2024-05-22T22:40:45.730 INFO:root:teuthology version: 0.0.1.dev287+gb9e3da8 2024-05-22T22:40:45.730 DEBUG:teuthology.run:Teuthology command: teuthology --name teuthology-2024-05-22_21:24:03-fs-squid-distro-default-smithi --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-05-22_21:24:03-fs-squid-distro-default-smithi/7721199 --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} -- /home/teuthworker/archive/teuthology-2024-05-22_21:24:03-fs-squid-distro-default-smithi/7721199/orig.config.yaml 2024-05-22T22:40:45.784 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-22T22:40:45.893 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-05-22_21:24:03-fs-squid-distro-default-smithi/7721199 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7721199' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-05-22_21:24:03-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 1 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 2e1f76222e88afe0f1cda37775b2d45d88289794 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 2e1f76222e88afe0f1cda37775b2d45d88289794 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 935217f1cb4dc722d5303f5ad364c883e9133376 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4697 sha1: 2e1f76222e88afe0f1cda37775b2d45d88289794 sleep_before_teardown: 0 subset: 12/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_935217f1cb4dc722d5303f5ad364c883e9133376/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 935217f1cb4dc722d5303f5ad364c883e9133376 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi084.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEBZHhDGokHl/h5QIjsY8e1W1dVIIfHcxwlXm9BCVfD04SEfTYnC5J5XgdHhNhYOJuYcbu79hcSAKBOHZuKNYoc= smithi096.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAodCWHaPoq5AHw+TLI4rLyg11vQuNtiacsuxGMOBuX2EZYid0LDGN3MCjhW1l+6T5GfoVnvvoHKxVcWVsqpyxY= smithi204.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGoAyfTfiwrVkHBhatljqIeunKt5TM1NeVHnmchzl39dDQ1wGm6lHisdtXfl73bIXCfs+UQL77MicpBsOIqbWZo= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a timestamp: 2024-05-22_21:24:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-22T22:40:45.893 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_935217f1cb4dc722d5303f5ad364c883e9133376/qa; will attempt to use it 2024-05-22T22:40:45.894 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_935217f1cb4dc722d5303f5ad364c883e9133376/qa/tasks 2024-05-22T22:40:45.894 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-22T22:40:45.895 INFO:teuthology.task.internal:Checking packages... 2024-05-22T22:40:45.915 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2e1f76222e88afe0f1cda37775b2d45d88289794' 2024-05-22T22:40:45.915 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-22T22:40:45.915 INFO:teuthology.packaging:ref: None 2024-05-22T22:40:45.915 INFO:teuthology.packaging:tag: None 2024-05-22T22:40:45.916 INFO:teuthology.packaging:branch: squid 2024-05-22T22:40:45.916 INFO:teuthology.packaging:sha1: 2e1f76222e88afe0f1cda37775b2d45d88289794 2024-05-22T22:40:45.916 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-05-22T22:40:46.105 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2804.g935217f1 2024-05-22T22:40:46.107 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-22T22:40:46.113 INFO:teuthology.task.internal:no buildpackages task found 2024-05-22T22:40:46.113 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-22T22:40:46.181 INFO:teuthology.task.internal:Saving configuration 2024-05-22T22:40:46.200 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-22T22:40:46.206 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-22T22:40:46.231 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi084.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-22_21:24:03-fs-squid-distro-default-smithi/7721199', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-22 22:35:28.407020', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEBZHhDGokHl/h5QIjsY8e1W1dVIIfHcxwlXm9BCVfD04SEfTYnC5J5XgdHhNhYOJuYcbu79hcSAKBOHZuKNYoc='} 2024-05-22T22:40:46.255 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-22_21:24:03-fs-squid-distro-default-smithi/7721199', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-22 22:35:28.405829', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAodCWHaPoq5AHw+TLI4rLyg11vQuNtiacsuxGMOBuX2EZYid0LDGN3MCjhW1l+6T5GfoVnvvoHKxVcWVsqpyxY='} 2024-05-22T22:40:46.277 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-22_21:24:03-fs-squid-distro-default-smithi/7721199', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-22 22:35:28.404416', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGoAyfTfiwrVkHBhatljqIeunKt5TM1NeVHnmchzl39dDQ1wGm6lHisdtXfl73bIXCfs+UQL77MicpBsOIqbWZo='} 2024-05-22T22:40:46.277 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-22T22:40:46.283 INFO:teuthology.task.internal:roles: ubuntu@smithi084.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-22T22:40:46.283 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-22T22:40:46.283 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-22T22:40:46.283 INFO:teuthology.run_tasks:Running task console_log... 2024-05-22T22:40:46.374 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff8f2ccfbe0>, signals=[15]) 2024-05-22T22:40:46.374 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-22T22:40:46.380 INFO:teuthology.task.internal:Opening connections... 2024-05-22T22:40:46.380 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi084.front.sepia.ceph.com 2024-05-22T22:40:46.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-22T22:40:46.456 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi096.front.sepia.ceph.com 2024-05-22T22:40:46.457 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-22T22:40:46.531 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi204.front.sepia.ceph.com 2024-05-22T22:40:46.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-22T22:40:46.605 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-22T22:40:46.611 DEBUG:teuthology.orchestra.run.smithi084:> uname -m 2024-05-22T22:40:46.628 INFO:teuthology.orchestra.run.smithi084.stdout:x86_64 2024-05-22T22:40:46.629 DEBUG:teuthology.orchestra.run.smithi084:> cat /etc/os-release 2024-05-22T22:40:46.686 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="CentOS Stream" 2024-05-22T22:40:46.686 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="9" 2024-05-22T22:40:46.687 INFO:teuthology.orchestra.run.smithi084.stdout:ID="centos" 2024-05-22T22:40:46.687 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE="rhel fedora" 2024-05-22T22:40:46.687 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="9" 2024-05-22T22:40:46.687 INFO:teuthology.orchestra.run.smithi084.stdout:PLATFORM_ID="platform:el9" 2024-05-22T22:40:46.687 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-22T22:40:46.687 INFO:teuthology.orchestra.run.smithi084.stdout:ANSI_COLOR="0;31" 2024-05-22T22:40:46.687 INFO:teuthology.orchestra.run.smithi084.stdout:LOGO="fedora-logo-icon" 2024-05-22T22:40:46.687 INFO:teuthology.orchestra.run.smithi084.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-22T22:40:46.687 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://centos.org/" 2024-05-22T22:40:46.687 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-22T22:40:46.687 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-22T22:40:46.687 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-22T22:40:46.688 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server 2024-05-22T22:40:46.711 DEBUG:teuthology.orchestra.run.smithi096:> uname -m 2024-05-22T22:40:46.729 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2024-05-22T22:40:46.729 DEBUG:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2024-05-22T22:40:46.787 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="CentOS Stream" 2024-05-22T22:40:46.787 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="9" 2024-05-22T22:40:46.787 INFO:teuthology.orchestra.run.smithi096.stdout:ID="centos" 2024-05-22T22:40:46.787 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="rhel fedora" 2024-05-22T22:40:46.787 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="9" 2024-05-22T22:40:46.787 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el9" 2024-05-22T22:40:46.788 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-22T22:40:46.788 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31" 2024-05-22T22:40:46.788 INFO:teuthology.orchestra.run.smithi096.stdout:LOGO="fedora-logo-icon" 2024-05-22T22:40:46.788 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-22T22:40:46.788 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://centos.org/" 2024-05-22T22:40:46.788 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-22T22:40:46.788 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-22T22:40:46.788 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-22T22:40:46.788 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2024-05-22T22:40:46.811 DEBUG:teuthology.orchestra.run.smithi204:> uname -m 2024-05-22T22:40:46.828 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2024-05-22T22:40:46.828 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2024-05-22T22:40:46.886 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="CentOS Stream" 2024-05-22T22:40:46.886 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="9" 2024-05-22T22:40:46.886 INFO:teuthology.orchestra.run.smithi204.stdout:ID="centos" 2024-05-22T22:40:46.886 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="rhel fedora" 2024-05-22T22:40:46.886 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="9" 2024-05-22T22:40:46.887 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el9" 2024-05-22T22:40:46.887 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-22T22:40:46.887 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31" 2024-05-22T22:40:46.887 INFO:teuthology.orchestra.run.smithi204.stdout:LOGO="fedora-logo-icon" 2024-05-22T22:40:46.887 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-22T22:40:46.887 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://centos.org/" 2024-05-22T22:40:46.887 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-22T22:40:46.887 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-22T22:40:46.887 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-22T22:40:46.887 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2024-05-22T22:40:46.911 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-22T22:40:46.920 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-22T22:40:46.925 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-22T22:40:46.925 DEBUG:teuthology.orchestra.run.smithi084:> test '!' -e /home/ubuntu/cephtest 2024-05-22T22:40:46.928 DEBUG:teuthology.orchestra.run.smithi096:> test '!' -e /home/ubuntu/cephtest 2024-05-22T22:40:46.930 DEBUG:teuthology.orchestra.run.smithi204:> test '!' -e /home/ubuntu/cephtest 2024-05-22T22:40:46.947 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-22T22:40:46.952 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-22T22:40:46.953 DEBUG:teuthology.orchestra.run.smithi084:> test -z $(ls -A /var/lib/ceph) 2024-05-22T22:40:46.985 DEBUG:teuthology.orchestra.run.smithi096:> test -z $(ls -A /var/lib/ceph) 2024-05-22T22:40:46.987 DEBUG:teuthology.orchestra.run.smithi204:> test -z $(ls -A /var/lib/ceph) 2024-05-22T22:40:47.281 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-22T22:40:47.357 INFO:teuthology.run_tasks:Running task kernel... 2024-05-22T22:40:47.370 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-22T22:40:47.370 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-22T22:40:47.371 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-22T22:40:47.371 DEBUG:teuthology.orchestra.run.smithi084:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-22T22:40:47.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T22:40:47.388 DEBUG:teuthology.orchestra.run.smithi084:> uname -r 2024-05-22T22:40:47.445 INFO:teuthology.orchestra.run.smithi084.stdout:5.14.0-437.el9.x86_64 2024-05-22T22:40:47.445 INFO:teuthology.task.kernel:Running kernel on smithi084: 5.14.0-437.el9.x86_64 2024-05-22T22:40:47.446 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum install -y kernel 2024-05-22T22:40:53.767 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.1 MB 00:04 2024-05-22T22:41:00.159 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - AppStream 3.6 MB/s | 19 MB 00:05 2024-05-22T22:41:04.409 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-05-22T22:41:06.243 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 16 kB 00:00 2024-05-22T22:41:09.452 INFO:teuthology.orchestra.run.smithi084.stdout:Extra Packages for Enterprise Linux 6.8 MB/s | 21 MB 00:03 2024-05-22T22:41:14.730 INFO:teuthology.orchestra.run.smithi084.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-05-22T22:41:16.599 INFO:teuthology.orchestra.run.smithi084.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-22T22:41:16.599 INFO:teuthology.orchestra.run.smithi084.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-22T22:41:16.690 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-05-22T22:41:16.697 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-05-22T22:41:16.697 INFO:teuthology.orchestra.run.smithi084.stdout: Package Architecture Version Repository Size 2024-05-22T22:41:16.698 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-05-22T22:41:16.698 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-05-22T22:41:16.698 INFO:teuthology.orchestra.run.smithi084.stdout: kernel x86_64 5.14.0-447.el9 baseos 576 k 2024-05-22T22:41:16.698 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-05-22T22:41:16.698 INFO:teuthology.orchestra.run.smithi084.stdout: kernel-core x86_64 5.14.0-447.el9 baseos 16 M 2024-05-22T22:41:16.698 INFO:teuthology.orchestra.run.smithi084.stdout: kernel-modules x86_64 5.14.0-447.el9 baseos 34 M 2024-05-22T22:41:16.698 INFO:teuthology.orchestra.run.smithi084.stdout: kernel-modules-core x86_64 5.14.0-447.el9 baseos 28 M 2024-05-22T22:41:16.698 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:41:16.698 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-05-22T22:41:16.698 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-05-22T22:41:16.699 INFO:teuthology.orchestra.run.smithi084.stdout:Install 4 Packages 2024-05-22T22:41:16.699 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:41:16.699 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 78 M 2024-05-22T22:41:16.699 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 123 M 2024-05-22T22:41:16.699 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-05-22T22:41:17.478 INFO:teuthology.orchestra.run.smithi084.stdout:(1/4): kernel-5.14.0-447.el9.x86_64.rpm 868 kB/s | 576 kB 00:00 2024-05-22T22:41:29.492 INFO:teuthology.orchestra.run.smithi084.stdout:(2/4): kernel-core-5.14.0-447.el9.x86_64.rpm 1.2 MB/s | 16 MB 00:12 2024-05-22T22:41:38.274 INFO:teuthology.orchestra.run.smithi084.stdout:(3/4): kernel-modules-core-5.14.0-447.el9.x86_6 1.3 MB/s | 28 MB 00:20 2024-05-22T22:41:40.766 INFO:teuthology.orchestra.run.smithi084.stdout:(4/4): kernel-modules-5.14.0-447.el9.x86_64.rpm 1.4 MB/s | 34 MB 00:23 2024-05-22T22:41:40.766 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-05-22T22:41:40.766 INFO:teuthology.orchestra.run.smithi084.stdout:Total 3.2 MB/s | 78 MB 00:24 2024-05-22T22:41:41.450 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-05-22T22:41:41.793 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-05-22T22:41:41.793 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-05-22T22:41:42.737 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-05-22T22:41:42.737 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-05-22T22:41:45.116 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-05-22T22:41:45.746 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : kernel-modules-core-5.14.0-447.el9.x86_64 1/4 2024-05-22T22:41:45.824 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-22T22:41:48.474 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-22T22:41:48.610 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-22T22:41:53.297 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-22T22:41:53.413 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : kernel-5.14.0-447.el9.x86_64 4/4 2024-05-22T22:41:58.150 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-22T22:42:27.371 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 4/4 2024-05-22T22:42:27.408 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 4/4 2024-05-22T22:42:27.712 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: kernel-5.14.0-447.el9.x86_64 4/4 2024-05-22T22:42:27.712 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : kernel-5.14.0-447.el9.x86_64 1/4 2024-05-22T22:42:27.712 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-22T22:42:27.712 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-22T22:42:28.529 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-22T22:42:28.529 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:42:28.529 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-05-22T22:42:28.529 INFO:teuthology.orchestra.run.smithi084.stdout: kernel-5.14.0-447.el9.x86_64 2024-05-22T22:42:28.529 INFO:teuthology.orchestra.run.smithi084.stdout: kernel-core-5.14.0-447.el9.x86_64 2024-05-22T22:42:28.530 INFO:teuthology.orchestra.run.smithi084.stdout: kernel-modules-5.14.0-447.el9.x86_64 2024-05-22T22:42:28.530 INFO:teuthology.orchestra.run.smithi084.stdout: kernel-modules-core-5.14.0-447.el9.x86_64 2024-05-22T22:42:28.530 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:42:28.530 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-05-22T22:42:28.886 DEBUG:teuthology.orchestra.run.smithi084:> rpm -q kernel | sort -rV | head -n 1 2024-05-22T22:42:28.962 INFO:teuthology.orchestra.run.smithi084.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-22T22:42:28.962 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-22T22:42:28.962 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-447.el9.x86_64 2024-05-22T22:42:28.962 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-22T22:42:28.962 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-22T22:42:28.963 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-22T22:42:28.963 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi084.front.sepia.ceph.com, path=None, version=distro) 2024-05-22T22:42:28.963 DEBUG:teuthology.orchestra.run.smithi084:> rpm -q kernel | sort -rV | head -n 1 2024-05-22T22:42:29.012 INFO:teuthology.orchestra.run.smithi084.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-22T22:42:29.012 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-22T22:42:29.012 DEBUG:teuthology.orchestra.run.smithi084:> sudo rpm -qi grub2-tools 2024-05-22T22:42:29.042 INFO:teuthology.orchestra.run.smithi084.stdout:Name : grub2-tools 2024-05-22T22:42:29.043 INFO:teuthology.orchestra.run.smithi084.stdout:Epoch : 1 2024-05-22T22:42:29.043 INFO:teuthology.orchestra.run.smithi084.stdout:Version : 2.06 2024-05-22T22:42:29.043 INFO:teuthology.orchestra.run.smithi084.stdout:Release : 68.el9 2024-05-22T22:42:29.043 INFO:teuthology.orchestra.run.smithi084.stdout:Architecture: x86_64 2024-05-22T22:42:29.043 INFO:teuthology.orchestra.run.smithi084.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-22T22:42:29.043 INFO:teuthology.orchestra.run.smithi084.stdout:Group : Unspecified 2024-05-22T22:42:29.043 INFO:teuthology.orchestra.run.smithi084.stdout:Size : 8264254 2024-05-22T22:42:29.043 INFO:teuthology.orchestra.run.smithi084.stdout:License : GPLv3+ 2024-05-22T22:42:29.043 INFO:teuthology.orchestra.run.smithi084.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-22T22:42:29.043 INFO:teuthology.orchestra.run.smithi084.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-22T22:42:29.044 INFO:teuthology.orchestra.run.smithi084.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-22T22:42:29.044 INFO:teuthology.orchestra.run.smithi084.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-22T22:42:29.044 INFO:teuthology.orchestra.run.smithi084.stdout:Packager : builder@centos.org 2024-05-22T22:42:29.044 INFO:teuthology.orchestra.run.smithi084.stdout:Vendor : CentOS 2024-05-22T22:42:29.044 INFO:teuthology.orchestra.run.smithi084.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-22T22:42:29.044 INFO:teuthology.orchestra.run.smithi084.stdout:Summary : Support tools for GRUB. 2024-05-22T22:42:29.044 INFO:teuthology.orchestra.run.smithi084.stdout:Description : 2024-05-22T22:42:29.044 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:42:29.044 INFO:teuthology.orchestra.run.smithi084.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-22T22:42:29.044 INFO:teuthology.orchestra.run.smithi084.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-22T22:42:29.044 INFO:teuthology.orchestra.run.smithi084.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-22T22:42:29.044 INFO:teuthology.orchestra.run.smithi084.stdout:hardware devices. 2024-05-22T22:42:29.044 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:42:29.044 INFO:teuthology.orchestra.run.smithi084.stdout:This subpackage provides tools for support of all platforms. 2024-05-22T22:42:29.045 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-22T22:42:29.045 INFO:teuthology.task.kernel:Updating grub on smithi084 to boot 5.14.0-447.el9.x86_64 2024-05-22T22:42:29.045 DEBUG:teuthology.orchestra.run.smithi084:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-22T22:42:29.732 INFO:teuthology.orchestra.run.smithi084.stderr:Generating grub configuration file ... 2024-05-22T22:42:31.104 INFO:teuthology.orchestra.run.smithi084.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-22T22:42:31.142 INFO:teuthology.orchestra.run.smithi084.stderr:done 2024-05-22T22:42:31.144 DEBUG:teuthology.orchestra.run.smithi084:> mktemp 2024-05-22T22:42:31.159 INFO:teuthology.orchestra.run.smithi084.stdout:/tmp/tmp.ddOhH8U5cZ 2024-05-22T22:42:31.159 DEBUG:teuthology.orchestra.run.smithi084:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ddOhH8U5cZ 2024-05-22T22:42:31.234 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0666 /tmp/tmp.ddOhH8U5cZ 2024-05-22T22:42:31.365 DEBUG:teuthology.orchestra.remote:smithi084:/tmp/tmp.ddOhH8U5cZ is 6KB 2024-05-22T22:42:31.414 DEBUG:teuthology.orchestra.run.smithi084:> rm -fr /tmp/tmp.ddOhH8U5cZ 2024-05-22T22:42:31.430 DEBUG:teuthology.orchestra.run.smithi084:> sudo /bin/ls /boot/loader/entries || true 2024-05-22T22:42:31.491 INFO:teuthology.orchestra.run.smithi084.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-22T22:42:31.491 INFO:teuthology.orchestra.run.smithi084.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-22T22:42:31.492 INFO:teuthology.orchestra.run.smithi084.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-22T22:42:31.492 INFO:teuthology.orchestra.run.smithi084.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64.conf 2024-05-22T22:42:31.493 DEBUG:teuthology.orchestra.run.smithi084:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64 2024-05-22T22:42:31.646 DEBUG:teuthology.orchestra.run.smithi084:> sudo shutdown -r now 2024-05-22T22:43:01.649 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-22T22:43:01.649 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-05-22T22:43:01.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-22T22:44:01.653 DEBUG:teuthology.orchestra.remote:timed out 2024-05-22T22:44:10.654 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-05-22T22:44:10.655 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-22T22:44:12.343 DEBUG:teuthology.orchestra.run.smithi084:> true 2024-05-22T22:44:12.761 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-05-22T22:44:12.761 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-447.el9.x86_64"... 2024-05-22T22:44:12.762 DEBUG:teuthology.orchestra.run.smithi084:> uname -r 2024-05-22T22:44:12.776 INFO:teuthology.orchestra.run.smithi084.stdout:5.14.0-447.el9.x86_64 2024-05-22T22:44:12.777 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-447.el9.x86_64 vs 5.14.0-447.el9.x86_64 2024-05-22T22:44:12.777 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-22T22:44:12.777 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-22T22:44:13.778 DEBUG:teuthology.parallel:result is None 2024-05-22T22:44:13.778 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-22T22:44:13.785 INFO:teuthology.task.internal:Creating test directory... 2024-05-22T22:44:13.785 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-22T22:44:13.787 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-22T22:44:13.790 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-22T22:44:13.808 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-22T22:44:13.868 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-22T22:44:13.879 INFO:teuthology.task.internal:Creating archive directory... 2024-05-22T22:44:13.880 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-22T22:44:13.882 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-22T22:44:13.885 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-22T22:44:14.009 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-22T22:44:14.015 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-22T22:44:14.015 DEBUG:teuthology.orchestra.run.smithi096:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-22T22:44:14.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T22:44:14.032 DEBUG:teuthology.orchestra.run.smithi204:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-22T22:44:14.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T22:44:14.049 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-22T22:44:14.051 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-22T22:44:14.074 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-22T22:44:14.104 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-22T22:44:14.121 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-22T22:44:14.131 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-22T22:44:14.134 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-22T22:44:14.162 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-22T22:44:14.162 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-22T22:44:14.164 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-22T22:44:14.171 INFO:teuthology.task.internal:Configuring sudo... 2024-05-22T22:44:14.172 DEBUG:teuthology.orchestra.run.smithi084:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-22T22:44:14.174 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-22T22:44:14.176 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-22T22:44:14.230 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-22T22:44:14.239 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-22T22:44:14.239 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-22T22:44:14.242 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-22T22:44:14.244 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-22T22:44:14.287 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-22T22:44:14.335 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-22T22:44:14.383 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-22T22:44:14.469 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-22T22:44:14.508 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-22T22:44:14.508 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-22T22:44:14.595 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-22T22:44:14.640 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-22T22:44:14.691 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-22T22:44:14.776 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-22T22:44:14.817 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-22T22:44:14.817 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-22T22:44:14.883 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-22T22:44:14.927 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-22T22:44:14.984 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-22T22:44:15.067 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-22T22:44:15.111 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-22T22:44:15.111 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-22T22:44:15.177 DEBUG:teuthology.orchestra.run.smithi084:> sudo service rsyslog restart 2024-05-22T22:44:15.179 DEBUG:teuthology.orchestra.run.smithi096:> sudo service rsyslog restart 2024-05-22T22:44:15.182 DEBUG:teuthology.orchestra.run.smithi204:> sudo service rsyslog restart 2024-05-22T22:44:15.215 INFO:teuthology.orchestra.run.smithi084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-22T22:44:15.219 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-22T22:44:15.256 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-22T22:44:15.675 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-22T22:44:15.683 INFO:teuthology.task.internal:Starting timer... 2024-05-22T22:44:15.683 INFO:teuthology.run_tasks:Running task pcp... 2024-05-22T22:44:15.707 INFO:teuthology.run_tasks:Running task selinux... 2024-05-22T22:44:15.715 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-22T22:44:15.716 DEBUG:teuthology.orchestra.run.smithi084:> sudo service auditd rotate 2024-05-22T22:44:15.787 INFO:teuthology.orchestra.run.smithi084.stdout:Rotating logs: 2024-05-22T22:44:15.789 DEBUG:teuthology.orchestra.run.smithi096:> sudo service auditd rotate 2024-05-22T22:44:15.849 INFO:teuthology.orchestra.run.smithi096.stdout:Rotating logs: 2024-05-22T22:44:15.851 DEBUG:teuthology.orchestra.run.smithi204:> sudo service auditd rotate 2024-05-22T22:44:15.930 INFO:teuthology.orchestra.run.smithi204.stdout:Rotating logs: 2024-05-22T22:44:15.931 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-22T22:44:15.931 DEBUG:teuthology.orchestra.run.smithi084:> /usr/sbin/getenforce 2024-05-22T22:44:15.959 INFO:teuthology.orchestra.run.smithi084.stdout:Permissive 2024-05-22T22:44:15.959 DEBUG:teuthology.orchestra.run.smithi096:> /usr/sbin/getenforce 2024-05-22T22:44:15.992 INFO:teuthology.orchestra.run.smithi096.stdout:Permissive 2024-05-22T22:44:15.992 DEBUG:teuthology.orchestra.run.smithi204:> /usr/sbin/getenforce 2024-05-22T22:44:16.019 INFO:teuthology.orchestra.run.smithi204.stdout:Permissive 2024-05-22T22:44:16.019 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi084.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi096.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi204.front.sepia.ceph.com': 'permissive'} 2024-05-22T22:44:16.019 DEBUG:teuthology.orchestra.run.smithi084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-22T22:44:16.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T22:44:16.045 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-22T22:44:16.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T22:44:16.073 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-22T22:44:16.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T22:44:16.099 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-22T22:44:16.100 DEBUG:teuthology.orchestra.run.smithi084:> sudo /usr/sbin/setenforce permissive 2024-05-22T22:44:16.270 DEBUG:teuthology.orchestra.run.smithi096:> sudo /usr/sbin/setenforce permissive 2024-05-22T22:44:16.295 DEBUG:teuthology.orchestra.run.smithi204:> sudo /usr/sbin/setenforce permissive 2024-05-22T22:44:16.322 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-22T22:44:16.331 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-22T22:44:16.344 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-22T22:44:16.345 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi084.front.sepia.ceph.com,smithi096.front.sepia.ceph.com,smithi204.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-22T22:48:49.652 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi084.front.sepia.ceph.com'), Remote(name='ubuntu@smithi096.front.sepia.ceph.com'), Remote(name='ubuntu@smithi204.front.sepia.ceph.com')] 2024-05-22T22:48:49.653 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-05-22T22:48:49.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-22T22:48:49.732 DEBUG:teuthology.orchestra.run.smithi084:> true 2024-05-22T22:48:49.813 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-05-22T22:48:49.813 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-05-22T22:48:49.814 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-22T22:48:49.888 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-05-22T22:48:49.973 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-05-22T22:48:49.973 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-05-22T22:48:49.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-22T22:48:50.048 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-05-22T22:48:50.135 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-05-22T22:48:50.136 INFO:teuthology.run_tasks:Running task clock... 2024-05-22T22:48:50.146 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-22T22:48:50.146 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-22T22:48:50.147 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-22T22:48:50.149 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-22T22:48:50.149 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-22T22:48:50.151 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-22T22:48:50.151 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-22T22:48:50.176 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-22T22:48:50.177 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-22T22:48:50.190 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-22T22:48:50.191 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-22T22:48:50.207 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-22T22:48:50.212 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: ntpd: command not found 2024-05-22T22:48:50.216 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: ntpd: command not found 2024-05-22T22:48:50.221 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-22T22:48:50.222 INFO:teuthology.orchestra.run.smithi096.stdout:506 Cannot talk to daemon 2024-05-22T22:48:50.230 INFO:teuthology.orchestra.run.smithi084.stdout:506 Cannot talk to daemon 2024-05-22T22:48:50.235 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-22T22:48:50.241 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-22T22:48:50.246 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: ntpd: command not found 2024-05-22T22:48:50.248 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-22T22:48:50.256 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-22T22:48:50.257 INFO:teuthology.orchestra.run.smithi204.stdout:506 Cannot talk to daemon 2024-05-22T22:48:50.271 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-22T22:48:50.284 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-22T22:48:50.303 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-05-22T22:48:50.306 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-22T22:48:50.307 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-05-22T22:48:50.307 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-22T22:48:50.307 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-22T22:48:50.307 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-22T22:48:50.307 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-22T22:48:50.311 INFO:teuthology.orchestra.run.smithi084.stderr:bash: line 1: ntpq: command not found 2024-05-22T22:48:50.314 INFO:teuthology.orchestra.run.smithi084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-22T22:48:50.314 INFO:teuthology.orchestra.run.smithi084.stdout:=============================================================================== 2024-05-22T22:48:50.315 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-22T22:48:50.315 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-22T22:48:50.315 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-22T22:48:50.315 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-22T22:48:50.337 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-05-22T22:48:50.340 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-22T22:48:50.340 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-05-22T22:48:50.340 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-22T22:48:50.340 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-22T22:48:50.341 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-22T22:48:50.341 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-22T22:48:50.342 INFO:teuthology.run_tasks:Running task install... 2024-05-22T22:48:50.349 DEBUG:teuthology.task.install:project ceph 2024-05-22T22:48:50.350 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2e1f76222e88afe0f1cda37775b2d45d88289794'}} 2024-05-22T22:48:50.350 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2e1f76222e88afe0f1cda37775b2d45d88289794'} 2024-05-22T22:48:50.350 INFO:teuthology.task.install:Using flavor: default 2024-05-22T22:48:50.355 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-22T22:48:50.355 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-22T22:48:50.355 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2e1f76222e88afe0f1cda37775b2d45d88289794', 'tag': None, 'wait_for_package': False} 2024-05-22T22:48:50.356 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2e1f76222e88afe0f1cda37775b2d45d88289794 2024-05-22T22:48:50.358 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2e1f76222e88afe0f1cda37775b2d45d88289794', 'tag': None, 'wait_for_package': False} 2024-05-22T22:48:50.358 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2e1f76222e88afe0f1cda37775b2d45d88289794 2024-05-22T22:48:50.359 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2e1f76222e88afe0f1cda37775b2d45d88289794', 'tag': None, 'wait_for_package': False} 2024-05-22T22:48:50.359 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2e1f76222e88afe0f1cda37775b2d45d88289794 2024-05-22T22:48:50.503 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2e1f76222e88afe0f1cda37775b2d45d88289794/centos/9/flavors/default/ 2024-05-22T22:48:50.504 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2765.g2e1f7622 2024-05-22T22:48:50.573 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2e1f76222e88afe0f1cda37775b2d45d88289794/centos/9/flavors/default/ 2024-05-22T22:48:50.573 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2765.g2e1f7622 2024-05-22T22:48:50.577 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2e1f76222e88afe0f1cda37775b2d45d88289794/centos/9/flavors/default/ 2024-05-22T22:48:50.577 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2765.g2e1f7622 2024-05-22T22:48:50.677 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2e1f76222e88afe0f1cda37775b2d45d88289794/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2e1f76222e88afe0f1cda37775b2d45d88289794/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2e1f76222e88afe0f1cda37775b2d45d88289794/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-22T22:48:50.677 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-22T22:48:50.677 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-22T22:48:50.687 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2e1f76222e88afe0f1cda37775b2d45d88289794/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2e1f76222e88afe0f1cda37775b2d45d88289794/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2e1f76222e88afe0f1cda37775b2d45d88289794/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-22T22:48:50.687 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-22T22:48:50.687 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-22T22:48:50.702 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-22T22:48:50.702 DEBUG:teuthology.orchestra.run.smithi096:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2e1f76222e88afe0f1cda37775b2d45d88289794/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-22T22:48:50.712 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-22T22:48:50.712 DEBUG:teuthology.orchestra.run.smithi204:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2e1f76222e88afe0f1cda37775b2d45d88289794/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-22T22:48:50.743 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2e1f76222e88afe0f1cda37775b2d45d88289794/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2e1f76222e88afe0f1cda37775b2d45d88289794/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2e1f76222e88afe0f1cda37775b2d45d88289794/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-22T22:48:50.743 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-22T22:48:50.743 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-22T22:48:50.768 DEBUG:teuthology.orchestra.run.smithi096:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-22T22:48:50.769 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-22T22:48:50.769 DEBUG:teuthology.orchestra.run.smithi084:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2e1f76222e88afe0f1cda37775b2d45d88289794/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-22T22:48:50.778 DEBUG:teuthology.orchestra.run.smithi204:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-22T22:48:50.838 DEBUG:teuthology.orchestra.run.smithi084:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-22T22:48:50.841 DEBUG:teuthology.orchestra.run.smithi096:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-22T22:48:50.850 DEBUG:teuthology.orchestra.run.smithi204:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-22T22:48:50.904 INFO:teuthology.orchestra.run.smithi096.stdout:check_obsoletes = 1 2024-05-22T22:48:50.906 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum clean all 2024-05-22T22:48:50.910 DEBUG:teuthology.orchestra.run.smithi084:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-22T22:48:50.913 INFO:teuthology.orchestra.run.smithi204.stdout:check_obsoletes = 1 2024-05-22T22:48:50.915 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum clean all 2024-05-22T22:48:50.974 INFO:teuthology.orchestra.run.smithi084.stdout:check_obsoletes = 1 2024-05-22T22:48:50.976 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum clean all 2024-05-22T22:48:51.131 INFO:teuthology.orchestra.run.smithi096.stdout:45 files removed 2024-05-22T22:48:51.140 INFO:teuthology.orchestra.run.smithi204.stdout:45 files removed 2024-05-22T22:48:51.157 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-22T22:48:51.167 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-22T22:48:51.208 INFO:teuthology.orchestra.run.smithi084.stdout:45 files removed 2024-05-22T22:48:51.234 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-22T22:48:51.804 INFO:teuthology.orchestra.run.smithi096.stdout:ceph packages for x86_64 217 kB/s | 83 kB 00:00 2024-05-22T22:48:51.823 INFO:teuthology.orchestra.run.smithi084.stdout:ceph packages for x86_64 264 kB/s | 83 kB 00:00 2024-05-22T22:48:51.841 INFO:teuthology.orchestra.run.smithi204.stdout:ceph packages for x86_64 203 kB/s | 83 kB 00:00 2024-05-22T22:48:52.182 INFO:teuthology.orchestra.run.smithi084.stdout:ceph noarch packages 37 kB/s | 11 kB 00:00 2024-05-22T22:48:52.224 INFO:teuthology.orchestra.run.smithi204.stdout:ceph noarch packages 34 kB/s | 11 kB 00:00 2024-05-22T22:48:52.246 INFO:teuthology.orchestra.run.smithi096.stdout:ceph noarch packages 29 kB/s | 11 kB 00:00 2024-05-22T22:48:52.533 INFO:teuthology.orchestra.run.smithi204.stdout:ceph source packages 6.9 kB/s | 1.9 kB 00:00 2024-05-22T22:48:52.555 INFO:teuthology.orchestra.run.smithi096.stdout:ceph source packages 6.9 kB/s | 1.9 kB 00:00 2024-05-22T22:48:52.574 INFO:teuthology.orchestra.run.smithi084.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2024-05-22T22:48:53.250 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.1 MB 00:00 2024-05-22T22:48:53.425 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.1 MB 00:00 2024-05-22T22:48:55.259 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-05-22T22:48:55.285 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-05-22T22:48:56.113 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - BaseOS 2.3 MB/s | 8.1 MB 00:03 2024-05-22T22:48:59.176 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.2 MB 00:00 2024-05-22T22:49:00.578 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.2 MB 00:01 2024-05-22T22:49:00.786 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-05-22T22:49:02.048 INFO:teuthology.orchestra.run.smithi084.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-05-22T22:49:02.154 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 16 kB 00:00 2024-05-22T22:49:03.908 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-05-22T22:49:06.255 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - AppStream 2.1 MB/s | 19 MB 00:09 2024-05-22T22:49:07.120 INFO:teuthology.orchestra.run.smithi084.stdout:lab-extras 44 kB/s | 1.7 kB 00:00 2024-05-22T22:49:09.097 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-05-22T22:49:09.354 INFO:teuthology.orchestra.run.smithi084.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-22T22:49:09.355 INFO:teuthology.orchestra.run.smithi084.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-22T22:49:09.366 INFO:teuthology.orchestra.run.smithi084.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-22T22:49:09.367 INFO:teuthology.orchestra.run.smithi084.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-22T22:49:09.368 INFO:teuthology.orchestra.run.smithi084.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-22T22:49:09.369 INFO:teuthology.orchestra.run.smithi084.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-22T22:49:09.370 INFO:teuthology.orchestra.run.smithi084.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-22T22:49:09.372 INFO:teuthology.orchestra.run.smithi084.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-22T22:49:09.373 INFO:teuthology.orchestra.run.smithi084.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-22T22:49:09.435 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-05-22T22:49:09.446 INFO:teuthology.orchestra.run.smithi084.stdout:======================================================================================= 2024-05-22T22:49:09.446 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-05-22T22:49:09.446 INFO:teuthology.orchestra.run.smithi084.stdout:======================================================================================= 2024-05-22T22:49:09.446 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-05-22T22:49:09.447 INFO:teuthology.orchestra.run.smithi084.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-22T22:49:09.447 INFO:teuthology.orchestra.run.smithi084.stdout: ceph x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 6.6 k 2024-05-22T22:49:09.447 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 5.4 M 2024-05-22T22:49:09.447 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-fuse x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 889 k 2024-05-22T22:49:09.447 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 145 k 2024-05-22T22:49:09.447 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 1.5 M 2024-05-22T22:49:09.447 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 142 k 2024-05-22T22:49:09.447 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 3.5 M 2024-05-22T22:49:09.447 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 7.4 M 2024-05-22T22:49:09.447 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-rook noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 49 k 2024-05-22T22:49:09.447 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-radosgw x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 10 M 2024-05-22T22:49:09.448 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-test x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 48 M 2024-05-22T22:49:09.448 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-volume noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 264 k 2024-05-22T22:49:09.448 INFO:teuthology.orchestra.run.smithi084.stdout: cephadm noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 763 k 2024-05-22T22:49:09.448 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-mirror x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 222 k 2024-05-22T22:49:09.448 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-shell x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 27 k 2024-05-22T22:49:09.448 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-top noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 18 k 2024-05-22T22:49:09.448 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-22T22:49:09.448 INFO:teuthology.orchestra.run.smithi084.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-22T22:49:09.448 INFO:teuthology.orchestra.run.smithi084.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-22T22:49:09.448 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs-devel x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 32 k 2024-05-22T22:49:09.449 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs2 x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 745 k 2024-05-22T22:49:09.449 INFO:teuthology.orchestra.run.smithi084.stdout: librados-devel x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 127 k 2024-05-22T22:49:09.449 INFO:teuthology.orchestra.run.smithi084.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-22T22:49:09.449 INFO:teuthology.orchestra.run.smithi084.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-22T22:49:09.449 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-22T22:49:09.449 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-22T22:49:09.449 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-22T22:49:09.449 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-22T22:49:09.449 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cephfs x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 162 k 2024-05-22T22:49:09.449 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 319 k 2024-05-22T22:49:09.449 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rbd x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 300 k 2024-05-22T22:49:09.450 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rgw x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 99 k 2024-05-22T22:49:09.450 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-fuse x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 85 k 2024-05-22T22:49:09.450 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-mirror x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 3.1 M 2024-05-22T22:49:09.450 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-nbd x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 171 k 2024-05-22T22:49:09.450 INFO:teuthology.orchestra.run.smithi084.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-22T22:49:09.450 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrading: 2024-05-22T22:49:09.450 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-22T22:49:09.450 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-22T22:49:09.450 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-22T22:49:09.450 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-22T22:49:09.450 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-22T22:49:09.451 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-22T22:49:09.451 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-22T22:49:09.451 INFO:teuthology.orchestra.run.smithi084.stdout: librados2 x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 3.4 M 2024-05-22T22:49:09.451 INFO:teuthology.orchestra.run.smithi084.stdout: librbd1 x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 3.1 M 2024-05-22T22:49:09.451 INFO:teuthology.orchestra.run.smithi084.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-22T22:49:09.451 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-22T22:49:09.451 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-05-22T22:49:09.451 INFO:teuthology.orchestra.run.smithi084.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-22T22:49:09.451 INFO:teuthology.orchestra.run.smithi084.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-22T22:49:09.451 INFO:teuthology.orchestra.run.smithi084.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-22T22:49:09.451 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-common x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 21 M 2024-05-22T22:49:09.451 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 32 k 2024-05-22T22:49:09.452 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mds x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 2.4 M 2024-05-22T22:49:09.452 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 249 k 2024-05-22T22:49:09.452 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mon x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 4.7 M 2024-05-22T22:49:09.452 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-osd x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 17 M 2024-05-22T22:49:09.452 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 16 k 2024-05-22T22:49:09.452 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-selinux x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 25 k 2024-05-22T22:49:09.452 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-22T22:49:09.452 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-22T22:49:09.452 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-22T22:49:09.452 INFO:teuthology.orchestra.run.smithi084.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-22T22:49:09.452 INFO:teuthology.orchestra.run.smithi084.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-22T22:49:09.452 INFO:teuthology.orchestra.run.smithi084.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-22T22:49:09.453 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-22T22:49:09.453 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-22T22:49:09.453 INFO:teuthology.orchestra.run.smithi084.stdout: libcephsqlite x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 163 k 2024-05-22T22:49:09.453 INFO:teuthology.orchestra.run.smithi084.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-22T22:49:09.453 INFO:teuthology.orchestra.run.smithi084.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-22T22:49:09.453 INFO:teuthology.orchestra.run.smithi084.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-22T22:49:09.453 INFO:teuthology.orchestra.run.smithi084.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-22T22:49:09.453 INFO:teuthology.orchestra.run.smithi084.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-22T22:49:09.453 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1 x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 501 k 2024-05-22T22:49:09.453 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-22T22:49:09.453 INFO:teuthology.orchestra.run.smithi084.stdout: librgw2 x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 5.3 M 2024-05-22T22:49:09.454 INFO:teuthology.orchestra.run.smithi084.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-22T22:49:09.454 INFO:teuthology.orchestra.run.smithi084.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-22T22:49:09.454 INFO:teuthology.orchestra.run.smithi084.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-22T22:49:09.454 INFO:teuthology.orchestra.run.smithi084.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-22T22:49:09.454 INFO:teuthology.orchestra.run.smithi084.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-22T22:49:09.454 INFO:teuthology.orchestra.run.smithi084.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-22T22:49:09.454 INFO:teuthology.orchestra.run.smithi084.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-22T22:49:09.454 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-22T22:49:09.454 INFO:teuthology.orchestra.run.smithi084.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-22T22:49:09.454 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-22T22:49:09.455 INFO:teuthology.orchestra.run.smithi084.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-22T22:49:09.455 INFO:teuthology.orchestra.run.smithi084.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-22T22:49:09.455 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-22T22:49:09.455 INFO:teuthology.orchestra.run.smithi084.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-22T22:49:09.455 INFO:teuthology.orchestra.run.smithi084.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-22T22:49:09.455 INFO:teuthology.orchestra.run.smithi084.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-22T22:49:09.455 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-22T22:49:09.455 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 45 k 2024-05-22T22:49:09.455 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-common x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 135 k 2024-05-22T22:49:09.455 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-22T22:49:09.455 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-22T22:49:09.455 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-22T22:49:09.456 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-22T22:49:09.456 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-22T22:49:09.456 INFO:teuthology.orchestra.run.smithi084.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-22T22:49:09.456 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-22T22:49:09.456 INFO:teuthology.orchestra.run.smithi084.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-22T22:49:09.456 INFO:teuthology.orchestra.run.smithi084.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-22T22:49:09.456 INFO:teuthology.orchestra.run.smithi084.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-22T22:49:09.456 INFO:teuthology.orchestra.run.smithi084.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-22T22:49:09.456 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-22T22:49:09.456 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-22T22:49:09.456 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-22T22:49:09.456 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-22T22:49:09.457 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-22T22:49:09.457 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-22T22:49:09.457 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-22T22:49:09.457 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-22T22:49:09.457 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-22T22:49:09.457 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-22T22:49:09.457 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-22T22:49:09.457 INFO:teuthology.orchestra.run.smithi084.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-22T22:49:09.457 INFO:teuthology.orchestra.run.smithi084.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-22T22:49:09.457 INFO:teuthology.orchestra.run.smithi084.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-22T22:49:09.457 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-22T22:49:09.457 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-22T22:49:09.458 INFO:teuthology.orchestra.run.smithi084.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-22T22:49:09.458 INFO:teuthology.orchestra.run.smithi084.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-22T22:49:09.458 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-22T22:49:09.458 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-22T22:49:09.458 INFO:teuthology.orchestra.run.smithi084.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-22T22:49:09.458 INFO:teuthology.orchestra.run.smithi084.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-22T22:49:09.458 INFO:teuthology.orchestra.run.smithi084.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-22T22:49:09.458 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-22T22:49:09.458 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-22T22:49:09.458 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-22T22:49:09.458 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-22T22:49:09.458 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-22T22:49:09.458 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-22T22:49:09.459 INFO:teuthology.orchestra.run.smithi084.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-22T22:49:09.459 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-22T22:49:09.459 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-22T22:49:09.459 INFO:teuthology.orchestra.run.smithi084.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-22T22:49:09.459 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-22T22:49:09.459 INFO:teuthology.orchestra.run.smithi084.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-22T22:49:09.459 INFO:teuthology.orchestra.run.smithi084.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-22T22:49:09.459 INFO:teuthology.orchestra.run.smithi084.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-22T22:49:09.459 INFO:teuthology.orchestra.run.smithi084.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-22T22:49:09.459 INFO:teuthology.orchestra.run.smithi084.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-22T22:49:09.459 INFO:teuthology.orchestra.run.smithi084.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-22T22:49:09.459 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-22T22:49:09.460 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-22T22:49:09.460 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-22T22:49:09.460 INFO:teuthology.orchestra.run.smithi084.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-22T22:49:09.460 INFO:teuthology.orchestra.run.smithi084.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-22T22:49:09.460 INFO:teuthology.orchestra.run.smithi084.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-22T22:49:09.460 INFO:teuthology.orchestra.run.smithi084.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-22T22:49:09.460 INFO:teuthology.orchestra.run.smithi084.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-22T22:49:09.460 INFO:teuthology.orchestra.run.smithi084.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-22T22:49:09.460 INFO:teuthology.orchestra.run.smithi084.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-22T22:49:09.460 INFO:teuthology.orchestra.run.smithi084.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-22T22:49:09.460 INFO:teuthology.orchestra.run.smithi084.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-22T22:49:09.460 INFO:teuthology.orchestra.run.smithi084.stdout:Installing weak dependencies: 2024-05-22T22:49:09.460 INFO:teuthology.orchestra.run.smithi084.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-22T22:49:09.461 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:49:09.461 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-05-22T22:49:09.461 INFO:teuthology.orchestra.run.smithi084.stdout:======================================================================================= 2024-05-22T22:49:09.461 INFO:teuthology.orchestra.run.smithi084.stdout:Install 148 Packages 2024-05-22T22:49:09.461 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrade 11 Packages 2024-05-22T22:49:09.461 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:49:09.462 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 225 M 2024-05-22T22:49:09.462 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-05-22T22:49:10.280 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-05-22T22:49:10.555 INFO:teuthology.orchestra.run.smithi084.stdout:(1/159): ceph-19.0.0-2765.g2e1f7622.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-05-22T22:49:10.888 INFO:teuthology.orchestra.run.smithi084.stdout:(2/159): ceph-fuse-19.0.0-2765.g2e1f7622.el9.x8 2.6 MB/s | 889 kB 00:00 2024-05-22T22:49:10.955 INFO:teuthology.orchestra.run.smithi084.stdout:(3/159): ceph-immutable-object-cache-19.0.0-276 2.1 MB/s | 145 kB 00:00 2024-05-22T22:49:11.214 INFO:teuthology.orchestra.run.smithi084.stdout:(4/159): ceph-mds-19.0.0-2765.g2e1f7622.el9.x86 9.1 MB/s | 2.4 MB 00:00 2024-05-22T22:49:11.257 INFO:teuthology.orchestra.run.smithi204.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-22T22:49:11.258 INFO:teuthology.orchestra.run.smithi204.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-22T22:49:11.269 INFO:teuthology.orchestra.run.smithi204.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-22T22:49:11.270 INFO:teuthology.orchestra.run.smithi204.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-22T22:49:11.270 INFO:teuthology.orchestra.run.smithi204.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-22T22:49:11.271 INFO:teuthology.orchestra.run.smithi204.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-22T22:49:11.273 INFO:teuthology.orchestra.run.smithi204.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-22T22:49:11.275 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-22T22:49:11.276 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-22T22:49:11.337 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-05-22T22:49:11.352 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-05-22T22:49:11.352 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-05-22T22:49:11.352 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-05-22T22:49:11.353 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-05-22T22:49:11.353 INFO:teuthology.orchestra.run.smithi204.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-22T22:49:11.353 INFO:teuthology.orchestra.run.smithi204.stdout: ceph x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 6.6 k 2024-05-22T22:49:11.353 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 5.4 M 2024-05-22T22:49:11.353 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 889 k 2024-05-22T22:49:11.353 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 145 k 2024-05-22T22:49:11.353 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 1.5 M 2024-05-22T22:49:11.353 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 142 k 2024-05-22T22:49:11.353 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 3.5 M 2024-05-22T22:49:11.353 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 7.4 M 2024-05-22T22:49:11.353 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 49 k 2024-05-22T22:49:11.353 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 10 M 2024-05-22T22:49:11.353 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 48 M 2024-05-22T22:49:11.354 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 264 k 2024-05-22T22:49:11.354 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 763 k 2024-05-22T22:49:11.354 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 222 k 2024-05-22T22:49:11.354 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 27 k 2024-05-22T22:49:11.354 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 18 k 2024-05-22T22:49:11.354 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-22T22:49:11.354 INFO:teuthology.orchestra.run.smithi204.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-22T22:49:11.354 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-22T22:49:11.354 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 32 k 2024-05-22T22:49:11.354 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2 x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 745 k 2024-05-22T22:49:11.354 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 127 k 2024-05-22T22:49:11.354 INFO:teuthology.orchestra.run.smithi204.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-22T22:49:11.355 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-22T22:49:11.355 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-22T22:49:11.355 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-22T22:49:11.355 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-22T22:49:11.355 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-22T22:49:11.355 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 162 k 2024-05-22T22:49:11.355 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 319 k 2024-05-22T22:49:11.355 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 300 k 2024-05-22T22:49:11.355 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 99 k 2024-05-22T22:49:11.355 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 85 k 2024-05-22T22:49:11.355 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 3.1 M 2024-05-22T22:49:11.355 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 171 k 2024-05-22T22:49:11.356 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-22T22:49:11.356 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrading: 2024-05-22T22:49:11.356 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-22T22:49:11.356 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-22T22:49:11.356 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-22T22:49:11.356 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-22T22:49:11.356 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-22T22:49:11.356 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-22T22:49:11.356 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-22T22:49:11.356 INFO:teuthology.orchestra.run.smithi204.stdout: librados2 x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 3.4 M 2024-05-22T22:49:11.356 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1 x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 3.1 M 2024-05-22T22:49:11.356 INFO:teuthology.orchestra.run.smithi204.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-22T22:49:11.356 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-22T22:49:11.357 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-05-22T22:49:11.357 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-22T22:49:11.357 INFO:teuthology.orchestra.run.smithi204.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-22T22:49:11.357 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-22T22:49:11.357 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 21 M 2024-05-22T22:49:11.357 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 32 k 2024-05-22T22:49:11.357 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 2.4 M 2024-05-22T22:49:11.357 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 249 k 2024-05-22T22:49:11.357 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 4.7 M 2024-05-22T22:49:11.357 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 17 M 2024-05-22T22:49:11.357 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 16 k 2024-05-22T22:49:11.357 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 25 k 2024-05-22T22:49:11.357 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-22T22:49:11.358 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-22T22:49:11.358 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-22T22:49:11.358 INFO:teuthology.orchestra.run.smithi204.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-22T22:49:11.358 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-22T22:49:11.358 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-22T22:49:11.358 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-22T22:49:11.358 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-22T22:49:11.358 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 163 k 2024-05-22T22:49:11.358 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-22T22:49:11.358 INFO:teuthology.orchestra.run.smithi204.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-22T22:49:11.358 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-22T22:49:11.358 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-22T22:49:11.359 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-22T22:49:11.359 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1 x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 501 k 2024-05-22T22:49:11.359 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-22T22:49:11.359 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2 x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 5.3 M 2024-05-22T22:49:11.359 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-22T22:49:11.359 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-22T22:49:11.359 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-22T22:49:11.359 INFO:teuthology.orchestra.run.smithi204.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-22T22:49:11.359 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-22T22:49:11.359 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-22T22:49:11.359 INFO:teuthology.orchestra.run.smithi204.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-22T22:49:11.359 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-22T22:49:11.359 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-22T22:49:11.360 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-22T22:49:11.360 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-22T22:49:11.360 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-22T22:49:11.360 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-22T22:49:11.360 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-22T22:49:11.360 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-22T22:49:11.360 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-22T22:49:11.360 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-22T22:49:11.360 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 45 k 2024-05-22T22:49:11.360 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 135 k 2024-05-22T22:49:11.360 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-22T22:49:11.360 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-22T22:49:11.360 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-22T22:49:11.360 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-22T22:49:11.361 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-22T22:49:11.361 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-22T22:49:11.361 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-22T22:49:11.361 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-22T22:49:11.361 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-22T22:49:11.361 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-22T22:49:11.361 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-22T22:49:11.361 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-22T22:49:11.361 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-22T22:49:11.361 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-22T22:49:11.361 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-22T22:49:11.361 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-22T22:49:11.361 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-22T22:49:11.362 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-22T22:49:11.362 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-22T22:49:11.362 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-22T22:49:11.362 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-22T22:49:11.362 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-22T22:49:11.362 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-22T22:49:11.362 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-22T22:49:11.362 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-22T22:49:11.362 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-22T22:49:11.362 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-22T22:49:11.362 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-22T22:49:11.362 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-22T22:49:11.362 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-22T22:49:11.362 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-22T22:49:11.363 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-22T22:49:11.363 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-22T22:49:11.363 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-22T22:49:11.363 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-22T22:49:11.363 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-22T22:49:11.363 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-22T22:49:11.363 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-22T22:49:11.363 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-22T22:49:11.363 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-22T22:49:11.363 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-22T22:49:11.363 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-22T22:49:11.364 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-22T22:49:11.364 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-22T22:49:11.364 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-22T22:49:11.364 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-22T22:49:11.364 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-22T22:49:11.364 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-22T22:49:11.364 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-22T22:49:11.364 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-22T22:49:11.364 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-22T22:49:11.364 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-22T22:49:11.364 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-22T22:49:11.364 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-22T22:49:11.364 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-22T22:49:11.364 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-22T22:49:11.364 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-22T22:49:11.365 INFO:teuthology.orchestra.run.smithi204.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-22T22:49:11.365 INFO:teuthology.orchestra.run.smithi204.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-22T22:49:11.365 INFO:teuthology.orchestra.run.smithi204.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-22T22:49:11.365 INFO:teuthology.orchestra.run.smithi204.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-22T22:49:11.365 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-22T22:49:11.365 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-22T22:49:11.365 INFO:teuthology.orchestra.run.smithi204.stdout:Installing weak dependencies: 2024-05-22T22:49:11.365 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-22T22:49:11.365 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-22T22:49:11.365 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-05-22T22:49:11.365 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-05-22T22:49:11.365 INFO:teuthology.orchestra.run.smithi204.stdout:Install 148 Packages 2024-05-22T22:49:11.366 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrade 11 Packages 2024-05-22T22:49:11.366 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-22T22:49:11.366 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 225 M 2024-05-22T22:49:11.366 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-05-22T22:49:11.739 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 16 kB 00:00 2024-05-22T22:49:11.766 INFO:teuthology.orchestra.run.smithi204.stdout:(1/159): ceph-19.0.0-2765.g2e1f7622.el9.x86_64. 33 kB/s | 6.6 kB 00:00 2024-05-22T22:49:11.789 INFO:teuthology.orchestra.run.smithi084.stdout:(5/159): ceph-mgr-19.0.0-2765.g2e1f7622.el9.x86 2.6 MB/s | 1.5 MB 00:00 2024-05-22T22:49:12.006 INFO:teuthology.orchestra.run.smithi084.stdout:(6/159): ceph-base-19.0.0-2765.g2e1f7622.el9.x8 2.9 MB/s | 5.4 MB 00:01 2024-05-22T22:49:12.398 INFO:teuthology.orchestra.run.smithi084.stdout:(7/159): ceph-mon-19.0.0-2765.g2e1f7622.el9.x86 7.7 MB/s | 4.7 MB 00:00 2024-05-22T22:49:12.491 INFO:teuthology.orchestra.run.smithi204.stdout:(2/159): ceph-fuse-19.0.0-2765.g2e1f7622.el9.x8 1.2 MB/s | 889 kB 00:00 2024-05-22T22:49:12.675 INFO:teuthology.orchestra.run.smithi204.stdout:(3/159): ceph-immutable-object-cache-19.0.0-276 790 kB/s | 145 kB 00:00 2024-05-22T22:49:13.708 INFO:teuthology.orchestra.run.smithi204.stdout:(4/159): ceph-mds-19.0.0-2765.g2e1f7622.el9.x86 2.3 MB/s | 2.4 MB 00:01 2024-05-22T22:49:13.784 INFO:teuthology.orchestra.run.smithi204.stdout:(5/159): ceph-base-19.0.0-2765.g2e1f7622.el9.x8 2.4 MB/s | 5.4 MB 00:02 2024-05-22T22:49:14.350 INFO:teuthology.orchestra.run.smithi204.stdout:(6/159): ceph-mgr-19.0.0-2765.g2e1f7622.el9.x86 2.3 MB/s | 1.5 MB 00:00 2024-05-22T22:49:14.540 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 7.9 MB/s | 21 MB 00:02 2024-05-22T22:49:15.073 INFO:teuthology.orchestra.run.smithi084.stdout:(8/159): ceph-radosgw-19.0.0-2765.g2e1f7622.el9 3.8 MB/s | 10 MB 00:02 2024-05-22T22:49:15.173 INFO:teuthology.orchestra.run.smithi084.stdout:(9/159): ceph-selinux-19.0.0-2765.g2e1f7622.el9 252 kB/s | 25 kB 00:00 2024-05-22T22:49:15.384 INFO:teuthology.orchestra.run.smithi204.stdout:(7/159): ceph-mon-19.0.0-2765.g2e1f7622.el9.x86 2.9 MB/s | 4.7 MB 00:01 2024-05-22T22:49:15.884 INFO:teuthology.orchestra.run.smithi204.stdout:(8/159): ceph-common-19.0.0-2765.g2e1f7622.el9. 4.9 MB/s | 21 MB 00:04 2024-05-22T22:49:15.951 INFO:teuthology.orchestra.run.smithi204.stdout:(9/159): ceph-selinux-19.0.0-2765.g2e1f7622.el9 379 kB/s | 25 kB 00:00 2024-05-22T22:49:16.331 INFO:teuthology.orchestra.run.smithi084.stdout:(10/159): ceph-osd-19.0.0-2765.g2e1f7622.el9.x8 3.9 MB/s | 17 MB 00:04 2024-05-22T22:49:16.423 INFO:teuthology.orchestra.run.smithi084.stdout:(11/159): cephfs-mirror-19.0.0-2765.g2e1f7622.e 2.4 MB/s | 222 kB 00:00 2024-05-22T22:49:16.507 INFO:teuthology.orchestra.run.smithi084.stdout:(12/159): cephfs-shell-19.0.0-2765.g2e1f7622.el 325 kB/s | 27 kB 00:00 2024-05-22T22:49:16.590 INFO:teuthology.orchestra.run.smithi084.stdout:(13/159): libcephfs-devel-19.0.0-2765.g2e1f7622 381 kB/s | 32 kB 00:00 2024-05-22T22:49:16.866 INFO:teuthology.orchestra.run.smithi084.stdout:(14/159): libcephfs2-19.0.0-2765.g2e1f7622.el9. 2.6 MB/s | 745 kB 00:00 2024-05-22T22:49:16.966 INFO:teuthology.orchestra.run.smithi084.stdout:(15/159): libcephsqlite-19.0.0-2765.g2e1f7622.e 1.6 MB/s | 163 kB 00:00 2024-05-22T22:49:17.050 INFO:teuthology.orchestra.run.smithi084.stdout:(16/159): librados-devel-19.0.0-2765.g2e1f7622. 1.5 MB/s | 127 kB 00:00 2024-05-22T22:49:17.275 INFO:teuthology.orchestra.run.smithi084.stdout:(17/159): libradosstriper1-19.0.0-2765.g2e1f762 2.2 MB/s | 501 kB 00:00 2024-05-22T22:49:17.825 INFO:teuthology.orchestra.run.smithi084.stdout:(18/159): ceph-common-19.0.0-2765.g2e1f7622.el9 2.7 MB/s | 21 MB 00:07 2024-05-22T22:49:17.917 INFO:teuthology.orchestra.run.smithi084.stdout:(19/159): python3-ceph-argparse-19.0.0-2765.g2e 494 kB/s | 45 kB 00:00 2024-05-22T22:49:18.009 INFO:teuthology.orchestra.run.smithi084.stdout:(20/159): python3-ceph-common-19.0.0-2765.g2e1f 1.4 MB/s | 135 kB 00:00 2024-05-22T22:49:18.168 INFO:teuthology.orchestra.run.smithi084.stdout:(21/159): python3-cephfs-19.0.0-2765.g2e1f7622. 1.0 MB/s | 162 kB 00:00 2024-05-22T22:49:18.401 INFO:teuthology.orchestra.run.smithi084.stdout:(22/159): python3-rados-19.0.0-2765.g2e1f7622.e 1.3 MB/s | 319 kB 00:00 2024-05-22T22:49:18.635 INFO:teuthology.orchestra.run.smithi084.stdout:(23/159): python3-rbd-19.0.0-2765.g2e1f7622.el9 1.3 MB/s | 300 kB 00:00 2024-05-22T22:49:18.735 INFO:teuthology.orchestra.run.smithi084.stdout:(24/159): python3-rgw-19.0.0-2765.g2e1f7622.el9 996 kB/s | 99 kB 00:00 2024-05-22T22:49:18.835 INFO:teuthology.orchestra.run.smithi084.stdout:(25/159): rbd-fuse-19.0.0-2765.g2e1f7622.el9.x8 852 kB/s | 85 kB 00:00 2024-05-22T22:49:19.036 INFO:teuthology.orchestra.run.smithi084.stdout:(26/159): librgw2-19.0.0-2765.g2e1f7622.el9.x86 3.0 MB/s | 5.3 MB 00:01 2024-05-22T22:49:19.068 INFO:teuthology.orchestra.run.smithi204.stdout:(10/159): ceph-osd-19.0.0-2765.g2e1f7622.el9.x8 3.6 MB/s | 17 MB 00:04 2024-05-22T22:49:19.103 INFO:teuthology.orchestra.run.smithi084.stdout:(27/159): rbd-nbd-19.0.0-2765.g2e1f7622.el9.x86 2.5 MB/s | 171 kB 00:00 2024-05-22T22:49:19.177 INFO:teuthology.orchestra.run.smithi204.stdout:(11/159): cephfs-mirror-19.0.0-2765.g2e1f7622.e 2.0 MB/s | 222 kB 00:00 2024-05-22T22:49:19.203 INFO:teuthology.orchestra.run.smithi084.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2765.g 317 kB/s | 32 kB 00:00 2024-05-22T22:49:19.260 INFO:teuthology.orchestra.run.smithi204.stdout:(12/159): cephfs-shell-19.0.0-2765.g2e1f7622.el 325 kB/s | 27 kB 00:00 2024-05-22T22:49:19.303 INFO:teuthology.orchestra.run.smithi084.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2765.g2e1f762 1.4 MB/s | 142 kB 00:00 2024-05-22T22:49:19.419 INFO:teuthology.orchestra.run.smithi204.stdout:(13/159): ceph-radosgw-19.0.0-2765.g2e1f7622.el 2.5 MB/s | 10 MB 00:04 2024-05-22T22:49:19.444 INFO:teuthology.orchestra.run.smithi204.stdout:(14/159): libcephfs-devel-19.0.0-2765.g2e1f7622 173 kB/s | 32 kB 00:00 2024-05-22T22:49:19.536 INFO:teuthology.orchestra.run.smithi204.stdout:(15/159): libcephsqlite-19.0.0-2765.g2e1f7622.e 1.7 MB/s | 163 kB 00:00 2024-05-22T22:49:19.682 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-05-22T22:49:19.812 INFO:teuthology.orchestra.run.smithi084.stdout:(30/159): rbd-mirror-19.0.0-2765.g2e1f7622.el9. 3.2 MB/s | 3.1 MB 00:00 2024-05-22T22:49:19.904 INFO:teuthology.orchestra.run.smithi084.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2765.g2e1f7 5.9 MB/s | 3.5 MB 00:00 2024-05-22T22:49:19.970 INFO:teuthology.orchestra.run.smithi204.stdout:(16/159): ceph-test-19.0.0-2765.g2e1f7622.el9.x 12 MB/s | 48 MB 00:04 2024-05-22T22:49:19.971 INFO:teuthology.orchestra.run.smithi084.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2765.g2e 3.6 MB/s | 249 kB 00:00 2024-05-22T22:49:19.995 INFO:teuthology.orchestra.run.smithi204.stdout:(17/159): librados-devel-19.0.0-2765.g2e1f7622. 276 kB/s | 127 kB 00:00 2024-05-22T22:49:20.029 INFO:teuthology.orchestra.run.smithi204.stdout:(18/159): libcephfs2-19.0.0-2765.g2e1f7622.el9. 1.2 MB/s | 745 kB 00:00 2024-05-22T22:49:20.029 INFO:teuthology.orchestra.run.smithi084.stdout:(33/159): ceph-mgr-rook-19.0.0-2765.g2e1f7622.e 849 kB/s | 49 kB 00:00 2024-05-22T22:49:20.054 INFO:teuthology.orchestra.run.smithi204.stdout:(19/159): libradosstriper1-19.0.0-2765.g2e1f762 5.9 MB/s | 501 kB 00:00 2024-05-22T22:49:20.079 INFO:teuthology.orchestra.run.smithi204.stdout:(20/159): python3-ceph-argparse-19.0.0-2765.g2e 901 kB/s | 45 kB 00:00 2024-05-22T22:49:20.088 INFO:teuthology.orchestra.run.smithi084.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2765.g2 281 kB/s | 16 kB 00:00 2024-05-22T22:49:20.104 INFO:teuthology.orchestra.run.smithi204.stdout:(21/159): python3-ceph-common-19.0.0-2765.g2e1f 2.6 MB/s | 135 kB 00:00 2024-05-22T22:49:20.163 INFO:teuthology.orchestra.run.smithi204.stdout:(22/159): python3-cephfs-19.0.0-2765.g2e1f7622. 1.9 MB/s | 162 kB 00:00 2024-05-22T22:49:20.189 INFO:teuthology.orchestra.run.smithi204.stdout:(23/159): python3-rados-19.0.0-2765.g2e1f7622.e 3.7 MB/s | 319 kB 00:00 2024-05-22T22:49:20.197 INFO:teuthology.orchestra.run.smithi084.stdout:(35/159): ceph-volume-19.0.0-2765.g2e1f7622.el9 2.4 MB/s | 264 kB 00:00 2024-05-22T22:49:20.239 INFO:teuthology.orchestra.run.smithi204.stdout:(24/159): python3-rgw-19.0.0-2765.g2e1f7622.el9 1.9 MB/s | 99 kB 00:00 2024-05-22T22:49:20.264 INFO:teuthology.orchestra.run.smithi204.stdout:(25/159): python3-rbd-19.0.0-2765.g2e1f7622.el9 2.9 MB/s | 300 kB 00:00 2024-05-22T22:49:20.306 INFO:teuthology.orchestra.run.smithi204.stdout:(26/159): rbd-fuse-19.0.0-2765.g2e1f7622.el9.x8 1.2 MB/s | 85 kB 00:00 2024-05-22T22:49:20.364 INFO:teuthology.orchestra.run.smithi084.stdout:(36/159): cephadm-19.0.0-2765.g2e1f7622.el9.noa 4.5 MB/s | 763 kB 00:00 2024-05-22T22:49:20.381 INFO:teuthology.orchestra.run.smithi204.stdout:(27/159): rbd-nbd-19.0.0-2765.g2e1f7622.el9.x86 2.2 MB/s | 171 kB 00:00 2024-05-22T22:49:20.422 INFO:teuthology.orchestra.run.smithi084.stdout:(37/159): cephfs-top-19.0.0-2765.g2e1f7622.el9. 313 kB/s | 18 kB 00:00 2024-05-22T22:49:20.449 INFO:teuthology.orchestra.run.smithi204.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2765.g 473 kB/s | 32 kB 00:00 2024-05-22T22:49:20.499 INFO:teuthology.orchestra.run.smithi204.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2765.g2e1f762 2.8 MB/s | 142 kB 00:00 2024-05-22T22:49:20.816 INFO:teuthology.orchestra.run.smithi204.stdout:(30/159): ceph-mgr-dashboard-19.0.0-2765.g2e1f7 11 MB/s | 3.5 MB 00:00 2024-05-22T22:49:20.999 INFO:teuthology.orchestra.run.smithi204.stdout:(31/159): librgw2-19.0.0-2765.g2e1f7622.el9.x86 5.3 MB/s | 5.3 MB 00:01 2024-05-22T22:49:21.098 INFO:teuthology.orchestra.run.smithi084.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 119 kB/s | 80 kB 00:00 2024-05-22T22:49:21.108 INFO:teuthology.orchestra.run.smithi204.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2765.g2e 2.2 MB/s | 249 kB 00:00 2024-05-22T22:49:21.167 INFO:teuthology.orchestra.run.smithi204.stdout:(33/159): rbd-mirror-19.0.0-2765.g2e1f7622.el9. 3.4 MB/s | 3.1 MB 00:00 2024-05-22T22:49:21.184 INFO:teuthology.orchestra.run.smithi204.stdout:(34/159): ceph-mgr-rook-19.0.0-2765.g2e1f7622.e 654 kB/s | 49 kB 00:00 2024-05-22T22:49:21.242 INFO:teuthology.orchestra.run.smithi204.stdout:(35/159): ceph-prometheus-alerts-19.0.0-2765.g2 219 kB/s | 16 kB 00:00 2024-05-22T22:49:21.368 INFO:teuthology.orchestra.run.smithi204.stdout:(36/159): ceph-volume-19.0.0-2765.g2e1f7622.el9 1.4 MB/s | 264 kB 00:00 2024-05-22T22:49:21.460 INFO:teuthology.orchestra.run.smithi204.stdout:(37/159): ceph-mgr-diskprediction-local-19.0.0- 12 MB/s | 7.4 MB 00:00 2024-05-22T22:49:21.477 INFO:teuthology.orchestra.run.smithi204.stdout:(38/159): cephfs-top-19.0.0-2765.g2e1f7622.el9. 168 kB/s | 18 kB 00:00 2024-05-22T22:49:21.510 INFO:teuthology.orchestra.run.smithi204.stdout:(39/159): cephadm-19.0.0-2765.g2e1f7622.el9.noa 2.8 MB/s | 763 kB 00:00 2024-05-22T22:49:21.590 INFO:teuthology.orchestra.run.smithi084.stdout:(39/159): ceph-mgr-diskprediction-local-19.0.0- 4.2 MB/s | 7.4 MB 00:01 2024-05-22T22:49:21.791 INFO:teuthology.orchestra.run.smithi096.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-22T22:49:21.792 INFO:teuthology.orchestra.run.smithi096.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-22T22:49:21.803 INFO:teuthology.orchestra.run.smithi096.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-22T22:49:21.804 INFO:teuthology.orchestra.run.smithi096.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-22T22:49:21.804 INFO:teuthology.orchestra.run.smithi096.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-22T22:49:21.805 INFO:teuthology.orchestra.run.smithi096.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-22T22:49:21.807 INFO:teuthology.orchestra.run.smithi096.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-22T22:49:21.809 INFO:teuthology.orchestra.run.smithi096.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-22T22:49:21.810 INFO:teuthology.orchestra.run.smithi096.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-22T22:49:21.872 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-05-22T22:49:21.889 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-05-22T22:49:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-05-22T22:49:21.889 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-05-22T22:49:21.889 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-05-22T22:49:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-22T22:49:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: ceph x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 6.6 k 2024-05-22T22:49:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 5.4 M 2024-05-22T22:49:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 889 k 2024-05-22T22:49:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 145 k 2024-05-22T22:49:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 1.5 M 2024-05-22T22:49:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 142 k 2024-05-22T22:49:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 3.5 M 2024-05-22T22:49:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 7.4 M 2024-05-22T22:49:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 49 k 2024-05-22T22:49:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 10 M 2024-05-22T22:49:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 48 M 2024-05-22T22:49:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 264 k 2024-05-22T22:49:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 763 k 2024-05-22T22:49:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-mirror x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 222 k 2024-05-22T22:49:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-shell x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 27 k 2024-05-22T22:49:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-top noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 18 k 2024-05-22T22:49:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-22T22:49:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-22T22:49:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-22T22:49:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 32 k 2024-05-22T22:49:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2 x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 745 k 2024-05-22T22:49:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 127 k 2024-05-22T22:49:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-22T22:49:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-22T22:49:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-22T22:49:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-22T22:49:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-22T22:49:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-22T22:49:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 162 k 2024-05-22T22:49:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 319 k 2024-05-22T22:49:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 300 k 2024-05-22T22:49:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 99 k 2024-05-22T22:49:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 85 k 2024-05-22T22:49:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 3.1 M 2024-05-22T22:49:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 171 k 2024-05-22T22:49:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-22T22:49:21.892 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrading: 2024-05-22T22:49:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-22T22:49:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-22T22:49:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-22T22:49:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-22T22:49:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-22T22:49:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-22T22:49:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-22T22:49:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: librados2 x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 3.4 M 2024-05-22T22:49:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1 x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 3.1 M 2024-05-22T22:49:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-22T22:49:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-22T22:49:21.893 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-05-22T22:49:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-22T22:49:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-22T22:49:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-22T22:49:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 21 M 2024-05-22T22:49:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 32 k 2024-05-22T22:49:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 2.4 M 2024-05-22T22:49:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 249 k 2024-05-22T22:49:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 4.7 M 2024-05-22T22:49:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 17 M 2024-05-22T22:49:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2765.g2e1f7622.el9 ceph-noarch 16 k 2024-05-22T22:49:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 25 k 2024-05-22T22:49:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-22T22:49:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-22T22:49:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-22T22:49:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-22T22:49:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-22T22:49:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-22T22:49:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-22T22:49:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-22T22:49:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 163 k 2024-05-22T22:49:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-22T22:49:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-22T22:49:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-22T22:49:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-22T22:49:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-22T22:49:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1 x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 501 k 2024-05-22T22:49:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-22T22:49:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2 x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 5.3 M 2024-05-22T22:49:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-22T22:49:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-22T22:49:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-22T22:49:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-22T22:49:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-22T22:49:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-22T22:49:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-22T22:49:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-22T22:49:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-22T22:49:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-22T22:49:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-22T22:49:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-22T22:49:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-22T22:49:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-22T22:49:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-22T22:49:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-22T22:49:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-22T22:49:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 45 k 2024-05-22T22:49:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common x86_64 2:19.0.0-2765.g2e1f7622.el9 ceph 135 k 2024-05-22T22:49:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-22T22:49:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-22T22:49:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-22T22:49:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-22T22:49:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-22T22:49:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-22T22:49:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-22T22:49:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-22T22:49:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-22T22:49:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-22T22:49:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-22T22:49:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-22T22:49:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-22T22:49:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-22T22:49:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-22T22:49:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-22T22:49:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-22T22:49:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-22T22:49:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-22T22:49:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-22T22:49:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-22T22:49:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-22T22:49:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-22T22:49:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-22T22:49:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-22T22:49:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-22T22:49:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-22T22:49:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-22T22:49:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-22T22:49:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-22T22:49:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-22T22:49:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-22T22:49:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-22T22:49:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-22T22:49:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-22T22:49:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-22T22:49:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-22T22:49:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-22T22:49:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-22T22:49:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-22T22:49:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-22T22:49:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-22T22:49:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-22T22:49:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-22T22:49:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-22T22:49:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-22T22:49:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-22T22:49:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-22T22:49:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-22T22:49:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-22T22:49:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-22T22:49:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-22T22:49:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-22T22:49:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-22T22:49:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-22T22:49:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-22T22:49:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-22T22:49:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-22T22:49:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-22T22:49:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-22T22:49:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-22T22:49:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-22T22:49:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-22T22:49:21.902 INFO:teuthology.orchestra.run.smithi096.stdout:Installing weak dependencies: 2024-05-22T22:49:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-22T22:49:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-22T22:49:21.902 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-05-22T22:49:21.902 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-05-22T22:49:21.902 INFO:teuthology.orchestra.run.smithi096.stdout:Install 148 Packages 2024-05-22T22:49:21.902 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrade 11 Packages 2024-05-22T22:49:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-22T22:49:21.903 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 225 M 2024-05-22T22:49:21.903 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-05-22T22:49:22.148 INFO:teuthology.orchestra.run.smithi084.stdout:(40/159): libgfortran-11.4.1-3.el9.x86_64.rpm 765 kB/s | 803 kB 00:01 2024-05-22T22:49:22.152 INFO:teuthology.orchestra.run.smithi204.stdout:(40/159): fuse-2.9.9-15.el9.x86_64.rpm 116 kB/s | 80 kB 00:00 2024-05-22T22:49:22.311 INFO:teuthology.orchestra.run.smithi204.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 244 kB/s | 195 kB 00:00 2024-05-22T22:49:22.390 INFO:teuthology.orchestra.run.smithi084.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 244 kB/s | 195 kB 00:00 2024-05-22T22:49:22.494 INFO:teuthology.orchestra.run.smithi204.stdout:(42/159): libgfortran-11.4.1-3.el9.x86_64.rpm 789 kB/s | 803 kB 00:01 2024-05-22T22:49:22.850 INFO:teuthology.orchestra.run.smithi096.stdout:(1/159): ceph-19.0.0-2765.g2e1f7622.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-05-22T22:49:22.978 INFO:teuthology.orchestra.run.smithi204.stdout:(43/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 307 kB/s | 253 kB 00:00 2024-05-22T22:49:22.999 INFO:teuthology.orchestra.run.smithi084.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 298 kB/s | 253 kB 00:00 2024-05-22T22:49:23.125 INFO:teuthology.orchestra.run.smithi096.stdout:(2/159): ceph-fuse-19.0.0-2765.g2e1f7622.el9.x8 3.2 MB/s | 889 kB 00:00 2024-05-22T22:49:23.233 INFO:teuthology.orchestra.run.smithi096.stdout:(3/159): ceph-immutable-object-cache-19.0.0-276 1.3 MB/s | 145 kB 00:00 2024-05-22T22:49:23.270 INFO:teuthology.orchestra.run.smithi204.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 138 kB/s | 106 kB 00:00 2024-05-22T22:49:23.373 INFO:teuthology.orchestra.run.smithi204.stdout:(45/159): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:01 2024-05-22T22:49:23.432 INFO:teuthology.orchestra.run.smithi084.stdout:(43/159): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:01 2024-05-22T22:49:23.667 INFO:teuthology.orchestra.run.smithi096.stdout:(4/159): ceph-base-19.0.0-2765.g2e1f7622.el9.x8 5.2 MB/s | 5.4 MB 00:01 2024-05-22T22:49:23.707 INFO:teuthology.orchestra.run.smithi084.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 150 kB/s | 106 kB 00:00 2024-05-22T22:49:23.717 INFO:teuthology.orchestra.run.smithi096.stdout:(5/159): ceph-mds-19.0.0-2765.g2e1f7622.el9.x86 4.9 MB/s | 2.4 MB 00:00 2024-05-22T22:49:23.771 INFO:teuthology.orchestra.run.smithi204.stdout:(46/159): python3-pycparser-2.20-6.el9.noarch.r 171 kB/s | 135 kB 00:00 2024-05-22T22:49:23.884 INFO:teuthology.orchestra.run.smithi096.stdout:(6/159): ceph-mgr-19.0.0-2765.g2e1f7622.el9.x86 6.9 MB/s | 1.5 MB 00:00 2024-05-22T22:49:24.054 INFO:teuthology.orchestra.run.smithi204.stdout:(47/159): python3-requests-2.25.1-8.el9.noarch. 159 kB/s | 125 kB 00:00 2024-05-22T22:49:24.189 INFO:teuthology.orchestra.run.smithi204.stdout:(48/159): python3-urllib3-1.26.5-5.el9.noarch.r 263 kB/s | 215 kB 00:00 2024-05-22T22:49:24.241 INFO:teuthology.orchestra.run.smithi084.stdout:(45/159): ceph-test-19.0.0-2765.g2e1f7622.el9.x 5.3 MB/s | 48 MB 00:09 2024-05-22T22:49:24.526 INFO:teuthology.orchestra.run.smithi096.stdout:(7/159): ceph-mon-19.0.0-2765.g2e1f7622.el9.x86 5.8 MB/s | 4.7 MB 00:00 2024-05-22T22:49:24.608 INFO:teuthology.orchestra.run.smithi084.stdout:(46/159): python3-pycparser-2.20-6.el9.noarch.r 115 kB/s | 135 kB 00:01 2024-05-22T22:49:24.730 INFO:teuthology.orchestra.run.smithi204.stdout:(49/159): automake-1.16.2-8.el9.noarch.rpm 719 kB/s | 689 kB 00:00 2024-05-22T22:49:24.802 INFO:teuthology.orchestra.run.smithi084.stdout:(47/159): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-05-22T22:49:24.825 INFO:teuthology.orchestra.run.smithi084.stdout:(48/159): python3-requests-2.25.1-8.el9.noarch. 112 kB/s | 125 kB 00:01 2024-05-22T22:49:24.867 INFO:teuthology.orchestra.run.smithi084.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-05-22T22:49:24.897 INFO:teuthology.orchestra.run.smithi204.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 151 kB/s | 107 kB 00:00 2024-05-22T22:49:24.917 INFO:teuthology.orchestra.run.smithi084.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-05-22T22:49:24.993 INFO:teuthology.orchestra.run.smithi084.stdout:(51/159): boost-program-options-1.75.0-8.el9.x8 640 kB/s | 107 kB 00:00 2024-05-22T22:49:24.997 INFO:teuthology.orchestra.run.smithi204.stdout:(51/159): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-05-22T22:49:25.018 INFO:teuthology.orchestra.run.smithi084.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 3.1 MB/s | 313 kB 00:00 2024-05-22T22:49:25.043 INFO:teuthology.orchestra.run.smithi084.stdout:(53/159): python3-urllib3-1.26.5-5.el9.noarch.r 268 kB/s | 215 kB 00:00 2024-05-22T22:49:25.069 INFO:teuthology.orchestra.run.smithi084.stdout:(54/159): flexiblas-3.0.4-8.el9.x86_64.rpm 431 kB/s | 33 kB 00:00 2024-05-22T22:49:25.094 INFO:teuthology.orchestra.run.smithi084.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-05-22T22:49:25.127 INFO:teuthology.orchestra.run.smithi096.stdout:(8/159): ceph-common-19.0.0-2765.g2e1f7622.el9. 8.4 MB/s | 21 MB 00:02 2024-05-22T22:49:25.161 INFO:teuthology.orchestra.run.smithi084.stdout:(56/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-05-22T22:49:25.186 INFO:teuthology.orchestra.run.smithi084.stdout:(57/159): libaio-devel-0.3.111-13.el9.x86_64.rp 95 kB/s | 11 kB 00:00 2024-05-22T22:49:25.202 INFO:teuthology.orchestra.run.smithi096.stdout:(9/159): ceph-selinux-19.0.0-2765.g2e1f7622.el9 336 kB/s | 25 kB 00:00 2024-05-22T22:49:25.211 INFO:teuthology.orchestra.run.smithi084.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-05-22T22:49:25.237 INFO:teuthology.orchestra.run.smithi084.stdout:(59/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2024-05-22T22:49:25.271 INFO:teuthology.orchestra.run.smithi084.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 582 kB 00:00 2024-05-22T22:49:25.287 INFO:teuthology.orchestra.run.smithi084.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 509 kB/s | 25 kB 00:00 2024-05-22T22:49:25.304 INFO:teuthology.orchestra.run.smithi084.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-05-22T22:49:25.355 INFO:teuthology.orchestra.run.smithi084.stdout:(63/159): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-05-22T22:49:25.380 INFO:teuthology.orchestra.run.smithi084.stdout:(64/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-05-22T22:49:25.397 INFO:teuthology.orchestra.run.smithi204.stdout:(52/159): elfutils-libelf-devel-0.191-4.el9.x86 33 kB/s | 22 kB 00:00 2024-05-22T22:49:25.405 INFO:teuthology.orchestra.run.smithi084.stdout:(65/159): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-05-22T22:49:25.430 INFO:teuthology.orchestra.run.smithi084.stdout:(66/159): openblas-0.3.21-2.el9.x86_64.rpm 468 kB/s | 35 kB 00:00 2024-05-22T22:49:25.572 INFO:teuthology.orchestra.run.smithi084.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-22T22:49:25.631 INFO:teuthology.orchestra.run.smithi084.stdout:(68/159): postgresql-contrib-13.14-1.el9.x86_64 15 MB/s | 911 kB 00:00 2024-05-22T22:49:25.647 INFO:teuthology.orchestra.run.smithi204.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 52 kB/s | 33 kB 00:00 2024-05-22T22:49:25.706 INFO:teuthology.orchestra.run.smithi084.stdout:(69/159): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2024-05-22T22:49:25.731 INFO:teuthology.orchestra.run.smithi084.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-05-22T22:49:25.752 INFO:teuthology.orchestra.run.smithi096.stdout:(10/159): ceph-osd-19.0.0-2765.g2e1f7622.el9.x8 9.0 MB/s | 17 MB 00:01 2024-05-22T22:49:25.798 INFO:teuthology.orchestra.run.smithi084.stdout:(71/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-05-22T22:49:25.832 INFO:teuthology.orchestra.run.smithi084.stdout:(72/159): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-05-22T22:49:25.844 INFO:teuthology.orchestra.run.smithi096.stdout:(11/159): cephfs-mirror-19.0.0-2765.g2e1f7622.e 2.4 MB/s | 222 kB 00:00 2024-05-22T22:49:25.857 INFO:teuthology.orchestra.run.smithi084.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-22T22:49:25.898 INFO:teuthology.orchestra.run.smithi204.stdout:(54/159): flex-2.6.4-9.el9.x86_64.rpm 313 kB/s | 313 kB 00:01 2024-05-22T22:49:25.908 INFO:teuthology.orchestra.run.smithi084.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-05-22T22:49:25.911 INFO:teuthology.orchestra.run.smithi096.stdout:(12/159): cephfs-shell-19.0.0-2765.g2e1f7622.el 406 kB/s | 27 kB 00:00 2024-05-22T22:49:25.983 INFO:teuthology.orchestra.run.smithi084.stdout:(75/159): postgresql-server-13.14-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-05-22T22:49:26.008 INFO:teuthology.orchestra.run.smithi084.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-05-22T22:49:26.033 INFO:teuthology.orchestra.run.smithi084.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-05-22T22:49:26.059 INFO:teuthology.orchestra.run.smithi084.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-05-22T22:49:26.078 INFO:teuthology.orchestra.run.smithi096.stdout:(13/159): ceph-radosgw-19.0.0-2765.g2e1f7622.el 6.6 MB/s | 10 MB 00:01 2024-05-22T22:49:26.084 INFO:teuthology.orchestra.run.smithi084.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-05-22T22:49:26.095 INFO:teuthology.orchestra.run.smithi096.stdout:(14/159): libcephfs-devel-19.0.0-2765.g2e1f7622 173 kB/s | 32 kB 00:00 2024-05-22T22:49:26.101 INFO:teuthology.orchestra.run.smithi084.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-05-22T22:49:26.126 INFO:teuthology.orchestra.run.smithi084.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 828 kB/s | 35 kB 00:00 2024-05-22T22:49:26.189 INFO:teuthology.orchestra.run.smithi204.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 35 kB/s | 18 kB 00:00 2024-05-22T22:49:26.195 INFO:teuthology.orchestra.run.smithi096.stdout:(15/159): libcephsqlite-19.0.0-2765.g2e1f7622.e 1.6 MB/s | 163 kB 00:00 2024-05-22T22:49:26.210 INFO:teuthology.orchestra.run.smithi084.stdout:(82/159): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-05-22T22:49:26.235 INFO:teuthology.orchestra.run.smithi084.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-05-22T22:49:26.245 INFO:teuthology.orchestra.run.smithi096.stdout:(16/159): libcephfs2-19.0.0-2765.g2e1f7622.el9. 4.4 MB/s | 745 kB 00:00 2024-05-22T22:49:26.260 INFO:teuthology.orchestra.run.smithi084.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-05-22T22:49:26.286 INFO:teuthology.orchestra.run.smithi084.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-22T22:49:26.287 INFO:teuthology.orchestra.run.smithi096.stdout:(17/159): librados-devel-19.0.0-2765.g2e1f7622. 1.3 MB/s | 127 kB 00:00 2024-05-22T22:49:26.311 INFO:teuthology.orchestra.run.smithi084.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 836 kB/s | 42 kB 00:00 2024-05-22T22:49:26.337 INFO:teuthology.orchestra.run.smithi084.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-05-22T22:49:26.362 INFO:teuthology.orchestra.run.smithi084.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-22T22:49:26.387 INFO:teuthology.orchestra.run.smithi084.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-22T22:49:26.412 INFO:teuthology.orchestra.run.smithi084.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-22T22:49:26.423 INFO:teuthology.orchestra.run.smithi204.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-05-22T22:49:26.429 INFO:teuthology.orchestra.run.smithi096.stdout:(18/159): libradosstriper1-19.0.0-2765.g2e1f762 2.7 MB/s | 501 kB 00:00 2024-05-22T22:49:26.437 INFO:teuthology.orchestra.run.smithi084.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-22T22:49:26.521 INFO:teuthology.orchestra.run.smithi084.stdout:(92/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-05-22T22:49:26.529 INFO:teuthology.orchestra.run.smithi096.stdout:(19/159): python3-ceph-argparse-19.0.0-2765.g2e 452 kB/s | 45 kB 00:00 2024-05-22T22:49:26.532 INFO:teuthology.orchestra.run.smithi204.stdout:(57/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.6 MB/s | 3.0 MB 00:01 2024-05-22T22:49:26.546 INFO:teuthology.orchestra.run.smithi084.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-05-22T22:49:26.572 INFO:teuthology.orchestra.run.smithi084.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-05-22T22:49:26.597 INFO:teuthology.orchestra.run.smithi084.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-22T22:49:26.622 INFO:teuthology.orchestra.run.smithi084.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-05-22T22:49:26.648 INFO:teuthology.orchestra.run.smithi084.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-05-22T22:49:26.654 INFO:teuthology.orchestra.run.smithi096.stdout:(20/159): python3-ceph-common-19.0.0-2765.g2e1f 1.1 MB/s | 135 kB 00:00 2024-05-22T22:49:26.757 INFO:teuthology.orchestra.run.smithi084.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 2.2 MB/s | 305 kB 00:00 2024-05-22T22:49:26.763 INFO:teuthology.orchestra.run.smithi096.stdout:(21/159): python3-cephfs-19.0.0-2765.g2e1f7622. 1.5 MB/s | 162 kB 00:00 2024-05-22T22:49:26.782 INFO:teuthology.orchestra.run.smithi084.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 432 kB/s | 58 kB 00:00 2024-05-22T22:49:26.807 INFO:teuthology.orchestra.run.smithi084.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-05-22T22:49:26.824 INFO:teuthology.orchestra.run.smithi084.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-05-22T22:49:26.841 INFO:teuthology.orchestra.run.smithi084.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-05-22T22:49:26.880 INFO:teuthology.orchestra.run.smithi096.stdout:(22/159): python3-rados-19.0.0-2765.g2e1f7622.e 2.7 MB/s | 319 kB 00:00 2024-05-22T22:49:26.891 INFO:teuthology.orchestra.run.smithi084.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 333 kB/s | 22 kB 00:00 2024-05-22T22:49:26.982 INFO:teuthology.orchestra.run.smithi204.stdout:(58/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 202 kB/s | 160 kB 00:00 2024-05-22T22:49:26.997 INFO:teuthology.orchestra.run.smithi096.stdout:(23/159): librgw2-19.0.0-2765.g2e1f7622.el9.x86 7.5 MB/s | 5.3 MB 00:00 2024-05-22T22:49:27.022 INFO:teuthology.orchestra.run.smithi096.stdout:(24/159): python3-rbd-19.0.0-2765.g2e1f7622.el9 2.1 MB/s | 300 kB 00:00 2024-05-22T22:49:27.066 INFO:teuthology.orchestra.run.smithi204.stdout:(59/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 73 kB/s | 45 kB 00:00 2024-05-22T22:49:27.081 INFO:teuthology.orchestra.run.smithi096.stdout:(25/159): python3-rgw-19.0.0-2765.g2e1f7622.el9 1.2 MB/s | 99 kB 00:00 2024-05-22T22:49:27.100 INFO:teuthology.orchestra.run.smithi084.stdout:(104/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-05-22T22:49:27.114 INFO:teuthology.orchestra.run.smithi096.stdout:(26/159): rbd-fuse-19.0.0-2765.g2e1f7622.el9.x8 925 kB/s | 85 kB 00:00 2024-05-22T22:49:27.150 INFO:teuthology.orchestra.run.smithi084.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 2.1 MB/s | 548 kB 00:00 2024-05-22T22:49:27.209 INFO:teuthology.orchestra.run.smithi084.stdout:(106/159): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-05-22T22:49:27.234 INFO:teuthology.orchestra.run.smithi084.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 233 kB/s | 19 kB 00:00 2024-05-22T22:49:27.240 INFO:teuthology.orchestra.run.smithi096.stdout:(27/159): rbd-nbd-19.0.0-2765.g2e1f7622.el9.x86 1.3 MB/s | 171 kB 00:00 2024-05-22T22:49:27.267 INFO:teuthology.orchestra.run.smithi084.stdout:(108/159): gperftools-libs-2.9.1-2.el9.x86_64.r 1.8 MB/s | 309 kB 00:00 2024-05-22T22:49:27.293 INFO:teuthology.orchestra.run.smithi084.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 444 kB/s | 26 kB 00:00 2024-05-22T22:49:27.318 INFO:teuthology.orchestra.run.smithi084.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 970 kB/s | 49 kB 00:00 2024-05-22T22:49:27.335 INFO:teuthology.orchestra.run.smithi084.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-05-22T22:49:27.348 INFO:teuthology.orchestra.run.smithi096.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2765.g 292 kB/s | 32 kB 00:00 2024-05-22T22:49:27.360 INFO:teuthology.orchestra.run.smithi084.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 3.5 MB/s | 151 kB 00:00 2024-05-22T22:49:27.402 INFO:teuthology.orchestra.run.smithi084.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-05-22T22:49:27.436 INFO:teuthology.orchestra.run.smithi084.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-05-22T22:49:27.449 INFO:teuthology.orchestra.run.smithi096.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2765.g2e1f762 1.4 MB/s | 142 kB 00:00 2024-05-22T22:49:27.461 INFO:teuthology.orchestra.run.smithi084.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 739 kB/s | 43 kB 00:00 2024-05-22T22:49:27.487 INFO:teuthology.orchestra.run.smithi084.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-05-22T22:49:27.512 INFO:teuthology.orchestra.run.smithi084.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-05-22T22:49:27.532 INFO:teuthology.orchestra.run.smithi096.stdout:(30/159): rbd-mirror-19.0.0-2765.g2e1f7622.el9. 6.8 MB/s | 3.1 MB 00:00 2024-05-22T22:49:27.537 INFO:teuthology.orchestra.run.smithi084.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-05-22T22:49:27.558 INFO:teuthology.orchestra.run.smithi204.stdout:(60/159): librdkafka-1.6.1-102.el9.x86_64.rpm 647 kB/s | 662 kB 00:01 2024-05-22T22:49:27.571 INFO:teuthology.orchestra.run.smithi084.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-05-22T22:49:27.596 INFO:teuthology.orchestra.run.smithi084.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-05-22T22:49:27.622 INFO:teuthology.orchestra.run.smithi084.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-05-22T22:49:27.675 INFO:teuthology.orchestra.run.smithi204.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 42 kB/s | 25 kB 00:00 2024-05-22T22:49:27.697 INFO:teuthology.orchestra.run.smithi084.stdout:(122/159): libarrow-9.0.0-11.el9.x86_64.rpm 9.0 MB/s | 4.4 MB 00:00 2024-05-22T22:49:27.714 INFO:teuthology.orchestra.run.smithi084.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-05-22T22:49:27.739 INFO:teuthology.orchestra.run.smithi084.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 257 kB/s | 11 kB 00:00 2024-05-22T22:49:27.764 INFO:teuthology.orchestra.run.smithi084.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-05-22T22:49:27.790 INFO:teuthology.orchestra.run.smithi084.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-05-22T22:49:27.815 INFO:teuthology.orchestra.run.smithi084.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-05-22T22:49:27.840 INFO:teuthology.orchestra.run.smithi084.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-05-22T22:49:27.890 INFO:teuthology.orchestra.run.smithi084.stdout:(129/159): python3-grpcio-1.46.7-10.el9.x86_64. 6.9 MB/s | 2.0 MB 00:00 2024-05-22T22:49:27.916 INFO:teuthology.orchestra.run.smithi084.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 617 kB/s | 46 kB 00:00 2024-05-22T22:49:27.941 INFO:teuthology.orchestra.run.smithi084.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-05-22T22:49:27.966 INFO:teuthology.orchestra.run.smithi084.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-05-22T22:49:28.000 INFO:teuthology.orchestra.run.smithi084.stdout:(133/159): python3-kubernetes-26.1.0-2.el9.noar 5.5 MB/s | 1.0 MB 00:00 2024-05-22T22:49:28.017 INFO:teuthology.orchestra.run.smithi204.stdout:(62/159): libtool-2.4.6-45.el9.x86_64.rpm 563 kB/s | 582 kB 00:01 2024-05-22T22:49:28.025 INFO:teuthology.orchestra.run.smithi084.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 281 kB/s | 16 kB 00:00 2024-05-22T22:49:28.050 INFO:teuthology.orchestra.run.smithi084.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-05-22T22:49:28.076 INFO:teuthology.orchestra.run.smithi084.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-05-22T22:49:28.101 INFO:teuthology.orchestra.run.smithi084.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-05-22T22:49:28.116 INFO:teuthology.orchestra.run.smithi096.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2765.g2e1f7 5.3 MB/s | 3.5 MB 00:00 2024-05-22T22:49:28.126 INFO:teuthology.orchestra.run.smithi084.stdout:(138/159): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-05-22T22:49:28.152 INFO:teuthology.orchestra.run.smithi084.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-05-22T22:49:28.177 INFO:teuthology.orchestra.run.smithi084.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-05-22T22:49:28.184 INFO:teuthology.orchestra.run.smithi204.stdout:(63/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 76 kB/s | 47 kB 00:00 2024-05-22T22:49:28.202 INFO:teuthology.orchestra.run.smithi084.stdout:(141/159): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-05-22T22:49:28.226 INFO:teuthology.orchestra.run.smithi096.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2765.g2e 2.2 MB/s | 249 kB 00:00 2024-05-22T22:49:28.236 INFO:teuthology.orchestra.run.smithi084.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 607 kB/s | 51 kB 00:00 2024-05-22T22:49:28.253 INFO:teuthology.orchestra.run.smithi084.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-05-22T22:49:28.270 INFO:teuthology.orchestra.run.smithi084.stdout:(144/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-05-22T22:49:28.295 INFO:teuthology.orchestra.run.smithi084.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 477 kB/s | 20 kB 00:00 2024-05-22T22:49:28.316 INFO:teuthology.orchestra.run.smithi096.stdout:(33/159): ceph-mgr-rook-19.0.0-2765.g2e1f7622.e 538 kB/s | 49 kB 00:00 2024-05-22T22:49:28.320 INFO:teuthology.orchestra.run.smithi084.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-05-22T22:49:28.346 INFO:teuthology.orchestra.run.smithi084.stdout:(147/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-05-22T22:49:28.388 INFO:teuthology.orchestra.run.smithi084.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-05-22T22:49:28.409 INFO:teuthology.orchestra.run.smithi096.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2765.g2 179 kB/s | 16 kB 00:00 2024-05-22T22:49:28.526 INFO:teuthology.orchestra.run.smithi096.stdout:(35/159): ceph-volume-19.0.0-2765.g2e1f7622.el9 2.2 MB/s | 264 kB 00:00 2024-05-22T22:49:28.534 INFO:teuthology.orchestra.run.smithi204.stdout:(64/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 341 kB/s | 292 kB 00:00 2024-05-22T22:49:28.668 INFO:teuthology.orchestra.run.smithi096.stdout:(36/159): ceph-mgr-diskprediction-local-19.0.0- 6.5 MB/s | 7.4 MB 00:01 2024-05-22T22:49:28.718 INFO:teuthology.orchestra.run.smithi096.stdout:(37/159): cephadm-19.0.0-2765.g2e1f7622.el9.noa 3.9 MB/s | 763 kB 00:00 2024-05-22T22:49:28.768 INFO:teuthology.orchestra.run.smithi096.stdout:(38/159): cephfs-top-19.0.0-2765.g2e1f7622.el9. 182 kB/s | 18 kB 00:00 2024-05-22T22:49:28.809 INFO:teuthology.orchestra.run.smithi204.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 56 kB/s | 35 kB 00:00 2024-05-22T22:49:28.868 INFO:teuthology.orchestra.run.smithi204.stdout:(66/159): lua-5.4.4-4.el9.x86_64.rpm 222 kB/s | 188 kB 00:00 2024-05-22T22:49:28.885 INFO:teuthology.orchestra.run.smithi096.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 481 kB/s | 80 kB 00:00 2024-05-22T22:49:28.985 INFO:teuthology.orchestra.run.smithi096.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-05-22T22:49:29.077 INFO:teuthology.orchestra.run.smithi096.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-05-22T22:49:29.111 INFO:teuthology.orchestra.run.smithi096.stdout:(42/159): libgfortran-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 803 kB 00:00 2024-05-22T22:49:29.211 INFO:teuthology.orchestra.run.smithi096.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-05-22T22:49:29.270 INFO:teuthology.orchestra.run.smithi096.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-22T22:49:29.271 INFO:teuthology.orchestra.run.smithi084.stdout:(149/159): librbd1-19.0.0-2765.g2e1f7622.el9.x8 3.4 MB/s | 3.1 MB 00:00 2024-05-22T22:49:29.320 INFO:teuthology.orchestra.run.smithi096.stdout:(45/159): python3-cryptography-36.0.1-4.el9.x86 5.1 MB/s | 1.2 MB 00:00 2024-05-22T22:49:29.345 INFO:teuthology.orchestra.run.smithi096.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-05-22T22:49:29.379 INFO:teuthology.orchestra.run.smithi096.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-05-22T22:49:29.413 INFO:teuthology.orchestra.run.smithi084.stdout:(150/159): librados2-19.0.0-2765.g2e1f7622.el9. 3.1 MB/s | 3.4 MB 00:01 2024-05-22T22:49:30.038 INFO:teuthology.orchestra.run.smithi084.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 66 kB/s | 41 kB 00:00 2024-05-22T22:49:30.155 INFO:teuthology.orchestra.run.smithi096.stdout:(48/159): ceph-test-19.0.0-2765.g2e1f7622.el9.x 9.8 MB/s | 48 MB 00:04 2024-05-22T22:49:30.197 INFO:teuthology.orchestra.run.smithi084.stdout:(152/159): NetworkManager-1.47.90-1.el9.x86_64. 1.3 MB/s | 2.3 MB 00:01 2024-05-22T22:49:30.243 INFO:teuthology.orchestra.run.smithi204.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 1.2 MB/s | 1.6 MB 00:01 2024-05-22T22:49:30.318 INFO:teuthology.orchestra.run.smithi204.stdout:(68/159): openblas-openmp-0.3.21-2.el9.x86_64.r 2.7 MB/s | 4.7 MB 00:01 2024-05-22T22:49:30.355 INFO:teuthology.orchestra.run.smithi096.stdout:(49/159): automake-1.16.2-8.el9.noarch.rpm 683 kB/s | 689 kB 00:01 2024-05-22T22:49:30.380 INFO:teuthology.orchestra.run.smithi084.stdout:(153/159): NetworkManager-libnm-1.47.90-1.el9.x 1.6 MB/s | 1.8 MB 00:01 2024-05-22T22:49:30.402 INFO:teuthology.orchestra.run.smithi204.stdout:(69/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 2.8 MB/s | 4.4 MB 00:01 2024-05-22T22:49:30.455 INFO:teuthology.orchestra.run.smithi096.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 217 kB/s | 22 kB 00:00 2024-05-22T22:49:30.497 INFO:teuthology.orchestra.run.smithi096.stdout:(51/159): bison-3.7.4-5.el9.x86_64.rpm 844 kB/s | 943 kB 00:01 2024-05-22T22:49:30.564 INFO:teuthology.orchestra.run.smithi096.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-05-22T22:49:30.589 INFO:teuthology.orchestra.run.smithi096.stdout:(53/159): boost-program-options-1.75.0-8.el9.x8 246 kB/s | 107 kB 00:00 2024-05-22T22:49:30.614 INFO:teuthology.orchestra.run.smithi096.stdout:(54/159): flexiblas-3.0.4-8.el9.x86_64.rpm 279 kB/s | 33 kB 00:00 2024-05-22T22:49:30.689 INFO:teuthology.orchestra.run.smithi096.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 182 kB/s | 18 kB 00:00 2024-05-22T22:49:30.715 INFO:teuthology.orchestra.run.smithi096.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-05-22T22:49:30.815 INFO:teuthology.orchestra.run.smithi096.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 452 kB/s | 45 kB 00:00 2024-05-22T22:49:30.874 INFO:teuthology.orchestra.run.smithi096.stdout:(58/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 871 kB/s | 160 kB 00:00 2024-05-22T22:49:30.881 INFO:teuthology.orchestra.run.smithi084.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 54 kB/s | 37 kB 00:00 2024-05-22T22:49:30.949 INFO:teuthology.orchestra.run.smithi096.stdout:(59/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.8 MB/s | 3.0 MB 00:00 2024-05-22T22:49:30.989 INFO:teuthology.orchestra.run.smithi084.stdout:(155/159): NetworkManager-tui-1.47.90-1.el9.x86 261 kB/s | 248 kB 00:00 2024-05-22T22:49:31.049 INFO:teuthology.orchestra.run.smithi096.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 255 kB/s | 25 kB 00:00 2024-05-22T22:49:31.083 INFO:teuthology.orchestra.run.smithi096.stdout:(61/159): librdkafka-1.6.1-102.el9.x86_64.rpm 2.4 MB/s | 662 kB 00:00 2024-05-22T22:49:31.144 INFO:teuthology.orchestra.run.smithi204.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 161 kB/s | 133 kB 00:00 2024-05-22T22:49:31.150 INFO:teuthology.orchestra.run.smithi096.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-05-22T22:49:31.184 INFO:teuthology.orchestra.run.smithi096.stdout:(63/159): libtool-2.4.6-45.el9.x86_64.rpm 1.8 MB/s | 582 kB 00:00 2024-05-22T22:49:31.209 INFO:teuthology.orchestra.run.smithi096.stdout:(64/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-05-22T22:49:31.259 INFO:teuthology.orchestra.run.smithi096.stdout:(65/159): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-05-22T22:49:31.284 INFO:teuthology.orchestra.run.smithi096.stdout:(66/159): openblas-0.3.21-2.el9.x86_64.rpm 350 kB/s | 35 kB 00:00 2024-05-22T22:49:31.289 INFO:teuthology.orchestra.run.smithi084.stdout:(156/159): elfutils-libelf-0.191-4.el9.x86_64.r 228 kB/s | 207 kB 00:00 2024-05-22T22:49:31.377 INFO:teuthology.orchestra.run.smithi204.stdout:(71/159): postgresql-contrib-13.14-1.el9.x86_64 804 kB/s | 911 kB 00:01 2024-05-22T22:49:31.710 INFO:teuthology.orchestra.run.smithi096.stdout:(67/159): openblas-openmp-0.3.21-2.el9.x86_64.r 9.5 MB/s | 4.7 MB 00:00 2024-05-22T22:49:31.711 INFO:teuthology.orchestra.run.smithi204.stdout:(72/159): postgresql-server-13.14-1.el9.x86_64. 4.4 MB/s | 5.8 MB 00:01 2024-05-22T22:49:31.887 INFO:teuthology.orchestra.run.smithi096.stdout:(68/159): postgresql-contrib-13.14-1.el9.x86_64 5.0 MB/s | 911 kB 00:00 2024-05-22T22:49:31.927 INFO:teuthology.orchestra.run.smithi096.stdout:(69/159): postgresql-13.14-1.el9.x86_64.rpm 2.5 MB/s | 1.6 MB 00:00 2024-05-22T22:49:31.940 INFO:teuthology.orchestra.run.smithi084.stdout:(157/159): elfutils-libs-0.191-4.el9.x86_64.rpm 245 kB/s | 260 kB 00:01 2024-05-22T22:49:31.994 INFO:teuthology.orchestra.run.smithi096.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-05-22T22:49:32.112 INFO:teuthology.orchestra.run.smithi204.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 119 kB/s | 87 kB 00:00 2024-05-22T22:49:32.728 INFO:teuthology.orchestra.run.smithi204.stdout:(74/159): protobuf-3.14.0-13.el9.x86_64.rpm 652 kB/s | 1.0 MB 00:01 2024-05-22T22:49:32.762 INFO:teuthology.orchestra.run.smithi204.stdout:(75/159): python3-colorama-0.4.6-3.el9.noarch.r 94 kB/s | 61 kB 00:00 2024-05-22T22:49:32.827 INFO:teuthology.orchestra.run.smithi096.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 1.2 MB/s | 1.0 MB 00:00 2024-05-22T22:49:32.856 INFO:teuthology.orchestra.run.smithi084.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 1.5 MB/s | 2.4 MB 00:01 2024-05-22T22:49:32.961 INFO:teuthology.orchestra.run.smithi096.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 657 kB/s | 87 kB 00:00 2024-05-22T22:49:33.112 INFO:teuthology.orchestra.run.smithi204.stdout:(76/159): python3-babel-2.9.1-2.el9.noarch.rpm 4.3 MB/s | 6.0 MB 00:01 2024-05-22T22:49:33.206 INFO:teuthology.orchestra.run.smithi084.stdout:(159/159): openssl-3.2.1-1.el9.x86_64.rpm 623 kB/s | 1.3 MB 00:02 2024-05-22T22:49:33.210 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-05-22T22:49:33.210 INFO:teuthology.orchestra.run.smithi084.stdout:Total 9.5 MB/s | 225 MB 00:23 2024-05-22T22:49:33.596 INFO:teuthology.orchestra.run.smithi204.stdout:(77/159): python3-devel-3.9.18-3.el9.x86_64.rpm 282 kB/s | 244 kB 00:00 2024-05-22T22:49:33.629 INFO:teuthology.orchestra.run.smithi204.stdout:(78/159): python3-jinja2-2.11.3-6.el9.noarch.rp 286 kB/s | 248 kB 00:00 2024-05-22T22:49:33.705 INFO:teuthology.orchestra.run.smithi204.stdout:(79/159): python3-jsonpatch-1.21-16.el9.noarch. 43 kB/s | 26 kB 00:00 2024-05-22T22:49:34.122 INFO:teuthology.orchestra.run.smithi204.stdout:(80/159): python3-jsonpointer-2.0-4.el9.noarch. 36 kB/s | 19 kB 00:00 2024-05-22T22:49:34.297 INFO:teuthology.orchestra.run.smithi204.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 59 kB/s | 35 kB 00:00 2024-05-22T22:49:34.369 INFO:teuthology.orchestra.run.smithi096.stdout:(73/159): postgresql-server-13.14-1.el9.x86_64. 2.4 MB/s | 5.8 MB 00:02 2024-05-22T22:49:34.422 INFO:teuthology.orchestra.run.smithi204.stdout:(82/159): python3-mako-1.1.4-6.el9.noarch.rpm 217 kB/s | 172 kB 00:00 2024-05-22T22:49:34.446 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-05-22T22:49:34.486 INFO:teuthology.orchestra.run.smithi096.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 525 kB/s | 61 kB 00:00 2024-05-22T22:49:34.602 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-05-22T22:49:34.602 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-05-22T22:49:34.603 INFO:teuthology.orchestra.run.smithi096.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.0 MB/s | 244 kB 00:00 2024-05-22T22:49:34.712 INFO:teuthology.orchestra.run.smithi096.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-05-22T22:49:34.846 INFO:teuthology.orchestra.run.smithi096.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 192 kB/s | 26 kB 00:00 2024-05-22T22:49:34.954 INFO:teuthology.orchestra.run.smithi096.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 177 kB/s | 19 kB 00:00 2024-05-22T22:49:35.063 INFO:teuthology.orchestra.run.smithi096.stdout:(79/159): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-05-22T22:49:35.163 INFO:teuthology.orchestra.run.smithi096.stdout:(80/159): python3-markupsafe-1.1.1-12.el9.x86_6 347 kB/s | 35 kB 00:00 2024-05-22T22:49:35.231 INFO:teuthology.orchestra.run.smithi204.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 268 kB/s | 250 kB 00:00 2024-05-22T22:49:35.290 INFO:teuthology.orchestra.run.smithi204.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 256 kB/s | 222 kB 00:00 2024-05-22T22:49:35.305 INFO:teuthology.orchestra.run.smithi096.stdout:(81/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.1 MB/s | 4.4 MB 00:04 2024-05-22T22:49:35.647 INFO:teuthology.orchestra.run.smithi096.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 734 kB/s | 250 kB 00:00 2024-05-22T22:49:35.907 INFO:teuthology.orchestra.run.smithi204.stdout:(85/159): python3-prettytable-0.7.2-27.el9.noar 68 kB/s | 42 kB 00:00 2024-05-22T22:49:35.923 INFO:teuthology.orchestra.run.smithi204.stdout:(86/159): python3-packaging-20.9-5.el9.noarch.r 112 kB/s | 77 kB 00:00 2024-05-22T22:49:35.989 INFO:teuthology.orchestra.run.smithi096.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 650 kB/s | 222 kB 00:00 2024-05-22T22:49:36.181 INFO:teuthology.orchestra.run.smithi096.stdout:(84/159): python3-packaging-20.9-5.el9.noarch.r 402 kB/s | 77 kB 00:00 2024-05-22T22:49:36.266 INFO:teuthology.orchestra.run.smithi204.stdout:(87/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.4 MB/s | 5.1 MB 00:02 2024-05-22T22:49:36.281 INFO:teuthology.orchestra.run.smithi096.stdout:(85/159): python3-prettytable-0.7.2-27.el9.noar 423 kB/s | 42 kB 00:00 2024-05-22T22:49:36.698 INFO:teuthology.orchestra.run.smithi096.stdout:(86/159): python3-protobuf-3.14.0-13.el9.noarch 646 kB/s | 269 kB 00:00 2024-05-22T22:49:36.724 INFO:teuthology.orchestra.run.smithi204.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 198 kB/s | 159 kB 00:00 2024-05-22T22:49:36.758 INFO:teuthology.orchestra.run.smithi204.stdout:(89/159): python3-protobuf-3.14.0-13.el9.noarch 317 kB/s | 269 kB 00:00 2024-05-22T22:49:36.772 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-05-22T22:49:36.773 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-05-22T22:49:36.998 INFO:teuthology.orchestra.run.smithi096.stdout:(87/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 529 kB/s | 159 kB 00:00 2024-05-22T22:49:37.116 INFO:teuthology.orchestra.run.smithi204.stdout:(90/159): python3-pyasn1-modules-0.4.8-6.el9.no 329 kB/s | 279 kB 00:00 2024-05-22T22:49:37.400 INFO:teuthology.orchestra.run.smithi204.stdout:(91/159): python3-pytz-2021.1-5.el9.noarch.rpm 75 kB/s | 51 kB 00:00 2024-05-22T22:49:37.415 INFO:teuthology.orchestra.run.smithi096.stdout:(88/159): python3-pyasn1-modules-0.4.8-6.el9.no 670 kB/s | 279 kB 00:00 2024-05-22T22:49:37.425 INFO:teuthology.orchestra.run.smithi204.stdout:(92/159): python3-requests-oauthlib-1.3.0-12.el 80 kB/s | 54 kB 00:00 2024-05-22T22:49:37.515 INFO:teuthology.orchestra.run.smithi096.stdout:(89/159): python3-pytz-2021.1-5.el9.noarch.rpm 509 kB/s | 51 kB 00:00 2024-05-22T22:49:37.616 INFO:teuthology.orchestra.run.smithi096.stdout:(90/159): python3-requests-oauthlib-1.3.0-12.el 537 kB/s | 54 kB 00:00 2024-05-22T22:49:38.017 INFO:teuthology.orchestra.run.smithi204.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 68 kB/s | 42 kB 00:00 2024-05-22T22:49:38.051 INFO:teuthology.orchestra.run.smithi204.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 71 kB/s | 44 kB 00:00 2024-05-22T22:49:38.591 INFO:teuthology.orchestra.run.smithi096.stdout:(91/159): python3-babel-2.9.1-2.el9.noarch.rpm 1.1 MB/s | 6.0 MB 00:05 2024-05-22T22:49:38.634 INFO:teuthology.orchestra.run.smithi204.stdout:(95/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-05-22T22:49:38.660 INFO:teuthology.orchestra.run.smithi204.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 59 kB/s | 36 kB 00:00 2024-05-22T22:49:38.691 INFO:teuthology.orchestra.run.smithi096.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 417 kB/s | 42 kB 00:00 2024-05-22T22:49:38.792 INFO:teuthology.orchestra.run.smithi096.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 442 kB/s | 44 kB 00:00 2024-05-22T22:49:38.944 INFO:teuthology.orchestra.run.smithi204.stdout:(97/159): qatlib-24.02.0-1.el9.x86_64.rpm 238 kB/s | 221 kB 00:00 2024-05-22T22:49:39.142 INFO:teuthology.orchestra.run.smithi096.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 631 kB/s | 221 kB 00:00 2024-05-22T22:49:39.242 INFO:teuthology.orchestra.run.smithi096.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 359 kB/s | 36 kB 00:00 2024-05-22T22:49:39.277 INFO:teuthology.orchestra.run.smithi204.stdout:(98/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 74 kB/s | 47 kB 00:00 2024-05-22T22:49:39.343 INFO:teuthology.orchestra.run.smithi096.stdout:(96/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 474 kB/s | 47 kB 00:00 2024-05-22T22:49:39.486 INFO:teuthology.orchestra.run.smithi204.stdout:(99/159): socat-1.7.4.1-5.el9.x86_64.rpm 369 kB/s | 305 kB 00:00 2024-05-22T22:49:39.603 INFO:teuthology.orchestra.run.smithi204.stdout:(100/159): uuid-1.6.2-55.el9.x86_64.rpm 88 kB/s | 58 kB 00:00 2024-05-22T22:49:39.759 INFO:teuthology.orchestra.run.smithi096.stdout:(97/159): socat-1.7.4.1-5.el9.x86_64.rpm 731 kB/s | 305 kB 00:00 2024-05-22T22:49:39.859 INFO:teuthology.orchestra.run.smithi096.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 579 kB/s | 58 kB 00:00 2024-05-22T22:49:39.926 INFO:teuthology.orchestra.run.smithi096.stdout:(99/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.1 MB/s | 5.1 MB 00:04 2024-05-22T22:49:39.995 INFO:teuthology.orchestra.run.smithi204.stdout:(101/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 131 kB/s | 94 kB 00:00 2024-05-22T22:49:40.044 INFO:teuthology.orchestra.run.smithi096.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 510 kB/s | 94 kB 00:00 2024-05-22T22:49:40.045 INFO:teuthology.orchestra.run.smithi204.stdout:(102/159): lua-devel-5.4.4-4.el9.x86_64.rpm 450 kB/s | 22 kB 00:00 2024-05-22T22:49:40.110 INFO:teuthology.orchestra.run.smithi096.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 347 kB/s | 64 kB 00:00 2024-05-22T22:49:40.146 INFO:teuthology.orchestra.run.smithi204.stdout:(103/159): protobuf-compiler-3.14.0-13.el9.x86_ 8.4 MB/s | 863 kB 00:00 2024-05-22T22:49:40.146 INFO:teuthology.orchestra.run.smithi096.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 447 kB/s | 45 kB 00:00 2024-05-22T22:49:40.171 INFO:teuthology.orchestra.run.smithi204.stdout:(104/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 93 kB/s | 64 kB 00:00 2024-05-22T22:49:40.211 INFO:teuthology.orchestra.run.smithi096.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 223 kB/s | 22 kB 00:00 2024-05-22T22:49:40.221 INFO:teuthology.orchestra.run.smithi204.stdout:(105/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 73 kB/s | 45 kB 00:00 2024-05-22T22:49:40.370 INFO:teuthology.orchestra.run.smithi096.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 3.7 MB/s | 863 kB 00:00 2024-05-22T22:49:40.596 INFO:teuthology.orchestra.run.smithi204.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 52 kB/s | 19 kB 00:00 2024-05-22T22:49:40.779 INFO:teuthology.orchestra.run.smithi204.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 507 kB/s | 309 kB 00:00 2024-05-22T22:49:40.945 INFO:teuthology.orchestra.run.smithi096.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 747 kB/s | 548 kB 00:00 2024-05-22T22:49:40.963 INFO:teuthology.orchestra.run.smithi204.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 142 kB/s | 26 kB 00:00 2024-05-22T22:49:41.028 INFO:teuthology.orchestra.run.smithi096.stdout:(106/159): gperftools-libs-2.9.1-2.el9.x86_64.r 469 kB/s | 309 kB 00:00 2024-05-22T22:49:41.047 INFO:teuthology.orchestra.run.smithi204.stdout:(109/159): abseil-cpp-20211102.0-3.el9.x86_64.r 609 kB/s | 548 kB 00:00 2024-05-22T22:49:41.053 INFO:teuthology.orchestra.run.smithi096.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 179 kB/s | 19 kB 00:00 2024-05-22T22:49:41.139 INFO:teuthology.orchestra.run.smithi204.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 277 kB/s | 49 kB 00:00 2024-05-22T22:49:41.170 INFO:teuthology.orchestra.run.smithi096.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 223 kB/s | 26 kB 00:00 2024-05-22T22:49:41.206 INFO:teuthology.orchestra.run.smithi204.stdout:(111/159): libarrow-9.0.0-11.el9.x86_64.rpm 7.2 MB/s | 4.4 MB 00:00 2024-05-22T22:49:41.231 INFO:teuthology.orchestra.run.smithi204.stdout:(112/159): libunwind-1.6.2-1.el9.x86_64.rpm 367 kB/s | 67 kB 00:00 2024-05-22T22:49:41.256 INFO:teuthology.orchestra.run.smithi204.stdout:(113/159): luarocks-3.9.2-5.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-05-22T22:49:41.287 INFO:teuthology.orchestra.run.smithi096.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 416 kB/s | 49 kB 00:00 2024-05-22T22:49:41.357 INFO:teuthology.orchestra.run.smithi204.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.6 MB/s | 839 kB 00:00 2024-05-22T22:49:41.390 INFO:teuthology.orchestra.run.smithi204.stdout:(115/159): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-05-22T22:49:41.405 INFO:teuthology.orchestra.run.smithi096.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 578 kB/s | 67 kB 00:00 2024-05-22T22:49:41.407 INFO:teuthology.orchestra.run.smithi204.stdout:(116/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 289 kB/s | 43 kB 00:00 2024-05-22T22:49:41.458 INFO:teuthology.orchestra.run.smithi204.stdout:(117/159): python3-cachetools-4.2.4-1.el9.noarc 324 kB/s | 32 kB 00:00 2024-05-22T22:49:41.499 INFO:teuthology.orchestra.run.smithi204.stdout:(118/159): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-05-22T22:49:41.525 INFO:teuthology.orchestra.run.smithi204.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-05-22T22:49:41.538 INFO:teuthology.orchestra.run.smithi096.stdout:(111/159): libarrow-9.0.0-11.el9.x86_64.rpm 8.7 MB/s | 4.4 MB 00:00 2024-05-22T22:49:41.566 INFO:teuthology.orchestra.run.smithi204.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-05-22T22:49:41.588 INFO:teuthology.orchestra.run.smithi096.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 825 kB/s | 151 kB 00:00 2024-05-22T22:49:41.609 INFO:teuthology.orchestra.run.smithi204.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-05-22T22:49:41.634 INFO:teuthology.orchestra.run.smithi204.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 1.8 MB/s | 206 kB 00:00 2024-05-22T22:49:41.672 INFO:teuthology.orchestra.run.smithi096.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-05-22T22:49:41.734 INFO:teuthology.orchestra.run.smithi204.stdout:(123/159): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-05-22T22:49:41.756 INFO:teuthology.orchestra.run.smithi096.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-05-22T22:49:41.759 INFO:teuthology.orchestra.run.smithi204.stdout:(124/159): python3-grpcio-tools-1.46.7-10.el9.x 963 kB/s | 144 kB 00:00 2024-05-22T22:49:41.784 INFO:teuthology.orchestra.run.smithi204.stdout:(125/159): python3-jaraco-8.2.1-3.el9.noarch.rp 71 kB/s | 11 kB 00:00 2024-05-22T22:49:41.789 INFO:teuthology.orchestra.run.smithi096.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 372 kB/s | 43 kB 00:00 2024-05-22T22:49:41.835 INFO:teuthology.orchestra.run.smithi204.stdout:(126/159): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-05-22T22:49:41.864 INFO:teuthology.orchestra.run.smithi096.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 297 kB/s | 32 kB 00:00 2024-05-22T22:49:41.885 INFO:teuthology.orchestra.run.smithi204.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-05-22T22:49:41.906 INFO:teuthology.orchestra.run.smithi096.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-05-22T22:49:41.910 INFO:teuthology.orchestra.run.smithi204.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 155 kB/s | 19 kB 00:00 2024-05-22T22:49:41.936 INFO:teuthology.orchestra.run.smithi204.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 195 kB/s | 20 kB 00:00 2024-05-22T22:49:41.982 INFO:teuthology.orchestra.run.smithi096.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-05-22T22:49:42.015 INFO:teuthology.orchestra.run.smithi096.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-05-22T22:49:42.019 INFO:teuthology.orchestra.run.smithi204.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-05-22T22:49:42.045 INFO:teuthology.orchestra.run.smithi204.stdout:(131/159): python3-logutils-0.3.5-21.el9.noarch 346 kB/s | 46 kB 00:00 2024-05-22T22:49:42.070 INFO:teuthology.orchestra.run.smithi204.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 588 kB/s | 79 kB 00:00 2024-05-22T22:49:42.120 INFO:teuthology.orchestra.run.smithi204.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 574 kB/s | 58 kB 00:00 2024-05-22T22:49:42.154 INFO:teuthology.orchestra.run.smithi204.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-05-22T22:49:42.204 INFO:teuthology.orchestra.run.smithi204.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 123 kB/s | 16 kB 00:00 2024-05-22T22:49:42.229 INFO:teuthology.orchestra.run.smithi204.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 826 kB/s | 90 kB 00:00 2024-05-22T22:49:42.255 INFO:teuthology.orchestra.run.smithi204.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 280 kB/s | 28 kB 00:00 2024-05-22T22:49:42.266 INFO:teuthology.orchestra.run.smithi096.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.0 MB/s | 303 kB 00:00 2024-05-22T22:49:42.291 INFO:teuthology.orchestra.run.smithi096.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 746 kB/s | 206 kB 00:00 2024-05-22T22:49:42.305 INFO:teuthology.orchestra.run.smithi204.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 307 kB/s | 31 kB 00:00 2024-05-22T22:49:42.339 INFO:teuthology.orchestra.run.smithi204.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-05-22T22:49:42.364 INFO:teuthology.orchestra.run.smithi204.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 542 kB/s | 59 kB 00:00 2024-05-22T22:49:42.406 INFO:teuthology.orchestra.run.smithi204.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 354 kB/s | 36 kB 00:00 2024-05-22T22:49:42.408 INFO:teuthology.orchestra.run.smithi096.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-05-22T22:49:42.440 INFO:teuthology.orchestra.run.smithi204.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 506 kB/s | 51 kB 00:00 2024-05-22T22:49:42.473 INFO:teuthology.orchestra.run.smithi204.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-05-22T22:49:42.508 INFO:teuthology.orchestra.run.smithi096.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-05-22T22:49:42.515 INFO:teuthology.orchestra.run.smithi204.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 822 kB/s | 90 kB 00:00 2024-05-22T22:49:42.566 INFO:teuthology.orchestra.run.smithi204.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-05-22T22:49:42.567 INFO:teuthology.orchestra.run.smithi096.stdout:(124/159): python3-grpcio-1.46.7-10.el9.x86_64. 6.8 MB/s | 2.0 MB 00:00 2024-05-22T22:49:42.591 INFO:teuthology.orchestra.run.smithi204.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 170 kB/s | 20 kB 00:00 2024-05-22T22:49:42.609 INFO:teuthology.orchestra.run.smithi096.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-05-22T22:49:42.624 INFO:teuthology.orchestra.run.smithi204.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-05-22T22:49:42.676 INFO:teuthology.orchestra.run.smithi096.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 213 kB/s | 23 kB 00:00 2024-05-22T22:49:42.718 INFO:teuthology.orchestra.run.smithi096.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 179 kB/s | 19 kB 00:00 2024-05-22T22:49:42.785 INFO:teuthology.orchestra.run.smithi096.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-05-22T22:49:42.852 INFO:teuthology.orchestra.run.smithi096.stdout:(129/159): python3-kubernetes-26.1.0-2.el9.noar 7.7 MB/s | 1.0 MB 00:00 2024-05-22T22:49:42.902 INFO:teuthology.orchestra.run.smithi096.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 396 kB/s | 46 kB 00:00 2024-05-22T22:49:42.961 INFO:teuthology.orchestra.run.smithi096.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 725 kB/s | 79 kB 00:00 2024-05-22T22:49:43.011 INFO:teuthology.orchestra.run.smithi096.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 531 kB/s | 58 kB 00:00 2024-05-22T22:49:43.078 INFO:teuthology.orchestra.run.smithi096.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-05-22T22:49:43.125 INFO:teuthology.orchestra.run.smithi204.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 2.8 MB/s | 1.6 MB 00:00 2024-05-22T22:49:43.136 INFO:teuthology.orchestra.run.smithi096.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 131 kB/s | 16 kB 00:00 2024-05-22T22:49:43.187 INFO:teuthology.orchestra.run.smithi096.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 828 kB/s | 90 kB 00:00 2024-05-22T22:49:43.222 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-05-22T22:49:43.279 INFO:teuthology.orchestra.run.smithi096.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 199 kB/s | 28 kB 00:00 2024-05-22T22:49:43.302 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-22T22:49:43.304 INFO:teuthology.orchestra.run.smithi096.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 263 kB/s | 31 kB 00:00 2024-05-22T22:49:43.342 INFO:teuthology.orchestra.run.smithi204.stdout:(149/159): librados2-19.0.0-2765.g2e1f7622.el9. 4.5 MB/s | 3.4 MB 00:00 2024-05-22T22:49:43.347 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-22T22:49:43.404 INFO:teuthology.orchestra.run.smithi096.stdout:(138/159): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-05-22T22:49:43.415 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-22T22:49:43.429 INFO:teuthology.orchestra.run.smithi096.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 472 kB/s | 59 kB 00:00 2024-05-22T22:49:43.453 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-22T22:49:43.478 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-22T22:49:43.500 INFO:teuthology.orchestra.run.smithi204.stdout:(150/159): librbd1-19.0.0-2765.g2e1f7622.el9.x8 3.6 MB/s | 3.1 MB 00:00 2024-05-22T22:49:43.521 INFO:teuthology.orchestra.run.smithi096.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 306 kB/s | 36 kB 00:00 2024-05-22T22:49:43.531 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-22T22:49:43.547 INFO:teuthology.orchestra.run.smithi096.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 436 kB/s | 51 kB 00:00 2024-05-22T22:49:43.639 INFO:teuthology.orchestra.run.smithi096.stdout:(142/159): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-05-22T22:49:43.664 INFO:teuthology.orchestra.run.smithi096.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 767 kB/s | 90 kB 00:00 2024-05-22T22:49:43.773 INFO:teuthology.orchestra.run.smithi096.stdout:(144/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-05-22T22:49:43.798 INFO:teuthology.orchestra.run.smithi096.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 149 kB/s | 20 kB 00:00 2024-05-22T22:49:43.865 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-22T22:49:43.898 INFO:teuthology.orchestra.run.smithi096.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-05-22T22:49:43.957 INFO:teuthology.orchestra.run.smithi096.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-05-22T22:49:43.971 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : librados2-2:19.0.0-2765.g2e1f7622.el9.x86_64 8/170 2024-05-22T22:49:44.093 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados2-2:19.0.0-2765.g2e1f7622.el9.x86_64 8/170 2024-05-22T22:49:44.134 INFO:teuthology.orchestra.run.smithi204.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 65 kB/s | 41 kB 00:00 2024-05-22T22:49:44.256 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephfs2-2:19.0.0-2765.g2e1f7622.el9.x86_64 9/170 2024-05-22T22:49:44.337 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephfs2-2:19.0.0-2765.g2e1f7622.el9.x86_64 9/170 2024-05-22T22:49:44.368 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rados-2:19.0.0-2765.g2e1f7622.el9.x86_64 10/170 2024-05-22T22:49:44.418 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-22T22:49:44.610 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-22T22:49:44.667 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-22T22:49:44.674 INFO:teuthology.orchestra.run.smithi096.stdout:(148/159): librbd1-19.0.0-2765.g2e1f7622.el9.x8 4.4 MB/s | 3.1 MB 00:00 2024-05-22T22:49:44.695 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephsqlite-2:19.0.0-2765.g2e1f7622.el9.x86_64 14/170 2024-05-22T22:49:44.766 INFO:teuthology.orchestra.run.smithi096.stdout:(149/159): librados2-19.0.0-2765.g2e1f7622.el9. 3.9 MB/s | 3.4 MB 00:00 2024-05-22T22:49:44.798 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2765.g2e1f7622.el9.x86_64 14/170 2024-05-22T22:49:44.821 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libradosstriper1-2:19.0.0-2765.g2e1f7622.el9.x86 15/170 2024-05-22T22:49:44.883 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2765.g2e1f7622.el9.x86 15/170 2024-05-22T22:49:44.905 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-22T22:49:44.931 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-22T22:49:44.942 INFO:teuthology.orchestra.run.smithi204.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 307 kB/s | 248 kB 00:00 2024-05-22T22:49:45.001 INFO:teuthology.orchestra.run.smithi204.stdout:(153/159): NetworkManager-libnm-1.47.90-1.el9.x 1.1 MB/s | 1.8 MB 00:01 2024-05-22T22:49:45.107 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-22T22:49:45.158 INFO:teuthology.orchestra.run.smithi096.stdout:(150/159): NetworkManager-1.47.90-1.el9.x86_64. 4.8 MB/s | 2.3 MB 00:00 2024-05-22T22:49:45.216 INFO:teuthology.orchestra.run.smithi096.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 715 kB/s | 41 kB 00:00 2024-05-22T22:49:45.250 INFO:teuthology.orchestra.run.smithi096.stdout:(152/159): NetworkManager-libnm-1.47.90-1.el9.x 3.7 MB/s | 1.8 MB 00:00 2024-05-22T22:49:45.275 INFO:teuthology.orchestra.run.smithi096.stdout:(153/159): NetworkManager-tui-1.47.90-1.el9.x86 4.1 MB/s | 248 kB 00:00 2024-05-22T22:49:45.276 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-22T22:49:45.309 INFO:teuthology.orchestra.run.smithi096.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 626 kB/s | 37 kB 00:00 2024-05-22T22:49:45.334 INFO:teuthology.orchestra.run.smithi096.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 3.5 MB/s | 207 kB 00:00 2024-05-22T22:49:45.393 INFO:teuthology.orchestra.run.smithi096.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 3.0 MB/s | 260 kB 00:00 2024-05-22T22:49:45.410 INFO:teuthology.orchestra.run.smithi204.stdout:(154/159): NetworkManager-1.47.90-1.el9.x86_64. 1.0 MB/s | 2.3 MB 00:02 2024-05-22T22:49:45.452 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-22T22:49:45.501 INFO:teuthology.orchestra.run.smithi096.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 8.1 MB/s | 1.3 MB 00:00 2024-05-22T22:49:45.513 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-22T22:49:45.560 INFO:teuthology.orchestra.run.smithi204.stdout:(155/159): elfutils-debuginfod-client-0.191-4.e 59 kB/s | 37 kB 00:00 2024-05-22T22:49:45.615 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-22T22:49:45.706 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-22T22:49:45.734 INFO:teuthology.orchestra.run.smithi096.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 7.0 MB/s | 2.4 MB 00:00 2024-05-22T22:49:45.794 INFO:teuthology.orchestra.run.smithi204.stdout:(156/159): elfutils-libelf-0.191-4.el9.x86_64.r 261 kB/s | 207 kB 00:00 2024-05-22T22:49:45.909 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-22T22:49:46.054 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-22T22:49:46.147 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-22T22:49:46.186 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-05-22T22:49:46.229 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-05-22T22:49:46.252 INFO:teuthology.orchestra.run.smithi204.stdout:(157/159): elfutils-libs-0.191-4.el9.x86_64.rpm 308 kB/s | 260 kB 00:00 2024-05-22T22:49:46.293 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-05-22T22:49:46.328 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-05-22T22:49:46.371 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-05-22T22:49:46.405 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-05-22T22:49:46.432 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-05-22T22:49:46.500 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-05-22T22:49:46.555 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-05-22T22:49:46.593 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-05-22T22:49:46.628 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-05-22T22:49:46.661 INFO:teuthology.orchestra.run.smithi204.stdout:(158/159): openssl-3.2.1-1.el9.x86_64.rpm 1.2 MB/s | 1.3 MB 00:01 2024-05-22T22:49:46.704 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-05-22T22:49:46.739 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ceph-common-2:19.0.0-2765.g2e1f7622.el9. 34/170 2024-05-22T22:49:46.782 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ceph-argparse-2:19.0.0-2765.g2e1f7622.el 35/170 2024-05-22T22:49:46.811 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cephfs-2:19.0.0-2765.g2e1f7622.el9.x86_6 36/170 2024-05-22T22:49:46.880 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-05-22T22:49:46.926 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-05-22T22:49:47.000 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-05-22T22:49:47.061 INFO:teuthology.orchestra.run.smithi204.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 1.9 MB/s | 2.4 MB 00:01 2024-05-22T22:49:47.068 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-05-22T22:49:47.068 INFO:teuthology.orchestra.run.smithi204.stdout:Total 6.3 MB/s | 225 MB 00:35 2024-05-22T22:49:47.133 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-05-22T22:49:47.187 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-05-22T22:49:47.713 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-05-22T22:49:47.790 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-05-22T22:49:47.819 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-05-22T22:49:47.850 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-05-22T22:49:47.897 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-05-22T22:49:47.937 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librados-devel-2:19.0.0-2765.g2e1f7622.el9.x86_6 47/170 2024-05-22T22:49:47.980 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-05-22T22:49:48.133 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-05-22T22:49:48.198 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-05-22T22:49:48.244 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-05-22T22:49:48.252 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-05-22T22:49:48.317 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-22T22:49:48.348 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-22T22:49:48.385 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-22T22:49:48.405 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-05-22T22:49:48.405 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-05-22T22:49:48.431 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-22T22:49:48.467 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-22T22:49:48.511 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-22T22:49:48.554 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-22T22:49:48.688 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-22T22:49:48.730 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-22T22:49:48.771 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-22T22:49:48.866 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-22T22:49:48.896 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-22T22:49:48.946 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-22T22:49:48.970 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-22T22:49:48.986 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-22T22:49:49.071 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-22T22:49:49.196 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-22T22:49:49.236 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-22T22:49:49.267 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-22T22:49:49.304 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-22T22:49:49.345 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-22T22:49:49.377 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-22T22:49:49.406 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-22T22:49:49.441 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-22T22:49:49.474 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-22T22:49:49.516 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-22T22:49:49.551 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-22T22:49:49.641 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-22T22:49:49.711 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-22T22:49:49.746 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-22T22:49:49.793 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-22T22:49:49.889 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-22T22:49:49.939 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-22T22:49:50.239 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-05-22T22:49:50.240 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-05-22T22:49:50.671 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-22T22:49:50.728 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-22T22:49:50.761 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-22T22:49:50.862 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-22T22:49:50.909 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-22T22:49:50.952 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-22T22:49:51.061 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-22T22:49:51.106 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-22T22:49:51.317 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-22T22:49:51.354 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-22T22:49:51.438 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-22T22:49:52.083 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-22T22:49:52.202 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-22T22:49:53.463 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-22T22:49:53.557 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-22T22:49:53.585 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-22T22:49:53.618 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-22T22:49:53.929 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-22T22:49:53.973 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : librbd1-2:19.0.0-2765.g2e1f7622.el9.x86_64 103/170 2024-05-22T22:49:54.063 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librbd1-2:19.0.0-2765.g2e1f7622.el9.x86_64 103/170 2024-05-22T22:49:54.094 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rbd-2:19.0.0-2765.g2e1f7622.el9.x86_64 104/170 2024-05-22T22:49:54.131 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-22T22:49:54.601 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-22T22:49:54.626 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librgw2-2:19.0.0-2765.g2e1f7622.el9.x86_64 107/170 2024-05-22T22:49:54.703 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librgw2-2:19.0.0-2765.g2e1f7622.el9.x86_64 107/170 2024-05-22T22:49:54.727 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rgw-2:19.0.0-2765.g2e1f7622.el9.x86_64 108/170 2024-05-22T22:49:56.428 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-05-22T22:49:56.516 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-22T22:49:56.554 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-22T22:49:56.608 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-22T22:49:56.642 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-22T22:49:56.668 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-22T22:49:56.710 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-22T22:49:56.915 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-common-2:19.0.0-2765.g2e1f7622.el9.x86_64 109/170 2024-05-22T22:49:56.949 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-common-2:19.0.0-2765.g2e1f7622.el9.x86_64 109/170 2024-05-22T22:49:57.030 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-22T22:49:57.135 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librados2-2:19.0.0-2765.g2e1f7622.el9.x86_64 8/170 2024-05-22T22:49:57.176 INFO:teuthology.orchestra.run.smithi096.stdout:(159/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 851 kB/s | 16 MB 00:19 2024-05-22T22:49:57.181 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-05-22T22:49:57.181 INFO:teuthology.orchestra.run.smithi096.stdout:Total 6.4 MB/s | 225 MB 00:35 2024-05-22T22:49:57.258 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:19.0.0-2765.g2e1f7622.el9.x86_64 8/170 2024-05-22T22:49:57.287 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs2-2:19.0.0-2765.g2e1f7622.el9.x86_64 9/170 2024-05-22T22:49:57.378 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephfs2-2:19.0.0-2765.g2e1f7622.el9.x86_64 9/170 2024-05-22T22:49:57.407 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rados-2:19.0.0-2765.g2e1f7622.el9.x86_64 10/170 2024-05-22T22:49:57.449 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-22T22:49:57.494 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-common-2:19.0.0-2765.g2e1f7622.el9.x86_64 109/170 2024-05-22T22:49:57.527 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-base-2:19.0.0-2765.g2e1f7622.el9.x86_64 110/170 2024-05-22T22:49:57.601 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-base-2:19.0.0-2765.g2e1f7622.el9.x86_64 110/170 2024-05-22T22:49:57.601 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-22T22:49:57.602 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-22T22:49:57.602 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:49:57.628 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-selinux-2:19.0.0-2765.g2e1f7622.el9.x86_64 111/170 2024-05-22T22:49:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-22T22:49:57.723 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-22T22:49:57.742 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephsqlite-2:19.0.0-2765.g2e1f7622.el9.x86_64 14/170 2024-05-22T22:49:57.845 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2765.g2e1f7622.el9.x86_64 14/170 2024-05-22T22:49:57.868 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libradosstriper1-2:19.0.0-2765.g2e1f7622.el9.x86 15/170 2024-05-22T22:49:57.931 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2765.g2e1f7622.el9.x86 15/170 2024-05-22T22:49:57.953 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-22T22:49:57.974 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-22T22:49:58.162 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-22T22:49:58.293 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-22T22:49:58.351 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-05-22T22:49:58.467 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-22T22:49:58.510 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-05-22T22:49:58.510 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-05-22T22:49:58.528 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-22T22:49:58.618 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-22T22:49:58.704 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-22T22:49:58.894 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-22T22:49:59.002 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-22T22:49:59.098 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-22T22:49:59.136 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-05-22T22:49:59.171 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-05-22T22:49:59.236 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-05-22T22:49:59.270 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-05-22T22:49:59.314 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-05-22T22:49:59.347 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-05-22T22:49:59.375 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-05-22T22:49:59.434 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-05-22T22:49:59.488 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-05-22T22:49:59.519 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-05-22T22:49:59.555 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-05-22T22:49:59.630 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-05-22T22:49:59.665 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-common-2:19.0.0-2765.g2e1f7622.el9. 34/170 2024-05-22T22:49:59.708 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-argparse-2:19.0.0-2765.g2e1f7622.el 35/170 2024-05-22T22:49:59.737 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cephfs-2:19.0.0-2765.g2e1f7622.el9.x86_6 36/170 2024-05-22T22:49:59.807 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-05-22T22:49:59.852 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-05-22T22:49:59.918 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-05-22T22:50:00.053 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-05-22T22:50:00.106 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-05-22T22:50:00.399 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-05-22T22:50:00.399 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-05-22T22:50:00.637 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-05-22T22:50:00.719 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-05-22T22:50:00.747 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-05-22T22:50:00.783 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-05-22T22:50:00.832 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-05-22T22:50:00.870 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librados-devel-2:19.0.0-2765.g2e1f7622.el9.x86_6 47/170 2024-05-22T22:50:00.912 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-05-22T22:50:01.061 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-05-22T22:50:01.115 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-05-22T22:50:01.171 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-05-22T22:50:01.251 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-22T22:50:01.291 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-22T22:50:01.327 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-22T22:50:01.374 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-22T22:50:01.410 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-22T22:50:01.454 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-22T22:50:01.495 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-22T22:50:01.632 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-22T22:50:01.672 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-22T22:50:01.714 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-22T22:50:01.808 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-22T22:50:01.839 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-22T22:50:01.879 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-22T22:50:01.904 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-22T22:50:01.929 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-22T22:50:02.006 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-22T22:50:02.131 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-22T22:50:02.170 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-22T22:50:02.198 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-22T22:50:02.233 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-22T22:50:02.271 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-22T22:50:02.303 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-22T22:50:02.324 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-22T22:50:02.351 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-22T22:50:02.384 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-22T22:50:02.426 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-22T22:50:02.460 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-22T22:50:02.547 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-22T22:50:02.590 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-22T22:50:02.622 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-22T22:50:02.673 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-22T22:50:02.776 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-22T22:50:02.824 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-22T22:50:03.553 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-22T22:50:03.596 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-22T22:50:03.630 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-22T22:50:03.729 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-22T22:50:03.785 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-22T22:50:03.830 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-22T22:50:03.918 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-22T22:50:03.949 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-22T22:50:04.127 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-22T22:50:04.155 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-22T22:50:04.234 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-22T22:50:04.886 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-22T22:50:04.996 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-22T22:50:06.233 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-22T22:50:06.327 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-22T22:50:06.355 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-22T22:50:06.388 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-22T22:50:06.587 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-05-22T22:50:06.696 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-22T22:50:06.699 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-22T22:50:06.734 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-22T22:50:06.751 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librbd1-2:19.0.0-2765.g2e1f7622.el9.x86_64 103/170 2024-05-22T22:50:06.787 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-22T22:50:06.821 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-22T22:50:06.843 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:19.0.0-2765.g2e1f7622.el9.x86_64 103/170 2024-05-22T22:50:06.847 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-22T22:50:06.873 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rbd-2:19.0.0-2765.g2e1f7622.el9.x86_64 104/170 2024-05-22T22:50:06.889 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-22T22:50:06.909 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-22T22:50:07.274 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-22T22:50:07.323 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librados2-2:19.0.0-2765.g2e1f7622.el9.x86_64 8/170 2024-05-22T22:50:07.380 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-22T22:50:07.413 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librgw2-2:19.0.0-2765.g2e1f7622.el9.x86_64 107/170 2024-05-22T22:50:07.444 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:19.0.0-2765.g2e1f7622.el9.x86_64 8/170 2024-05-22T22:50:07.482 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs2-2:19.0.0-2765.g2e1f7622.el9.x86_64 9/170 2024-05-22T22:50:07.484 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librgw2-2:19.0.0-2765.g2e1f7622.el9.x86_64 107/170 2024-05-22T22:50:07.514 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rgw-2:19.0.0-2765.g2e1f7622.el9.x86_64 108/170 2024-05-22T22:50:07.572 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephfs2-2:19.0.0-2765.g2e1f7622.el9.x86_64 9/170 2024-05-22T22:50:07.603 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rados-2:19.0.0-2765.g2e1f7622.el9.x86_64 10/170 2024-05-22T22:50:07.636 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-22T22:50:07.837 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-22T22:50:07.893 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-22T22:50:07.921 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephsqlite-2:19.0.0-2765.g2e1f7622.el9.x86_64 14/170 2024-05-22T22:50:08.023 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2765.g2e1f7622.el9.x86_64 14/170 2024-05-22T22:50:08.047 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libradosstriper1-2:19.0.0-2765.g2e1f7622.el9.x86 15/170 2024-05-22T22:50:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2765.g2e1f7622.el9.x86 15/170 2024-05-22T22:50:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-22T22:50:08.160 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-22T22:50:08.348 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-22T22:50:08.475 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-22T22:50:08.630 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-22T22:50:08.707 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-22T22:50:08.818 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-22T22:50:08.892 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-22T22:50:09.090 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-22T22:50:09.171 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2765.g2e1f7622.el9.x86_64 111/170 2024-05-22T22:50:09.171 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /sys 2024-05-22T22:50:09.171 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /proc 2024-05-22T22:50:09.171 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /mnt 2024-05-22T22:50:09.171 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /var/tmp 2024-05-22T22:50:09.172 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /home 2024-05-22T22:50:09.172 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /root 2024-05-22T22:50:09.172 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /tmp 2024-05-22T22:50:09.172 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:50:09.196 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-22T22:50:09.290 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-22T22:50:09.334 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-volume-2:19.0.0-2765.g2e1f7622.el9.noarch 112/170 2024-05-22T22:50:09.339 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-05-22T22:50:09.350 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-volume-2:19.0.0-2765.g2e1f7622.el9.noarch 112/170 2024-05-22T22:50:09.351 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:09.351 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-22T22:50:09.351 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:50:09.373 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-05-22T22:50:09.436 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-05-22T22:50:09.472 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-05-22T22:50:09.516 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-05-22T22:50:09.557 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-05-22T22:50:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-05-22T22:50:09.652 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-05-22T22:50:09.706 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-05-22T22:50:09.738 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-05-22T22:50:09.756 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.0.0-2765.g2e1f7622.el9.x86_64 109/170 2024-05-22T22:50:09.773 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-05-22T22:50:09.857 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-05-22T22:50:09.883 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-common-2:19.0.0-2765.g2e1f7622.el9.x86_64 109/170 2024-05-22T22:50:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-common-2:19.0.0-2765.g2e1f7622.el9. 34/170 2024-05-22T22:50:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-argparse-2:19.0.0-2765.g2e1f7622.el 35/170 2024-05-22T22:50:09.964 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cephfs-2:19.0.0-2765.g2e1f7622.el9.x86_6 36/170 2024-05-22T22:50:10.033 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-05-22T22:50:10.079 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-05-22T22:50:10.154 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-05-22T22:50:10.284 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-05-22T22:50:10.331 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-05-22T22:50:10.427 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.0.0-2765.g2e1f7622.el9.x86_64 109/170 2024-05-22T22:50:10.465 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-base-2:19.0.0-2765.g2e1f7622.el9.x86_64 110/170 2024-05-22T22:50:10.556 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-base-2:19.0.0-2765.g2e1f7622.el9.x86_64 110/170 2024-05-22T22:50:10.556 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-22T22:50:10.556 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-22T22:50:10.557 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-22T22:50:10.582 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-selinux-2:19.0.0-2765.g2e1f7622.el9.x86_64 111/170 2024-05-22T22:50:10.837 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-osd-2:19.0.0-2765.g2e1f7622.el9.x86_64 113/170 2024-05-22T22:50:10.843 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-05-22T22:50:10.865 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-osd-2:19.0.0-2765.g2e1f7622.el9.x86_64 113/170 2024-05-22T22:50:10.865 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:10.866 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-22T22:50:10.866 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-22T22:50:10.866 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-22T22:50:10.866 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:50:10.927 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-05-22T22:50:10.956 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-05-22T22:50:10.984 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-05-22T22:50:11.033 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-05-22T22:50:11.071 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librados-devel-2:19.0.0-2765.g2e1f7622.el9.x86_6 47/170 2024-05-22T22:50:11.096 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mds-2:19.0.0-2765.g2e1f7622.el9.x86_64 114/170 2024-05-22T22:50:11.104 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-05-22T22:50:11.120 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mds-2:19.0.0-2765.g2e1f7622.el9.x86_64 114/170 2024-05-22T22:50:11.120 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:11.120 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-22T22:50:11.120 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-22T22:50:11.120 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-22T22:50:11.120 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:50:11.267 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-05-22T22:50:11.316 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-05-22T22:50:11.370 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-05-22T22:50:11.443 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-22T22:50:11.475 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-22T22:50:11.511 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-22T22:50:11.555 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mon-2:19.0.0-2765.g2e1f7622.el9.x86_64 115/170 2024-05-22T22:50:11.557 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-22T22:50:11.579 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mon-2:19.0.0-2765.g2e1f7622.el9.x86_64 115/170 2024-05-22T22:50:11.579 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:11.579 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-22T22:50:11.579 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-22T22:50:11.579 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-22T22:50:11.579 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:50:11.594 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-22T22:50:11.628 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-22T22:50:11.670 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-22T22:50:11.677 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-22T22:50:11.734 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-22T22:50:11.789 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-22T22:50:11.807 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-22T22:50:11.856 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-22T22:50:11.897 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-22T22:50:11.982 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-22T22:50:11.991 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-22T22:50:12.031 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-22T22:50:12.031 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-22T22:50:12.063 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-22T22:50:12.088 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-22T22:50:12.108 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-22T22:50:12.112 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-22T22:50:12.186 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-22T22:50:12.196 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-22T22:50:12.313 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-22T22:50:12.335 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-22T22:50:12.353 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-22T22:50:12.381 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-22T22:50:12.382 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-22T22:50:12.413 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-22T22:50:12.432 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-22T22:50:12.440 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-22T22:50:12.471 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-22T22:50:12.503 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-22T22:50:12.533 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-22T22:50:12.559 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-22T22:50:12.593 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-22T22:50:12.634 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-22T22:50:12.668 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-22T22:50:12.755 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-22T22:50:12.785 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: cephadm-2:19.0.0-2765.g2e1f7622.el9.noarch 127/170 2024-05-22T22:50:12.798 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-22T22:50:12.814 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephadm-2:19.0.0-2765.g2e1f7622.el9.noarch 127/170 2024-05-22T22:50:12.830 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-22T22:50:12.845 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2765.g2e1f7622.e 128/170 2024-05-22T22:50:12.877 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-22T22:50:12.907 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2765.g2e1f7622. 129/170 2024-05-22T22:50:12.937 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2765.g2e1f7622.el9.noa 130/170 2024-05-22T22:50:12.973 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-22T22:50:13.031 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-22T22:50:13.752 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-22T22:50:13.795 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-22T22:50:13.828 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-22T22:50:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-22T22:50:13.963 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2765.g2e1f7622.el9.noa 130/170 2024-05-22T22:50:13.983 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-22T22:50:13.997 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2765.g2e1f7622.el9.n 131/170 2024-05-22T22:50:14.028 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-22T22:50:14.107 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-22T22:50:14.148 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-22T22:50:14.324 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-22T22:50:14.354 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-22T22:50:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-22T22:50:14.961 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2765.g2e1f7622.el9.n 131/170 2024-05-22T22:50:14.989 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2765.g2e1 132/170 2024-05-22T22:50:15.061 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-22T22:50:15.091 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2765.g2e1 132/170 2024-05-22T22:50:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-22T22:50:15.262 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2765.g2e1f7622.el 133/170 2024-05-22T22:50:15.293 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-2:19.0.0-2765.g2e1f7622.el9.x86_64 134/170 2024-05-22T22:50:15.316 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2765.g2e1f7622.el9.x86_64 134/170 2024-05-22T22:50:15.316 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:15.317 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-22T22:50:15.317 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-22T22:50:15.317 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-22T22:50:15.317 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:50:15.365 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-rook-2:19.0.0-2765.g2e1f7622.el9.noarch 135/170 2024-05-22T22:50:15.376 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2765.g2e1f7622.el9.noarch 135/170 2024-05-22T22:50:15.473 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-2:19.0.0-2765.g2e1f7622.el9.x86_64 136/170 2024-05-22T22:50:15.531 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-fuse-2:19.0.0-2765.g2e1f7622.el9.x86_64 137/170 2024-05-22T22:50:16.384 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-22T22:50:16.427 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-22T22:50:16.459 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-radosgw-2:19.0.0-2765.g2e1f7622.el9.x86_64 139/170 2024-05-22T22:50:16.466 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-22T22:50:16.483 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2765.g2e1f7622.el9.x86_64 139/170 2024-05-22T22:50:16.483 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:16.483 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-22T22:50:16.483 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-22T22:50:16.483 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-22T22:50:16.483 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:50:16.493 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-22T22:50:16.526 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2765.g2e1f7 140/170 2024-05-22T22:50:16.527 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-22T22:50:16.549 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2765.g2e1f7 140/170 2024-05-22T22:50:16.550 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:16.550 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-22T22:50:16.550 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:50:16.594 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephfs-mirror-2:19.0.0-2765.g2e1f7622.el9.x86_64 141/170 2024-05-22T22:50:16.617 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2765.g2e1f7622.el9.x86_64 141/170 2024-05-22T22:50:16.617 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:16.617 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-22T22:50:16.617 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-22T22:50:16.618 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-22T22:50:16.618 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:50:16.832 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-22T22:50:16.873 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librbd1-2:19.0.0-2765.g2e1f7622.el9.x86_64 103/170 2024-05-22T22:50:16.919 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-mirror-2:19.0.0-2765.g2e1f7622.el9.x86_64 142/170 2024-05-22T22:50:16.942 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2765.g2e1f7622.el9.x86_64 142/170 2024-05-22T22:50:16.942 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:16.943 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-22T22:50:16.943 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-22T22:50:16.943 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-22T22:50:16.943 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:50:16.963 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:19.0.0-2765.g2e1f7622.el9.x86_64 103/170 2024-05-22T22:50:16.994 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rbd-2:19.0.0-2765.g2e1f7622.el9.x86_64 104/170 2024-05-22T22:50:17.030 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-22T22:50:17.498 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-22T22:50:17.526 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librgw2-2:19.0.0-2765.g2e1f7622.el9.x86_64 107/170 2024-05-22T22:50:17.597 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librgw2-2:19.0.0-2765.g2e1f7622.el9.x86_64 107/170 2024-05-22T22:50:17.619 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rgw-2:19.0.0-2765.g2e1f7622.el9.x86_64 108/170 2024-05-22T22:50:19.866 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.0.0-2765.g2e1f7622.el9.x86_64 109/170 2024-05-22T22:50:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-common-2:19.0.0-2765.g2e1f7622.el9.x86_64 109/170 2024-05-22T22:50:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.0.0-2765.g2e1f7622.el9.x86_64 109/170 2024-05-22T22:50:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-base-2:19.0.0-2765.g2e1f7622.el9.x86_64 110/170 2024-05-22T22:50:20.577 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-base-2:19.0.0-2765.g2e1f7622.el9.x86_64 110/170 2024-05-22T22:50:20.577 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-22T22:50:20.577 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-22T22:50:20.577 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-22T22:50:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-selinux-2:19.0.0-2765.g2e1f7622.el9.x86_64 111/170 2024-05-22T22:50:21.455 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-test-2:19.0.0-2765.g2e1f7622.el9.x86_64 143/170 2024-05-22T22:50:21.497 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-fuse-2:19.0.0-2765.g2e1f7622.el9.x86_64 144/170 2024-05-22T22:50:21.537 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-nbd-2:19.0.0-2765.g2e1f7622.el9.x86_64 145/170 2024-05-22T22:50:21.569 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-22T22:50:21.597 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-22T22:50:21.782 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephfs-shell-2:19.0.0-2765.g2e1f7622.el9.x86_64 148/170 2024-05-22T22:50:21.834 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-22T22:50:21.868 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephfs-devel-2:19.0.0-2765.g2e1f7622.el9.x86_ 150/170 2024-05-22T22:50:21.908 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-22T22:50:21.937 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-22T22:50:22.000 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-22T22:50:22.206 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2765.g2e1f7622.el9.x86_64 111/170 2024-05-22T22:50:22.207 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /sys 2024-05-22T22:50:22.207 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /proc 2024-05-22T22:50:22.207 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /mnt 2024-05-22T22:50:22.207 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /var/tmp 2024-05-22T22:50:22.207 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /home 2024-05-22T22:50:22.207 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /root 2024-05-22T22:50:22.207 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /tmp 2024-05-22T22:50:22.207 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-22T22:50:22.369 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-volume-2:19.0.0-2765.g2e1f7622.el9.noarch 112/170 2024-05-22T22:50:22.385 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-volume-2:19.0.0-2765.g2e1f7622.el9.noarch 112/170 2024-05-22T22:50:22.385 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:22.385 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-22T22:50:22.385 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-22T22:50:22.568 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-22T22:50:22.606 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-22T22:50:22.626 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-22T22:50:23.442 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephfs-top-2:19.0.0-2765.g2e1f7622.el9.noarch 155/170 2024-05-22T22:50:23.543 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-22T22:50:23.593 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-22T22:50:23.685 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-22T22:50:23.721 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-22T22:50:23.755 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-22T22:50:23.780 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-22T22:50:23.806 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-22T22:50:23.806 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-22T22:50:23.881 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-osd-2:19.0.0-2765.g2e1f7622.el9.x86_64 113/170 2024-05-22T22:50:23.890 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-22T22:50:23.906 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-osd-2:19.0.0-2765.g2e1f7622.el9.x86_64 113/170 2024-05-22T22:50:23.906 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:23.906 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-22T22:50:23.906 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-22T22:50:23.906 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-22T22:50:23.906 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-22T22:50:23.915 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-22T22:50:23.915 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-22T22:50:23.999 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-22T22:50:24.024 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-22T22:50:24.030 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-22T22:50:24.030 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-22T22:50:24.131 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mds-2:19.0.0-2765.g2e1f7622.el9.x86_64 114/170 2024-05-22T22:50:24.154 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mds-2:19.0.0-2765.g2e1f7622.el9.x86_64 114/170 2024-05-22T22:50:24.154 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:24.154 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-22T22:50:24.154 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-22T22:50:24.154 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-22T22:50:24.154 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-22T22:50:24.218 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-22T22:50:24.218 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-22T22:50:24.295 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-22T22:50:24.335 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-22T22:50:24.401 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-22T22:50:24.599 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mon-2:19.0.0-2765.g2e1f7622.el9.x86_64 115/170 2024-05-22T22:50:24.622 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mon-2:19.0.0-2765.g2e1f7622.el9.x86_64 115/170 2024-05-22T22:50:24.622 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:24.622 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-22T22:50:24.623 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-22T22:50:24.623 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-22T22:50:24.623 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-22T22:50:24.720 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-22T22:50:24.778 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-22T22:50:24.833 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-22T22:50:25.020 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-22T22:50:25.066 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-22T22:50:25.145 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-22T22:50:25.242 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-22T22:50:25.382 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-22T22:50:25.433 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-22T22:50:25.474 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-22T22:50:25.510 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-22T22:50:25.878 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephadm-2:19.0.0-2765.g2e1f7622.el9.noarch 127/170 2024-05-22T22:50:25.909 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephadm-2:19.0.0-2765.g2e1f7622.el9.noarch 127/170 2024-05-22T22:50:25.940 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2765.g2e1f7622.e 128/170 2024-05-22T22:50:26.002 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2765.g2e1f7622. 129/170 2024-05-22T22:50:26.040 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2765.g2e1f7622.el9.noa 130/170 2024-05-22T22:50:27.073 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2765.g2e1f7622.el9.noa 130/170 2024-05-22T22:50:27.108 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2765.g2e1f7622.el9.n 131/170 2024-05-22T22:50:28.075 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2765.g2e1f7622.el9.n 131/170 2024-05-22T22:50:28.108 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2765.g2e1 132/170 2024-05-22T22:50:28.211 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2765.g2e1 132/170 2024-05-22T22:50:28.395 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2765.g2e1f7622.el 133/170 2024-05-22T22:50:28.428 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-2:19.0.0-2765.g2e1f7622.el9.x86_64 134/170 2024-05-22T22:50:28.451 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2765.g2e1f7622.el9.x86_64 134/170 2024-05-22T22:50:28.451 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:28.451 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-22T22:50:28.452 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-22T22:50:28.452 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-22T22:50:28.452 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-22T22:50:28.495 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-rook-2:19.0.0-2765.g2e1f7622.el9.noarch 135/170 2024-05-22T22:50:28.506 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2765.g2e1f7622.el9.noarch 135/170 2024-05-22T22:50:28.604 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-2:19.0.0-2765.g2e1f7622.el9.x86_64 136/170 2024-05-22T22:50:28.651 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-fuse-2:19.0.0-2765.g2e1f7622.el9.x86_64 137/170 2024-05-22T22:50:29.562 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-22T22:50:29.598 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-radosgw-2:19.0.0-2765.g2e1f7622.el9.x86_64 139/170 2024-05-22T22:50:29.620 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2765.g2e1f7622.el9.x86_64 139/170 2024-05-22T22:50:29.621 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:29.621 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-22T22:50:29.621 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-22T22:50:29.621 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-22T22:50:29.621 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-22T22:50:29.665 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2765.g2e1f7 140/170 2024-05-22T22:50:29.686 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2765.g2e1f7 140/170 2024-05-22T22:50:29.687 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:29.687 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-22T22:50:29.687 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-22T22:50:29.732 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-mirror-2:19.0.0-2765.g2e1f7622.el9.x86_64 141/170 2024-05-22T22:50:29.754 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2765.g2e1f7622.el9.x86_64 141/170 2024-05-22T22:50:29.754 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:29.754 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-22T22:50:29.754 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-22T22:50:29.755 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-22T22:50:29.755 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-22T22:50:29.975 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-22T22:50:29.976 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-2:19.0.0-2765.g2e1f7622.el9.x86_64 1/170 2024-05-22T22:50:29.976 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-base-2:19.0.0-2765.g2e1f7622.el9.x86_64 2/170 2024-05-22T22:50:29.976 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-common-2:19.0.0-2765.g2e1f7622.el9.x86_64 3/170 2024-05-22T22:50:29.976 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-fuse-2:19.0.0-2765.g2e1f7622.el9.x86_64 4/170 2024-05-22T22:50:29.976 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2765.g2e1f7 5/170 2024-05-22T22:50:29.976 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mds-2:19.0.0-2765.g2e1f7622.el9.x86_64 6/170 2024-05-22T22:50:29.976 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-2:19.0.0-2765.g2e1f7622.el9.x86_64 7/170 2024-05-22T22:50:29.976 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mon-2:19.0.0-2765.g2e1f7622.el9.x86_64 8/170 2024-05-22T22:50:29.976 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-osd-2:19.0.0-2765.g2e1f7622.el9.x86_64 9/170 2024-05-22T22:50:29.976 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-radosgw-2:19.0.0-2765.g2e1f7622.el9.x86_64 10/170 2024-05-22T22:50:29.976 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-selinux-2:19.0.0-2765.g2e1f7622.el9.x86_64 11/170 2024-05-22T22:50:29.977 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-test-2:19.0.0-2765.g2e1f7622.el9.x86_64 12/170 2024-05-22T22:50:29.977 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephfs-mirror-2:19.0.0-2765.g2e1f7622.el9.x86_64 13/170 2024-05-22T22:50:29.977 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephfs-shell-2:19.0.0-2765.g2e1f7622.el9.x86_64 14/170 2024-05-22T22:50:29.977 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephfs-devel-2:19.0.0-2765.g2e1f7622.el9.x86_ 15/170 2024-05-22T22:50:29.977 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephfs2-2:19.0.0-2765.g2e1f7622.el9.x86_64 16/170 2024-05-22T22:50:29.977 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephsqlite-2:19.0.0-2765.g2e1f7622.el9.x86_64 17/170 2024-05-22T22:50:29.977 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados-devel-2:19.0.0-2765.g2e1f7622.el9.x86_6 18/170 2024-05-22T22:50:29.977 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libradosstriper1-2:19.0.0-2765.g2e1f7622.el9.x86 19/170 2024-05-22T22:50:29.977 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librgw2-2:19.0.0-2765.g2e1f7622.el9.x86_64 20/170 2024-05-22T22:50:29.977 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2765.g2e1f7622.el 21/170 2024-05-22T22:50:29.977 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ceph-common-2:19.0.0-2765.g2e1f7622.el9. 22/170 2024-05-22T22:50:29.977 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cephfs-2:19.0.0-2765.g2e1f7622.el9.x86_6 23/170 2024-05-22T22:50:29.977 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rados-2:19.0.0-2765.g2e1f7622.el9.x86_64 24/170 2024-05-22T22:50:29.978 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rbd-2:19.0.0-2765.g2e1f7622.el9.x86_64 25/170 2024-05-22T22:50:29.978 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rgw-2:19.0.0-2765.g2e1f7622.el9.x86_64 26/170 2024-05-22T22:50:29.978 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-fuse-2:19.0.0-2765.g2e1f7622.el9.x86_64 27/170 2024-05-22T22:50:29.978 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-mirror-2:19.0.0-2765.g2e1f7622.el9.x86_64 28/170 2024-05-22T22:50:29.978 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-nbd-2:19.0.0-2765.g2e1f7622.el9.x86_64 29/170 2024-05-22T22:50:29.978 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2765.g2e1f7622. 30/170 2024-05-22T22:50:29.978 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2765.g2e1f7622.el9.noa 31/170 2024-05-22T22:50:29.978 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2765.g2e1f7622.el9.n 32/170 2024-05-22T22:50:29.978 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2765.g2e1 33/170 2024-05-22T22:50:29.978 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2765.g2e1f7622.el 34/170 2024-05-22T22:50:29.978 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2765.g2e1f7622.el9.noarch 35/170 2024-05-22T22:50:29.978 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2765.g2e1f7622.e 36/170 2024-05-22T22:50:29.978 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-volume-2:19.0.0-2765.g2e1f7622.el9.noarch 37/170 2024-05-22T22:50:29.979 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephadm-2:19.0.0-2765.g2e1f7622.el9.noarch 38/170 2024-05-22T22:50:29.979 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephfs-top-2:19.0.0-2765.g2e1f7622.el9.noarch 39/170 2024-05-22T22:50:29.979 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-22T22:50:29.979 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-22T22:50:29.979 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-22T22:50:29.979 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-22T22:50:29.979 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-22T22:50:29.979 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-22T22:50:29.979 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-22T22:50:29.979 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-22T22:50:29.979 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-22T22:50:29.979 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-22T22:50:29.980 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-22T22:50:29.980 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-22T22:50:29.980 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-22T22:50:29.980 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-22T22:50:29.980 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-22T22:50:29.980 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-22T22:50:29.980 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-22T22:50:29.980 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-22T22:50:29.980 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-22T22:50:29.980 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-22T22:50:29.980 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-22T22:50:29.980 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-22T22:50:29.980 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-22T22:50:29.981 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-22T22:50:29.981 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-22T22:50:29.981 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-22T22:50:29.981 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-22T22:50:29.981 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-22T22:50:29.981 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-22T22:50:29.981 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-22T22:50:29.981 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-22T22:50:29.981 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-22T22:50:29.981 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-22T22:50:29.981 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-22T22:50:29.981 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-22T22:50:29.981 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-22T22:50:29.982 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-22T22:50:29.982 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-22T22:50:29.982 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-22T22:50:29.982 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-22T22:50:29.982 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-22T22:50:29.982 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-22T22:50:29.982 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-22T22:50:29.982 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-22T22:50:29.982 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-22T22:50:29.982 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-22T22:50:29.982 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-22T22:50:29.983 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-22T22:50:29.983 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-22T22:50:29.983 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-22T22:50:29.983 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-22T22:50:29.983 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-22T22:50:29.983 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-22T22:50:29.983 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-22T22:50:29.983 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-22T22:50:29.983 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-22T22:50:29.983 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-22T22:50:29.983 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-22T22:50:29.983 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-22T22:50:29.983 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-22T22:50:29.984 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-22T22:50:29.984 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-22T22:50:29.984 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-22T22:50:29.984 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-22T22:50:29.984 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-22T22:50:29.984 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-22T22:50:29.984 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-22T22:50:29.984 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-22T22:50:29.984 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-22T22:50:29.984 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-22T22:50:29.984 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-22T22:50:29.984 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-22T22:50:29.985 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-22T22:50:29.985 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-22T22:50:29.985 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-22T22:50:29.985 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-22T22:50:29.985 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-22T22:50:29.985 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-22T22:50:29.985 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-22T22:50:29.985 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-22T22:50:29.985 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-22T22:50:29.985 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-22T22:50:29.985 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-22T22:50:29.985 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-22T22:50:29.986 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-22T22:50:29.986 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-22T22:50:29.986 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-22T22:50:29.986 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-22T22:50:29.986 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-22T22:50:29.986 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-22T22:50:29.986 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-22T22:50:29.986 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-22T22:50:29.986 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-22T22:50:29.986 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-22T22:50:29.986 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-22T22:50:29.986 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-22T22:50:29.987 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-22T22:50:29.987 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-22T22:50:29.987 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-22T22:50:29.987 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-22T22:50:29.987 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-22T22:50:29.987 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-22T22:50:29.987 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-22T22:50:29.987 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-22T22:50:29.987 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-22T22:50:29.987 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-22T22:50:29.987 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-22T22:50:29.988 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-22T22:50:29.988 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-22T22:50:29.988 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados2-2:19.0.0-2765.g2e1f7622.el9.x86_64 149/170 2024-05-22T22:50:29.988 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-22T22:50:29.988 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librbd1-2:19.0.0-2765.g2e1f7622.el9.x86_64 151/170 2024-05-22T22:50:29.988 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-22T22:50:29.988 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-22T22:50:29.988 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-22T22:50:29.988 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-22T22:50:29.988 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-22T22:50:29.988 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-22T22:50:29.988 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-22T22:50:29.989 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-22T22:50:29.989 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-22T22:50:29.989 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-22T22:50:29.989 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-22T22:50:29.989 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-22T22:50:29.989 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-22T22:50:29.989 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-22T22:50:29.989 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-22T22:50:29.989 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-22T22:50:29.989 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-22T22:50:29.989 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-22T22:50:30.058 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-mirror-2:19.0.0-2765.g2e1f7622.el9.x86_64 142/170 2024-05-22T22:50:30.081 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2765.g2e1f7622.el9.x86_64 142/170 2024-05-22T22:50:30.081 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:30.081 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-22T22:50:30.081 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-22T22:50:30.081 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-22T22:50:30.081 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-22T22:50:32.376 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2765.g2e1f7622.el9.x86_64 111/170 2024-05-22T22:50:32.376 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /sys 2024-05-22T22:50:32.376 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /proc 2024-05-22T22:50:32.376 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /mnt 2024-05-22T22:50:32.376 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /var/tmp 2024-05-22T22:50:32.376 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /home 2024-05-22T22:50:32.376 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /root 2024-05-22T22:50:32.377 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /tmp 2024-05-22T22:50:32.377 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-22T22:50:32.547 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-volume-2:19.0.0-2765.g2e1f7622.el9.noarch 112/170 2024-05-22T22:50:32.562 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-volume-2:19.0.0-2765.g2e1f7622.el9.noarch 112/170 2024-05-22T22:50:32.562 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:32.562 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-22T22:50:32.562 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-22T22:50:32.858 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-22T22:50:32.859 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:50:32.859 INFO:teuthology.orchestra.run.smithi084.stdout:Upgraded: 2024-05-22T22:50:32.859 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-22T22:50:32.859 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-22T22:50:32.859 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-22T22:50:32.860 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-22T22:50:32.860 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-22T22:50:32.860 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-22T22:50:32.860 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-22T22:50:32.860 INFO:teuthology.orchestra.run.smithi084.stdout: librados2-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.860 INFO:teuthology.orchestra.run.smithi084.stdout: librbd1-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.861 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-22T22:50:32.861 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-22T22:50:32.861 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-05-22T22:50:32.861 INFO:teuthology.orchestra.run.smithi084.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-22T22:50:32.861 INFO:teuthology.orchestra.run.smithi084.stdout: automake-1.16.2-8.el9.noarch 2024-05-22T22:50:32.861 INFO:teuthology.orchestra.run.smithi084.stdout: bison-3.7.4-5.el9.x86_64 2024-05-22T22:50:32.862 INFO:teuthology.orchestra.run.smithi084.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-22T22:50:32.862 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.862 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.862 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-common-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.862 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-fuse-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.863 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-grafana-dashboards-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:32.863 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-immutable-object-cache-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.863 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mds-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.863 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.863 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-cephadm-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:32.863 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:32.864 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:32.864 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:32.864 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-rook-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:32.864 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mon-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.864 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-osd-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.865 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-prometheus-alerts-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:32.865 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-radosgw-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.865 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-selinux-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.865 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-test-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.865 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-volume-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:32.865 INFO:teuthology.orchestra.run.smithi084.stdout: cephadm-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:32.866 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-mirror-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.866 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-shell-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.866 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-top-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:32.866 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-22T22:50:32.866 INFO:teuthology.orchestra.run.smithi084.stdout: flex-2.6.4-9.el9.x86_64 2024-05-22T22:50:32.867 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-22T22:50:32.867 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-22T22:50:32.867 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-22T22:50:32.867 INFO:teuthology.orchestra.run.smithi084.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-22T22:50:32.867 INFO:teuthology.orchestra.run.smithi084.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-22T22:50:32.867 INFO:teuthology.orchestra.run.smithi084.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-22T22:50:32.868 INFO:teuthology.orchestra.run.smithi084.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-22T22:50:32.868 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-22T22:50:32.868 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-22T22:50:32.868 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs-devel-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.868 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs2-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.868 INFO:teuthology.orchestra.run.smithi084.stdout: libcephsqlite-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.869 INFO:teuthology.orchestra.run.smithi084.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-22T22:50:32.869 INFO:teuthology.orchestra.run.smithi084.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-22T22:50:32.869 INFO:teuthology.orchestra.run.smithi084.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-22T22:50:32.869 INFO:teuthology.orchestra.run.smithi084.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-22T22:50:32.869 INFO:teuthology.orchestra.run.smithi084.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-22T22:50:32.869 INFO:teuthology.orchestra.run.smithi084.stdout: librados-devel-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.870 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.870 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-22T22:50:32.870 INFO:teuthology.orchestra.run.smithi084.stdout: librgw2-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.870 INFO:teuthology.orchestra.run.smithi084.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-22T22:50:32.870 INFO:teuthology.orchestra.run.smithi084.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-22T22:50:32.870 INFO:teuthology.orchestra.run.smithi084.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-22T22:50:32.871 INFO:teuthology.orchestra.run.smithi084.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-22T22:50:32.871 INFO:teuthology.orchestra.run.smithi084.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-22T22:50:32.871 INFO:teuthology.orchestra.run.smithi084.stdout: lua-5.4.4-4.el9.x86_64 2024-05-22T22:50:32.871 INFO:teuthology.orchestra.run.smithi084.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-22T22:50:32.871 INFO:teuthology.orchestra.run.smithi084.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-22T22:50:32.871 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-22T22:50:32.871 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-22T22:50:32.872 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-22T22:50:32.872 INFO:teuthology.orchestra.run.smithi084.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-22T22:50:32.872 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-13.14-1.el9.x86_64 2024-05-22T22:50:32.872 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-22T22:50:32.872 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-22T22:50:32.872 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-22T22:50:32.872 INFO:teuthology.orchestra.run.smithi084.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-22T22:50:32.873 INFO:teuthology.orchestra.run.smithi084.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-22T22:50:32.873 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-22T22:50:32.873 INFO:teuthology.orchestra.run.smithi084.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-22T22:50:32.873 INFO:teuthology.orchestra.run.smithi084.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-22T22:50:32.873 INFO:teuthology.orchestra.run.smithi084.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-22T22:50:32.873 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-22T22:50:32.873 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.873 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-common-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.873 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cephfs-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.873 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-22T22:50:32.873 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-22T22:50:32.873 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-22T22:50:32.874 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-22T22:50:32.874 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-22T22:50:32.874 INFO:teuthology.orchestra.run.smithi084.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-22T22:50:32.874 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-22T22:50:32.874 INFO:teuthology.orchestra.run.smithi084.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-22T22:50:32.874 INFO:teuthology.orchestra.run.smithi084.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-22T22:50:32.874 INFO:teuthology.orchestra.run.smithi084.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-22T22:50:32.874 INFO:teuthology.orchestra.run.smithi084.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-22T22:50:32.874 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-22T22:50:32.874 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-22T22:50:32.874 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-22T22:50:32.874 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-22T22:50:32.875 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-22T22:50:32.875 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-22T22:50:32.875 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-22T22:50:32.875 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-22T22:50:32.875 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-22T22:50:32.875 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-22T22:50:32.875 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-22T22:50:32.875 INFO:teuthology.orchestra.run.smithi084.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-22T22:50:32.875 INFO:teuthology.orchestra.run.smithi084.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-22T22:50:32.875 INFO:teuthology.orchestra.run.smithi084.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-22T22:50:32.875 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-22T22:50:32.875 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-22T22:50:32.876 INFO:teuthology.orchestra.run.smithi084.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-22T22:50:32.876 INFO:teuthology.orchestra.run.smithi084.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-22T22:50:32.876 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-22T22:50:32.876 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ply-3.11-14.el9.noarch 2024-05-22T22:50:32.876 INFO:teuthology.orchestra.run.smithi084.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-22T22:50:32.876 INFO:teuthology.orchestra.run.smithi084.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-22T22:50:32.876 INFO:teuthology.orchestra.run.smithi084.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-22T22:50:32.876 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-22T22:50:32.876 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-22T22:50:32.876 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-22T22:50:32.876 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-22T22:50:32.876 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-22T22:50:32.876 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-22T22:50:32.877 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.877 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rbd-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.877 INFO:teuthology.orchestra.run.smithi084.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-22T22:50:32.877 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-22T22:50:32.877 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-22T22:50:32.877 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rgw-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.877 INFO:teuthology.orchestra.run.smithi084.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-22T22:50:32.877 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-22T22:50:32.877 INFO:teuthology.orchestra.run.smithi084.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-22T22:50:32.877 INFO:teuthology.orchestra.run.smithi084.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-22T22:50:32.877 INFO:teuthology.orchestra.run.smithi084.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-22T22:50:32.877 INFO:teuthology.orchestra.run.smithi084.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-22T22:50:32.877 INFO:teuthology.orchestra.run.smithi084.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-22T22:50:32.877 INFO:teuthology.orchestra.run.smithi084.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-22T22:50:32.878 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-22T22:50:32.878 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-22T22:50:32.878 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-22T22:50:32.878 INFO:teuthology.orchestra.run.smithi084.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-22T22:50:32.878 INFO:teuthology.orchestra.run.smithi084.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-22T22:50:32.878 INFO:teuthology.orchestra.run.smithi084.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-22T22:50:32.878 INFO:teuthology.orchestra.run.smithi084.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-22T22:50:32.878 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-fuse-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.878 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-mirror-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.878 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-nbd-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:32.878 INFO:teuthology.orchestra.run.smithi084.stdout: re2-1:20211101-3.el9.x86_64 2024-05-22T22:50:32.878 INFO:teuthology.orchestra.run.smithi084.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-22T22:50:32.878 INFO:teuthology.orchestra.run.smithi084.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-22T22:50:32.879 INFO:teuthology.orchestra.run.smithi084.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-22T22:50:32.879 INFO:teuthology.orchestra.run.smithi084.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-22T22:50:32.879 INFO:teuthology.orchestra.run.smithi084.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-22T22:50:32.879 INFO:teuthology.orchestra.run.smithi084.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-22T22:50:32.879 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:50:32.880 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-05-22T22:50:33.277 DEBUG:teuthology.parallel:result is None 2024-05-22T22:50:33.998 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-osd-2:19.0.0-2765.g2e1f7622.el9.x86_64 113/170 2024-05-22T22:50:34.024 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-osd-2:19.0.0-2765.g2e1f7622.el9.x86_64 113/170 2024-05-22T22:50:34.024 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:34.024 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-22T22:50:34.024 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-22T22:50:34.024 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-22T22:50:34.024 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-22T22:50:34.249 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mds-2:19.0.0-2765.g2e1f7622.el9.x86_64 114/170 2024-05-22T22:50:34.271 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mds-2:19.0.0-2765.g2e1f7622.el9.x86_64 114/170 2024-05-22T22:50:34.271 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:34.271 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-22T22:50:34.271 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-22T22:50:34.271 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-22T22:50:34.271 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-22T22:50:34.603 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-test-2:19.0.0-2765.g2e1f7622.el9.x86_64 143/170 2024-05-22T22:50:34.645 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-fuse-2:19.0.0-2765.g2e1f7622.el9.x86_64 144/170 2024-05-22T22:50:34.694 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-nbd-2:19.0.0-2765.g2e1f7622.el9.x86_64 145/170 2024-05-22T22:50:34.716 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mon-2:19.0.0-2765.g2e1f7622.el9.x86_64 115/170 2024-05-22T22:50:34.734 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-22T22:50:34.738 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mon-2:19.0.0-2765.g2e1f7622.el9.x86_64 115/170 2024-05-22T22:50:34.738 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:34.738 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-22T22:50:34.738 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-22T22:50:34.738 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-22T22:50:34.738 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-22T22:50:34.761 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-22T22:50:34.846 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-22T22:50:34.894 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-22T22:50:34.945 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-shell-2:19.0.0-2765.g2e1f7622.el9.x86_64 148/170 2024-05-22T22:50:34.949 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-22T22:50:34.991 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-22T22:50:35.024 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs-devel-2:19.0.0-2765.g2e1f7622.el9.x86_ 150/170 2024-05-22T22:50:35.064 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-22T22:50:35.094 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-22T22:50:35.132 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-22T22:50:35.133 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-22T22:50:35.183 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-22T22:50:35.261 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-22T22:50:35.349 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-22T22:50:35.479 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-22T22:50:35.525 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-22T22:50:35.557 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-22T22:50:35.585 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-22T22:50:35.731 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-22T22:50:35.763 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-22T22:50:35.783 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-22T22:50:35.928 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephadm-2:19.0.0-2765.g2e1f7622.el9.noarch 127/170 2024-05-22T22:50:35.970 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephadm-2:19.0.0-2765.g2e1f7622.el9.noarch 127/170 2024-05-22T22:50:36.007 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2765.g2e1f7622.e 128/170 2024-05-22T22:50:36.070 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2765.g2e1f7622. 129/170 2024-05-22T22:50:36.099 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2765.g2e1f7622.el9.noa 130/170 2024-05-22T22:50:36.636 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-top-2:19.0.0-2765.g2e1f7622.el9.noarch 155/170 2024-05-22T22:50:36.710 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-22T22:50:36.751 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-22T22:50:36.832 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-22T22:50:36.864 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-22T22:50:36.889 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-22T22:50:36.914 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-22T22:50:36.949 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-22T22:50:36.950 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-22T22:50:37.042 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-22T22:50:37.066 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-22T22:50:37.067 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-22T22:50:37.134 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2765.g2e1f7622.el9.noa 130/170 2024-05-22T22:50:37.142 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-22T22:50:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2765.g2e1f7622.el9.n 131/170 2024-05-22T22:50:37.168 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-22T22:50:37.172 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-22T22:50:37.173 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-22T22:50:37.353 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-22T22:50:37.353 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-22T22:50:37.438 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-22T22:50:37.487 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-22T22:50:37.552 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-22T22:50:38.132 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2765.g2e1f7622.el9.n 131/170 2024-05-22T22:50:38.159 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2765.g2e1 132/170 2024-05-22T22:50:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2765.g2e1 132/170 2024-05-22T22:50:38.422 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2765.g2e1f7622.el 133/170 2024-05-22T22:50:38.461 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-2:19.0.0-2765.g2e1f7622.el9.x86_64 134/170 2024-05-22T22:50:38.484 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2765.g2e1f7622.el9.x86_64 134/170 2024-05-22T22:50:38.484 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:38.484 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-22T22:50:38.484 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-22T22:50:38.484 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-22T22:50:38.484 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-22T22:50:38.529 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-rook-2:19.0.0-2765.g2e1f7622.el9.noarch 135/170 2024-05-22T22:50:38.539 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2765.g2e1f7622.el9.noarch 135/170 2024-05-22T22:50:38.637 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-2:19.0.0-2765.g2e1f7622.el9.x86_64 136/170 2024-05-22T22:50:38.685 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-fuse-2:19.0.0-2765.g2e1f7622.el9.x86_64 137/170 2024-05-22T22:50:39.584 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-22T22:50:39.615 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-radosgw-2:19.0.0-2765.g2e1f7622.el9.x86_64 139/170 2024-05-22T22:50:39.637 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2765.g2e1f7622.el9.x86_64 139/170 2024-05-22T22:50:39.637 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:39.637 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-22T22:50:39.637 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-22T22:50:39.637 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-22T22:50:39.637 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-22T22:50:39.682 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2765.g2e1f7 140/170 2024-05-22T22:50:39.704 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2765.g2e1f7 140/170 2024-05-22T22:50:39.704 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:39.704 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-22T22:50:39.704 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-22T22:50:39.749 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-mirror-2:19.0.0-2765.g2e1f7622.el9.x86_64 141/170 2024-05-22T22:50:39.771 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2765.g2e1f7622.el9.x86_64 141/170 2024-05-22T22:50:39.771 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:39.771 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-22T22:50:39.772 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-22T22:50:39.772 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-22T22:50:39.772 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-22T22:50:40.067 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-mirror-2:19.0.0-2765.g2e1f7622.el9.x86_64 142/170 2024-05-22T22:50:40.089 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2765.g2e1f7622.el9.x86_64 142/170 2024-05-22T22:50:40.089 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-22T22:50:40.089 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-22T22:50:40.089 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-22T22:50:40.089 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-22T22:50:40.089 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-22T22:50:43.169 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-22T22:50:43.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-2:19.0.0-2765.g2e1f7622.el9.x86_64 1/170 2024-05-22T22:50:43.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-base-2:19.0.0-2765.g2e1f7622.el9.x86_64 2/170 2024-05-22T22:50:43.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-common-2:19.0.0-2765.g2e1f7622.el9.x86_64 3/170 2024-05-22T22:50:43.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-fuse-2:19.0.0-2765.g2e1f7622.el9.x86_64 4/170 2024-05-22T22:50:43.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2765.g2e1f7 5/170 2024-05-22T22:50:43.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mds-2:19.0.0-2765.g2e1f7622.el9.x86_64 6/170 2024-05-22T22:50:43.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-2:19.0.0-2765.g2e1f7622.el9.x86_64 7/170 2024-05-22T22:50:43.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mon-2:19.0.0-2765.g2e1f7622.el9.x86_64 8/170 2024-05-22T22:50:43.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-osd-2:19.0.0-2765.g2e1f7622.el9.x86_64 9/170 2024-05-22T22:50:43.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-radosgw-2:19.0.0-2765.g2e1f7622.el9.x86_64 10/170 2024-05-22T22:50:43.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-selinux-2:19.0.0-2765.g2e1f7622.el9.x86_64 11/170 2024-05-22T22:50:43.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-test-2:19.0.0-2765.g2e1f7622.el9.x86_64 12/170 2024-05-22T22:50:43.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-mirror-2:19.0.0-2765.g2e1f7622.el9.x86_64 13/170 2024-05-22T22:50:43.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-shell-2:19.0.0-2765.g2e1f7622.el9.x86_64 14/170 2024-05-22T22:50:43.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs-devel-2:19.0.0-2765.g2e1f7622.el9.x86_ 15/170 2024-05-22T22:50:43.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs2-2:19.0.0-2765.g2e1f7622.el9.x86_64 16/170 2024-05-22T22:50:43.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephsqlite-2:19.0.0-2765.g2e1f7622.el9.x86_64 17/170 2024-05-22T22:50:43.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados-devel-2:19.0.0-2765.g2e1f7622.el9.x86_6 18/170 2024-05-22T22:50:43.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libradosstriper1-2:19.0.0-2765.g2e1f7622.el9.x86 19/170 2024-05-22T22:50:43.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librgw2-2:19.0.0-2765.g2e1f7622.el9.x86_64 20/170 2024-05-22T22:50:43.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2765.g2e1f7622.el 21/170 2024-05-22T22:50:43.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-common-2:19.0.0-2765.g2e1f7622.el9. 22/170 2024-05-22T22:50:43.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cephfs-2:19.0.0-2765.g2e1f7622.el9.x86_6 23/170 2024-05-22T22:50:43.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rados-2:19.0.0-2765.g2e1f7622.el9.x86_64 24/170 2024-05-22T22:50:43.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rbd-2:19.0.0-2765.g2e1f7622.el9.x86_64 25/170 2024-05-22T22:50:43.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rgw-2:19.0.0-2765.g2e1f7622.el9.x86_64 26/170 2024-05-22T22:50:43.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-fuse-2:19.0.0-2765.g2e1f7622.el9.x86_64 27/170 2024-05-22T22:50:43.183 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-mirror-2:19.0.0-2765.g2e1f7622.el9.x86_64 28/170 2024-05-22T22:50:43.183 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-nbd-2:19.0.0-2765.g2e1f7622.el9.x86_64 29/170 2024-05-22T22:50:43.183 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2765.g2e1f7622. 30/170 2024-05-22T22:50:43.184 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2765.g2e1f7622.el9.noa 31/170 2024-05-22T22:50:43.184 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2765.g2e1f7622.el9.n 32/170 2024-05-22T22:50:43.184 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2765.g2e1 33/170 2024-05-22T22:50:43.184 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2765.g2e1f7622.el 34/170 2024-05-22T22:50:43.184 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2765.g2e1f7622.el9.noarch 35/170 2024-05-22T22:50:43.184 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2765.g2e1f7622.e 36/170 2024-05-22T22:50:43.184 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-volume-2:19.0.0-2765.g2e1f7622.el9.noarch 37/170 2024-05-22T22:50:43.184 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephadm-2:19.0.0-2765.g2e1f7622.el9.noarch 38/170 2024-05-22T22:50:43.184 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-top-2:19.0.0-2765.g2e1f7622.el9.noarch 39/170 2024-05-22T22:50:43.184 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-22T22:50:43.184 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-22T22:50:43.184 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-22T22:50:43.184 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-22T22:50:43.185 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-22T22:50:43.185 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-22T22:50:43.185 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-22T22:50:43.185 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-22T22:50:43.185 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-22T22:50:43.185 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-22T22:50:43.185 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-22T22:50:43.185 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-22T22:50:43.185 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-22T22:50:43.185 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-22T22:50:43.185 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-22T22:50:43.185 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-22T22:50:43.185 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-22T22:50:43.186 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-22T22:50:43.186 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-22T22:50:43.186 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-22T22:50:43.186 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-22T22:50:43.186 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-22T22:50:43.186 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-22T22:50:43.186 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-22T22:50:43.186 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-22T22:50:43.186 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-22T22:50:43.186 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-22T22:50:43.186 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-22T22:50:43.186 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-22T22:50:43.186 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-22T22:50:43.186 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-22T22:50:43.187 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-22T22:50:43.187 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-22T22:50:43.187 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-22T22:50:43.187 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-22T22:50:43.187 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-22T22:50:43.187 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-22T22:50:43.187 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-22T22:50:43.187 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-22T22:50:43.187 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-22T22:50:43.188 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-22T22:50:43.188 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-22T22:50:43.188 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-22T22:50:43.188 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-22T22:50:43.188 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-22T22:50:43.188 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-22T22:50:43.188 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-22T22:50:43.188 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-22T22:50:43.188 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-22T22:50:43.188 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-22T22:50:43.188 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-22T22:50:43.189 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-22T22:50:43.189 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-22T22:50:43.189 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-22T22:50:43.189 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-22T22:50:43.189 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-22T22:50:43.189 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-22T22:50:43.189 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-22T22:50:43.189 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-22T22:50:43.189 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-22T22:50:43.190 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-22T22:50:43.190 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-22T22:50:43.190 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-22T22:50:43.190 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-22T22:50:43.190 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-22T22:50:43.190 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-22T22:50:43.190 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-22T22:50:43.190 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-22T22:50:43.190 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-22T22:50:43.190 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-22T22:50:43.190 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-22T22:50:43.190 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-22T22:50:43.190 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-22T22:50:43.190 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-22T22:50:43.191 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-22T22:50:43.191 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-22T22:50:43.191 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-22T22:50:43.191 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-22T22:50:43.191 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-22T22:50:43.191 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-22T22:50:43.191 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-22T22:50:43.191 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-22T22:50:43.191 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-22T22:50:43.191 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-22T22:50:43.191 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-22T22:50:43.191 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-22T22:50:43.191 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-22T22:50:43.191 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-22T22:50:43.192 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-22T22:50:43.192 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-22T22:50:43.192 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-22T22:50:43.192 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-22T22:50:43.192 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-22T22:50:43.192 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-22T22:50:43.192 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-22T22:50:43.192 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-22T22:50:43.192 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-22T22:50:43.192 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-22T22:50:43.192 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-22T22:50:43.192 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-22T22:50:43.192 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-22T22:50:43.193 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-22T22:50:43.193 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-22T22:50:43.193 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-22T22:50:43.193 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-22T22:50:43.193 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-22T22:50:43.193 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-22T22:50:43.193 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-22T22:50:43.193 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-22T22:50:43.193 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:19.0.0-2765.g2e1f7622.el9.x86_64 149/170 2024-05-22T22:50:43.193 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-22T22:50:43.193 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:19.0.0-2765.g2e1f7622.el9.x86_64 151/170 2024-05-22T22:50:43.193 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-22T22:50:43.193 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-22T22:50:43.193 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-22T22:50:43.194 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-22T22:50:43.194 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-22T22:50:43.194 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-22T22:50:43.194 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-22T22:50:43.194 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-22T22:50:43.194 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-22T22:50:43.194 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-22T22:50:43.194 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-22T22:50:43.194 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-22T22:50:43.194 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-22T22:50:43.194 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-22T22:50:43.194 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-22T22:50:43.194 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-22T22:50:43.195 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-22T22:50:43.195 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-22T22:50:44.604 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-test-2:19.0.0-2765.g2e1f7622.el9.x86_64 143/170 2024-05-22T22:50:44.654 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-fuse-2:19.0.0-2765.g2e1f7622.el9.x86_64 144/170 2024-05-22T22:50:44.694 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-nbd-2:19.0.0-2765.g2e1f7622.el9.x86_64 145/170 2024-05-22T22:50:44.726 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-22T22:50:44.754 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-22T22:50:44.936 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-shell-2:19.0.0-2765.g2e1f7622.el9.x86_64 148/170 2024-05-22T22:50:44.992 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-22T22:50:45.024 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs-devel-2:19.0.0-2765.g2e1f7622.el9.x86_ 150/170 2024-05-22T22:50:45.057 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-22T22:50:45.086 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-22T22:50:45.132 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-22T22:50:45.778 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-22T22:50:45.822 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-22T22:50:45.842 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-22T22:50:46.037 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-22T22:50:46.037 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-22T22:50:46.037 INFO:teuthology.orchestra.run.smithi204.stdout:Upgraded: 2024-05-22T22:50:46.037 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-22T22:50:46.037 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-22T22:50:46.037 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-22T22:50:46.037 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-22T22:50:46.037 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-22T22:50:46.038 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-22T22:50:46.038 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-22T22:50:46.038 INFO:teuthology.orchestra.run.smithi204.stdout: librados2-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.038 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.038 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-22T22:50:46.038 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-22T22:50:46.038 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-05-22T22:50:46.038 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-22T22:50:46.038 INFO:teuthology.orchestra.run.smithi204.stdout: automake-1.16.2-8.el9.noarch 2024-05-22T22:50:46.038 INFO:teuthology.orchestra.run.smithi204.stdout: bison-3.7.4-5.el9.x86_64 2024-05-22T22:50:46.038 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-22T22:50:46.038 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.039 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.039 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.039 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.039 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:46.039 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.039 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.039 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.039 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:46.039 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:46.039 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:46.039 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:46.039 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:46.039 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.040 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.040 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:46.040 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.040 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.040 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.040 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:46.040 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:46.040 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.040 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.040 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:46.040 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-22T22:50:46.040 INFO:teuthology.orchestra.run.smithi204.stdout: flex-2.6.4-9.el9.x86_64 2024-05-22T22:50:46.041 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-22T22:50:46.041 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-22T22:50:46.041 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-22T22:50:46.041 INFO:teuthology.orchestra.run.smithi204.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-22T22:50:46.041 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-22T22:50:46.041 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-22T22:50:46.041 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-22T22:50:46.041 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-22T22:50:46.041 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-22T22:50:46.041 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.041 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.041 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.041 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-22T22:50:46.042 INFO:teuthology.orchestra.run.smithi204.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-22T22:50:46.042 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-22T22:50:46.042 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-22T22:50:46.042 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-22T22:50:46.042 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.042 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.042 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-22T22:50:46.042 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.042 INFO:teuthology.orchestra.run.smithi204.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-22T22:50:46.042 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-22T22:50:46.042 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-22T22:50:46.042 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-22T22:50:46.043 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-22T22:50:46.043 INFO:teuthology.orchestra.run.smithi204.stdout: lua-5.4.4-4.el9.x86_64 2024-05-22T22:50:46.043 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-22T22:50:46.043 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-22T22:50:46.043 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-22T22:50:46.043 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-22T22:50:46.043 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-22T22:50:46.043 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-22T22:50:46.043 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-13.14-1.el9.x86_64 2024-05-22T22:50:46.043 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-22T22:50:46.043 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-22T22:50:46.043 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-22T22:50:46.043 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-22T22:50:46.043 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-22T22:50:46.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-22T22:50:46.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-22T22:50:46.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-22T22:50:46.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-22T22:50:46.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-22T22:50:46.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-22T22:50:46.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-22T22:50:46.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-22T22:50:46.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-22T22:50:46.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-22T22:50:46.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-22T22:50:46.045 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-22T22:50:46.045 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-22T22:50:46.045 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-22T22:50:46.045 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-22T22:50:46.045 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-22T22:50:46.045 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-22T22:50:46.045 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-22T22:50:46.045 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-22T22:50:46.045 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-22T22:50:46.045 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-22T22:50:46.045 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-22T22:50:46.045 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-22T22:50:46.045 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-22T22:50:46.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-22T22:50:46.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-22T22:50:46.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-22T22:50:46.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-22T22:50:46.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-22T22:50:46.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-22T22:50:46.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-22T22:50:46.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-22T22:50:46.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-22T22:50:46.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-22T22:50:46.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-22T22:50:46.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply-3.11-14.el9.noarch 2024-05-22T22:50:46.047 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-22T22:50:46.047 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-22T22:50:46.047 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-22T22:50:46.047 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-22T22:50:46.047 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-22T22:50:46.047 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-22T22:50:46.047 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-22T22:50:46.047 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-22T22:50:46.047 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-22T22:50:46.047 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.047 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.047 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-22T22:50:46.047 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-22T22:50:46.047 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-22T22:50:46.048 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.048 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-22T22:50:46.048 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-22T22:50:46.048 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-22T22:50:46.048 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-22T22:50:46.048 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-22T22:50:46.048 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-22T22:50:46.048 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-22T22:50:46.048 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-22T22:50:46.048 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-22T22:50:46.048 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-22T22:50:46.048 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-22T22:50:46.048 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-22T22:50:46.049 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-22T22:50:46.049 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-22T22:50:46.049 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-22T22:50:46.049 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.049 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.049 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:46.049 INFO:teuthology.orchestra.run.smithi204.stdout: re2-1:20211101-3.el9.x86_64 2024-05-22T22:50:46.049 INFO:teuthology.orchestra.run.smithi204.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-22T22:50:46.049 INFO:teuthology.orchestra.run.smithi204.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-22T22:50:46.049 INFO:teuthology.orchestra.run.smithi204.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-22T22:50:46.049 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-22T22:50:46.049 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-22T22:50:46.050 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-22T22:50:46.050 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-22T22:50:46.050 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-05-22T22:50:46.494 DEBUG:teuthology.parallel:result is None 2024-05-22T22:50:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-top-2:19.0.0-2765.g2e1f7622.el9.noarch 155/170 2024-05-22T22:50:46.778 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-22T22:50:46.819 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-22T22:50:46.900 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-22T22:50:46.931 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-22T22:50:46.956 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-22T22:50:46.982 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-22T22:50:47.009 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-22T22:50:47.009 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-22T22:50:47.093 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-22T22:50:47.134 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-22T22:50:47.134 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-22T22:50:47.227 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-22T22:50:47.252 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-22T22:50:47.257 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-22T22:50:47.257 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-22T22:50:47.438 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-22T22:50:47.438 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-22T22:50:47.522 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-22T22:50:47.547 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-22T22:50:47.610 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-22T22:50:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-22T22:50:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-2:19.0.0-2765.g2e1f7622.el9.x86_64 1/170 2024-05-22T22:50:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-base-2:19.0.0-2765.g2e1f7622.el9.x86_64 2/170 2024-05-22T22:50:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-common-2:19.0.0-2765.g2e1f7622.el9.x86_64 3/170 2024-05-22T22:50:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-fuse-2:19.0.0-2765.g2e1f7622.el9.x86_64 4/170 2024-05-22T22:50:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2765.g2e1f7 5/170 2024-05-22T22:50:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mds-2:19.0.0-2765.g2e1f7622.el9.x86_64 6/170 2024-05-22T22:50:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-2:19.0.0-2765.g2e1f7622.el9.x86_64 7/170 2024-05-22T22:50:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mon-2:19.0.0-2765.g2e1f7622.el9.x86_64 8/170 2024-05-22T22:50:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-osd-2:19.0.0-2765.g2e1f7622.el9.x86_64 9/170 2024-05-22T22:50:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-radosgw-2:19.0.0-2765.g2e1f7622.el9.x86_64 10/170 2024-05-22T22:50:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-selinux-2:19.0.0-2765.g2e1f7622.el9.x86_64 11/170 2024-05-22T22:50:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-test-2:19.0.0-2765.g2e1f7622.el9.x86_64 12/170 2024-05-22T22:50:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-mirror-2:19.0.0-2765.g2e1f7622.el9.x86_64 13/170 2024-05-22T22:50:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-shell-2:19.0.0-2765.g2e1f7622.el9.x86_64 14/170 2024-05-22T22:50:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs-devel-2:19.0.0-2765.g2e1f7622.el9.x86_ 15/170 2024-05-22T22:50:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs2-2:19.0.0-2765.g2e1f7622.el9.x86_64 16/170 2024-05-22T22:50:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephsqlite-2:19.0.0-2765.g2e1f7622.el9.x86_64 17/170 2024-05-22T22:50:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados-devel-2:19.0.0-2765.g2e1f7622.el9.x86_6 18/170 2024-05-22T22:50:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libradosstriper1-2:19.0.0-2765.g2e1f7622.el9.x86 19/170 2024-05-22T22:50:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librgw2-2:19.0.0-2765.g2e1f7622.el9.x86_64 20/170 2024-05-22T22:50:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2765.g2e1f7622.el 21/170 2024-05-22T22:50:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-common-2:19.0.0-2765.g2e1f7622.el9. 22/170 2024-05-22T22:50:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cephfs-2:19.0.0-2765.g2e1f7622.el9.x86_6 23/170 2024-05-22T22:50:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rados-2:19.0.0-2765.g2e1f7622.el9.x86_64 24/170 2024-05-22T22:50:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rbd-2:19.0.0-2765.g2e1f7622.el9.x86_64 25/170 2024-05-22T22:50:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rgw-2:19.0.0-2765.g2e1f7622.el9.x86_64 26/170 2024-05-22T22:50:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-fuse-2:19.0.0-2765.g2e1f7622.el9.x86_64 27/170 2024-05-22T22:50:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-mirror-2:19.0.0-2765.g2e1f7622.el9.x86_64 28/170 2024-05-22T22:50:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-nbd-2:19.0.0-2765.g2e1f7622.el9.x86_64 29/170 2024-05-22T22:50:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2765.g2e1f7622. 30/170 2024-05-22T22:50:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2765.g2e1f7622.el9.noa 31/170 2024-05-22T22:50:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2765.g2e1f7622.el9.n 32/170 2024-05-22T22:50:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2765.g2e1 33/170 2024-05-22T22:50:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2765.g2e1f7622.el 34/170 2024-05-22T22:50:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2765.g2e1f7622.el9.noarch 35/170 2024-05-22T22:50:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2765.g2e1f7622.e 36/170 2024-05-22T22:50:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-volume-2:19.0.0-2765.g2e1f7622.el9.noarch 37/170 2024-05-22T22:50:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephadm-2:19.0.0-2765.g2e1f7622.el9.noarch 38/170 2024-05-22T22:50:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-top-2:19.0.0-2765.g2e1f7622.el9.noarch 39/170 2024-05-22T22:50:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-22T22:50:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-22T22:50:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-22T22:50:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-22T22:50:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-22T22:50:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-22T22:50:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-22T22:50:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-22T22:50:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-22T22:50:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-22T22:50:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-22T22:50:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-22T22:50:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-22T22:50:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-22T22:50:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-22T22:50:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-22T22:50:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-22T22:50:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-22T22:50:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-22T22:50:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-22T22:50:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-22T22:50:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-22T22:50:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-22T22:50:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-22T22:50:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-22T22:50:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-22T22:50:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-22T22:50:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-22T22:50:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-22T22:50:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-22T22:50:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-22T22:50:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-22T22:50:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-22T22:50:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-22T22:50:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-22T22:50:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-22T22:50:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-22T22:50:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-22T22:50:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-22T22:50:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-22T22:50:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-22T22:50:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-22T22:50:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-22T22:50:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-22T22:50:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-22T22:50:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-22T22:50:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-22T22:50:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-22T22:50:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-22T22:50:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-22T22:50:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-22T22:50:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-22T22:50:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-22T22:50:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-22T22:50:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-22T22:50:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-22T22:50:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-22T22:50:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-22T22:50:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-22T22:50:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-22T22:50:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-22T22:50:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-22T22:50:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-22T22:50:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-22T22:50:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-22T22:50:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-22T22:50:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-22T22:50:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-22T22:50:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-22T22:50:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-22T22:50:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-22T22:50:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-22T22:50:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-22T22:50:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-22T22:50:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-22T22:50:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-22T22:50:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-22T22:50:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-22T22:50:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-22T22:50:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-22T22:50:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-22T22:50:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-22T22:50:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-22T22:50:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-22T22:50:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-22T22:50:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-22T22:50:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-22T22:50:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-22T22:50:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-22T22:50:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-22T22:50:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-22T22:50:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-22T22:50:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-22T22:50:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-22T22:50:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-22T22:50:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-22T22:50:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-22T22:50:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-22T22:50:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-22T22:50:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-22T22:50:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-22T22:50:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-22T22:50:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-22T22:50:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-22T22:50:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-22T22:50:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-22T22:50:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-22T22:50:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-22T22:50:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-22T22:50:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:19.0.0-2765.g2e1f7622.el9.x86_64 149/170 2024-05-22T22:50:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-22T22:50:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:19.0.0-2765.g2e1f7622.el9.x86_64 151/170 2024-05-22T22:50:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-22T22:50:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-22T22:50:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-22T22:50:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-22T22:50:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-22T22:50:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-22T22:50:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-22T22:50:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-22T22:50:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-22T22:50:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-22T22:50:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-22T22:50:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-22T22:50:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-22T22:50:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-22T22:50:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-22T22:50:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-22T22:50:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-22T22:50:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-22T22:50:55.937 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-22T22:50:55.937 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-22T22:50:55.937 INFO:teuthology.orchestra.run.smithi096.stdout:Upgraded: 2024-05-22T22:50:55.938 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-22T22:50:55.938 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-22T22:50:55.938 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-22T22:50:55.938 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-22T22:50:55.938 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-22T22:50:55.938 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-22T22:50:55.938 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-22T22:50:55.938 INFO:teuthology.orchestra.run.smithi096.stdout: librados2-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.938 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.938 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-22T22:50:55.938 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-22T22:50:55.938 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-05-22T22:50:55.939 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-22T22:50:55.939 INFO:teuthology.orchestra.run.smithi096.stdout: automake-1.16.2-8.el9.noarch 2024-05-22T22:50:55.939 INFO:teuthology.orchestra.run.smithi096.stdout: bison-3.7.4-5.el9.x86_64 2024-05-22T22:50:55.939 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-22T22:50:55.939 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.939 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.939 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.939 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.939 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:55.939 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.939 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.939 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.941 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:55.941 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:55.941 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:55.941 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:55.941 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:55.941 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.941 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.941 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:55.942 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.942 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.942 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.942 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:55.942 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:55.942 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-mirror-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.942 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-shell-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.942 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-top-2:19.0.0-2765.g2e1f7622.el9.noarch 2024-05-22T22:50:55.942 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-22T22:50:55.942 INFO:teuthology.orchestra.run.smithi096.stdout: flex-2.6.4-9.el9.x86_64 2024-05-22T22:50:55.942 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-22T22:50:55.942 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-22T22:50:55.942 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-22T22:50:55.943 INFO:teuthology.orchestra.run.smithi096.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-22T22:50:55.943 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-22T22:50:55.943 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-22T22:50:55.943 INFO:teuthology.orchestra.run.smithi096.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-22T22:50:55.943 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-22T22:50:55.943 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-22T22:50:55.943 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.943 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.943 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.943 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-22T22:50:55.943 INFO:teuthology.orchestra.run.smithi096.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-22T22:50:55.943 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-22T22:50:55.943 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-22T22:50:55.944 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-22T22:50:55.944 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.944 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.944 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-22T22:50:55.944 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.944 INFO:teuthology.orchestra.run.smithi096.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-22T22:50:55.944 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-22T22:50:55.944 INFO:teuthology.orchestra.run.smithi096.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-22T22:50:55.944 INFO:teuthology.orchestra.run.smithi096.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-22T22:50:55.944 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-22T22:50:55.944 INFO:teuthology.orchestra.run.smithi096.stdout: lua-5.4.4-4.el9.x86_64 2024-05-22T22:50:55.944 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-22T22:50:55.945 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-22T22:50:55.945 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-22T22:50:55.945 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-22T22:50:55.945 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-22T22:50:55.945 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-22T22:50:55.945 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-13.14-1.el9.x86_64 2024-05-22T22:50:55.945 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-22T22:50:55.945 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-22T22:50:55.945 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-22T22:50:55.945 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-22T22:50:55.945 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-22T22:50:55.945 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-22T22:50:55.945 INFO:teuthology.orchestra.run.smithi096.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-22T22:50:55.945 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-22T22:50:55.946 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-22T22:50:55.946 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-22T22:50:55.946 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.946 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.946 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.946 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-22T22:50:55.946 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-22T22:50:55.946 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-22T22:50:55.946 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-22T22:50:55.946 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-22T22:50:55.946 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-22T22:50:55.947 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-22T22:50:55.947 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-22T22:50:55.947 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-22T22:50:55.947 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-22T22:50:55.947 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-22T22:50:55.947 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-22T22:50:55.947 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-22T22:50:55.947 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-22T22:50:55.947 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-22T22:50:55.947 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-22T22:50:55.947 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-22T22:50:55.947 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-22T22:50:55.948 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-22T22:50:55.948 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-22T22:50:55.948 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-22T22:50:55.948 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-22T22:50:55.948 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-22T22:50:55.948 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-22T22:50:55.948 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-22T22:50:55.948 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-22T22:50:55.948 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-22T22:50:55.948 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-22T22:50:55.948 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-22T22:50:55.948 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-22T22:50:55.948 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply-3.11-14.el9.noarch 2024-05-22T22:50:55.948 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-22T22:50:55.949 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-22T22:50:55.949 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-22T22:50:55.949 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-22T22:50:55.949 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-22T22:50:55.949 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-22T22:50:55.949 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-22T22:50:55.949 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-22T22:50:55.949 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-22T22:50:55.949 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.949 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.949 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-22T22:50:55.949 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-22T22:50:55.949 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-22T22:50:55.950 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.950 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-22T22:50:55.950 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-22T22:50:55.950 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-22T22:50:55.950 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-22T22:50:55.950 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-22T22:50:55.950 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-22T22:50:55.950 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-22T22:50:55.950 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-22T22:50:55.950 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-22T22:50:55.950 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-22T22:50:55.950 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-22T22:50:55.951 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-22T22:50:55.951 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-22T22:50:55.951 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-22T22:50:55.951 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-22T22:50:55.951 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.951 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.951 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd-2:19.0.0-2765.g2e1f7622.el9.x86_64 2024-05-22T22:50:55.951 INFO:teuthology.orchestra.run.smithi096.stdout: re2-1:20211101-3.el9.x86_64 2024-05-22T22:50:55.951 INFO:teuthology.orchestra.run.smithi096.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-22T22:50:55.951 INFO:teuthology.orchestra.run.smithi096.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-22T22:50:55.951 INFO:teuthology.orchestra.run.smithi096.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-22T22:50:55.951 INFO:teuthology.orchestra.run.smithi096.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-22T22:50:55.951 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-22T22:50:55.951 INFO:teuthology.orchestra.run.smithi096.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-22T22:50:55.951 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-22T22:50:55.952 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-05-22T22:50:56.394 DEBUG:teuthology.parallel:result is None 2024-05-22T22:50:56.395 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2e1f76222e88afe0f1cda37775b2d45d88289794 2024-05-22T22:50:56.610 DEBUG:teuthology.orchestra.run.smithi084:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-22T22:50:56.636 INFO:teuthology.orchestra.run.smithi084.stdout:19.0.0-2765.g2e1f7622.el9 2024-05-22T22:50:56.637 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2765.g2e1f7622.el9 2024-05-22T22:50:56.637 INFO:teuthology.task.install:The correct ceph version 19.0.0-2765.g2e1f7622 is installed. 2024-05-22T22:50:56.639 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2e1f76222e88afe0f1cda37775b2d45d88289794 2024-05-22T22:50:56.784 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-22T22:50:56.805 INFO:teuthology.orchestra.run.smithi096.stdout:19.0.0-2765.g2e1f7622.el9 2024-05-22T22:50:56.806 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2765.g2e1f7622.el9 2024-05-22T22:50:56.806 INFO:teuthology.task.install:The correct ceph version 19.0.0-2765.g2e1f7622 is installed. 2024-05-22T22:50:56.808 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2e1f76222e88afe0f1cda37775b2d45d88289794 2024-05-22T22:50:57.023 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-22T22:50:57.049 INFO:teuthology.orchestra.run.smithi204.stdout:19.0.0-2765.g2e1f7622.el9 2024-05-22T22:50:57.049 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2765.g2e1f7622.el9 2024-05-22T22:50:57.049 INFO:teuthology.task.install:The correct ceph version 19.0.0-2765.g2e1f7622 is installed. 2024-05-22T22:50:57.051 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-22T22:50:57.051 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-22T22:50:57.052 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-22T22:50:57.083 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-22T22:50:57.083 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-22T22:50:57.109 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-22T22:50:57.109 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-22T22:50:57.140 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-22T22:50:57.140 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-22T22:50:57.140 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/daemon-helper 2024-05-22T22:50:57.171 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-22T22:50:57.239 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-22T22:50:57.239 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/daemon-helper 2024-05-22T22:50:57.262 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-22T22:50:57.325 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-22T22:50:57.326 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/daemon-helper 2024-05-22T22:50:57.354 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-22T22:50:57.422 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-22T22:50:57.423 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-22T22:50:57.423 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-22T22:50:57.450 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-22T22:50:57.517 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-22T22:50:57.517 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-22T22:50:57.541 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-22T22:50:57.605 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-22T22:50:57.605 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-22T22:50:57.629 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-22T22:50:57.693 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-22T22:50:57.693 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-22T22:50:57.693 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/stdin-killer 2024-05-22T22:50:57.719 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-22T22:50:57.784 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-22T22:50:57.785 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/stdin-killer 2024-05-22T22:50:57.808 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-22T22:50:57.871 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-22T22:50:57.872 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/stdin-killer 2024-05-22T22:50:57.895 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-22T22:50:57.959 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-22T22:50:58.049 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '2e1f76222e88afe0f1cda37775b2d45d88289794', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-05-22T22:50:58.049 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 2024-05-22T22:50:58.050 INFO:tasks.cephadm:Cluster fsid is c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:50:58.050 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-22T22:50:58.050 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.84', 'mon.b': '172.21.15.96', 'mon.c': '172.21.15.204'} 2024-05-22T22:50:58.050 INFO:tasks.cephadm:First mon is mon.a on smithi084 2024-05-22T22:50:58.050 INFO:tasks.cephadm:First mgr is x 2024-05-22T22:50:58.050 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-22T22:50:58.051 DEBUG:teuthology.orchestra.run.smithi084:> sudo hostname $(hostname -s) 2024-05-22T22:50:58.076 DEBUG:teuthology.orchestra.run.smithi096:> sudo hostname $(hostname -s) 2024-05-22T22:50:58.099 DEBUG:teuthology.orchestra.run.smithi204:> sudo hostname $(hostname -s) 2024-05-22T22:50:58.123 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-22T22:50:58.124 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2e1f76222e88afe0f1cda37775b2d45d88289794 2024-05-22T22:50:58.343 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2e1f76222e88afe0f1cda37775b2d45d88289794', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55202/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2765-g2e1f7622', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2765.g2e1f7622'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/2e1f76222e88afe0f1cda37775b2d45d88289794/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-22 16:50:39.768604', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/2e1f76222e88afe0f1cda37775b2d45d88289794/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-22T22:50:58.477 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 2e1f76222e88afe0f1cda37775b2d45d88289794 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2e1f76222e88afe0f1cda37775b2d45d88289794 2024-05-22T22:50:58.479 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/2e1f76222e88afe0f1cda37775b2d45d88289794/centos/9/x86_64/flavors/default/cephadm 2024-05-22T22:50:58.479 DEBUG:teuthology.orchestra.run.smithi084:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2e1f76222e88afe0f1cda37775b2d45d88289794/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-22T22:51:00.036 INFO:teuthology.orchestra.run.smithi084.stdout:-rw-r--r--. 1 ubuntu ubuntu 781817 May 22 22:51 /home/ubuntu/cephtest/cephadm 2024-05-22T22:51:00.037 DEBUG:teuthology.orchestra.run.smithi096:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2e1f76222e88afe0f1cda37775b2d45d88289794/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-22T22:51:01.557 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-r--r--. 1 ubuntu ubuntu 781817 May 22 22:51 /home/ubuntu/cephtest/cephadm 2024-05-22T22:51:01.557 DEBUG:teuthology.orchestra.run.smithi204:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2e1f76222e88afe0f1cda37775b2d45d88289794/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-22T22:51:02.704 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r--. 1 ubuntu ubuntu 781817 May 22 22:51 /home/ubuntu/cephtest/cephadm 2024-05-22T22:51:02.705 DEBUG:teuthology.orchestra.run.smithi084:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-22T22:51:02.724 DEBUG:teuthology.orchestra.run.smithi096:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-22T22:51:02.743 DEBUG:teuthology.orchestra.run.smithi204:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-22T22:51:02.778 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 on all hosts... 2024-05-22T22:51:02.778 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 pull 2024-05-22T22:51:02.780 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 pull 2024-05-22T22:51:02.786 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 pull 2024-05-22T22:51:03.593 INFO:teuthology.orchestra.run.smithi204.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794... 2024-05-22T22:51:03.606 INFO:teuthology.orchestra.run.smithi084.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794... 2024-05-22T22:51:03.693 INFO:teuthology.orchestra.run.smithi096.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794... 2024-05-22T22:51:23.731 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T22:51:23.732 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 19.0.0-2765-g2e1f7622 (2e1f76222e88afe0f1cda37775b2d45d88289794) squid (rc)", 2024-05-22T22:51:23.732 INFO:teuthology.orchestra.run.smithi084.stdout: "image_id": "f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41", 2024-05-22T22:51:23.732 INFO:teuthology.orchestra.run.smithi084.stdout: "repo_digests": [ 2024-05-22T22:51:23.732 INFO:teuthology.orchestra.run.smithi084.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036" 2024-05-22T22:51:23.732 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-05-22T22:51:23.732 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T22:51:25.751 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-05-22T22:51:25.751 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 19.0.0-2765-g2e1f7622 (2e1f76222e88afe0f1cda37775b2d45d88289794) squid (rc)", 2024-05-22T22:51:25.751 INFO:teuthology.orchestra.run.smithi096.stdout: "image_id": "f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41", 2024-05-22T22:51:25.751 INFO:teuthology.orchestra.run.smithi096.stdout: "repo_digests": [ 2024-05-22T22:51:25.752 INFO:teuthology.orchestra.run.smithi096.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036" 2024-05-22T22:51:25.752 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-05-22T22:51:25.752 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-05-22T22:51:28.618 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-05-22T22:51:28.618 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 19.0.0-2765-g2e1f7622 (2e1f76222e88afe0f1cda37775b2d45d88289794) squid (rc)", 2024-05-22T22:51:28.618 INFO:teuthology.orchestra.run.smithi204.stdout: "image_id": "f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41", 2024-05-22T22:51:28.618 INFO:teuthology.orchestra.run.smithi204.stdout: "repo_digests": [ 2024-05-22T22:51:28.618 INFO:teuthology.orchestra.run.smithi204.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036" 2024-05-22T22:51:28.619 INFO:teuthology.orchestra.run.smithi204.stdout: ] 2024-05-22T22:51:28.619 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-05-22T22:51:28.649 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /etc/ceph 2024-05-22T22:51:28.681 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /etc/ceph 2024-05-22T22:51:28.712 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph 2024-05-22T22:51:28.743 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 777 /etc/ceph 2024-05-22T22:51:28.771 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 777 /etc/ceph 2024-05-22T22:51:28.799 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 777 /etc/ceph 2024-05-22T22:51:28.826 INFO:tasks.cephadm:Writing seed config... 2024-05-22T22:51:28.827 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-22T22:51:28.827 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-22T22:51:28.827 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-22T22:51:28.827 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-22T22:51:28.828 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-22T22:51:28.828 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-22T22:51:28.828 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-22T22:51:28.828 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-22T22:51:28.828 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-22T22:51:28.828 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-22T22:51:28.828 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-22T22:51:28.828 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-22T22:51:28.828 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-22T22:51:28.828 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-22T22:51:28.828 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-22T22:51:28.828 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-22T22:51:28.828 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-22T22:51:28.828 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-22T22:51:28.829 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-22T22:51:28.829 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-22T22:51:28.829 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-22T22:51:28.829 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-22T22:51:28.829 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-22T22:51:28.829 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-22T22:51:28.829 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-22T22:51:28.829 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-22T22:51:28.829 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-22T22:51:28.829 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-22T22:51:28.829 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-22T22:51:28.829 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-22T22:51:28.830 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-22T22:51:28.830 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-22T22:51:28.830 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-22T22:51:28.830 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-22T22:51:28.830 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-22T22:51:28.830 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-22T22:51:28.830 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-22T22:51:28.830 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-22T22:51:28.830 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-22T22:51:28.830 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-22T22:51:28.830 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-22T22:51:28.830 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-22T22:51:28.831 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-22T22:51:28.831 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-22T22:51:28.831 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-22T22:51:28.831 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-22T22:51:28.831 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-22T22:51:28.832 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-22T22:51:28.850 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c1135a12-188d-11ef-bc9a-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-22T22:51:28.850 DEBUG:teuthology.orchestra.run.smithi084:mon.a> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mon.a.service 2024-05-22T22:51:28.892 DEBUG:teuthology.orchestra.run.smithi084:mgr.x> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mgr.x.service 2024-05-22T22:51:28.936 INFO:tasks.cephadm:Bootstrapping... 2024-05-22T22:51:28.936 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 -v bootstrap --fsid c1135a12-188d-11ef-bc9a-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.84 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-22T22:51:29.133 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-05-22T22:51:29.134 INFO:teuthology.orchestra.run.smithi084.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794', '-v', 'bootstrap', '--fsid', 'c1135a12-188d-11ef-bc9a-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.84', '--skip-admin-label'] 2024-05-22T22:51:29.134 INFO:teuthology.orchestra.run.smithi084.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-22T22:51:29.134 INFO:teuthology.orchestra.run.smithi084.stdout:Verifying podman|docker is present... 2024-05-22T22:51:29.160 INFO:teuthology.orchestra.run.smithi084.stdout:/bin/podman: stdout 5.0.2 2024-05-22T22:51:29.160 INFO:teuthology.orchestra.run.smithi084.stdout:Verifying lvm2 is present... 2024-05-22T22:51:29.160 INFO:teuthology.orchestra.run.smithi084.stdout:Verifying time synchronization is in place... 2024-05-22T22:51:29.168 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-22T22:51:29.168 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-22T22:51:29.175 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-22T22:51:29.175 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout inactive 2024-05-22T22:51:29.185 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout enabled 2024-05-22T22:51:29.189 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout active 2024-05-22T22:51:29.189 INFO:teuthology.orchestra.run.smithi084.stdout:Unit chronyd.service is enabled and running 2024-05-22T22:51:29.189 INFO:teuthology.orchestra.run.smithi084.stdout:Repeating the final host check... 2024-05-22T22:51:29.214 INFO:teuthology.orchestra.run.smithi084.stdout:/bin/podman: stdout 5.0.2 2024-05-22T22:51:29.214 INFO:teuthology.orchestra.run.smithi084.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-22T22:51:29.214 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl is present 2024-05-22T22:51:29.214 INFO:teuthology.orchestra.run.smithi084.stdout:lvcreate is present 2024-05-22T22:51:29.221 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-22T22:51:29.221 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-22T22:51:29.227 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-22T22:51:29.228 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout inactive 2024-05-22T22:51:29.235 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout enabled 2024-05-22T22:51:29.242 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout active 2024-05-22T22:51:29.243 INFO:teuthology.orchestra.run.smithi084.stdout:Unit chronyd.service is enabled and running 2024-05-22T22:51:29.243 INFO:teuthology.orchestra.run.smithi084.stdout:Host looks OK 2024-05-22T22:51:29.243 INFO:teuthology.orchestra.run.smithi084.stdout:Cluster fsid: c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:51:29.243 INFO:teuthology.orchestra.run.smithi084.stdout:Acquiring lock 139682574984576 on /run/cephadm/c1135a12-188d-11ef-bc9a-c7b262605968.lock 2024-05-22T22:51:29.243 INFO:teuthology.orchestra.run.smithi084.stdout:Lock 139682574984576 acquired on /run/cephadm/c1135a12-188d-11ef-bc9a-c7b262605968.lock 2024-05-22T22:51:29.243 INFO:teuthology.orchestra.run.smithi084.stdout:Verifying IP 172.21.15.84 port 3300 ... 2024-05-22T22:51:29.243 INFO:teuthology.orchestra.run.smithi084.stdout:Verifying IP 172.21.15.84 port 6789 ... 2024-05-22T22:51:29.244 INFO:teuthology.orchestra.run.smithi084.stdout:Base mon IP(s) is [172.21.15.84:3300, 172.21.15.84:6789], mon addrv is [v2:172.21.15.84:3300,v1:172.21.15.84:6789] 2024-05-22T22:51:29.247 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.84 metric 100 2024-05-22T22:51:29.247 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.84 metric 100 2024-05-22T22:51:29.249 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-22T22:51:29.250 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-05-22T22:51:29.250 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1797sec hoplimit 64 pref medium 2024-05-22T22:51:29.252 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-22T22:51:29.252 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-22T22:51:29.252 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-22T22:51:29.252 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-22T22:51:29.252 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2024-05-22T22:51:29.252 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-22T22:51:29.253 INFO:teuthology.orchestra.run.smithi084.stdout:Mon IP `172.21.15.84` is in CIDR network `172.21.0.0/20` 2024-05-22T22:51:29.253 INFO:teuthology.orchestra.run.smithi084.stdout:Mon IP `172.21.15.84` is in CIDR network `172.21.0.0/20` 2024-05-22T22:51:29.253 INFO:teuthology.orchestra.run.smithi084.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-22T22:51:29.253 INFO:teuthology.orchestra.run.smithi084.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-22T22:51:29.254 INFO:teuthology.orchestra.run.smithi084.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794... 2024-05-22T22:51:31.168 INFO:teuthology.orchestra.run.smithi084.stdout:/bin/podman: stdout f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 2024-05-22T22:51:31.169 INFO:teuthology.orchestra.run.smithi084.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794... 2024-05-22T22:51:31.169 INFO:teuthology.orchestra.run.smithi084.stdout:/bin/podman: stderr Getting image source signatures 2024-05-22T22:51:31.169 INFO:teuthology.orchestra.run.smithi084.stdout:/bin/podman: stderr Copying blob sha256:f82b46892f110bee096f0ba81d48593d085ecabe0b8d4a508f5ad49a8eb9726e 2024-05-22T22:51:31.169 INFO:teuthology.orchestra.run.smithi084.stdout:/bin/podman: stderr Copying blob sha256:6ac36e7c57f227110df5cd65cfdca1cc05e0b17df6748637a6aa04c5c80f89c6 2024-05-22T22:51:31.170 INFO:teuthology.orchestra.run.smithi084.stdout:/bin/podman: stderr Copying config sha256:f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 2024-05-22T22:51:31.170 INFO:teuthology.orchestra.run.smithi084.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-22T22:51:32.105 INFO:teuthology.orchestra.run.smithi084.stdout:ceph: stdout ceph version 19.0.0-2765-g2e1f7622 (2e1f76222e88afe0f1cda37775b2d45d88289794) squid (rc) 2024-05-22T22:51:32.105 INFO:teuthology.orchestra.run.smithi084.stdout:Ceph version: ceph version 19.0.0-2765-g2e1f7622 (2e1f76222e88afe0f1cda37775b2d45d88289794) squid (rc) 2024-05-22T22:51:32.105 INFO:teuthology.orchestra.run.smithi084.stdout:Extracting ceph user uid/gid from container image... 2024-05-22T22:51:32.976 INFO:teuthology.orchestra.run.smithi084.stdout:stat: stdout 167 167 2024-05-22T22:51:32.976 INFO:teuthology.orchestra.run.smithi084.stdout:Creating initial keys... 2024-05-22T22:51:33.941 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph-authtool: stdout AQD1dk5msRA9ERAAF3qw99oPyW53hkVsdz0dFg== 2024-05-22T22:51:41.084 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph-authtool: stdout AQD3dk5mFYmDAxAAl6AeHhJKLU4ZBzuyae/+Zg== 2024-05-22T22:51:45.787 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph-authtool: stdout AQD/dk5mEXgvOBAAd8Qam8IEKZuk7QCTJAoMBw== 2024-05-22T22:51:45.787 INFO:teuthology.orchestra.run.smithi084.stdout:Creating initial monmap... 2024-05-22T22:51:47.576 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-22T22:51:47.576 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-22T22:51:47.576 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:51:47.577 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-22T22:51:47.577 INFO:teuthology.orchestra.run.smithi084.stdout:monmaptool for a [v2:172.21.15.84:3300,v1:172.21.15.84:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-22T22:51:47.577 INFO:teuthology.orchestra.run.smithi084.stdout:setting min_mon_release = quincy 2024-05-22T22:51:47.577 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/monmaptool: set fsid to c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:51:47.577 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-22T22:51:47.577 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:51:47.577 INFO:teuthology.orchestra.run.smithi084.stdout:Creating mon... 2024-05-22T22:51:48.493 INFO:teuthology.orchestra.run.smithi084.stdout:create mon.a on 2024-05-22T22:51:48.693 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-22T22:51:48.831 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-22T22:51:49.019 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c1135a12-188d-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-c1135a12-188d-11ef-bc9a-c7b262605968.target. 2024-05-22T22:51:49.020 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c1135a12-188d-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-c1135a12-188d-11ef-bc9a-c7b262605968.target. 2024-05-22T22:51:49.245 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mon.a 2024-05-22T22:51:49.245 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mon.a.service: Unit ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mon.a.service not loaded. 2024-05-22T22:51:49.409 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c1135a12-188d-11ef-bc9a-c7b262605968.target.wants/ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mon.a.service → /etc/systemd/system/ceph-c1135a12-188d-11ef-bc9a-c7b262605968@.service. 2024-05-22T22:51:49.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:49 smithi084 ceph-mon[31819]: mkfs c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:51:49.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:49 smithi084 ceph-mon[31819]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-22T22:51:49.922 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-22T22:51:49.922 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout disabled 2024-05-22T22:51:49.929 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-22T22:51:49.929 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout inactive 2024-05-22T22:51:49.929 INFO:teuthology.orchestra.run.smithi084.stdout:firewalld.service is not enabled 2024-05-22T22:51:49.929 INFO:teuthology.orchestra.run.smithi084.stdout:Not possible to enable service . firewalld.service is not available 2024-05-22T22:51:49.930 INFO:teuthology.orchestra.run.smithi084.stdout:Waiting for mon to start... 2024-05-22T22:51:49.930 INFO:teuthology.orchestra.run.smithi084.stdout:Waiting for mon... 2024-05-22T22:51:50.158 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:49 smithi084 bash[31800]: 03590bbfefdd5081b9f1abbe7fd5533be03245d94bc7dff89746bd398ef970de 2024-05-22T22:51:50.158 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:49 smithi084 systemd[1]: Started Ceph mon.a for c1135a12-188d-11ef-bc9a-c7b262605968. 2024-05-22T22:51:51.364 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:50 smithi084 ceph-mon[31819]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-22T22:51:51.365 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:50 smithi084 ceph-mon[31819]: monmap epoch 1 2024-05-22T22:51:51.365 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:50 smithi084 ceph-mon[31819]: fsid c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:51:51.365 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:50 smithi084 ceph-mon[31819]: last_changed 2024-05-22T22:51:46.137964+0000 2024-05-22T22:51:51.365 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:50 smithi084 ceph-mon[31819]: created 2024-05-22T22:51:46.137964+0000 2024-05-22T22:51:51.365 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:50 smithi084 ceph-mon[31819]: min_mon_release 19 (squid) 2024-05-22T22:51:51.365 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:50 smithi084 ceph-mon[31819]: election_strategy: 1 2024-05-22T22:51:51.365 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:50 smithi084 ceph-mon[31819]: 0: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.a 2024-05-22T22:51:51.365 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:50 smithi084 ceph-mon[31819]: fsmap 2024-05-22T22:51:51.365 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:50 smithi084 ceph-mon[31819]: osdmap e1: 0 total, 0 up, 0 in 2024-05-22T22:51:51.365 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:50 smithi084 ceph-mon[31819]: mgrmap e1: no daemons active 2024-05-22T22:51:51.365 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:50 smithi084 ceph-mon[31819]: from='client.? 172.21.15.84:0/399023403' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-22T22:51:51.808 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout cluster: 2024-05-22T22:51:51.815 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout id: c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:51:51.815 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-22T22:51:51.815 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-05-22T22:51:51.815 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout services: 2024-05-22T22:51:51.815 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.551906s) 2024-05-22T22:51:51.815 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-22T22:51:51.815 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-22T22:51:51.815 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-05-22T22:51:51.815 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout data: 2024-05-22T22:51:51.816 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-22T22:51:51.816 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-22T22:51:51.816 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-22T22:51:51.816 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout pgs: 2024-05-22T22:51:51.816 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-05-22T22:51:51.816 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.352+0000 7f5002e5d700 1 Processor -- start 2024-05-22T22:51:51.816 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.352+0000 7f5002e5d700 1 -- start start 2024-05-22T22:51:51.816 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.352+0000 7f5002e5d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ffc107e10 0x7f4ffc108230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:51:51.816 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.352+0000 7f5002e5d700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc108800 con 0x7f4ffc107e10 2024-05-22T22:51:51.816 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.353+0000 7f5000bf9700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ffc107e10 0x7f4ffc108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:51:51.816 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.353+0000 7f5000bf9700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ffc107e10 0x7f4ffc108230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53852/0 (socket says 172.21.15.84:53852) 2024-05-22T22:51:51.816 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.353+0000 7f5000bf9700 1 -- 172.21.15.84:0/3330979000 learned_addr learned my addr 172.21.15.84:0/3330979000 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:51:51.817 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.353+0000 7f5000bf9700 1 -- 172.21.15.84:0/3330979000 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ffc108f70 con 0x7f4ffc107e10 2024-05-22T22:51:51.817 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.353+0000 7f5000bf9700 1 --2- 172.21.15.84:0/3330979000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ffc107e10 0x7f4ffc108230 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00098a0 tx=0x7f4ff0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250623f9582f7a40 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:51:51.817 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.353+0000 7f4ffb7fe700 1 -- 172.21.15.84:0/3330979000 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ff0004160 con 0x7f4ffc107e10 2024-05-22T22:51:51.817 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.353+0000 7f4ffb7fe700 1 -- 172.21.15.84:0/3330979000 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ff0004300 con 0x7f4ffc107e10 2024-05-22T22:51:51.817 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.353+0000 7f4ffb7fe700 1 -- 172.21.15.84:0/3330979000 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ff00036a0 con 0x7f4ffc107e10 2024-05-22T22:51:51.817 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.354+0000 7f5002e5d700 1 -- 172.21.15.84:0/3330979000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ffc107e10 msgr2=0x7f4ffc108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:51:51.817 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.354+0000 7f5002e5d700 1 --2- 172.21.15.84:0/3330979000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ffc107e10 0x7f4ffc108230 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00098a0 tx=0x7f4ff0009bb0 comp rx=0 tx=0).stop 2024-05-22T22:51:51.817 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.354+0000 7f5002e5d700 1 -- 172.21.15.84:0/3330979000 shutdown_connections 2024-05-22T22:51:51.817 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.354+0000 7f5002e5d700 1 --2- 172.21.15.84:0/3330979000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ffc107e10 0x7f4ffc108230 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:51:51.817 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.354+0000 7f5002e5d700 1 -- 172.21.15.84:0/3330979000 >> 172.21.15.84:0/3330979000 conn(0x7f4ffc1033f0 msgr2=0x7f4ffc105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:51:51.817 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.354+0000 7f5002e5d700 1 -- 172.21.15.84:0/3330979000 shutdown_connections 2024-05-22T22:51:51.817 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.354+0000 7f5002e5d700 1 -- 172.21.15.84:0/3330979000 wait complete. 2024-05-22T22:51:51.818 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.354+0000 7f5002e5d700 1 Processor -- start 2024-05-22T22:51:51.818 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.354+0000 7f5002e5d700 1 -- start start 2024-05-22T22:51:51.818 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.355+0000 7f5002e5d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ffc107e10 0x7f4ffc19b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:51:51.818 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.355+0000 7f5002e5d700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc1094a0 con 0x7f4ffc107e10 2024-05-22T22:51:51.818 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.355+0000 7f5000bf9700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ffc107e10 0x7f4ffc19b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:51:51.818 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.355+0000 7f5000bf9700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ffc107e10 0x7f4ffc19b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53868/0 (socket says 172.21.15.84:53868) 2024-05-22T22:51:51.818 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.355+0000 7f5000bf9700 1 -- 172.21.15.84:0/399023403 learned_addr learned my addr 172.21.15.84:0/399023403 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:51:51.818 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.355+0000 7f5000bf9700 1 -- 172.21.15.84:0/399023403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ffc19b9b0 con 0x7f4ffc107e10 2024-05-22T22:51:51.818 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.355+0000 7f5000bf9700 1 --2- 172.21.15.84:0/399023403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ffc107e10 0x7f4ffc19b470 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0000c00 tx=0x7f4ff0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:51:51.818 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.355+0000 7f4ff9ffb700 1 -- 172.21.15.84:0/399023403 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ff0003db0 con 0x7f4ffc107e10 2024-05-22T22:51:51.818 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.355+0000 7f4ff9ffb700 1 -- 172.21.15.84:0/399023403 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ff0011630 con 0x7f4ffc107e10 2024-05-22T22:51:51.818 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.355+0000 7f4ff9ffb700 1 -- 172.21.15.84:0/399023403 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ff0011950 con 0x7f4ffc107e10 2024-05-22T22:51:51.818 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.355+0000 7f5002e5d700 1 -- 172.21.15.84:0/399023403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ffc19bc40 con 0x7f4ffc107e10 2024-05-22T22:51:51.819 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.356+0000 7f5002e5d700 1 -- 172.21.15.84:0/399023403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ffc19e930 con 0x7f4ffc107e10 2024-05-22T22:51:51.819 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.356+0000 7f4ff9ffb700 1 -- 172.21.15.84:0/399023403 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4ff0011af0 con 0x7f4ffc107e10 2024-05-22T22:51:51.819 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.356+0000 7f4ff9ffb700 1 -- 172.21.15.84:0/399023403 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4ff001c450 con 0x7f4ffc107e10 2024-05-22T22:51:51.819 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.356+0000 7f5002e5d700 1 -- 172.21.15.84:0/399023403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ffc0639c0 con 0x7f4ffc107e10 2024-05-22T22:51:51.819 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.359+0000 7f4ff9ffb700 1 -- 172.21.15.84:0/399023403 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4ff0032020 con 0x7f4ffc107e10 2024-05-22T22:51:51.819 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.427+0000 7f5002e5d700 1 -- 172.21.15.84:0/399023403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f4ffc02dd40 con 0x7f4ffc107e10 2024-05-22T22:51:51.819 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.428+0000 7f4ff9ffb700 1 -- 172.21.15.84:0/399023403 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f4ff0004260 con 0x7f4ffc107e10 2024-05-22T22:51:51.819 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.428+0000 7f5002e5d700 1 -- 172.21.15.84:0/399023403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ffc107e10 msgr2=0x7f4ffc19b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:51:51.819 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.428+0000 7f5002e5d700 1 --2- 172.21.15.84:0/399023403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ffc107e10 0x7f4ffc19b470 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0000c00 tx=0x7f4ff0004750 comp rx=0 tx=0).stop 2024-05-22T22:51:51.819 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.429+0000 7f5002e5d700 1 -- 172.21.15.84:0/399023403 shutdown_connections 2024-05-22T22:51:51.819 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.429+0000 7f5002e5d700 1 --2- 172.21.15.84:0/399023403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ffc107e10 0x7f4ffc19b470 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:51:51.819 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.429+0000 7f5002e5d700 1 -- 172.21.15.84:0/399023403 >> 172.21.15.84:0/399023403 conn(0x7f4ffc1033f0 msgr2=0x7f4ffc192070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:51:51.820 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.429+0000 7f5002e5d700 1 -- 172.21.15.84:0/399023403 shutdown_connections 2024-05-22T22:51:51.820 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:50.429+0000 7f5002e5d700 1 -- 172.21.15.84:0/399023403 wait complete. 2024-05-22T22:51:51.820 INFO:teuthology.orchestra.run.smithi084.stdout:mon is available 2024-05-22T22:51:51.820 INFO:teuthology.orchestra.run.smithi084.stdout:Assimilating anything we can from ceph.conf... 2024-05-22T22:51:52.614 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:52 smithi084 ceph-mon[31819]: from='client.? 172.21.15.84:0/1665454182' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-22T22:51:52.614 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:52 smithi084 ceph-mon[31819]: from='client.? 172.21.15.84:0/1665454182' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-22T22:51:53.103 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-05-22T22:51:53.103 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout [global] 2024-05-22T22:51:53.103 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout fsid = c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:51:53.104 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.84:3300,v1:172.21.15.84:6789] 2024-05-22T22:51:53.104 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-22T22:51:53.104 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-22T22:51:53.104 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-22T22:51:53.104 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-22T22:51:53.104 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-05-22T22:51:53.104 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-22T22:51:53.104 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-22T22:51:53.104 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-05-22T22:51:53.104 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout [osd] 2024-05-22T22:51:53.104 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-22T22:51:53.104 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-22T22:51:53.105 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-22T22:51:53.105 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-22T22:51:53.105 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-22T22:51:53.105 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.246+0000 7f0d84060700 1 Processor -- start 2024-05-22T22:51:53.105 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.246+0000 7f0d84060700 1 -- start start 2024-05-22T22:51:53.105 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.246+0000 7f0d84060700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d7c107e10 0x7f0d7c108230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:51:53.105 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.246+0000 7f0d84060700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c108800 con 0x7f0d7c107e10 2024-05-22T22:51:53.105 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.247+0000 7f0d81dfc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d7c107e10 0x7f0d7c108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:51:53.105 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.247+0000 7f0d81dfc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d7c107e10 0x7f0d7c108230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58626/0 (socket says 172.21.15.84:58626) 2024-05-22T22:51:53.105 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.247+0000 7f0d81dfc700 1 -- 172.21.15.84:0/967039109 learned_addr learned my addr 172.21.15.84:0/967039109 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:51:53.105 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.247+0000 7f0d81dfc700 1 -- 172.21.15.84:0/967039109 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7c108f70 con 0x7f0d7c107e10 2024-05-22T22:51:53.105 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.247+0000 7f0d81dfc700 1 --2- 172.21.15.84:0/967039109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d7c107e10 0x7f0d7c108230 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0098a0 tx=0x7f0d6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c8633c74b76fb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:51:53.105 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.247+0000 7f0d80dfa700 1 -- 172.21.15.84:0/967039109 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d6c004160 con 0x7f0d7c107e10 2024-05-22T22:51:53.106 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.247+0000 7f0d80dfa700 1 -- 172.21.15.84:0/967039109 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d6c004300 con 0x7f0d7c107e10 2024-05-22T22:51:53.106 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.247+0000 7f0d80dfa700 1 -- 172.21.15.84:0/967039109 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d6c0036a0 con 0x7f0d7c107e10 2024-05-22T22:51:53.106 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.248+0000 7f0d84060700 1 -- 172.21.15.84:0/967039109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d7c107e10 msgr2=0x7f0d7c108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:51:53.106 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.248+0000 7f0d84060700 1 --2- 172.21.15.84:0/967039109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d7c107e10 0x7f0d7c108230 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0098a0 tx=0x7f0d6c009bb0 comp rx=0 tx=0).stop 2024-05-22T22:51:53.106 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.248+0000 7f0d84060700 1 -- 172.21.15.84:0/967039109 shutdown_connections 2024-05-22T22:51:53.106 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.248+0000 7f0d84060700 1 --2- 172.21.15.84:0/967039109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d7c107e10 0x7f0d7c108230 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:51:53.106 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.248+0000 7f0d84060700 1 -- 172.21.15.84:0/967039109 >> 172.21.15.84:0/967039109 conn(0x7f0d7c1033f0 msgr2=0x7f0d7c105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:51:53.106 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.248+0000 7f0d84060700 1 -- 172.21.15.84:0/967039109 shutdown_connections 2024-05-22T22:51:53.106 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.248+0000 7f0d84060700 1 -- 172.21.15.84:0/967039109 wait complete. 2024-05-22T22:51:53.106 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.248+0000 7f0d84060700 1 Processor -- start 2024-05-22T22:51:53.106 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.249+0000 7f0d84060700 1 -- start start 2024-05-22T22:51:53.106 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.249+0000 7f0d84060700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d7c107e10 0x7f0d7c07db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:51:53.107 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.249+0000 7f0d84060700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c1094a0 con 0x7f0d7c107e10 2024-05-22T22:51:53.107 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.249+0000 7f0d81dfc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d7c107e10 0x7f0d7c07db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:51:53.107 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.249+0000 7f0d81dfc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d7c107e10 0x7f0d7c07db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58634/0 (socket says 172.21.15.84:58634) 2024-05-22T22:51:53.107 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.249+0000 7f0d81dfc700 1 -- 172.21.15.84:0/1665454182 learned_addr learned my addr 172.21.15.84:0/1665454182 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:51:53.107 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.249+0000 7f0d81dfc700 1 -- 172.21.15.84:0/1665454182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7c07e0c0 con 0x7f0d7c107e10 2024-05-22T22:51:53.107 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.249+0000 7f0d81dfc700 1 --2- 172.21.15.84:0/1665454182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d7c107e10 0x7f0d7c07db80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c000c00 tx=0x7f0d6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:51:53.107 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.249+0000 7f0d72ffd700 1 -- 172.21.15.84:0/1665454182 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d6c003db0 con 0x7f0d7c107e10 2024-05-22T22:51:53.107 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.249+0000 7f0d72ffd700 1 -- 172.21.15.84:0/1665454182 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d6c011630 con 0x7f0d7c107e10 2024-05-22T22:51:53.107 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.249+0000 7f0d84060700 1 -- 172.21.15.84:0/1665454182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7c07c2b0 con 0x7f0d7c107e10 2024-05-22T22:51:53.107 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.249+0000 7f0d72ffd700 1 -- 172.21.15.84:0/1665454182 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d6c011950 con 0x7f0d7c107e10 2024-05-22T22:51:53.107 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.250+0000 7f0d84060700 1 -- 172.21.15.84:0/1665454182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d7c07c7f0 con 0x7f0d7c107e10 2024-05-22T22:51:53.107 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.250+0000 7f0d72ffd700 1 -- 172.21.15.84:0/1665454182 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0d6c011af0 con 0x7f0d7c107e10 2024-05-22T22:51:53.107 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.250+0000 7f0d72ffd700 1 -- 172.21.15.84:0/1665454182 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0d6c01c790 con 0x7f0d7c107e10 2024-05-22T22:51:53.108 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.251+0000 7f0d84060700 1 -- 172.21.15.84:0/1665454182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d7c0639c0 con 0x7f0d7c107e10 2024-05-22T22:51:53.108 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.253+0000 7f0d72ffd700 1 -- 172.21.15.84:0/1665454182 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0d6c032020 con 0x7f0d7c107e10 2024-05-22T22:51:53.108 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.319+0000 7f0d84060700 1 -- 172.21.15.84:0/1665454182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f0d7c1949c0 con 0x7f0d7c107e10 2024-05-22T22:51:53.108 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.322+0000 7f0d72ffd700 1 -- 172.21.15.84:0/1665454182 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f0d7c1949c0 con 0x7f0d7c107e10 2024-05-22T22:51:53.108 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.322+0000 7f0d72ffd700 1 -- 172.21.15.84:0/1665454182 <== mon.0 v2:172.21.15.84:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0d6c01ce50 con 0x7f0d7c107e10 2024-05-22T22:51:53.108 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.323+0000 7f0d84060700 1 -- 172.21.15.84:0/1665454182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d7c107e10 msgr2=0x7f0d7c07db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:51:53.108 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.323+0000 7f0d84060700 1 --2- 172.21.15.84:0/1665454182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d7c107e10 0x7f0d7c07db80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c000c00 tx=0x7f0d6c004750 comp rx=0 tx=0).stop 2024-05-22T22:51:53.108 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.323+0000 7f0d84060700 1 -- 172.21.15.84:0/1665454182 shutdown_connections 2024-05-22T22:51:53.108 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.323+0000 7f0d84060700 1 --2- 172.21.15.84:0/1665454182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d7c107e10 0x7f0d7c07db80 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:51:53.108 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.323+0000 7f0d84060700 1 -- 172.21.15.84:0/1665454182 >> 172.21.15.84:0/1665454182 conn(0x7f0d7c1033f0 msgr2=0x7f0d7c192070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:51:53.108 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.323+0000 7f0d84060700 1 -- 172.21.15.84:0/1665454182 shutdown_connections 2024-05-22T22:51:53.108 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:52.323+0000 7f0d84060700 1 -- 172.21.15.84:0/1665454182 wait complete. 2024-05-22T22:51:53.108 INFO:teuthology.orchestra.run.smithi084.stdout:Generating new minimal ceph.conf... 2024-05-22T22:51:53.864 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:53 smithi084 ceph-mon[31819]: from='client.? 172.21.15.84:0/1051495613' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:51:54.963 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.505+0000 7fedbe7b7700 1 Processor -- start 2024-05-22T22:51:54.963 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.505+0000 7fedbe7b7700 1 -- start start 2024-05-22T22:51:54.964 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.505+0000 7fedbe7b7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedb81071c0 0x7fedb81075e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:51:54.964 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.505+0000 7fedbe7b7700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fedb8107bb0 con 0x7fedb81071c0 2024-05-22T22:51:54.964 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.505+0000 7fedb7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedb81071c0 0x7fedb81075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:51:54.964 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.505+0000 7fedb7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedb81071c0 0x7fedb81075e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58640/0 (socket says 172.21.15.84:58640) 2024-05-22T22:51:54.964 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.505+0000 7fedb7fff700 1 -- 172.21.15.84:0/3989161376 learned_addr learned my addr 172.21.15.84:0/3989161376 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:51:54.964 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.506+0000 7fedb7fff700 1 -- 172.21.15.84:0/3989161376 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb8107ee0 con 0x7fedb81071c0 2024-05-22T22:51:54.965 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.506+0000 7fedb7fff700 1 --2- 172.21.15.84:0/3989161376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedb81071c0 0x7fedb81075e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feda80098a0 tx=0x7feda8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e411c0df055978 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:51:54.965 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.506+0000 7fedb6ffd700 1 -- 172.21.15.84:0/3989161376 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feda8004160 con 0x7fedb81071c0 2024-05-22T22:51:54.965 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.506+0000 7fedb6ffd700 1 -- 172.21.15.84:0/3989161376 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feda8004300 con 0x7fedb81071c0 2024-05-22T22:51:54.965 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.506+0000 7fedb6ffd700 1 -- 172.21.15.84:0/3989161376 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feda8011b50 con 0x7fedb81071c0 2024-05-22T22:51:54.965 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.506+0000 7fedbe7b7700 1 -- 172.21.15.84:0/3989161376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedb81071c0 msgr2=0x7fedb81075e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:51:54.966 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.507+0000 7fedbe7b7700 1 --2- 172.21.15.84:0/3989161376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedb81071c0 0x7fedb81075e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feda80098a0 tx=0x7feda8009bb0 comp rx=0 tx=0).stop 2024-05-22T22:51:54.966 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.507+0000 7fedbe7b7700 1 -- 172.21.15.84:0/3989161376 shutdown_connections 2024-05-22T22:51:54.966 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.507+0000 7fedbe7b7700 1 --2- 172.21.15.84:0/3989161376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedb81071c0 0x7fedb81075e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:51:54.966 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.507+0000 7fedbe7b7700 1 -- 172.21.15.84:0/3989161376 >> 172.21.15.84:0/3989161376 conn(0x7fedb8101a40 msgr2=0x7fedb8103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:51:54.966 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.507+0000 7fedbe7b7700 1 -- 172.21.15.84:0/3989161376 shutdown_connections 2024-05-22T22:51:54.967 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.507+0000 7fedbe7b7700 1 -- 172.21.15.84:0/3989161376 wait complete. 2024-05-22T22:51:54.967 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.507+0000 7fedbe7b7700 1 Processor -- start 2024-05-22T22:51:54.967 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.507+0000 7fedbe7b7700 1 -- start start 2024-05-22T22:51:54.967 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.508+0000 7fedbe7b7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedb81071c0 0x7fedb819bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:51:54.967 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.508+0000 7fedbe7b7700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fedb810b290 con 0x7fedb81071c0 2024-05-22T22:51:54.967 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.508+0000 7fedb7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedb81071c0 0x7fedb819bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:51:54.968 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.508+0000 7fedb7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedb81071c0 0x7fedb819bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58652/0 (socket says 172.21.15.84:58652) 2024-05-22T22:51:54.968 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.508+0000 7fedb7fff700 1 -- 172.21.15.84:0/1051495613 learned_addr learned my addr 172.21.15.84:0/1051495613 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:51:54.968 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.508+0000 7fedb7fff700 1 -- 172.21.15.84:0/1051495613 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb819c270 con 0x7fedb81071c0 2024-05-22T22:51:54.968 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.508+0000 7fedb7fff700 1 --2- 172.21.15.84:0/1051495613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedb81071c0 0x7fedb819bd30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feda8011670 tx=0x7feda8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:51:54.968 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.508+0000 7fedb57fa700 1 -- 172.21.15.84:0/1051495613 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feda80039c0 con 0x7fedb81071c0 2024-05-22T22:51:54.969 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.508+0000 7fedb57fa700 1 -- 172.21.15.84:0/1051495613 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feda8011e50 con 0x7fedb81071c0 2024-05-22T22:51:54.969 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.508+0000 7fedbe7b7700 1 -- 172.21.15.84:0/1051495613 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb819c500 con 0x7fedb81071c0 2024-05-22T22:51:54.969 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.508+0000 7fedb57fa700 1 -- 172.21.15.84:0/1051495613 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feda801a5c0 con 0x7fedb81071c0 2024-05-22T22:51:54.969 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.508+0000 7fedbe7b7700 1 -- 172.21.15.84:0/1051495613 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb819f1f0 con 0x7fedb81071c0 2024-05-22T22:51:54.969 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.509+0000 7fedb57fa700 1 -- 172.21.15.84:0/1051495613 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feda801ac00 con 0x7fedb81071c0 2024-05-22T22:51:54.969 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.509+0000 7fedb57fa700 1 -- 172.21.15.84:0/1051495613 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7feda8019700 con 0x7fedb81071c0 2024-05-22T22:51:54.970 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.510+0000 7fedbe7b7700 1 -- 172.21.15.84:0/1051495613 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedb80639c0 con 0x7fedb81071c0 2024-05-22T22:51:54.970 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.512+0000 7fedb57fa700 1 -- 172.21.15.84:0/1051495613 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7feda8003b60 con 0x7fedb81071c0 2024-05-22T22:51:54.970 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.578+0000 7fedbe7b7700 1 -- 172.21.15.84:0/1051495613 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fedb802deb0 con 0x7fedb81071c0 2024-05-22T22:51:54.970 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.579+0000 7fedb57fa700 1 -- 172.21.15.84:0/1051495613 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7feda80199a0 con 0x7fedb81071c0 2024-05-22T22:51:54.970 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.580+0000 7fedbe7b7700 1 -- 172.21.15.84:0/1051495613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedb81071c0 msgr2=0x7fedb819bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:51:54.970 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.580+0000 7fedbe7b7700 1 --2- 172.21.15.84:0/1051495613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedb81071c0 0x7fedb819bd30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feda8011670 tx=0x7feda8011b60 comp rx=0 tx=0).stop 2024-05-22T22:51:54.970 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.580+0000 7fedbe7b7700 1 -- 172.21.15.84:0/1051495613 shutdown_connections 2024-05-22T22:51:54.970 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.580+0000 7fedbe7b7700 1 --2- 172.21.15.84:0/1051495613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedb81071c0 0x7fedb819bd30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:51:54.970 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.580+0000 7fedbe7b7700 1 -- 172.21.15.84:0/1051495613 >> 172.21.15.84:0/1051495613 conn(0x7fedb8101a40 msgr2=0x7fedb8102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:51:54.971 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.580+0000 7fedbe7b7700 1 -- 172.21.15.84:0/1051495613 shutdown_connections 2024-05-22T22:51:54.971 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:53.580+0000 7fedbe7b7700 1 -- 172.21.15.84:0/1051495613 wait complete. 2024-05-22T22:51:54.971 INFO:teuthology.orchestra.run.smithi084.stdout:Restarting the monitor... 2024-05-22T22:51:55.294 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:54 smithi084 systemd[1]: Stopping Ceph mon.a for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T22:51:55.294 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:55 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-a[31815]: 2024-05-22T22:51:55.231+0000 7f6864dcd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T22:51:55.294 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:55 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-a[31815]: 2024-05-22T22:51:55.231+0000 7f6864dcd700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-22T22:51:55.571 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:55 smithi084 podman[32063]: 2024-05-22 22:51:55.30316499 +0000 UTC m=+0.155720757 container died 03590bbfefdd5081b9f1abbe7fd5533be03245d94bc7dff89746bd398ef970de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-2e1f762, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-22T22:51:55.864 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:55 smithi084 podman[32063]: 2024-05-22 22:51:55.672672391 +0000 UTC m=+0.525228157 container cleanup 03590bbfefdd5081b9f1abbe7fd5533be03245d94bc7dff89746bd398ef970de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-2e1f762, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-22T22:51:55.864 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:55 smithi084 bash[32063]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-a 2024-05-22T22:51:56.225 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:55 smithi084 podman[32076]: 2024-05-22 22:51:55.890168992 +0000 UTC m=+0.588683928 container remove 03590bbfefdd5081b9f1abbe7fd5533be03245d94bc7dff89746bd398ef970de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-2e1f762, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-22T22:51:56.225 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 systemd[1]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-22T22:51:56.225 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 systemd[1]: Stopped Ceph mon.a for c1135a12-188d-11ef-bc9a-c7b262605968. 2024-05-22T22:51:56.225 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 systemd[1]: Starting Ceph mon.a for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T22:51:56.614 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 podman[32168]: 2024-05-22 22:51:56.137824722 +0000 UTC m=+0.016937475 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 2024-05-22T22:51:56.873 INFO:teuthology.orchestra.run.smithi084.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-22T22:51:56.998 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 podman[32168]: 2024-05-22 22:51:56.652041436 +0000 UTC m=+0.531154177 container create 16db2f34d801f4e3ada4ff7250f589fc39df5c0b7b5c46890567978aabd8656f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, RELEASE=squid-2e1f762, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-22T22:51:56.998 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 podman[32168]: 2024-05-22 22:51:56.804362166 +0000 UTC m=+0.683474909 container init 16db2f34d801f4e3ada4ff7250f589fc39df5c0b7b5c46890567978aabd8656f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-2e1f762, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-22T22:51:56.998 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 podman[32168]: 2024-05-22 22:51:56.807489 +0000 UTC m=+0.686601743 container start 16db2f34d801f4e3ada4ff7250f589fc39df5c0b7b5c46890567978aabd8656f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-a, RELEASE=squid-2e1f762, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-22T22:51:56.998 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: set uid:gid to 167:167 (ceph:ceph) 2024-05-22T22:51:56.998 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: ceph version 19.0.0-2765-g2e1f7622 (2e1f76222e88afe0f1cda37775b2d45d88289794) squid (rc), process ceph-mon, pid 2 2024-05-22T22:51:56.998 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: pidfile_write: ignore empty --pid-file 2024-05-22T22:51:56.998 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: load: jerasure load: lrc 2024-05-22T22:51:56.998 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: RocksDB version: 7.9.2 2024-05-22T22:51:56.998 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Git sha 0 2024-05-22T22:51:56.999 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Compile date 2024-05-22 16:19:42 2024-05-22T22:51:56.999 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: DB SUMMARY 2024-05-22T22:51:56.999 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: DB Session ID: 0IUXDX6KU9VU2JXC2E4B 2024-05-22T22:51:56.999 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: CURRENT file: CURRENT 2024-05-22T22:51:56.999 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: IDENTITY file: IDENTITY 2024-05-22T22:51:56.999 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-22T22:51:56.999 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-22T22:51:56.999 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104720 ; 2024-05-22T22:51:56.999 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.error_if_exists: 0 2024-05-22T22:51:56.999 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.create_if_missing: 0 2024-05-22T22:51:56.999 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.paranoid_checks: 1 2024-05-22T22:51:56.999 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-22T22:51:57.001 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-22T22:51:57.001 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-22T22:51:57.001 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.env: 0x55aefdf0bc60 2024-05-22T22:51:57.001 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.fs: PosixFileSystem 2024-05-22T22:51:57.001 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.info_log: 0x55af000d9ae0 2024-05-22T22:51:57.001 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_file_opening_threads: 16 2024-05-22T22:51:57.001 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.statistics: (nil) 2024-05-22T22:51:57.001 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.use_fsync: 0 2024-05-22T22:51:57.001 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_log_file_size: 0 2024-05-22T22:51:57.001 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-22T22:51:57.002 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-22T22:51:57.002 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.keep_log_file_num: 1000 2024-05-22T22:51:57.002 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.recycle_log_file_num: 0 2024-05-22T22:51:57.002 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.allow_fallocate: 1 2024-05-22T22:51:57.002 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.allow_mmap_reads: 0 2024-05-22T22:51:57.002 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.allow_mmap_writes: 0 2024-05-22T22:51:57.002 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.use_direct_reads: 0 2024-05-22T22:51:57.002 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-22T22:51:57.002 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.create_missing_column_families: 0 2024-05-22T22:51:57.002 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.db_log_dir: 2024-05-22T22:51:57.002 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.wal_dir: 2024-05-22T22:51:57.002 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-22T22:51:57.002 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-22T22:51:57.003 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-22T22:51:57.003 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-22T22:51:57.003 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-22T22:51:57.003 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-22T22:51:57.003 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.advise_random_on_open: 1 2024-05-22T22:51:57.003 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.db_write_buffer_size: 0 2024-05-22T22:51:57.003 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.write_buffer_manager: 0x55af000aa1e0 2024-05-22T22:51:57.003 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-22T22:51:57.003 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-22T22:51:57.003 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-22T22:51:57.003 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.rate_limiter: (nil) 2024-05-22T22:51:57.003 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-22T22:51:57.003 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.wal_recovery_mode: 2 2024-05-22T22:51:57.004 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.enable_thread_tracking: 0 2024-05-22T22:51:57.004 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.enable_pipelined_write: 0 2024-05-22T22:51:57.004 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.unordered_write: 0 2024-05-22T22:51:57.004 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-22T22:51:57.004 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-22T22:51:57.004 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-22T22:51:57.004 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-22T22:51:57.004 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.row_cache: None 2024-05-22T22:51:57.004 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.wal_filter: None 2024-05-22T22:51:57.004 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-22T22:51:57.004 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.allow_ingest_behind: 0 2024-05-22T22:51:57.004 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.two_write_queues: 0 2024-05-22T22:51:57.004 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.manual_wal_flush: 0 2024-05-22T22:51:57.005 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.wal_compression: 0 2024-05-22T22:51:57.005 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.atomic_flush: 0 2024-05-22T22:51:57.005 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-22T22:51:57.005 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-22T22:51:57.005 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-22T22:51:57.005 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.log_readahead_size: 0 2024-05-22T22:51:57.005 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-22T22:51:57.005 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.best_efforts_recovery: 0 2024-05-22T22:51:57.005 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-22T22:51:57.005 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-22T22:51:57.005 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.allow_data_in_errors: 0 2024-05-22T22:51:57.005 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.db_host_id: __hostname__ 2024-05-22T22:51:57.005 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-22T22:51:57.005 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_background_jobs: 2 2024-05-22T22:51:57.006 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_background_compactions: -1 2024-05-22T22:51:57.006 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_subcompactions: 1 2024-05-22T22:51:57.006 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-22T22:51:57.006 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-22T22:51:57.006 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-22T22:51:57.006 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_total_wal_size: 0 2024-05-22T22:51:57.006 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-22T22:51:57.006 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-22T22:51:57.006 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-22T22:51:57.006 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-22T22:51:57.006 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_open_files: -1 2024-05-22T22:51:57.006 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.bytes_per_sync: 0 2024-05-22T22:51:57.006 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-22T22:51:57.006 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-22T22:51:57.007 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.compaction_readahead_size: 0 2024-05-22T22:51:57.007 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_background_flushes: -1 2024-05-22T22:51:57.007 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Compression algorithms supported: 2024-05-22T22:51:57.007 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-22T22:51:57.007 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: kZSTD supported: 0 2024-05-22T22:51:57.007 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: kXpressCompression supported: 0 2024-05-22T22:51:57.007 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: kLZ4HCCompression supported: 1 2024-05-22T22:51:57.007 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: kZlibCompression supported: 1 2024-05-22T22:51:57.007 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: kSnappyCompression supported: 1 2024-05-22T22:51:57.007 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: kLZ4Compression supported: 1 2024-05-22T22:51:57.007 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: kBZip2Compression supported: 0 2024-05-22T22:51:57.007 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-22T22:51:57.007 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-22T22:51:57.007 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-22T22:51:57.008 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-22T22:51:57.008 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-22T22:51:57.008 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.merge_operator: 2024-05-22T22:51:57.008 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.compaction_filter: None 2024-05-22T22:51:57.008 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.compaction_filter_factory: None 2024-05-22T22:51:57.008 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.sst_partitioner_factory: None 2024-05-22T22:51:57.008 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-22T22:51:57.008 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-22T22:51:57.008 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55af000d9be0) 2024-05-22T22:51:57.008 INFO:journalctl@ceph.mon.a.smithi084.stdout: cache_index_and_filter_blocks: 1 2024-05-22T22:51:57.008 INFO:journalctl@ceph.mon.a.smithi084.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-22T22:51:57.008 INFO:journalctl@ceph.mon.a.smithi084.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-22T22:51:57.008 INFO:journalctl@ceph.mon.a.smithi084.stdout: pin_top_level_index_and_filter: 1 2024-05-22T22:51:57.008 INFO:journalctl@ceph.mon.a.smithi084.stdout: index_type: 0 2024-05-22T22:51:57.008 INFO:journalctl@ceph.mon.a.smithi084.stdout: data_block_index_type: 0 2024-05-22T22:51:57.009 INFO:journalctl@ceph.mon.a.smithi084.stdout: index_shortening: 1 2024-05-22T22:51:57.009 INFO:journalctl@ceph.mon.a.smithi084.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-22T22:51:57.009 INFO:journalctl@ceph.mon.a.smithi084.stdout: checksum: 4 2024-05-22T22:51:57.009 INFO:journalctl@ceph.mon.a.smithi084.stdout: no_block_cache: 0 2024-05-22T22:51:57.009 INFO:journalctl@ceph.mon.a.smithi084.stdout: block_cache: 0x55af000db090 2024-05-22T22:51:57.009 INFO:journalctl@ceph.mon.a.smithi084.stdout: block_cache_name: BinnedLRUCache 2024-05-22T22:51:57.009 INFO:journalctl@ceph.mon.a.smithi084.stdout: block_cache_options: 2024-05-22T22:51:57.009 INFO:journalctl@ceph.mon.a.smithi084.stdout: capacity : 536870912 2024-05-22T22:51:57.009 INFO:journalctl@ceph.mon.a.smithi084.stdout: num_shard_bits : 4 2024-05-22T22:51:57.009 INFO:journalctl@ceph.mon.a.smithi084.stdout: strict_capacity_limit : 0 2024-05-22T22:51:57.009 INFO:journalctl@ceph.mon.a.smithi084.stdout: high_pri_pool_ratio: 0.000 2024-05-22T22:51:57.009 INFO:journalctl@ceph.mon.a.smithi084.stdout: block_cache_compressed: (nil) 2024-05-22T22:51:57.009 INFO:journalctl@ceph.mon.a.smithi084.stdout: persistent_cache: (nil) 2024-05-22T22:51:57.009 INFO:journalctl@ceph.mon.a.smithi084.stdout: block_size: 4096 2024-05-22T22:51:57.010 INFO:journalctl@ceph.mon.a.smithi084.stdout: block_size_deviation: 10 2024-05-22T22:51:57.010 INFO:journalctl@ceph.mon.a.smithi084.stdout: block_restart_interval: 16 2024-05-22T22:51:57.010 INFO:journalctl@ceph.mon.a.smithi084.stdout: index_block_restart_interval: 1 2024-05-22T22:51:57.010 INFO:journalctl@ceph.mon.a.smithi084.stdout: metadata_block_size: 4096 2024-05-22T22:51:57.010 INFO:journalctl@ceph.mon.a.smithi084.stdout: partition_filters: 0 2024-05-22T22:51:57.010 INFO:journalctl@ceph.mon.a.smithi084.stdout: use_delta_encoding: 1 2024-05-22T22:51:57.010 INFO:journalctl@ceph.mon.a.smithi084.stdout: filter_policy: bloomfilter 2024-05-22T22:51:57.010 INFO:journalctl@ceph.mon.a.smithi084.stdout: whole_key_filtering: 1 2024-05-22T22:51:57.010 INFO:journalctl@ceph.mon.a.smithi084.stdout: verify_compression: 0 2024-05-22T22:51:57.010 INFO:journalctl@ceph.mon.a.smithi084.stdout: read_amp_bytes_per_bit: 0 2024-05-22T22:51:57.010 INFO:journalctl@ceph.mon.a.smithi084.stdout: format_version: 5 2024-05-22T22:51:57.010 INFO:journalctl@ceph.mon.a.smithi084.stdout: enable_index_compression: 1 2024-05-22T22:51:57.010 INFO:journalctl@ceph.mon.a.smithi084.stdout: block_align: 0 2024-05-22T22:51:57.011 INFO:journalctl@ceph.mon.a.smithi084.stdout: max_auto_readahead_size: 262144 2024-05-22T22:51:57.011 INFO:journalctl@ceph.mon.a.smithi084.stdout: prepopulate_block_cache: 0 2024-05-22T22:51:57.011 INFO:journalctl@ceph.mon.a.smithi084.stdout: initial_auto_readahead_size: 8192 2024-05-22T22:51:57.011 INFO:journalctl@ceph.mon.a.smithi084.stdout: num_file_reads_for_auto_readahead: 2 2024-05-22T22:51:57.011 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.write_buffer_size: 33554432 2024-05-22T22:51:57.011 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_write_buffer_number: 2 2024-05-22T22:51:57.011 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.compression: NoCompression 2024-05-22T22:51:57.011 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.bottommost_compression: Disabled 2024-05-22T22:51:57.011 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.prefix_extractor: nullptr 2024-05-22T22:51:57.011 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-22T22:51:57.011 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.num_levels: 7 2024-05-22T22:51:57.011 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-22T22:51:57.011 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-22T22:51:57.012 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-22T22:51:57.012 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-22T22:51:57.012 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-22T22:51:57.012 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-22T22:51:57.012 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-22T22:51:57.012 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-22T22:51:57.012 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-22T22:51:57.012 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-22T22:51:57.012 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-22T22:51:57.012 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-22T22:51:57.012 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-22T22:51:57.012 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.compression_opts.level: 32767 2024-05-22T22:51:57.013 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.compression_opts.strategy: 0 2024-05-22T22:51:57.013 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-22T22:51:57.013 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-22T22:51:57.013 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-22T22:51:57.013 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-22T22:51:57.013 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.compression_opts.enabled: false 2024-05-22T22:51:57.013 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-22T22:51:57.013 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-22T22:51:57.013 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-22T22:51:57.013 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-22T22:51:57.013 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.target_file_size_base: 67108864 2024-05-22T22:51:57.013 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-22T22:51:57.013 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-22T22:51:57.014 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-22T22:51:57.014 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-22T22:51:57.014 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-22T22:51:57.014 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-22T22:51:57.014 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-22T22:51:57.014 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-22T22:51:57.014 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-22T22:51:57.014 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-22T22:51:57.014 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-22T22:51:57.014 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-22T22:51:57.014 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-22T22:51:57.014 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-22T22:51:57.014 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.arena_block_size: 1048576 2024-05-22T22:51:57.015 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-22T22:51:57.015 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-22T22:51:57.015 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.disable_auto_compactions: 0 2024-05-22T22:51:57.015 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-22T22:51:57.015 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-22T22:51:57.015 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-22T22:51:57.015 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-22T22:51:57.015 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-22T22:51:57.015 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-22T22:51:57.015 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-22T22:51:57.015 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-22T22:51:57.015 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-22T22:51:57.015 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-22T22:51:57.015 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.table_properties_collectors: 2024-05-22T22:51:57.016 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.inplace_update_support: 0 2024-05-22T22:51:57.016 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-22T22:51:57.016 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-22T22:51:57.016 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-22T22:51:57.016 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-22T22:51:57.016 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.bloom_locality: 0 2024-05-22T22:51:57.016 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.max_successive_merges: 0 2024-05-22T22:51:57.016 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-22T22:51:57.016 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.paranoid_file_checks: 0 2024-05-22T22:51:57.016 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.force_consistency_checks: 1 2024-05-22T22:51:57.016 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.report_bg_io_stats: 0 2024-05-22T22:51:57.016 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.ttl: 2592000 2024-05-22T22:51:57.016 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-22T22:51:57.016 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-22T22:51:57.016 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-22T22:51:57.017 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.enable_blob_files: false 2024-05-22T22:51:57.017 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.min_blob_size: 0 2024-05-22T22:51:57.017 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.blob_file_size: 268435456 2024-05-22T22:51:57.017 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-22T22:51:57.017 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-22T22:51:57.017 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-22T22:51:57.017 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-22T22:51:57.017 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-22T22:51:57.017 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.blob_file_starting_level: 0 2024-05-22T22:51:57.017 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-22T22:51:57.017 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-22T22:51:57.017 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-22T22:51:57.017 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2f4ae0ed-129b-47b5-9d02-44882e631e82 2024-05-22T22:51:57.017 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716418316838179, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-22T22:51:57.018 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-22T22:51:57.018 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716418316839179, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99809, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97721, "index_size": 306, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14165, "raw_average_key_size": 49, "raw_value_size": 90060, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716418316, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2f4ae0ed-129b-47b5-9d02-44882e631e82", "db_session_id": "0IUXDX6KU9VU2JXC2E4B", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-22T22:51:57.018 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716418316839268, "job": 1, "event": "recovery_finished"} 2024-05-22T22:51:57.018 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-22T22:51:57.018 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-22T22:51:57.018 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55af001ca000 2024-05-22T22:51:57.018 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: DB pointer 0x55af001b4000 2024-05-22T22:51:57.018 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-22T22:51:57.018 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-22T22:51:57.018 INFO:journalctl@ceph.mon.a.smithi084.stdout: ** DB Stats ** 2024-05-22T22:51:57.018 INFO:journalctl@ceph.mon.a.smithi084.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-22T22:51:57.018 INFO:journalctl@ceph.mon.a.smithi084.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-22T22:51:57.019 INFO:journalctl@ceph.mon.a.smithi084.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-22T22:51:57.019 INFO:journalctl@ceph.mon.a.smithi084.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-22T22:51:57.019 INFO:journalctl@ceph.mon.a.smithi084.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-22T22:51:57.019 INFO:journalctl@ceph.mon.a.smithi084.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-22T22:51:57.019 INFO:journalctl@ceph.mon.a.smithi084.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-22T22:51:57.019 INFO:journalctl@ceph.mon.a.smithi084.stdout: 2024-05-22T22:51:57.019 INFO:journalctl@ceph.mon.a.smithi084.stdout: ** Compaction Stats [default] ** 2024-05-22T22:51:57.019 INFO:journalctl@ceph.mon.a.smithi084.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-22T22:51:57.019 INFO:journalctl@ceph.mon.a.smithi084.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-22T22:51:57.019 INFO:journalctl@ceph.mon.a.smithi084.stdout: L0 2/0 99.30 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 154.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-22T22:51:57.019 INFO:journalctl@ceph.mon.a.smithi084.stdout: Sum 2/0 99.30 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 154.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-22T22:51:57.019 INFO:journalctl@ceph.mon.a.smithi084.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 154.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-22T22:51:57.019 INFO:journalctl@ceph.mon.a.smithi084.stdout: 2024-05-22T22:51:57.020 INFO:journalctl@ceph.mon.a.smithi084.stdout: ** Compaction Stats [default] ** 2024-05-22T22:51:57.020 INFO:journalctl@ceph.mon.a.smithi084.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-22T22:51:57.020 INFO:journalctl@ceph.mon.a.smithi084.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-22T22:51:57.020 INFO:journalctl@ceph.mon.a.smithi084.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 154.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-22T22:51:57.020 INFO:journalctl@ceph.mon.a.smithi084.stdout: 2024-05-22T22:51:57.020 INFO:journalctl@ceph.mon.a.smithi084.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-22T22:51:57.020 INFO:journalctl@ceph.mon.a.smithi084.stdout: 2024-05-22T22:51:57.020 INFO:journalctl@ceph.mon.a.smithi084.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-22T22:51:57.020 INFO:journalctl@ceph.mon.a.smithi084.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-22T22:51:57.020 INFO:journalctl@ceph.mon.a.smithi084.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-22T22:51:57.020 INFO:journalctl@ceph.mon.a.smithi084.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-22T22:51:57.020 INFO:journalctl@ceph.mon.a.smithi084.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-22T22:51:57.020 INFO:journalctl@ceph.mon.a.smithi084.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-22T22:51:57.020 INFO:journalctl@ceph.mon.a.smithi084.stdout: Cumulative compaction: 0.00 GB write, 31.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-22T22:51:57.020 INFO:journalctl@ceph.mon.a.smithi084.stdout: Interval compaction: 0.00 GB write, 31.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-22T22:51:57.021 INFO:journalctl@ceph.mon.a.smithi084.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-22T22:51:57.021 INFO:journalctl@ceph.mon.a.smithi084.stdout: Block cache BinnedLRUCache@0x55af000db090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-22T22:51:57.021 INFO:journalctl@ceph.mon.a.smithi084.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-22T22:51:57.021 INFO:journalctl@ceph.mon.a.smithi084.stdout: 2024-05-22T22:51:57.021 INFO:journalctl@ceph.mon.a.smithi084.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-22T22:51:57.021 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: starting mon.a rank 0 at public addrs [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] at bind addrs [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:51:57.021 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: mon.a@-1(???) e1 preinit fsid c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:51:57.021 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: mon.a@-1(???).mds e1 new map 2024-05-22T22:51:57.021 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: mon.a@-1(???).mds e1 print_map 2024-05-22T22:51:57.021 INFO:journalctl@ceph.mon.a.smithi084.stdout: e1 2024-05-22T22:51:57.021 INFO:journalctl@ceph.mon.a.smithi084.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-22T22:51:57.021 INFO:journalctl@ceph.mon.a.smithi084.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-22T22:51:57.021 INFO:journalctl@ceph.mon.a.smithi084.stdout: legacy client fscid: -1 2024-05-22T22:51:57.021 INFO:journalctl@ceph.mon.a.smithi084.stdout: 2024-05-22T22:51:57.022 INFO:journalctl@ceph.mon.a.smithi084.stdout: No filesystems configured 2024-05-22T22:51:57.022 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-22T22:51:57.022 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-22T22:51:57.022 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-22T22:51:57.022 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-22T22:51:57.022 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-22T22:51:57.022 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 bash[32168]: 16db2f34d801f4e3ada4ff7250f589fc39df5c0b7b5c46890567978aabd8656f 2024-05-22T22:51:57.022 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 systemd[1]: Started Ceph mon.a for c1135a12-188d-11ef-bc9a-c7b262605968. 2024-05-22T22:51:57.022 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-22T22:51:57.022 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: monmap epoch 1 2024-05-22T22:51:57.022 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: fsid c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:51:57.022 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: last_changed 2024-05-22T22:51:46.137964+0000 2024-05-22T22:51:57.022 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: created 2024-05-22T22:51:46.137964+0000 2024-05-22T22:51:57.022 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: min_mon_release 19 (squid) 2024-05-22T22:51:57.023 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: election_strategy: 1 2024-05-22T22:51:57.023 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: 0: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.a 2024-05-22T22:51:57.023 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: fsmap 2024-05-22T22:51:57.023 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: osdmap e1: 0 total, 0 up, 0 in 2024-05-22T22:51:57.023 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:56 smithi084 ceph-mon[32186]: mgrmap e1: no daemons active 2024-05-22T22:51:58.020 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.377+0000 7f58de905700 1 Processor -- start 2024-05-22T22:51:58.024 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.377+0000 7f58de905700 1 -- start start 2024-05-22T22:51:58.024 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.377+0000 7f58de905700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d807c330 0x7f58d807c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:51:58.024 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.377+0000 7f58de905700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f58d807dfc0 con 0x7f58d807c330 2024-05-22T22:51:58.024 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.377+0000 7f58d7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d807c330 0x7f58d807c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:51:58.024 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.377+0000 7f58d7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d807c330 0x7f58d807c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58664/0 (socket says 172.21.15.84:58664) 2024-05-22T22:51:58.024 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.377+0000 7f58d7fff700 1 -- 172.21.15.84:0/1029805790 learned_addr learned my addr 172.21.15.84:0/1029805790 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:51:58.024 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.378+0000 7f58d7fff700 1 -- 172.21.15.84:0/1029805790 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d807cc90 con 0x7f58d807c330 2024-05-22T22:51:58.025 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.378+0000 7f58d7fff700 1 --2- 172.21.15.84:0/1029805790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d807c330 0x7f58d807c750 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f58c80098a0 tx=0x7f58c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf1db6a8b33abe6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:51:58.025 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.378+0000 7f58d77fe700 1 -- 172.21.15.84:0/1029805790 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58c8004160 con 0x7f58d807c330 2024-05-22T22:51:58.025 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.378+0000 7f58d77fe700 1 -- 172.21.15.84:0/1029805790 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f58c8004300 con 0x7f58d807c330 2024-05-22T22:51:58.025 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.378+0000 7f58d77fe700 1 -- 172.21.15.84:0/1029805790 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58c8011b50 con 0x7f58d807c330 2024-05-22T22:51:58.025 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.379+0000 7f58de905700 1 -- 172.21.15.84:0/1029805790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d807c330 msgr2=0x7f58d807c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:51:58.025 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.379+0000 7f58de905700 1 --2- 172.21.15.84:0/1029805790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d807c330 0x7f58d807c750 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f58c80098a0 tx=0x7f58c8009bb0 comp rx=0 tx=0).stop 2024-05-22T22:51:58.025 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.379+0000 7f58de905700 1 -- 172.21.15.84:0/1029805790 shutdown_connections 2024-05-22T22:51:58.025 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.379+0000 7f58de905700 1 --2- 172.21.15.84:0/1029805790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d807c330 0x7f58d807c750 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:51:58.026 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.379+0000 7f58de905700 1 -- 172.21.15.84:0/1029805790 >> 172.21.15.84:0/1029805790 conn(0x7f58d8103450 msgr2=0x7f58d8105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:51:58.026 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.379+0000 7f58de905700 1 -- 172.21.15.84:0/1029805790 shutdown_connections 2024-05-22T22:51:58.026 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.379+0000 7f58de905700 1 -- 172.21.15.84:0/1029805790 wait complete. 2024-05-22T22:51:58.026 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.379+0000 7f58de905700 1 Processor -- start 2024-05-22T22:51:58.026 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.379+0000 7f58de905700 1 -- start start 2024-05-22T22:51:58.026 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.380+0000 7f58de905700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d807c330 0x7f58d819e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:51:58.026 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.380+0000 7f58de905700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f58d810e500 con 0x7f58d807c330 2024-05-22T22:51:58.026 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.380+0000 7f58d7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d807c330 0x7f58d819e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:51:58.026 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.380+0000 7f58d7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d807c330 0x7f58d819e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58676/0 (socket says 172.21.15.84:58676) 2024-05-22T22:51:58.026 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.380+0000 7f58d7fff700 1 -- 172.21.15.84:0/3463360817 learned_addr learned my addr 172.21.15.84:0/3463360817 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:51:58.026 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.380+0000 7f58d7fff700 1 -- 172.21.15.84:0/3463360817 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d819e610 con 0x7f58d807c330 2024-05-22T22:51:58.026 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.380+0000 7f58d7fff700 1 --2- 172.21.15.84:0/3463360817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d807c330 0x7f58d819e0d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009fb0 tx=0x7f58c8011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:51:58.027 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.380+0000 7f58d5ffb700 1 -- 172.21.15.84:0/3463360817 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58c80038b0 con 0x7f58d807c330 2024-05-22T22:51:58.027 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.380+0000 7f58d5ffb700 1 -- 172.21.15.84:0/3463360817 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f58c8011e50 con 0x7f58d807c330 2024-05-22T22:51:58.027 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.380+0000 7f58de905700 1 -- 172.21.15.84:0/3463360817 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d819e8a0 con 0x7f58d807c330 2024-05-22T22:51:58.027 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.380+0000 7f58d5ffb700 1 -- 172.21.15.84:0/3463360817 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58c8019420 con 0x7f58d807c330 2024-05-22T22:51:58.027 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.380+0000 7f58de905700 1 -- 172.21.15.84:0/3463360817 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d81a1590 con 0x7f58d807c330 2024-05-22T22:51:58.027 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.381+0000 7f58d5ffb700 1 -- 172.21.15.84:0/3463360817 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f58c8019a60 con 0x7f58d807c330 2024-05-22T22:51:58.027 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.381+0000 7f58d5ffb700 1 -- 172.21.15.84:0/3463360817 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f58c8024730 con 0x7f58d807c330 2024-05-22T22:51:58.027 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.381+0000 7f58de905700 1 -- 172.21.15.84:0/3463360817 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d80639c0 con 0x7f58d807c330 2024-05-22T22:51:58.027 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.384+0000 7f58d5ffb700 1 -- 172.21.15.84:0/3463360817 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f58c8043dd0 con 0x7f58d807c330 2024-05-22T22:51:58.027 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.449+0000 7f58de905700 1 -- 172.21.15.84:0/3463360817 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f58d81a1a40 con 0x7f58d807c330 2024-05-22T22:51:58.027 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.451+0000 7f58d5ffb700 1 -- 172.21.15.84:0/3463360817 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f58c8032030 con 0x7f58d807c330 2024-05-22T22:51:58.027 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.452+0000 7f58de905700 1 -- 172.21.15.84:0/3463360817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d807c330 msgr2=0x7f58d819e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:51:58.027 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.452+0000 7f58de905700 1 --2- 172.21.15.84:0/3463360817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d807c330 0x7f58d819e0d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009fb0 tx=0x7f58c8011bf0 comp rx=0 tx=0).stop 2024-05-22T22:51:58.028 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.452+0000 7f58de905700 1 -- 172.21.15.84:0/3463360817 shutdown_connections 2024-05-22T22:51:58.028 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.452+0000 7f58de905700 1 --2- 172.21.15.84:0/3463360817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d807c330 0x7f58d819e0d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:51:58.028 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.452+0000 7f58de905700 1 -- 172.21.15.84:0/3463360817 >> 172.21.15.84:0/3463360817 conn(0x7f58d8103450 msgr2=0x7f58d8103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:51:58.028 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.452+0000 7f58de905700 1 -- 172.21.15.84:0/3463360817 shutdown_connections 2024-05-22T22:51:58.028 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:57.452+0000 7f58de905700 1 -- 172.21.15.84:0/3463360817 wait complete. 2024-05-22T22:51:58.028 INFO:teuthology.orchestra.run.smithi084.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-22T22:51:58.042 INFO:teuthology.orchestra.run.smithi084.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-22T22:51:58.042 INFO:teuthology.orchestra.run.smithi084.stdout:Creating mgr... 2024-05-22T22:51:58.042 INFO:teuthology.orchestra.run.smithi084.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-22T22:51:58.043 INFO:teuthology.orchestra.run.smithi084.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-22T22:51:58.288 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mgr.x 2024-05-22T22:51:58.288 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mgr.x.service: Unit ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mgr.x.service not loaded. 2024-05-22T22:51:58.435 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c1135a12-188d-11ef-bc9a-c7b262605968.target.wants/ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-c1135a12-188d-11ef-bc9a-c7b262605968@.service. 2024-05-22T22:51:58.751 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:58 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3463360817' entity='client.admin' 2024-05-22T22:51:58.935 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-22T22:51:58.935 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout disabled 2024-05-22T22:51:58.942 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-22T22:51:58.942 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout inactive 2024-05-22T22:51:58.942 INFO:teuthology.orchestra.run.smithi084.stdout:firewalld.service is not enabled 2024-05-22T22:51:58.943 INFO:teuthology.orchestra.run.smithi084.stdout:Not possible to enable service . firewalld.service is not available 2024-05-22T22:51:58.951 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-22T22:51:58.951 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout disabled 2024-05-22T22:51:58.959 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-22T22:51:58.959 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout inactive 2024-05-22T22:51:58.959 INFO:teuthology.orchestra.run.smithi084.stdout:firewalld.service is not enabled 2024-05-22T22:51:58.959 INFO:teuthology.orchestra.run.smithi084.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-22T22:51:58.959 INFO:teuthology.orchestra.run.smithi084.stdout:Waiting for mgr to start... 2024-05-22T22:51:58.959 INFO:teuthology.orchestra.run.smithi084.stdout:Waiting for mgr... 2024-05-22T22:51:59.688 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:51:59 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3377895686' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-22T22:52:00.148 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-05-22T22:52:00.150 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout { 2024-05-22T22:52:00.151 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "fsid": "c1135a12-188d-11ef-bc9a-c7b262605968", 2024-05-22T22:52:00.151 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "health": { 2024-05-22T22:52:00.151 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-22T22:52:00.151 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-22T22:52:00.151 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-22T22:52:00.151 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:00.151 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-22T22:52:00.151 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-22T22:52:00.151 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 0 2024-05-22T22:52:00.151 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout ], 2024-05-22T22:52:00.151 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-22T22:52:00.151 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "a" 2024-05-22T22:52:00.152 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout ], 2024-05-22T22:52:00.152 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-22T22:52:00.152 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-22T22:52:00.152 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-22T22:52:00.152 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-22T22:52:00.152 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-22T22:52:00.152 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:00.152 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-22T22:52:00.152 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-22T22:52:00.152 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-22T22:52:00.152 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-22T22:52:00.152 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-22T22:52:00.165 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-22T22:52:00.166 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-22T22:52:00.166 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-22T22:52:00.166 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:00.166 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-22T22:52:00.166 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-22T22:52:00.167 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-22T22:52:00.167 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-22T22:52:00.167 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-22T22:52:00.167 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-22T22:52:00.167 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-22T22:52:00.167 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-22T22:52:00.168 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-22T22:52:00.168 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:00.168 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-22T22:52:00.168 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-22T22:52:00.168 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-22T22:52:00.168 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-22T22:52:00.168 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:00.169 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-22T22:52:00.169 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-22T22:52:00.169 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-22T22:52:00.169 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-22T22:52:00.169 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-22T22:52:00.169 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-22T22:52:00.169 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "restful" 2024-05-22T22:52:00.170 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout ], 2024-05-22T22:52:00.170 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-22T22:52:00.170 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:00.170 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-22T22:52:00.170 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-22T22:52:00.170 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "modified": "2024-05-22T22:51:49.878683+0000", 2024-05-22T22:52:00.170 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-22T22:52:00.171 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:00.171 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-22T22:52:00.171 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout } 2024-05-22T22:52:00.171 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.430+0000 7f617b75a700 1 Processor -- start 2024-05-22T22:52:00.171 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.430+0000 7f617b75a700 1 -- start start 2024-05-22T22:52:00.171 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.430+0000 7f617b75a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6174108890 0x7f617410ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:00.171 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.430+0000 7f617b75a700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f617407be60 con 0x7f6174108890 2024-05-22T22:52:00.172 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.431+0000 7f61794f6700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6174108890 0x7f617410ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:00.172 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.431+0000 7f61794f6700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6174108890 0x7f617410ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58688/0 (socket says 172.21.15.84:58688) 2024-05-22T22:52:00.172 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.431+0000 7f61794f6700 1 -- 172.21.15.84:0/512849481 learned_addr learned my addr 172.21.15.84:0/512849481 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:00.172 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.431+0000 7f61794f6700 1 -- 172.21.15.84:0/512849481 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617410b1c0 con 0x7f6174108890 2024-05-22T22:52:00.172 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.431+0000 7f61794f6700 1 --2- 172.21.15.84:0/512849481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6174108890 0x7f617410ac80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f61700098a0 tx=0x7f6170009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d979f5bfc605b911 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:00.172 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.431+0000 7f616bfff700 1 -- 172.21.15.84:0/512849481 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6170004160 con 0x7f6174108890 2024-05-22T22:52:00.172 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.432+0000 7f616bfff700 1 -- 172.21.15.84:0/512849481 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6170004300 con 0x7f6174108890 2024-05-22T22:52:00.172 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.432+0000 7f616bfff700 1 -- 172.21.15.84:0/512849481 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6170011b50 con 0x7f6174108890 2024-05-22T22:52:00.172 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.432+0000 7f617b75a700 1 -- 172.21.15.84:0/512849481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6174108890 msgr2=0x7f617410ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:00.172 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.432+0000 7f617b75a700 1 --2- 172.21.15.84:0/512849481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6174108890 0x7f617410ac80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f61700098a0 tx=0x7f6170009bb0 comp rx=0 tx=0).stop 2024-05-22T22:52:00.172 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.432+0000 7f617b75a700 1 -- 172.21.15.84:0/512849481 shutdown_connections 2024-05-22T22:52:00.173 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.432+0000 7f617b75a700 1 --2- 172.21.15.84:0/512849481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6174108890 0x7f617410ac80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:00.173 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.432+0000 7f617b75a700 1 -- 172.21.15.84:0/512849481 >> 172.21.15.84:0/512849481 conn(0x7f6174102260 msgr2=0x7f61741046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:00.173 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.432+0000 7f617b75a700 1 -- 172.21.15.84:0/512849481 shutdown_connections 2024-05-22T22:52:00.173 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.433+0000 7f617b75a700 1 -- 172.21.15.84:0/512849481 wait complete. 2024-05-22T22:52:00.173 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.433+0000 7f617b75a700 1 Processor -- start 2024-05-22T22:52:00.173 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.433+0000 7f617b75a700 1 -- start start 2024-05-22T22:52:00.173 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.433+0000 7f617b75a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6174108890 0x7f617419c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:00.173 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.433+0000 7f617b75a700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f617410c190 con 0x7f6174108890 2024-05-22T22:52:00.173 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.434+0000 7f61794f6700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6174108890 0x7f617419c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:00.173 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.434+0000 7f61794f6700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6174108890 0x7f617419c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58704/0 (socket says 172.21.15.84:58704) 2024-05-22T22:52:00.173 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.434+0000 7f61794f6700 1 -- 172.21.15.84:0/3377895686 learned_addr learned my addr 172.21.15.84:0/3377895686 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:00.173 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.434+0000 7f61794f6700 1 -- 172.21.15.84:0/3377895686 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617419c580 con 0x7f6174108890 2024-05-22T22:52:00.173 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.434+0000 7f61794f6700 1 --2- 172.21.15.84:0/3377895686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6174108890 0x7f617419c040 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6170011b60 tx=0x7f6170011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:00.173 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.434+0000 7f616a7fc700 1 -- 172.21.15.84:0/3377895686 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6170003b20 con 0x7f6174108890 2024-05-22T22:52:00.174 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.434+0000 7f616a7fc700 1 -- 172.21.15.84:0/3377895686 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6170003cc0 con 0x7f6174108890 2024-05-22T22:52:00.174 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.434+0000 7f617b75a700 1 -- 172.21.15.84:0/3377895686 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f617419c810 con 0x7f6174108890 2024-05-22T22:52:00.174 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.434+0000 7f616a7fc700 1 -- 172.21.15.84:0/3377895686 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6170024840 con 0x7f6174108890 2024-05-22T22:52:00.174 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.434+0000 7f617b75a700 1 -- 172.21.15.84:0/3377895686 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f617419f500 con 0x7f6174108890 2024-05-22T22:52:00.174 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.434+0000 7f616a7fc700 1 -- 172.21.15.84:0/3377895686 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6170022020 con 0x7f6174108890 2024-05-22T22:52:00.174 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.434+0000 7f616a7fc700 1 -- 172.21.15.84:0/3377895686 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6170011e00 con 0x7f6174108890 2024-05-22T22:52:00.174 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.435+0000 7f617b75a700 1 -- 172.21.15.84:0/3377895686 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61740639c0 con 0x7f6174108890 2024-05-22T22:52:00.174 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.438+0000 7f616a7fc700 1 -- 172.21.15.84:0/3377895686 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f6170043dd0 con 0x7f6174108890 2024-05-22T22:52:00.174 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.565+0000 7f617b75a700 1 -- 172.21.15.84:0/3377895686 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f617402dbc0 con 0x7f6174108890 2024-05-22T22:52:00.174 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.566+0000 7f616a7fc700 1 -- 172.21.15.84:0/3377895686 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6170032020 con 0x7f6174108890 2024-05-22T22:52:00.174 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.567+0000 7f617b75a700 1 -- 172.21.15.84:0/3377895686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6174108890 msgr2=0x7f617419c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:00.174 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.567+0000 7f617b75a700 1 --2- 172.21.15.84:0/3377895686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6174108890 0x7f617419c040 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6170011b60 tx=0x7f6170011c40 comp rx=0 tx=0).stop 2024-05-22T22:52:00.174 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.567+0000 7f617b75a700 1 -- 172.21.15.84:0/3377895686 shutdown_connections 2024-05-22T22:52:00.175 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.567+0000 7f617b75a700 1 --2- 172.21.15.84:0/3377895686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6174108890 0x7f617419c040 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:00.175 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.567+0000 7f617b75a700 1 -- 172.21.15.84:0/3377895686 >> 172.21.15.84:0/3377895686 conn(0x7f6174102260 msgr2=0x7f6174102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:00.175 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.567+0000 7f617b75a700 1 -- 172.21.15.84:0/3377895686 shutdown_connections 2024-05-22T22:52:00.175 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:51:59.567+0000 7f617b75a700 1 -- 172.21.15.84:0/3377895686 wait complete. 2024-05-22T22:52:00.175 INFO:teuthology.orchestra.run.smithi084.stdout:mgr not available, waiting (1/15)... 2024-05-22T22:52:01.347 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:01 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:01.188+0000 7f58afc47200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-22T22:52:01.347 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:01 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:01.356+0000 7f58afc47200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-22T22:52:01.929 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:01 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:01.646+0000 7f58afc47200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-22T22:52:01.929 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:01 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:01.737+0000 7f58afc47200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-22T22:52:01.929 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:01 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:01.938+0000 7f58afc47200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-22T22:52:02.281 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:02 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:02.130+0000 7f58afc47200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-22T22:52:02.864 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:02 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/107860287' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-22T22:52:02.865 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:02 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:02.765+0000 7f58afc47200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-22T22:52:03.226 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:03 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:03.203+0000 7f58afc47200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-22T22:52:03.229 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-05-22T22:52:03.232 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout { 2024-05-22T22:52:03.232 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "fsid": "c1135a12-188d-11ef-bc9a-c7b262605968", 2024-05-22T22:52:03.232 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "health": { 2024-05-22T22:52:03.232 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-22T22:52:03.232 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-22T22:52:03.232 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-22T22:52:03.233 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:03.233 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-22T22:52:03.233 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-22T22:52:03.233 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 0 2024-05-22T22:52:03.233 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout ], 2024-05-22T22:52:03.233 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-22T22:52:03.233 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "a" 2024-05-22T22:52:03.233 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout ], 2024-05-22T22:52:03.233 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-22T22:52:03.233 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-22T22:52:03.233 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-22T22:52:03.233 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-22T22:52:03.233 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-22T22:52:03.233 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:03.233 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-22T22:52:03.234 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-22T22:52:03.234 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-22T22:52:03.234 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-22T22:52:03.234 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-22T22:52:03.242 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-22T22:52:03.242 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-22T22:52:03.242 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-22T22:52:03.243 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:03.243 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-22T22:52:03.243 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-22T22:52:03.243 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-22T22:52:03.243 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-22T22:52:03.243 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-22T22:52:03.243 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-22T22:52:03.243 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-22T22:52:03.243 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-22T22:52:03.243 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-22T22:52:03.243 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:03.244 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-22T22:52:03.244 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-22T22:52:03.244 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-22T22:52:03.244 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-22T22:52:03.244 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:03.244 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-22T22:52:03.244 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-22T22:52:03.244 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-22T22:52:03.244 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-22T22:52:03.244 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-22T22:52:03.244 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-22T22:52:03.245 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "restful" 2024-05-22T22:52:03.245 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout ], 2024-05-22T22:52:03.245 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-22T22:52:03.245 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:03.245 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-22T22:52:03.245 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-22T22:52:03.245 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "modified": "2024-05-22T22:51:49.878683+0000", 2024-05-22T22:52:03.245 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-22T22:52:03.245 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:03.245 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-22T22:52:03.246 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout } 2024-05-22T22:52:03.246 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.593+0000 7fd368a45700 1 Processor -- start 2024-05-22T22:52:03.246 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.593+0000 7fd368a45700 1 -- start start 2024-05-22T22:52:03.246 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.593+0000 7fd368a45700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd364108850 0x7fd36410ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:03.246 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.594+0000 7fd368a45700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd36407be80 con 0x7fd364108850 2024-05-22T22:52:03.246 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.594+0000 7fd36259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd364108850 0x7fd36410ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:03.246 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.594+0000 7fd36259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd364108850 0x7fd36410ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46744/0 (socket says 172.21.15.84:46744) 2024-05-22T22:52:03.246 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.594+0000 7fd36259c700 1 -- 172.21.15.84:0/3687256882 learned_addr learned my addr 172.21.15.84:0/3687256882 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:03.246 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.594+0000 7fd36259c700 1 -- 172.21.15.84:0/3687256882 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36410b180 con 0x7fd364108850 2024-05-22T22:52:03.246 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.594+0000 7fd36259c700 1 --2- 172.21.15.84:0/3687256882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd364108850 0x7fd36410ac40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd3540098a0 tx=0x7fd354009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83bf0f6eda22541f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:03.246 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.595+0000 7fd36159a700 1 -- 172.21.15.84:0/3687256882 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd354004160 con 0x7fd364108850 2024-05-22T22:52:03.247 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.595+0000 7fd36159a700 1 -- 172.21.15.84:0/3687256882 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd354004300 con 0x7fd364108850 2024-05-22T22:52:03.247 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.595+0000 7fd36159a700 1 -- 172.21.15.84:0/3687256882 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd354011b50 con 0x7fd364108850 2024-05-22T22:52:03.247 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.595+0000 7fd368a45700 1 -- 172.21.15.84:0/3687256882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd364108850 msgr2=0x7fd36410ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:03.247 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.595+0000 7fd368a45700 1 --2- 172.21.15.84:0/3687256882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd364108850 0x7fd36410ac40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd3540098a0 tx=0x7fd354009bb0 comp rx=0 tx=0).stop 2024-05-22T22:52:03.247 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.595+0000 7fd368a45700 1 -- 172.21.15.84:0/3687256882 shutdown_connections 2024-05-22T22:52:03.247 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.595+0000 7fd368a45700 1 --2- 172.21.15.84:0/3687256882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd364108850 0x7fd36410ac40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:03.247 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.595+0000 7fd368a45700 1 -- 172.21.15.84:0/3687256882 >> 172.21.15.84:0/3687256882 conn(0x7fd364102260 msgr2=0x7fd364104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:03.247 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.596+0000 7fd368a45700 1 -- 172.21.15.84:0/3687256882 shutdown_connections 2024-05-22T22:52:03.247 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.596+0000 7fd368a45700 1 -- 172.21.15.84:0/3687256882 wait complete. 2024-05-22T22:52:03.247 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.596+0000 7fd368a45700 1 Processor -- start 2024-05-22T22:52:03.247 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.597+0000 7fd368a45700 1 -- start start 2024-05-22T22:52:03.247 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.597+0000 7fd368a45700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd364108850 0x7fd3641080d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:03.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.597+0000 7fd368a45700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd36410c1a0 con 0x7fd364108850 2024-05-22T22:52:03.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.597+0000 7fd36259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd364108850 0x7fd3641080d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:03.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.597+0000 7fd36259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd364108850 0x7fd3641080d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46754/0 (socket says 172.21.15.84:46754) 2024-05-22T22:52:03.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.597+0000 7fd36259c700 1 -- 172.21.15.84:0/107860287 learned_addr learned my addr 172.21.15.84:0/107860287 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:03.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.597+0000 7fd36259c700 1 -- 172.21.15.84:0/107860287 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd364108610 con 0x7fd364108850 2024-05-22T22:52:03.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.597+0000 7fd36259c700 1 --2- 172.21.15.84:0/107860287 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd364108850 0x7fd3641080d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd354011670 tx=0x7fd354011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:03.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.597+0000 7fd3537fe700 1 -- 172.21.15.84:0/107860287 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3540039c0 con 0x7fd364108850 2024-05-22T22:52:03.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.598+0000 7fd3537fe700 1 -- 172.21.15.84:0/107860287 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd354011e50 con 0x7fd364108850 2024-05-22T22:52:03.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.598+0000 7fd368a45700 1 -- 172.21.15.84:0/107860287 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd364106800 con 0x7fd364108850 2024-05-22T22:52:03.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.598+0000 7fd3537fe700 1 -- 172.21.15.84:0/107860287 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd35401a5c0 con 0x7fd364108850 2024-05-22T22:52:03.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.598+0000 7fd368a45700 1 -- 172.21.15.84:0/107860287 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd364106d40 con 0x7fd364108850 2024-05-22T22:52:03.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.598+0000 7fd3537fe700 1 -- 172.21.15.84:0/107860287 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd35401ac50 con 0x7fd364108850 2024-05-22T22:52:03.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.598+0000 7fd3537fe700 1 -- 172.21.15.84:0/107860287 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd354019700 con 0x7fd364108850 2024-05-22T22:52:03.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.599+0000 7fd368a45700 1 -- 172.21.15.84:0/107860287 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3640639c0 con 0x7fd364108850 2024-05-22T22:52:03.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.601+0000 7fd3537fe700 1 -- 172.21.15.84:0/107860287 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd354003b60 con 0x7fd364108850 2024-05-22T22:52:03.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.673+0000 7fd368a45700 1 -- 172.21.15.84:0/107860287 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd364107640 con 0x7fd364108850 2024-05-22T22:52:03.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.674+0000 7fd3537fe700 1 -- 172.21.15.84:0/107860287 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fd354019900 con 0x7fd364108850 2024-05-22T22:52:03.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.675+0000 7fd368a45700 1 -- 172.21.15.84:0/107860287 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd364108850 msgr2=0x7fd3641080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:03.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.675+0000 7fd368a45700 1 --2- 172.21.15.84:0/107860287 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd364108850 0x7fd3641080d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd354011670 tx=0x7fd354011b60 comp rx=0 tx=0).stop 2024-05-22T22:52:03.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.675+0000 7fd368a45700 1 -- 172.21.15.84:0/107860287 shutdown_connections 2024-05-22T22:52:03.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.675+0000 7fd368a45700 1 --2- 172.21.15.84:0/107860287 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd364108850 0x7fd3641080d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:03.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.675+0000 7fd368a45700 1 -- 172.21.15.84:0/107860287 >> 172.21.15.84:0/107860287 conn(0x7fd364102260 msgr2=0x7fd364102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:03.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.675+0000 7fd368a45700 1 -- 172.21.15.84:0/107860287 shutdown_connections 2024-05-22T22:52:03.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:02.675+0000 7fd368a45700 1 -- 172.21.15.84:0/107860287 wait complete. 2024-05-22T22:52:03.249 INFO:teuthology.orchestra.run.smithi084.stdout:mgr not available, waiting (2/15)... 2024-05-22T22:52:03.492 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:03 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:03.302+0000 7f58afc47200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-22T22:52:03.492 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:03 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:03.502+0000 7f58afc47200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-22T22:52:04.345 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:04 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:04.024+0000 7f58afc47200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-22T22:52:04.346 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:04 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:04.115+0000 7f58afc47200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-22T22:52:04.614 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:04 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:04.354+0000 7f58afc47200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-22T22:52:04.614 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:04 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:04.487+0000 7f58afc47200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-22T22:52:04.615 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:04 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:04.582+0000 7f58afc47200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-22T22:52:04.988 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:04 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:04.698+0000 7f58afc47200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-22T22:52:04.988 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:04 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:04.996+0000 7f58afc47200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-22T22:52:05.614 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:05 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:05.614+0000 7f58afc47200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-22T22:52:06.114 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:05 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/598777773' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-22T22:52:06.115 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:05 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:05.843+0000 7f58afc47200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-22T22:52:06.388 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-05-22T22:52:06.388 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout { 2024-05-22T22:52:06.388 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "fsid": "c1135a12-188d-11ef-bc9a-c7b262605968", 2024-05-22T22:52:06.388 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "health": { 2024-05-22T22:52:06.388 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-22T22:52:06.388 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-22T22:52:06.388 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-22T22:52:06.389 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:06.389 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-22T22:52:06.389 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-22T22:52:06.389 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 0 2024-05-22T22:52:06.389 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout ], 2024-05-22T22:52:06.389 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-22T22:52:06.389 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "a" 2024-05-22T22:52:06.389 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout ], 2024-05-22T22:52:06.389 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-22T22:52:06.389 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-22T22:52:06.389 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-22T22:52:06.389 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-22T22:52:06.390 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-22T22:52:06.390 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:06.390 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-22T22:52:06.390 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-22T22:52:06.390 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-22T22:52:06.390 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-22T22:52:06.390 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-22T22:52:06.398 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-22T22:52:06.398 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-22T22:52:06.398 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-22T22:52:06.398 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:06.399 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-22T22:52:06.399 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-22T22:52:06.399 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-22T22:52:06.399 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-22T22:52:06.399 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-22T22:52:06.399 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-22T22:52:06.399 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-22T22:52:06.399 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-22T22:52:06.399 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-22T22:52:06.399 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:06.399 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-22T22:52:06.399 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-22T22:52:06.400 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-22T22:52:06.400 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-22T22:52:06.400 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:06.400 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-22T22:52:06.400 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-22T22:52:06.400 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-22T22:52:06.400 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-22T22:52:06.400 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-22T22:52:06.400 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-22T22:52:06.400 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "restful" 2024-05-22T22:52:06.400 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout ], 2024-05-22T22:52:06.401 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-22T22:52:06.401 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:06.401 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-22T22:52:06.401 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-22T22:52:06.401 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "modified": "2024-05-22T22:51:49.878683+0000", 2024-05-22T22:52:06.401 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-22T22:52:06.401 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:06.401 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-22T22:52:06.401 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout } 2024-05-22T22:52:06.401 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.681+0000 7f91f23c7700 1 Processor -- start 2024-05-22T22:52:06.401 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.681+0000 7f91f23c7700 1 -- start start 2024-05-22T22:52:06.401 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.681+0000 7f91f23c7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ec108870 0x7f91ec10ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:06.402 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.681+0000 7f91f23c7700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f91ec07be80 con 0x7f91ec108870 2024-05-22T22:52:06.402 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.681+0000 7f91ebfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ec108870 0x7f91ec10ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:06.402 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.681+0000 7f91ebfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ec108870 0x7f91ec10ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46768/0 (socket says 172.21.15.84:46768) 2024-05-22T22:52:06.402 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.682+0000 7f91ebfff700 1 -- 172.21.15.84:0/1793719476 learned_addr learned my addr 172.21.15.84:0/1793719476 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:06.402 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.682+0000 7f91ebfff700 1 -- 172.21.15.84:0/1793719476 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ec10b1a0 con 0x7f91ec108870 2024-05-22T22:52:06.402 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.682+0000 7f91ebfff700 1 --2- 172.21.15.84:0/1793719476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ec108870 0x7f91ec10ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f91dc0098a0 tx=0x7f91dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1469f5fe1fbfb60 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:06.402 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.682+0000 7f91eaffd700 1 -- 172.21.15.84:0/1793719476 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91dc004160 con 0x7f91ec108870 2024-05-22T22:52:06.402 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.682+0000 7f91eaffd700 1 -- 172.21.15.84:0/1793719476 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f91dc004300 con 0x7f91ec108870 2024-05-22T22:52:06.402 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.682+0000 7f91eaffd700 1 -- 172.21.15.84:0/1793719476 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91dc011b50 con 0x7f91ec108870 2024-05-22T22:52:06.402 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.683+0000 7f91f23c7700 1 -- 172.21.15.84:0/1793719476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ec108870 msgr2=0x7f91ec10ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:06.403 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.683+0000 7f91f23c7700 1 --2- 172.21.15.84:0/1793719476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ec108870 0x7f91ec10ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f91dc0098a0 tx=0x7f91dc009bb0 comp rx=0 tx=0).stop 2024-05-22T22:52:06.403 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.683+0000 7f91f23c7700 1 -- 172.21.15.84:0/1793719476 shutdown_connections 2024-05-22T22:52:06.403 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.683+0000 7f91f23c7700 1 --2- 172.21.15.84:0/1793719476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ec108870 0x7f91ec10ac60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:06.403 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.683+0000 7f91f23c7700 1 -- 172.21.15.84:0/1793719476 >> 172.21.15.84:0/1793719476 conn(0x7f91ec102260 msgr2=0x7f91ec1046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:06.403 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.683+0000 7f91f23c7700 1 -- 172.21.15.84:0/1793719476 shutdown_connections 2024-05-22T22:52:06.403 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.683+0000 7f91f23c7700 1 -- 172.21.15.84:0/1793719476 wait complete. 2024-05-22T22:52:06.403 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.683+0000 7f91f23c7700 1 Processor -- start 2024-05-22T22:52:06.403 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.684+0000 7f91f23c7700 1 -- start start 2024-05-22T22:52:06.403 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.684+0000 7f91f23c7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ec108870 0x7f91ec19c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:06.403 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.684+0000 7f91f23c7700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f91ec10c1c0 con 0x7f91ec108870 2024-05-22T22:52:06.403 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.684+0000 7f91ebfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ec108870 0x7f91ec19c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:06.404 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.684+0000 7f91ebfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ec108870 0x7f91ec19c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46774/0 (socket says 172.21.15.84:46774) 2024-05-22T22:52:06.404 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.684+0000 7f91ebfff700 1 -- 172.21.15.84:0/598777773 learned_addr learned my addr 172.21.15.84:0/598777773 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:06.404 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.684+0000 7f91ebfff700 1 -- 172.21.15.84:0/598777773 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ec19c580 con 0x7f91ec108870 2024-05-22T22:52:06.404 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.684+0000 7f91ebfff700 1 --2- 172.21.15.84:0/598777773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ec108870 0x7f91ec19c040 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f91dc011b60 tx=0x7f91dc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:06.404 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.685+0000 7f91e97fa700 1 -- 172.21.15.84:0/598777773 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91dc003b20 con 0x7f91ec108870 2024-05-22T22:52:06.404 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.685+0000 7f91e97fa700 1 -- 172.21.15.84:0/598777773 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f91dc003cc0 con 0x7f91ec108870 2024-05-22T22:52:06.404 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.685+0000 7f91f23c7700 1 -- 172.21.15.84:0/598777773 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91ec19c810 con 0x7f91ec108870 2024-05-22T22:52:06.404 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.685+0000 7f91e97fa700 1 -- 172.21.15.84:0/598777773 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91dc024840 con 0x7f91ec108870 2024-05-22T22:52:06.404 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.685+0000 7f91f23c7700 1 -- 172.21.15.84:0/598777773 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91ec19f4b0 con 0x7f91ec108870 2024-05-22T22:52:06.404 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.685+0000 7f91e97fa700 1 -- 172.21.15.84:0/598777773 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f91dc022020 con 0x7f91ec108870 2024-05-22T22:52:06.405 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.685+0000 7f91e97fa700 1 -- 172.21.15.84:0/598777773 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f91dc011e00 con 0x7f91ec108870 2024-05-22T22:52:06.405 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.686+0000 7f91f23c7700 1 -- 172.21.15.84:0/598777773 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91ec0639c0 con 0x7f91ec108870 2024-05-22T22:52:06.405 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.689+0000 7f91e97fa700 1 -- 172.21.15.84:0/598777773 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f91dc043dd0 con 0x7f91ec108870 2024-05-22T22:52:06.405 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.761+0000 7f91f23c7700 1 -- 172.21.15.84:0/598777773 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f91ec02dbc0 con 0x7f91ec108870 2024-05-22T22:52:06.405 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.761+0000 7f91e97fa700 1 -- 172.21.15.84:0/598777773 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f91dc032030 con 0x7f91ec108870 2024-05-22T22:52:06.405 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.762+0000 7f91f23c7700 1 -- 172.21.15.84:0/598777773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ec108870 msgr2=0x7f91ec19c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:06.405 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.762+0000 7f91f23c7700 1 --2- 172.21.15.84:0/598777773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ec108870 0x7f91ec19c040 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f91dc011b60 tx=0x7f91dc011c40 comp rx=0 tx=0).stop 2024-05-22T22:52:06.405 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.762+0000 7f91f23c7700 1 -- 172.21.15.84:0/598777773 shutdown_connections 2024-05-22T22:52:06.405 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.762+0000 7f91f23c7700 1 --2- 172.21.15.84:0/598777773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ec108870 0x7f91ec19c040 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:06.405 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.762+0000 7f91f23c7700 1 -- 172.21.15.84:0/598777773 >> 172.21.15.84:0/598777773 conn(0x7f91ec102260 msgr2=0x7f91ec102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:06.406 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.762+0000 7f91f23c7700 1 -- 172.21.15.84:0/598777773 shutdown_connections 2024-05-22T22:52:06.406 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:05.762+0000 7f91f23c7700 1 -- 172.21.15.84:0/598777773 wait complete. 2024-05-22T22:52:06.406 INFO:teuthology.orchestra.run.smithi084.stdout:mgr not available, waiting (3/15)... 2024-05-22T22:52:07.114 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:06 smithi084 ceph-mon[32186]: Activating manager daemon x 2024-05-22T22:52:07.115 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:06 smithi084 ceph-mon[32186]: mgrmap e2: x(active, starting, since 0.00308336s) 2024-05-22T22:52:07.115 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:06 smithi084 ceph-mon[32186]: from='mgr.14100 172.21.15.84:0/3697937054' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-22T22:52:07.115 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:06 smithi084 ceph-mon[32186]: from='mgr.14100 172.21.15.84:0/3697937054' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-22T22:52:07.115 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:06 smithi084 ceph-mon[32186]: from='mgr.14100 172.21.15.84:0/3697937054' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-22T22:52:07.115 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:06 smithi084 ceph-mon[32186]: from='mgr.14100 172.21.15.84:0/3697937054' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-22T22:52:07.115 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:06 smithi084 ceph-mon[32186]: from='mgr.14100 172.21.15.84:0/3697937054' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-22T22:52:07.115 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:06 smithi084 ceph-mon[32186]: Manager daemon x is now available 2024-05-22T22:52:07.115 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:06 smithi084 ceph-mon[32186]: from='mgr.14100 172.21.15.84:0/3697937054' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-22T22:52:07.115 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:06 smithi084 ceph-mon[32186]: from='mgr.14100 172.21.15.84:0/3697937054' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-22T22:52:07.115 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:06 smithi084 ceph-mon[32186]: from='mgr.14100 172.21.15.84:0/3697937054' entity='mgr.x' 2024-05-22T22:52:07.115 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:06 smithi084 ceph-mon[32186]: from='mgr.14100 172.21.15.84:0/3697937054' entity='mgr.x' 2024-05-22T22:52:07.116 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:06 smithi084 ceph-mon[32186]: from='mgr.14100 172.21.15.84:0/3697937054' entity='mgr.x' 2024-05-22T22:52:08.114 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:07 smithi084 ceph-mon[32186]: mgrmap e3: x(active, since 1.00694s) 2024-05-22T22:52:09.097 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:08 smithi084 ceph-mon[32186]: mgrmap e4: x(active, since 2s) 2024-05-22T22:52:09.669 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-05-22T22:52:09.669 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout { 2024-05-22T22:52:09.669 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "fsid": "c1135a12-188d-11ef-bc9a-c7b262605968", 2024-05-22T22:52:09.669 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "health": { 2024-05-22T22:52:09.669 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-22T22:52:09.670 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-22T22:52:09.670 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-22T22:52:09.670 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:09.670 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-22T22:52:09.670 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-22T22:52:09.670 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 0 2024-05-22T22:52:09.670 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout ], 2024-05-22T22:52:09.670 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-22T22:52:09.670 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "a" 2024-05-22T22:52:09.670 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout ], 2024-05-22T22:52:09.670 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-05-22T22:52:09.670 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-22T22:52:09.670 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-22T22:52:09.670 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-22T22:52:09.671 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-22T22:52:09.671 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:09.671 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-22T22:52:09.671 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-22T22:52:09.671 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-22T22:52:09.671 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-22T22:52:09.671 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-22T22:52:09.679 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-22T22:52:09.679 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-22T22:52:09.679 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-22T22:52:09.679 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:09.679 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-22T22:52:09.679 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-22T22:52:09.679 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-22T22:52:09.680 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-22T22:52:09.680 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-22T22:52:09.680 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-22T22:52:09.680 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-22T22:52:09.680 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-22T22:52:09.680 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-22T22:52:09.680 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:09.680 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-22T22:52:09.680 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-22T22:52:09.680 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-22T22:52:09.680 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-22T22:52:09.680 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:09.680 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-22T22:52:09.681 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-22T22:52:09.681 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-22T22:52:09.681 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-22T22:52:09.681 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-22T22:52:09.681 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-22T22:52:09.681 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "restful" 2024-05-22T22:52:09.681 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout ], 2024-05-22T22:52:09.681 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-22T22:52:09.681 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:09.681 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-22T22:52:09.681 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-22T22:52:09.681 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "modified": "2024-05-22T22:51:49.878683+0000", 2024-05-22T22:52:09.681 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-22T22:52:09.682 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-05-22T22:52:09.682 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-22T22:52:09.682 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout } 2024-05-22T22:52:09.682 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.818+0000 7f4a4d520700 1 Processor -- start 2024-05-22T22:52:09.682 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.818+0000 7f4a4d520700 1 -- start start 2024-05-22T22:52:09.682 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.819+0000 7f4a4d520700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a48106610 0x7f4a48106a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:09.682 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.819+0000 7f4a4d520700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4807bec0 con 0x7f4a48106610 2024-05-22T22:52:09.682 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.819+0000 7f4a46ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a48106610 0x7f4a48106a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:09.682 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.819+0000 7f4a46ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a48106610 0x7f4a48106a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46844/0 (socket says 172.21.15.84:46844) 2024-05-22T22:52:09.682 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.819+0000 7f4a46ffd700 1 -- 172.21.15.84:0/1512853994 learned_addr learned my addr 172.21.15.84:0/1512853994 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:09.682 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.819+0000 7f4a46ffd700 1 -- 172.21.15.84:0/1512853994 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a48106f70 con 0x7f4a48106610 2024-05-22T22:52:09.682 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.819+0000 7f4a46ffd700 1 --2- 172.21.15.84:0/1512853994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a48106610 0x7f4a48106a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009e80 tx=0x7f4a4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7158e30bbfe281fb server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:09.683 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.819+0000 7f4a45ffb700 1 -- 172.21.15.84:0/1512853994 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a40017070 con 0x7f4a48106610 2024-05-22T22:52:09.683 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.819+0000 7f4a45ffb700 1 -- 172.21.15.84:0/1512853994 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4a4000b8c0 con 0x7f4a48106610 2024-05-22T22:52:09.683 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.819+0000 7f4a45ffb700 1 -- 172.21.15.84:0/1512853994 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a400045c0 con 0x7f4a48106610 2024-05-22T22:52:09.683 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.820+0000 7f4a4d520700 1 -- 172.21.15.84:0/1512853994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a48106610 msgr2=0x7f4a48106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:09.683 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.820+0000 7f4a4d520700 1 --2- 172.21.15.84:0/1512853994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a48106610 0x7f4a48106a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009e80 tx=0x7f4a4000b330 comp rx=0 tx=0).stop 2024-05-22T22:52:09.683 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.820+0000 7f4a4d520700 1 -- 172.21.15.84:0/1512853994 shutdown_connections 2024-05-22T22:52:09.683 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.820+0000 7f4a4d520700 1 --2- 172.21.15.84:0/1512853994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a48106610 0x7f4a48106a30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:09.683 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.820+0000 7f4a4d520700 1 -- 172.21.15.84:0/1512853994 >> 172.21.15.84:0/1512853994 conn(0x7f4a48102230 msgr2=0x7f4a48104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:09.683 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.820+0000 7f4a4d520700 1 -- 172.21.15.84:0/1512853994 shutdown_connections 2024-05-22T22:52:09.683 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.820+0000 7f4a4d520700 1 -- 172.21.15.84:0/1512853994 wait complete. 2024-05-22T22:52:09.683 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.821+0000 7f4a4d520700 1 Processor -- start 2024-05-22T22:52:09.683 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.821+0000 7f4a4d520700 1 -- start start 2024-05-22T22:52:09.683 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.821+0000 7f4a4d520700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a48106610 0x7f4a4819e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:09.683 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.821+0000 7f4a4d520700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4a48108000 con 0x7f4a48106610 2024-05-22T22:52:09.683 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.821+0000 7f4a46ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a48106610 0x7f4a4819e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:09.684 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.821+0000 7f4a46ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a48106610 0x7f4a4819e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46856/0 (socket says 172.21.15.84:46856) 2024-05-22T22:52:09.684 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.821+0000 7f4a46ffd700 1 -- 172.21.15.84:0/2178320756 learned_addr learned my addr 172.21.15.84:0/2178320756 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:09.684 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.821+0000 7f4a46ffd700 1 -- 172.21.15.84:0/2178320756 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4819e6f0 con 0x7f4a48106610 2024-05-22T22:52:09.684 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.822+0000 7f4a46ffd700 1 --2- 172.21.15.84:0/2178320756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a48106610 0x7f4a4819e1b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4a400072a0 tx=0x7f4a4000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:09.684 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.822+0000 7f4a3ffff700 1 -- 172.21.15.84:0/2178320756 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a40020020 con 0x7f4a48106610 2024-05-22T22:52:09.684 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.822+0000 7f4a3ffff700 1 -- 172.21.15.84:0/2178320756 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4a40003b50 con 0x7f4a48106610 2024-05-22T22:52:09.684 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.822+0000 7f4a4d520700 1 -- 172.21.15.84:0/2178320756 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a4819e980 con 0x7f4a48106610 2024-05-22T22:52:09.684 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.822+0000 7f4a3ffff700 1 -- 172.21.15.84:0/2178320756 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a40017040 con 0x7f4a48106610 2024-05-22T22:52:09.684 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.822+0000 7f4a4d520700 1 -- 172.21.15.84:0/2178320756 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a481a1670 con 0x7f4a48106610 2024-05-22T22:52:09.684 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.823+0000 7f4a3ffff700 1 -- 172.21.15.84:0/2178320756 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f4a40023020 con 0x7f4a48106610 2024-05-22T22:52:09.684 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.823+0000 7f4a3ffff700 1 --2- 172.21.15.84:0/2178320756 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7f4a34039000 0x7f4a3403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:09.684 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.823+0000 7f4a4d520700 1 -- 172.21.15.84:0/2178320756 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a480639c0 con 0x7f4a48106610 2024-05-22T22:52:09.685 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.823+0000 7f4a3ffff700 1 -- 172.21.15.84:0/2178320756 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4a4000f050 con 0x7f4a48106610 2024-05-22T22:52:09.685 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.823+0000 7f4a467fc700 1 --2- 172.21.15.84:0/2178320756 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7f4a34039000 0x7f4a3403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:09.685 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.823+0000 7f4a467fc700 1 --2- 172.21.15.84:0/2178320756 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7f4a34039000 0x7f4a3403b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4a30006fd0 tx=0x7f4a30006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:09.685 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:08.828+0000 7f4a3ffff700 1 -- 172.21.15.84:0/2178320756 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4a4002a020 con 0x7f4a48106610 2024-05-22T22:52:09.685 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:09.105+0000 7f4a4d520700 1 -- 172.21.15.84:0/2178320756 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4a4802dbc0 con 0x7f4a48106610 2024-05-22T22:52:09.685 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:09.106+0000 7f4a3ffff700 1 -- 172.21.15.84:0/2178320756 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f4a4001b530 con 0x7f4a48106610 2024-05-22T22:52:09.685 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:09.109+0000 7f4a4d520700 1 -- 172.21.15.84:0/2178320756 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7f4a34039000 msgr2=0x7f4a3403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:09.685 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:09.109+0000 7f4a4d520700 1 --2- 172.21.15.84:0/2178320756 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7f4a34039000 0x7f4a3403b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4a30006fd0 tx=0x7f4a30006e40 comp rx=0 tx=0).stop 2024-05-22T22:52:09.685 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:09.109+0000 7f4a4d520700 1 -- 172.21.15.84:0/2178320756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a48106610 msgr2=0x7f4a4819e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:09.685 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:09.109+0000 7f4a4d520700 1 --2- 172.21.15.84:0/2178320756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a48106610 0x7f4a4819e1b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4a400072a0 tx=0x7f4a4000bd50 comp rx=0 tx=0).stop 2024-05-22T22:52:09.685 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:09.109+0000 7f4a4d520700 1 -- 172.21.15.84:0/2178320756 shutdown_connections 2024-05-22T22:52:09.685 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:09.109+0000 7f4a4d520700 1 --2- 172.21.15.84:0/2178320756 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7f4a34039000 0x7f4a3403b4c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:09.685 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:09.109+0000 7f4a4d520700 1 --2- 172.21.15.84:0/2178320756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a48106610 0x7f4a4819e1b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:09.686 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:09.109+0000 7f4a4d520700 1 -- 172.21.15.84:0/2178320756 >> 172.21.15.84:0/2178320756 conn(0x7f4a48102230 msgr2=0x7f4a48194b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:09.686 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:09.109+0000 7f4a4d520700 1 -- 172.21.15.84:0/2178320756 shutdown_connections 2024-05-22T22:52:09.686 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:09.109+0000 7f4a4d520700 1 -- 172.21.15.84:0/2178320756 wait complete. 2024-05-22T22:52:09.686 INFO:teuthology.orchestra.run.smithi084.stdout:mgr is available 2024-05-22T22:52:10.057 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:09 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2178320756' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-22T22:52:10.854 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-05-22T22:52:10.854 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout [global] 2024-05-22T22:52:10.855 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout fsid = c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:52:10.855 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-22T22:52:10.855 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-22T22:52:10.855 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-22T22:52:10.855 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-22T22:52:10.855 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-05-22T22:52:10.855 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-22T22:52:10.855 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-22T22:52:10.855 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-05-22T22:52:10.855 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout [osd] 2024-05-22T22:52:10.855 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-22T22:52:10.855 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-22T22:52:10.856 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-22T22:52:10.856 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-22T22:52:10.856 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-22T22:52:10.856 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.066+0000 7fab23151700 1 Processor -- start 2024-05-22T22:52:10.856 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.066+0000 7fab23151700 1 -- start start 2024-05-22T22:52:10.856 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.066+0000 7fab23151700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab1c1066f0 0x7fab1c106b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:10.856 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.066+0000 7fab23151700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c07be20 con 0x7fab1c1066f0 2024-05-22T22:52:10.856 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.066+0000 7fab20eed700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab1c1066f0 0x7fab1c106b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:10.856 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.066+0000 7fab20eed700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab1c1066f0 0x7fab1c106b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46862/0 (socket says 172.21.15.84:46862) 2024-05-22T22:52:10.856 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.066+0000 7fab20eed700 1 -- 172.21.15.84:0/892588285 learned_addr learned my addr 172.21.15.84:0/892588285 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:10.856 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.067+0000 7fab20eed700 1 -- 172.21.15.84:0/892588285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab1c107050 con 0x7fab1c1066f0 2024-05-22T22:52:10.857 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.067+0000 7fab20eed700 1 --2- 172.21.15.84:0/892588285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab1c1066f0 0x7fab1c106b10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fab100098a0 tx=0x7fab10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63755ff7d036274 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:10.857 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.067+0000 7fab1b7fe700 1 -- 172.21.15.84:0/892588285 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab10004160 con 0x7fab1c1066f0 2024-05-22T22:52:10.857 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.067+0000 7fab1b7fe700 1 -- 172.21.15.84:0/892588285 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fab10004300 con 0x7fab1c1066f0 2024-05-22T22:52:10.857 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.067+0000 7fab1b7fe700 1 -- 172.21.15.84:0/892588285 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab10011b50 con 0x7fab1c1066f0 2024-05-22T22:52:10.857 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.068+0000 7fab23151700 1 -- 172.21.15.84:0/892588285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab1c1066f0 msgr2=0x7fab1c106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:10.857 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.068+0000 7fab23151700 1 --2- 172.21.15.84:0/892588285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab1c1066f0 0x7fab1c106b10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fab100098a0 tx=0x7fab10009bb0 comp rx=0 tx=0).stop 2024-05-22T22:52:10.857 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.068+0000 7fab23151700 1 -- 172.21.15.84:0/892588285 shutdown_connections 2024-05-22T22:52:10.857 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.068+0000 7fab23151700 1 --2- 172.21.15.84:0/892588285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab1c1066f0 0x7fab1c106b10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:10.857 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.068+0000 7fab23151700 1 -- 172.21.15.84:0/892588285 >> 172.21.15.84:0/892588285 conn(0x7fab1c1022b0 msgr2=0x7fab1c1046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:10.857 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.068+0000 7fab23151700 1 -- 172.21.15.84:0/892588285 shutdown_connections 2024-05-22T22:52:10.857 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.068+0000 7fab23151700 1 -- 172.21.15.84:0/892588285 wait complete. 2024-05-22T22:52:10.858 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.068+0000 7fab23151700 1 Processor -- start 2024-05-22T22:52:10.858 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.068+0000 7fab23151700 1 -- start start 2024-05-22T22:52:10.858 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.069+0000 7fab23151700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab1c1066f0 0x7fab1c19df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:10.858 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.069+0000 7fab23151700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c1080e0 con 0x7fab1c1066f0 2024-05-22T22:52:10.858 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.069+0000 7fab20eed700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab1c1066f0 0x7fab1c19df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:10.858 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.069+0000 7fab20eed700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab1c1066f0 0x7fab1c19df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46876/0 (socket says 172.21.15.84:46876) 2024-05-22T22:52:10.858 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.069+0000 7fab20eed700 1 -- 172.21.15.84:0/876433563 learned_addr learned my addr 172.21.15.84:0/876433563 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:10.858 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.069+0000 7fab20eed700 1 -- 172.21.15.84:0/876433563 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab1c19e470 con 0x7fab1c1066f0 2024-05-22T22:52:10.858 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.069+0000 7fab20eed700 1 --2- 172.21.15.84:0/876433563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab1c1066f0 0x7fab1c19df30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fab10011670 tx=0x7fab10011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:10.858 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.069+0000 7fab19ffb700 1 -- 172.21.15.84:0/876433563 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab100039c0 con 0x7fab1c1066f0 2024-05-22T22:52:10.858 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.069+0000 7fab19ffb700 1 -- 172.21.15.84:0/876433563 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fab10011e50 con 0x7fab1c1066f0 2024-05-22T22:52:10.858 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.069+0000 7fab23151700 1 -- 172.21.15.84:0/876433563 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab1c19e700 con 0x7fab1c1066f0 2024-05-22T22:52:10.858 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.069+0000 7fab19ffb700 1 -- 172.21.15.84:0/876433563 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab1001a5c0 con 0x7fab1c1066f0 2024-05-22T22:52:10.859 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.069+0000 7fab23151700 1 -- 172.21.15.84:0/876433563 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab1c1a13f0 con 0x7fab1c1066f0 2024-05-22T22:52:10.859 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.070+0000 7fab19ffb700 1 -- 172.21.15.84:0/876433563 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fab10024430 con 0x7fab1c1066f0 2024-05-22T22:52:10.859 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.070+0000 7fab19ffb700 1 --2- 172.21.15.84:0/876433563 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7fab0c039000 0x7fab0c03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:10.859 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.070+0000 7fab23151700 1 -- 172.21.15.84:0/876433563 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab1c0639c0 con 0x7fab1c1066f0 2024-05-22T22:52:10.859 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.071+0000 7fab19ffb700 1 -- 172.21.15.84:0/876433563 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fab1004d250 con 0x7fab1c1066f0 2024-05-22T22:52:10.859 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.071+0000 7fab1bfff700 1 --2- 172.21.15.84:0/876433563 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7fab0c039000 0x7fab0c03b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:10.859 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.071+0000 7fab1bfff700 1 --2- 172.21.15.84:0/876433563 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7fab0c039000 0x7fab0c03b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fab08006fd0 tx=0x7fab08006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:10.859 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.075+0000 7fab19ffb700 1 -- 172.21.15.84:0/876433563 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fab10003b60 con 0x7fab1c1066f0 2024-05-22T22:52:10.859 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.269+0000 7fab23151700 1 -- 172.21.15.84:0/876433563 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fab1c02deb0 con 0x7fab1c1066f0 2024-05-22T22:52:10.859 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.270+0000 7fab19ffb700 1 -- 172.21.15.84:0/876433563 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7fab10032190 con 0x7fab1c1066f0 2024-05-22T22:52:10.859 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.272+0000 7fab23151700 1 -- 172.21.15.84:0/876433563 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7fab0c039000 msgr2=0x7fab0c03b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:10.859 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.272+0000 7fab23151700 1 --2- 172.21.15.84:0/876433563 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7fab0c039000 0x7fab0c03b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fab08006fd0 tx=0x7fab08006e40 comp rx=0 tx=0).stop 2024-05-22T22:52:10.859 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.272+0000 7fab23151700 1 -- 172.21.15.84:0/876433563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab1c1066f0 msgr2=0x7fab1c19df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:10.860 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.272+0000 7fab23151700 1 --2- 172.21.15.84:0/876433563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab1c1066f0 0x7fab1c19df30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fab10011670 tx=0x7fab10011b60 comp rx=0 tx=0).stop 2024-05-22T22:52:10.860 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.272+0000 7fab23151700 1 -- 172.21.15.84:0/876433563 shutdown_connections 2024-05-22T22:52:10.860 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.272+0000 7fab23151700 1 --2- 172.21.15.84:0/876433563 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7fab0c039000 0x7fab0c03b4c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:10.860 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.272+0000 7fab23151700 1 --2- 172.21.15.84:0/876433563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab1c1066f0 0x7fab1c19df30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:10.860 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.272+0000 7fab23151700 1 -- 172.21.15.84:0/876433563 >> 172.21.15.84:0/876433563 conn(0x7fab1c1022b0 msgr2=0x7fab1c194b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:10.860 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.272+0000 7fab23151700 1 -- 172.21.15.84:0/876433563 shutdown_connections 2024-05-22T22:52:10.860 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:10.273+0000 7fab23151700 1 -- 172.21.15.84:0/876433563 wait complete. 2024-05-22T22:52:10.860 INFO:teuthology.orchestra.run.smithi084.stdout:Enabling cephadm module... 2024-05-22T22:52:11.114 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:10 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/876433563' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-22T22:52:12.114 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:11 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1523373781' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-22T22:52:12.115 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:11 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: ignoring --setuser ceph since I am not root 2024-05-22T22:52:12.115 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:11 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: ignoring --setgroup ceph since I am not root 2024-05-22T22:52:12.115 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:12 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:12.024+0000 7fdd1cc01200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-22T22:52:12.427 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:12 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:12.213+0000 7fdd1cc01200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-22T22:52:12.427 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:12 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:12.317+0000 7fdd1cc01200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-22T22:52:12.434 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.300+0000 7fdcd4db3700 1 Processor -- start 2024-05-22T22:52:12.434 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.300+0000 7fdcd4db3700 1 -- start start 2024-05-22T22:52:12.434 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.300+0000 7fdcd4db3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd01088d0 0x7fdcd010acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:12.435 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.300+0000 7fdcd4db3700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd007be40 con 0x7fdcd01088d0 2024-05-22T22:52:12.435 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.301+0000 7fdcce59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd01088d0 0x7fdcd010acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:12.435 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.301+0000 7fdcce59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd01088d0 0x7fdcd010acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46890/0 (socket says 172.21.15.84:46890) 2024-05-22T22:52:12.435 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.301+0000 7fdcce59c700 1 -- 172.21.15.84:0/4220161773 learned_addr learned my addr 172.21.15.84:0/4220161773 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:12.435 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.301+0000 7fdcce59c700 1 -- 172.21.15.84:0/4220161773 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd010b200 con 0x7fdcd01088d0 2024-05-22T22:52:12.435 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.301+0000 7fdcce59c700 1 --2- 172.21.15.84:0/4220161773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd01088d0 0x7fdcd010acc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00098a0 tx=0x7fdcc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523b2140071cb772 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:12.435 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.301+0000 7fdccdd9b700 1 -- 172.21.15.84:0/4220161773 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdcc0004160 con 0x7fdcd01088d0 2024-05-22T22:52:12.435 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.301+0000 7fdccdd9b700 1 -- 172.21.15.84:0/4220161773 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdcc0004300 con 0x7fdcd01088d0 2024-05-22T22:52:12.435 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.301+0000 7fdccdd9b700 1 -- 172.21.15.84:0/4220161773 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdcc0011b50 con 0x7fdcd01088d0 2024-05-22T22:52:12.435 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.302+0000 7fdcd4db3700 1 -- 172.21.15.84:0/4220161773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd01088d0 msgr2=0x7fdcd010acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:12.435 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.302+0000 7fdcd4db3700 1 --2- 172.21.15.84:0/4220161773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd01088d0 0x7fdcd010acc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00098a0 tx=0x7fdcc0009bb0 comp rx=0 tx=0).stop 2024-05-22T22:52:12.435 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.302+0000 7fdcd4db3700 1 -- 172.21.15.84:0/4220161773 shutdown_connections 2024-05-22T22:52:12.436 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.302+0000 7fdcd4db3700 1 --2- 172.21.15.84:0/4220161773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd01088d0 0x7fdcd010acc0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:12.436 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.302+0000 7fdcd4db3700 1 -- 172.21.15.84:0/4220161773 >> 172.21.15.84:0/4220161773 conn(0x7fdcd0102280 msgr2=0x7fdcd01046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:12.436 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.302+0000 7fdcd4db3700 1 -- 172.21.15.84:0/4220161773 shutdown_connections 2024-05-22T22:52:12.436 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.302+0000 7fdcd4db3700 1 -- 172.21.15.84:0/4220161773 wait complete. 2024-05-22T22:52:12.436 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.302+0000 7fdcd4db3700 1 Processor -- start 2024-05-22T22:52:12.436 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.303+0000 7fdcd4db3700 1 -- start start 2024-05-22T22:52:12.436 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.303+0000 7fdcd4db3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd01088d0 0x7fdcd019bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:12.436 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.303+0000 7fdcd4db3700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd010c1c0 con 0x7fdcd01088d0 2024-05-22T22:52:12.436 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.303+0000 7fdcce59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd01088d0 0x7fdcd019bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:12.436 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.303+0000 7fdcce59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd01088d0 0x7fdcd019bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46900/0 (socket says 172.21.15.84:46900) 2024-05-22T22:52:12.436 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.303+0000 7fdcce59c700 1 -- 172.21.15.84:0/1523373781 learned_addr learned my addr 172.21.15.84:0/1523373781 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:12.436 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.303+0000 7fdcce59c700 1 -- 172.21.15.84:0/1523373781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd019c2a0 con 0x7fdcd01088d0 2024-05-22T22:52:12.436 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.303+0000 7fdcce59c700 1 --2- 172.21.15.84:0/1523373781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd01088d0 0x7fdcd019bd60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0011670 tx=0x7fdcc0011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:12.437 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.303+0000 7fdcc77fe700 1 -- 172.21.15.84:0/1523373781 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdcc00039c0 con 0x7fdcd01088d0 2024-05-22T22:52:12.437 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.303+0000 7fdcc77fe700 1 -- 172.21.15.84:0/1523373781 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdcc0011e50 con 0x7fdcd01088d0 2024-05-22T22:52:12.437 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.303+0000 7fdcc77fe700 1 -- 172.21.15.84:0/1523373781 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdcc001a5c0 con 0x7fdcd01088d0 2024-05-22T22:52:12.437 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.303+0000 7fdcd4db3700 1 -- 172.21.15.84:0/1523373781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd019c530 con 0x7fdcd01088d0 2024-05-22T22:52:12.437 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.303+0000 7fdcd4db3700 1 -- 172.21.15.84:0/1523373781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd019f220 con 0x7fdcd01088d0 2024-05-22T22:52:12.437 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.304+0000 7fdcc77fe700 1 -- 172.21.15.84:0/1523373781 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fdcc0024430 con 0x7fdcd01088d0 2024-05-22T22:52:12.437 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.304+0000 7fdcc77fe700 1 --2- 172.21.15.84:0/1523373781 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7fdcbc039000 0x7fdcbc03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:12.437 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.304+0000 7fdcc77fe700 1 -- 172.21.15.84:0/1523373781 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdcc004d440 con 0x7fdcd01088d0 2024-05-22T22:52:12.437 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.304+0000 7fdcd4db3700 1 -- 172.21.15.84:0/1523373781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcd00639c0 con 0x7fdcd01088d0 2024-05-22T22:52:12.437 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.304+0000 7fdcc7fff700 1 --2- 172.21.15.84:0/1523373781 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7fdcbc039000 0x7fdcbc03b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:12.437 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.305+0000 7fdcc7fff700 1 --2- 172.21.15.84:0/1523373781 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7fdcbc039000 0x7fdcbc03b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8006fd0 tx=0x7fdcb8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:12.437 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.309+0000 7fdcc77fe700 1 -- 172.21.15.84:0/1523373781 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdcc004ce60 con 0x7fdcd01088d0 2024-05-22T22:52:12.437 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.559+0000 7fdcd4db3700 1 -- 172.21.15.84:0/1523373781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fdcd019f3f0 con 0x7fdcd01088d0 2024-05-22T22:52:12.438 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.856+0000 7fdcc77fe700 1 -- 172.21.15.84:0/1523373781 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fdcc0032030 con 0x7fdcd01088d0 2024-05-22T22:52:12.438 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.857+0000 7fdcc77fe700 1 -- 172.21.15.84:0/1523373781 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fdcc002d910 con 0x7fdcd01088d0 2024-05-22T22:52:12.438 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.859+0000 7fdcd4db3700 1 -- 172.21.15.84:0/1523373781 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7fdcbc039000 msgr2=0x7fdcbc03b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:12.438 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.859+0000 7fdcd4db3700 1 --2- 172.21.15.84:0/1523373781 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7fdcbc039000 0x7fdcbc03b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8006fd0 tx=0x7fdcb8006e40 comp rx=0 tx=0).stop 2024-05-22T22:52:12.438 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.859+0000 7fdcd4db3700 1 -- 172.21.15.84:0/1523373781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd01088d0 msgr2=0x7fdcd019bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:12.438 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.859+0000 7fdcd4db3700 1 --2- 172.21.15.84:0/1523373781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd01088d0 0x7fdcd019bd60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0011670 tx=0x7fdcc0011b60 comp rx=0 tx=0).stop 2024-05-22T22:52:12.438 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.859+0000 7fdcd4db3700 1 -- 172.21.15.84:0/1523373781 shutdown_connections 2024-05-22T22:52:12.438 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.859+0000 7fdcd4db3700 1 --2- 172.21.15.84:0/1523373781 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7fdcbc039000 0x7fdcbc03b4c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:12.438 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.859+0000 7fdcd4db3700 1 --2- 172.21.15.84:0/1523373781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd01088d0 0x7fdcd019bd60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:12.438 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.859+0000 7fdcd4db3700 1 -- 172.21.15.84:0/1523373781 >> 172.21.15.84:0/1523373781 conn(0x7fdcd0102280 msgr2=0x7fdcd0102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:12.438 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.860+0000 7fdcd4db3700 1 -- 172.21.15.84:0/1523373781 shutdown_connections 2024-05-22T22:52:12.438 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:11.860+0000 7fdcd4db3700 1 -- 172.21.15.84:0/1523373781 wait complete. 2024-05-22T22:52:12.723 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:12 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:12.638+0000 7fdd1cc01200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-22T22:52:12.723 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:12 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:12.731+0000 7fdd1cc01200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-22T22:52:13.060 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:12 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1523373781' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-22T22:52:13.060 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:12 smithi084 ceph-mon[32186]: mgrmap e5: x(active, since 6s) 2024-05-22T22:52:13.712 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout { 2024-05-22T22:52:13.712 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-22T22:52:13.712 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-22T22:52:13.712 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-22T22:52:13.712 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-22T22:52:13.712 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout } 2024-05-22T22:52:13.712 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.863+0000 7f6460abf700 1 Processor -- start 2024-05-22T22:52:13.712 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.863+0000 7f6460abf700 1 -- start start 2024-05-22T22:52:13.712 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.864+0000 7f6460abf700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f645c108840 0x7f645c10ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:13.712 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.864+0000 7f6460abf700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f645c07be80 con 0x7f645c108840 2024-05-22T22:52:13.713 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.864+0000 7f645a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f645c108840 0x7f645c10ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:13.713 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.864+0000 7f645a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f645c108840 0x7f645c10ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43324/0 (socket says 172.21.15.84:43324) 2024-05-22T22:52:13.713 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.864+0000 7f645a59c700 1 -- 172.21.15.84:0/3887676433 learned_addr learned my addr 172.21.15.84:0/3887676433 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:13.713 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.864+0000 7f645a59c700 1 -- 172.21.15.84:0/3887676433 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c10b170 con 0x7f645c108840 2024-05-22T22:52:13.713 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.864+0000 7f645a59c700 1 --2- 172.21.15.84:0/3887676433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f645c108840 0x7f645c10ac30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f644c0098a0 tx=0x7f644c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9296d84e104cd274 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:13.713 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.864+0000 7f645959a700 1 -- 172.21.15.84:0/3887676433 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f644c004160 con 0x7f645c108840 2024-05-22T22:52:13.713 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.865+0000 7f645959a700 1 -- 172.21.15.84:0/3887676433 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f644c004300 con 0x7f645c108840 2024-05-22T22:52:13.713 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.865+0000 7f645959a700 1 -- 172.21.15.84:0/3887676433 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f644c011b50 con 0x7f645c108840 2024-05-22T22:52:13.713 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.865+0000 7f6460abf700 1 -- 172.21.15.84:0/3887676433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f645c108840 msgr2=0x7f645c10ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:13.713 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.865+0000 7f6460abf700 1 --2- 172.21.15.84:0/3887676433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f645c108840 0x7f645c10ac30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f644c0098a0 tx=0x7f644c009bb0 comp rx=0 tx=0).stop 2024-05-22T22:52:13.713 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.865+0000 7f6460abf700 1 -- 172.21.15.84:0/3887676433 shutdown_connections 2024-05-22T22:52:13.713 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.865+0000 7f6460abf700 1 --2- 172.21.15.84:0/3887676433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f645c108840 0x7f645c10ac30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:13.713 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.865+0000 7f6460abf700 1 -- 172.21.15.84:0/3887676433 >> 172.21.15.84:0/3887676433 conn(0x7f645c102210 msgr2=0x7f645c104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:13.713 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.865+0000 7f6460abf700 1 -- 172.21.15.84:0/3887676433 shutdown_connections 2024-05-22T22:52:13.714 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.865+0000 7f6460abf700 1 -- 172.21.15.84:0/3887676433 wait complete. 2024-05-22T22:52:13.714 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.866+0000 7f6460abf700 1 Processor -- start 2024-05-22T22:52:13.714 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.866+0000 7f6460abf700 1 -- start start 2024-05-22T22:52:13.714 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.866+0000 7f6460abf700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f645c108840 0x7f645c1080c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:13.714 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.866+0000 7f6460abf700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f645c10c190 con 0x7f645c108840 2024-05-22T22:52:13.714 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.866+0000 7f645a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f645c108840 0x7f645c1080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:13.714 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.866+0000 7f645a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f645c108840 0x7f645c1080c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43332/0 (socket says 172.21.15.84:43332) 2024-05-22T22:52:13.714 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.866+0000 7f645a59c700 1 -- 172.21.15.84:0/361911122 learned_addr learned my addr 172.21.15.84:0/361911122 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:13.714 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.867+0000 7f645a59c700 1 -- 172.21.15.84:0/361911122 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c108600 con 0x7f645c108840 2024-05-22T22:52:13.714 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.867+0000 7f645a59c700 1 --2- 172.21.15.84:0/361911122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f645c108840 0x7f645c1080c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f644c011670 tx=0x7f644c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:13.714 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.867+0000 7f644b7fe700 1 -- 172.21.15.84:0/361911122 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f644c0039c0 con 0x7f645c108840 2024-05-22T22:52:13.714 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.867+0000 7f644b7fe700 1 -- 172.21.15.84:0/361911122 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f644c011e50 con 0x7f645c108840 2024-05-22T22:52:13.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.867+0000 7f6460abf700 1 -- 172.21.15.84:0/361911122 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645c1067f0 con 0x7f645c108840 2024-05-22T22:52:13.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.867+0000 7f644b7fe700 1 -- 172.21.15.84:0/361911122 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f644c01a5c0 con 0x7f645c108840 2024-05-22T22:52:13.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.867+0000 7f6460abf700 1 -- 172.21.15.84:0/361911122 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f645c106d30 con 0x7f645c108840 2024-05-22T22:52:13.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.868+0000 7f644b7fe700 1 -- 172.21.15.84:0/361911122 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f644c01a870 con 0x7f645c108840 2024-05-22T22:52:13.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.868+0000 7f6460abf700 1 -- 172.21.15.84:0/361911122 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f645c0639c0 con 0x7f645c108840 2024-05-22T22:52:13.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.868+0000 7f644b7fe700 1 --2- 172.21.15.84:0/361911122 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7f6444039050 0x7f644403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:13.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.868+0000 7f644b7fe700 1 -- 172.21.15.84:0/361911122 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f644c04d260 con 0x7f645c108840 2024-05-22T22:52:13.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.868+0000 7f6459d9b700 1 -- 172.21.15.84:0/361911122 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7f6444039050 msgr2=0x7f644403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/4029123070 2024-05-22T22:52:13.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.868+0000 7f6459d9b700 1 --2- 172.21.15.84:0/361911122 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7f6444039050 0x7f644403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-22T22:52:13.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:12.873+0000 7f644b7fe700 1 -- 172.21.15.84:0/361911122 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f644c019c70 con 0x7f645c108840 2024-05-22T22:52:13.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:13.069+0000 7f6459d9b700 1 -- 172.21.15.84:0/361911122 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7f6444039050 msgr2=0x7f644403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/4029123070 2024-05-22T22:52:13.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:13.069+0000 7f6459d9b700 1 --2- 172.21.15.84:0/361911122 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7f6444039050 0x7f644403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-22T22:52:13.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:13.150+0000 7f6460abf700 1 -- 172.21.15.84:0/361911122 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f645c107660 con 0x7f645c108840 2024-05-22T22:52:13.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:13.150+0000 7f644b7fe700 1 -- 172.21.15.84:0/361911122 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f644c003b60 con 0x7f645c108840 2024-05-22T22:52:13.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:13.153+0000 7f6460abf700 1 -- 172.21.15.84:0/361911122 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7f6444039050 msgr2=0x7f644403b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-22T22:52:13.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:13.153+0000 7f6460abf700 1 --2- 172.21.15.84:0/361911122 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7f6444039050 0x7f644403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:13.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:13.153+0000 7f6460abf700 1 -- 172.21.15.84:0/361911122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f645c108840 msgr2=0x7f645c1080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:13.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:13.153+0000 7f6460abf700 1 --2- 172.21.15.84:0/361911122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f645c108840 0x7f645c1080c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f644c011670 tx=0x7f644c011b60 comp rx=0 tx=0).stop 2024-05-22T22:52:13.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:13.153+0000 7f6460abf700 1 -- 172.21.15.84:0/361911122 shutdown_connections 2024-05-22T22:52:13.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:13.153+0000 7f6460abf700 1 --2- 172.21.15.84:0/361911122 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7f6444039050 0x7f644403b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:13.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:13.153+0000 7f6460abf700 1 --2- 172.21.15.84:0/361911122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f645c108840 0x7f645c1080c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:13.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:13.153+0000 7f6460abf700 1 -- 172.21.15.84:0/361911122 >> 172.21.15.84:0/361911122 conn(0x7f645c102210 msgr2=0x7f645c102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:13.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:13.153+0000 7f6460abf700 1 -- 172.21.15.84:0/361911122 shutdown_connections 2024-05-22T22:52:13.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:13.153+0000 7f6460abf700 1 -- 172.21.15.84:0/361911122 wait complete. 2024-05-22T22:52:13.716 INFO:teuthology.orchestra.run.smithi084.stdout:Waiting for the mgr to restart... 2024-05-22T22:52:13.716 INFO:teuthology.orchestra.run.smithi084.stdout:Waiting for mgr epoch 5... 2024-05-22T22:52:14.097 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:13 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/361911122' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-22T22:52:14.098 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:13 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:13.929+0000 7fdd1cc01200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-22T22:52:14.098 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:14 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:14.106+0000 7fdd1cc01200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-22T22:52:14.352 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:14 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:14.276+0000 7fdd1cc01200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-22T22:52:14.615 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:14 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:14.562+0000 7fdd1cc01200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-22T22:52:15.035 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:14 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:14.653+0000 7fdd1cc01200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-22T22:52:15.035 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:14 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:14.853+0000 7fdd1cc01200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-22T22:52:15.364 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:15 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:15.043+0000 7fdd1cc01200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-22T22:52:15.864 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:15 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:15.632+0000 7fdd1cc01200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-22T22:52:16.331 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:16 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:16.047+0000 7fdd1cc01200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-22T22:52:16.331 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:16 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:16.142+0000 7fdd1cc01200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-22T22:52:16.331 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:16 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:16.339+0000 7fdd1cc01200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-22T22:52:17.115 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:16 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:16.851+0000 7fdd1cc01200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-22T22:52:17.115 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:16 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:16.941+0000 7fdd1cc01200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-22T22:52:17.509 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:17 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:17.180+0000 7fdd1cc01200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-22T22:52:17.509 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:17 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:17.310+0000 7fdd1cc01200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-22T22:52:17.509 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:17 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:17.403+0000 7fdd1cc01200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-22T22:52:17.509 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:17 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:17.517+0000 7fdd1cc01200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-22T22:52:18.114 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:17 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:17.811+0000 7fdd1cc01200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-22T22:52:18.657 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:18 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:18.400+0000 7fdd1cc01200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-22T22:52:18.658 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:18 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:18.613+0000 7fdd1cc01200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-22T22:52:19.115 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:18 smithi084 ceph-mon[32186]: Active manager daemon x restarted 2024-05-22T22:52:19.115 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:18 smithi084 ceph-mon[32186]: Activating manager daemon x 2024-05-22T22:52:19.115 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:18 smithi084 ceph-mon[32186]: osdmap e2: 0 total, 0 up, 0 in 2024-05-22T22:52:19.115 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:18 smithi084 ceph-mon[32186]: mgrmap e6: x(active, starting, since 0.00351335s) 2024-05-22T22:52:19.115 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:18 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-22T22:52:19.115 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:18 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-22T22:52:19.115 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:18 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-22T22:52:19.115 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:18 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-22T22:52:19.115 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:18 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-22T22:52:19.116 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:18 smithi084 ceph-mon[32186]: Manager daemon x is now available 2024-05-22T22:52:19.116 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:18 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' 2024-05-22T22:52:19.116 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:18 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' 2024-05-22T22:52:19.116 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:18 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:52:19.116 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:18 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:52:19.116 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:18 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-22T22:52:19.116 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:18 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-22T22:52:19.865 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:19 smithi084 ceph-mon[32186]: Found migration_current of "None". Setting to last migration. 2024-05-22T22:52:19.865 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:19 smithi084 ceph-mon[32186]: mgrmap e7: x(active, since 1.0061s) 2024-05-22T22:52:20.215 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout { 2024-05-22T22:52:20.215 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-22T22:52:20.215 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-22T22:52:20.215 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout } 2024-05-22T22:52:20.215 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.155+0000 7ff2b9963700 1 Processor -- start 2024-05-22T22:52:20.215 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.155+0000 7ff2b9963700 1 -- start start 2024-05-22T22:52:20.215 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.156+0000 7ff2b9963700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b4108820 0x7ff2b410ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:20.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.156+0000 7ff2b9963700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b407bea0 con 0x7ff2b4108820 2024-05-22T22:52:20.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.156+0000 7ff2b2ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b4108820 0x7ff2b410ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:20.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.156+0000 7ff2b2ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b4108820 0x7ff2b410ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43334/0 (socket says 172.21.15.84:43334) 2024-05-22T22:52:20.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.156+0000 7ff2b2ffd700 1 -- 172.21.15.84:0/3047532297 learned_addr learned my addr 172.21.15.84:0/3047532297 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:20.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.156+0000 7ff2b2ffd700 1 -- 172.21.15.84:0/3047532297 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b410b150 con 0x7ff2b4108820 2024-05-22T22:52:20.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.156+0000 7ff2b2ffd700 1 --2- 172.21.15.84:0/3047532297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b4108820 0x7ff2b410ac10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40098a0 tx=0x7ff2a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708439a42fb95110 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:20.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.156+0000 7ff2b1ffb700 1 -- 172.21.15.84:0/3047532297 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2a4004160 con 0x7ff2b4108820 2024-05-22T22:52:20.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.156+0000 7ff2b1ffb700 1 -- 172.21.15.84:0/3047532297 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff2a4004300 con 0x7ff2b4108820 2024-05-22T22:52:20.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.156+0000 7ff2b1ffb700 1 -- 172.21.15.84:0/3047532297 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2a4011b50 con 0x7ff2b4108820 2024-05-22T22:52:20.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.157+0000 7ff2b9963700 1 -- 172.21.15.84:0/3047532297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b4108820 msgr2=0x7ff2b410ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:20.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.157+0000 7ff2b9963700 1 --2- 172.21.15.84:0/3047532297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b4108820 0x7ff2b410ac10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40098a0 tx=0x7ff2a4009bb0 comp rx=0 tx=0).stop 2024-05-22T22:52:20.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.157+0000 7ff2b9963700 1 -- 172.21.15.84:0/3047532297 shutdown_connections 2024-05-22T22:52:20.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.157+0000 7ff2b9963700 1 --2- 172.21.15.84:0/3047532297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b4108820 0x7ff2b410ac10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:20.217 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.157+0000 7ff2b9963700 1 -- 172.21.15.84:0/3047532297 >> 172.21.15.84:0/3047532297 conn(0x7ff2b4102210 msgr2=0x7ff2b4104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:20.217 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.157+0000 7ff2b9963700 1 -- 172.21.15.84:0/3047532297 shutdown_connections 2024-05-22T22:52:20.217 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.157+0000 7ff2b9963700 1 -- 172.21.15.84:0/3047532297 wait complete. 2024-05-22T22:52:20.217 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.158+0000 7ff2b9963700 1 Processor -- start 2024-05-22T22:52:20.217 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.158+0000 7ff2b9963700 1 -- start start 2024-05-22T22:52:20.217 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.158+0000 7ff2b9963700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b4108820 0x7ff2b41080a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:20.217 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.158+0000 7ff2b9963700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b410c100 con 0x7ff2b4108820 2024-05-22T22:52:20.217 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.158+0000 7ff2b2ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b4108820 0x7ff2b41080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:20.217 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.158+0000 7ff2b2ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b4108820 0x7ff2b41080a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43350/0 (socket says 172.21.15.84:43350) 2024-05-22T22:52:20.217 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.158+0000 7ff2b2ffd700 1 -- 172.21.15.84:0/4000324278 learned_addr learned my addr 172.21.15.84:0/4000324278 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:20.217 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.159+0000 7ff2b2ffd700 1 -- 172.21.15.84:0/4000324278 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b41085e0 con 0x7ff2b4108820 2024-05-22T22:52:20.217 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.159+0000 7ff2b2ffd700 1 --2- 172.21.15.84:0/4000324278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b4108820 0x7ff2b41080a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4011b60 tx=0x7ff2a4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:20.217 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.159+0000 7ff2b8961700 1 -- 172.21.15.84:0/4000324278 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2a4003b20 con 0x7ff2b4108820 2024-05-22T22:52:20.218 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.159+0000 7ff2b9963700 1 -- 172.21.15.84:0/4000324278 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b41067d0 con 0x7ff2b4108820 2024-05-22T22:52:20.218 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.159+0000 7ff2b8961700 1 -- 172.21.15.84:0/4000324278 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff2a4003cc0 con 0x7ff2b4108820 2024-05-22T22:52:20.218 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.159+0000 7ff2b8961700 1 -- 172.21.15.84:0/4000324278 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2a4024840 con 0x7ff2b4108820 2024-05-22T22:52:20.218 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.159+0000 7ff2b9963700 1 -- 172.21.15.84:0/4000324278 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b4106d10 con 0x7ff2b4108820 2024-05-22T22:52:20.218 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.160+0000 7ff2b8961700 1 -- 172.21.15.84:0/4000324278 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7ff2a4022020 con 0x7ff2b4108820 2024-05-22T22:52:20.218 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.160+0000 7ff2b8961700 1 --2- 172.21.15.84:0/4000324278 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7ff29c0390a0 0x7ff29c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:20.218 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.160+0000 7ff2b9963700 1 -- 172.21.15.84:0/4000324278 --> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff2b40639c0 con 0x7ff29c0390a0 2024-05-22T22:52:20.218 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.160+0000 7ff2b8961700 1 -- 172.21.15.84:0/4000324278 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff2a4011e00 con 0x7ff2b4108820 2024-05-22T22:52:20.218 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.160+0000 7ff2b27fc700 1 -- 172.21.15.84:0/4000324278 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7ff29c0390a0 msgr2=0x7ff29c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/4029123070 2024-05-22T22:52:20.218 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.160+0000 7ff2b27fc700 1 --2- 172.21.15.84:0/4000324278 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7ff29c0390a0 0x7ff29c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-22T22:52:20.218 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.361+0000 7ff2b27fc700 1 -- 172.21.15.84:0/4000324278 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7ff29c0390a0 msgr2=0x7ff29c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/4029123070 2024-05-22T22:52:20.218 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.361+0000 7ff2b27fc700 1 --2- 172.21.15.84:0/4000324278 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7ff29c0390a0 0x7ff29c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-22T22:52:20.218 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.762+0000 7ff2b27fc700 1 -- 172.21.15.84:0/4000324278 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7ff29c0390a0 msgr2=0x7ff29c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/4029123070 2024-05-22T22:52:20.218 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:14.762+0000 7ff2b27fc700 1 --2- 172.21.15.84:0/4000324278 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7ff29c0390a0 0x7ff29c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-22T22:52:20.219 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:15.563+0000 7ff2b27fc700 1 -- 172.21.15.84:0/4000324278 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7ff29c0390a0 msgr2=0x7ff29c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/4029123070 2024-05-22T22:52:20.219 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:15.563+0000 7ff2b27fc700 1 --2- 172.21.15.84:0/4000324278 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7ff29c0390a0 0x7ff29c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-22T22:52:20.219 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:17.164+0000 7ff2b27fc700 1 -- 172.21.15.84:0/4000324278 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7ff29c0390a0 msgr2=0x7ff29c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/4029123070 2024-05-22T22:52:20.219 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:17.164+0000 7ff2b27fc700 1 --2- 172.21.15.84:0/4000324278 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7ff29c0390a0 0x7ff29c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-22T22:52:20.219 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:18.617+0000 7ff2b8961700 1 -- 172.21.15.84:0/4000324278 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7ff2a402d910 con 0x7ff2b4108820 2024-05-22T22:52:20.219 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:18.617+0000 7ff2b8961700 1 -- 172.21.15.84:0/4000324278 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7ff29c0390a0 msgr2=0x7ff29c03b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-22T22:52:20.219 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:18.617+0000 7ff2b8961700 1 --2- 172.21.15.84:0/4000324278 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7ff29c0390a0 0x7ff29c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:20.219 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:19.619+0000 7ff2b8961700 1 -- 172.21.15.84:0/4000324278 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7ff2a404eb70 con 0x7ff2b4108820 2024-05-22T22:52:20.219 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:19.619+0000 7ff2b8961700 1 --2- 172.21.15.84:0/4000324278 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7ff29c03c7e0 0x7ff29c03ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:20.219 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:19.619+0000 7ff2b8961700 1 -- 172.21.15.84:0/4000324278 --> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff2a402be50 con 0x7ff29c03c7e0 2024-05-22T22:52:20.219 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:19.619+0000 7ff2b27fc700 1 --2- 172.21.15.84:0/4000324278 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7ff29c03c7e0 0x7ff29c03ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:20.219 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:19.620+0000 7ff2b27fc700 1 --2- 172.21.15.84:0/4000324278 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7ff29c03c7e0 0x7ff29c03ebd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8003a10 tx=0x7ff2a80092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:20.219 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:19.620+0000 7ff2b8961700 1 -- 172.21.15.84:0/4000324278 <== mgr.14120 v2:172.21.15.84:6800/3278959854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7ff2a402be50 con 0x7ff29c03c7e0 2024-05-22T22:52:20.220 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:19.632+0000 7ff2b9963700 1 -- 172.21.15.84:0/4000324278 --> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff2b4103430 con 0x7ff29c03c7e0 2024-05-22T22:52:20.220 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:19.632+0000 7ff2b8961700 1 -- 172.21.15.84:0/4000324278 <== mgr.14120 v2:172.21.15.84:6800/3278959854 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7ff2b4103430 con 0x7ff29c03c7e0 2024-05-22T22:52:20.220 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:19.632+0000 7ff2b9963700 1 -- 172.21.15.84:0/4000324278 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7ff29c03c7e0 msgr2=0x7ff29c03ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:20.220 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:19.632+0000 7ff2b9963700 1 --2- 172.21.15.84:0/4000324278 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7ff29c03c7e0 0x7ff29c03ebd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8003a10 tx=0x7ff2a80092b0 comp rx=0 tx=0).stop 2024-05-22T22:52:20.220 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:19.632+0000 7ff2b9963700 1 -- 172.21.15.84:0/4000324278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b4108820 msgr2=0x7ff2b41080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:20.220 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:19.632+0000 7ff2b9963700 1 --2- 172.21.15.84:0/4000324278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b4108820 0x7ff2b41080a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4011b60 tx=0x7ff2a4011c40 comp rx=0 tx=0).stop 2024-05-22T22:52:20.220 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:19.632+0000 7ff2b9963700 1 -- 172.21.15.84:0/4000324278 shutdown_connections 2024-05-22T22:52:20.220 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:19.632+0000 7ff2b9963700 1 --2- 172.21.15.84:0/4000324278 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7ff29c03c7e0 0x7ff29c03ebd0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:20.220 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:19.632+0000 7ff2b9963700 1 --2- 172.21.15.84:0/4000324278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b4108820 0x7ff2b41080a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:20.220 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:19.632+0000 7ff2b9963700 1 --2- 172.21.15.84:0/4000324278 >> [v2:172.21.15.84:6800/4029123070,v1:172.21.15.84:6801/4029123070] conn(0x7ff29c0390a0 0x7ff29c03b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:20.220 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:19.632+0000 7ff2b9963700 1 -- 172.21.15.84:0/4000324278 >> 172.21.15.84:0/4000324278 conn(0x7ff2b4102210 msgr2=0x7ff2b4102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:20.220 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:19.632+0000 7ff2b9963700 1 -- 172.21.15.84:0/4000324278 shutdown_connections 2024-05-22T22:52:20.220 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:19.632+0000 7ff2b9963700 1 -- 172.21.15.84:0/4000324278 wait complete. 2024-05-22T22:52:20.221 INFO:teuthology.orchestra.run.smithi084.stdout:mgr epoch 5 is available 2024-05-22T22:52:20.221 INFO:teuthology.orchestra.run.smithi084.stdout:Setting orchestrator backend to cephadm... 2024-05-22T22:52:20.865 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:20 smithi084 ceph-mon[32186]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-22T22:52:20.865 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:20 smithi084 ceph-mon[32186]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-22T22:52:20.865 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:20 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' 2024-05-22T22:52:20.865 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:20 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' 2024-05-22T22:52:21.478 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.661+0000 7fa999953700 1 Processor -- start 2024-05-22T22:52:21.478 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.661+0000 7fa999953700 1 -- start start 2024-05-22T22:52:21.478 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.661+0000 7fa999953700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa994106690 0x7fa994106ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:21.478 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.661+0000 7fa999953700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa99407be80 con 0x7fa994106690 2024-05-22T22:52:21.478 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.661+0000 7fa992ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa994106690 0x7fa994106ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:21.478 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.661+0000 7fa992ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa994106690 0x7fa994106ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43424/0 (socket says 172.21.15.84:43424) 2024-05-22T22:52:21.479 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.661+0000 7fa992ffd700 1 -- 172.21.15.84:0/3408204579 learned_addr learned my addr 172.21.15.84:0/3408204579 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:21.479 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.661+0000 7fa992ffd700 1 -- 172.21.15.84:0/3408204579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa994106ff0 con 0x7fa994106690 2024-05-22T22:52:21.479 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.661+0000 7fa992ffd700 1 --2- 172.21.15.84:0/3408204579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa994106690 0x7fa994106ab0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa9880098a0 tx=0x7fa988009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f7471196db390a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:21.479 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.662+0000 7fa991ffb700 1 -- 172.21.15.84:0/3408204579 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa988004160 con 0x7fa994106690 2024-05-22T22:52:21.479 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.662+0000 7fa991ffb700 1 -- 172.21.15.84:0/3408204579 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa988004300 con 0x7fa994106690 2024-05-22T22:52:21.479 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.662+0000 7fa991ffb700 1 -- 172.21.15.84:0/3408204579 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa988011b50 con 0x7fa994106690 2024-05-22T22:52:21.479 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.662+0000 7fa999953700 1 -- 172.21.15.84:0/3408204579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa994106690 msgr2=0x7fa994106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:21.479 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.662+0000 7fa999953700 1 --2- 172.21.15.84:0/3408204579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa994106690 0x7fa994106ab0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa9880098a0 tx=0x7fa988009bb0 comp rx=0 tx=0).stop 2024-05-22T22:52:21.479 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.662+0000 7fa999953700 1 -- 172.21.15.84:0/3408204579 shutdown_connections 2024-05-22T22:52:21.479 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.662+0000 7fa999953700 1 --2- 172.21.15.84:0/3408204579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa994106690 0x7fa994106ab0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:21.479 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.662+0000 7fa999953700 1 -- 172.21.15.84:0/3408204579 >> 172.21.15.84:0/3408204579 conn(0x7fa9941022b0 msgr2=0x7fa9941046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:21.479 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.662+0000 7fa999953700 1 -- 172.21.15.84:0/3408204579 shutdown_connections 2024-05-22T22:52:21.479 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.662+0000 7fa999953700 1 -- 172.21.15.84:0/3408204579 wait complete. 2024-05-22T22:52:21.480 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.663+0000 7fa999953700 1 Processor -- start 2024-05-22T22:52:21.480 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.663+0000 7fa999953700 1 -- start start 2024-05-22T22:52:21.480 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.663+0000 7fa999953700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa994106690 0x7fa99419df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:21.480 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.663+0000 7fa999953700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa994108080 con 0x7fa994106690 2024-05-22T22:52:21.480 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.663+0000 7fa992ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa994106690 0x7fa99419df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:21.480 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.663+0000 7fa992ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa994106690 0x7fa99419df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43432/0 (socket says 172.21.15.84:43432) 2024-05-22T22:52:21.480 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.663+0000 7fa992ffd700 1 -- 172.21.15.84:0/2639033290 learned_addr learned my addr 172.21.15.84:0/2639033290 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:21.480 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.664+0000 7fa992ffd700 1 -- 172.21.15.84:0/2639033290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99419e4a0 con 0x7fa994106690 2024-05-22T22:52:21.480 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.664+0000 7fa992ffd700 1 --2- 172.21.15.84:0/2639033290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa994106690 0x7fa99419df60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa988011670 tx=0x7fa988011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:21.480 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.664+0000 7fa998951700 1 -- 172.21.15.84:0/2639033290 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9880039c0 con 0x7fa994106690 2024-05-22T22:52:21.480 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.664+0000 7fa998951700 1 -- 172.21.15.84:0/2639033290 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa988011e50 con 0x7fa994106690 2024-05-22T22:52:21.480 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.664+0000 7fa998951700 1 -- 172.21.15.84:0/2639033290 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa98801a5c0 con 0x7fa994106690 2024-05-22T22:52:21.480 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.665+0000 7fa999953700 1 -- 172.21.15.84:0/2639033290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa99419e730 con 0x7fa994106690 2024-05-22T22:52:21.480 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.665+0000 7fa999953700 1 -- 172.21.15.84:0/2639033290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9941a1420 con 0x7fa994106690 2024-05-22T22:52:21.481 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.666+0000 7fa998951700 1 -- 172.21.15.84:0/2639033290 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fa988024430 con 0x7fa994106690 2024-05-22T22:52:21.481 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.666+0000 7fa999953700 1 -- 172.21.15.84:0/2639033290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9940639c0 con 0x7fa994106690 2024-05-22T22:52:21.481 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.666+0000 7fa998951700 1 --2- 172.21.15.84:0/2639033290 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fa97c038f40 0x7fa97c03b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:21.481 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.666+0000 7fa998951700 1 -- 172.21.15.84:0/2639033290 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa98804d270 con 0x7fa994106690 2024-05-22T22:52:21.481 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.666+0000 7fa9927fc700 1 --2- 172.21.15.84:0/2639033290 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fa97c038f40 0x7fa97c03b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:21.481 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.666+0000 7fa9927fc700 1 --2- 172.21.15.84:0/2639033290 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fa97c038f40 0x7fa97c03b400 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa984006fd0 tx=0x7fa984006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:21.481 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.671+0000 7fa998951700 1 -- 172.21.15.84:0/2639033290 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa988003b60 con 0x7fa994106690 2024-05-22T22:52:21.481 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.737+0000 7fa998951700 1 -- 172.21.15.84:0/2639033290 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa98802d3f0 con 0x7fa994106690 2024-05-22T22:52:21.481 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.891+0000 7fa999953700 1 -- 172.21.15.84:0/2639033290 --> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fa9940008d0 con 0x7fa97c038f40 2024-05-22T22:52:21.481 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.898+0000 7fa998951700 1 -- 172.21.15.84:0/2639033290 <== mgr.14120 v2:172.21.15.84:6800/3278959854 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa9940008d0 con 0x7fa97c038f40 2024-05-22T22:52:21.481 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.900+0000 7fa999953700 1 -- 172.21.15.84:0/2639033290 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fa97c038f40 msgr2=0x7fa97c03b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:21.481 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.900+0000 7fa999953700 1 --2- 172.21.15.84:0/2639033290 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fa97c038f40 0x7fa97c03b400 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa984006fd0 tx=0x7fa984006e40 comp rx=0 tx=0).stop 2024-05-22T22:52:21.481 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.901+0000 7fa999953700 1 -- 172.21.15.84:0/2639033290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa994106690 msgr2=0x7fa99419df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:21.482 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.901+0000 7fa999953700 1 --2- 172.21.15.84:0/2639033290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa994106690 0x7fa99419df60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa988011670 tx=0x7fa988011b60 comp rx=0 tx=0).stop 2024-05-22T22:52:21.482 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.901+0000 7fa999953700 1 -- 172.21.15.84:0/2639033290 shutdown_connections 2024-05-22T22:52:21.482 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.901+0000 7fa999953700 1 --2- 172.21.15.84:0/2639033290 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fa97c038f40 0x7fa97c03b400 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:21.482 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.901+0000 7fa999953700 1 --2- 172.21.15.84:0/2639033290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa994106690 0x7fa99419df60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:21.482 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.901+0000 7fa999953700 1 -- 172.21.15.84:0/2639033290 >> 172.21.15.84:0/2639033290 conn(0x7fa9941022b0 msgr2=0x7fa994194b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:21.483 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.901+0000 7fa999953700 1 -- 172.21.15.84:0/2639033290 shutdown_connections 2024-05-22T22:52:21.485 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:20.901+0000 7fa999953700 1 -- 172.21.15.84:0/2639033290 wait complete. 2024-05-22T22:52:22.093 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:21 smithi084 ceph-mon[32186]: mgrmap e8: x(active, since 2s) 2024-05-22T22:52:22.093 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:21 smithi084 ceph-mon[32186]: [22/May/2024:22:52:20] ENGINE Bus STARTING 2024-05-22T22:52:22.093 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:21 smithi084 ceph-mon[32186]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:52:22.093 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:21 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' 2024-05-22T22:52:22.093 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:21 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:52:22.093 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:21 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:52:22.668 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-22T22:52:22.668 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.892+0000 7f27e099f700 1 Processor -- start 2024-05-22T22:52:22.668 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.892+0000 7f27e099f700 1 -- start start 2024-05-22T22:52:22.668 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.892+0000 7f27e099f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27dc105ed0 0x7f27dc1062f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:22.668 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.892+0000 7f27e099f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc1068c0 con 0x7f27dc105ed0 2024-05-22T22:52:22.668 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.892+0000 7f27db7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27dc105ed0 0x7f27dc1062f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:22.668 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.892+0000 7f27db7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27dc105ed0 0x7f27dc1062f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35530/0 (socket says 172.21.15.84:35530) 2024-05-22T22:52:22.668 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.892+0000 7f27db7fe700 1 -- 172.21.15.84:0/4251395934 learned_addr learned my addr 172.21.15.84:0/4251395934 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:22.668 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.892+0000 7f27db7fe700 1 -- 172.21.15.84:0/4251395934 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27dc108e60 con 0x7f27dc105ed0 2024-05-22T22:52:22.668 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.893+0000 7f27db7fe700 1 --2- 172.21.15.84:0/4251395934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27dc105ed0 0x7f27dc1062f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0098a0 tx=0x7f27cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6553385adfe649d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:22.668 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.893+0000 7f27da7fc700 1 -- 172.21.15.84:0/4251395934 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27cc004160 con 0x7f27dc105ed0 2024-05-22T22:52:22.668 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.893+0000 7f27da7fc700 1 -- 172.21.15.84:0/4251395934 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f27cc004300 con 0x7f27dc105ed0 2024-05-22T22:52:22.669 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.893+0000 7f27da7fc700 1 -- 172.21.15.84:0/4251395934 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27cc011b50 con 0x7f27dc105ed0 2024-05-22T22:52:22.669 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.893+0000 7f27e099f700 1 -- 172.21.15.84:0/4251395934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27dc105ed0 msgr2=0x7f27dc1062f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:22.669 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.893+0000 7f27e099f700 1 --2- 172.21.15.84:0/4251395934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27dc105ed0 0x7f27dc1062f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0098a0 tx=0x7f27cc009bb0 comp rx=0 tx=0).stop 2024-05-22T22:52:22.669 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.893+0000 7f27e099f700 1 -- 172.21.15.84:0/4251395934 shutdown_connections 2024-05-22T22:52:22.669 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.893+0000 7f27e099f700 1 --2- 172.21.15.84:0/4251395934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27dc105ed0 0x7f27dc1062f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:22.669 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.893+0000 7f27e099f700 1 -- 172.21.15.84:0/4251395934 >> 172.21.15.84:0/4251395934 conn(0x7f27dc101860 msgr2=0x7f27dc103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:22.669 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.893+0000 7f27e099f700 1 -- 172.21.15.84:0/4251395934 shutdown_connections 2024-05-22T22:52:22.669 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.893+0000 7f27e099f700 1 -- 172.21.15.84:0/4251395934 wait complete. 2024-05-22T22:52:22.669 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.894+0000 7f27e099f700 1 Processor -- start 2024-05-22T22:52:22.669 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.894+0000 7f27e099f700 1 -- start start 2024-05-22T22:52:22.669 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.895+0000 7f27e099f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27dc105ed0 0x7f27dc19bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:22.669 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.895+0000 7f27e099f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc10c210 con 0x7f27dc105ed0 2024-05-22T22:52:22.670 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.895+0000 7f27db7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27dc105ed0 0x7f27dc19bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:22.670 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.895+0000 7f27db7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27dc105ed0 0x7f27dc19bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35544/0 (socket says 172.21.15.84:35544) 2024-05-22T22:52:22.670 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.895+0000 7f27db7fe700 1 -- 172.21.15.84:0/3004413523 learned_addr learned my addr 172.21.15.84:0/3004413523 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:22.670 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.895+0000 7f27db7fe700 1 -- 172.21.15.84:0/3004413523 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27dc19c0c0 con 0x7f27dc105ed0 2024-05-22T22:52:22.670 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.895+0000 7f27db7fe700 1 --2- 172.21.15.84:0/3004413523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27dc105ed0 0x7f27dc19bb80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f27cc011670 tx=0x7f27cc011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:22.670 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.895+0000 7f27d8ff9700 1 -- 172.21.15.84:0/3004413523 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27cc0039c0 con 0x7f27dc105ed0 2024-05-22T22:52:22.670 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.895+0000 7f27d8ff9700 1 -- 172.21.15.84:0/3004413523 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f27cc011e50 con 0x7f27dc105ed0 2024-05-22T22:52:22.670 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.895+0000 7f27e099f700 1 -- 172.21.15.84:0/3004413523 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27dc19c350 con 0x7f27dc105ed0 2024-05-22T22:52:22.670 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.895+0000 7f27d8ff9700 1 -- 172.21.15.84:0/3004413523 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27cc01a5c0 con 0x7f27dc105ed0 2024-05-22T22:52:22.670 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.895+0000 7f27e099f700 1 -- 172.21.15.84:0/3004413523 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27dc19f040 con 0x7f27dc105ed0 2024-05-22T22:52:22.671 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.896+0000 7f27d8ff9700 1 -- 172.21.15.84:0/3004413523 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f27cc01a870 con 0x7f27dc105ed0 2024-05-22T22:52:22.671 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.896+0000 7f27e099f700 1 -- 172.21.15.84:0/3004413523 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27dc0639c0 con 0x7f27dc105ed0 2024-05-22T22:52:22.671 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.896+0000 7f27d8ff9700 1 --2- 172.21.15.84:0/3004413523 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f27c4039050 0x7f27c403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:22.671 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.896+0000 7f27d8ff9700 1 -- 172.21.15.84:0/3004413523 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f27cc04d460 con 0x7f27dc105ed0 2024-05-22T22:52:22.671 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.896+0000 7f27daffd700 1 --2- 172.21.15.84:0/3004413523 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f27c4039050 0x7f27c403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:22.671 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.897+0000 7f27daffd700 1 --2- 172.21.15.84:0/3004413523 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f27c4039050 0x7f27c403b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f27d0006fd0 tx=0x7f27d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:22.671 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:21.901+0000 7f27d8ff9700 1 -- 172.21.15.84:0/3004413523 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f27cc019c70 con 0x7f27dc105ed0 2024-05-22T22:52:22.671 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:22.101+0000 7f27e099f700 1 -- 172.21.15.84:0/3004413523 --> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f27dc102a10 con 0x7f27c4039050 2024-05-22T22:52:22.671 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:22.102+0000 7f27d8ff9700 1 -- 172.21.15.84:0/3004413523 <== mgr.14120 v2:172.21.15.84:6800/3278959854 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f27dc102a10 con 0x7f27c4039050 2024-05-22T22:52:22.671 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:22.104+0000 7f27e099f700 1 -- 172.21.15.84:0/3004413523 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f27c4039050 msgr2=0x7f27c403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:22.671 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:22.104+0000 7f27e099f700 1 --2- 172.21.15.84:0/3004413523 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f27c4039050 0x7f27c403b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f27d0006fd0 tx=0x7f27d0006e40 comp rx=0 tx=0).stop 2024-05-22T22:52:22.671 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:22.104+0000 7f27e099f700 1 -- 172.21.15.84:0/3004413523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27dc105ed0 msgr2=0x7f27dc19bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:22.671 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:22.104+0000 7f27e099f700 1 --2- 172.21.15.84:0/3004413523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27dc105ed0 0x7f27dc19bb80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f27cc011670 tx=0x7f27cc011b60 comp rx=0 tx=0).stop 2024-05-22T22:52:22.672 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:22.104+0000 7f27e099f700 1 -- 172.21.15.84:0/3004413523 shutdown_connections 2024-05-22T22:52:22.672 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:22.104+0000 7f27e099f700 1 --2- 172.21.15.84:0/3004413523 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f27c4039050 0x7f27c403b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:22.672 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:22.104+0000 7f27e099f700 1 --2- 172.21.15.84:0/3004413523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27dc105ed0 0x7f27dc19bb80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:22.672 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:22.104+0000 7f27e099f700 1 -- 172.21.15.84:0/3004413523 >> 172.21.15.84:0/3004413523 conn(0x7f27dc101860 msgr2=0x7f27dc102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:22.672 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:22.104+0000 7f27e099f700 1 -- 172.21.15.84:0/3004413523 shutdown_connections 2024-05-22T22:52:22.672 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:22.104+0000 7f27e099f700 1 -- 172.21.15.84:0/3004413523 wait complete. 2024-05-22T22:52:22.672 INFO:teuthology.orchestra.run.smithi084.stdout:Generating ssh key... 2024-05-22T22:52:22.980 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:22 smithi084 ceph-mon[32186]: [22/May/2024:22:52:20] ENGINE Serving on https://172.21.15.84:7150 2024-05-22T22:52:22.980 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:22 smithi084 ceph-mon[32186]: [22/May/2024:22:52:21] ENGINE Serving on http://172.21.15.84:8765 2024-05-22T22:52:22.980 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:22 smithi084 ceph-mon[32186]: [22/May/2024:22:52:21] ENGINE Bus STARTED 2024-05-22T22:52:23.615 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:23 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: Generating public/private rsa key pair. 2024-05-22T22:52:23.615 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:23 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: Your identification has been saved in /tmp/tmpwdu1bkt0/key. 2024-05-22T22:52:23.615 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:23 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: Your public key has been saved in /tmp/tmpwdu1bkt0/key.pub. 2024-05-22T22:52:23.615 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:23 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: The key fingerprint is: 2024-05-22T22:52:23.615 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:23 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: SHA256:IcSiZGFXov005fEKalyl7BRnOKBaiUg9TjEAY0R+A5U ceph-c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:52:23.615 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:23 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: The key's randomart image is: 2024-05-22T22:52:23.616 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:23 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: +---[RSA 3072]----+ 2024-05-22T22:52:23.616 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:23 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: |*B*=*++.* | 2024-05-22T22:52:23.616 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:23 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: |=++Eo++O o | 2024-05-22T22:52:23.616 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:23 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: |o+B+o.X.o . | 2024-05-22T22:52:23.616 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:23 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: | ooo.B + o | 2024-05-22T22:52:23.616 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:23 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: |. + o S | 2024-05-22T22:52:23.616 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:23 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: | . | 2024-05-22T22:52:23.616 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:23 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: | | 2024-05-22T22:52:23.616 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:23 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: | | 2024-05-22T22:52:23.616 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:23 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: | | 2024-05-22T22:52:23.616 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:23 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: +----[SHA256]-----+ 2024-05-22T22:52:24.000 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:23 smithi084 ceph-mon[32186]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:52:24.000 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:23 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' 2024-05-22T22:52:24.001 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:23 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' 2024-05-22T22:52:24.007 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.088+0000 7f0e36d52700 1 Processor -- start 2024-05-22T22:52:24.007 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.088+0000 7f0e36d52700 1 -- start start 2024-05-22T22:52:24.007 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.089+0000 7f0e36d52700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e30108840 0x7f0e3010ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:24.007 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.089+0000 7f0e36d52700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3007be80 con 0x7f0e30108840 2024-05-22T22:52:24.007 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.089+0000 7f0e34aee700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e30108840 0x7f0e3010ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:24.007 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.089+0000 7f0e34aee700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e30108840 0x7f0e3010ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35546/0 (socket says 172.21.15.84:35546) 2024-05-22T22:52:24.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.089+0000 7f0e34aee700 1 -- 172.21.15.84:0/893205737 learned_addr learned my addr 172.21.15.84:0/893205737 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:24.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.089+0000 7f0e34aee700 1 -- 172.21.15.84:0/893205737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3010b170 con 0x7f0e30108840 2024-05-22T22:52:24.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.089+0000 7f0e34aee700 1 --2- 172.21.15.84:0/893205737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e30108840 0x7f0e3010ac30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0e28009e80 tx=0x7f0e2800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc949d810fa38e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:24.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.089+0000 7f0e2f7fe700 1 -- 172.21.15.84:0/893205737 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e28017070 con 0x7f0e30108840 2024-05-22T22:52:24.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.089+0000 7f0e2f7fe700 1 -- 172.21.15.84:0/893205737 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0e2800b8c0 con 0x7f0e30108840 2024-05-22T22:52:24.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.089+0000 7f0e2f7fe700 1 -- 172.21.15.84:0/893205737 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e280045c0 con 0x7f0e30108840 2024-05-22T22:52:24.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.090+0000 7f0e36d52700 1 -- 172.21.15.84:0/893205737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e30108840 msgr2=0x7f0e3010ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:24.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.090+0000 7f0e36d52700 1 --2- 172.21.15.84:0/893205737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e30108840 0x7f0e3010ac30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0e28009e80 tx=0x7f0e2800b330 comp rx=0 tx=0).stop 2024-05-22T22:52:24.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.090+0000 7f0e36d52700 1 -- 172.21.15.84:0/893205737 shutdown_connections 2024-05-22T22:52:24.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.090+0000 7f0e36d52700 1 --2- 172.21.15.84:0/893205737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e30108840 0x7f0e3010ac30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:24.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.090+0000 7f0e36d52700 1 -- 172.21.15.84:0/893205737 >> 172.21.15.84:0/893205737 conn(0x7f0e30102210 msgr2=0x7f0e30104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:24.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.090+0000 7f0e36d52700 1 -- 172.21.15.84:0/893205737 shutdown_connections 2024-05-22T22:52:24.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.090+0000 7f0e36d52700 1 -- 172.21.15.84:0/893205737 wait complete. 2024-05-22T22:52:24.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.091+0000 7f0e36d52700 1 Processor -- start 2024-05-22T22:52:24.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.091+0000 7f0e36d52700 1 -- start start 2024-05-22T22:52:24.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.091+0000 7f0e36d52700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e30108840 0x7f0e3019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:24.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.091+0000 7f0e36d52700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3010c190 con 0x7f0e30108840 2024-05-22T22:52:24.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.091+0000 7f0e34aee700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e30108840 0x7f0e3019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:24.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.091+0000 7f0e34aee700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e30108840 0x7f0e3019bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35550/0 (socket says 172.21.15.84:35550) 2024-05-22T22:52:24.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.091+0000 7f0e34aee700 1 -- 172.21.15.84:0/4110950499 learned_addr learned my addr 172.21.15.84:0/4110950499 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:24.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.091+0000 7f0e34aee700 1 -- 172.21.15.84:0/4110950499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3019c1f0 con 0x7f0e30108840 2024-05-22T22:52:24.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.091+0000 7f0e34aee700 1 --2- 172.21.15.84:0/4110950499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e30108840 0x7f0e3019bcb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0e280072a0 tx=0x7f0e2800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:24.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.091+0000 7f0e2dffb700 1 -- 172.21.15.84:0/4110950499 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e28020020 con 0x7f0e30108840 2024-05-22T22:52:24.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.091+0000 7f0e36d52700 1 -- 172.21.15.84:0/4110950499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e3019c480 con 0x7f0e30108840 2024-05-22T22:52:24.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.091+0000 7f0e2dffb700 1 -- 172.21.15.84:0/4110950499 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0e28003b50 con 0x7f0e30108840 2024-05-22T22:52:24.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.091+0000 7f0e2dffb700 1 -- 172.21.15.84:0/4110950499 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e28017040 con 0x7f0e30108840 2024-05-22T22:52:24.010 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.092+0000 7f0e36d52700 1 -- 172.21.15.84:0/4110950499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e3019f170 con 0x7f0e30108840 2024-05-22T22:52:24.010 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.092+0000 7f0e2dffb700 1 -- 172.21.15.84:0/4110950499 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0e28023020 con 0x7f0e30108840 2024-05-22T22:52:24.010 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.093+0000 7f0e36d52700 1 -- 172.21.15.84:0/4110950499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e300639c0 con 0x7f0e30108840 2024-05-22T22:52:24.010 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.093+0000 7f0e2dffb700 1 --2- 172.21.15.84:0/4110950499 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f0e18039000 0x7f0e1803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:24.010 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.093+0000 7f0e2dffb700 1 -- 172.21.15.84:0/4110950499 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0e2800f050 con 0x7f0e30108840 2024-05-22T22:52:24.010 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.093+0000 7f0e2ffff700 1 --2- 172.21.15.84:0/4110950499 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f0e18039000 0x7f0e1803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:24.010 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.093+0000 7f0e2ffff700 1 --2- 172.21.15.84:0/4110950499 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f0e18039000 0x7f0e1803b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0e24006fd0 tx=0x7f0e24006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:24.010 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.098+0000 7f0e2dffb700 1 -- 172.21.15.84:0/4110950499 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0e2802a020 con 0x7f0e30108840 2024-05-22T22:52:24.010 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.292+0000 7f0e36d52700 1 -- 172.21.15.84:0/4110950499 --> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f0e30103430 con 0x7f0e18039000 2024-05-22T22:52:24.010 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.443+0000 7f0e2dffb700 1 -- 172.21.15.84:0/4110950499 <== mgr.14120 v2:172.21.15.84:6800/3278959854 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0e30103430 con 0x7f0e18039000 2024-05-22T22:52:24.010 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.445+0000 7f0e36d52700 1 -- 172.21.15.84:0/4110950499 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f0e18039000 msgr2=0x7f0e1803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:24.010 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.445+0000 7f0e36d52700 1 --2- 172.21.15.84:0/4110950499 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f0e18039000 0x7f0e1803b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0e24006fd0 tx=0x7f0e24006e40 comp rx=0 tx=0).stop 2024-05-22T22:52:24.011 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.445+0000 7f0e36d52700 1 -- 172.21.15.84:0/4110950499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e30108840 msgr2=0x7f0e3019bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:24.011 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.445+0000 7f0e36d52700 1 --2- 172.21.15.84:0/4110950499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e30108840 0x7f0e3019bcb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0e280072a0 tx=0x7f0e2800bd50 comp rx=0 tx=0).stop 2024-05-22T22:52:24.011 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.445+0000 7f0e36d52700 1 -- 172.21.15.84:0/4110950499 shutdown_connections 2024-05-22T22:52:24.011 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.445+0000 7f0e36d52700 1 --2- 172.21.15.84:0/4110950499 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f0e18039000 0x7f0e1803b4c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:24.011 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.445+0000 7f0e36d52700 1 --2- 172.21.15.84:0/4110950499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e30108840 0x7f0e3019bcb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:24.011 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.445+0000 7f0e36d52700 1 -- 172.21.15.84:0/4110950499 >> 172.21.15.84:0/4110950499 conn(0x7f0e30102210 msgr2=0x7f0e30102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:24.011 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.445+0000 7f0e36d52700 1 -- 172.21.15.84:0/4110950499 shutdown_connections 2024-05-22T22:52:24.011 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:23.445+0000 7f0e36d52700 1 -- 172.21.15.84:0/4110950499 wait complete. 2024-05-22T22:52:25.006 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:24 smithi084 ceph-mon[32186]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:52:25.007 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:24 smithi084 ceph-mon[32186]: Generating ssh key... 2024-05-22T22:52:25.265 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyMcocFllQBQmqrgABP1nMuSuzyzqWTLEKmlJZRkJzY1P9PMklATrhQuTd1vhVTjh5HTbLsaX/wKsv269gmsjmykHyiMWTXWmXlUJTqoYWTD/hX1G+W3Ejkja76VkuJzPnU0P98tXAPBgmG3z15LmVkPi/aS0Qxe6fygADVs5NvYCJdrIk5foFOeiHL5+XVuTEg3dbdHlxtoqUfafdtXar5P945X8RIl84/ljEQqwyy1vFvwQLoNcBiNZPRY05+c5GmBFBhedJOv7QpnfgutgEE0hQZMHHNRNXMDNWFbCoHd2fsCBF6s0V7IvEdolun9c3oq0sBI7tEqVcn3CLNNpkrvh/n68Kz2DXRF85qYTwi+C1F2OV0gGEJA6GknlcyD8HxpgluJfEeQUB1tmScOxhMMr2mtdlTyeAnM6UbCYsU0K9YMtV+UQL3MZ5Kl0M7c7xRcYCHoOBYRRTMOMjpgiHZzaclaoVPuS8lQ4yj4fd5IVpcDryCbNPCHRgc/l9b0E= ceph-c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:52:25.265 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.425+0000 7f4b7e4e3700 1 Processor -- start 2024-05-22T22:52:25.265 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.425+0000 7f4b7e4e3700 1 -- start start 2024-05-22T22:52:25.265 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.425+0000 7f4b7e4e3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b78106c90 0x7f4b781070b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:25.265 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.425+0000 7f4b7e4e3700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4b78107680 con 0x7f4b78106c90 2024-05-22T22:52:25.265 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.426+0000 7f4b77fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b78106c90 0x7f4b781070b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:25.265 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.426+0000 7f4b77fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b78106c90 0x7f4b781070b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35554/0 (socket says 172.21.15.84:35554) 2024-05-22T22:52:25.265 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.426+0000 7f4b77fff700 1 -- 172.21.15.84:0/2166891162 learned_addr learned my addr 172.21.15.84:0/2166891162 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:25.265 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.426+0000 7f4b77fff700 1 -- 172.21.15.84:0/2166891162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b78107e90 con 0x7f4b78106c90 2024-05-22T22:52:25.266 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.426+0000 7f4b77fff700 1 --2- 172.21.15.84:0/2166891162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b78106c90 0x7f4b781070b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009e80 tx=0x7f4b6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4d1cfc7022f67d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:25.266 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.426+0000 7f4b76ffd700 1 -- 172.21.15.84:0/2166891162 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b68017070 con 0x7f4b78106c90 2024-05-22T22:52:25.266 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.426+0000 7f4b76ffd700 1 -- 172.21.15.84:0/2166891162 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4b6800b8c0 con 0x7f4b78106c90 2024-05-22T22:52:25.266 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.426+0000 7f4b76ffd700 1 -- 172.21.15.84:0/2166891162 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b680045c0 con 0x7f4b78106c90 2024-05-22T22:52:25.266 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.427+0000 7f4b7e4e3700 1 -- 172.21.15.84:0/2166891162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b78106c90 msgr2=0x7f4b781070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:25.266 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.427+0000 7f4b7e4e3700 1 --2- 172.21.15.84:0/2166891162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b78106c90 0x7f4b781070b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009e80 tx=0x7f4b6800b330 comp rx=0 tx=0).stop 2024-05-22T22:52:25.266 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.427+0000 7f4b7e4e3700 1 -- 172.21.15.84:0/2166891162 shutdown_connections 2024-05-22T22:52:25.266 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.427+0000 7f4b7e4e3700 1 --2- 172.21.15.84:0/2166891162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b78106c90 0x7f4b781070b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:25.266 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.427+0000 7f4b7e4e3700 1 -- 172.21.15.84:0/2166891162 >> 172.21.15.84:0/2166891162 conn(0x7f4b78102210 msgr2=0x7f4b78104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:25.266 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.427+0000 7f4b7e4e3700 1 -- 172.21.15.84:0/2166891162 shutdown_connections 2024-05-22T22:52:25.266 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.427+0000 7f4b7e4e3700 1 -- 172.21.15.84:0/2166891162 wait complete. 2024-05-22T22:52:25.266 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.427+0000 7f4b7e4e3700 1 Processor -- start 2024-05-22T22:52:25.267 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.428+0000 7f4b7e4e3700 1 -- start start 2024-05-22T22:52:25.267 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.428+0000 7f4b7e4e3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b78106c90 0x7f4b7819bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:25.267 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.428+0000 7f4b7e4e3700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7810b240 con 0x7f4b78106c90 2024-05-22T22:52:25.267 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.428+0000 7f4b77fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b78106c90 0x7f4b7819bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:25.267 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.428+0000 7f4b77fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b78106c90 0x7f4b7819bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35562/0 (socket says 172.21.15.84:35562) 2024-05-22T22:52:25.267 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.428+0000 7f4b77fff700 1 -- 172.21.15.84:0/2915775933 learned_addr learned my addr 172.21.15.84:0/2915775933 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:25.267 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.428+0000 7f4b77fff700 1 -- 172.21.15.84:0/2915775933 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b7819c260 con 0x7f4b78106c90 2024-05-22T22:52:25.267 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.428+0000 7f4b77fff700 1 --2- 172.21.15.84:0/2915775933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b78106c90 0x7f4b7819bd20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009530 tx=0x7f4b6800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:25.267 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.429+0000 7f4b757fa700 1 -- 172.21.15.84:0/2915775933 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b68020020 con 0x7f4b78106c90 2024-05-22T22:52:25.267 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.429+0000 7f4b757fa700 1 -- 172.21.15.84:0/2915775933 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4b68003b50 con 0x7f4b78106c90 2024-05-22T22:52:25.267 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.429+0000 7f4b7e4e3700 1 -- 172.21.15.84:0/2915775933 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b7819c4f0 con 0x7f4b78106c90 2024-05-22T22:52:25.267 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.429+0000 7f4b757fa700 1 -- 172.21.15.84:0/2915775933 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b68017040 con 0x7f4b78106c90 2024-05-22T22:52:25.268 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.429+0000 7f4b7e4e3700 1 -- 172.21.15.84:0/2915775933 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b7819f1d0 con 0x7f4b78106c90 2024-05-22T22:52:25.268 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.430+0000 7f4b7e4e3700 1 -- 172.21.15.84:0/2915775933 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b780639c0 con 0x7f4b78106c90 2024-05-22T22:52:25.268 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.430+0000 7f4b757fa700 1 -- 172.21.15.84:0/2915775933 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4b68023020 con 0x7f4b78106c90 2024-05-22T22:52:25.268 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.430+0000 7f4b757fa700 1 --2- 172.21.15.84:0/2915775933 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f4b60039050 0x7f4b6003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:25.268 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.430+0000 7f4b757fa700 1 -- 172.21.15.84:0/2915775933 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4b6800f050 con 0x7f4b78106c90 2024-05-22T22:52:25.268 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.430+0000 7f4b777fe700 1 --2- 172.21.15.84:0/2915775933 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f4b60039050 0x7f4b6003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:25.268 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.430+0000 7f4b777fe700 1 --2- 172.21.15.84:0/2915775933 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f4b60039050 0x7f4b6003b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c006fd0 tx=0x7f4b6c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:25.268 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.435+0000 7f4b757fa700 1 -- 172.21.15.84:0/2915775933 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4b6802a020 con 0x7f4b78106c90 2024-05-22T22:52:25.268 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.629+0000 7f4b7e4e3700 1 -- 172.21.15.84:0/2915775933 --> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f4b7802df70 con 0x7f4b60039050 2024-05-22T22:52:25.268 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.630+0000 7f4b757fa700 1 -- 172.21.15.84:0/2915775933 <== mgr.14120 v2:172.21.15.84:6800/3278959854 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f4b7802df70 con 0x7f4b60039050 2024-05-22T22:52:25.268 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.632+0000 7f4b7e4e3700 1 -- 172.21.15.84:0/2915775933 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f4b60039050 msgr2=0x7f4b6003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:25.269 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.632+0000 7f4b7e4e3700 1 --2- 172.21.15.84:0/2915775933 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f4b60039050 0x7f4b6003b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c006fd0 tx=0x7f4b6c006e40 comp rx=0 tx=0).stop 2024-05-22T22:52:25.269 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.632+0000 7f4b7e4e3700 1 -- 172.21.15.84:0/2915775933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b78106c90 msgr2=0x7f4b7819bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:25.269 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.632+0000 7f4b7e4e3700 1 --2- 172.21.15.84:0/2915775933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b78106c90 0x7f4b7819bd20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009530 tx=0x7f4b6800bd50 comp rx=0 tx=0).stop 2024-05-22T22:52:25.269 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.632+0000 7f4b7e4e3700 1 -- 172.21.15.84:0/2915775933 shutdown_connections 2024-05-22T22:52:25.269 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.632+0000 7f4b7e4e3700 1 --2- 172.21.15.84:0/2915775933 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f4b60039050 0x7f4b6003b510 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:25.269 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.632+0000 7f4b7e4e3700 1 --2- 172.21.15.84:0/2915775933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b78106c90 0x7f4b7819bd20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:25.269 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.632+0000 7f4b7e4e3700 1 -- 172.21.15.84:0/2915775933 >> 172.21.15.84:0/2915775933 conn(0x7f4b78102210 msgr2=0x7f4b781030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:25.269 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.632+0000 7f4b7e4e3700 1 -- 172.21.15.84:0/2915775933 shutdown_connections 2024-05-22T22:52:25.269 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:24.632+0000 7f4b7e4e3700 1 -- 172.21.15.84:0/2915775933 wait complete. 2024-05-22T22:52:25.269 INFO:teuthology.orchestra.run.smithi084.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-22T22:52:25.270 INFO:teuthology.orchestra.run.smithi084.stdout:Adding key to root@localhost authorized_keys... 2024-05-22T22:52:25.270 INFO:teuthology.orchestra.run.smithi084.stdout:Adding host smithi084... 2024-05-22T22:52:25.865 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:25 smithi084 ceph-mon[32186]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:52:27.115 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:26 smithi084 ceph-mon[32186]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi084", "addr": "172.21.15.84", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:52:28.016 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:27 smithi084 ceph-mon[32186]: Deploying cephadm binary to smithi084 2024-05-22T22:52:30.572 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout Added host 'smithi084' with addr '172.21.15.84' 2024-05-22T22:52:30.572 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.674+0000 7fb884885700 1 Processor -- start 2024-05-22T22:52:30.572 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.674+0000 7fb884885700 1 -- start start 2024-05-22T22:52:30.572 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.674+0000 7fb884885700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8801060d0 0x7fb8801064f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:30.572 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.674+0000 7fb884885700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb880106ac0 con 0x7fb8801060d0 2024-05-22T22:52:30.573 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.674+0000 7fb87e59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8801060d0 0x7fb8801064f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:30.573 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.674+0000 7fb87e59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8801060d0 0x7fb8801064f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35574/0 (socket says 172.21.15.84:35574) 2024-05-22T22:52:30.573 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.674+0000 7fb87e59c700 1 -- 172.21.15.84:0/1826243866 learned_addr learned my addr 172.21.15.84:0/1826243866 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:30.573 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.674+0000 7fb87e59c700 1 -- 172.21.15.84:0/1826243866 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb880109060 con 0x7fb8801060d0 2024-05-22T22:52:30.573 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.675+0000 7fb87e59c700 1 --2- 172.21.15.84:0/1826243866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8801060d0 0x7fb8801064f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb8680098a0 tx=0x7fb868009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0afde1a518129b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:30.573 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.675+0000 7fb87d59a700 1 -- 172.21.15.84:0/1826243866 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb868004160 con 0x7fb8801060d0 2024-05-22T22:52:30.573 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.675+0000 7fb87d59a700 1 -- 172.21.15.84:0/1826243866 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb868004300 con 0x7fb8801060d0 2024-05-22T22:52:30.573 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.675+0000 7fb87d59a700 1 -- 172.21.15.84:0/1826243866 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb868011b50 con 0x7fb8801060d0 2024-05-22T22:52:30.573 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.675+0000 7fb884885700 1 -- 172.21.15.84:0/1826243866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8801060d0 msgr2=0x7fb8801064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:30.573 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.675+0000 7fb884885700 1 --2- 172.21.15.84:0/1826243866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8801060d0 0x7fb8801064f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb8680098a0 tx=0x7fb868009bb0 comp rx=0 tx=0).stop 2024-05-22T22:52:30.573 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.675+0000 7fb884885700 1 -- 172.21.15.84:0/1826243866 shutdown_connections 2024-05-22T22:52:30.573 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.675+0000 7fb884885700 1 --2- 172.21.15.84:0/1826243866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8801060d0 0x7fb8801064f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:30.574 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.675+0000 7fb884885700 1 -- 172.21.15.84:0/1826243866 >> 172.21.15.84:0/1826243866 conn(0x7fb880101a40 msgr2=0x7fb880103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:30.574 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.675+0000 7fb884885700 1 -- 172.21.15.84:0/1826243866 shutdown_connections 2024-05-22T22:52:30.574 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.675+0000 7fb884885700 1 -- 172.21.15.84:0/1826243866 wait complete. 2024-05-22T22:52:30.574 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.676+0000 7fb884885700 1 Processor -- start 2024-05-22T22:52:30.574 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.676+0000 7fb884885700 1 -- start start 2024-05-22T22:52:30.574 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.676+0000 7fb884885700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8801060d0 0x7fb88019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:30.574 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.676+0000 7fb884885700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb88010c410 con 0x7fb8801060d0 2024-05-22T22:52:30.574 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.677+0000 7fb87e59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8801060d0 0x7fb88019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:30.574 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.677+0000 7fb87e59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8801060d0 0x7fb88019bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35588/0 (socket says 172.21.15.84:35588) 2024-05-22T22:52:30.574 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.677+0000 7fb87e59c700 1 -- 172.21.15.84:0/519693529 learned_addr learned my addr 172.21.15.84:0/519693529 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:30.574 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.677+0000 7fb87e59c700 1 -- 172.21.15.84:0/519693529 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88019c270 con 0x7fb8801060d0 2024-05-22T22:52:30.574 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.677+0000 7fb87e59c700 1 --2- 172.21.15.84:0/519693529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8801060d0 0x7fb88019bd30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb868011670 tx=0x7fb868011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:30.575 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.677+0000 7fb8777fe700 1 -- 172.21.15.84:0/519693529 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8680039c0 con 0x7fb8801060d0 2024-05-22T22:52:30.575 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.677+0000 7fb884885700 1 -- 172.21.15.84:0/519693529 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb88019c500 con 0x7fb8801060d0 2024-05-22T22:52:30.575 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.677+0000 7fb8777fe700 1 -- 172.21.15.84:0/519693529 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb868011e50 con 0x7fb8801060d0 2024-05-22T22:52:30.575 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.677+0000 7fb8777fe700 1 -- 172.21.15.84:0/519693529 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb86801a5c0 con 0x7fb8801060d0 2024-05-22T22:52:30.575 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.677+0000 7fb884885700 1 -- 172.21.15.84:0/519693529 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb88019f1f0 con 0x7fb8801060d0 2024-05-22T22:52:30.575 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.678+0000 7fb8777fe700 1 -- 172.21.15.84:0/519693529 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb86801a8c0 con 0x7fb8801060d0 2024-05-22T22:52:30.575 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.678+0000 7fb884885700 1 -- 172.21.15.84:0/519693529 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8800639c0 con 0x7fb8801060d0 2024-05-22T22:52:30.575 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.678+0000 7fb8777fe700 1 --2- 172.21.15.84:0/519693529 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fb86c039000 0x7fb86c03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:30.575 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.678+0000 7fb8777fe700 1 -- 172.21.15.84:0/519693529 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb86804d6d0 con 0x7fb8801060d0 2024-05-22T22:52:30.575 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.678+0000 7fb87dd9b700 1 --2- 172.21.15.84:0/519693529 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fb86c039000 0x7fb86c03b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:30.575 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.679+0000 7fb87dd9b700 1 --2- 172.21.15.84:0/519693529 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fb86c039000 0x7fb86c03b4c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb870006fd0 tx=0x7fb870006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:30.575 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.683+0000 7fb8777fe700 1 -- 172.21.15.84:0/519693529 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb868019c70 con 0x7fb8801060d0 2024-05-22T22:52:30.575 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:25.885+0000 7fb884885700 1 -- 172.21.15.84:0/519693529 --> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi084", "addr": "172.21.15.84", "target": ["mon-mgr", ""]}) -- 0x7fb880102bf0 con 0x7fb86c039000 2024-05-22T22:52:30.576 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:29.988+0000 7fb8777fe700 1 -- 172.21.15.84:0/519693529 <== mgr.14120 v2:172.21.15.84:6800/3278959854 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fb880102bf0 con 0x7fb86c039000 2024-05-22T22:52:30.576 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:29.990+0000 7fb884885700 1 -- 172.21.15.84:0/519693529 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fb86c039000 msgr2=0x7fb86c03b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:30.576 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:29.990+0000 7fb884885700 1 --2- 172.21.15.84:0/519693529 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fb86c039000 0x7fb86c03b4c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb870006fd0 tx=0x7fb870006e40 comp rx=0 tx=0).stop 2024-05-22T22:52:30.576 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:29.990+0000 7fb884885700 1 -- 172.21.15.84:0/519693529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8801060d0 msgr2=0x7fb88019bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:30.576 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:29.990+0000 7fb884885700 1 --2- 172.21.15.84:0/519693529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8801060d0 0x7fb88019bd30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb868011670 tx=0x7fb868011b60 comp rx=0 tx=0).stop 2024-05-22T22:52:30.576 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:29.991+0000 7fb884885700 1 -- 172.21.15.84:0/519693529 shutdown_connections 2024-05-22T22:52:30.576 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:29.991+0000 7fb884885700 1 --2- 172.21.15.84:0/519693529 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fb86c039000 0x7fb86c03b4c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:30.576 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:29.991+0000 7fb884885700 1 --2- 172.21.15.84:0/519693529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8801060d0 0x7fb88019bd30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:30.576 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:29.991+0000 7fb884885700 1 -- 172.21.15.84:0/519693529 >> 172.21.15.84:0/519693529 conn(0x7fb880101a40 msgr2=0x7fb8801024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:30.576 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:29.991+0000 7fb884885700 1 -- 172.21.15.84:0/519693529 shutdown_connections 2024-05-22T22:52:30.576 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:29.991+0000 7fb884885700 1 -- 172.21.15.84:0/519693529 wait complete. 2024-05-22T22:52:30.576 INFO:teuthology.orchestra.run.smithi084.stdout:Deploying unmanaged mon service... 2024-05-22T22:52:31.240 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:30 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' 2024-05-22T22:52:31.240 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:30 smithi084 ceph-mon[32186]: Added host smithi084 2024-05-22T22:52:31.240 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:30 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:52:32.006 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-22T22:52:32.006 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.014+0000 7f144e11d700 1 Processor -- start 2024-05-22T22:52:32.006 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.014+0000 7f144e11d700 1 -- start start 2024-05-22T22:52:32.006 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.015+0000 7f144e11d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1448106400 0x7f1448106820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:32.006 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.015+0000 7f144e11d700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1448106df0 con 0x7f1448106400 2024-05-22T22:52:32.006 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.015+0000 7f14477fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1448106400 0x7f1448106820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:32.006 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.015+0000 7f14477fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1448106400 0x7f1448106820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35594/0 (socket says 172.21.15.84:35594) 2024-05-22T22:52:32.006 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.015+0000 7f14477fe700 1 -- 172.21.15.84:0/3038742567 learned_addr learned my addr 172.21.15.84:0/3038742567 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:32.006 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.016+0000 7f14477fe700 1 -- 172.21.15.84:0/3038742567 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1448107600 con 0x7f1448106400 2024-05-22T22:52:32.006 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.016+0000 7f14477fe700 1 --2- 172.21.15.84:0/3038742567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1448106400 0x7f1448106820 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f144000af90 tx=0x7f144000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f804b8661e2921 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:32.007 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.016+0000 7f14467fc700 1 -- 172.21.15.84:0/3038742567 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1440018070 con 0x7f1448106400 2024-05-22T22:52:32.007 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.016+0000 7f14467fc700 1 -- 172.21.15.84:0/3038742567 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f144000c9a0 con 0x7f1448106400 2024-05-22T22:52:32.007 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.016+0000 7f14467fc700 1 -- 172.21.15.84:0/3038742567 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14400045e0 con 0x7f1448106400 2024-05-22T22:52:32.007 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.017+0000 7f144e11d700 1 -- 172.21.15.84:0/3038742567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1448106400 msgr2=0x7f1448106820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:32.007 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.017+0000 7f144e11d700 1 --2- 172.21.15.84:0/3038742567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1448106400 0x7f1448106820 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f144000af90 tx=0x7f144000c330 comp rx=0 tx=0).stop 2024-05-22T22:52:32.007 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.017+0000 7f144e11d700 1 -- 172.21.15.84:0/3038742567 shutdown_connections 2024-05-22T22:52:32.007 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.017+0000 7f144e11d700 1 --2- 172.21.15.84:0/3038742567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1448106400 0x7f1448106820 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:32.007 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.017+0000 7f144e11d700 1 -- 172.21.15.84:0/3038742567 >> 172.21.15.84:0/3038742567 conn(0x7f14481019e0 msgr2=0x7f1448103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:32.007 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.017+0000 7f144e11d700 1 -- 172.21.15.84:0/3038742567 shutdown_connections 2024-05-22T22:52:32.007 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.017+0000 7f144e11d700 1 -- 172.21.15.84:0/3038742567 wait complete. 2024-05-22T22:52:32.007 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.018+0000 7f144e11d700 1 Processor -- start 2024-05-22T22:52:32.007 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.018+0000 7f144e11d700 1 -- start start 2024-05-22T22:52:32.007 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.018+0000 7f144e11d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1448106400 0x7f144818efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:32.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.018+0000 7f144e11d700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1448107d80 con 0x7f1448106400 2024-05-22T22:52:32.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.018+0000 7f14477fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1448106400 0x7f144818efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:32.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.018+0000 7f14477fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1448106400 0x7f144818efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35608/0 (socket says 172.21.15.84:35608) 2024-05-22T22:52:32.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.018+0000 7f14477fe700 1 -- 172.21.15.84:0/202161541 learned_addr learned my addr 172.21.15.84:0/202161541 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:32.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.018+0000 7f14477fe700 1 -- 172.21.15.84:0/202161541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144818f500 con 0x7f1448106400 2024-05-22T22:52:32.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.019+0000 7f14477fe700 1 --2- 172.21.15.84:0/202161541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1448106400 0x7f144818efc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f144000c770 tx=0x7f144000de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:32.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.019+0000 7f1444ff9700 1 -- 172.21.15.84:0/202161541 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1440021020 con 0x7f1448106400 2024-05-22T22:52:32.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.019+0000 7f144e11d700 1 -- 172.21.15.84:0/202161541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f144818d6f0 con 0x7f1448106400 2024-05-22T22:52:32.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.019+0000 7f1444ff9700 1 -- 172.21.15.84:0/202161541 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1440019040 con 0x7f1448106400 2024-05-22T22:52:32.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.019+0000 7f1444ff9700 1 -- 172.21.15.84:0/202161541 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1440018070 con 0x7f1448106400 2024-05-22T22:52:32.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.019+0000 7f144e11d700 1 -- 172.21.15.84:0/202161541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f144818dc30 con 0x7f1448106400 2024-05-22T22:52:32.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.020+0000 7f1444ff9700 1 -- 172.21.15.84:0/202161541 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1440024020 con 0x7f1448106400 2024-05-22T22:52:32.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.020+0000 7f144e11d700 1 -- 172.21.15.84:0/202161541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14480639c0 con 0x7f1448106400 2024-05-22T22:52:32.008 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.020+0000 7f1444ff9700 1 --2- 172.21.15.84:0/202161541 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f1434039000 0x7f143403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:32.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.020+0000 7f1444ff9700 1 -- 172.21.15.84:0/202161541 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1440010050 con 0x7f1448106400 2024-05-22T22:52:32.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.020+0000 7f1446ffd700 1 --2- 172.21.15.84:0/202161541 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f1434039000 0x7f143403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:32.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.021+0000 7f1446ffd700 1 --2- 172.21.15.84:0/202161541 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f1434039000 0x7f143403b4c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1430006fd0 tx=0x7f1430006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:32.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.025+0000 7f1444ff9700 1 -- 172.21.15.84:0/202161541 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f144002b020 con 0x7f1448106400 2024-05-22T22:52:32.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.248+0000 7f144e11d700 1 -- 172.21.15.84:0/202161541 --> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f1448195640 con 0x7f1434039000 2024-05-22T22:52:32.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.255+0000 7f1444ff9700 1 -- 172.21.15.84:0/202161541 <== mgr.14120 v2:172.21.15.84:6800/3278959854 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1448195640 con 0x7f1434039000 2024-05-22T22:52:32.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.257+0000 7f144e11d700 1 -- 172.21.15.84:0/202161541 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f1434039000 msgr2=0x7f143403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:32.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.257+0000 7f144e11d700 1 --2- 172.21.15.84:0/202161541 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f1434039000 0x7f143403b4c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1430006fd0 tx=0x7f1430006e40 comp rx=0 tx=0).stop 2024-05-22T22:52:32.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.257+0000 7f144e11d700 1 -- 172.21.15.84:0/202161541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1448106400 msgr2=0x7f144818efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:32.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.257+0000 7f144e11d700 1 --2- 172.21.15.84:0/202161541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1448106400 0x7f144818efc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f144000c770 tx=0x7f144000de20 comp rx=0 tx=0).stop 2024-05-22T22:52:32.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.257+0000 7f144e11d700 1 -- 172.21.15.84:0/202161541 shutdown_connections 2024-05-22T22:52:32.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.257+0000 7f144e11d700 1 --2- 172.21.15.84:0/202161541 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f1434039000 0x7f143403b4c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:32.009 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.257+0000 7f144e11d700 1 --2- 172.21.15.84:0/202161541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1448106400 0x7f144818efc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:32.010 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.257+0000 7f144e11d700 1 -- 172.21.15.84:0/202161541 >> 172.21.15.84:0/202161541 conn(0x7f14481019e0 msgr2=0x7f14481928a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:32.010 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.258+0000 7f144e11d700 1 -- 172.21.15.84:0/202161541 shutdown_connections 2024-05-22T22:52:32.010 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:31.258+0000 7f144e11d700 1 -- 172.21.15.84:0/202161541 wait complete. 2024-05-22T22:52:32.010 INFO:teuthology.orchestra.run.smithi084.stdout:Deploying unmanaged mgr service... 2024-05-22T22:52:32.293 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:32 smithi084 ceph-mon[32186]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:52:32.293 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:32 smithi084 ceph-mon[32186]: Saving service mon spec with placement count:5 2024-05-22T22:52:32.293 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:32 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' 2024-05-22T22:52:33.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-22T22:52:33.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.402+0000 7f15bede3700 1 Processor -- start 2024-05-22T22:52:33.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.402+0000 7f15bede3700 1 -- start start 2024-05-22T22:52:33.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.402+0000 7f15bede3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b8107e50 0x7f15b8108270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:33.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.402+0000 7f15bede3700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f15b8108840 con 0x7f15b8107e50 2024-05-22T22:52:33.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.402+0000 7f15bcb7f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b8107e50 0x7f15b8108270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:33.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.403+0000 7f15bcb7f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b8107e50 0x7f15b8108270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44874/0 (socket says 172.21.15.84:44874) 2024-05-22T22:52:33.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.403+0000 7f15bcb7f700 1 -- 172.21.15.84:0/3255176007 learned_addr learned my addr 172.21.15.84:0/3255176007 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:33.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.403+0000 7f15bcb7f700 1 -- 172.21.15.84:0/3255176007 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b8108fb0 con 0x7f15b8107e50 2024-05-22T22:52:33.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.403+0000 7f15bcb7f700 1 --2- 172.21.15.84:0/3255176007 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b8107e50 0x7f15b8108270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009e80 tx=0x7f15b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af22425327f073a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:33.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.403+0000 7f15af7fe700 1 -- 172.21.15.84:0/3255176007 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15b4017070 con 0x7f15b8107e50 2024-05-22T22:52:33.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.403+0000 7f15af7fe700 1 -- 172.21.15.84:0/3255176007 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f15b400b8c0 con 0x7f15b8107e50 2024-05-22T22:52:33.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.403+0000 7f15af7fe700 1 -- 172.21.15.84:0/3255176007 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15b40045c0 con 0x7f15b8107e50 2024-05-22T22:52:33.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.404+0000 7f15bede3700 1 -- 172.21.15.84:0/3255176007 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b8107e50 msgr2=0x7f15b8108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:33.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.404+0000 7f15bede3700 1 --2- 172.21.15.84:0/3255176007 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b8107e50 0x7f15b8108270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009e80 tx=0x7f15b400b330 comp rx=0 tx=0).stop 2024-05-22T22:52:33.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.404+0000 7f15bede3700 1 -- 172.21.15.84:0/3255176007 shutdown_connections 2024-05-22T22:52:33.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.404+0000 7f15bede3700 1 --2- 172.21.15.84:0/3255176007 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b8107e50 0x7f15b8108270 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:33.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.404+0000 7f15bede3700 1 -- 172.21.15.84:0/3255176007 >> 172.21.15.84:0/3255176007 conn(0x7f15b8103450 msgr2=0x7f15b8105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:33.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.404+0000 7f15bede3700 1 -- 172.21.15.84:0/3255176007 shutdown_connections 2024-05-22T22:52:33.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.404+0000 7f15bede3700 1 -- 172.21.15.84:0/3255176007 wait complete. 2024-05-22T22:52:33.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.404+0000 7f15bede3700 1 Processor -- start 2024-05-22T22:52:33.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.404+0000 7f15bede3700 1 -- start start 2024-05-22T22:52:33.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.404+0000 7f15bede3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b8107e50 0x7f15b819bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:33.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.404+0000 7f15bede3700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f15b810c360 con 0x7f15b8107e50 2024-05-22T22:52:33.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.404+0000 7f15bcb7f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b8107e50 0x7f15b819bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:33.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.405+0000 7f15bcb7f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b8107e50 0x7f15b819bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44886/0 (socket says 172.21.15.84:44886) 2024-05-22T22:52:33.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.405+0000 7f15bcb7f700 1 -- 172.21.15.84:0/2655786387 learned_addr learned my addr 172.21.15.84:0/2655786387 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:33.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.405+0000 7f15bcb7f700 1 -- 172.21.15.84:0/2655786387 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b819c330 con 0x7f15b8107e50 2024-05-22T22:52:33.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.405+0000 7f15bcb7f700 1 --2- 172.21.15.84:0/2655786387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b8107e50 0x7f15b819bdf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f15b4004210 tx=0x7f15b400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:33.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.405+0000 7f15adffb700 1 -- 172.21.15.84:0/2655786387 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15b4020020 con 0x7f15b8107e50 2024-05-22T22:52:33.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.405+0000 7f15bede3700 1 -- 172.21.15.84:0/2655786387 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b819c5c0 con 0x7f15b8107e50 2024-05-22T22:52:33.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.405+0000 7f15adffb700 1 -- 172.21.15.84:0/2655786387 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f15b4003b50 con 0x7f15b8107e50 2024-05-22T22:52:33.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.405+0000 7f15adffb700 1 -- 172.21.15.84:0/2655786387 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15b4017040 con 0x7f15b8107e50 2024-05-22T22:52:33.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.405+0000 7f15bede3700 1 -- 172.21.15.84:0/2655786387 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b819f2b0 con 0x7f15b8107e50 2024-05-22T22:52:33.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.406+0000 7f15adffb700 1 -- 172.21.15.84:0/2655786387 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f15b4023020 con 0x7f15b8107e50 2024-05-22T22:52:33.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.406+0000 7f15bede3700 1 -- 172.21.15.84:0/2655786387 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b80639c0 con 0x7f15b8107e50 2024-05-22T22:52:33.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.406+0000 7f15adffb700 1 --2- 172.21.15.84:0/2655786387 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f15a0039050 0x7f15a003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:33.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.406+0000 7f15adffb700 1 -- 172.21.15.84:0/2655786387 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f15b400f050 con 0x7f15b8107e50 2024-05-22T22:52:33.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.406+0000 7f15affff700 1 --2- 172.21.15.84:0/2655786387 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f15a0039050 0x7f15a003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:33.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.407+0000 7f15affff700 1 --2- 172.21.15.84:0/2655786387 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f15a0039050 0x7f15a003b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f15a8006fd0 tx=0x7f15a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:33.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.411+0000 7f15adffb700 1 -- 172.21.15.84:0/2655786387 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f15b402a020 con 0x7f15b8107e50 2024-05-22T22:52:33.252 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.626+0000 7f15bede3700 1 -- 172.21.15.84:0/2655786387 --> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f15b8104670 con 0x7f15a0039050 2024-05-22T22:52:33.252 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.630+0000 7f15adffb700 1 -- 172.21.15.84:0/2655786387 <== mgr.14120 v2:172.21.15.84:6800/3278959854 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f15b8104670 con 0x7f15a0039050 2024-05-22T22:52:33.252 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.633+0000 7f15bede3700 1 -- 172.21.15.84:0/2655786387 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f15a0039050 msgr2=0x7f15a003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:33.252 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.633+0000 7f15bede3700 1 --2- 172.21.15.84:0/2655786387 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f15a0039050 0x7f15a003b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f15a8006fd0 tx=0x7f15a8006e40 comp rx=0 tx=0).stop 2024-05-22T22:52:33.252 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.633+0000 7f15bede3700 1 -- 172.21.15.84:0/2655786387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b8107e50 msgr2=0x7f15b819bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:33.252 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.633+0000 7f15bede3700 1 --2- 172.21.15.84:0/2655786387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b8107e50 0x7f15b819bdf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f15b4004210 tx=0x7f15b400bd50 comp rx=0 tx=0).stop 2024-05-22T22:52:33.252 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.633+0000 7f15bede3700 1 -- 172.21.15.84:0/2655786387 shutdown_connections 2024-05-22T22:52:33.252 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.633+0000 7f15bede3700 1 --2- 172.21.15.84:0/2655786387 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f15a0039050 0x7f15a003b510 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:33.252 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.633+0000 7f15bede3700 1 --2- 172.21.15.84:0/2655786387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b8107e50 0x7f15b819bdf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:33.252 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.633+0000 7f15bede3700 1 -- 172.21.15.84:0/2655786387 >> 172.21.15.84:0/2655786387 conn(0x7f15b8103450 msgr2=0x7f15b8103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:33.252 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.633+0000 7f15bede3700 1 -- 172.21.15.84:0/2655786387 shutdown_connections 2024-05-22T22:52:33.252 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:32.633+0000 7f15bede3700 1 -- 172.21.15.84:0/2655786387 wait complete. 2024-05-22T22:52:33.986 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:33 smithi084 ceph-mon[32186]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:52:33.986 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:33 smithi084 ceph-mon[32186]: Saving service mgr spec with placement count:2 2024-05-22T22:52:33.986 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:33 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' 2024-05-22T22:52:35.235 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:35 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' 2024-05-22T22:52:35.235 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:35 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' 2024-05-22T22:52:35.235 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:35 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/946370689' entity='client.admin' 2024-05-22T22:52:35.242 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.385+0000 7febaf5db700 1 Processor -- start 2024-05-22T22:52:35.242 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.385+0000 7febaf5db700 1 -- start start 2024-05-22T22:52:35.242 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.385+0000 7febaf5db700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feba8108880 0x7feba810ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:35.242 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.385+0000 7febaf5db700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7feba807be80 con 0x7feba8108880 2024-05-22T22:52:35.242 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.386+0000 7febad377700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feba8108880 0x7feba810ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:35.242 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.386+0000 7febad377700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feba8108880 0x7feba810ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44888/0 (socket says 172.21.15.84:44888) 2024-05-22T22:52:35.242 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.386+0000 7febad377700 1 -- 172.21.15.84:0/55034128 learned_addr learned my addr 172.21.15.84:0/55034128 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:35.242 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.386+0000 7febad377700 1 -- 172.21.15.84:0/55034128 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba810b1b0 con 0x7feba8108880 2024-05-22T22:52:35.242 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.386+0000 7febad377700 1 --2- 172.21.15.84:0/55034128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feba8108880 0x7feba810ac70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7feb980098a0 tx=0x7feb98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7f2de70544408c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:35.243 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.386+0000 7feb9ffff700 1 -- 172.21.15.84:0/55034128 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb98004160 con 0x7feba8108880 2024-05-22T22:52:35.243 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.386+0000 7feb9ffff700 1 -- 172.21.15.84:0/55034128 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb98004300 con 0x7feba8108880 2024-05-22T22:52:35.243 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.386+0000 7feb9ffff700 1 -- 172.21.15.84:0/55034128 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb98011b20 con 0x7feba8108880 2024-05-22T22:52:35.243 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.387+0000 7febaf5db700 1 -- 172.21.15.84:0/55034128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feba8108880 msgr2=0x7feba810ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:35.243 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.387+0000 7febaf5db700 1 --2- 172.21.15.84:0/55034128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feba8108880 0x7feba810ac70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7feb980098a0 tx=0x7feb98009bb0 comp rx=0 tx=0).stop 2024-05-22T22:52:35.243 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.387+0000 7febaf5db700 1 -- 172.21.15.84:0/55034128 shutdown_connections 2024-05-22T22:52:35.243 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.387+0000 7febaf5db700 1 --2- 172.21.15.84:0/55034128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feba8108880 0x7feba810ac70 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:35.243 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.387+0000 7febaf5db700 1 -- 172.21.15.84:0/55034128 >> 172.21.15.84:0/55034128 conn(0x7feba81022b0 msgr2=0x7feba81046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:35.243 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.387+0000 7febaf5db700 1 -- 172.21.15.84:0/55034128 shutdown_connections 2024-05-22T22:52:35.243 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.387+0000 7febaf5db700 1 -- 172.21.15.84:0/55034128 wait complete. 2024-05-22T22:52:35.243 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.388+0000 7febaf5db700 1 Processor -- start 2024-05-22T22:52:35.243 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.388+0000 7febaf5db700 1 -- start start 2024-05-22T22:52:35.243 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.388+0000 7febaf5db700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feba8108880 0x7feba8108100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:35.244 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.388+0000 7febaf5db700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7feba810c210 con 0x7feba8108880 2024-05-22T22:52:35.244 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.388+0000 7febad377700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feba8108880 0x7feba8108100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:35.244 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.388+0000 7febad377700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feba8108880 0x7feba8108100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44900/0 (socket says 172.21.15.84:44900) 2024-05-22T22:52:35.244 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.388+0000 7febad377700 1 -- 172.21.15.84:0/946370689 learned_addr learned my addr 172.21.15.84:0/946370689 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:35.244 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.388+0000 7febad377700 1 -- 172.21.15.84:0/946370689 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba8108640 con 0x7feba8108880 2024-05-22T22:52:35.244 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.388+0000 7febad377700 1 --2- 172.21.15.84:0/946370689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feba8108880 0x7feba8108100 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7feb980117e0 tx=0x7feb98011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:35.244 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.388+0000 7feb9e7fc700 1 -- 172.21.15.84:0/946370689 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb98003bf0 con 0x7feba8108880 2024-05-22T22:52:35.244 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.389+0000 7febaf5db700 1 -- 172.21.15.84:0/946370689 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba8106830 con 0x7feba8108880 2024-05-22T22:52:35.244 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.389+0000 7feb9e7fc700 1 -- 172.21.15.84:0/946370689 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb9801a910 con 0x7feba8108880 2024-05-22T22:52:35.244 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.389+0000 7feb9e7fc700 1 -- 172.21.15.84:0/946370689 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb98024970 con 0x7feba8108880 2024-05-22T22:52:35.244 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.389+0000 7febaf5db700 1 -- 172.21.15.84:0/946370689 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba8106cf0 con 0x7feba8108880 2024-05-22T22:52:35.244 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.390+0000 7feb9e7fc700 1 -- 172.21.15.84:0/946370689 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7feb98022020 con 0x7feba8108880 2024-05-22T22:52:35.245 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.390+0000 7febaf5db700 1 -- 172.21.15.84:0/946370689 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb8c005360 con 0x7feba8108880 2024-05-22T22:52:35.245 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.390+0000 7feb9e7fc700 1 --2- 172.21.15.84:0/946370689 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7feb94039000 0x7feb9403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:35.245 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.390+0000 7feb9e7fc700 1 -- 172.21.15.84:0/946370689 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7feb9804d320 con 0x7feba8108880 2024-05-22T22:52:35.245 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.390+0000 7febacb76700 1 --2- 172.21.15.84:0/946370689 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7feb94039000 0x7feb9403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:35.245 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.391+0000 7febacb76700 1 --2- 172.21.15.84:0/946370689 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7feb94039000 0x7feb9403b4c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7feba4006fd0 tx=0x7feba4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:35.245 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.395+0000 7feb9e7fc700 1 -- 172.21.15.84:0/946370689 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7feb98011e20 con 0x7feba8108880 2024-05-22T22:52:35.245 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.598+0000 7febaf5db700 1 -- 172.21.15.84:0/946370689 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7feb8c005650 con 0x7feba8108880 2024-05-22T22:52:35.245 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.600+0000 7feb9e7fc700 1 -- 172.21.15.84:0/946370689 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7feb98016260 con 0x7feba8108880 2024-05-22T22:52:35.245 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.604+0000 7febaf5db700 1 -- 172.21.15.84:0/946370689 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7feb94039000 msgr2=0x7feb9403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:35.245 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.604+0000 7febaf5db700 1 --2- 172.21.15.84:0/946370689 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7feb94039000 0x7feb9403b4c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7feba4006fd0 tx=0x7feba4006e40 comp rx=0 tx=0).stop 2024-05-22T22:52:35.245 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.604+0000 7febaf5db700 1 -- 172.21.15.84:0/946370689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feba8108880 msgr2=0x7feba8108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:35.245 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.604+0000 7febaf5db700 1 --2- 172.21.15.84:0/946370689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feba8108880 0x7feba8108100 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7feb980117e0 tx=0x7feb98011c10 comp rx=0 tx=0).stop 2024-05-22T22:52:35.246 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.604+0000 7febaf5db700 1 -- 172.21.15.84:0/946370689 shutdown_connections 2024-05-22T22:52:35.246 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.604+0000 7febaf5db700 1 --2- 172.21.15.84:0/946370689 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7feb94039000 0x7feb9403b4c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:35.246 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.604+0000 7febaf5db700 1 --2- 172.21.15.84:0/946370689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feba8108880 0x7feba8108100 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:35.246 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.604+0000 7febaf5db700 1 -- 172.21.15.84:0/946370689 >> 172.21.15.84:0/946370689 conn(0x7feba81022b0 msgr2=0x7feba8102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:35.246 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.604+0000 7febaf5db700 1 -- 172.21.15.84:0/946370689 shutdown_connections 2024-05-22T22:52:35.246 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:34.604+0000 7febaf5db700 1 -- 172.21.15.84:0/946370689 wait complete. 2024-05-22T22:52:36.583 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.681+0000 7f4d0f076700 1 Processor -- start 2024-05-22T22:52:36.584 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.681+0000 7f4d0f076700 1 -- start start 2024-05-22T22:52:36.584 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.681+0000 7f4d0f076700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d08108890 0x7f4d0810ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:36.584 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.681+0000 7f4d0f076700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0807be80 con 0x7f4d08108890 2024-05-22T22:52:36.584 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.681+0000 7f4d0ce12700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d08108890 0x7f4d0810ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:36.584 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.681+0000 7f4d0ce12700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d08108890 0x7f4d0810ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44902/0 (socket says 172.21.15.84:44902) 2024-05-22T22:52:36.584 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.681+0000 7f4d0ce12700 1 -- 172.21.15.84:0/75201428 learned_addr learned my addr 172.21.15.84:0/75201428 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:36.584 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.681+0000 7f4d0ce12700 1 -- 172.21.15.84:0/75201428 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0810b1c0 con 0x7f4d08108890 2024-05-22T22:52:36.584 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.681+0000 7f4d0ce12700 1 --2- 172.21.15.84:0/75201428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d08108890 0x7f4d0810ac80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc0098a0 tx=0x7f4cfc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec87a4abc8a8b91 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:36.584 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.681+0000 7f4d077fe700 1 -- 172.21.15.84:0/75201428 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cfc004160 con 0x7f4d08108890 2024-05-22T22:52:36.584 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.682+0000 7f4d077fe700 1 -- 172.21.15.84:0/75201428 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cfc004300 con 0x7f4d08108890 2024-05-22T22:52:36.584 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.682+0000 7f4d077fe700 1 -- 172.21.15.84:0/75201428 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cfc011b20 con 0x7f4d08108890 2024-05-22T22:52:36.584 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.682+0000 7f4d0f076700 1 -- 172.21.15.84:0/75201428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d08108890 msgr2=0x7f4d0810ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:36.585 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.682+0000 7f4d0f076700 1 --2- 172.21.15.84:0/75201428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d08108890 0x7f4d0810ac80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc0098a0 tx=0x7f4cfc009bb0 comp rx=0 tx=0).stop 2024-05-22T22:52:36.585 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.682+0000 7f4d0f076700 1 -- 172.21.15.84:0/75201428 shutdown_connections 2024-05-22T22:52:36.585 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.682+0000 7f4d0f076700 1 --2- 172.21.15.84:0/75201428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d08108890 0x7f4d0810ac80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:36.585 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.682+0000 7f4d0f076700 1 -- 172.21.15.84:0/75201428 >> 172.21.15.84:0/75201428 conn(0x7f4d08102260 msgr2=0x7f4d081046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:36.585 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.682+0000 7f4d0f076700 1 -- 172.21.15.84:0/75201428 shutdown_connections 2024-05-22T22:52:36.585 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.682+0000 7f4d0f076700 1 -- 172.21.15.84:0/75201428 wait complete. 2024-05-22T22:52:36.585 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.683+0000 7f4d0f076700 1 Processor -- start 2024-05-22T22:52:36.585 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.683+0000 7f4d0f076700 1 -- start start 2024-05-22T22:52:36.585 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.683+0000 7f4d0f076700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d08108890 0x7f4d0819bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:36.585 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.683+0000 7f4d0f076700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0810c220 con 0x7f4d08108890 2024-05-22T22:52:36.585 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.683+0000 7f4d0ce12700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d08108890 0x7f4d0819bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:36.585 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.683+0000 7f4d0ce12700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d08108890 0x7f4d0819bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44904/0 (socket says 172.21.15.84:44904) 2024-05-22T22:52:36.585 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.683+0000 7f4d0ce12700 1 -- 172.21.15.84:0/3173682923 learned_addr learned my addr 172.21.15.84:0/3173682923 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:36.586 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.683+0000 7f4d0ce12700 1 -- 172.21.15.84:0/3173682923 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0819c220 con 0x7f4d08108890 2024-05-22T22:52:36.586 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.683+0000 7f4d0ce12700 1 --2- 172.21.15.84:0/3173682923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d08108890 0x7f4d0819bce0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc0117e0 tx=0x7f4cfc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:36.586 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.684+0000 7f4d05ffb700 1 -- 172.21.15.84:0/3173682923 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cfc003bf0 con 0x7f4d08108890 2024-05-22T22:52:36.586 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.684+0000 7f4d05ffb700 1 -- 172.21.15.84:0/3173682923 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cfc01a910 con 0x7f4d08108890 2024-05-22T22:52:36.586 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.684+0000 7f4d0f076700 1 -- 172.21.15.84:0/3173682923 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d0819c4b0 con 0x7f4d08108890 2024-05-22T22:52:36.586 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.684+0000 7f4d05ffb700 1 -- 172.21.15.84:0/3173682923 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cfc024970 con 0x7f4d08108890 2024-05-22T22:52:36.586 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.684+0000 7f4d0f076700 1 -- 172.21.15.84:0/3173682923 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d0819c910 con 0x7f4d08108890 2024-05-22T22:52:36.586 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.685+0000 7f4d05ffb700 1 -- 172.21.15.84:0/3173682923 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4cfc022020 con 0x7f4d08108890 2024-05-22T22:52:36.586 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.685+0000 7f4d0f076700 1 -- 172.21.15.84:0/3173682923 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d080639c0 con 0x7f4d08108890 2024-05-22T22:52:36.586 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.685+0000 7f4d05ffb700 1 --2- 172.21.15.84:0/3173682923 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f4cf8039000 0x7f4cf803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:36.586 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.685+0000 7f4d05ffb700 1 -- 172.21.15.84:0/3173682923 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4cfc04d520 con 0x7f4d08108890 2024-05-22T22:52:36.586 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.685+0000 7f4d07fff700 1 --2- 172.21.15.84:0/3173682923 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f4cf8039000 0x7f4cf803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:36.586 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.685+0000 7f4d07fff700 1 --2- 172.21.15.84:0/3173682923 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f4cf8039000 0x7f4cf803b4c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4006fd0 tx=0x7f4cf4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:36.587 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.690+0000 7f4d05ffb700 1 -- 172.21.15.84:0/3173682923 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4cfc011e20 con 0x7f4d08108890 2024-05-22T22:52:36.587 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.928+0000 7f4d0f076700 1 -- 172.21.15.84:0/3173682923 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f4d0819f370 con 0x7f4d08108890 2024-05-22T22:52:36.587 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.930+0000 7f4d05ffb700 1 -- 172.21.15.84:0/3173682923 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f4cfc00ba30 con 0x7f4d08108890 2024-05-22T22:52:36.587 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.933+0000 7f4d0f076700 1 -- 172.21.15.84:0/3173682923 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f4cf8039000 msgr2=0x7f4cf803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:36.587 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.933+0000 7f4d0f076700 1 --2- 172.21.15.84:0/3173682923 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f4cf8039000 0x7f4cf803b4c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4006fd0 tx=0x7f4cf4006e40 comp rx=0 tx=0).stop 2024-05-22T22:52:36.587 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.933+0000 7f4d0f076700 1 -- 172.21.15.84:0/3173682923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d08108890 msgr2=0x7f4d0819bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:36.587 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.933+0000 7f4d0f076700 1 --2- 172.21.15.84:0/3173682923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d08108890 0x7f4d0819bce0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc0117e0 tx=0x7f4cfc011c10 comp rx=0 tx=0).stop 2024-05-22T22:52:36.587 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.933+0000 7f4d0f076700 1 -- 172.21.15.84:0/3173682923 shutdown_connections 2024-05-22T22:52:36.587 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.933+0000 7f4d0f076700 1 --2- 172.21.15.84:0/3173682923 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f4cf8039000 0x7f4cf803b4c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:36.587 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.933+0000 7f4d0f076700 1 --2- 172.21.15.84:0/3173682923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d08108890 0x7f4d0819bce0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:36.587 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.933+0000 7f4d0f076700 1 -- 172.21.15.84:0/3173682923 >> 172.21.15.84:0/3173682923 conn(0x7f4d08102260 msgr2=0x7f4d08102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:36.587 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.933+0000 7f4d0f076700 1 -- 172.21.15.84:0/3173682923 shutdown_connections 2024-05-22T22:52:36.587 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:35.933+0000 7f4d0f076700 1 -- 172.21.15.84:0/3173682923 wait complete. 2024-05-22T22:52:36.587 INFO:teuthology.orchestra.run.smithi084.stdout:Enabling the dashboard module... 2024-05-22T22:52:37.115 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:36 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3173682923' entity='client.admin' 2024-05-22T22:52:38.776 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:38 smithi084 ceph-mon[32186]: from='mgr.14120 172.21.15.84:0/1139647952' entity='mgr.x' 2024-05-22T22:52:38.777 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:38 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3349953603' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-22T22:52:38.777 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:38 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: ignoring --setuser ceph since I am not root 2024-05-22T22:52:38.777 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:38 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: ignoring --setgroup ceph since I am not root 2024-05-22T22:52:38.777 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:38 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:38.643+0000 7f73fb8b1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-22T22:52:39.043 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:38 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:38.828+0000 7f73fb8b1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-22T22:52:39.043 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:38 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:38.929+0000 7f73fb8b1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-22T22:52:39.050 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.426+0000 7f1f4e8bd700 1 Processor -- start 2024-05-22T22:52:39.050 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.426+0000 7f1f4e8bd700 1 -- start start 2024-05-22T22:52:39.050 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.426+0000 7f1f4e8bd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f48107180 0x7f1f481075a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:39.050 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.426+0000 7f1f4e8bd700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1f48107b70 con 0x7f1f48107180 2024-05-22T22:52:39.050 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.427+0000 7f1f47fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f48107180 0x7f1f481075a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:39.050 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.427+0000 7f1f47fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f48107180 0x7f1f481075a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44906/0 (socket says 172.21.15.84:44906) 2024-05-22T22:52:39.051 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.427+0000 7f1f47fff700 1 -- 172.21.15.84:0/2886058388 learned_addr learned my addr 172.21.15.84:0/2886058388 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:39.051 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.427+0000 7f1f47fff700 1 -- 172.21.15.84:0/2886058388 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f48107ea0 con 0x7f1f48107180 2024-05-22T22:52:39.051 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.427+0000 7f1f47fff700 1 --2- 172.21.15.84:0/2886058388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f48107180 0x7f1f481075a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0098a0 tx=0x7f1f3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1053828b73f17139 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:39.051 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.427+0000 7f1f46ffd700 1 -- 172.21.15.84:0/2886058388 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f3c004160 con 0x7f1f48107180 2024-05-22T22:52:39.051 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.427+0000 7f1f46ffd700 1 -- 172.21.15.84:0/2886058388 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f3c004300 con 0x7f1f48107180 2024-05-22T22:52:39.051 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.427+0000 7f1f46ffd700 1 -- 172.21.15.84:0/2886058388 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f3c011b20 con 0x7f1f48107180 2024-05-22T22:52:39.051 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.428+0000 7f1f4e8bd700 1 -- 172.21.15.84:0/2886058388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f48107180 msgr2=0x7f1f481075a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:39.051 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.428+0000 7f1f4e8bd700 1 --2- 172.21.15.84:0/2886058388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f48107180 0x7f1f481075a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0098a0 tx=0x7f1f3c009bb0 comp rx=0 tx=0).stop 2024-05-22T22:52:39.051 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.428+0000 7f1f4e8bd700 1 -- 172.21.15.84:0/2886058388 shutdown_connections 2024-05-22T22:52:39.051 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.428+0000 7f1f4e8bd700 1 --2- 172.21.15.84:0/2886058388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f48107180 0x7f1f481075a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:39.051 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.428+0000 7f1f4e8bd700 1 -- 172.21.15.84:0/2886058388 >> 172.21.15.84:0/2886058388 conn(0x7f1f481019e0 msgr2=0x7f1f48103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:39.051 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.428+0000 7f1f4e8bd700 1 -- 172.21.15.84:0/2886058388 shutdown_connections 2024-05-22T22:52:39.051 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.428+0000 7f1f4e8bd700 1 -- 172.21.15.84:0/2886058388 wait complete. 2024-05-22T22:52:39.052 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.429+0000 7f1f4e8bd700 1 Processor -- start 2024-05-22T22:52:39.052 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.429+0000 7f1f4e8bd700 1 -- start start 2024-05-22T22:52:39.052 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.429+0000 7f1f4e8bd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f48107180 0x7f1f48106780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:39.052 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.429+0000 7f1f4e8bd700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4810b250 con 0x7f1f48107180 2024-05-22T22:52:39.052 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.430+0000 7f1f47fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f48107180 0x7f1f48106780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:39.052 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.430+0000 7f1f47fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f48107180 0x7f1f48106780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44912/0 (socket says 172.21.15.84:44912) 2024-05-22T22:52:39.052 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.430+0000 7f1f47fff700 1 -- 172.21.15.84:0/3349953603 learned_addr learned my addr 172.21.15.84:0/3349953603 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:39.052 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.430+0000 7f1f47fff700 1 -- 172.21.15.84:0/3349953603 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4819e9d0 con 0x7f1f48107180 2024-05-22T22:52:39.052 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.430+0000 7f1f47fff700 1 --2- 172.21.15.84:0/3349953603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f48107180 0x7f1f48106780 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0117e0 tx=0x7f1f3c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:39.052 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.430+0000 7f1f457fa700 1 -- 172.21.15.84:0/3349953603 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f3c003bf0 con 0x7f1f48107180 2024-05-22T22:52:39.052 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.430+0000 7f1f457fa700 1 -- 172.21.15.84:0/3349953603 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f3c01a910 con 0x7f1f48107180 2024-05-22T22:52:39.052 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.430+0000 7f1f4e8bd700 1 -- 172.21.15.84:0/3349953603 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f4819eba0 con 0x7f1f48107180 2024-05-22T22:52:39.052 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.430+0000 7f1f457fa700 1 -- 172.21.15.84:0/3349953603 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f3c024970 con 0x7f1f48107180 2024-05-22T22:52:39.053 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.430+0000 7f1f4e8bd700 1 -- 172.21.15.84:0/3349953603 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f4819ef40 con 0x7f1f48107180 2024-05-22T22:52:39.053 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.431+0000 7f1f457fa700 1 -- 172.21.15.84:0/3349953603 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1f3c022020 con 0x7f1f48107180 2024-05-22T22:52:39.053 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.431+0000 7f1f4e8bd700 1 -- 172.21.15.84:0/3349953603 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f480639c0 con 0x7f1f48107180 2024-05-22T22:52:39.053 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.432+0000 7f1f457fa700 1 --2- 172.21.15.84:0/3349953603 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f1f30039000 0x7f1f3003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:39.053 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.432+0000 7f1f457fa700 1 -- 172.21.15.84:0/3349953603 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1f3c04d490 con 0x7f1f48107180 2024-05-22T22:52:39.053 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.432+0000 7f1f477fe700 1 --2- 172.21.15.84:0/3349953603 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f1f30039000 0x7f1f3003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:39.053 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.432+0000 7f1f477fe700 1 --2- 172.21.15.84:0/3349953603 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f1f30039000 0x7f1f3003b4c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1f38006fd0 tx=0x7f1f38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:39.053 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.437+0000 7f1f457fa700 1 -- 172.21.15.84:0/3349953603 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1f3c011e20 con 0x7f1f48107180 2024-05-22T22:52:39.053 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:37.702+0000 7f1f4e8bd700 1 -- 172.21.15.84:0/3349953603 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f1f4819f350 con 0x7f1f48107180 2024-05-22T22:52:39.053 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:38.460+0000 7f1f457fa700 1 -- 172.21.15.84:0/3349953603 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f1f3c0172b0 con 0x7f1f48107180 2024-05-22T22:52:39.053 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:38.462+0000 7f1f457fa700 1 -- 172.21.15.84:0/3349953603 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f1f3c02d910 con 0x7f1f48107180 2024-05-22T22:52:39.053 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:38.464+0000 7f1f4e8bd700 1 -- 172.21.15.84:0/3349953603 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f1f30039000 msgr2=0x7f1f3003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:39.053 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:38.464+0000 7f1f4e8bd700 1 --2- 172.21.15.84:0/3349953603 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f1f30039000 0x7f1f3003b4c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1f38006fd0 tx=0x7f1f38006e40 comp rx=0 tx=0).stop 2024-05-22T22:52:39.054 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:38.464+0000 7f1f4e8bd700 1 -- 172.21.15.84:0/3349953603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f48107180 msgr2=0x7f1f48106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:39.054 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:38.464+0000 7f1f4e8bd700 1 --2- 172.21.15.84:0/3349953603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f48107180 0x7f1f48106780 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0117e0 tx=0x7f1f3c011c10 comp rx=0 tx=0).stop 2024-05-22T22:52:39.054 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:38.464+0000 7f1f4e8bd700 1 -- 172.21.15.84:0/3349953603 shutdown_connections 2024-05-22T22:52:39.054 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:38.464+0000 7f1f4e8bd700 1 --2- 172.21.15.84:0/3349953603 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7f1f30039000 0x7f1f3003b4c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:39.054 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:38.464+0000 7f1f4e8bd700 1 --2- 172.21.15.84:0/3349953603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f48107180 0x7f1f48106780 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:39.054 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:38.464+0000 7f1f4e8bd700 1 -- 172.21.15.84:0/3349953603 >> 172.21.15.84:0/3349953603 conn(0x7f1f481019e0 msgr2=0x7f1f481024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:39.054 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:38.464+0000 7f1f4e8bd700 1 -- 172.21.15.84:0/3349953603 shutdown_connections 2024-05-22T22:52:39.054 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:38.464+0000 7f1f4e8bd700 1 -- 172.21.15.84:0/3349953603 wait complete. 2024-05-22T22:52:39.298 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:39 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:39.243+0000 7f73fb8b1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-22T22:52:39.615 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:39 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:39.335+0000 7f73fb8b1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-22T22:52:39.616 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:39 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3349953603' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-22T22:52:39.616 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:39 smithi084 ceph-mon[32186]: mgrmap e9: x(active, since 19s) 2024-05-22T22:52:40.383 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout { 2024-05-22T22:52:40.383 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-22T22:52:40.383 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-22T22:52:40.383 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-22T22:52:40.383 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-22T22:52:40.383 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout } 2024-05-22T22:52:40.383 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.518+0000 7fe368d58700 1 Processor -- start 2024-05-22T22:52:40.383 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.518+0000 7fe368d58700 1 -- start start 2024-05-22T22:52:40.383 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.518+0000 7fe368d58700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe364108870 0x7fe36410ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:40.383 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.518+0000 7fe368d58700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe36407be60 con 0x7fe364108870 2024-05-22T22:52:40.383 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.518+0000 7fe36259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe364108870 0x7fe36410ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:40.383 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.518+0000 7fe36259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe364108870 0x7fe36410ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44936/0 (socket says 172.21.15.84:44936) 2024-05-22T22:52:40.384 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.518+0000 7fe36259c700 1 -- 172.21.15.84:0/3935308908 learned_addr learned my addr 172.21.15.84:0/3935308908 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:40.384 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.519+0000 7fe36259c700 1 -- 172.21.15.84:0/3935308908 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36410b1a0 con 0x7fe364108870 2024-05-22T22:52:40.384 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.519+0000 7fe36259c700 1 --2- 172.21.15.84:0/3935308908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe364108870 0x7fe36410ac60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe3540098a0 tx=0x7fe354009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43a057b89973261 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:40.384 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.519+0000 7fe36159a700 1 -- 172.21.15.84:0/3935308908 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe354004160 con 0x7fe364108870 2024-05-22T22:52:40.384 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.519+0000 7fe36159a700 1 -- 172.21.15.84:0/3935308908 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe354004300 con 0x7fe364108870 2024-05-22T22:52:40.384 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.519+0000 7fe36159a700 1 -- 172.21.15.84:0/3935308908 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe354011b20 con 0x7fe364108870 2024-05-22T22:52:40.384 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.520+0000 7fe368d58700 1 -- 172.21.15.84:0/3935308908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe364108870 msgr2=0x7fe36410ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:40.384 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.520+0000 7fe368d58700 1 --2- 172.21.15.84:0/3935308908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe364108870 0x7fe36410ac60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe3540098a0 tx=0x7fe354009bb0 comp rx=0 tx=0).stop 2024-05-22T22:52:40.384 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.520+0000 7fe368d58700 1 -- 172.21.15.84:0/3935308908 shutdown_connections 2024-05-22T22:52:40.384 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.520+0000 7fe368d58700 1 --2- 172.21.15.84:0/3935308908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe364108870 0x7fe36410ac60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:40.384 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.520+0000 7fe368d58700 1 -- 172.21.15.84:0/3935308908 >> 172.21.15.84:0/3935308908 conn(0x7fe364102260 msgr2=0x7fe364104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:40.384 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.520+0000 7fe368d58700 1 -- 172.21.15.84:0/3935308908 shutdown_connections 2024-05-22T22:52:40.384 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.520+0000 7fe368d58700 1 -- 172.21.15.84:0/3935308908 wait complete. 2024-05-22T22:52:40.384 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.521+0000 7fe368d58700 1 Processor -- start 2024-05-22T22:52:40.385 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.521+0000 7fe368d58700 1 -- start start 2024-05-22T22:52:40.385 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.521+0000 7fe368d58700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe364108870 0x7fe36419bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:40.385 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.521+0000 7fe368d58700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe36410c1b0 con 0x7fe364108870 2024-05-22T22:52:40.385 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.521+0000 7fe36259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe364108870 0x7fe36419bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:40.385 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.521+0000 7fe36259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe364108870 0x7fe36419bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44952/0 (socket says 172.21.15.84:44952) 2024-05-22T22:52:40.385 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.521+0000 7fe36259c700 1 -- 172.21.15.84:0/594078202 learned_addr learned my addr 172.21.15.84:0/594078202 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:40.385 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.521+0000 7fe36259c700 1 -- 172.21.15.84:0/594078202 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36419c1c0 con 0x7fe364108870 2024-05-22T22:52:40.385 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.522+0000 7fe36259c700 1 --2- 172.21.15.84:0/594078202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe364108870 0x7fe36419bc80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe3540117e0 tx=0x7fe354011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:40.385 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.522+0000 7fe3537fe700 1 -- 172.21.15.84:0/594078202 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe354003bf0 con 0x7fe364108870 2024-05-22T22:52:40.385 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.522+0000 7fe3537fe700 1 -- 172.21.15.84:0/594078202 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe35401a910 con 0x7fe364108870 2024-05-22T22:52:40.385 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.522+0000 7fe368d58700 1 -- 172.21.15.84:0/594078202 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe36419c450 con 0x7fe364108870 2024-05-22T22:52:40.385 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.522+0000 7fe3537fe700 1 -- 172.21.15.84:0/594078202 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe354024970 con 0x7fe364108870 2024-05-22T22:52:40.385 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.522+0000 7fe368d58700 1 -- 172.21.15.84:0/594078202 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe36419c8b0 con 0x7fe364108870 2024-05-22T22:52:40.385 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.523+0000 7fe3537fe700 1 -- 172.21.15.84:0/594078202 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fe354022020 con 0x7fe364108870 2024-05-22T22:52:40.386 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.523+0000 7fe368d58700 1 -- 172.21.15.84:0/594078202 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3640639c0 con 0x7fe364108870 2024-05-22T22:52:40.386 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.523+0000 7fe3537fe700 1 --2- 172.21.15.84:0/594078202 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fe34c0390b0 0x7fe34c03b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:40.386 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.523+0000 7fe3537fe700 1 -- 172.21.15.84:0/594078202 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe35404d3a0 con 0x7fe364108870 2024-05-22T22:52:40.386 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.523+0000 7fe361d9b700 1 -- 172.21.15.84:0/594078202 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fe34c0390b0 msgr2=0x7fe34c03b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/3278959854 2024-05-22T22:52:40.386 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.523+0000 7fe361d9b700 1 --2- 172.21.15.84:0/594078202 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fe34c0390b0 0x7fe34c03b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-22T22:52:40.386 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.528+0000 7fe3537fe700 1 -- 172.21.15.84:0/594078202 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe354011e20 con 0x7fe364108870 2024-05-22T22:52:40.386 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.724+0000 7fe361d9b700 1 -- 172.21.15.84:0/594078202 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fe34c0390b0 msgr2=0x7fe34c03b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/3278959854 2024-05-22T22:52:40.386 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.724+0000 7fe361d9b700 1 --2- 172.21.15.84:0/594078202 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fe34c0390b0 0x7fe34c03b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-22T22:52:40.386 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.818+0000 7fe368d58700 1 -- 172.21.15.84:0/594078202 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fe36419f6f0 con 0x7fe364108870 2024-05-22T22:52:40.386 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.818+0000 7fe3537fe700 1 -- 172.21.15.84:0/594078202 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fe35400ba30 con 0x7fe364108870 2024-05-22T22:52:40.386 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.821+0000 7fe368d58700 1 -- 172.21.15.84:0/594078202 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fe34c0390b0 msgr2=0x7fe34c03b570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-22T22:52:40.386 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.821+0000 7fe368d58700 1 --2- 172.21.15.84:0/594078202 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fe34c0390b0 0x7fe34c03b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:40.387 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.821+0000 7fe368d58700 1 -- 172.21.15.84:0/594078202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe364108870 msgr2=0x7fe36419bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:40.387 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.821+0000 7fe368d58700 1 --2- 172.21.15.84:0/594078202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe364108870 0x7fe36419bc80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe3540117e0 tx=0x7fe354011c10 comp rx=0 tx=0).stop 2024-05-22T22:52:40.387 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.821+0000 7fe368d58700 1 -- 172.21.15.84:0/594078202 shutdown_connections 2024-05-22T22:52:40.387 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.821+0000 7fe368d58700 1 --2- 172.21.15.84:0/594078202 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fe34c0390b0 0x7fe34c03b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:40.387 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.821+0000 7fe368d58700 1 --2- 172.21.15.84:0/594078202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe364108870 0x7fe36419bc80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:40.387 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.821+0000 7fe368d58700 1 -- 172.21.15.84:0/594078202 >> 172.21.15.84:0/594078202 conn(0x7fe364102260 msgr2=0x7fe364102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:40.387 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.821+0000 7fe368d58700 1 -- 172.21.15.84:0/594078202 shutdown_connections 2024-05-22T22:52:40.387 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:39.821+0000 7fe368d58700 1 -- 172.21.15.84:0/594078202 wait complete. 2024-05-22T22:52:40.387 INFO:teuthology.orchestra.run.smithi084.stdout:Waiting for the mgr to restart... 2024-05-22T22:52:40.387 INFO:teuthology.orchestra.run.smithi084.stdout:Waiting for mgr epoch 9... 2024-05-22T22:52:40.626 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:40 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/594078202' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-22T22:52:40.627 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:40 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:40.583+0000 7f73fb8b1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-22T22:52:41.011 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:40 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:40.759+0000 7f73fb8b1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-22T22:52:41.011 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:40 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:40.933+0000 7f73fb8b1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-22T22:52:41.307 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:41 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:41.224+0000 7f73fb8b1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-22T22:52:41.307 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:41 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:41.315+0000 7f73fb8b1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-22T22:52:41.615 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:41 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:41.517+0000 7f73fb8b1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-22T22:52:42.115 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:41 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:41.711+0000 7f73fb8b1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-22T22:52:42.615 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:42 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:42.310+0000 7f73fb8b1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-22T22:52:43.028 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:42 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:42.734+0000 7f73fb8b1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-22T22:52:43.029 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:42 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:42.833+0000 7f73fb8b1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-22T22:52:43.029 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:43 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:43.036+0000 7f73fb8b1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-22T22:52:43.814 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:43 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:43.556+0000 7f73fb8b1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-22T22:52:43.814 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:43 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:43.651+0000 7f73fb8b1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-22T22:52:44.115 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:43 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:43.899+0000 7f73fb8b1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-22T22:52:44.115 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:44 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:44.034+0000 7f73fb8b1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-22T22:52:44.565 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:44 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:44.132+0000 7f73fb8b1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-22T22:52:44.565 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:44 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:44.250+0000 7f73fb8b1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-22T22:52:44.565 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:44 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:44.573+0000 7f73fb8b1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-22T22:52:45.438 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:45 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:45.172+0000 7f73fb8b1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-22T22:52:45.438 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:52:45 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:52:45.393+0000 7f73fb8b1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-22T22:52:45.865 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:45 smithi084 ceph-mon[32186]: Active manager daemon x restarted 2024-05-22T22:52:45.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:45 smithi084 ceph-mon[32186]: Activating manager daemon x 2024-05-22T22:52:45.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:45 smithi084 ceph-mon[32186]: osdmap e3: 0 total, 0 up, 0 in 2024-05-22T22:52:45.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:45 smithi084 ceph-mon[32186]: mgrmap e10: x(active, starting, since 0.00340073s) 2024-05-22T22:52:45.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-22T22:52:45.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-22T22:52:45.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-22T22:52:45.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-22T22:52:45.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-22T22:52:45.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:45 smithi084 ceph-mon[32186]: Manager daemon x is now available 2024-05-22T22:52:45.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:52:45.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-22T22:52:46.615 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:46 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-22T22:52:46.615 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:46 smithi084 ceph-mon[32186]: mgrmap e11: x(active, since 1.00593s) 2024-05-22T22:52:46.991 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout { 2024-05-22T22:52:46.992 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-22T22:52:46.992 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-22T22:52:46.992 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout } 2024-05-22T22:52:46.992 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.813+0000 7fb58d0a5700 1 Processor -- start 2024-05-22T22:52:46.992 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.813+0000 7fb58d0a5700 1 -- start start 2024-05-22T22:52:46.992 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.813+0000 7fb58d0a5700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb588108860 0x7fb58810ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:46.992 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.813+0000 7fb58d0a5700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb58807be80 con 0x7fb588108860 2024-05-22T22:52:46.992 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.814+0000 7fb586d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb588108860 0x7fb58810ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:46.992 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.814+0000 7fb586d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb588108860 0x7fb58810ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44958/0 (socket says 172.21.15.84:44958) 2024-05-22T22:52:46.992 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.814+0000 7fb586d9d700 1 -- 172.21.15.84:0/1980195192 learned_addr learned my addr 172.21.15.84:0/1980195192 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:46.992 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.814+0000 7fb586d9d700 1 -- 172.21.15.84:0/1980195192 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58810b190 con 0x7fb588108860 2024-05-22T22:52:46.992 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.814+0000 7fb586d9d700 1 --2- 172.21.15.84:0/1980195192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb588108860 0x7fb58810ac50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb578009e80 tx=0x7fb57800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebeedf3def4d0611 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:46.993 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.814+0000 7fb585d9b700 1 -- 172.21.15.84:0/1980195192 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb578017070 con 0x7fb588108860 2024-05-22T22:52:46.993 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.814+0000 7fb585d9b700 1 -- 172.21.15.84:0/1980195192 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb57800b8c0 con 0x7fb588108860 2024-05-22T22:52:46.993 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.814+0000 7fb585d9b700 1 -- 172.21.15.84:0/1980195192 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5780045c0 con 0x7fb588108860 2024-05-22T22:52:46.993 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.815+0000 7fb58d0a5700 1 -- 172.21.15.84:0/1980195192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb588108860 msgr2=0x7fb58810ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:46.993 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.815+0000 7fb58d0a5700 1 --2- 172.21.15.84:0/1980195192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb588108860 0x7fb58810ac50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb578009e80 tx=0x7fb57800b330 comp rx=0 tx=0).stop 2024-05-22T22:52:46.993 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.815+0000 7fb58d0a5700 1 -- 172.21.15.84:0/1980195192 shutdown_connections 2024-05-22T22:52:46.993 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.815+0000 7fb58d0a5700 1 --2- 172.21.15.84:0/1980195192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb588108860 0x7fb58810ac50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:46.993 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.815+0000 7fb58d0a5700 1 -- 172.21.15.84:0/1980195192 >> 172.21.15.84:0/1980195192 conn(0x7fb588102250 msgr2=0x7fb588104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:46.993 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.815+0000 7fb58d0a5700 1 -- 172.21.15.84:0/1980195192 shutdown_connections 2024-05-22T22:52:46.993 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.815+0000 7fb58d0a5700 1 -- 172.21.15.84:0/1980195192 wait complete. 2024-05-22T22:52:46.993 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.816+0000 7fb58d0a5700 1 Processor -- start 2024-05-22T22:52:46.994 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.816+0000 7fb58d0a5700 1 -- start start 2024-05-22T22:52:46.994 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.816+0000 7fb58d0a5700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb588108860 0x7fb58819bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:46.994 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.816+0000 7fb58d0a5700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb58810c1f0 con 0x7fb588108860 2024-05-22T22:52:46.994 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.816+0000 7fb586d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb588108860 0x7fb58819bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:46.994 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.816+0000 7fb586d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb588108860 0x7fb58819bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44968/0 (socket says 172.21.15.84:44968) 2024-05-22T22:52:46.994 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.816+0000 7fb586d9d700 1 -- 172.21.15.84:0/3326462999 learned_addr learned my addr 172.21.15.84:0/3326462999 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:46.994 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.817+0000 7fb586d9d700 1 -- 172.21.15.84:0/3326462999 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58819c200 con 0x7fb588108860 2024-05-22T22:52:46.994 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.817+0000 7fb586d9d700 1 --2- 172.21.15.84:0/3326462999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb588108860 0x7fb58819bcc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb57800b7b0 tx=0x7fb57800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:46.994 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.817+0000 7fb57ffff700 1 -- 172.21.15.84:0/3326462999 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb578020020 con 0x7fb588108860 2024-05-22T22:52:46.994 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.817+0000 7fb57ffff700 1 -- 172.21.15.84:0/3326462999 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb578003d90 con 0x7fb588108860 2024-05-22T22:52:46.994 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.817+0000 7fb57ffff700 1 -- 172.21.15.84:0/3326462999 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb578017040 con 0x7fb588108860 2024-05-22T22:52:46.994 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.817+0000 7fb58d0a5700 1 -- 172.21.15.84:0/3326462999 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb58819c490 con 0x7fb588108860 2024-05-22T22:52:46.994 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.817+0000 7fb58d0a5700 1 -- 172.21.15.84:0/3326462999 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb58819f180 con 0x7fb588108860 2024-05-22T22:52:46.995 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.818+0000 7fb57ffff700 1 -- 172.21.15.84:0/3326462999 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fb578023020 con 0x7fb588108860 2024-05-22T22:52:46.995 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.818+0000 7fb57ffff700 1 --2- 172.21.15.84:0/3326462999 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fb574039050 0x7fb57403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:46.995 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.818+0000 7fb57ffff700 1 -- 172.21.15.84:0/3326462999 --> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb57403bc20 con 0x7fb574039050 2024-05-22T22:52:46.995 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.818+0000 7fb57ffff700 1 -- 172.21.15.84:0/3326462999 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb57800f050 con 0x7fb588108860 2024-05-22T22:52:46.995 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.818+0000 7fb58659c700 1 -- 172.21.15.84:0/3326462999 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fb574039050 msgr2=0x7fb57403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/3278959854 2024-05-22T22:52:46.995 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:40.818+0000 7fb58659c700 1 --2- 172.21.15.84:0/3326462999 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fb574039050 0x7fb57403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-22T22:52:46.995 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:41.019+0000 7fb58659c700 1 -- 172.21.15.84:0/3326462999 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fb574039050 msgr2=0x7fb57403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/3278959854 2024-05-22T22:52:46.995 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:41.019+0000 7fb58659c700 1 --2- 172.21.15.84:0/3326462999 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fb574039050 0x7fb57403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-22T22:52:46.995 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:41.420+0000 7fb58659c700 1 -- 172.21.15.84:0/3326462999 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fb574039050 msgr2=0x7fb57403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/3278959854 2024-05-22T22:52:46.995 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:41.420+0000 7fb58659c700 1 --2- 172.21.15.84:0/3326462999 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fb574039050 0x7fb57403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-22T22:52:46.995 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:42.221+0000 7fb58659c700 1 -- 172.21.15.84:0/3326462999 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fb574039050 msgr2=0x7fb57403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/3278959854 2024-05-22T22:52:46.995 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:42.221+0000 7fb58659c700 1 --2- 172.21.15.84:0/3326462999 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fb574039050 0x7fb57403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-22T22:52:46.995 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:43.822+0000 7fb58659c700 1 -- 172.21.15.84:0/3326462999 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fb574039050 msgr2=0x7fb57403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/3278959854 2024-05-22T22:52:46.996 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:43.822+0000 7fb58659c700 1 --2- 172.21.15.84:0/3326462999 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fb574039050 0x7fb57403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-22T22:52:46.996 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:45.396+0000 7fb57ffff700 1 -- 172.21.15.84:0/3326462999 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7fb578023d80 con 0x7fb588108860 2024-05-22T22:52:46.996 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:45.396+0000 7fb57ffff700 1 -- 172.21.15.84:0/3326462999 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fb574039050 msgr2=0x7fb57403b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-22T22:52:46.996 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:45.396+0000 7fb57ffff700 1 --2- 172.21.15.84:0/3326462999 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fb574039050 0x7fb57403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:46.996 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:46.398+0000 7fb57ffff700 1 -- 172.21.15.84:0/3326462999 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fb57804e780 con 0x7fb588108860 2024-05-22T22:52:46.996 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:46.398+0000 7fb57ffff700 1 --2- 172.21.15.84:0/3326462999 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb57403cb10 0x7fb57403ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:46.996 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:46.398+0000 7fb57ffff700 1 -- 172.21.15.84:0/3326462999 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb578022e70 con 0x7fb57403cb10 2024-05-22T22:52:46.996 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:46.399+0000 7fb58659c700 1 --2- 172.21.15.84:0/3326462999 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb57403cb10 0x7fb57403ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:46.996 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:46.399+0000 7fb58659c700 1 --2- 172.21.15.84:0/3326462999 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb57403cb10 0x7fb57403ef00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb570003a10 tx=0x7fb5700092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:46.996 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:46.400+0000 7fb57ffff700 1 -- 172.21.15.84:0/3326462999 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fb578022e70 con 0x7fb57403cb10 2024-05-22T22:52:46.996 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:46.414+0000 7fb58d0a5700 1 -- 172.21.15.84:0/3326462999 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fb5880639c0 con 0x7fb57403cb10 2024-05-22T22:52:46.996 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:46.414+0000 7fb57ffff700 1 -- 172.21.15.84:0/3326462999 <== mgr.14152 v2:172.21.15.84:6800/4223589140 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fb5880639c0 con 0x7fb57403cb10 2024-05-22T22:52:46.997 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:46.415+0000 7fb58d0a5700 1 -- 172.21.15.84:0/3326462999 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb57403cb10 msgr2=0x7fb57403ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:46.997 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:46.415+0000 7fb58d0a5700 1 --2- 172.21.15.84:0/3326462999 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb57403cb10 0x7fb57403ef00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb570003a10 tx=0x7fb5700092b0 comp rx=0 tx=0).stop 2024-05-22T22:52:46.997 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:46.415+0000 7fb58d0a5700 1 -- 172.21.15.84:0/3326462999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb588108860 msgr2=0x7fb58819bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:46.997 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:46.415+0000 7fb58d0a5700 1 --2- 172.21.15.84:0/3326462999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb588108860 0x7fb58819bcc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb57800b7b0 tx=0x7fb57800bd50 comp rx=0 tx=0).stop 2024-05-22T22:52:46.997 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:46.415+0000 7fb58d0a5700 1 -- 172.21.15.84:0/3326462999 shutdown_connections 2024-05-22T22:52:46.997 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:46.415+0000 7fb58d0a5700 1 --2- 172.21.15.84:0/3326462999 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb57403cb10 0x7fb57403ef00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:46.997 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:46.415+0000 7fb58d0a5700 1 --2- 172.21.15.84:0/3326462999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb588108860 0x7fb58819bcc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:46.997 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:46.415+0000 7fb58d0a5700 1 --2- 172.21.15.84:0/3326462999 >> [v2:172.21.15.84:6800/3278959854,v1:172.21.15.84:6801/3278959854] conn(0x7fb574039050 0x7fb57403b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:46.997 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:46.415+0000 7fb58d0a5700 1 -- 172.21.15.84:0/3326462999 >> 172.21.15.84:0/3326462999 conn(0x7fb588102250 msgr2=0x7fb588102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:46.997 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:46.415+0000 7fb58d0a5700 1 -- 172.21.15.84:0/3326462999 shutdown_connections 2024-05-22T22:52:46.997 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:46.415+0000 7fb58d0a5700 1 -- 172.21.15.84:0/3326462999 wait complete. 2024-05-22T22:52:46.997 INFO:teuthology.orchestra.run.smithi084.stdout:mgr epoch 9 is available 2024-05-22T22:52:46.997 INFO:teuthology.orchestra.run.smithi084.stdout:Generating a dashboard self-signed certificate... 2024-05-22T22:52:47.615 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:47 smithi084 ceph-mon[32186]: [22/May/2024:22:52:46] ENGINE Bus STARTING 2024-05-22T22:52:47.615 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:47 smithi084 ceph-mon[32186]: [22/May/2024:22:52:46] ENGINE Serving on http://172.21.15.84:8765 2024-05-22T22:52:47.616 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:47 smithi084 ceph-mon[32186]: [22/May/2024:22:52:46] ENGINE Serving on https://172.21.15.84:7150 2024-05-22T22:52:47.616 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:47 smithi084 ceph-mon[32186]: [22/May/2024:22:52:46] ENGINE Bus STARTED 2024-05-22T22:52:47.616 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:47 smithi084 ceph-mon[32186]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-22T22:52:47.616 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:47 smithi084 ceph-mon[32186]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-22T22:52:48.332 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-22T22:52:48.332 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.412+0000 7f7827bb3700 1 Processor -- start 2024-05-22T22:52:48.332 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.412+0000 7f7827bb3700 1 -- start start 2024-05-22T22:52:48.333 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.413+0000 7f7827bb3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7820108870 0x7f782010ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:48.333 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.413+0000 7f7827bb3700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f782007bea0 con 0x7f7820108870 2024-05-22T22:52:48.333 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.413+0000 7f782594f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7820108870 0x7f782010ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:48.333 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.413+0000 7f782594f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7820108870 0x7f782010ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36882/0 (socket says 172.21.15.84:36882) 2024-05-22T22:52:48.333 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.413+0000 7f782594f700 1 -- 172.21.15.84:0/3132450230 learned_addr learned my addr 172.21.15.84:0/3132450230 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:48.333 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.413+0000 7f782594f700 1 -- 172.21.15.84:0/3132450230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782010b1a0 con 0x7f7820108870 2024-05-22T22:52:48.333 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.413+0000 7f782594f700 1 --2- 172.21.15.84:0/3132450230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7820108870 0x7f782010ac60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f78100098a0 tx=0x7f7810009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1057ac01965ccf59 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:48.333 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.413+0000 7f782494d700 1 -- 172.21.15.84:0/3132450230 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7810004160 con 0x7f7820108870 2024-05-22T22:52:48.333 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.414+0000 7f782494d700 1 -- 172.21.15.84:0/3132450230 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7810004300 con 0x7f7820108870 2024-05-22T22:52:48.333 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.414+0000 7f782494d700 1 -- 172.21.15.84:0/3132450230 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7810011b20 con 0x7f7820108870 2024-05-22T22:52:48.333 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.414+0000 7f7827bb3700 1 -- 172.21.15.84:0/3132450230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7820108870 msgr2=0x7f782010ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:48.333 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.414+0000 7f7827bb3700 1 --2- 172.21.15.84:0/3132450230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7820108870 0x7f782010ac60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f78100098a0 tx=0x7f7810009bb0 comp rx=0 tx=0).stop 2024-05-22T22:52:48.334 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.414+0000 7f7827bb3700 1 -- 172.21.15.84:0/3132450230 shutdown_connections 2024-05-22T22:52:48.334 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.414+0000 7f7827bb3700 1 --2- 172.21.15.84:0/3132450230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7820108870 0x7f782010ac60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:48.334 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.414+0000 7f7827bb3700 1 -- 172.21.15.84:0/3132450230 >> 172.21.15.84:0/3132450230 conn(0x7f7820102280 msgr2=0x7f78201046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:48.334 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.414+0000 7f7827bb3700 1 -- 172.21.15.84:0/3132450230 shutdown_connections 2024-05-22T22:52:48.334 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.414+0000 7f7827bb3700 1 -- 172.21.15.84:0/3132450230 wait complete. 2024-05-22T22:52:48.334 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.415+0000 7f7827bb3700 1 Processor -- start 2024-05-22T22:52:48.334 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.415+0000 7f7827bb3700 1 -- start start 2024-05-22T22:52:48.334 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.415+0000 7f7827bb3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7820108870 0x7f782019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:48.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.415+0000 7f7827bb3700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f782010c1a0 con 0x7f7820108870 2024-05-22T22:52:48.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.415+0000 7f782594f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7820108870 0x7f782019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:48.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.415+0000 7f782594f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7820108870 0x7f782019bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36894/0 (socket says 172.21.15.84:36894) 2024-05-22T22:52:48.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.415+0000 7f782594f700 1 -- 172.21.15.84:0/2216718464 learned_addr learned my addr 172.21.15.84:0/2216718464 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:48.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.415+0000 7f782594f700 1 -- 172.21.15.84:0/2216718464 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782019c1d0 con 0x7f7820108870 2024-05-22T22:52:48.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.415+0000 7f782594f700 1 --2- 172.21.15.84:0/2216718464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7820108870 0x7f782019bc90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f78100117e0 tx=0x7f7810011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:48.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.416+0000 7f7816ffd700 1 -- 172.21.15.84:0/2216718464 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7810003bf0 con 0x7f7820108870 2024-05-22T22:52:48.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.416+0000 7f7816ffd700 1 -- 172.21.15.84:0/2216718464 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f781001a910 con 0x7f7820108870 2024-05-22T22:52:48.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.416+0000 7f7827bb3700 1 -- 172.21.15.84:0/2216718464 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f782019c4c0 con 0x7f7820108870 2024-05-22T22:52:48.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.416+0000 7f7816ffd700 1 -- 172.21.15.84:0/2216718464 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7810024970 con 0x7f7820108870 2024-05-22T22:52:48.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.416+0000 7f7827bb3700 1 -- 172.21.15.84:0/2216718464 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f782019f150 con 0x7f7820108870 2024-05-22T22:52:48.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.417+0000 7f7827bb3700 1 -- 172.21.15.84:0/2216718464 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78200639c0 con 0x7f7820108870 2024-05-22T22:52:48.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.417+0000 7f7816ffd700 1 -- 172.21.15.84:0/2216718464 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f7810022020 con 0x7f7820108870 2024-05-22T22:52:48.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.417+0000 7f7816ffd700 1 --2- 172.21.15.84:0/2216718464 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f780c038f90 0x7f780c03b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:48.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.417+0000 7f7816ffd700 1 -- 172.21.15.84:0/2216718464 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f781004d240 con 0x7f7820108870 2024-05-22T22:52:48.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.417+0000 7f782514e700 1 --2- 172.21.15.84:0/2216718464 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f780c038f90 0x7f780c03b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:48.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.417+0000 7f782514e700 1 --2- 172.21.15.84:0/2216718464 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f780c038f90 0x7f780c03b450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f781c006fd0 tx=0x7f781c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:48.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.422+0000 7f7816ffd700 1 -- 172.21.15.84:0/2216718464 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7810011e20 con 0x7f7820108870 2024-05-22T22:52:48.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.452+0000 7f7816ffd700 1 -- 172.21.15.84:0/2216718464 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f781001a430 con 0x7f7820108870 2024-05-22T22:52:48.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.626+0000 7f7827bb3700 1 -- 172.21.15.84:0/2216718464 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f7820103450 con 0x7f780c038f90 2024-05-22T22:52:48.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.765+0000 7f7816ffd700 1 -- 172.21.15.84:0/2216718464 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f7820103450 con 0x7f780c038f90 2024-05-22T22:52:48.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.767+0000 7f7827bb3700 1 -- 172.21.15.84:0/2216718464 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f780c038f90 msgr2=0x7f780c03b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:48.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.767+0000 7f7827bb3700 1 --2- 172.21.15.84:0/2216718464 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f780c038f90 0x7f780c03b450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f781c006fd0 tx=0x7f781c006e40 comp rx=0 tx=0).stop 2024-05-22T22:52:48.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.767+0000 7f7827bb3700 1 -- 172.21.15.84:0/2216718464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7820108870 msgr2=0x7f782019bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:48.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.767+0000 7f7827bb3700 1 --2- 172.21.15.84:0/2216718464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7820108870 0x7f782019bc90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f78100117e0 tx=0x7f7810011c10 comp rx=0 tx=0).stop 2024-05-22T22:52:48.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.767+0000 7f7827bb3700 1 -- 172.21.15.84:0/2216718464 shutdown_connections 2024-05-22T22:52:48.337 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.767+0000 7f7827bb3700 1 --2- 172.21.15.84:0/2216718464 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f780c038f90 0x7f780c03b450 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:48.337 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.767+0000 7f7827bb3700 1 --2- 172.21.15.84:0/2216718464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7820108870 0x7f782019bc90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:48.337 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.767+0000 7f7827bb3700 1 -- 172.21.15.84:0/2216718464 >> 172.21.15.84:0/2216718464 conn(0x7f7820102280 msgr2=0x7f7820102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:48.337 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.768+0000 7f7827bb3700 1 -- 172.21.15.84:0/2216718464 shutdown_connections 2024-05-22T22:52:48.337 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:47.768+0000 7f7827bb3700 1 -- 172.21.15.84:0/2216718464 wait complete. 2024-05-22T22:52:48.337 INFO:teuthology.orchestra.run.smithi084.stdout:Creating initial admin user... 2024-05-22T22:52:48.583 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:48 smithi084 ceph-mon[32186]: mgrmap e12: x(active, since 2s) 2024-05-22T22:52:48.583 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:52:48.583 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:52:48.583 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:52:49.806 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:49 smithi084 ceph-mon[32186]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:52:49.806 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:49 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:52:49.806 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:49 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:52:49.812 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$d2YoMkkS8gPzJZOzjqY59OD2z2huTxgd9xuvzxd92hmY7e9zcTZny", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1716418369, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-22T22:52:49.812 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.792+0000 7f5850bde700 1 Processor -- start 2024-05-22T22:52:49.812 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.792+0000 7f5850bde700 1 -- start start 2024-05-22T22:52:49.812 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.792+0000 7f5850bde700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c108820 0x7f584c10ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:49.812 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.793+0000 7f5850bde700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f584c07bec0 con 0x7f584c108820 2024-05-22T22:52:49.812 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.793+0000 7f584a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c108820 0x7f584c10ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:49.812 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.793+0000 7f584a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c108820 0x7f584c10ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36908/0 (socket says 172.21.15.84:36908) 2024-05-22T22:52:49.812 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.793+0000 7f584a59c700 1 -- 172.21.15.84:0/1590172653 learned_addr learned my addr 172.21.15.84:0/1590172653 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:49.812 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.793+0000 7f584a59c700 1 -- 172.21.15.84:0/1590172653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584c10b150 con 0x7f584c108820 2024-05-22T22:52:49.813 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.793+0000 7f584a59c700 1 --2- 172.21.15.84:0/1590172653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c108820 0x7f584c10ac10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f583c009e80 tx=0x7f583c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a093ec381fdf5f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:49.813 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.793+0000 7f584959a700 1 -- 172.21.15.84:0/1590172653 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f583c017070 con 0x7f584c108820 2024-05-22T22:52:49.813 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.793+0000 7f584959a700 1 -- 172.21.15.84:0/1590172653 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f583c00b8c0 con 0x7f584c108820 2024-05-22T22:52:49.813 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.793+0000 7f584959a700 1 -- 172.21.15.84:0/1590172653 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f583c0045c0 con 0x7f584c108820 2024-05-22T22:52:49.813 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.794+0000 7f5850bde700 1 -- 172.21.15.84:0/1590172653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c108820 msgr2=0x7f584c10ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:49.813 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.794+0000 7f5850bde700 1 --2- 172.21.15.84:0/1590172653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c108820 0x7f584c10ac10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f583c009e80 tx=0x7f583c00b330 comp rx=0 tx=0).stop 2024-05-22T22:52:49.813 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.794+0000 7f5850bde700 1 -- 172.21.15.84:0/1590172653 shutdown_connections 2024-05-22T22:52:49.813 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.794+0000 7f5850bde700 1 --2- 172.21.15.84:0/1590172653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c108820 0x7f584c10ac10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:49.813 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.794+0000 7f5850bde700 1 -- 172.21.15.84:0/1590172653 >> 172.21.15.84:0/1590172653 conn(0x7f584c102250 msgr2=0x7f584c104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:49.813 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.794+0000 7f5850bde700 1 -- 172.21.15.84:0/1590172653 shutdown_connections 2024-05-22T22:52:49.813 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.794+0000 7f5850bde700 1 -- 172.21.15.84:0/1590172653 wait complete. 2024-05-22T22:52:49.813 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.795+0000 7f5850bde700 1 Processor -- start 2024-05-22T22:52:49.813 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.795+0000 7f5850bde700 1 -- start start 2024-05-22T22:52:49.814 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.795+0000 7f5850bde700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c108820 0x7f584c19bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:49.814 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.795+0000 7f5850bde700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f584c10c1c0 con 0x7f584c108820 2024-05-22T22:52:49.814 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.795+0000 7f584a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c108820 0x7f584c19bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:49.814 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.795+0000 7f584a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c108820 0x7f584c19bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36916/0 (socket says 172.21.15.84:36916) 2024-05-22T22:52:49.814 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.795+0000 7f584a59c700 1 -- 172.21.15.84:0/3181960748 learned_addr learned my addr 172.21.15.84:0/3181960748 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:49.814 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.796+0000 7f584a59c700 1 -- 172.21.15.84:0/3181960748 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584c19c200 con 0x7f584c108820 2024-05-22T22:52:49.814 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.796+0000 7f584a59c700 1 --2- 172.21.15.84:0/3181960748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c108820 0x7f584c19bcc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f583c009fd0 tx=0x7f583c003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:49.814 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.796+0000 7f58437fe700 1 -- 172.21.15.84:0/3181960748 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f583c020020 con 0x7f584c108820 2024-05-22T22:52:49.814 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.796+0000 7f5850bde700 1 -- 172.21.15.84:0/3181960748 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f584c19c490 con 0x7f584c108820 2024-05-22T22:52:49.814 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.796+0000 7f58437fe700 1 -- 172.21.15.84:0/3181960748 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f583c00bba0 con 0x7f584c108820 2024-05-22T22:52:49.814 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.796+0000 7f58437fe700 1 -- 172.21.15.84:0/3181960748 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f583c017040 con 0x7f584c108820 2024-05-22T22:52:49.814 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.796+0000 7f5850bde700 1 -- 172.21.15.84:0/3181960748 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f584c19f180 con 0x7f584c108820 2024-05-22T22:52:49.814 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.797+0000 7f58437fe700 1 -- 172.21.15.84:0/3181960748 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f583c023020 con 0x7f584c108820 2024-05-22T22:52:49.815 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.797+0000 7f5850bde700 1 -- 172.21.15.84:0/3181960748 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f584c0639c0 con 0x7f584c108820 2024-05-22T22:52:49.815 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.797+0000 7f58437fe700 1 --2- 172.21.15.84:0/3181960748 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f5838039100 0x7f583803b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:49.815 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.797+0000 7f58437fe700 1 -- 172.21.15.84:0/3181960748 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f583c00f050 con 0x7f584c108820 2024-05-22T22:52:49.815 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.797+0000 7f5849d9b700 1 --2- 172.21.15.84:0/3181960748 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f5838039100 0x7f583803b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:49.815 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.798+0000 7f5849d9b700 1 --2- 172.21.15.84:0/3181960748 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f5838039100 0x7f583803b5c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5834006fd0 tx=0x7f5834006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:49.815 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:48.802+0000 7f58437fe700 1 -- 172.21.15.84:0/3181960748 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f583c02a020 con 0x7f584c108820 2024-05-22T22:52:49.815 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:49.023+0000 7f5850bde700 1 -- 172.21.15.84:0/3181960748 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f584c103420 con 0x7f5838039100 2024-05-22T22:52:49.816 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:49.253+0000 7f58437fe700 1 -- 172.21.15.84:0/3181960748 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f584c103420 con 0x7f5838039100 2024-05-22T22:52:49.816 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:49.255+0000 7f5850bde700 1 -- 172.21.15.84:0/3181960748 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f5838039100 msgr2=0x7f583803b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:49.816 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:49.255+0000 7f5850bde700 1 --2- 172.21.15.84:0/3181960748 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f5838039100 0x7f583803b5c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5834006fd0 tx=0x7f5834006e40 comp rx=0 tx=0).stop 2024-05-22T22:52:49.817 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:49.255+0000 7f5850bde700 1 -- 172.21.15.84:0/3181960748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c108820 msgr2=0x7f584c19bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:49.817 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:49.255+0000 7f5850bde700 1 --2- 172.21.15.84:0/3181960748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c108820 0x7f584c19bcc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f583c009fd0 tx=0x7f583c003d00 comp rx=0 tx=0).stop 2024-05-22T22:52:49.817 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:49.255+0000 7f5850bde700 1 -- 172.21.15.84:0/3181960748 shutdown_connections 2024-05-22T22:52:49.817 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:49.255+0000 7f5850bde700 1 --2- 172.21.15.84:0/3181960748 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f5838039100 0x7f583803b5c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:49.817 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:49.255+0000 7f5850bde700 1 --2- 172.21.15.84:0/3181960748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c108820 0x7f584c19bcc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:49.817 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:49.255+0000 7f5850bde700 1 -- 172.21.15.84:0/3181960748 >> 172.21.15.84:0/3181960748 conn(0x7f584c102250 msgr2=0x7f584c102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:49.817 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:49.255+0000 7f5850bde700 1 -- 172.21.15.84:0/3181960748 shutdown_connections 2024-05-22T22:52:49.817 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:49.255+0000 7f5850bde700 1 -- 172.21.15.84:0/3181960748 wait complete. 2024-05-22T22:52:49.817 INFO:teuthology.orchestra.run.smithi084.stdout:Fetching dashboard port number... 2024-05-22T22:52:50.615 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:50 smithi084 ceph-mon[32186]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:52:50.616 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:50 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1144843757' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-22T22:52:51.030 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 8443 2024-05-22T22:52:51.031 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.201+0000 7fa434c56700 1 Processor -- start 2024-05-22T22:52:51.031 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.201+0000 7fa434c56700 1 -- start start 2024-05-22T22:52:51.031 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.201+0000 7fa434c56700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa430106cb0 0x7fa4301070d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:51.031 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.201+0000 7fa434c56700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa4301076a0 con 0x7fa430106cb0 2024-05-22T22:52:51.031 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.201+0000 7fa42e59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa430106cb0 0x7fa4301070d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:51.031 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.202+0000 7fa42e59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa430106cb0 0x7fa4301070d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36922/0 (socket says 172.21.15.84:36922) 2024-05-22T22:52:51.031 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.202+0000 7fa42e59c700 1 -- 172.21.15.84:0/3744465107 learned_addr learned my addr 172.21.15.84:0/3744465107 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:51.031 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.202+0000 7fa42e59c700 1 -- 172.21.15.84:0/3744465107 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa430107eb0 con 0x7fa430106cb0 2024-05-22T22:52:51.031 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.202+0000 7fa42e59c700 1 --2- 172.21.15.84:0/3744465107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa430106cb0 0x7fa4301070d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa42000bb40 tx=0x7fa42000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc22cb5bb022b15c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:51.031 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.202+0000 7fa42d59a700 1 -- 172.21.15.84:0/3744465107 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa42000dbe0 con 0x7fa430106cb0 2024-05-22T22:52:51.031 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.202+0000 7fa42d59a700 1 -- 172.21.15.84:0/3744465107 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa420004510 con 0x7fa430106cb0 2024-05-22T22:52:51.031 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.202+0000 7fa42d59a700 1 -- 172.21.15.84:0/3744465107 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa420010470 con 0x7fa430106cb0 2024-05-22T22:52:51.031 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.203+0000 7fa434c56700 1 -- 172.21.15.84:0/3744465107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa430106cb0 msgr2=0x7fa4301070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:51.032 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.203+0000 7fa434c56700 1 --2- 172.21.15.84:0/3744465107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa430106cb0 0x7fa4301070d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa42000bb40 tx=0x7fa42000be50 comp rx=0 tx=0).stop 2024-05-22T22:52:51.032 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.203+0000 7fa434c56700 1 -- 172.21.15.84:0/3744465107 shutdown_connections 2024-05-22T22:52:51.032 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.203+0000 7fa434c56700 1 --2- 172.21.15.84:0/3744465107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa430106cb0 0x7fa4301070d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:51.032 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.203+0000 7fa434c56700 1 -- 172.21.15.84:0/3744465107 >> 172.21.15.84:0/3744465107 conn(0x7fa430102250 msgr2=0x7fa430104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:51.032 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.203+0000 7fa434c56700 1 -- 172.21.15.84:0/3744465107 shutdown_connections 2024-05-22T22:52:51.032 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.203+0000 7fa434c56700 1 -- 172.21.15.84:0/3744465107 wait complete. 2024-05-22T22:52:51.032 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.204+0000 7fa434c56700 1 Processor -- start 2024-05-22T22:52:51.032 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.204+0000 7fa434c56700 1 -- start start 2024-05-22T22:52:51.032 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.204+0000 7fa434c56700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa430106cb0 0x7fa430199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:51.032 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.204+0000 7fa434c56700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa43007afe0 con 0x7fa430106cb0 2024-05-22T22:52:51.032 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.204+0000 7fa42e59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa430106cb0 0x7fa430199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:51.032 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.204+0000 7fa42e59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa430106cb0 0x7fa430199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36936/0 (socket says 172.21.15.84:36936) 2024-05-22T22:52:51.033 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.204+0000 7fa42e59c700 1 -- 172.21.15.84:0/1144843757 learned_addr learned my addr 172.21.15.84:0/1144843757 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:51.033 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.205+0000 7fa42e59c700 1 -- 172.21.15.84:0/1144843757 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43019a0a0 con 0x7fa430106cb0 2024-05-22T22:52:51.033 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.205+0000 7fa42e59c700 1 --2- 172.21.15.84:0/1144843757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa430106cb0 0x7fa430199b60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa42000b340 tx=0x7fa420003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:51.033 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.205+0000 7fa41f7fe700 1 -- 172.21.15.84:0/1144843757 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa420027070 con 0x7fa430106cb0 2024-05-22T22:52:51.033 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.205+0000 7fa434c56700 1 -- 172.21.15.84:0/1144843757 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa43019a330 con 0x7fa430106cb0 2024-05-22T22:52:51.033 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.205+0000 7fa41f7fe700 1 -- 172.21.15.84:0/1144843757 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa420010c50 con 0x7fa430106cb0 2024-05-22T22:52:51.033 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.205+0000 7fa41f7fe700 1 -- 172.21.15.84:0/1144843757 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa420022be0 con 0x7fa430106cb0 2024-05-22T22:52:51.033 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.205+0000 7fa434c56700 1 -- 172.21.15.84:0/1144843757 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa43019d020 con 0x7fa430106cb0 2024-05-22T22:52:51.033 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.206+0000 7fa41f7fe700 1 -- 172.21.15.84:0/1144843757 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fa42002a070 con 0x7fa430106cb0 2024-05-22T22:52:51.033 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.206+0000 7fa41f7fe700 1 --2- 172.21.15.84:0/1144843757 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa418039050 0x7fa41803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:51.033 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.206+0000 7fa434c56700 1 -- 172.21.15.84:0/1144843757 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4300639c0 con 0x7fa430106cb0 2024-05-22T22:52:51.033 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.206+0000 7fa41f7fe700 1 -- 172.21.15.84:0/1144843757 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa42001f070 con 0x7fa430106cb0 2024-05-22T22:52:51.033 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.207+0000 7fa42dd9b700 1 --2- 172.21.15.84:0/1144843757 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa418039050 0x7fa41803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:51.034 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.207+0000 7fa42dd9b700 1 --2- 172.21.15.84:0/1144843757 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa418039050 0x7fa41803b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa424006fd0 tx=0x7fa424006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:51.034 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.212+0000 7fa41f7fe700 1 -- 172.21.15.84:0/1144843757 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa420015070 con 0x7fa430106cb0 2024-05-22T22:52:51.034 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.419+0000 7fa434c56700 1 -- 172.21.15.84:0/1144843757 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fa43002de00 con 0x7fa430106cb0 2024-05-22T22:52:51.034 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.419+0000 7fa41f7fe700 1 -- 172.21.15.84:0/1144843757 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fa420017300 con 0x7fa430106cb0 2024-05-22T22:52:51.034 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.421+0000 7fa434c56700 1 -- 172.21.15.84:0/1144843757 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa418039050 msgr2=0x7fa41803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:51.034 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.421+0000 7fa434c56700 1 --2- 172.21.15.84:0/1144843757 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa418039050 0x7fa41803b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa424006fd0 tx=0x7fa424006e40 comp rx=0 tx=0).stop 2024-05-22T22:52:51.034 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.421+0000 7fa434c56700 1 -- 172.21.15.84:0/1144843757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa430106cb0 msgr2=0x7fa430199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:51.034 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.421+0000 7fa434c56700 1 --2- 172.21.15.84:0/1144843757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa430106cb0 0x7fa430199b60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa42000b340 tx=0x7fa420003e80 comp rx=0 tx=0).stop 2024-05-22T22:52:51.034 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.421+0000 7fa434c56700 1 -- 172.21.15.84:0/1144843757 shutdown_connections 2024-05-22T22:52:51.034 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.421+0000 7fa434c56700 1 --2- 172.21.15.84:0/1144843757 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa418039050 0x7fa41803b510 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:51.034 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.421+0000 7fa434c56700 1 --2- 172.21.15.84:0/1144843757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa430106cb0 0x7fa430199b60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:51.034 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.421+0000 7fa434c56700 1 -- 172.21.15.84:0/1144843757 >> 172.21.15.84:0/1144843757 conn(0x7fa430102250 msgr2=0x7fa430190680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:51.035 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.421+0000 7fa434c56700 1 -- 172.21.15.84:0/1144843757 shutdown_connections 2024-05-22T22:52:51.035 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:50.421+0000 7fa434c56700 1 -- 172.21.15.84:0/1144843757 wait complete. 2024-05-22T22:52:51.038 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-22T22:52:51.038 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout disabled 2024-05-22T22:52:51.046 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-22T22:52:51.046 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout inactive 2024-05-22T22:52:51.046 INFO:teuthology.orchestra.run.smithi084.stdout:firewalld.service is not enabled 2024-05-22T22:52:51.046 INFO:teuthology.orchestra.run.smithi084.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-22T22:52:51.048 INFO:teuthology.orchestra.run.smithi084.stdout:Ceph Dashboard is now available at: 2024-05-22T22:52:51.048 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:52:51.048 INFO:teuthology.orchestra.run.smithi084.stdout: URL: https://smithi084.front.sepia.ceph.com:8443/ 2024-05-22T22:52:51.048 INFO:teuthology.orchestra.run.smithi084.stdout: User: admin 2024-05-22T22:52:51.048 INFO:teuthology.orchestra.run.smithi084.stdout: Password: sbxxgoikbr 2024-05-22T22:52:51.048 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:52:51.048 INFO:teuthology.orchestra.run.smithi084.stdout:Saving cluster configuration to /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config directory 2024-05-22T22:52:51.049 INFO:teuthology.orchestra.run.smithi084.stdout:Enabling autotune for osd_memory_target 2024-05-22T22:52:52.473 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.472+0000 7f3158ba7700 1 Processor -- start 2024-05-22T22:52:52.473 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.472+0000 7f3158ba7700 1 -- start start 2024-05-22T22:52:52.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.472+0000 7f3158ba7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3154108850 0x7f315410ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:52.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.472+0000 7f3158ba7700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f315407be80 con 0x7f3154108850 2024-05-22T22:52:52.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.472+0000 7f315259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3154108850 0x7f315410ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:52.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.472+0000 7f315259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3154108850 0x7f315410ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40014/0 (socket says 172.21.15.84:40014) 2024-05-22T22:52:52.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.472+0000 7f315259c700 1 -- 172.21.15.84:0/3453479110 learned_addr learned my addr 172.21.15.84:0/3453479110 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:52.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.472+0000 7f315259c700 1 -- 172.21.15.84:0/3453479110 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f315410b180 con 0x7f3154108850 2024-05-22T22:52:52.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.472+0000 7f315259c700 1 --2- 172.21.15.84:0/3453479110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3154108850 0x7f315410ac40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f31440098a0 tx=0x7f3144009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd4bda7728c4a9d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:52.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.473+0000 7f315159a700 1 -- 172.21.15.84:0/3453479110 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3144004160 con 0x7f3154108850 2024-05-22T22:52:52.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.473+0000 7f315159a700 1 -- 172.21.15.84:0/3453479110 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3144004300 con 0x7f3154108850 2024-05-22T22:52:52.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.473+0000 7f315159a700 1 -- 172.21.15.84:0/3453479110 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3144011b20 con 0x7f3154108850 2024-05-22T22:52:52.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.473+0000 7f3158ba7700 1 -- 172.21.15.84:0/3453479110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3154108850 msgr2=0x7f315410ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:52.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.473+0000 7f3158ba7700 1 --2- 172.21.15.84:0/3453479110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3154108850 0x7f315410ac40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f31440098a0 tx=0x7f3144009bb0 comp rx=0 tx=0).stop 2024-05-22T22:52:52.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.473+0000 7f3158ba7700 1 -- 172.21.15.84:0/3453479110 shutdown_connections 2024-05-22T22:52:52.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.473+0000 7f3158ba7700 1 --2- 172.21.15.84:0/3453479110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3154108850 0x7f315410ac40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:52.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.473+0000 7f3158ba7700 1 -- 172.21.15.84:0/3453479110 >> 172.21.15.84:0/3453479110 conn(0x7f3154102200 msgr2=0x7f3154104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:52.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.473+0000 7f3158ba7700 1 -- 172.21.15.84:0/3453479110 shutdown_connections 2024-05-22T22:52:52.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.473+0000 7f3158ba7700 1 -- 172.21.15.84:0/3453479110 wait complete. 2024-05-22T22:52:52.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.474+0000 7f3158ba7700 1 Processor -- start 2024-05-22T22:52:52.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.474+0000 7f3158ba7700 1 -- start start 2024-05-22T22:52:52.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.474+0000 7f3158ba7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3154108850 0x7f315419bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:52.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.474+0000 7f3158ba7700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f315410c1e0 con 0x7f3154108850 2024-05-22T22:52:52.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.474+0000 7f315259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3154108850 0x7f315419bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:52.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.474+0000 7f315259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3154108850 0x7f315419bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40026/0 (socket says 172.21.15.84:40026) 2024-05-22T22:52:52.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.474+0000 7f315259c700 1 -- 172.21.15.84:0/1359784423 learned_addr learned my addr 172.21.15.84:0/1359784423 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:52.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.475+0000 7f315259c700 1 -- 172.21.15.84:0/1359784423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f315419c1f0 con 0x7f3154108850 2024-05-22T22:52:52.476 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.475+0000 7f315259c700 1 --2- 172.21.15.84:0/1359784423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3154108850 0x7f315419bcb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f31440117e0 tx=0x7f3144011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:52.476 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.475+0000 7f31437fe700 1 -- 172.21.15.84:0/1359784423 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3144003bf0 con 0x7f3154108850 2024-05-22T22:52:52.476 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.475+0000 7f3158ba7700 1 -- 172.21.15.84:0/1359784423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f315419c480 con 0x7f3154108850 2024-05-22T22:52:52.476 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.475+0000 7f31437fe700 1 -- 172.21.15.84:0/1359784423 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f314401a910 con 0x7f3154108850 2024-05-22T22:52:52.476 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.475+0000 7f31437fe700 1 -- 172.21.15.84:0/1359784423 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3144024970 con 0x7f3154108850 2024-05-22T22:52:52.476 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.475+0000 7f3158ba7700 1 -- 172.21.15.84:0/1359784423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f315419c8e0 con 0x7f3154108850 2024-05-22T22:52:52.476 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.476+0000 7f31437fe700 1 -- 172.21.15.84:0/1359784423 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f3144022020 con 0x7f3154108850 2024-05-22T22:52:52.476 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.476+0000 7f31437fe700 1 --2- 172.21.15.84:0/1359784423 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f313c0390f0 0x7f313c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:52.476 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.476+0000 7f3158ba7700 1 -- 172.21.15.84:0/1359784423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31540639c0 con 0x7f3154108850 2024-05-22T22:52:52.476 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.476+0000 7f31437fe700 1 -- 172.21.15.84:0/1359784423 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f314404d810 con 0x7f3154108850 2024-05-22T22:52:52.476 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.476+0000 7f3151d9b700 1 --2- 172.21.15.84:0/1359784423 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f313c0390f0 0x7f313c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:52.476 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.477+0000 7f3151d9b700 1 --2- 172.21.15.84:0/1359784423 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f313c0390f0 0x7f313c03b5b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3148006fd0 tx=0x7f3148006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:52.476 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.481+0000 7f31437fe700 1 -- 172.21.15.84:0/1359784423 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3144011e20 con 0x7f3154108850 2024-05-22T22:52:52.476 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.683+0000 7f3158ba7700 1 -- 172.21.15.84:0/1359784423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f3154106640 con 0x7f3154108850 2024-05-22T22:52:52.477 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.684+0000 7f31437fe700 1 -- 172.21.15.84:0/1359784423 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f314400ba30 con 0x7f3154108850 2024-05-22T22:52:52.477 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.686+0000 7f3158ba7700 1 -- 172.21.15.84:0/1359784423 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f313c0390f0 msgr2=0x7f313c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:52.477 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.686+0000 7f3158ba7700 1 --2- 172.21.15.84:0/1359784423 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f313c0390f0 0x7f313c03b5b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3148006fd0 tx=0x7f3148006e40 comp rx=0 tx=0).stop 2024-05-22T22:52:52.477 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.686+0000 7f3158ba7700 1 -- 172.21.15.84:0/1359784423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3154108850 msgr2=0x7f315419bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:52.477 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.686+0000 7f3158ba7700 1 --2- 172.21.15.84:0/1359784423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3154108850 0x7f315419bcb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f31440117e0 tx=0x7f3144011c10 comp rx=0 tx=0).stop 2024-05-22T22:52:52.477 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.686+0000 7f3158ba7700 1 -- 172.21.15.84:0/1359784423 shutdown_connections 2024-05-22T22:52:52.477 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.686+0000 7f3158ba7700 1 --2- 172.21.15.84:0/1359784423 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f313c0390f0 0x7f313c03b5b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:52.477 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.686+0000 7f3158ba7700 1 --2- 172.21.15.84:0/1359784423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3154108850 0x7f315419bcb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:52.477 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.686+0000 7f3158ba7700 1 -- 172.21.15.84:0/1359784423 >> 172.21.15.84:0/1359784423 conn(0x7f3154102200 msgr2=0x7f3154102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:52.477 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.686+0000 7f3158ba7700 1 -- 172.21.15.84:0/1359784423 shutdown_connections 2024-05-22T22:52:52.477 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:51.686+0000 7f3158ba7700 1 -- 172.21.15.84:0/1359784423 wait complete. 2024-05-22T22:52:52.825 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:52 smithi084 ceph-mon[32186]: mgrmap e13: x(active, since 6s) 2024-05-22T22:52:54.247 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.352+0000 7fe4c2f5f700 1 Processor -- start 2024-05-22T22:52:54.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.352+0000 7fe4c2f5f700 1 -- start start 2024-05-22T22:52:54.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.352+0000 7fe4c2f5f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4bc108890 0x7fe4bc10ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:54.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.352+0000 7fe4c2f5f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe4bc07be80 con 0x7fe4bc108890 2024-05-22T22:52:54.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.352+0000 7fe4c0cfb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4bc108890 0x7fe4bc10ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:54.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.353+0000 7fe4c0cfb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4bc108890 0x7fe4bc10ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40030/0 (socket says 172.21.15.84:40030) 2024-05-22T22:52:54.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.353+0000 7fe4c0cfb700 1 -- 172.21.15.84:0/1291820094 learned_addr learned my addr 172.21.15.84:0/1291820094 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:54.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.353+0000 7fe4c0cfb700 1 -- 172.21.15.84:0/1291820094 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4bc10b1c0 con 0x7fe4bc108890 2024-05-22T22:52:54.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.353+0000 7fe4c0cfb700 1 --2- 172.21.15.84:0/1291820094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4bc108890 0x7fe4bc10ac80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00098a0 tx=0x7fe4b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5139c0ee0899044 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:54.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.353+0000 7fe4bb7fe700 1 -- 172.21.15.84:0/1291820094 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4b0004160 con 0x7fe4bc108890 2024-05-22T22:52:54.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.353+0000 7fe4bb7fe700 1 -- 172.21.15.84:0/1291820094 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b0004300 con 0x7fe4bc108890 2024-05-22T22:52:54.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.353+0000 7fe4bb7fe700 1 -- 172.21.15.84:0/1291820094 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4b0011b20 con 0x7fe4bc108890 2024-05-22T22:52:54.248 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.354+0000 7fe4c2f5f700 1 -- 172.21.15.84:0/1291820094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4bc108890 msgr2=0x7fe4bc10ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:54.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.354+0000 7fe4c2f5f700 1 --2- 172.21.15.84:0/1291820094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4bc108890 0x7fe4bc10ac80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00098a0 tx=0x7fe4b0009bb0 comp rx=0 tx=0).stop 2024-05-22T22:52:54.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.354+0000 7fe4c2f5f700 1 -- 172.21.15.84:0/1291820094 shutdown_connections 2024-05-22T22:52:54.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.354+0000 7fe4c2f5f700 1 --2- 172.21.15.84:0/1291820094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4bc108890 0x7fe4bc10ac80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:54.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.354+0000 7fe4c2f5f700 1 -- 172.21.15.84:0/1291820094 >> 172.21.15.84:0/1291820094 conn(0x7fe4bc102260 msgr2=0x7fe4bc1046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:54.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.354+0000 7fe4c2f5f700 1 -- 172.21.15.84:0/1291820094 shutdown_connections 2024-05-22T22:52:54.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.354+0000 7fe4c2f5f700 1 -- 172.21.15.84:0/1291820094 wait complete. 2024-05-22T22:52:54.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.354+0000 7fe4c2f5f700 1 Processor -- start 2024-05-22T22:52:54.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.354+0000 7fe4c2f5f700 1 -- start start 2024-05-22T22:52:54.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.355+0000 7fe4c2f5f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4bc108890 0x7fe4bc108110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:54.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.355+0000 7fe4c2f5f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe4bc10c220 con 0x7fe4bc108890 2024-05-22T22:52:54.249 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.355+0000 7fe4c0cfb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4bc108890 0x7fe4bc108110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:54.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.355+0000 7fe4c0cfb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4bc108890 0x7fe4bc108110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40042/0 (socket says 172.21.15.84:40042) 2024-05-22T22:52:54.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.355+0000 7fe4c0cfb700 1 -- 172.21.15.84:0/3937792387 learned_addr learned my addr 172.21.15.84:0/3937792387 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:54.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.355+0000 7fe4c0cfb700 1 -- 172.21.15.84:0/3937792387 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4bc108650 con 0x7fe4bc108890 2024-05-22T22:52:54.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.355+0000 7fe4c0cfb700 1 --2- 172.21.15.84:0/3937792387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4bc108890 0x7fe4bc108110 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00117e0 tx=0x7fe4b0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:54.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.355+0000 7fe4b9ffb700 1 -- 172.21.15.84:0/3937792387 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4b0003bf0 con 0x7fe4bc108890 2024-05-22T22:52:54.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.355+0000 7fe4b9ffb700 1 -- 172.21.15.84:0/3937792387 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b001a910 con 0x7fe4bc108890 2024-05-22T22:52:54.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.355+0000 7fe4c2f5f700 1 -- 172.21.15.84:0/3937792387 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4bc106840 con 0x7fe4bc108890 2024-05-22T22:52:54.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.355+0000 7fe4b9ffb700 1 -- 172.21.15.84:0/3937792387 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4b0024970 con 0x7fe4bc108890 2024-05-22T22:52:54.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.355+0000 7fe4c2f5f700 1 -- 172.21.15.84:0/3937792387 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4bc106d00 con 0x7fe4bc108890 2024-05-22T22:52:54.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.356+0000 7fe4b9ffb700 1 -- 172.21.15.84:0/3937792387 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe4b0022020 con 0x7fe4bc108890 2024-05-22T22:52:54.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.356+0000 7fe4c2f5f700 1 -- 172.21.15.84:0/3937792387 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4bc0639c0 con 0x7fe4bc108890 2024-05-22T22:52:54.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.357+0000 7fe4b9ffb700 1 --2- 172.21.15.84:0/3937792387 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe4ac0390f0 0x7fe4ac03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:54.250 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.357+0000 7fe4b9ffb700 1 -- 172.21.15.84:0/3937792387 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe4b004d730 con 0x7fe4bc108890 2024-05-22T22:52:54.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.357+0000 7fe4bbfff700 1 --2- 172.21.15.84:0/3937792387 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe4ac0390f0 0x7fe4ac03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:54.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.357+0000 7fe4bbfff700 1 --2- 172.21.15.84:0/3937792387 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe4ac0390f0 0x7fe4ac03b5b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8006fd0 tx=0x7fe4a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:54.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.361+0000 7fe4b9ffb700 1 -- 172.21.15.84:0/3937792387 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe4b0011e20 con 0x7fe4bc108890 2024-05-22T22:52:54.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.668+0000 7fe4c2f5f700 1 -- 172.21.15.84:0/3937792387 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fe4bc106f80 con 0x7fe4bc108890 2024-05-22T22:52:54.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.671+0000 7fe4b9ffb700 1 -- 172.21.15.84:0/3937792387 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7fe4b000ba30 con 0x7fe4bc108890 2024-05-22T22:52:54.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.673+0000 7fe4c2f5f700 1 -- 172.21.15.84:0/3937792387 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe4ac0390f0 msgr2=0x7fe4ac03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:54.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.673+0000 7fe4c2f5f700 1 --2- 172.21.15.84:0/3937792387 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe4ac0390f0 0x7fe4ac03b5b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8006fd0 tx=0x7fe4a8006e40 comp rx=0 tx=0).stop 2024-05-22T22:52:54.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.673+0000 7fe4c2f5f700 1 -- 172.21.15.84:0/3937792387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4bc108890 msgr2=0x7fe4bc108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:54.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.673+0000 7fe4c2f5f700 1 --2- 172.21.15.84:0/3937792387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4bc108890 0x7fe4bc108110 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00117e0 tx=0x7fe4b0011c10 comp rx=0 tx=0).stop 2024-05-22T22:52:54.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.673+0000 7fe4c2f5f700 1 -- 172.21.15.84:0/3937792387 shutdown_connections 2024-05-22T22:52:54.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.673+0000 7fe4c2f5f700 1 --2- 172.21.15.84:0/3937792387 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe4ac0390f0 0x7fe4ac03b5b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:54.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.673+0000 7fe4c2f5f700 1 --2- 172.21.15.84:0/3937792387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4bc108890 0x7fe4bc108110 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:54.251 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.673+0000 7fe4c2f5f700 1 -- 172.21.15.84:0/3937792387 >> 172.21.15.84:0/3937792387 conn(0x7fe4bc102260 msgr2=0x7fe4bc102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:54.252 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.673+0000 7fe4c2f5f700 1 -- 172.21.15.84:0/3937792387 shutdown_connections 2024-05-22T22:52:54.252 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-05-22T22:52:53.673+0000 7fe4c2f5f700 1 -- 172.21.15.84:0/3937792387 wait complete. 2024-05-22T22:52:54.252 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-22T22:52:54.252 INFO:teuthology.orchestra.run.smithi084.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-22T22:52:54.252 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:52:54.252 INFO:teuthology.orchestra.run.smithi084.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-22T22:52:54.252 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:52:54.252 INFO:teuthology.orchestra.run.smithi084.stdout:Or, if you are only running a single cluster on this host: 2024-05-22T22:52:54.252 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:52:54.252 INFO:teuthology.orchestra.run.smithi084.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-22T22:52:54.252 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:52:54.252 INFO:teuthology.orchestra.run.smithi084.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-22T22:52:54.252 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:52:54.252 INFO:teuthology.orchestra.run.smithi084.stdout: ceph telemetry on 2024-05-22T22:52:54.253 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:52:54.253 INFO:teuthology.orchestra.run.smithi084.stdout:For more information see: 2024-05-22T22:52:54.253 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:52:54.253 INFO:teuthology.orchestra.run.smithi084.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-22T22:52:54.253 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:52:54.253 INFO:teuthology.orchestra.run.smithi084.stdout:Bootstrap complete. 2024-05-22T22:52:54.289 INFO:tasks.cephadm:Fetching config... 2024-05-22T22:52:54.290 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-22T22:52:54.290 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-22T22:52:54.306 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-22T22:52:54.306 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-22T22:52:54.306 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-22T22:52:54.362 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-22T22:52:54.362 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-22T22:52:54.362 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-22T22:52:54.427 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-22T22:52:54.427 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-22T22:52:54.428 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-22T22:52:54.484 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-22T22:52:54.484 DEBUG:teuthology.orchestra.run.smithi084:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyMcocFllQBQmqrgABP1nMuSuzyzqWTLEKmlJZRkJzY1P9PMklATrhQuTd1vhVTjh5HTbLsaX/wKsv269gmsjmykHyiMWTXWmXlUJTqoYWTD/hX1G+W3Ejkja76VkuJzPnU0P98tXAPBgmG3z15LmVkPi/aS0Qxe6fygADVs5NvYCJdrIk5foFOeiHL5+XVuTEg3dbdHlxtoqUfafdtXar5P945X8RIl84/ljEQqwyy1vFvwQLoNcBiNZPRY05+c5GmBFBhedJOv7QpnfgutgEE0hQZMHHNRNXMDNWFbCoHd2fsCBF6s0V7IvEdolun9c3oq0sBI7tEqVcn3CLNNpkrvh/n68Kz2DXRF85qYTwi+C1F2OV0gGEJA6GknlcyD8HxpgluJfEeQUB1tmScOxhMMr2mtdlTyeAnM6UbCYsU0K9YMtV+UQL3MZ5Kl0M7c7xRcYCHoOBYRRTMOMjpgiHZzaclaoVPuS8lQ4yj4fd5IVpcDryCbNPCHRgc/l9b0E= ceph-c1135a12-188d-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-22T22:52:54.558 INFO:teuthology.orchestra.run.smithi084.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyMcocFllQBQmqrgABP1nMuSuzyzqWTLEKmlJZRkJzY1P9PMklATrhQuTd1vhVTjh5HTbLsaX/wKsv269gmsjmykHyiMWTXWmXlUJTqoYWTD/hX1G+W3Ejkja76VkuJzPnU0P98tXAPBgmG3z15LmVkPi/aS0Qxe6fygADVs5NvYCJdrIk5foFOeiHL5+XVuTEg3dbdHlxtoqUfafdtXar5P945X8RIl84/ljEQqwyy1vFvwQLoNcBiNZPRY05+c5GmBFBhedJOv7QpnfgutgEE0hQZMHHNRNXMDNWFbCoHd2fsCBF6s0V7IvEdolun9c3oq0sBI7tEqVcn3CLNNpkrvh/n68Kz2DXRF85qYTwi+C1F2OV0gGEJA6GknlcyD8HxpgluJfEeQUB1tmScOxhMMr2mtdlTyeAnM6UbCYsU0K9YMtV+UQL3MZ5Kl0M7c7xRcYCHoOBYRRTMOMjpgiHZzaclaoVPuS8lQ4yj4fd5IVpcDryCbNPCHRgc/l9b0E= ceph-c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:52:54.569 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyMcocFllQBQmqrgABP1nMuSuzyzqWTLEKmlJZRkJzY1P9PMklATrhQuTd1vhVTjh5HTbLsaX/wKsv269gmsjmykHyiMWTXWmXlUJTqoYWTD/hX1G+W3Ejkja76VkuJzPnU0P98tXAPBgmG3z15LmVkPi/aS0Qxe6fygADVs5NvYCJdrIk5foFOeiHL5+XVuTEg3dbdHlxtoqUfafdtXar5P945X8RIl84/ljEQqwyy1vFvwQLoNcBiNZPRY05+c5GmBFBhedJOv7QpnfgutgEE0hQZMHHNRNXMDNWFbCoHd2fsCBF6s0V7IvEdolun9c3oq0sBI7tEqVcn3CLNNpkrvh/n68Kz2DXRF85qYTwi+C1F2OV0gGEJA6GknlcyD8HxpgluJfEeQUB1tmScOxhMMr2mtdlTyeAnM6UbCYsU0K9YMtV+UQL3MZ5Kl0M7c7xRcYCHoOBYRRTMOMjpgiHZzaclaoVPuS8lQ4yj4fd5IVpcDryCbNPCHRgc/l9b0E= ceph-c1135a12-188d-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-22T22:52:54.608 INFO:teuthology.orchestra.run.smithi096.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyMcocFllQBQmqrgABP1nMuSuzyzqWTLEKmlJZRkJzY1P9PMklATrhQuTd1vhVTjh5HTbLsaX/wKsv269gmsjmykHyiMWTXWmXlUJTqoYWTD/hX1G+W3Ejkja76VkuJzPnU0P98tXAPBgmG3z15LmVkPi/aS0Qxe6fygADVs5NvYCJdrIk5foFOeiHL5+XVuTEg3dbdHlxtoqUfafdtXar5P945X8RIl84/ljEQqwyy1vFvwQLoNcBiNZPRY05+c5GmBFBhedJOv7QpnfgutgEE0hQZMHHNRNXMDNWFbCoHd2fsCBF6s0V7IvEdolun9c3oq0sBI7tEqVcn3CLNNpkrvh/n68Kz2DXRF85qYTwi+C1F2OV0gGEJA6GknlcyD8HxpgluJfEeQUB1tmScOxhMMr2mtdlTyeAnM6UbCYsU0K9YMtV+UQL3MZ5Kl0M7c7xRcYCHoOBYRRTMOMjpgiHZzaclaoVPuS8lQ4yj4fd5IVpcDryCbNPCHRgc/l9b0E= ceph-c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:52:54.620 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyMcocFllQBQmqrgABP1nMuSuzyzqWTLEKmlJZRkJzY1P9PMklATrhQuTd1vhVTjh5HTbLsaX/wKsv269gmsjmykHyiMWTXWmXlUJTqoYWTD/hX1G+W3Ejkja76VkuJzPnU0P98tXAPBgmG3z15LmVkPi/aS0Qxe6fygADVs5NvYCJdrIk5foFOeiHL5+XVuTEg3dbdHlxtoqUfafdtXar5P945X8RIl84/ljEQqwyy1vFvwQLoNcBiNZPRY05+c5GmBFBhedJOv7QpnfgutgEE0hQZMHHNRNXMDNWFbCoHd2fsCBF6s0V7IvEdolun9c3oq0sBI7tEqVcn3CLNNpkrvh/n68Kz2DXRF85qYTwi+C1F2OV0gGEJA6GknlcyD8HxpgluJfEeQUB1tmScOxhMMr2mtdlTyeAnM6UbCYsU0K9YMtV+UQL3MZ5Kl0M7c7xRcYCHoOBYRRTMOMjpgiHZzaclaoVPuS8lQ4yj4fd5IVpcDryCbNPCHRgc/l9b0E= ceph-c1135a12-188d-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-22T22:52:54.661 INFO:teuthology.orchestra.run.smithi204.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyMcocFllQBQmqrgABP1nMuSuzyzqWTLEKmlJZRkJzY1P9PMklATrhQuTd1vhVTjh5HTbLsaX/wKsv269gmsjmykHyiMWTXWmXlUJTqoYWTD/hX1G+W3Ejkja76VkuJzPnU0P98tXAPBgmG3z15LmVkPi/aS0Qxe6fygADVs5NvYCJdrIk5foFOeiHL5+XVuTEg3dbdHlxtoqUfafdtXar5P945X8RIl84/ljEQqwyy1vFvwQLoNcBiNZPRY05+c5GmBFBhedJOv7QpnfgutgEE0hQZMHHNRNXMDNWFbCoHd2fsCBF6s0V7IvEdolun9c3oq0sBI7tEqVcn3CLNNpkrvh/n68Kz2DXRF85qYTwi+C1F2OV0gGEJA6GknlcyD8HxpgluJfEeQUB1tmScOxhMMr2mtdlTyeAnM6UbCYsU0K9YMtV+UQL3MZ5Kl0M7c7xRcYCHoOBYRRTMOMjpgiHZzaclaoVPuS8lQ4yj4fd5IVpcDryCbNPCHRgc/l9b0E= ceph-c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:52:54.671 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-22T22:52:54.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:54 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3937792387' entity='client.admin' 2024-05-22T22:52:54.910 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T22:52:57.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.138+0000 7f6ead910700 1 -- 172.21.15.84:0/3509161767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ea8100cc0 msgr2=0x7f6ea81010e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:57.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.138+0000 7f6ead910700 1 --2- 172.21.15.84:0/3509161767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ea8100cc0 0x7f6ea81010e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6e98009910 tx=0x7f6e98009c20 comp rx=0 tx=0).stop 2024-05-22T22:52:57.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.138+0000 7f6ead910700 1 -- 172.21.15.84:0/3509161767 shutdown_connections 2024-05-22T22:52:57.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.138+0000 7f6ead910700 1 --2- 172.21.15.84:0/3509161767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ea8100cc0 0x7f6ea81010e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:57.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.138+0000 7f6ead910700 1 -- 172.21.15.84:0/3509161767 >> 172.21.15.84:0/3509161767 conn(0x7f6ea80fc260 msgr2=0x7f6ea80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:57.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.138+0000 7f6ead910700 1 -- 172.21.15.84:0/3509161767 shutdown_connections 2024-05-22T22:52:57.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.138+0000 7f6ead910700 1 -- 172.21.15.84:0/3509161767 wait complete. 2024-05-22T22:52:57.132 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.139+0000 7f6ead910700 1 Processor -- start 2024-05-22T22:52:57.132 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.139+0000 7f6ead910700 1 -- start start 2024-05-22T22:52:57.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.139+0000 7f6ead910700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ea8100cc0 0x7f6ea806a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:57.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.139+0000 7f6ead910700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea8106d50 con 0x7f6ea8100cc0 2024-05-22T22:52:57.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.139+0000 7f6ea6ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ea8100cc0 0x7f6ea806a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:57.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.139+0000 7f6ea6ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ea8100cc0 0x7f6ea806a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40070/0 (socket says 172.21.15.84:40070) 2024-05-22T22:52:57.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.139+0000 7f6ea6ffd700 1 -- 172.21.15.84:0/1167553671 learned_addr learned my addr 172.21.15.84:0/1167553671 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:57.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.139+0000 7f6ea6ffd700 1 -- 172.21.15.84:0/1167553671 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea819a1b0 con 0x7f6ea8100cc0 2024-05-22T22:52:57.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.140+0000 7f6ea6ffd700 1 --2- 172.21.15.84:0/1167553671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ea8100cc0 0x7f6ea806a260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6e98006010 tx=0x7f6e98005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:57.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.140+0000 7f6eac90e700 1 -- 172.21.15.84:0/1167553671 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e98027070 con 0x7f6ea8100cc0 2024-05-22T22:52:57.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.140+0000 7f6ead910700 1 -- 172.21.15.84:0/1167553671 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea819a380 con 0x7f6ea8100cc0 2024-05-22T22:52:57.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.140+0000 7f6eac90e700 1 -- 172.21.15.84:0/1167553671 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e9800be40 con 0x7f6ea8100cc0 2024-05-22T22:52:57.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.140+0000 7f6eac90e700 1 -- 172.21.15.84:0/1167553671 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e9800f460 con 0x7f6ea8100cc0 2024-05-22T22:52:57.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.140+0000 7f6ead910700 1 -- 172.21.15.84:0/1167553671 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea819a7a0 con 0x7f6ea8100cc0 2024-05-22T22:52:57.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.141+0000 7f6eac90e700 1 -- 172.21.15.84:0/1167553671 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6e9802a070 con 0x7f6ea8100cc0 2024-05-22T22:52:57.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.141+0000 7f6ead910700 1 -- 172.21.15.84:0/1167553671 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ea81074d0 con 0x7f6ea8100cc0 2024-05-22T22:52:57.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.141+0000 7f6eac90e700 1 --2- 172.21.15.84:0/1167553671 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f6e940390f0 0x7f6e9403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:57.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.141+0000 7f6eac90e700 1 -- 172.21.15.84:0/1167553671 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6e9801e030 con 0x7f6ea8100cc0 2024-05-22T22:52:57.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.141+0000 7f6e9ffff700 1 --2- 172.21.15.84:0/1167553671 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f6e940390f0 0x7f6e9403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:57.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.141+0000 7f6e9ffff700 1 --2- 172.21.15.84:0/1167553671 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f6e940390f0 0x7f6e9403b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6e90006fd0 tx=0x7f6e90006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:57.137 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.146+0000 7f6eac90e700 1 -- 172.21.15.84:0/1167553671 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6e98013070 con 0x7f6ea8100cc0 2024-05-22T22:52:57.336 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.344+0000 7f6ead910700 1 -- 172.21.15.84:0/1167553671 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f6ea802dd80 con 0x7f6ea8100cc0 2024-05-22T22:52:57.339 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.348+0000 7f6eac90e700 1 -- 172.21.15.84:0/1167553671 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f6e980310c0 con 0x7f6ea8100cc0 2024-05-22T22:52:57.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.350+0000 7f6ead910700 1 -- 172.21.15.84:0/1167553671 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f6e940390f0 msgr2=0x7f6e9403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:57.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.350+0000 7f6ead910700 1 --2- 172.21.15.84:0/1167553671 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f6e940390f0 0x7f6e9403b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6e90006fd0 tx=0x7f6e90006e40 comp rx=0 tx=0).stop 2024-05-22T22:52:57.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.350+0000 7f6ead910700 1 -- 172.21.15.84:0/1167553671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ea8100cc0 msgr2=0x7f6ea806a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:57.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.350+0000 7f6ead910700 1 --2- 172.21.15.84:0/1167553671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ea8100cc0 0x7f6ea806a260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6e98006010 tx=0x7f6e98005050 comp rx=0 tx=0).stop 2024-05-22T22:52:57.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.350+0000 7f6ead910700 1 -- 172.21.15.84:0/1167553671 shutdown_connections 2024-05-22T22:52:57.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.350+0000 7f6ead910700 1 --2- 172.21.15.84:0/1167553671 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f6e940390f0 0x7f6e9403b5b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:57.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.350+0000 7f6ead910700 1 --2- 172.21.15.84:0/1167553671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ea8100cc0 0x7f6ea806a260 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:57.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.350+0000 7f6ead910700 1 -- 172.21.15.84:0/1167553671 >> 172.21.15.84:0/1167553671 conn(0x7f6ea80fc260 msgr2=0x7f6ea80fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:57.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.350+0000 7f6ead910700 1 -- 172.21.15.84:0/1167553671 shutdown_connections 2024-05-22T22:52:57.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:57.350+0000 7f6ead910700 1 -- 172.21.15.84:0/1167553671 wait complete. 2024-05-22T22:52:57.895 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-22T22:52:57.896 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-22T22:52:58.116 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:52:58.116 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:52:58.116 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi084", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:52:58.116 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:52:58.116 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:52:58.116 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:52:58.116 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:57 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1167553671' entity='client.admin' 2024-05-22T22:52:58.116 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:52:58.117 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:52:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:52:58.129 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T22:52:59.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.603+0000 7fb7fc506700 1 -- 172.21.15.84:0/3714281187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7f4100cc0 msgr2=0x7fb7f41010e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.603+0000 7fb7fc506700 1 --2- 172.21.15.84:0/3714281187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7f4100cc0 0x7fb7f41010e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009910 tx=0x7fb7e4009c20 comp rx=0 tx=0).stop 2024-05-22T22:52:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.603+0000 7fb7fc506700 1 -- 172.21.15.84:0/3714281187 shutdown_connections 2024-05-22T22:52:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.603+0000 7fb7fc506700 1 --2- 172.21.15.84:0/3714281187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7f4100cc0 0x7fb7f41010e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.603+0000 7fb7fc506700 1 -- 172.21.15.84:0/3714281187 >> 172.21.15.84:0/3714281187 conn(0x7fb7f40fc260 msgr2=0x7fb7f40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.603+0000 7fb7fc506700 1 -- 172.21.15.84:0/3714281187 shutdown_connections 2024-05-22T22:52:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.603+0000 7fb7fc506700 1 -- 172.21.15.84:0/3714281187 wait complete. 2024-05-22T22:52:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.604+0000 7fb7fc506700 1 Processor -- start 2024-05-22T22:52:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.604+0000 7fb7fc506700 1 -- start start 2024-05-22T22:52:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.604+0000 7fb7fc506700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7f4100cc0 0x7fb7f4197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.604+0000 7fb7fc506700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f4106d50 con 0x7fb7f4100cc0 2024-05-22T22:52:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.604+0000 7fb7fa2a2700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7f4100cc0 0x7fb7f4197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:59.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.604+0000 7fb7fa2a2700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7f4100cc0 0x7fb7f4197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40094/0 (socket says 172.21.15.84:40094) 2024-05-22T22:52:59.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.604+0000 7fb7fa2a2700 1 -- 172.21.15.84:0/2107203051 learned_addr learned my addr 172.21.15.84:0/2107203051 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:52:59.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.604+0000 7fb7fa2a2700 1 -- 172.21.15.84:0/2107203051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f4197a40 con 0x7fb7f4100cc0 2024-05-22T22:52:59.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.604+0000 7fb7fa2a2700 1 --2- 172.21.15.84:0/2107203051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7f4100cc0 0x7fb7f4197500 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4005850 tx=0x7fb7e4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:59.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.604+0000 7fb7eb7fe700 1 -- 172.21.15.84:0/2107203051 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7e4026070 con 0x7fb7f4100cc0 2024-05-22T22:52:59.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.605+0000 7fb7eb7fe700 1 -- 172.21.15.84:0/2107203051 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7e40044a0 con 0x7fb7f4100cc0 2024-05-22T22:52:59.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.605+0000 7fb7fc506700 1 -- 172.21.15.84:0/2107203051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f4197cd0 con 0x7fb7f4100cc0 2024-05-22T22:52:59.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.605+0000 7fb7eb7fe700 1 -- 172.21.15.84:0/2107203051 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7e4003ab0 con 0x7fb7f4100cc0 2024-05-22T22:52:59.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.605+0000 7fb7fc506700 1 -- 172.21.15.84:0/2107203051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f419a9c0 con 0x7fb7f4100cc0 2024-05-22T22:52:59.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.605+0000 7fb7eb7fe700 1 -- 172.21.15.84:0/2107203051 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb7e4029070 con 0x7fb7f4100cc0 2024-05-22T22:52:59.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.605+0000 7fb7fc506700 1 -- 172.21.15.84:0/2107203051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f41074d0 con 0x7fb7f4100cc0 2024-05-22T22:52:59.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.606+0000 7fb7eb7fe700 1 --2- 172.21.15.84:0/2107203051 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb7e0039150 0x7fb7e003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:52:59.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.606+0000 7fb7eb7fe700 1 -- 172.21.15.84:0/2107203051 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb7e401e070 con 0x7fb7f4100cc0 2024-05-22T22:52:59.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.606+0000 7fb7f9aa1700 1 --2- 172.21.15.84:0/2107203051 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb7e0039150 0x7fb7e003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:52:59.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.606+0000 7fb7f9aa1700 1 --2- 172.21.15.84:0/2107203051 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb7e0039150 0x7fb7e003b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0006fd0 tx=0x7fb7f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:52:59.602 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.611+0000 7fb7eb7fe700 1 -- 172.21.15.84:0/2107203051 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb7e4013070 con 0x7fb7f4100cc0 2024-05-22T22:52:59.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.810+0000 7fb7fc506700 1 -- 172.21.15.84:0/2107203051 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fb7f402dcd0 con 0x7fb7e0039150 2024-05-22T22:52:59.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.813+0000 7fb7eb7fe700 1 -- 172.21.15.84:0/2107203051 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb7f402dcd0 con 0x7fb7e0039150 2024-05-22T22:52:59.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.816+0000 7fb7fc506700 1 -- 172.21.15.84:0/2107203051 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb7e0039150 msgr2=0x7fb7e003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:59.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.816+0000 7fb7fc506700 1 --2- 172.21.15.84:0/2107203051 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb7e0039150 0x7fb7e003b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0006fd0 tx=0x7fb7f0006e40 comp rx=0 tx=0).stop 2024-05-22T22:52:59.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.816+0000 7fb7fc506700 1 -- 172.21.15.84:0/2107203051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7f4100cc0 msgr2=0x7fb7f4197500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:52:59.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.816+0000 7fb7fc506700 1 --2- 172.21.15.84:0/2107203051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7f4100cc0 0x7fb7f4197500 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4005850 tx=0x7fb7e4005030 comp rx=0 tx=0).stop 2024-05-22T22:52:59.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.816+0000 7fb7fc506700 1 -- 172.21.15.84:0/2107203051 shutdown_connections 2024-05-22T22:52:59.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.816+0000 7fb7fc506700 1 --2- 172.21.15.84:0/2107203051 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb7e0039150 0x7fb7e003b610 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:59.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.816+0000 7fb7fc506700 1 --2- 172.21.15.84:0/2107203051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7f4100cc0 0x7fb7f4197500 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:52:59.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.816+0000 7fb7fc506700 1 -- 172.21.15.84:0/2107203051 >> 172.21.15.84:0/2107203051 conn(0x7fb7f40fc260 msgr2=0x7fb7f4069c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:52:59.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.816+0000 7fb7fc506700 1 -- 172.21.15.84:0/2107203051 shutdown_connections 2024-05-22T22:52:59.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:52:59.816+0000 7fb7fc506700 1 -- 172.21.15.84:0/2107203051 wait complete. 2024-05-22T22:53:00.574 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi096 2024-05-22T22:53:00.574 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-22T22:53:00.574 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/etc/ceph/ceph.conf 2024-05-22T22:53:00.592 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-22T22:53:00.592 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-22T22:53:00.651 INFO:tasks.cephadm:Adding host smithi096 to orchestrator... 2024-05-22T22:53:00.651 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph orch host add smithi096 2024-05-22T22:53:00.890 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T22:53:01.104 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:00 smithi084 ceph-mon[32186]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:53:01.104 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:00 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:01.104 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:00 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:53:01.104 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:00 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:01.104 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:00 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:53:01.104 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:00 smithi084 ceph-mon[32186]: Updating smithi084:/etc/ceph/ceph.conf 2024-05-22T22:53:01.104 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:00 smithi084 ceph-mon[32186]: Updating smithi084:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-22T22:53:01.104 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:00 smithi084 ceph-mon[32186]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-05-22T22:53:01.104 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:00 smithi084 ceph-mon[32186]: Updating smithi084:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-22T22:53:01.104 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:00 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:01.105 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:00 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:01.105 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:00 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:02.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.425+0000 7f94eb9cb700 1 -- 172.21.15.84:0/15214730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94e4100ee0 msgr2=0x7f94e41012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:02.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.425+0000 7f94eb9cb700 1 --2- 172.21.15.84:0/15214730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94e4100ee0 0x7f94e41012c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009910 tx=0x7f94e0009c20 comp rx=0 tx=0).stop 2024-05-22T22:53:02.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.425+0000 7f94eb9cb700 1 -- 172.21.15.84:0/15214730 shutdown_connections 2024-05-22T22:53:02.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.425+0000 7f94eb9cb700 1 --2- 172.21.15.84:0/15214730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94e4100ee0 0x7f94e41012c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:02.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.425+0000 7f94eb9cb700 1 -- 172.21.15.84:0/15214730 >> 172.21.15.84:0/15214730 conn(0x7f94e40fc7b0 msgr2=0x7f94e40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:53:02.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.425+0000 7f94eb9cb700 1 -- 172.21.15.84:0/15214730 shutdown_connections 2024-05-22T22:53:02.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.425+0000 7f94eb9cb700 1 -- 172.21.15.84:0/15214730 wait complete. 2024-05-22T22:53:02.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.426+0000 7f94eb9cb700 1 Processor -- start 2024-05-22T22:53:02.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.426+0000 7f94eb9cb700 1 -- start start 2024-05-22T22:53:02.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.426+0000 7f94eb9cb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94e4100ee0 0x7f94e4198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:02.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.426+0000 7f94eb9cb700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f94e410b2b0 con 0x7f94e4100ee0 2024-05-22T22:53:02.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.426+0000 7f94e9767700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94e4100ee0 0x7f94e4198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:02.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.426+0000 7f94e9767700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94e4100ee0 0x7f94e4198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42892/0 (socket says 172.21.15.84:42892) 2024-05-22T22:53:02.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.426+0000 7f94e9767700 1 -- 172.21.15.84:0/971843294 learned_addr learned my addr 172.21.15.84:0/971843294 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:53:02.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.426+0000 7f94e9767700 1 -- 172.21.15.84:0/971843294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e4198c40 con 0x7f94e4100ee0 2024-05-22T22:53:02.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.426+0000 7f94e9767700 1 --2- 172.21.15.84:0/971843294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94e4100ee0 0x7f94e4198700 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f94e0005850 tx=0x7f94e0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:53:02.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.427+0000 7f94da7fc700 1 -- 172.21.15.84:0/971843294 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94e0026070 con 0x7f94e4100ee0 2024-05-22T22:53:02.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.427+0000 7f94da7fc700 1 -- 172.21.15.84:0/971843294 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94e000bbd0 con 0x7f94e4100ee0 2024-05-22T22:53:02.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.427+0000 7f94da7fc700 1 -- 172.21.15.84:0/971843294 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94e0003970 con 0x7f94e4100ee0 2024-05-22T22:53:02.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.427+0000 7f94eb9cb700 1 -- 172.21.15.84:0/971843294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e4198f30 con 0x7f94e4100ee0 2024-05-22T22:53:02.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.427+0000 7f94eb9cb700 1 -- 172.21.15.84:0/971843294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e41993f0 con 0x7f94e4100ee0 2024-05-22T22:53:02.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.428+0000 7f94da7fc700 1 -- 172.21.15.84:0/971843294 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f94e0029070 con 0x7f94e4100ee0 2024-05-22T22:53:02.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.428+0000 7f94eb9cb700 1 -- 172.21.15.84:0/971843294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94e41023c0 con 0x7f94e4100ee0 2024-05-22T22:53:02.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.428+0000 7f94da7fc700 1 --2- 172.21.15.84:0/971843294 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f94d0039150 0x7f94d003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:02.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.428+0000 7f94da7fc700 1 -- 172.21.15.84:0/971843294 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f94e001e070 con 0x7f94e4100ee0 2024-05-22T22:53:02.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.428+0000 7f94e8f66700 1 --2- 172.21.15.84:0/971843294 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f94d0039150 0x7f94d003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:02.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.428+0000 7f94e8f66700 1 --2- 172.21.15.84:0/971843294 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f94d0039150 0x7f94d003b610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f94d4006fd0 tx=0x7f94d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:53:02.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.433+0000 7f94da7fc700 1 -- 172.21.15.84:0/971843294 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f94e0050030 con 0x7f94e4100ee0 2024-05-22T22:53:02.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:02.633+0000 7f94eb9cb700 1 -- 172.21.15.84:0/971843294 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi096", "target": ["mon-mgr", ""]}) -- 0x7f94e40639c0 con 0x7f94d0039150 2024-05-22T22:53:03.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:03 smithi084 ceph-mon[32186]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi096", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:53:03.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:03 smithi084 ceph-mon[32186]: Deploying cephadm binary to smithi096 2024-05-22T22:53:06.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:06.737+0000 7f94da7fc700 1 -- 172.21.15.84:0/971843294 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f94e40639c0 con 0x7f94d0039150 2024-05-22T22:53:06.730 INFO:teuthology.orchestra.run.smithi084.stdout:Added host 'smithi096' with addr '172.21.15.96' 2024-05-22T22:53:06.731 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:06 smithi084 ceph-mon[32186]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:06.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:06.739+0000 7f94eb9cb700 1 -- 172.21.15.84:0/971843294 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f94d0039150 msgr2=0x7f94d003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:06.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:06.739+0000 7f94eb9cb700 1 --2- 172.21.15.84:0/971843294 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f94d0039150 0x7f94d003b610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f94d4006fd0 tx=0x7f94d4006e40 comp rx=0 tx=0).stop 2024-05-22T22:53:06.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:06.739+0000 7f94eb9cb700 1 -- 172.21.15.84:0/971843294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94e4100ee0 msgr2=0x7f94e4198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:06.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:06.739+0000 7f94eb9cb700 1 --2- 172.21.15.84:0/971843294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94e4100ee0 0x7f94e4198700 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f94e0005850 tx=0x7f94e0005030 comp rx=0 tx=0).stop 2024-05-22T22:53:06.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:06.739+0000 7f94eb9cb700 1 -- 172.21.15.84:0/971843294 shutdown_connections 2024-05-22T22:53:06.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:06.739+0000 7f94eb9cb700 1 --2- 172.21.15.84:0/971843294 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f94d0039150 0x7f94d003b610 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:06.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:06.739+0000 7f94eb9cb700 1 --2- 172.21.15.84:0/971843294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94e4100ee0 0x7f94e4198700 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:06.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:06.739+0000 7f94eb9cb700 1 -- 172.21.15.84:0/971843294 >> 172.21.15.84:0/971843294 conn(0x7f94e40fc7b0 msgr2=0x7f94e41028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:53:06.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:06.740+0000 7f94eb9cb700 1 -- 172.21.15.84:0/971843294 shutdown_connections 2024-05-22T22:53:06.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:06.740+0000 7f94eb9cb700 1 -- 172.21.15.84:0/971843294 wait complete. 2024-05-22T22:53:07.327 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-22T22:53:07.563 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T22:53:07.937 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:07 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:07.938 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:07 smithi084 ceph-mon[32186]: Added host smithi096 2024-05-22T22:53:07.938 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:07 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:53:07.938 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:07 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:07.938 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:07 smithi084 ceph-mon[32186]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:08.847 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:08 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:09.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.069+0000 7f3869590700 1 -- 172.21.15.84:0/1557411888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3864100eb0 msgr2=0x7f3864101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:09.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.069+0000 7f3869590700 1 --2- 172.21.15.84:0/1557411888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3864100eb0 0x7f3864101290 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3854009910 tx=0x7f3854009c20 comp rx=0 tx=0).stop 2024-05-22T22:53:09.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.069+0000 7f3869590700 1 -- 172.21.15.84:0/1557411888 shutdown_connections 2024-05-22T22:53:09.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.069+0000 7f3869590700 1 --2- 172.21.15.84:0/1557411888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3864100eb0 0x7f3864101290 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:09.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.069+0000 7f3869590700 1 -- 172.21.15.84:0/1557411888 >> 172.21.15.84:0/1557411888 conn(0x7f38640fc760 msgr2=0x7f38640feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:53:09.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.069+0000 7f3869590700 1 -- 172.21.15.84:0/1557411888 shutdown_connections 2024-05-22T22:53:09.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.069+0000 7f3869590700 1 -- 172.21.15.84:0/1557411888 wait complete. 2024-05-22T22:53:09.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.069+0000 7f3869590700 1 Processor -- start 2024-05-22T22:53:09.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.070+0000 7f3869590700 1 -- start start 2024-05-22T22:53:09.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.070+0000 7f3869590700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3864100eb0 0x7f386419c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:09.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.070+0000 7f3869590700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f386410b280 con 0x7f3864100eb0 2024-05-22T22:53:09.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.070+0000 7f3862ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3864100eb0 0x7f386419c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:09.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.070+0000 7f3862ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3864100eb0 0x7f386419c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42900/0 (socket says 172.21.15.84:42900) 2024-05-22T22:53:09.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.070+0000 7f3862ffd700 1 -- 172.21.15.84:0/1339792243 learned_addr learned my addr 172.21.15.84:0/1339792243 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:53:09.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.070+0000 7f3862ffd700 1 -- 172.21.15.84:0/1339792243 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f386419c940 con 0x7f3864100eb0 2024-05-22T22:53:09.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.070+0000 7f3862ffd700 1 --2- 172.21.15.84:0/1339792243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3864100eb0 0x7f386419c400 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3854005850 tx=0x7f3854005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:53:09.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.070+0000 7f3860ff9700 1 -- 172.21.15.84:0/1339792243 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3854026070 con 0x7f3864100eb0 2024-05-22T22:53:09.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.071+0000 7f3869590700 1 -- 172.21.15.84:0/1339792243 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f386419cc30 con 0x7f3864100eb0 2024-05-22T22:53:09.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.071+0000 7f3860ff9700 1 -- 172.21.15.84:0/1339792243 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f385400bbd0 con 0x7f3864100eb0 2024-05-22T22:53:09.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.071+0000 7f3860ff9700 1 -- 172.21.15.84:0/1339792243 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3854003970 con 0x7f3864100eb0 2024-05-22T22:53:09.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.071+0000 7f3869590700 1 -- 172.21.15.84:0/1339792243 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38641965c0 con 0x7f3864100eb0 2024-05-22T22:53:09.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.071+0000 7f3860ff9700 1 -- 172.21.15.84:0/1339792243 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f3854029070 con 0x7f3864100eb0 2024-05-22T22:53:09.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.072+0000 7f3869590700 1 -- 172.21.15.84:0/1339792243 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3864102390 con 0x7f3864100eb0 2024-05-22T22:53:09.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.072+0000 7f3860ff9700 1 --2- 172.21.15.84:0/1339792243 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f385003b300 0x7f385003d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:09.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.072+0000 7f3860ff9700 1 -- 172.21.15.84:0/1339792243 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f385401e070 con 0x7f3864100eb0 2024-05-22T22:53:09.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.072+0000 7f385a7fc700 1 --2- 172.21.15.84:0/1339792243 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f385003b300 0x7f385003d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:09.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.072+0000 7f385a7fc700 1 --2- 172.21.15.84:0/1339792243 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f385003b300 0x7f385003d7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f384c006fd0 tx=0x7f384c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:53:09.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.077+0000 7f3860ff9700 1 -- 172.21.15.84:0/1339792243 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3854013070 con 0x7f3864100eb0 2024-05-22T22:53:09.267 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.274+0000 7f3869590700 1 -- 172.21.15.84:0/1339792243 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3864197af0 con 0x7f385003b300 2024-05-22T22:53:09.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.275+0000 7f3860ff9700 1 -- 172.21.15.84:0/1339792243 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f3864197af0 con 0x7f385003b300 2024-05-22T22:53:09.268 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:53:09.268 INFO:teuthology.orchestra.run.smithi084.stdout:[{"addr": "172.21.15.84", "hostname": "smithi084", "labels": [], "status": ""}, {"addr": "172.21.15.96", "hostname": "smithi096", "labels": [], "status": ""}] 2024-05-22T22:53:09.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.277+0000 7f3869590700 1 -- 172.21.15.84:0/1339792243 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f385003b300 msgr2=0x7f385003d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:09.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.277+0000 7f3869590700 1 --2- 172.21.15.84:0/1339792243 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f385003b300 0x7f385003d7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f384c006fd0 tx=0x7f384c006e40 comp rx=0 tx=0).stop 2024-05-22T22:53:09.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.277+0000 7f3869590700 1 -- 172.21.15.84:0/1339792243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3864100eb0 msgr2=0x7f386419c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:09.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.277+0000 7f3869590700 1 --2- 172.21.15.84:0/1339792243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3864100eb0 0x7f386419c400 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3854005850 tx=0x7f3854005030 comp rx=0 tx=0).stop 2024-05-22T22:53:09.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.277+0000 7f3869590700 1 -- 172.21.15.84:0/1339792243 shutdown_connections 2024-05-22T22:53:09.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.277+0000 7f3869590700 1 --2- 172.21.15.84:0/1339792243 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f385003b300 0x7f385003d7c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:09.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.277+0000 7f3869590700 1 --2- 172.21.15.84:0/1339792243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3864100eb0 0x7f386419c400 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:09.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.277+0000 7f3869590700 1 -- 172.21.15.84:0/1339792243 >> 172.21.15.84:0/1339792243 conn(0x7f38640fc760 msgr2=0x7f3864102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:53:09.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.277+0000 7f3869590700 1 -- 172.21.15.84:0/1339792243 shutdown_connections 2024-05-22T22:53:09.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:09.278+0000 7f3869590700 1 -- 172.21.15.84:0/1339792243 wait complete. 2024-05-22T22:53:09.915 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi204 2024-05-22T22:53:09.915 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-22T22:53:09.915 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.conf 2024-05-22T22:53:09.934 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-22T22:53:09.934 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-22T22:53:09.992 INFO:tasks.cephadm:Adding host smithi204 to orchestrator... 2024-05-22T22:53:09.992 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph orch host add smithi204 2024-05-22T22:53:10.224 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T22:53:10.616 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:10 smithi084 ceph-mon[32186]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-22T22:53:10.616 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:10 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:10.616 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:10 smithi084 ceph-mon[32186]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:11.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.627+0000 7fb8f2d2f700 1 -- 172.21.15.84:0/3900647675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8ec100f10 msgr2=0x7fb8ec1012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:11.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.627+0000 7fb8f2d2f700 1 --2- 172.21.15.84:0/3900647675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8ec100f10 0x7fb8ec1012f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009910 tx=0x7fb8e0009c20 comp rx=0 tx=0).stop 2024-05-22T22:53:11.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.627+0000 7fb8f2d2f700 1 -- 172.21.15.84:0/3900647675 shutdown_connections 2024-05-22T22:53:11.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.627+0000 7fb8f2d2f700 1 --2- 172.21.15.84:0/3900647675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8ec100f10 0x7fb8ec1012f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:11.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.627+0000 7fb8f2d2f700 1 -- 172.21.15.84:0/3900647675 >> 172.21.15.84:0/3900647675 conn(0x7fb8ec0767c0 msgr2=0x7fb8ec076bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:53:11.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.627+0000 7fb8f2d2f700 1 -- 172.21.15.84:0/3900647675 shutdown_connections 2024-05-22T22:53:11.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.627+0000 7fb8f2d2f700 1 -- 172.21.15.84:0/3900647675 wait complete. 2024-05-22T22:53:11.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.628+0000 7fb8f2d2f700 1 Processor -- start 2024-05-22T22:53:11.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.628+0000 7fb8f2d2f700 1 -- start start 2024-05-22T22:53:11.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.628+0000 7fb8f2d2f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8ec100f10 0x7fb8ec19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:11.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.628+0000 7fb8f2d2f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec10b290 con 0x7fb8ec100f10 2024-05-22T22:53:11.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.628+0000 7fb8f0acb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8ec100f10 0x7fb8ec19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:11.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.628+0000 7fb8f0acb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8ec100f10 0x7fb8ec19d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43328/0 (socket says 172.21.15.84:43328) 2024-05-22T22:53:11.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.628+0000 7fb8f0acb700 1 -- 172.21.15.84:0/317821359 learned_addr learned my addr 172.21.15.84:0/317821359 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:53:11.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.628+0000 7fb8f0acb700 1 -- 172.21.15.84:0/317821359 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8ec19db10 con 0x7fb8ec100f10 2024-05-22T22:53:11.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.628+0000 7fb8f0acb700 1 --2- 172.21.15.84:0/317821359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8ec100f10 0x7fb8ec19d5d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0005850 tx=0x7fb8e0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:53:11.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.628+0000 7fb8e9ffb700 1 -- 172.21.15.84:0/317821359 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8e0026070 con 0x7fb8ec100f10 2024-05-22T22:53:11.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.629+0000 7fb8e9ffb700 1 -- 172.21.15.84:0/317821359 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8e000bbd0 con 0x7fb8ec100f10 2024-05-22T22:53:11.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.629+0000 7fb8f2d2f700 1 -- 172.21.15.84:0/317821359 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8ec19de00 con 0x7fb8ec100f10 2024-05-22T22:53:11.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.629+0000 7fb8e9ffb700 1 -- 172.21.15.84:0/317821359 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8e0003970 con 0x7fb8ec100f10 2024-05-22T22:53:11.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.629+0000 7fb8f2d2f700 1 -- 172.21.15.84:0/317821359 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8ec197870 con 0x7fb8ec100f10 2024-05-22T22:53:11.624 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.629+0000 7fb8e9ffb700 1 -- 172.21.15.84:0/317821359 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb8e0029070 con 0x7fb8ec100f10 2024-05-22T22:53:11.624 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.630+0000 7fb8e9ffb700 1 --2- 172.21.15.84:0/317821359 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb8dc0390f0 0x7fb8dc03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:11.624 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.630+0000 7fb8e9ffb700 1 -- 172.21.15.84:0/317821359 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb8e001e070 con 0x7fb8ec100f10 2024-05-22T22:53:11.624 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.630+0000 7fb8f2d2f700 1 -- 172.21.15.84:0/317821359 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8ec1023a0 con 0x7fb8ec100f10 2024-05-22T22:53:11.624 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.630+0000 7fb8ebfff700 1 --2- 172.21.15.84:0/317821359 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb8dc0390f0 0x7fb8dc03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:11.624 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.630+0000 7fb8ebfff700 1 --2- 172.21.15.84:0/317821359 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb8dc0390f0 0x7fb8dc03b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8006fd0 tx=0x7fb8d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:53:11.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.635+0000 7fb8e9ffb700 1 -- 172.21.15.84:0/317821359 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb8e0030070 con 0x7fb8ec100f10 2024-05-22T22:53:11.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:11.832+0000 7fb8f2d2f700 1 -- 172.21.15.84:0/317821359 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}) -- 0x7fb8ec198bd0 con 0x7fb8dc0390f0 2024-05-22T22:53:12.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:12 smithi084 ceph-mon[32186]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:13.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:13 smithi084 ceph-mon[32186]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:53:13.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:13 smithi084 ceph-mon[32186]: Deploying cephadm binary to smithi204 2024-05-22T22:53:13.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:13 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:13.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:13 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:13.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:13 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:13.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:13 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:13.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:13 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi096", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:53:13.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:13 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:13.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:13 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:53:14.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:14 smithi084 ceph-mon[32186]: Updating smithi096:/etc/ceph/ceph.conf 2024-05-22T22:53:14.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:14 smithi084 ceph-mon[32186]: Updating smithi096:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-22T22:53:14.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:14 smithi084 ceph-mon[32186]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-05-22T22:53:14.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:14 smithi084 ceph-mon[32186]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:15.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:15.997+0000 7fb8e9ffb700 1 -- 172.21.15.84:0/317821359 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fb8ec198bd0 con 0x7fb8dc0390f0 2024-05-22T22:53:15.990 INFO:teuthology.orchestra.run.smithi084.stdout:Added host 'smithi204' with addr '172.21.15.204' 2024-05-22T22:53:15.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:15.999+0000 7fb8f2d2f700 1 -- 172.21.15.84:0/317821359 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb8dc0390f0 msgr2=0x7fb8dc03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:15.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:15.999+0000 7fb8f2d2f700 1 --2- 172.21.15.84:0/317821359 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb8dc0390f0 0x7fb8dc03b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8006fd0 tx=0x7fb8d8006e40 comp rx=0 tx=0).stop 2024-05-22T22:53:15.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:15.999+0000 7fb8f2d2f700 1 -- 172.21.15.84:0/317821359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8ec100f10 msgr2=0x7fb8ec19d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:15.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:15.999+0000 7fb8f2d2f700 1 --2- 172.21.15.84:0/317821359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8ec100f10 0x7fb8ec19d5d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0005850 tx=0x7fb8e0005030 comp rx=0 tx=0).stop 2024-05-22T22:53:15.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:15.999+0000 7fb8f2d2f700 1 -- 172.21.15.84:0/317821359 shutdown_connections 2024-05-22T22:53:15.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:15.999+0000 7fb8f2d2f700 1 --2- 172.21.15.84:0/317821359 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb8dc0390f0 0x7fb8dc03b5b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:15.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:15.999+0000 7fb8f2d2f700 1 --2- 172.21.15.84:0/317821359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8ec100f10 0x7fb8ec19d5d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:15.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:15.999+0000 7fb8f2d2f700 1 -- 172.21.15.84:0/317821359 >> 172.21.15.84:0/317821359 conn(0x7fb8ec0767c0 msgr2=0x7fb8ec069780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:53:15.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:15.999+0000 7fb8f2d2f700 1 -- 172.21.15.84:0/317821359 shutdown_connections 2024-05-22T22:53:15.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:15.999+0000 7fb8f2d2f700 1 -- 172.21.15.84:0/317821359 wait complete. 2024-05-22T22:53:16.616 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:16 smithi084 ceph-mon[32186]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:16.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:16 smithi084 ceph-mon[32186]: Updating smithi096:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-22T22:53:16.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:16 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:16.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:16 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:16.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:16 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:16.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:16 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:16.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:16 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:53:16.672 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-22T22:53:16.904 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T22:53:17.616 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:17 smithi084 ceph-mon[32186]: Added host smithi204 2024-05-22T22:53:17.616 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:17 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:17.616 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:17 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:18.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.411+0000 7f3c85c59700 1 -- 172.21.15.84:0/384414028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c80101fd0 msgr2=0x7f3c801023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:18.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.411+0000 7f3c85c59700 1 --2- 172.21.15.84:0/384414028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c80101fd0 0x7f3c801023b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000b0a0 tx=0x7f3c7000b3b0 comp rx=0 tx=0).stop 2024-05-22T22:53:18.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.411+0000 7f3c85c59700 1 -- 172.21.15.84:0/384414028 shutdown_connections 2024-05-22T22:53:18.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.411+0000 7f3c85c59700 1 --2- 172.21.15.84:0/384414028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c80101fd0 0x7f3c801023b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:18.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.411+0000 7f3c85c59700 1 -- 172.21.15.84:0/384414028 >> 172.21.15.84:0/384414028 conn(0x7f3c800fd8a0 msgr2=0x7f3c800ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:53:18.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.411+0000 7f3c85c59700 1 -- 172.21.15.84:0/384414028 shutdown_connections 2024-05-22T22:53:18.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.411+0000 7f3c85c59700 1 -- 172.21.15.84:0/384414028 wait complete. 2024-05-22T22:53:18.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.411+0000 7f3c85c59700 1 Processor -- start 2024-05-22T22:53:18.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.412+0000 7f3c85c59700 1 -- start start 2024-05-22T22:53:18.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.412+0000 7f3c85c59700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c80101fd0 0x7f3c80074660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:18.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.412+0000 7f3c85c59700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8010c3a0 con 0x7f3c80101fd0 2024-05-22T22:53:18.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.412+0000 7f3c7f7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c80101fd0 0x7f3c80074660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:18.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.412+0000 7f3c7f7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c80101fd0 0x7f3c80074660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43336/0 (socket says 172.21.15.84:43336) 2024-05-22T22:53:18.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.412+0000 7f3c7f7fe700 1 -- 172.21.15.84:0/320298684 learned_addr learned my addr 172.21.15.84:0/320298684 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:53:18.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.412+0000 7f3c7f7fe700 1 -- 172.21.15.84:0/320298684 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c80076550 con 0x7f3c80101fd0 2024-05-22T22:53:18.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.412+0000 7f3c7f7fe700 1 --2- 172.21.15.84:0/320298684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c80101fd0 0x7f3c80074660 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3c70015040 tx=0x7f3c70003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:53:18.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.413+0000 7f3c7cff9700 1 -- 172.21.15.84:0/320298684 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c70017040 con 0x7f3c80101fd0 2024-05-22T22:53:18.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.413+0000 7f3c85c59700 1 -- 172.21.15.84:0/320298684 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c80074c00 con 0x7f3c80101fd0 2024-05-22T22:53:18.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.413+0000 7f3c7cff9700 1 -- 172.21.15.84:0/320298684 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c70022070 con 0x7f3c80101fd0 2024-05-22T22:53:18.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.413+0000 7f3c7cff9700 1 -- 172.21.15.84:0/320298684 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c7001d5d0 con 0x7f3c80101fd0 2024-05-22T22:53:18.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.413+0000 7f3c85c59700 1 -- 172.21.15.84:0/320298684 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c800750c0 con 0x7f3c80101fd0 2024-05-22T22:53:18.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.414+0000 7f3c7cff9700 1 -- 172.21.15.84:0/320298684 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f3c7001d8d0 con 0x7f3c80101fd0 2024-05-22T22:53:18.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.414+0000 7f3c85c59700 1 -- 172.21.15.84:0/320298684 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c801034b0 con 0x7f3c80101fd0 2024-05-22T22:53:18.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.414+0000 7f3c7cff9700 1 --2- 172.21.15.84:0/320298684 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f3c6c0390f0 0x7f3c6c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:18.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.414+0000 7f3c7cff9700 1 -- 172.21.15.84:0/320298684 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3c7000f050 con 0x7f3c80101fd0 2024-05-22T22:53:18.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.414+0000 7f3c7effd700 1 --2- 172.21.15.84:0/320298684 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f3c6c0390f0 0x7f3c6c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:18.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.414+0000 7f3c7effd700 1 --2- 172.21.15.84:0/320298684 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f3c6c0390f0 0x7f3c6c03b5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3c68006fd0 tx=0x7f3c68006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:53:18.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.419+0000 7f3c7cff9700 1 -- 172.21.15.84:0/320298684 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3c70007e20 con 0x7f3c80101fd0 2024-05-22T22:53:18.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.621+0000 7f3c85c59700 1 -- 172.21.15.84:0/320298684 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3c80075b90 con 0x7f3c6c0390f0 2024-05-22T22:53:18.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.622+0000 7f3c7cff9700 1 -- 172.21.15.84:0/320298684 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f3c80075b90 con 0x7f3c6c0390f0 2024-05-22T22:53:18.615 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:53:18.615 INFO:teuthology.orchestra.run.smithi084.stdout:[{"addr": "172.21.15.84", "hostname": "smithi084", "labels": [], "status": ""}, {"addr": "172.21.15.96", "hostname": "smithi096", "labels": [], "status": ""}, {"addr": "172.21.15.204", "hostname": "smithi204", "labels": [], "status": ""}] 2024-05-22T22:53:18.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.624+0000 7f3c85c59700 1 -- 172.21.15.84:0/320298684 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f3c6c0390f0 msgr2=0x7f3c6c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:18.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.624+0000 7f3c85c59700 1 --2- 172.21.15.84:0/320298684 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f3c6c0390f0 0x7f3c6c03b5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3c68006fd0 tx=0x7f3c68006e40 comp rx=0 tx=0).stop 2024-05-22T22:53:18.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.624+0000 7f3c85c59700 1 -- 172.21.15.84:0/320298684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c80101fd0 msgr2=0x7f3c80074660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:18.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.624+0000 7f3c85c59700 1 --2- 172.21.15.84:0/320298684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c80101fd0 0x7f3c80074660 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3c70015040 tx=0x7f3c70003d50 comp rx=0 tx=0).stop 2024-05-22T22:53:18.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.624+0000 7f3c85c59700 1 -- 172.21.15.84:0/320298684 shutdown_connections 2024-05-22T22:53:18.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.624+0000 7f3c85c59700 1 --2- 172.21.15.84:0/320298684 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f3c6c0390f0 0x7f3c6c03b5b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:18.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.624+0000 7f3c85c59700 1 --2- 172.21.15.84:0/320298684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c80101fd0 0x7f3c80074660 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:18.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.624+0000 7f3c85c59700 1 -- 172.21.15.84:0/320298684 >> 172.21.15.84:0/320298684 conn(0x7f3c800fd8a0 msgr2=0x7f3c801039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:53:18.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.624+0000 7f3c85c59700 1 -- 172.21.15.84:0/320298684 shutdown_connections 2024-05-22T22:53:18.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:18.624+0000 7f3c85c59700 1 -- 172.21.15.84:0/320298684 wait complete. 2024-05-22T22:53:18.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:18 smithi084 ceph-mon[32186]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:19.201 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-22T22:53:19.201 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph osd crush tunables default 2024-05-22T22:53:19.434 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T22:53:19.835 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:19 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:19.835 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:19 smithi084 ceph-mon[32186]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-22T22:53:19.835 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:19 smithi084 ceph-mon[32186]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:20.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.918+0000 7f268337d700 1 -- 172.21.15.84:0/2391865753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f267c107d80 msgr2=0x7f267c108160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:20.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.918+0000 7f268337d700 1 --2- 172.21.15.84:0/2391865753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f267c107d80 0x7f267c108160 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2678009910 tx=0x7f2678009c20 comp rx=0 tx=0).stop 2024-05-22T22:53:20.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.918+0000 7f268337d700 1 -- 172.21.15.84:0/2391865753 shutdown_connections 2024-05-22T22:53:20.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.918+0000 7f268337d700 1 --2- 172.21.15.84:0/2391865753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f267c107d80 0x7f267c108160 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:20.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.918+0000 7f268337d700 1 -- 172.21.15.84:0/2391865753 >> 172.21.15.84:0/2391865753 conn(0x7f267c078290 msgr2=0x7f267c0786a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:53:20.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.918+0000 7f268337d700 1 -- 172.21.15.84:0/2391865753 shutdown_connections 2024-05-22T22:53:20.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.918+0000 7f268337d700 1 -- 172.21.15.84:0/2391865753 wait complete. 2024-05-22T22:53:20.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.919+0000 7f268337d700 1 Processor -- start 2024-05-22T22:53:20.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.919+0000 7f268337d700 1 -- start start 2024-05-22T22:53:20.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.919+0000 7f268337d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f267c107d80 0x7f267c101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:20.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.919+0000 7f268337d700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f267c10c3a0 con 0x7f267c107d80 2024-05-22T22:53:20.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.919+0000 7f2681119700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f267c107d80 0x7f267c101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:20.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.919+0000 7f2681119700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f267c107d80 0x7f267c101630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43346/0 (socket says 172.21.15.84:43346) 2024-05-22T22:53:20.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.919+0000 7f2681119700 1 -- 172.21.15.84:0/1451730735 learned_addr learned my addr 172.21.15.84:0/1451730735 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:53:20.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.919+0000 7f2681119700 1 -- 172.21.15.84:0/1451730735 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c103520 con 0x7f267c107d80 2024-05-22T22:53:20.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.919+0000 7f2681119700 1 --2- 172.21.15.84:0/1451730735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f267c107d80 0x7f267c101630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2678005850 tx=0x7f2678005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:53:20.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.919+0000 7f26727fc700 1 -- 172.21.15.84:0/1451730735 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2678026070 con 0x7f267c107d80 2024-05-22T22:53:20.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.919+0000 7f26727fc700 1 -- 172.21.15.84:0/1451730735 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f267800bc30 con 0x7f267c107d80 2024-05-22T22:53:20.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.919+0000 7f268337d700 1 -- 172.21.15.84:0/1451730735 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267c101bd0 con 0x7f267c107d80 2024-05-22T22:53:20.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.919+0000 7f26727fc700 1 -- 172.21.15.84:0/1451730735 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2678003a20 con 0x7f267c107d80 2024-05-22T22:53:20.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.919+0000 7f268337d700 1 -- 172.21.15.84:0/1451730735 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f267c102090 con 0x7f267c107d80 2024-05-22T22:53:20.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.920+0000 7f26727fc700 1 -- 172.21.15.84:0/1451730735 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f2678029070 con 0x7f267c107d80 2024-05-22T22:53:20.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.920+0000 7f26727fc700 1 --2- 172.21.15.84:0/1451730735 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f2668039150 0x7f266803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:20.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.920+0000 7f26727fc700 1 -- 172.21.15.84:0/1451730735 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f267801e070 con 0x7f267c107d80 2024-05-22T22:53:20.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.921+0000 7f2680918700 1 --2- 172.21.15.84:0/1451730735 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f2668039150 0x7f266803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:20.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.921+0000 7f268337d700 1 -- 172.21.15.84:0/1451730735 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f267c069f80 con 0x7f267c107d80 2024-05-22T22:53:20.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.921+0000 7f2680918700 1 --2- 172.21.15.84:0/1451730735 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f2668039150 0x7f266803b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f266c006fd0 tx=0x7f266c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:53:20.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:20.926+0000 7f26727fc700 1 -- 172.21.15.84:0/1451730735 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2678030070 con 0x7f267c107d80 2024-05-22T22:53:21.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:21.125+0000 7f268337d700 1 -- 172.21.15.84:0/1451730735 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f267c102b60 con 0x7f267c107d80 2024-05-22T22:53:21.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:21.176+0000 7f26727fc700 1 -- 172.21.15.84:0/1451730735 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f2678013070 con 0x7f267c107d80 2024-05-22T22:53:21.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:21.179+0000 7f268337d700 1 -- 172.21.15.84:0/1451730735 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f2668039150 msgr2=0x7f266803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:21.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:21.179+0000 7f268337d700 1 --2- 172.21.15.84:0/1451730735 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f2668039150 0x7f266803b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f266c006fd0 tx=0x7f266c006e40 comp rx=0 tx=0).stop 2024-05-22T22:53:21.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:21.179+0000 7f268337d700 1 -- 172.21.15.84:0/1451730735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f267c107d80 msgr2=0x7f267c101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:21.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:21.179+0000 7f268337d700 1 --2- 172.21.15.84:0/1451730735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f267c107d80 0x7f267c101630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2678005850 tx=0x7f2678005030 comp rx=0 tx=0).stop 2024-05-22T22:53:21.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:21.179+0000 7f268337d700 1 -- 172.21.15.84:0/1451730735 shutdown_connections 2024-05-22T22:53:21.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:21.179+0000 7f268337d700 1 --2- 172.21.15.84:0/1451730735 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f2668039150 0x7f266803b610 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:21.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:21.179+0000 7f268337d700 1 --2- 172.21.15.84:0/1451730735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f267c107d80 0x7f267c101630 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:21.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:21.179+0000 7f268337d700 1 -- 172.21.15.84:0/1451730735 >> 172.21.15.84:0/1451730735 conn(0x7f267c078290 msgr2=0x7f267c0fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:53:21.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:21.179+0000 7f268337d700 1 -- 172.21.15.84:0/1451730735 shutdown_connections 2024-05-22T22:53:21.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:21.179+0000 7f268337d700 1 -- 172.21.15.84:0/1451730735 wait complete. 2024-05-22T22:53:21.174 INFO:teuthology.orchestra.run.smithi084.stderr:adjusted tunables profile to default 2024-05-22T22:53:21.553 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:21 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1451730735' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-22T22:53:21.788 INFO:tasks.cephadm:Adding mon.a on smithi084 2024-05-22T22:53:21.788 INFO:tasks.cephadm:Adding mon.b on smithi096 2024-05-22T22:53:21.788 INFO:tasks.cephadm:Adding mon.c on smithi204 2024-05-22T22:53:21.789 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph orch apply mon '3;smithi084:172.21.15.84=a;smithi096:172.21.15.96=b;smithi204:172.21.15.204=c' 2024-05-22T22:53:22.028 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-22T22:53:22.616 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:22 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1451730735' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-22T22:53:22.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:22 smithi084 ceph-mon[32186]: osdmap e4: 0 total, 0 up, 0 in 2024-05-22T22:53:22.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:22 smithi084 ceph-mon[32186]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:22.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:22 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:22.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:22 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:22.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:22 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:22.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:22 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:22.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:22 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:53:22.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:22 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:22.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:22 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:53:23.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.601+0000 7f75d6c3d700 1 -- 172.21.15.204:0/2880637405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75d0100d20 msgr2=0x7f75d0101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:23.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.601+0000 7f75d6c3d700 1 --2- 172.21.15.204:0/2880637405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75d0100d20 0x7f75d0101100 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009910 tx=0x7f75cc009c20 comp rx=0 tx=0).stop 2024-05-22T22:53:23.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.601+0000 7f75d6c3d700 1 -- 172.21.15.204:0/2880637405 shutdown_connections 2024-05-22T22:53:23.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.601+0000 7f75d6c3d700 1 --2- 172.21.15.204:0/2880637405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75d0100d20 0x7f75d0101100 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:23.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.601+0000 7f75d6c3d700 1 -- 172.21.15.204:0/2880637405 >> 172.21.15.204:0/2880637405 conn(0x7f75d00fc780 msgr2=0x7f75d00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:53:23.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.601+0000 7f75d6c3d700 1 -- 172.21.15.204:0/2880637405 shutdown_connections 2024-05-22T22:53:23.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.601+0000 7f75d6c3d700 1 -- 172.21.15.204:0/2880637405 wait complete. 2024-05-22T22:53:23.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.601+0000 7f75d6c3d700 1 Processor -- start 2024-05-22T22:53:23.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.601+0000 7f75d6c3d700 1 -- start start 2024-05-22T22:53:23.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.602+0000 7f75d6c3d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75d0100d20 0x7f75d019ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:23.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.602+0000 7f75d6c3d700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f75d0112a20 con 0x7f75d0100d20 2024-05-22T22:53:23.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.602+0000 7f75d49d9700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75d0100d20 0x7f75d019ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:23.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.602+0000 7f75d49d9700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75d0100d20 0x7f75d019ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.204:32810/0 (socket says 172.21.15.204:32810) 2024-05-22T22:53:23.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.602+0000 7f75d49d9700 1 -- 172.21.15.204:0/2620855354 learned_addr learned my addr 172.21.15.204:0/2620855354 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-22T22:53:23.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.602+0000 7f75d49d9700 1 -- 172.21.15.204:0/2620855354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d019e410 con 0x7f75d0100d20 2024-05-22T22:53:23.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.602+0000 7f75d49d9700 1 --2- 172.21.15.204:0/2620855354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75d0100d20 0x7f75d019ded0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f75cc006010 tx=0x7f75cc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:53:23.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.603+0000 7f75c5ffb700 1 -- 172.21.15.204:0/2620855354 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75cc026070 con 0x7f75d0100d20 2024-05-22T22:53:23.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.603+0000 7f75d6c3d700 1 -- 172.21.15.204:0/2620855354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d019e6a0 con 0x7f75d0100d20 2024-05-22T22:53:23.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.603+0000 7f75c5ffb700 1 -- 172.21.15.204:0/2620855354 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75cc00bc30 con 0x7f75d0100d20 2024-05-22T22:53:23.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.603+0000 7f75c5ffb700 1 -- 172.21.15.204:0/2620855354 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75cc0039a0 con 0x7f75d0100d20 2024-05-22T22:53:23.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.603+0000 7f75d6c3d700 1 -- 172.21.15.204:0/2620855354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d019eb00 con 0x7f75d0100d20 2024-05-22T22:53:23.598 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.604+0000 7f75d6c3d700 1 -- 172.21.15.204:0/2620855354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d006aaf0 con 0x7f75d0100d20 2024-05-22T22:53:23.598 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.604+0000 7f75c5ffb700 1 -- 172.21.15.204:0/2620855354 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f75cc029070 con 0x7f75d0100d20 2024-05-22T22:53:23.598 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.604+0000 7f75c5ffb700 1 --2- 172.21.15.204:0/2620855354 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f75b8039150 0x7f75b803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:23.598 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.604+0000 7f75c5ffb700 1 -- 172.21.15.204:0/2620855354 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f75cc01e070 con 0x7f75d0100d20 2024-05-22T22:53:23.598 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.605+0000 7f75c7fff700 1 --2- 172.21.15.204:0/2620855354 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f75b8039150 0x7f75b803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:23.598 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.605+0000 7f75c7fff700 1 --2- 172.21.15.204:0/2620855354 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f75b8039150 0x7f75b803b610 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f75c0006fd0 tx=0x7f75c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:53:23.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.609+0000 7f75c5ffb700 1 -- 172.21.15.204:0/2620855354 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f75cc013070 con 0x7f75d0100d20 2024-05-22T22:53:23.811 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.818+0000 7f75d6c3d700 1 -- 172.21.15.204:0/2620855354 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi084:172.21.15.84=a;smithi096:172.21.15.96=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}) -- 0x7f75d00fe700 con 0x7f75b8039150 2024-05-22T22:53:23.818 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.826+0000 7f75c5ffb700 1 -- 172.21.15.204:0/2620855354 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f75d00fe700 con 0x7f75b8039150 2024-05-22T22:53:23.818 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mon update... 2024-05-22T22:53:23.821 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.828+0000 7f75d6c3d700 1 -- 172.21.15.204:0/2620855354 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f75b8039150 msgr2=0x7f75b803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:23.821 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.828+0000 7f75d6c3d700 1 --2- 172.21.15.204:0/2620855354 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f75b8039150 0x7f75b803b610 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f75c0006fd0 tx=0x7f75c0006e40 comp rx=0 tx=0).stop 2024-05-22T22:53:23.821 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.828+0000 7f75d6c3d700 1 -- 172.21.15.204:0/2620855354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75d0100d20 msgr2=0x7f75d019ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:23.821 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.828+0000 7f75d6c3d700 1 --2- 172.21.15.204:0/2620855354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75d0100d20 0x7f75d019ded0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f75cc006010 tx=0x7f75cc005030 comp rx=0 tx=0).stop 2024-05-22T22:53:23.821 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.828+0000 7f75d6c3d700 1 -- 172.21.15.204:0/2620855354 shutdown_connections 2024-05-22T22:53:23.821 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.828+0000 7f75d6c3d700 1 --2- 172.21.15.204:0/2620855354 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f75b8039150 0x7f75b803b610 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:23.821 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.828+0000 7f75d6c3d700 1 --2- 172.21.15.204:0/2620855354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75d0100d20 0x7f75d019ded0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:23.821 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.828+0000 7f75d6c3d700 1 -- 172.21.15.204:0/2620855354 >> 172.21.15.204:0/2620855354 conn(0x7f75d00fc780 msgr2=0x7f75d00fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:53:23.821 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.828+0000 7f75d6c3d700 1 -- 172.21.15.204:0/2620855354 shutdown_connections 2024-05-22T22:53:23.821 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:23.828+0000 7f75d6c3d700 1 -- 172.21.15.204:0/2620855354 wait complete. 2024-05-22T22:53:23.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:23 smithi084 ceph-mon[32186]: Updating smithi204:/etc/ceph/ceph.conf 2024-05-22T22:53:23.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:23 smithi084 ceph-mon[32186]: Updating smithi204:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-22T22:53:23.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:23 smithi084 ceph-mon[32186]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-05-22T22:53:23.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:23 smithi084 ceph-mon[32186]: Updating smithi204:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-22T22:53:23.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:23 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:23.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:23 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:23.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:23 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:24.380 DEBUG:teuthology.orchestra.run.smithi096:mon.b> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mon.b.service 2024-05-22T22:53:24.383 DEBUG:teuthology.orchestra.run.smithi204:mon.c> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mon.c.service 2024-05-22T22:53:24.386 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-22T22:53:24.386 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-22T22:53:24.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:24 smithi084 ceph-mon[32186]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:24.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:24 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:24.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:24 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:53:24.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:24 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:24.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:24 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:53:24.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:24 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:24.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:24 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-22T22:53:24.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:24 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:25.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:25 smithi084 ceph-mon[32186]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi084:172.21.15.84=a;smithi096:172.21.15.96=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:53:25.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:25 smithi084 ceph-mon[32186]: Saving service mon spec with placement smithi084:172.21.15.84=a;smithi096:172.21.15.96=b;smithi204:172.21.15.204=c;count:3 2024-05-22T22:53:25.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:25 smithi084 ceph-mon[32186]: Deploying daemon mon.c on smithi204 2024-05-22T22:53:26.018 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-22T22:53:26.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:26 smithi084 ceph-mon[32186]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:28.170 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.175+0000 7f6c0404e700 1 -- 172.21.15.204:0/2168145508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bfc106470 msgr2=0x7f6bfc106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:28.170 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.175+0000 7f6c0404e700 1 --2- 172.21.15.204:0/2168145508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bfc106470 0x7f6bfc106850 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6bec009910 tx=0x7f6bec009c20 comp rx=0 tx=0).stop 2024-05-22T22:53:28.170 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.175+0000 7f6c0404e700 1 -- 172.21.15.204:0/2168145508 shutdown_connections 2024-05-22T22:53:28.170 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.175+0000 7f6c0404e700 1 --2- 172.21.15.204:0/2168145508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bfc106470 0x7f6bfc106850 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:28.170 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.175+0000 7f6c0404e700 1 -- 172.21.15.204:0/2168145508 >> 172.21.15.204:0/2168145508 conn(0x7f6bfc0773e0 msgr2=0x7f6bfc0777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:53:28.170 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.175+0000 7f6c0404e700 1 -- 172.21.15.204:0/2168145508 shutdown_connections 2024-05-22T22:53:28.170 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.175+0000 7f6c0404e700 1 -- 172.21.15.204:0/2168145508 wait complete. 2024-05-22T22:53:28.170 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.176+0000 7f6c0404e700 1 Processor -- start 2024-05-22T22:53:28.171 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.176+0000 7f6c0404e700 1 -- start start 2024-05-22T22:53:28.171 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.176+0000 7f6c0404e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bfc106470 0x7f6bfc19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:28.171 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.176+0000 7f6c0404e700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6bfc10aa20 con 0x7f6bfc106470 2024-05-22T22:53:28.171 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.176+0000 7f6c01dea700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bfc106470 0x7f6bfc19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:28.171 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.176+0000 7f6c01dea700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bfc106470 0x7f6bfc19c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.204:32832/0 (socket says 172.21.15.204:32832) 2024-05-22T22:53:28.171 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.176+0000 7f6c01dea700 1 -- 172.21.15.204:0/3569971302 learned_addr learned my addr 172.21.15.204:0/3569971302 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-22T22:53:28.171 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.176+0000 7f6c01dea700 1 -- 172.21.15.204:0/3569971302 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bfc19c930 con 0x7f6bfc106470 2024-05-22T22:53:28.171 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.177+0000 7f6c01dea700 1 --2- 172.21.15.204:0/3569971302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bfc106470 0x7f6bfc19c3f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6bec005850 tx=0x7f6bec005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:53:28.171 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.177+0000 7f6bf2ffd700 1 -- 172.21.15.204:0/3569971302 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bec026070 con 0x7f6bfc106470 2024-05-22T22:53:28.171 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.177+0000 7f6c0404e700 1 -- 172.21.15.204:0/3569971302 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bfc19cc20 con 0x7f6bfc106470 2024-05-22T22:53:28.171 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.177+0000 7f6bf2ffd700 1 -- 172.21.15.204:0/3569971302 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bec00bbd0 con 0x7f6bfc106470 2024-05-22T22:53:28.171 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.177+0000 7f6bf2ffd700 1 -- 172.21.15.204:0/3569971302 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bec003970 con 0x7f6bfc106470 2024-05-22T22:53:28.172 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.177+0000 7f6c0404e700 1 -- 172.21.15.204:0/3569971302 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bfc1965b0 con 0x7f6bfc106470 2024-05-22T22:53:28.172 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.178+0000 7f6bf2ffd700 1 -- 172.21.15.204:0/3569971302 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6bec029070 con 0x7f6bfc106470 2024-05-22T22:53:28.173 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.178+0000 7f6bf2ffd700 1 --2- 172.21.15.204:0/3569971302 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f6be80390f0 0x7f6be803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:28.173 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.178+0000 7f6bf2ffd700 1 -- 172.21.15.204:0/3569971302 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6bec01e070 con 0x7f6bfc106470 2024-05-22T22:53:28.173 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.178+0000 7f6c0404e700 1 -- 172.21.15.204:0/3569971302 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bfc101b20 con 0x7f6bfc106470 2024-05-22T22:53:28.173 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.178+0000 7f6c015e9700 1 --2- 172.21.15.204:0/3569971302 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f6be80390f0 0x7f6be803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:28.173 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.179+0000 7f6c015e9700 1 --2- 172.21.15.204:0/3569971302 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f6be80390f0 0x7f6be803b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8006fd0 tx=0x7f6bf8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:53:28.176 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.184+0000 7f6bf2ffd700 1 -- 172.21.15.204:0/3569971302 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6bec030070 con 0x7f6bfc106470 2024-05-22T22:53:28.470 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.476+0000 7f6c0404e700 1 -- 172.21.15.204:0/3569971302 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6bfc0639c0 con 0x7f6bfc106470 2024-05-22T22:53:28.470 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.477+0000 7f6bf2ffd700 1 -- 172.21.15.204:0/3569971302 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f6bec013070 con 0x7f6bfc106470 2024-05-22T22:53:28.470 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-22T22:53:28.470 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":1,"fsid":"c1135a12-188d-11ef-bc9a-c7b262605968","modified":"2024-05-22T22:51:46.137964Z","created":"2024-05-22T22:51:46.137964Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:3300","nonce":0},{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-22T22:53:28.472 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.479+0000 7f6c0404e700 1 -- 172.21.15.204:0/3569971302 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f6be80390f0 msgr2=0x7f6be803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:28.472 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.479+0000 7f6c0404e700 1 --2- 172.21.15.204:0/3569971302 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f6be80390f0 0x7f6be803b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8006fd0 tx=0x7f6bf8006e40 comp rx=0 tx=0).stop 2024-05-22T22:53:28.472 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.479+0000 7f6c0404e700 1 -- 172.21.15.204:0/3569971302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bfc106470 msgr2=0x7f6bfc19c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:28.472 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.479+0000 7f6c0404e700 1 --2- 172.21.15.204:0/3569971302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bfc106470 0x7f6bfc19c3f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6bec005850 tx=0x7f6bec005030 comp rx=0 tx=0).stop 2024-05-22T22:53:28.472 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.479+0000 7f6c0404e700 1 -- 172.21.15.204:0/3569971302 shutdown_connections 2024-05-22T22:53:28.472 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.479+0000 7f6c0404e700 1 --2- 172.21.15.204:0/3569971302 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f6be80390f0 0x7f6be803b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:28.472 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.479+0000 7f6c0404e700 1 --2- 172.21.15.204:0/3569971302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bfc106470 0x7f6bfc19c3f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:28.473 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.479+0000 7f6c0404e700 1 -- 172.21.15.204:0/3569971302 >> 172.21.15.204:0/3569971302 conn(0x7f6bfc0773e0 msgr2=0x7f6bfc108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:53:28.473 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.479+0000 7f6c0404e700 1 -- 172.21.15.204:0/3569971302 shutdown_connections 2024-05-22T22:53:28.473 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:28.479+0000 7f6c0404e700 1 -- 172.21.15.204:0/3569971302 wait complete. 2024-05-22T22:53:28.473 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 1 2024-05-22T22:53:28.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:28 smithi084 ceph-mon[32186]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:30.223 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-22T22:53:30.223 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-22T22:53:30.466 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.c/config 2024-05-22T22:53:34.117 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-22T22:53:34.117 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: mon.a calling monitor election 2024-05-22T22:53:34.118 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-22T22:53:34.118 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:34.118 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-22T22:53:34.118 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-22T22:53:34.118 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: mon.c calling monitor election 2024-05-22T22:53:34.118 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:34.118 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-22T22:53:34.118 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-22T22:53:34.118 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:34.118 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-22T22:53:34.118 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-22T22:53:34.118 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: monmap epoch 2 2024-05-22T22:53:34.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: fsid c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:53:34.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: last_changed 2024-05-22T22:53:28.601321+0000 2024-05-22T22:53:34.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: created 2024-05-22T22:51:46.137964+0000 2024-05-22T22:53:34.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: min_mon_release 19 (squid) 2024-05-22T22:53:34.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: election_strategy: 1 2024-05-22T22:53:34.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: 0: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.a 2024-05-22T22:53:34.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-05-22T22:53:34.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: fsmap 2024-05-22T22:53:34.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: osdmap e4: 0 total, 0 up, 0 in 2024-05-22T22:53:34.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: mgrmap e13: x(active, since 48s) 2024-05-22T22:53:34.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: overall HEALTH_OK 2024-05-22T22:53:34.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:34.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:34.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:34.120 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-22T22:53:34.120 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:33 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:34.153 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:33 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:34.823 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.828+0000 7fec25b05700 1 -- 172.21.15.204:0/3244774708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec080035c0 msgr2=0x7fec08005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:34.823 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.828+0000 7fec25b05700 1 --2- 172.21.15.204:0/3244774708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec080035c0 0x7fec08005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fec20101620 tx=0x7fec1400fe60 comp rx=0 tx=0).stop 2024-05-22T22:53:34.823 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.829+0000 7fec25b05700 1 -- 172.21.15.204:0/3244774708 shutdown_connections 2024-05-22T22:53:34.823 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.829+0000 7fec25b05700 1 --2- 172.21.15.204:0/3244774708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec080035c0 0x7fec08005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:34.823 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.829+0000 7fec25b05700 1 -- 172.21.15.204:0/3244774708 >> 172.21.15.204:0/3244774708 conn(0x7fec20077240 msgr2=0x7fec20077650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:53:34.823 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.829+0000 7fec25b05700 1 -- 172.21.15.204:0/3244774708 shutdown_connections 2024-05-22T22:53:34.823 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.829+0000 7fec25b05700 1 -- 172.21.15.204:0/3244774708 wait complete. 2024-05-22T22:53:34.824 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.829+0000 7fec25b05700 1 Processor -- start 2024-05-22T22:53:34.824 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.830+0000 7fec25b05700 1 -- start start 2024-05-22T22:53:34.824 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.830+0000 7fec25b05700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec080035c0 0x7fec20101770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:34.825 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.830+0000 7fec25b05700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec20101cb0 0x7fec20102130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:34.825 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.830+0000 7fec25b05700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fec20109f40 con 0x7fec20101cb0 2024-05-22T22:53:34.825 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.830+0000 7fec25b05700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fec20109dc0 con 0x7fec080035c0 2024-05-22T22:53:34.825 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.830+0000 7fec1f7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec080035c0 0x7fec20101770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:34.825 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.830+0000 7fec1f7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec080035c0 0x7fec20101770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:47570/0 (socket says 172.21.15.204:47570) 2024-05-22T22:53:34.825 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.830+0000 7fec1f7fe700 1 -- 172.21.15.204:0/2480349898 learned_addr learned my addr 172.21.15.204:0/2480349898 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-22T22:53:34.825 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.830+0000 7fec1f7fe700 1 -- 172.21.15.204:0/2480349898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec080035c0 msgr2=0x7fec20101770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-22T22:53:34.825 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.830+0000 7fec1f7fe700 1 -- 172.21.15.204:0/2480349898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec080035c0 msgr2=0x7fec20101770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-22T22:53:34.825 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.830+0000 7fec1f7fe700 1 --2- 172.21.15.204:0/2480349898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec080035c0 0x7fec20101770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-22T22:53:34.825 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.830+0000 7fec1effd700 1 --2- 172.21.15.204:0/2480349898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec20101cb0 0x7fec20102130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:34.825 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.830+0000 7fec1f7fe700 1 --2- 172.21.15.204:0/2480349898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec080035c0 0x7fec20101770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-22T22:53:34.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.831+0000 7fec1effd700 1 -- 172.21.15.204:0/2480349898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec080035c0 msgr2=0x7fec20101770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-22T22:53:34.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.831+0000 7fec1effd700 1 --2- 172.21.15.204:0/2480349898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec080035c0 0x7fec20101770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:34.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.831+0000 7fec1effd700 1 -- 172.21.15.204:0/2480349898 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec20103690 con 0x7fec20101cb0 2024-05-22T22:53:34.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.831+0000 7fec1effd700 1 --2- 172.21.15.204:0/2480349898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec20101cb0 0x7fec20102130 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fec1000b4d0 tx=0x7fec1000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:53:34.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.831+0000 7fec1cff9700 1 -- 172.21.15.204:0/2480349898 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fec10011630 con 0x7fec20101cb0 2024-05-22T22:53:34.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.831+0000 7fec1cff9700 1 -- 172.21.15.204:0/2480349898 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec10011cb0 con 0x7fec20101cb0 2024-05-22T22:53:34.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.831+0000 7fec25b05700 1 -- 172.21.15.204:0/2480349898 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec201a3d00 con 0x7fec20101cb0 2024-05-22T22:53:34.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.831+0000 7fec1cff9700 1 -- 172.21.15.204:0/2480349898 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fec10010e60 con 0x7fec20101cb0 2024-05-22T22:53:34.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.831+0000 7fec25b05700 1 -- 172.21.15.204:0/2480349898 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec201a4240 con 0x7fec20101cb0 2024-05-22T22:53:34.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.832+0000 7fec1cff9700 1 -- 172.21.15.204:0/2480349898 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fec10018020 con 0x7fec20101cb0 2024-05-22T22:53:34.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.832+0000 7fec25b05700 1 -- 172.21.15.204:0/2480349898 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec2006a280 con 0x7fec20101cb0 2024-05-22T22:53:34.828 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.833+0000 7fec1cff9700 1 --2- 172.21.15.204:0/2480349898 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fec0c038e30 0x7fec0c03b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:34.828 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.833+0000 7fec1cff9700 1 -- 172.21.15.204:0/2480349898 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fec1004bc60 con 0x7fec20101cb0 2024-05-22T22:53:34.828 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.833+0000 7fec1f7fe700 1 --2- 172.21.15.204:0/2480349898 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fec0c038e30 0x7fec0c03b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:34.828 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.833+0000 7fec1f7fe700 1 --2- 172.21.15.204:0/2480349898 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fec0c038e30 0x7fec0c03b2f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fec1400fa10 tx=0x7fec14005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:53:34.832 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:34.839+0000 7fec1cff9700 1 -- 172.21.15.204:0/2480349898 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fec1001f020 con 0x7fec20101cb0 2024-05-22T22:53:35.111 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:35.117+0000 7fec25b05700 1 -- 172.21.15.204:0/2480349898 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fec2002dbc0 con 0x7fec20101cb0 2024-05-22T22:53:35.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:34 smithi204 ceph-mon[36211]: Deploying daemon mon.b on smithi096 2024-05-22T22:53:35.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:34 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-22T22:53:35.112 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:35.118+0000 7fec1cff9700 1 -- 172.21.15.204:0/2480349898 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fec10018310 con 0x7fec20101cb0 2024-05-22T22:53:35.113 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-22T22:53:35.113 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":2,"fsid":"c1135a12-188d-11ef-bc9a-c7b262605968","modified":"2024-05-22T22:53:28.601321Z","created":"2024-05-22T22:51:46.137964Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:3300","nonce":0},{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-22T22:53:35.115 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:35.120+0000 7fec25b05700 1 -- 172.21.15.204:0/2480349898 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fec0c038e30 msgr2=0x7fec0c03b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:35.115 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:35.120+0000 7fec25b05700 1 --2- 172.21.15.204:0/2480349898 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fec0c038e30 0x7fec0c03b2f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fec1400fa10 tx=0x7fec14005e30 comp rx=0 tx=0).stop 2024-05-22T22:53:35.115 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:35.120+0000 7fec25b05700 1 -- 172.21.15.204:0/2480349898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec20101cb0 msgr2=0x7fec20102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:35.115 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:35.120+0000 7fec25b05700 1 --2- 172.21.15.204:0/2480349898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec20101cb0 0x7fec20102130 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fec1000b4d0 tx=0x7fec1000b890 comp rx=0 tx=0).stop 2024-05-22T22:53:35.115 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:35.121+0000 7fec25b05700 1 -- 172.21.15.204:0/2480349898 shutdown_connections 2024-05-22T22:53:35.115 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:35.121+0000 7fec25b05700 1 --2- 172.21.15.204:0/2480349898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec080035c0 0x7fec20101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:35.115 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:35.121+0000 7fec25b05700 1 --2- 172.21.15.204:0/2480349898 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fec0c038e30 0x7fec0c03b2f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:35.115 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:35.121+0000 7fec25b05700 1 --2- 172.21.15.204:0/2480349898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec20101cb0 0x7fec20102130 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:35.115 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:35.121+0000 7fec25b05700 1 -- 172.21.15.204:0/2480349898 >> 172.21.15.204:0/2480349898 conn(0x7fec20077240 msgr2=0x7fec200fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:53:35.115 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:35.121+0000 7fec25b05700 1 -- 172.21.15.204:0/2480349898 shutdown_connections 2024-05-22T22:53:35.115 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:35.121+0000 7fec25b05700 1 -- 172.21.15.204:0/2480349898 wait complete. 2024-05-22T22:53:35.116 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 2 2024-05-22T22:53:35.116 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:34 smithi084 ceph-mon[32186]: Deploying daemon mon.b on smithi096 2024-05-22T22:53:35.116 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:34 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-22T22:53:35.902 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:35 smithi204 ceph-mon[36211]: from='client.? 172.21.15.204:0/2480349898' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-22T22:53:35.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:35 smithi204 ceph-mon[36211]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:36.117 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:35 smithi084 ceph-mon[32186]: from='client.? 172.21.15.204:0/2480349898' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-22T22:53:36.117 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:35 smithi084 ceph-mon[32186]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:36.630 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-22T22:53:36.630 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-22T22:53:36.868 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.c/config 2024-05-22T22:53:37.541 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:37 smithi096 systemd[1]: Started Ceph mon.b for c1135a12-188d-11ef-bc9a-c7b262605968. 2024-05-22T22:53:42.721 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.728+0000 7f19c1dd0700 1 -- 172.21.15.204:0/2253875678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19a80035c0 msgr2=0x7f19a8005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:42.722 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.728+0000 7f19c1dd0700 1 --2- 172.21.15.204:0/2253875678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19a80035c0 0x7f19a8005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f19bc101da0 tx=0x7f19b4009e80 comp rx=0 tx=0).stop 2024-05-22T22:53:42.722 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.728+0000 7f19c1dd0700 1 -- 172.21.15.204:0/2253875678 shutdown_connections 2024-05-22T22:53:42.722 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.728+0000 7f19c1dd0700 1 --2- 172.21.15.204:0/2253875678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19a80035c0 0x7f19a8005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:42.722 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.728+0000 7f19c1dd0700 1 -- 172.21.15.204:0/2253875678 >> 172.21.15.204:0/2253875678 conn(0x7f19bc0fd980 msgr2=0x7f19bc0ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:53:42.722 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.728+0000 7f19c1dd0700 1 -- 172.21.15.204:0/2253875678 shutdown_connections 2024-05-22T22:53:42.722 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.728+0000 7f19c1dd0700 1 -- 172.21.15.204:0/2253875678 wait complete. 2024-05-22T22:53:42.723 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.729+0000 7f19c1dd0700 1 Processor -- start 2024-05-22T22:53:42.724 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.729+0000 7f19c1dd0700 1 -- start start 2024-05-22T22:53:42.724 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.730+0000 7f19c1dd0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19a80035c0 0x7f19bc19d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:42.724 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.730+0000 7f19c1dd0700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19bc19dc30 0x7f19bc1978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:42.724 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.730+0000 7f19c1dd0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19bc197df0 0x7f19bc1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:42.724 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.730+0000 7f19c1dd0700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f19bc109eb0 con 0x7f19bc19dc30 2024-05-22T22:53:42.724 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.730+0000 7f19c1dd0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f19bc109d30 con 0x7f19a80035c0 2024-05-22T22:53:42.724 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.730+0000 7f19c1dd0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f19bc10a030 con 0x7f19bc197df0 2024-05-22T22:53:42.724 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.730+0000 7f19bbfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19bc197df0 0x7f19bc1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:42.724 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.730+0000 7f19bbfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19bc197df0 0x7f19bc1982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:41834/0 (socket says 172.21.15.204:41834) 2024-05-22T22:53:42.724 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.730+0000 7f19bbfff700 1 -- 172.21.15.204:0/801022987 learned_addr learned my addr 172.21.15.204:0/801022987 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-22T22:53:42.724 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.730+0000 7f19baffd700 1 --2- 172.21.15.204:0/801022987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19bc19dc30 0x7f19bc1978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:42.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.730+0000 7f19bbfff700 1 -- 172.21.15.204:0/801022987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19bc197df0 msgr2=0x7f19bc1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-22T22:53:42.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.730+0000 7f19bbfff700 1 -- 172.21.15.204:0/801022987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19bc197df0 msgr2=0x7f19bc1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-22T22:53:42.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.730+0000 7f19bbfff700 1 --2- 172.21.15.204:0/801022987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19bc197df0 0x7f19bc1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-22T22:53:42.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.730+0000 7f19bb7fe700 1 --2- 172.21.15.204:0/801022987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19a80035c0 0x7f19bc19d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:42.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.730+0000 7f19bbfff700 1 --2- 172.21.15.204:0/801022987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19bc197df0 0x7f19bc1982a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-22T22:53:42.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.730+0000 7f19bb7fe700 1 -- 172.21.15.204:0/801022987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19a80035c0 msgr2=0x7f19bc19d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-22T22:53:42.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.730+0000 7f19bb7fe700 1 -- 172.21.15.204:0/801022987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19a80035c0 msgr2=0x7f19bc19d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-22T22:53:42.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.730+0000 7f19bb7fe700 1 --2- 172.21.15.204:0/801022987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19a80035c0 0x7f19bc19d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-22T22:53:42.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.730+0000 7f19baffd700 1 -- 172.21.15.204:0/801022987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19bc197df0 msgr2=0x7f19bc1982a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-22T22:53:42.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.730+0000 7f19baffd700 1 --2- 172.21.15.204:0/801022987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19bc197df0 0x7f19bc1982a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:42.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.730+0000 7f19baffd700 1 -- 172.21.15.204:0/801022987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19a80035c0 msgr2=0x7f19bc19d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:42.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.730+0000 7f19bb7fe700 1 --2- 172.21.15.204:0/801022987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19a80035c0 0x7f19bc19d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-22T22:53:42.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.730+0000 7f19baffd700 1 --2- 172.21.15.204:0/801022987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19a80035c0 0x7f19bc19d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:42.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.730+0000 7f19baffd700 1 -- 172.21.15.204:0/801022987 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19bc1a45c0 con 0x7f19bc19dc30 2024-05-22T22:53:42.726 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.731+0000 7f19baffd700 1 --2- 172.21.15.204:0/801022987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19bc19dc30 0x7f19bc1978b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f19a400b550 tx=0x7f19a400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:53:42.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.731+0000 7f19b8ff9700 1 -- 172.21.15.204:0/801022987 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a4004230 con 0x7f19bc19dc30 2024-05-22T22:53:42.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.731+0000 7f19b8ff9700 1 -- 172.21.15.204:0/801022987 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19a40043d0 con 0x7f19bc19dc30 2024-05-22T22:53:42.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.731+0000 7f19c1dd0700 1 -- 172.21.15.204:0/801022987 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19bc1a47f0 con 0x7f19bc19dc30 2024-05-22T22:53:42.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.731+0000 7f19b8ff9700 1 -- 172.21.15.204:0/801022987 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a4011c20 con 0x7f19bc19dc30 2024-05-22T22:53:42.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.731+0000 7f19c1dd0700 1 -- 172.21.15.204:0/801022987 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19bc1a4d80 con 0x7f19bc19dc30 2024-05-22T22:53:42.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.732+0000 7f19b8ff9700 1 -- 172.21.15.204:0/801022987 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f19a401a400 con 0x7f19bc19dc30 2024-05-22T22:53:42.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.732+0000 7f19b8ff9700 1 --2- 172.21.15.204:0/801022987 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f199c039580 0x7f199c03ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:42.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.732+0000 7f19b8ff9700 1 -- 172.21.15.204:0/801022987 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f19a404bd70 con 0x7f19bc19dc30 2024-05-22T22:53:42.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.732+0000 7f19c1dd0700 1 -- 172.21.15.204:0/801022987 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19bc0639c0 con 0x7f19bc19dc30 2024-05-22T22:53:42.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.733+0000 7f19bb7fe700 1 --2- 172.21.15.204:0/801022987 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f199c039580 0x7f199c03ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:42.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.733+0000 7f19bb7fe700 1 --2- 172.21.15.204:0/801022987 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f199c039580 0x7f199c03ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f19b4009160 tx=0x7f19b400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:53:42.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:42.738+0000 7f19b8ff9700 1 -- 172.21.15.204:0/801022987 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f19a40151c0 con 0x7f19bc19dc30 2024-05-22T22:53:42.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:42.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-22T22:53:42.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: mon.c calling monitor election 2024-05-22T22:53:42.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: mon.a calling monitor election 2024-05-22T22:53:42.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:53:42.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-22T22:53:42.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:53:42.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:42.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:53:42.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:53:42.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:42.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:53:42.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:53:42.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-22T22:53:42.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: monmap epoch 3 2024-05-22T22:53:42.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: fsid c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:53:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: last_changed 2024-05-22T22:53:37.518995+0000 2024-05-22T22:53:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: created 2024-05-22T22:51:46.137964+0000 2024-05-22T22:53:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: min_mon_release 19 (squid) 2024-05-22T22:53:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: election_strategy: 1 2024-05-22T22:53:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: 0: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.a 2024-05-22T22:53:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-05-22T22:53:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-05-22T22:53:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: fsmap 2024-05-22T22:53:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: osdmap e4: 0 total, 0 up, 0 in 2024-05-22T22:53:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: mgrmap e13: x(active, since 57s) 2024-05-22T22:53:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: overall HEALTH_OK 2024-05-22T22:53:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:42 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:42.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:42.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-22T22:53:42.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: mon.c calling monitor election 2024-05-22T22:53:42.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: mon.a calling monitor election 2024-05-22T22:53:42.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:53:42.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-22T22:53:42.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:53:42.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:42.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:53:42.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:53:42.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:42.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:53:42.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:53:42.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-22T22:53:42.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: monmap epoch 3 2024-05-22T22:53:42.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: fsid c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:53:42.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: last_changed 2024-05-22T22:53:37.518995+0000 2024-05-22T22:53:42.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: created 2024-05-22T22:51:46.137964+0000 2024-05-22T22:53:42.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: min_mon_release 19 (squid) 2024-05-22T22:53:42.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: election_strategy: 1 2024-05-22T22:53:42.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: 0: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.a 2024-05-22T22:53:42.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-05-22T22:53:42.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-05-22T22:53:42.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: fsmap 2024-05-22T22:53:42.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: osdmap e4: 0 total, 0 up, 0 in 2024-05-22T22:53:42.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: mgrmap e13: x(active, since 57s) 2024-05-22T22:53:42.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: overall HEALTH_OK 2024-05-22T22:53:42.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:42.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:42 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:43.022 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:43.029+0000 7f19c1dd0700 1 -- 172.21.15.204:0/801022987 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f19bc1a5090 con 0x7f19bc19dc30 2024-05-22T22:53:43.023 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:43.029+0000 7f19b8ff9700 1 -- 172.21.15.204:0/801022987 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f19a4003c80 con 0x7f19bc19dc30 2024-05-22T22:53:43.023 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-22T22:53:43.024 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":3,"fsid":"c1135a12-188d-11ef-bc9a-c7b262605968","modified":"2024-05-22T22:53:37.518995Z","created":"2024-05-22T22:51:46.137964Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:3300","nonce":0},{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:3300","nonce":0},{"type":"v1","addr":"172.21.15.96:6789","nonce":0}]},"addr":"172.21.15.96:6789/0","public_addr":"172.21.15.96:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-22T22:53:43.026 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:43.032+0000 7f19c1dd0700 1 -- 172.21.15.204:0/801022987 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f199c039580 msgr2=0x7f199c03ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:43.026 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:43.032+0000 7f19c1dd0700 1 --2- 172.21.15.204:0/801022987 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f199c039580 0x7f199c03ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f19b4009160 tx=0x7f19b400b040 comp rx=0 tx=0).stop 2024-05-22T22:53:43.027 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:43.032+0000 7f19c1dd0700 1 -- 172.21.15.204:0/801022987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19bc19dc30 msgr2=0x7f19bc1978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:43.027 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:43.032+0000 7f19c1dd0700 1 --2- 172.21.15.204:0/801022987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19bc19dc30 0x7f19bc1978b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f19a400b550 tx=0x7f19a400b910 comp rx=0 tx=0).stop 2024-05-22T22:53:43.027 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:43.032+0000 7f19c1dd0700 1 -- 172.21.15.204:0/801022987 shutdown_connections 2024-05-22T22:53:43.027 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:43.032+0000 7f19c1dd0700 1 --2- 172.21.15.204:0/801022987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19bc197df0 0x7f19bc1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:43.027 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:43.032+0000 7f19c1dd0700 1 --2- 172.21.15.204:0/801022987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19a80035c0 0x7f19bc19d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:43.027 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:43.032+0000 7f19c1dd0700 1 --2- 172.21.15.204:0/801022987 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f199c039580 0x7f199c03ba40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:43.028 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:43.032+0000 7f19c1dd0700 1 --2- 172.21.15.204:0/801022987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19bc19dc30 0x7f19bc1978b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:43.028 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:43.032+0000 7f19c1dd0700 1 -- 172.21.15.204:0/801022987 >> 172.21.15.204:0/801022987 conn(0x7f19bc0fd980 msgr2=0x7f19bc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:53:43.028 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:43.032+0000 7f19c1dd0700 1 -- 172.21.15.204:0/801022987 shutdown_connections 2024-05-22T22:53:43.028 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:43.032+0000 7f19c1dd0700 1 -- 172.21.15.204:0/801022987 wait complete. 2024-05-22T22:53:43.028 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 3 2024-05-22T22:53:43.565 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-22T22:53:43.565 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph config generate-minimal-conf 2024-05-22T22:53:43.814 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T22:53:43.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:43 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:43.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:43 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:43.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:43 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:53:43.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:43 smithi084 ceph-mon[32186]: from='client.? 172.21.15.204:0/801022987' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-22T22:53:43.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:43 smithi084 ceph-mon[32186]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:43.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:43 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:53:43.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:43 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:43.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:43 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:43.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:43 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:53:43.874 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:43 smithi204 ceph-mon[36211]: from='client.? 172.21.15.204:0/801022987' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-22T22:53:43.874 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:43 smithi204 ceph-mon[36211]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:43.874 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:43 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:53:44.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:44.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-22T22:53:44.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: mon.c calling monitor election 2024-05-22T22:53:44.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: mon.a calling monitor election 2024-05-22T22:53:44.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:53:44.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-22T22:53:44.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:53:44.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:44.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:53:44.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:53:44.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:44.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:53:44.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:53:44.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-22T22:53:44.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: monmap epoch 3 2024-05-22T22:53:44.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: fsid c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:53:44.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: last_changed 2024-05-22T22:53:37.518995+0000 2024-05-22T22:53:44.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: created 2024-05-22T22:51:46.137964+0000 2024-05-22T22:53:44.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: min_mon_release 19 (squid) 2024-05-22T22:53:44.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: election_strategy: 1 2024-05-22T22:53:44.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: 0: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.a 2024-05-22T22:53:44.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-05-22T22:53:44.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-05-22T22:53:44.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: fsmap 2024-05-22T22:53:44.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: osdmap e4: 0 total, 0 up, 0 in 2024-05-22T22:53:44.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: mgrmap e13: x(active, since 57s) 2024-05-22T22:53:44.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: overall HEALTH_OK 2024-05-22T22:53:44.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:44.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:44.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:44.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:44.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:53:44.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: from='client.? 172.21.15.204:0/801022987' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-22T22:53:44.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:44.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:53:45.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.274+0000 7fa7b1912700 1 -- 172.21.15.84:0/521522537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7ac100f00 msgr2=0x7fa7ac1012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:45.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.274+0000 7fa7b1912700 1 --2- 172.21.15.84:0/521522537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7ac100f00 0x7fa7ac1012e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009910 tx=0x7fa7a0009c20 comp rx=0 tx=0).stop 2024-05-22T22:53:45.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.274+0000 7fa7b1912700 1 -- 172.21.15.84:0/521522537 shutdown_connections 2024-05-22T22:53:45.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.274+0000 7fa7b1912700 1 --2- 172.21.15.84:0/521522537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7ac100f00 0x7fa7ac1012e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:45.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.274+0000 7fa7b1912700 1 -- 172.21.15.84:0/521522537 >> 172.21.15.84:0/521522537 conn(0x7fa7ac0fc7f0 msgr2=0x7fa7ac0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:53:45.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.274+0000 7fa7b1912700 1 -- 172.21.15.84:0/521522537 shutdown_connections 2024-05-22T22:53:45.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.274+0000 7fa7b1912700 1 -- 172.21.15.84:0/521522537 wait complete. 2024-05-22T22:53:45.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.275+0000 7fa7b1912700 1 Processor -- start 2024-05-22T22:53:45.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.275+0000 7fa7b1912700 1 -- start start 2024-05-22T22:53:45.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.275+0000 7fa7b1912700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7ac100f00 0x7fa7ac19c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:45.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.276+0000 7fa7b1912700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7ac19cbf0 0x7fa7ac196830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:45.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.276+0000 7fa7b1912700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7ac196d70 0x7fa7ac197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:45.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.276+0000 7fa7b1912700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac10b290 con 0x7fa7ac19cbf0 2024-05-22T22:53:45.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.276+0000 7fa7b1912700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac10b110 con 0x7fa7ac100f00 2024-05-22T22:53:45.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.276+0000 7fa7b1912700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac10b410 con 0x7fa7ac196d70 2024-05-22T22:53:45.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.276+0000 7fa7aa7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7ac19cbf0 0x7fa7ac196830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:45.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.276+0000 7fa7aa7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7ac19cbf0 0x7fa7ac196830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43056/0 (socket says 172.21.15.84:43056) 2024-05-22T22:53:45.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.276+0000 7fa7aa7fc700 1 -- 172.21.15.84:0/2338765137 learned_addr learned my addr 172.21.15.84:0/2338765137 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:53:45.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.276+0000 7fa7ab7fe700 1 --2- 172.21.15.84:0/2338765137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7ac196d70 0x7fa7ac197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:45.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.276+0000 7fa7aaffd700 1 --2- 172.21.15.84:0/2338765137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7ac100f00 0x7fa7ac19c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:45.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.276+0000 7fa7aa7fc700 1 -- 172.21.15.84:0/2338765137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7ac196d70 msgr2=0x7fa7ac197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:45.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.276+0000 7fa7aa7fc700 1 --2- 172.21.15.84:0/2338765137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7ac196d70 0x7fa7ac197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:45.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.276+0000 7fa7aa7fc700 1 -- 172.21.15.84:0/2338765137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7ac100f00 msgr2=0x7fa7ac19c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:45.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.276+0000 7fa7aa7fc700 1 --2- 172.21.15.84:0/2338765137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7ac100f00 0x7fa7ac19c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:45.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.276+0000 7fa7aa7fc700 1 -- 172.21.15.84:0/2338765137 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7ac1a3510 con 0x7fa7ac19cbf0 2024-05-22T22:53:45.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.276+0000 7fa7aaffd700 1 --2- 172.21.15.84:0/2338765137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7ac100f00 0x7fa7ac19c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T22:53:45.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.276+0000 7fa7aa7fc700 1 --2- 172.21.15.84:0/2338765137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7ac19cbf0 0x7fa7ac196830 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa79c00b780 tx=0x7fa79c00bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:53:45.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.276+0000 7fa7b0910700 1 -- 172.21.15.84:0/2338765137 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c004230 con 0x7fa7ac19cbf0 2024-05-22T22:53:45.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.276+0000 7fa7b0910700 1 -- 172.21.15.84:0/2338765137 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa79c0043d0 con 0x7fa7ac19cbf0 2024-05-22T22:53:45.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.276+0000 7fa7b1912700 1 -- 172.21.15.84:0/2338765137 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7ac1a37a0 con 0x7fa7ac19cbf0 2024-05-22T22:53:45.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.276+0000 7fa7b0910700 1 -- 172.21.15.84:0/2338765137 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c011c60 con 0x7fa7ac19cbf0 2024-05-22T22:53:45.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.276+0000 7fa7b1912700 1 -- 172.21.15.84:0/2338765137 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7ac1a3d00 con 0x7fa7ac19cbf0 2024-05-22T22:53:45.273 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.277+0000 7fa7b0910700 1 -- 172.21.15.84:0/2338765137 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa79c01a400 con 0x7fa7ac19cbf0 2024-05-22T22:53:45.273 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.278+0000 7fa7b0910700 1 --2- 172.21.15.84:0/2338765137 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa794039580 0x7fa79403ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:45.273 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.278+0000 7fa7b0910700 1 -- 172.21.15.84:0/2338765137 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa79c04bde0 con 0x7fa7ac19cbf0 2024-05-22T22:53:45.273 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.278+0000 7fa7aaffd700 1 --2- 172.21.15.84:0/2338765137 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa794039580 0x7fa79403ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:45.273 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.278+0000 7fa7b1912700 1 -- 172.21.15.84:0/2338765137 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7ac1023e0 con 0x7fa7ac19cbf0 2024-05-22T22:53:45.273 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.278+0000 7fa7aaffd700 1 --2- 172.21.15.84:0/2338765137 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa794039580 0x7fa79403ba40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009750 tx=0x7fa7a0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:53:45.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.283+0000 7fa7b0910700 1 -- 172.21.15.84:0/2338765137 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa79c018510 con 0x7fa7ac19cbf0 2024-05-22T22:53:45.367 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: mon.b calling monitor election 2024-05-22T22:53:45.367 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: Updating smithi084:/etc/ceph/ceph.conf 2024-05-22T22:53:45.368 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: Updating smithi096:/etc/ceph/ceph.conf 2024-05-22T22:53:45.368 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: Updating smithi204:/etc/ceph/ceph.conf 2024-05-22T22:53:45.368 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: Updating smithi096:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-22T22:53:45.368 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: Updating smithi084:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-22T22:53:45.368 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: Updating smithi204:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-22T22:53:45.368 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: mon.b calling monitor election 2024-05-22T22:53:45.368 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: mon.c calling monitor election 2024-05-22T22:53:45.368 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: mon.a calling monitor election 2024-05-22T22:53:45.368 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-22T22:53:45.368 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: monmap epoch 3 2024-05-22T22:53:45.368 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: fsid c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:53:45.368 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: last_changed 2024-05-22T22:53:37.518995+0000 2024-05-22T22:53:45.368 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: created 2024-05-22T22:51:46.137964+0000 2024-05-22T22:53:45.369 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: min_mon_release 19 (squid) 2024-05-22T22:53:45.369 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: election_strategy: 1 2024-05-22T22:53:45.369 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: 0: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.a 2024-05-22T22:53:45.369 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-05-22T22:53:45.369 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-05-22T22:53:45.369 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: fsmap 2024-05-22T22:53:45.369 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: osdmap e4: 0 total, 0 up, 0 in 2024-05-22T22:53:45.369 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: mgrmap e13: x(active, since 59s) 2024-05-22T22:53:45.369 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: overall HEALTH_OK 2024-05-22T22:53:45.369 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.369 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.369 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.369 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-22T22:53:45.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-22T22:53:45.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:45.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.504+0000 7fa7b1912700 1 -- 172.21.15.84:0/2338765137 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fa7ac1a3fc0 con 0x7fa7ac19cbf0 2024-05-22T22:53:45.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.504+0000 7fa7b0910700 1 -- 172.21.15.84:0/2338765137 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fa79c018770 con 0x7fa7ac19cbf0 2024-05-22T22:53:45.498 INFO:teuthology.orchestra.run.smithi084.stdout:# minimal ceph.conf for c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:53:45.498 INFO:teuthology.orchestra.run.smithi084.stdout:[global] 2024-05-22T22:53:45.499 INFO:teuthology.orchestra.run.smithi084.stdout: fsid = c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:53:45.499 INFO:teuthology.orchestra.run.smithi084.stdout: mon_host = [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-22T22:53:45.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.507+0000 7fa7b1912700 1 -- 172.21.15.84:0/2338765137 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa794039580 msgr2=0x7fa79403ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:45.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.507+0000 7fa7b1912700 1 --2- 172.21.15.84:0/2338765137 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa794039580 0x7fa79403ba40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009750 tx=0x7fa7a0005fd0 comp rx=0 tx=0).stop 2024-05-22T22:53:45.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.507+0000 7fa7b1912700 1 -- 172.21.15.84:0/2338765137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7ac19cbf0 msgr2=0x7fa7ac196830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:45.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.507+0000 7fa7b1912700 1 --2- 172.21.15.84:0/2338765137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7ac19cbf0 0x7fa7ac196830 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa79c00b780 tx=0x7fa79c00bb40 comp rx=0 tx=0).stop 2024-05-22T22:53:45.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.507+0000 7fa7b1912700 1 -- 172.21.15.84:0/2338765137 shutdown_connections 2024-05-22T22:53:45.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.507+0000 7fa7b1912700 1 --2- 172.21.15.84:0/2338765137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7ac196d70 0x7fa7ac197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:45.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.507+0000 7fa7b1912700 1 --2- 172.21.15.84:0/2338765137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7ac100f00 0x7fa7ac19c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:45.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.507+0000 7fa7b1912700 1 --2- 172.21.15.84:0/2338765137 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa794039580 0x7fa79403ba40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:45.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.507+0000 7fa7b1912700 1 --2- 172.21.15.84:0/2338765137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7ac19cbf0 0x7fa7ac196830 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:45.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.507+0000 7fa7b1912700 1 -- 172.21.15.84:0/2338765137 >> 172.21.15.84:0/2338765137 conn(0x7fa7ac0fc7f0 msgr2=0x7fa7ac1028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:53:45.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.507+0000 7fa7b1912700 1 -- 172.21.15.84:0/2338765137 shutdown_connections 2024-05-22T22:53:45.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:45.507+0000 7fa7b1912700 1 -- 172.21.15.84:0/2338765137 wait complete. 2024-05-22T22:53:45.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: mon.b calling monitor election 2024-05-22T22:53:45.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: Updating smithi084:/etc/ceph/ceph.conf 2024-05-22T22:53:45.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: Updating smithi096:/etc/ceph/ceph.conf 2024-05-22T22:53:45.543 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: Updating smithi204:/etc/ceph/ceph.conf 2024-05-22T22:53:45.543 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: Updating smithi096:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-22T22:53:45.543 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: Updating smithi084:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-22T22:53:45.543 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: Updating smithi204:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-22T22:53:45.543 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: mon.b calling monitor election 2024-05-22T22:53:45.543 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: mon.c calling monitor election 2024-05-22T22:53:45.543 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: mon.a calling monitor election 2024-05-22T22:53:45.543 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-22T22:53:45.543 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: monmap epoch 3 2024-05-22T22:53:45.543 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: fsid c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:53:45.543 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: last_changed 2024-05-22T22:53:37.518995+0000 2024-05-22T22:53:45.543 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: created 2024-05-22T22:51:46.137964+0000 2024-05-22T22:53:45.543 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: min_mon_release 19 (squid) 2024-05-22T22:53:45.544 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: election_strategy: 1 2024-05-22T22:53:45.544 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: 0: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.a 2024-05-22T22:53:45.544 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-05-22T22:53:45.544 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-05-22T22:53:45.544 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: fsmap 2024-05-22T22:53:45.544 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: osdmap e4: 0 total, 0 up, 0 in 2024-05-22T22:53:45.544 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: mgrmap e13: x(active, since 59s) 2024-05-22T22:53:45.544 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: overall HEALTH_OK 2024-05-22T22:53:45.544 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.544 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.544 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.544 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.544 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-22T22:53:45.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-22T22:53:45.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:45.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: mon.b calling monitor election 2024-05-22T22:53:45.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: Updating smithi084:/etc/ceph/ceph.conf 2024-05-22T22:53:45.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: Updating smithi096:/etc/ceph/ceph.conf 2024-05-22T22:53:45.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: Updating smithi204:/etc/ceph/ceph.conf 2024-05-22T22:53:45.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: Updating smithi096:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-22T22:53:45.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: Updating smithi084:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-22T22:53:45.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: Updating smithi204:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-22T22:53:45.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: mon.b calling monitor election 2024-05-22T22:53:45.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: mon.c calling monitor election 2024-05-22T22:53:45.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: mon.a calling monitor election 2024-05-22T22:53:45.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-22T22:53:45.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: monmap epoch 3 2024-05-22T22:53:45.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: fsid c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T22:53:45.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: last_changed 2024-05-22T22:53:37.518995+0000 2024-05-22T22:53:45.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: created 2024-05-22T22:51:46.137964+0000 2024-05-22T22:53:45.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: min_mon_release 19 (squid) 2024-05-22T22:53:45.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: election_strategy: 1 2024-05-22T22:53:45.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: 0: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.a 2024-05-22T22:53:45.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-05-22T22:53:45.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-05-22T22:53:45.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: fsmap 2024-05-22T22:53:45.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: osdmap e4: 0 total, 0 up, 0 in 2024-05-22T22:53:45.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: mgrmap e13: x(active, since 59s) 2024-05-22T22:53:45.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: overall HEALTH_OK 2024-05-22T22:53:45.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:45.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-22T22:53:45.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-22T22:53:45.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:46.092 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-22T22:53:46.093 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-22T22:53:46.093 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/ceph.conf 2024-05-22T22:53:46.121 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-22T22:53:46.121 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-22T22:53:46.187 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-22T22:53:46.187 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.conf 2024-05-22T22:53:46.218 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-22T22:53:46.218 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-22T22:53:46.286 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-22T22:53:46.286 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.conf 2024-05-22T22:53:46.317 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-22T22:53:46.317 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-22T22:53:46.385 INFO:tasks.cephadm:Adding mgr.x on smithi084 2024-05-22T22:53:46.385 INFO:tasks.cephadm:Adding mgr.y on smithi096 2024-05-22T22:53:46.385 INFO:tasks.cephadm:Adding mgr.z on smithi204 2024-05-22T22:53:46.385 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph orch apply mgr '3;smithi084=x;smithi096=y;smithi204=z' 2024-05-22T22:53:46.541 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:46 smithi096 ceph-mon[36392]: Reconfiguring mon.a (unknown last config time)... 2024-05-22T22:53:46.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:46 smithi096 ceph-mon[36392]: Reconfiguring daemon mon.a on smithi084 2024-05-22T22:53:46.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:46 smithi096 ceph-mon[36392]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:46.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:46 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:46.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:46 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2338765137' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:46.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:46 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:53:46.544 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:46 smithi084 ceph-mon[32186]: Reconfiguring mon.a (unknown last config time)... 2024-05-22T22:53:46.544 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:46 smithi084 ceph-mon[32186]: Reconfiguring daemon mon.a on smithi084 2024-05-22T22:53:46.544 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:46 smithi084 ceph-mon[32186]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:46.545 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:46 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:46.545 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:46 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2338765137' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:46.545 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:46 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:53:46.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:46 smithi204 ceph-mon[36211]: Reconfiguring mon.a (unknown last config time)... 2024-05-22T22:53:46.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:46 smithi204 ceph-mon[36211]: Reconfiguring daemon mon.a on smithi084 2024-05-22T22:53:46.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:46 smithi204 ceph-mon[36211]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:46.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:46 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:46.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:46 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2338765137' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:46.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:46 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:53:46.659 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.c/config 2024-05-22T22:53:48.062 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.069+0000 7f70ef13f700 1 -- 172.21.15.204:0/4192229146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70e8101ea0 msgr2=0x7f70e81022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:48.062 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.069+0000 7f70ef13f700 1 --2- 172.21.15.204:0/4192229146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70e8101ea0 0x7f70e81022c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009910 tx=0x7f70dc009c20 comp rx=0 tx=0).stop 2024-05-22T22:53:48.063 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.069+0000 7f70ef13f700 1 -- 172.21.15.204:0/4192229146 shutdown_connections 2024-05-22T22:53:48.063 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.069+0000 7f70ef13f700 1 --2- 172.21.15.204:0/4192229146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70e8101ea0 0x7f70e81022c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:48.063 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.069+0000 7f70ef13f700 1 -- 172.21.15.204:0/4192229146 >> 172.21.15.204:0/4192229146 conn(0x7f70e80fd490 msgr2=0x7f70e80ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:53:48.063 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.069+0000 7f70ef13f700 1 -- 172.21.15.204:0/4192229146 shutdown_connections 2024-05-22T22:53:48.063 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.069+0000 7f70ef13f700 1 -- 172.21.15.204:0/4192229146 wait complete. 2024-05-22T22:53:48.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.070+0000 7f70ef13f700 1 Processor -- start 2024-05-22T22:53:48.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.070+0000 7f70ef13f700 1 -- start start 2024-05-22T22:53:48.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.070+0000 7f70ef13f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70e8101ea0 0x7f70e8076100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:48.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.070+0000 7f70ef13f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70e8074750 0x7f70e8074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:48.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.070+0000 7f70ef13f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70e8075110 0x7f70e81a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:48.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.070+0000 7f70ef13f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f70e8107ef0 con 0x7f70e8101ea0 2024-05-22T22:53:48.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.070+0000 7f70ef13f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f70e8107d70 con 0x7f70e8074750 2024-05-22T22:53:48.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.070+0000 7f70ef13f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f70e8108070 con 0x7f70e8075110 2024-05-22T22:53:48.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.070+0000 7f70ecedb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70e8101ea0 0x7f70e8076100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:48.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.070+0000 7f70ecedb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70e8101ea0 0x7f70e8076100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.204:38562/0 (socket says 172.21.15.204:38562) 2024-05-22T22:53:48.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.070+0000 7f70ecedb700 1 -- 172.21.15.204:0/1549815032 learned_addr learned my addr 172.21.15.204:0/1549815032 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-22T22:53:48.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.070+0000 7f70ed6dc700 1 --2- 172.21.15.204:0/1549815032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70e8075110 0x7f70e81a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:48.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.070+0000 7f70e7fff700 1 --2- 172.21.15.204:0/1549815032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70e8074750 0x7f70e8074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:48.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.070+0000 7f70ecedb700 1 -- 172.21.15.204:0/1549815032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70e8075110 msgr2=0x7f70e81a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:48.066 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.070+0000 7f70ecedb700 1 --2- 172.21.15.204:0/1549815032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70e8075110 0x7f70e81a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:48.066 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.070+0000 7f70ecedb700 1 -- 172.21.15.204:0/1549815032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70e8074750 msgr2=0x7f70e8074bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:48.066 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.070+0000 7f70ecedb700 1 --2- 172.21.15.204:0/1549815032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70e8074750 0x7f70e8074bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:48.066 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.070+0000 7f70ecedb700 1 -- 172.21.15.204:0/1549815032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e81a1d00 con 0x7f70e8101ea0 2024-05-22T22:53:48.066 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.070+0000 7f70e7fff700 1 --2- 172.21.15.204:0/1549815032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70e8074750 0x7f70e8074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T22:53:48.066 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.071+0000 7f70ecedb700 1 --2- 172.21.15.204:0/1549815032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70e8101ea0 0x7f70e8076100 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f70dc005e00 tx=0x7f70dc004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:53:48.066 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.071+0000 7f70e5ffb700 1 -- 172.21.15.204:0/1549815032 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc027070 con 0x7f70e8101ea0 2024-05-22T22:53:48.066 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.071+0000 7f70e5ffb700 1 -- 172.21.15.204:0/1549815032 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70dc00be40 con 0x7f70e8101ea0 2024-05-22T22:53:48.066 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.071+0000 7f70ef13f700 1 -- 172.21.15.204:0/1549815032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e81a1f90 con 0x7f70e8101ea0 2024-05-22T22:53:48.066 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.071+0000 7f70e5ffb700 1 -- 172.21.15.204:0/1549815032 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc00f460 con 0x7f70e8101ea0 2024-05-22T22:53:48.066 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.071+0000 7f70ef13f700 1 -- 172.21.15.204:0/1549815032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e81a2400 con 0x7f70e8101ea0 2024-05-22T22:53:48.067 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.072+0000 7f70ef13f700 1 -- 172.21.15.204:0/1549815032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e80639c0 con 0x7f70e8101ea0 2024-05-22T22:53:48.067 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.072+0000 7f70e5ffb700 1 -- 172.21.15.204:0/1549815032 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f70dc02a070 con 0x7f70e8101ea0 2024-05-22T22:53:48.067 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.072+0000 7f70e5ffb700 1 --2- 172.21.15.204:0/1549815032 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f70d8039580 0x7f70d803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:48.067 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.072+0000 7f70e5ffb700 1 -- 172.21.15.204:0/1549815032 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f70dc04d340 con 0x7f70e8101ea0 2024-05-22T22:53:48.067 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.072+0000 7f70e7fff700 1 --2- 172.21.15.204:0/1549815032 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f70d8039580 0x7f70d803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:48.067 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.073+0000 7f70e7fff700 1 --2- 172.21.15.204:0/1549815032 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f70d8039580 0x7f70d803ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f70d4009910 tx=0x7f70d4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:53:48.071 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.078+0000 7f70e5ffb700 1 -- 172.21.15.204:0/1549815032 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f70dc013070 con 0x7f70e8101ea0 2024-05-22T22:53:48.279 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.286+0000 7f70ef13f700 1 -- 172.21.15.204:0/1549815032 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi084=x;smithi096=y;smithi204=z", "target": ["mon-mgr", ""]}) -- 0x7f70e8062830 con 0x7f70d8039580 2024-05-22T22:53:48.285 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.292+0000 7f70e5ffb700 1 -- 172.21.15.204:0/1549815032 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f70e8062830 con 0x7f70d8039580 2024-05-22T22:53:48.285 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mgr update... 2024-05-22T22:53:48.289 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.294+0000 7f70ef13f700 1 -- 172.21.15.204:0/1549815032 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f70d8039580 msgr2=0x7f70d803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:48.290 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.294+0000 7f70ef13f700 1 --2- 172.21.15.204:0/1549815032 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f70d8039580 0x7f70d803ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f70d4009910 tx=0x7f70d4008040 comp rx=0 tx=0).stop 2024-05-22T22:53:48.290 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.294+0000 7f70ef13f700 1 -- 172.21.15.204:0/1549815032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70e8101ea0 msgr2=0x7f70e8076100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:48.290 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.294+0000 7f70ef13f700 1 --2- 172.21.15.204:0/1549815032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70e8101ea0 0x7f70e8076100 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f70dc005e00 tx=0x7f70dc004f80 comp rx=0 tx=0).stop 2024-05-22T22:53:48.290 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.294+0000 7f70ef13f700 1 -- 172.21.15.204:0/1549815032 shutdown_connections 2024-05-22T22:53:48.290 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.294+0000 7f70ef13f700 1 --2- 172.21.15.204:0/1549815032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70e8075110 0x7f70e81a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:48.290 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.294+0000 7f70ef13f700 1 --2- 172.21.15.204:0/1549815032 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f70d8039580 0x7f70d803ba40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:48.290 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.294+0000 7f70ef13f700 1 --2- 172.21.15.204:0/1549815032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70e8101ea0 0x7f70e8076100 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:48.290 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.294+0000 7f70ef13f700 1 --2- 172.21.15.204:0/1549815032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70e8074750 0x7f70e8074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:48.290 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.294+0000 7f70ef13f700 1 -- 172.21.15.204:0/1549815032 >> 172.21.15.204:0/1549815032 conn(0x7f70e80fd490 msgr2=0x7f70e80fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:53:48.290 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.294+0000 7f70ef13f700 1 -- 172.21.15.204:0/1549815032 shutdown_connections 2024-05-22T22:53:48.290 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:53:48.294+0000 7f70ef13f700 1 -- 172.21.15.204:0/1549815032 wait complete. 2024-05-22T22:53:48.292 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:48 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:48.292 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:48 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:48.292 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:48 smithi096 ceph-mon[36392]: Reconfiguring mon.b (monmap changed)... 2024-05-22T22:53:48.292 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:48 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-22T22:53:48.292 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:48 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-22T22:53:48.292 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:48 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:48.292 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:48 smithi096 ceph-mon[36392]: Reconfiguring daemon mon.b on smithi096 2024-05-22T22:53:48.292 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:48 smithi096 ceph-mon[36392]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:48.367 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:48.367 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:48.367 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:48 smithi084 ceph-mon[32186]: Reconfiguring mon.b (monmap changed)... 2024-05-22T22:53:48.367 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-22T22:53:48.367 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-22T22:53:48.368 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:48.368 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:48 smithi084 ceph-mon[32186]: Reconfiguring daemon mon.b on smithi096 2024-05-22T22:53:48.368 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:48 smithi084 ceph-mon[32186]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:48.403 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:48 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:48.403 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:48 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:48.403 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:48 smithi204 ceph-mon[36211]: Reconfiguring mon.b (monmap changed)... 2024-05-22T22:53:48.403 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:48 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-22T22:53:48.403 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:48 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-22T22:53:48.403 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:48 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:48.403 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:48 smithi204 ceph-mon[36211]: Reconfiguring daemon mon.b on smithi096 2024-05-22T22:53:48.403 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:48 smithi204 ceph-mon[36211]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:48.875 DEBUG:teuthology.orchestra.run.smithi096:mgr.y> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mgr.y.service 2024-05-22T22:53:48.877 DEBUG:teuthology.orchestra.run.smithi204:mgr.z> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mgr.z.service 2024-05-22T22:53:48.879 INFO:tasks.cephadm:Deploying OSDs... 2024-05-22T22:53:48.880 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-22T22:53:48.880 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/scratch_devs of=/dev/stdout 2024-05-22T22:53:48.900 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-22T22:53:48.900 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_1 2024-05-22T22:53:48.959 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-22T22:53:48.959 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-22T22:53:48.959 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-22T22:53:48.959 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-22T22:53:48.959 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-05-22T22:53:48.959 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-05-22 22:52:55.250168130 +0000 2024-05-22T22:53:48.959 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-05-22 22:47:56.625824105 +0000 2024-05-22T22:53:48.959 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-05-22 22:47:56.625824105 +0000 2024-05-22T22:53:48.959 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-05-22 22:47:56.625824105 +0000 2024-05-22T22:53:48.959 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-22T22:53:49.028 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-05-22T22:53:49.028 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-05-22T22:53:49.028 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000211568 s, 2.4 MB/s 2024-05-22T22:53:49.030 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-22T22:53:49.089 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_2 2024-05-22T22:53:49.147 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-22T22:53:49.147 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-22T22:53:49.147 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-22T22:53:49.147 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-22T22:53:49.147 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-05-22T22:53:49.147 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-05-22 22:52:55.282167609 +0000 2024-05-22T22:53:49.147 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-05-22 22:47:56.949818761 +0000 2024-05-22T22:53:49.148 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-05-22 22:47:56.949818761 +0000 2024-05-22T22:53:49.148 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-05-22 22:47:56.949818761 +0000 2024-05-22T22:53:49.148 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-22T22:53:49.213 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-05-22T22:53:49.213 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-05-22T22:53:49.214 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000257852 s, 2.0 MB/s 2024-05-22T22:53:49.215 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-22T22:53:49.273 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_3 2024-05-22T22:53:49.331 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-22T22:53:49.331 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-22T22:53:49.331 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-22T22:53:49.331 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-22T22:53:49.331 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-05-22T22:53:49.331 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-05-22 22:52:55.313167104 +0000 2024-05-22T22:53:49.331 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-05-22 22:47:57.273813416 +0000 2024-05-22T22:53:49.331 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-05-22 22:47:57.273813416 +0000 2024-05-22T22:53:49.331 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-05-22 22:47:57.273813416 +0000 2024-05-22T22:53:49.332 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-22T22:53:49.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:49 smithi204 ceph-mon[36211]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi084=x;smithi096=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:53:49.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:49 smithi204 ceph-mon[36211]: Saving service mgr spec with placement smithi084=x;smithi096=y;smithi204=z;count:3 2024-05-22T22:53:49.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:49 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:49.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:49 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:49.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:49 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:49.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:49 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-22T22:53:49.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:49 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-22T22:53:49.384 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:49 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:49.397 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-05-22T22:53:49.397 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-05-22T22:53:49.397 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000318289 s, 1.6 MB/s 2024-05-22T22:53:49.398 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-22T22:53:49.456 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_4 2024-05-22T22:53:49.513 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-22T22:53:49.513 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-22T22:53:49.513 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-22T22:53:49.513 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-22T22:53:49.513 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-05-22T22:53:49.513 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-05-22 22:52:55.344166599 +0000 2024-05-22T22:53:49.513 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-05-22 22:47:57.611807842 +0000 2024-05-22T22:53:49.513 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-05-22 22:47:57.611807842 +0000 2024-05-22T22:53:49.514 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-05-22 22:47:57.611807842 +0000 2024-05-22T22:53:49.514 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-22T22:53:49.541 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:49 smithi096 ceph-mon[36392]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi084=x;smithi096=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:53:49.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:49 smithi096 ceph-mon[36392]: Saving service mgr spec with placement smithi084=x;smithi096=y;smithi204=z;count:3 2024-05-22T22:53:49.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:49 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:49.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:49 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:49.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:49 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:49.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:49 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-22T22:53:49.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:49 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-22T22:53:49.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:49 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:49.575 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:49 smithi084 ceph-mon[32186]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi084=x;smithi096=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:53:49.576 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:49 smithi084 ceph-mon[32186]: Saving service mgr spec with placement smithi084=x;smithi096=y;smithi204=z;count:3 2024-05-22T22:53:49.576 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:49 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:49.576 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:49 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:49.576 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:49 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:49.576 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:49 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-22T22:53:49.576 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:49 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-22T22:53:49.576 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:49 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:49.577 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-05-22T22:53:49.578 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-05-22T22:53:49.578 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000215501 s, 2.4 MB/s 2024-05-22T22:53:49.579 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-22T22:53:49.636 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-22T22:53:49.636 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/scratch_devs of=/dev/stdout 2024-05-22T22:53:49.654 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-22T22:53:49.654 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_1 2024-05-22T22:53:49.712 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-22T22:53:49.713 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-22T22:53:49.713 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-22T22:53:49.713 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-22T22:53:49.713 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-05-22T22:53:49.713 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-05-22 22:53:37.004147608 +0000 2024-05-22T22:53:49.713 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-05-22 22:48:08.348199067 +0000 2024-05-22T22:53:49.713 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-05-22 22:48:08.348199067 +0000 2024-05-22T22:53:49.713 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-05-22 22:48:08.348199067 +0000 2024-05-22T22:53:49.713 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-22T22:53:49.779 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-05-22T22:53:49.779 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-05-22T22:53:49.779 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000294698 s, 1.7 MB/s 2024-05-22T22:53:49.781 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-22T22:53:49.840 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_2 2024-05-22T22:53:49.897 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-22T22:53:49.897 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-22T22:53:49.898 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-22T22:53:49.898 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-22T22:53:49.898 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-05-22T22:53:49.898 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-05-22 22:53:37.005147586 +0000 2024-05-22T22:53:49.898 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-05-22 22:48:08.727190311 +0000 2024-05-22T22:53:49.898 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-05-22 22:48:08.727190311 +0000 2024-05-22T22:53:49.898 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-05-22 22:48:08.727190311 +0000 2024-05-22T22:53:49.898 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-22T22:53:49.965 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-05-22T22:53:49.965 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-05-22T22:53:49.965 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000301064 s, 1.7 MB/s 2024-05-22T22:53:49.967 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-22T22:53:50.025 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_3 2024-05-22T22:53:50.081 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-22T22:53:50.081 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-22T22:53:50.081 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-05-22T22:53:50.081 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-22T22:53:50.082 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-05-22T22:53:50.082 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-05-22 22:53:37.005147586 +0000 2024-05-22T22:53:50.082 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-05-22 22:48:09.075182271 +0000 2024-05-22T22:53:50.082 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-05-22 22:48:09.075182271 +0000 2024-05-22T22:53:50.082 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-05-22 22:48:09.075182271 +0000 2024-05-22T22:53:50.082 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-22T22:53:50.146 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-05-22T22:53:50.146 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-05-22T22:53:50.146 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000203283 s, 2.5 MB/s 2024-05-22T22:53:50.147 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-22T22:53:50.204 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_4 2024-05-22T22:53:50.259 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-22T22:53:50.259 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-22T22:53:50.260 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-22T22:53:50.260 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-22T22:53:50.260 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-05-22T22:53:50.260 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-05-22 22:53:37.005147586 +0000 2024-05-22T22:53:50.260 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-05-22 22:48:09.427174139 +0000 2024-05-22T22:53:50.260 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-05-22 22:48:09.427174139 +0000 2024-05-22T22:53:50.260 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-05-22 22:48:09.427174139 +0000 2024-05-22T22:53:50.260 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-22T22:53:50.324 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-05-22T22:53:50.324 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-05-22T22:53:50.324 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000189632 s, 2.7 MB/s 2024-05-22T22:53:50.325 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-22T22:53:50.382 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-22T22:53:50.382 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/scratch_devs of=/dev/stdout 2024-05-22T22:53:50.399 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-22T22:53:50.400 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_1 2024-05-22T22:53:50.456 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-22T22:53:50.457 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-22T22:53:50.457 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-22T22:53:50.457 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-22T22:53:50.457 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-05-22T22:53:50.457 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-05-22 22:53:28.080032961 +0000 2024-05-22T22:53:50.457 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-05-22 22:48:06.323138838 +0000 2024-05-22T22:53:50.457 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-05-22 22:48:06.323138838 +0000 2024-05-22T22:53:50.457 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-05-22 22:48:06.323138838 +0000 2024-05-22T22:53:50.457 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-22T22:53:50.521 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-05-22T22:53:50.521 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-05-22T22:53:50.521 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000276584 s, 1.9 MB/s 2024-05-22T22:53:50.522 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-22T22:53:50.541 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:50 smithi096 ceph-mon[36392]: Reconfiguring mon.c (monmap changed)... 2024-05-22T22:53:50.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:50 smithi096 ceph-mon[36392]: Reconfiguring daemon mon.c on smithi204 2024-05-22T22:53:50.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:50 smithi096 ceph-mon[36392]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:50.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:50 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:50.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:50 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:50.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:50 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:53:50.579 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_2 2024-05-22T22:53:50.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:50 smithi084 ceph-mon[32186]: Reconfiguring mon.c (monmap changed)... 2024-05-22T22:53:50.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:50 smithi084 ceph-mon[32186]: Reconfiguring daemon mon.c on smithi204 2024-05-22T22:53:50.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:50 smithi084 ceph-mon[32186]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:50.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:50 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:50.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:50 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:50.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:50 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:53:50.634 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-22T22:53:50.634 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-22T22:53:50.634 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-22T22:53:50.634 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-22T22:53:50.634 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-05-22T22:53:50.634 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-05-22 22:53:28.081032949 +0000 2024-05-22T22:53:50.634 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-05-22 22:48:06.666134544 +0000 2024-05-22T22:53:50.634 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-05-22 22:48:06.666134544 +0000 2024-05-22T22:53:50.635 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-05-22 22:48:06.666134544 +0000 2024-05-22T22:53:50.635 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-22T22:53:50.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:50 smithi204 ceph-mon[36211]: Reconfiguring mon.c (monmap changed)... 2024-05-22T22:53:50.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:50 smithi204 ceph-mon[36211]: Reconfiguring daemon mon.c on smithi204 2024-05-22T22:53:50.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:50 smithi204 ceph-mon[36211]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:50.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:50 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:50.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:50 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:50.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:50 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:53:50.675 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-05-22T22:53:50.675 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-05-22T22:53:50.675 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000256048 s, 2.0 MB/s 2024-05-22T22:53:50.676 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-22T22:53:50.731 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_3 2024-05-22T22:53:50.786 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-22T22:53:50.786 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-22T22:53:50.786 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-22T22:53:50.786 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-22T22:53:50.786 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-05-22T22:53:50.786 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-05-22 22:53:28.081032949 +0000 2024-05-22T22:53:50.786 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-05-22 22:48:06.992130462 +0000 2024-05-22T22:53:50.786 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-05-22 22:48:06.992130462 +0000 2024-05-22T22:53:50.786 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-05-22 22:48:06.992130462 +0000 2024-05-22T22:53:50.786 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-22T22:53:50.848 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-05-22T22:53:50.849 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-05-22T22:53:50.849 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000203698 s, 2.5 MB/s 2024-05-22T22:53:50.850 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-22T22:53:50.906 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_4 2024-05-22T22:53:50.961 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-22T22:53:50.961 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-22T22:53:50.961 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-05-22T22:53:50.961 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-22T22:53:50.961 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-05-22T22:53:50.961 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-05-22 22:53:28.082032936 +0000 2024-05-22T22:53:50.961 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-05-22 22:48:07.317126394 +0000 2024-05-22T22:53:50.961 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-05-22 22:48:07.317126394 +0000 2024-05-22T22:53:50.961 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-05-22 22:48:07.317126394 +0000 2024-05-22T22:53:50.962 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-22T22:53:51.024 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-05-22T22:53:51.024 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-05-22T22:53:51.024 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000218165 s, 2.3 MB/s 2024-05-22T22:53:51.025 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-22T22:53:51.080 INFO:tasks.cephadm:Deploying osd.0 on smithi084 with /dev/vg_nvme/lv_4... 2024-05-22T22:53:51.080 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-22T22:53:51.314 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T22:53:51.367 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:51 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:51.367 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:51 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:53:51.367 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:51 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:51.367 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:51 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-22T22:53:51.367 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:51 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-22T22:53:51.367 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:51 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-22T22:53:51.367 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:51 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:51.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:51 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:51.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:51 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:53:51.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:51 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:51.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:51 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-22T22:53:51.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:51 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-22T22:53:51.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:51 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-22T22:53:51.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:51 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:51.791 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:51 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:51.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:51 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:53:51.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:51 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:51.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:51 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-22T22:53:51.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:51 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-22T22:53:51.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:51 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-22T22:53:51.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:51 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:52.403 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:52 smithi204 ceph-mon[36211]: Deploying daemon mgr.z on smithi204 2024-05-22T22:53:52.403 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:52 smithi204 ceph-mon[36211]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:52.589 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:52 smithi084 ceph-mon[32186]: Deploying daemon mgr.z on smithi204 2024-05-22T22:53:52.589 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:52 smithi084 ceph-mon[32186]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:52.791 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:52 smithi096 ceph-mon[36392]: Deploying daemon mgr.z on smithi204 2024-05-22T22:53:52.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:52 smithi096 ceph-mon[36392]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:52.818 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:52 smithi204 systemd[1]: Starting Ceph mgr.z for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T22:53:53.105 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:52 smithi204 podman[38047]: 2024-05-22 22:53:52.727577783 +0000 UTC m=+0.016990394 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T22:53:53.105 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:52 smithi204 podman[38047]: 2024-05-22 22:53:52.835362943 +0000 UTC m=+0.124775536 container create f67fd07efabb2fadfb2de9fe26b805056dd4333caff23ce2b97e558a6038f207 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2e1f762, org.label-schema.schema-version=1.0) 2024-05-22T22:53:53.105 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:53 smithi204 podman[38047]: 2024-05-22 22:53:53.029488581 +0000 UTC m=+0.318901174 container init f67fd07efabb2fadfb2de9fe26b805056dd4333caff23ce2b97e558a6038f207 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-2e1f762, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-22T22:53:53.106 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:53 smithi204 podman[38047]: 2024-05-22 22:53:53.03204107 +0000 UTC m=+0.321453670 container start f67fd07efabb2fadfb2de9fe26b805056dd4333caff23ce2b97e558a6038f207 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-2e1f762, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-05-22T22:53:53.106 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:53 smithi204 ceph-mgr[38067]: -- 172.21.15.204:0/2663594019 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aaf033e5a0 con 0x55aaf0313000 2024-05-22T22:53:53.106 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:53 smithi204 bash[38047]: f67fd07efabb2fadfb2de9fe26b805056dd4333caff23ce2b97e558a6038f207 2024-05-22T22:53:53.145 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-22T22:53:53.146 INFO:teuthology.orchestra.run.smithi084.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-22T22:53:53.146 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10+0 records in 2024-05-22T22:53:53.146 INFO:teuthology.orchestra.run.smithi084.stderr:10+0 records out 2024-05-22T22:53:53.146 INFO:teuthology.orchestra.run.smithi084.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0132097 s, 794 MB/s 2024-05-22T22:53:53.146 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping successful for: 2024-05-22T22:53:53.362 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:53 smithi204 systemd[1]: Started Ceph mgr.z for c1135a12-188d-11ef-bc9a-c7b262605968. 2024-05-22T22:53:53.362 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:53 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:53.185+0000 7eff7365c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-22T22:53:53.362 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:53 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:53.368+0000 7eff7365c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-22T22:53:53.653 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:53 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:53.471+0000 7eff7365c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-22T22:53:53.790 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi084:vg_nvme/lv_4 2024-05-22T22:53:54.025 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T22:53:54.153 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:53 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:53.780+0000 7eff7365c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-22T22:53:54.153 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:53 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:53.864+0000 7eff7365c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-22T22:53:54.421 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:54 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:54.422 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:54 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:54.422 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:54 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:54.422 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:54 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-22T22:53:54.422 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:54 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-22T22:53:54.422 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:54 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-22T22:53:54.422 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:54 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:54.422 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:54 smithi084 ceph-mon[32186]: Deploying daemon mgr.y on smithi096 2024-05-22T22:53:54.422 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:54 smithi084 ceph-mon[32186]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:54.474 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:54 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:54.474 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:54 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:54.474 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:54 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:54.474 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:54 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-22T22:53:54.474 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:54 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-22T22:53:54.475 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:54 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-22T22:53:54.475 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:54 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:54.475 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:54 smithi204 ceph-mon[36211]: Deploying daemon mgr.y on smithi096 2024-05-22T22:53:54.475 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:54 smithi204 ceph-mon[36211]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:54.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:54 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:54.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:54 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:54.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:54 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:54.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:54 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-22T22:53:54.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:54 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-22T22:53:54.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:54 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-22T22:53:54.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:54 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:54.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:54 smithi096 ceph-mon[36392]: Deploying daemon mgr.y on smithi096 2024-05-22T22:53:54.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:54 smithi096 ceph-mon[36392]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:55.292 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:53:55 smithi096 systemd[1]: Starting Ceph mgr.y for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T22:53:55.403 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:55 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:55.085+0000 7eff7365c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-22T22:53:55.403 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:55 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:55.254+0000 7eff7365c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-22T22:53:55.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.532+0000 7f12b6350700 1 -- 172.21.15.84:0/2044466751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b01035d0 msgr2=0x7f12b0103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:55.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.532+0000 7f12b6350700 1 --2- 172.21.15.84:0/2044466751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b01035d0 0x7f12b0103a30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f12a40099c0 tx=0x7f12a4009cd0 comp rx=0 tx=0).stop 2024-05-22T22:53:55.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.532+0000 7f12b6350700 1 -- 172.21.15.84:0/2044466751 shutdown_connections 2024-05-22T22:53:55.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.532+0000 7f12b6350700 1 --2- 172.21.15.84:0/2044466751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12b0103f70 0x7f12b010a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:55.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.532+0000 7f12b6350700 1 --2- 172.21.15.84:0/2044466751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12b01023d0 0x7f12b01027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:55.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.532+0000 7f12b6350700 1 --2- 172.21.15.84:0/2044466751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b01035d0 0x7f12b0103a30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:55.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.532+0000 7f12b6350700 1 -- 172.21.15.84:0/2044466751 >> 172.21.15.84:0/2044466751 conn(0x7f12b00fd930 msgr2=0x7f12b00ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:53:55.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.532+0000 7f12b6350700 1 -- 172.21.15.84:0/2044466751 shutdown_connections 2024-05-22T22:53:55.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.532+0000 7f12b6350700 1 -- 172.21.15.84:0/2044466751 wait complete. 2024-05-22T22:53:55.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.532+0000 7f12b6350700 1 Processor -- start 2024-05-22T22:53:55.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.532+0000 7f12b6350700 1 -- start start 2024-05-22T22:53:55.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.533+0000 7f12b6350700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b01023d0 0x7f12b0198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:55.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.533+0000 7f12b6350700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12b01035d0 0x7f12b0198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:55.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.533+0000 7f12b6350700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12b0103f70 0x7f12b019fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:55.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.533+0000 7f12b6350700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f12b010dfc0 con 0x7f12b01023d0 2024-05-22T22:53:55.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.533+0000 7f12b6350700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f12b010de40 con 0x7f12b01035d0 2024-05-22T22:53:55.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.533+0000 7f12b6350700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f12b010e140 con 0x7f12b0103f70 2024-05-22T22:53:55.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.533+0000 7f12affff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b01023d0 0x7f12b0198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:55.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.533+0000 7f12affff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b01023d0 0x7f12b0198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49260/0 (socket says 172.21.15.84:49260) 2024-05-22T22:53:55.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.533+0000 7f12affff700 1 -- 172.21.15.84:0/3854151671 learned_addr learned my addr 172.21.15.84:0/3854151671 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:53:55.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.533+0000 7f12af7fe700 1 --2- 172.21.15.84:0/3854151671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12b01035d0 0x7f12b0198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:55.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.533+0000 7f12b48ed700 1 --2- 172.21.15.84:0/3854151671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12b0103f70 0x7f12b019fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:55.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.533+0000 7f12affff700 1 -- 172.21.15.84:0/3854151671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12b0103f70 msgr2=0x7f12b019fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:55.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.533+0000 7f12affff700 1 --2- 172.21.15.84:0/3854151671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12b0103f70 0x7f12b019fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:55.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.533+0000 7f12affff700 1 -- 172.21.15.84:0/3854151671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12b01035d0 msgr2=0x7f12b0198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:53:55.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.533+0000 7f12affff700 1 --2- 172.21.15.84:0/3854151671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12b01035d0 0x7f12b0198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:53:55.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.533+0000 7f12affff700 1 -- 172.21.15.84:0/3854151671 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b01a01d0 con 0x7f12b01023d0 2024-05-22T22:53:55.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.533+0000 7f12af7fe700 1 --2- 172.21.15.84:0/3854151671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12b01035d0 0x7f12b0198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T22:53:55.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.533+0000 7f12affff700 1 --2- 172.21.15.84:0/3854151671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b01023d0 0x7f12b0198570 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f12a000e980 tx=0x7f12a000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:53:55.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.533+0000 7f12ad7fa700 1 -- 172.21.15.84:0/3854151671 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a0004230 con 0x7f12b01023d0 2024-05-22T22:53:55.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.533+0000 7f12ad7fa700 1 -- 172.21.15.84:0/3854151671 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12a00043d0 con 0x7f12b01023d0 2024-05-22T22:53:55.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.534+0000 7f12ad7fa700 1 -- 172.21.15.84:0/3854151671 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a0003dc0 con 0x7f12b01023d0 2024-05-22T22:53:55.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.534+0000 7f12b6350700 1 -- 172.21.15.84:0/3854151671 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b01a04c0 con 0x7f12b01023d0 2024-05-22T22:53:55.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.534+0000 7f12b6350700 1 -- 172.21.15.84:0/3854151671 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b01a0a00 con 0x7f12b01023d0 2024-05-22T22:53:55.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.534+0000 7f12ad7fa700 1 -- 172.21.15.84:0/3854151671 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f12a0021440 con 0x7f12b01023d0 2024-05-22T22:53:55.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.534+0000 7f12b6350700 1 -- 172.21.15.84:0/3854151671 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12b0069050 con 0x7f12b01023d0 2024-05-22T22:53:55.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.535+0000 7f12ad7fa700 1 --2- 172.21.15.84:0/3854151671 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1298039330 0x7f129803b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:53:55.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.535+0000 7f12ad7fa700 1 -- 172.21.15.84:0/3854151671 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f12a004c730 con 0x7f12b01023d0 2024-05-22T22:53:55.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.535+0000 7f12af7fe700 1 --2- 172.21.15.84:0/3854151671 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1298039330 0x7f129803b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:53:55.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.535+0000 7f12af7fe700 1 --2- 172.21.15.84:0/3854151671 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1298039330 0x7f129803b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f12a4000c00 tx=0x7f12a4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:53:55.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.540+0000 7f12ad7fa700 1 -- 172.21.15.84:0/3854151671 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f12a001f7e0 con 0x7f12b01023d0 2024-05-22T22:53:55.619 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:53:55 smithi096 podman[37751]: 2024-05-22 22:53:55.237847596 +0000 UTC m=+0.017364999 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T22:53:55.619 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:53:55 smithi096 podman[37751]: 2024-05-22 22:53:55.348988122 +0000 UTC m=+0.128505572 container create f4badc5353f9ad578122ed4e04c24053ff81529a90696c5983dfbad4fef01774 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-2e1f762) 2024-05-22T22:53:55.619 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:53:55 smithi096 podman[37751]: 2024-05-22 22:53:55.52633498 +0000 UTC m=+0.305852374 container init f4badc5353f9ad578122ed4e04c24053ff81529a90696c5983dfbad4fef01774 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-2e1f762, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-22T22:53:55.620 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:53:55 smithi096 podman[37751]: 2024-05-22 22:53:55.528957543 +0000 UTC m=+0.308474938 container start f4badc5353f9ad578122ed4e04c24053ff81529a90696c5983dfbad4fef01774 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2e1f762, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-22T22:53:55.620 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:53:55 smithi096 ceph-mgr[37770]: -- 172.21.15.96:0/2880612947 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56329a4aa5a0 con 0x56329a47ec00 2024-05-22T22:53:55.620 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:53:55 smithi096 bash[37751]: f4badc5353f9ad578122ed4e04c24053ff81529a90696c5983dfbad4fef01774 2024-05-22T22:53:55.709 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:55 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:55.427+0000 7eff7365c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-22T22:53:55.709 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:55 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:55.715+0000 7eff7365c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-22T22:53:55.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:53:55.748+0000 7f12b6350700 1 -- 172.21.15.84:0/3854151671 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f12b00627a0 con 0x7f1298039330 2024-05-22T22:53:55.968 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:53:55 smithi096 systemd[1]: Started Ceph mgr.y for c1135a12-188d-11ef-bc9a-c7b262605968. 2024-05-22T22:53:55.968 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:53:55 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:53:55.680+0000 7f4751f91200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-22T22:53:55.968 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:53:55 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:53:55.869+0000 7f4751f91200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-22T22:53:55.968 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:53:55 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:53:55.974+0000 7f4751f91200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-22T22:53:56.010 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:55 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:55.806+0000 7eff7365c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-22T22:53:56.010 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:56 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:56.016+0000 7eff7365c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-22T22:53:56.403 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:56 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:56.215+0000 7eff7365c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-22T22:53:56.598 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:56 smithi084 ceph-mon[32186]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:56.599 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:56 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:56.599 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:56 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:56.599 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:56 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:56.599 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:56 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:56.599 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:56 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:53:56.599 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:56 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:53:56.599 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:56 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:56.599 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:56 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:53:56.709 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:53:56 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:53:56.302+0000 7f4751f91200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-22T22:53:56.709 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:53:56 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:53:56.395+0000 7f4751f91200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-22T22:53:56.709 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:56 smithi096 ceph-mon[36392]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:56.709 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:56 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:56.709 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:56 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:56.710 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:56 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:56.710 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:56 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:56.710 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:56 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:53:56.710 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:56 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:53:56.710 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:56 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:56.710 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:56 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:53:56.737 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:56 smithi204 ceph-mon[36211]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:56.737 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:56 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:56.737 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:56 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:56.737 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:56 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:56.737 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:56 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:56.737 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:56 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:53:56.737 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:56 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:53:56.737 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:56 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:56.737 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:56 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:53:57.153 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:56 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:56.844+0000 7eff7365c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-22T22:53:57.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:57 smithi096 ceph-mon[36392]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:53:57.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:57 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:57.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:57 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:57.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:57 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:57.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:57 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:57.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:57 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:57.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:57 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:53:57.543 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:57 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:57.543 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:57 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-22T22:53:57.543 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:57 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-22T22:53:57.543 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:57 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:57.565 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:57 smithi084 ceph-mon[32186]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:53:57.565 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:57.565 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:57.565 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:57.565 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:57.565 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:57.565 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:53:57.565 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:57.565 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-22T22:53:57.565 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-22T22:53:57.565 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:57.568 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:57 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:57.273+0000 7eff7365c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-22T22:53:57.568 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:57 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:57.373+0000 7eff7365c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-22T22:53:57.568 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:57 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:57.574+0000 7eff7365c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-22T22:53:57.568 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:57 smithi204 ceph-mon[36211]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:53:57.568 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:57 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:57.568 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:57 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:57.568 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:57 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:57.568 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:57 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:57.568 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:57 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:57.568 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:57 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:53:57.568 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:57 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:53:57.569 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:57 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-22T22:53:57.569 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:57 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-22T22:53:57.569 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:57 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:53:58.018 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:53:57 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:53:57.678+0000 7f4751f91200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-22T22:53:58.018 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:53:57 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:53:57.851+0000 7f4751f91200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-22T22:53:58.018 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:53:58 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:53:58.024+0000 7f4751f91200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-22T22:53:58.403 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:58 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:58.097+0000 7eff7365c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-22T22:53:58.403 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:58 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:58.189+0000 7eff7365c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-22T22:53:58.604 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:53:58 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:53:58.319+0000 7f4751f91200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-22T22:53:58.604 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:53:58 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:53:58.411+0000 7f4751f91200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-22T22:53:58.604 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:53:58 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:53:58.610+0000 7f4751f91200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-22T22:53:58.604 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:58 smithi096 ceph-mon[36392]: Reconfiguring mgr.x (unknown last config time)... 2024-05-22T22:53:58.604 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:58 smithi096 ceph-mon[36392]: Reconfiguring daemon mgr.x on smithi084 2024-05-22T22:53:58.604 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:58 smithi096 ceph-mon[36392]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:58.604 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:58 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3923812227' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "762df065-fb4f-4c71-aab8-a62d2282c253"}]: dispatch 2024-05-22T22:53:58.604 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:58 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3923812227' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "762df065-fb4f-4c71-aab8-a62d2282c253"}]': finished 2024-05-22T22:53:58.604 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:58 smithi096 ceph-mon[36392]: osdmap e5: 1 total, 0 up, 1 in 2024-05-22T22:53:58.604 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:58 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-22T22:53:58.763 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:58 smithi084 ceph-mon[32186]: Reconfiguring mgr.x (unknown last config time)... 2024-05-22T22:53:58.763 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:58 smithi084 ceph-mon[32186]: Reconfiguring daemon mgr.x on smithi084 2024-05-22T22:53:58.763 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:58 smithi084 ceph-mon[32186]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:58.764 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:58 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3923812227' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "762df065-fb4f-4c71-aab8-a62d2282c253"}]: dispatch 2024-05-22T22:53:58.764 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:58 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3923812227' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "762df065-fb4f-4c71-aab8-a62d2282c253"}]': finished 2024-05-22T22:53:58.764 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:58 smithi084 ceph-mon[32186]: osdmap e5: 1 total, 0 up, 1 in 2024-05-22T22:53:58.764 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:58 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-22T22:53:58.773 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:58 smithi204 ceph-mon[36211]: Reconfiguring mgr.x (unknown last config time)... 2024-05-22T22:53:58.774 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:58 smithi204 ceph-mon[36211]: Reconfiguring daemon mgr.x on smithi084 2024-05-22T22:53:58.774 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:58 smithi204 ceph-mon[36211]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:53:58.774 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:58 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3923812227' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "762df065-fb4f-4c71-aab8-a62d2282c253"}]: dispatch 2024-05-22T22:53:58.774 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:58 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3923812227' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "762df065-fb4f-4c71-aab8-a62d2282c253"}]': finished 2024-05-22T22:53:58.774 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:58 smithi204 ceph-mon[36211]: osdmap e5: 1 total, 0 up, 1 in 2024-05-22T22:53:58.774 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:58 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-22T22:53:58.775 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:58 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:58.433+0000 7eff7365c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-22T22:53:58.775 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:58 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:58.567+0000 7eff7365c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-22T22:53:58.775 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:58 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:58.661+0000 7eff7365c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-22T22:53:58.775 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:58 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:58.780+0000 7eff7365c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-22T22:53:59.042 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:53:58 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:53:58.801+0000 7f4751f91200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-22T22:53:59.405 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:59 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:59.081+0000 7eff7365c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-22T22:53:59.779 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:53:59 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2076415046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:53:59.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:53:59 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2076415046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:53:59.792 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:53:59 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:53:59.393+0000 7f4751f91200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-22T22:53:59.890 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:53:59 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2076415046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:53:59.890 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:59 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:59.679+0000 7eff7365c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-22T22:53:59.890 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 22:53:59 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T22:53:59.896+0000 7eff7365c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-22T22:54:00.098 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:53:59 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:53:59.810+0000 7f4751f91200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-22T22:54:00.098 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:53:59 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:53:59.906+0000 7f4751f91200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-22T22:54:00.098 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:54:00 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:54:00.104+0000 7f4751f91200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-22T22:54:00.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:54:00.492+0000 7f12ad7fa700 1 -- 172.21.15.84:0/3854151671 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f12a0007d10 con 0x7f12b01023d0 2024-05-22T22:54:00.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:00 smithi096 ceph-mon[36392]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:00.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:00 smithi096 ceph-mon[36392]: Standby manager daemon z started 2024-05-22T22:54:00.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:00 smithi096 ceph-mon[36392]: from='mgr.? 172.21.15.204:0/3412735116' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-22T22:54:00.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:00 smithi096 ceph-mon[36392]: from='mgr.? 172.21.15.204:0/3412735116' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-22T22:54:00.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:00 smithi096 ceph-mon[36392]: from='mgr.? 172.21.15.204:0/3412735116' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-22T22:54:00.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:00 smithi096 ceph-mon[36392]: from='mgr.? 172.21.15.204:0/3412735116' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-22T22:54:00.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:00 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:00.793 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:54:00 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:54:00.620+0000 7f4751f91200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-22T22:54:00.793 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:54:00 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:54:00.712+0000 7f4751f91200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-22T22:54:00.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:00 smithi084 ceph-mon[32186]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:00.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:00 smithi084 ceph-mon[32186]: Standby manager daemon z started 2024-05-22T22:54:00.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:00 smithi084 ceph-mon[32186]: from='mgr.? 172.21.15.204:0/3412735116' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-22T22:54:00.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:00 smithi084 ceph-mon[32186]: from='mgr.? 172.21.15.204:0/3412735116' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-22T22:54:00.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:00 smithi084 ceph-mon[32186]: from='mgr.? 172.21.15.204:0/3412735116' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-22T22:54:00.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:00 smithi084 ceph-mon[32186]: from='mgr.? 172.21.15.204:0/3412735116' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-22T22:54:00.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:00 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:00.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:00 smithi204 ceph-mon[36211]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:00.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:00 smithi204 ceph-mon[36211]: Standby manager daemon z started 2024-05-22T22:54:00.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:00 smithi204 ceph-mon[36211]: from='mgr.? 172.21.15.204:0/3412735116' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-22T22:54:00.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:00 smithi204 ceph-mon[36211]: from='mgr.? 172.21.15.204:0/3412735116' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-22T22:54:00.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:00 smithi204 ceph-mon[36211]: from='mgr.? 172.21.15.204:0/3412735116' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-22T22:54:00.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:00 smithi204 ceph-mon[36211]: from='mgr.? 172.21.15.204:0/3412735116' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-22T22:54:00.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:00 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:01.285 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:54:00 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:54:00.952+0000 7f4751f91200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-22T22:54:01.285 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:54:01 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:54:01.084+0000 7f4751f91200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-22T22:54:01.285 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:54:01 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:54:01.177+0000 7f4751f91200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-22T22:54:01.285 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:54:01 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:54:01.291+0000 7f4751f91200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-22T22:54:01.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:01 smithi096 ceph-mon[36392]: mgrmap e14: x(active, since 75s), standbys: z 2024-05-22T22:54:01.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:01 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-22T22:54:01.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:01 smithi084 ceph-mon[32186]: mgrmap e14: x(active, since 75s), standbys: z 2024-05-22T22:54:01.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:01 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-22T22:54:01.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:01 smithi204 ceph-mon[36211]: mgrmap e14: x(active, since 75s), standbys: z 2024-05-22T22:54:01.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:01 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-22T22:54:02.042 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:54:01 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:54:01.587+0000 7f4751f91200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-22T22:54:02.488 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:54:02 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:54:02.178+0000 7f4751f91200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-22T22:54:02.489 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 22:54:02 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T22:54:02.391+0000 7f4751f91200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-22T22:54:02.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:54:02.506+0000 7f12ad7fa700 1 -- 172.21.15.84:0/3854151671 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f12a0021ca0 con 0x7f12b01023d0 2024-05-22T22:54:02.660 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:02 smithi084 ceph-mon[32186]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:02.660 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:02 smithi084 ceph-mon[32186]: Standby manager daemon y started 2024-05-22T22:54:02.660 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:02 smithi084 ceph-mon[32186]: from='mgr.? 172.21.15.96:0/1794026892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-22T22:54:02.660 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:02 smithi084 ceph-mon[32186]: from='mgr.? 172.21.15.96:0/1794026892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-22T22:54:02.660 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:02 smithi084 ceph-mon[32186]: from='mgr.? 172.21.15.96:0/1794026892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-22T22:54:02.660 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:02 smithi084 ceph-mon[32186]: from='mgr.? 172.21.15.96:0/1794026892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-22T22:54:02.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:02 smithi096 ceph-mon[36392]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:02.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:02 smithi096 ceph-mon[36392]: Standby manager daemon y started 2024-05-22T22:54:02.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:02 smithi096 ceph-mon[36392]: from='mgr.? 172.21.15.96:0/1794026892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-22T22:54:02.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:02 smithi096 ceph-mon[36392]: from='mgr.? 172.21.15.96:0/1794026892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-22T22:54:02.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:02 smithi096 ceph-mon[36392]: from='mgr.? 172.21.15.96:0/1794026892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-22T22:54:02.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:02 smithi096 ceph-mon[36392]: from='mgr.? 172.21.15.96:0/1794026892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-22T22:54:02.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:02 smithi204 ceph-mon[36211]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:02.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:02 smithi204 ceph-mon[36211]: Standby manager daemon y started 2024-05-22T22:54:02.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:02 smithi204 ceph-mon[36211]: from='mgr.? 172.21.15.96:0/1794026892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-22T22:54:02.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:02 smithi204 ceph-mon[36211]: from='mgr.? 172.21.15.96:0/1794026892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-22T22:54:02.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:02 smithi204 ceph-mon[36211]: from='mgr.? 172.21.15.96:0/1794026892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-22T22:54:02.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:02 smithi204 ceph-mon[36211]: from='mgr.? 172.21.15.96:0/1794026892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-22T22:54:03.791 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:03 smithi084 ceph-mon[32186]: mgrmap e15: x(active, since 77s), standbys: z, y 2024-05-22T22:54:03.791 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:03 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-22T22:54:03.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:03 smithi096 ceph-mon[36392]: mgrmap e15: x(active, since 77s), standbys: z, y 2024-05-22T22:54:03.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:03 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-22T22:54:03.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:03 smithi204 ceph-mon[36211]: mgrmap e15: x(active, since 77s), standbys: z, y 2024-05-22T22:54:03.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:03 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-22T22:54:04.788 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:04 smithi084 ceph-mon[32186]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:04.788 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:04 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:04.789 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:04 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:04.789 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:04 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:54:04.789 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:04 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:04.789 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:04 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:54:04.789 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:04 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:04.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:04 smithi096 ceph-mon[36392]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:04.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:04 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:04.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:04 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:04.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:04 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:54:04.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:04 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:04.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:04 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:54:04.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:04 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:04.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:04 smithi204 ceph-mon[36211]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:04.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:04 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:04.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:04 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:04.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:04 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:54:04.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:04 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:04.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:04 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:54:04.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:04 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:06.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:06 smithi084 ceph-mon[32186]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:06.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-22T22:54:06.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:06.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:06 smithi096 ceph-mon[36392]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:06.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-22T22:54:06.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:06.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:06 smithi204 ceph-mon[36211]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:06.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-22T22:54:06.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:07.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:07 smithi084 ceph-mon[32186]: Deploying daemon osd.0 on smithi084 2024-05-22T22:54:07.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:07 smithi096 ceph-mon[36392]: Deploying daemon osd.0 on smithi084 2024-05-22T22:54:07.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:07 smithi204 ceph-mon[36211]: Deploying daemon osd.0 on smithi084 2024-05-22T22:54:08.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:08 smithi096 ceph-mon[36392]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:08.867 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:08 smithi084 ceph-mon[32186]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:08.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:08 smithi204 ceph-mon[36211]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:10.791 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:10 smithi084 ceph-mon[32186]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:10.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:10 smithi096 ceph-mon[36392]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:10.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:10 smithi204 ceph-mon[36211]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:12.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:12 smithi096 ceph-mon[36392]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:12.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:12 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:12.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:12 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:12.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:12 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:54:12.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:12 smithi084 ceph-mon[32186]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:12.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:12 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:12.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:12 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:12.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:12 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:54:12.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:12 smithi204 ceph-mon[36211]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:12.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:12 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:12.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:12 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:12.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:12 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:54:14.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:14 smithi096 ceph-mon[36392]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:14.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:14 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:14.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:14 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:14.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:14 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:14.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:14 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:54:14.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:14 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:14.819 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:14 smithi084 ceph-mon[32186]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:14.819 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:14 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:14.819 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:14 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:14.819 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:14 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:14.819 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:14 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:54:14.820 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:14 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:14.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:14 smithi204 ceph-mon[36211]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:14.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:14 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:14.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:14 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:14.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:14 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:14.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:14 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:54:14.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:14 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:15.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:54:15.152+0000 7f12ad7fa700 1 -- 172.21.15.84:0/3854151671 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f12b00627a0 con 0x7f1298039330 2024-05-22T22:54:15.147 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 0 on host 'smithi084' 2024-05-22T22:54:15.149 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:54:15.155+0000 7f12b6350700 1 -- 172.21.15.84:0/3854151671 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1298039330 msgr2=0x7f129803b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:54:15.149 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:54:15.155+0000 7f12b6350700 1 --2- 172.21.15.84:0/3854151671 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1298039330 0x7f129803b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f12a4000c00 tx=0x7f12a4005fb0 comp rx=0 tx=0).stop 2024-05-22T22:54:15.149 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:54:15.155+0000 7f12b6350700 1 -- 172.21.15.84:0/3854151671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b01023d0 msgr2=0x7f12b0198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:54:15.149 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:54:15.155+0000 7f12b6350700 1 --2- 172.21.15.84:0/3854151671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b01023d0 0x7f12b0198570 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f12a000e980 tx=0x7f12a000ec90 comp rx=0 tx=0).stop 2024-05-22T22:54:15.149 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:54:15.155+0000 7f12b6350700 1 -- 172.21.15.84:0/3854151671 shutdown_connections 2024-05-22T22:54:15.149 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:54:15.155+0000 7f12b6350700 1 --2- 172.21.15.84:0/3854151671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12b0103f70 0x7f12b019fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:54:15.149 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:54:15.155+0000 7f12b6350700 1 --2- 172.21.15.84:0/3854151671 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1298039330 0x7f129803b7f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:54:15.150 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:54:15.155+0000 7f12b6350700 1 --2- 172.21.15.84:0/3854151671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b01023d0 0x7f12b0198570 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:54:15.150 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:54:15.155+0000 7f12b6350700 1 --2- 172.21.15.84:0/3854151671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12b01035d0 0x7f12b0198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:54:15.150 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:54:15.155+0000 7f12b6350700 1 -- 172.21.15.84:0/3854151671 >> 172.21.15.84:0/3854151671 conn(0x7f12b00fd930 msgr2=0x7f12b01068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:54:15.150 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:54:15.155+0000 7f12b6350700 1 -- 172.21.15.84:0/3854151671 shutdown_connections 2024-05-22T22:54:15.150 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:54:15.155+0000 7f12b6350700 1 -- 172.21.15.84:0/3854151671 wait complete. 2024-05-22T22:54:15.801 DEBUG:teuthology.orchestra.run.smithi084:osd.0> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.0.service 2024-05-22T22:54:15.803 INFO:tasks.cephadm:Deploying osd.1 on smithi096 with /dev/vg_nvme/lv_4... 2024-05-22T22:54:15.803 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-22T22:54:16.045 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.b/config 2024-05-22T22:54:16.368 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:16 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:16.368 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:16 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:16.368 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:16 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:54:16.368 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:16 smithi084 ceph-mon[32186]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:16.368 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:16 smithi084 ceph-mon[32186]: from='osd.0 [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-22T22:54:16.403 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:16 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:16.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:16 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:16.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:16 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:54:16.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:16 smithi204 ceph-mon[36211]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:16.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:16 smithi204 ceph-mon[36211]: from='osd.0 [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-22T22:54:16.465 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:16 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:16.465 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:16 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:16.465 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:16 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:54:16.465 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:16 smithi096 ceph-mon[36392]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:16.465 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:16 smithi096 ceph-mon[36392]: from='osd.0 [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-22T22:54:17.271 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:17 smithi096 ceph-mon[36392]: from='osd.0 [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-22T22:54:17.271 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:17 smithi096 ceph-mon[36392]: osdmap e6: 1 total, 0 up, 1 in 2024-05-22T22:54:17.271 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:17 smithi096 ceph-mon[36392]: from='osd.0 [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-22T22:54:17.271 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:17 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-22T22:54:17.293 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:17 smithi084 ceph-mon[32186]: from='osd.0 [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-22T22:54:17.293 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:17 smithi084 ceph-mon[32186]: osdmap e6: 1 total, 0 up, 1 in 2024-05-22T22:54:17.293 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:17 smithi084 ceph-mon[32186]: from='osd.0 [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-22T22:54:17.293 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:17 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-22T22:54:17.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:17 smithi204 ceph-mon[36211]: from='osd.0 [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-22T22:54:17.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:17 smithi204 ceph-mon[36211]: osdmap e6: 1 total, 0 up, 1 in 2024-05-22T22:54:17.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:17 smithi204 ceph-mon[36211]: from='osd.0 [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-22T22:54:17.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:17 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-22T22:54:17.829 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-22T22:54:17.829 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-22T22:54:17.829 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-05-22T22:54:17.829 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-05-22T22:54:17.829 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113155 s, 927 MB/s 2024-05-22T22:54:17.829 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-05-22T22:54:18.118 INFO:journalctl@ceph.osd.0.smithi084.stdout:May 22 22:54:17 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-0[44652]: 2024-05-22T22:54:17.666+0000 7f32f845b700 -1 osd.0 0 waiting for initial osdmap 2024-05-22T22:54:18.118 INFO:journalctl@ceph.osd.0.smithi084.stdout:May 22 22:54:17 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-0[44652]: 2024-05-22T22:54:17.673+0000 7f32f0246700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-22T22:54:18.394 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_4 2024-05-22T22:54:18.416 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:18 smithi096 ceph-mon[36392]: from='osd.0 [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-05-22T22:54:18.416 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:18 smithi096 ceph-mon[36392]: osdmap e7: 1 total, 0 up, 1 in 2024-05-22T22:54:18.416 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:18 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-22T22:54:18.416 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:18 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-22T22:54:18.416 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:18 smithi096 ceph-mon[36392]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:18.416 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:18 smithi096 ceph-mon[36392]: from='osd.0 [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826]' entity='osd.0' 2024-05-22T22:54:18.442 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:18 smithi084 ceph-mon[32186]: from='osd.0 [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-05-22T22:54:18.442 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:18 smithi084 ceph-mon[32186]: osdmap e7: 1 total, 0 up, 1 in 2024-05-22T22:54:18.442 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:18 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-22T22:54:18.442 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:18 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-22T22:54:18.442 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:18 smithi084 ceph-mon[32186]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:18.442 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:18 smithi084 ceph-mon[32186]: from='osd.0 [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826]' entity='osd.0' 2024-05-22T22:54:18.632 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.b/config 2024-05-22T22:54:18.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:18 smithi204 ceph-mon[36211]: from='osd.0 [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-05-22T22:54:18.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:18 smithi204 ceph-mon[36211]: osdmap e7: 1 total, 0 up, 1 in 2024-05-22T22:54:18.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:18 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-22T22:54:18.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:18 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-22T22:54:18.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:18 smithi204 ceph-mon[36211]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-22T22:54:18.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:18 smithi204 ceph-mon[36211]: from='osd.0 [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826]' entity='osd.0' 2024-05-22T22:54:19.292 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:19 smithi096 ceph-mon[36392]: purged_snaps scrub starts 2024-05-22T22:54:19.293 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:19 smithi096 ceph-mon[36392]: purged_snaps scrub ok 2024-05-22T22:54:19.293 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:19 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-22T22:54:19.293 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:19 smithi096 ceph-mon[36392]: osd.0 [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826] boot 2024-05-22T22:54:19.293 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:19 smithi096 ceph-mon[36392]: osdmap e8: 1 total, 1 up, 1 in 2024-05-22T22:54:19.293 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:19 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-22T22:54:19.293 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:19 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:19.293 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:19 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:19.293 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:19 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:54:19.293 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:19 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:19.293 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:19 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:19.294 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:19 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:54:19.294 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:19 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:19.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:19 smithi084 ceph-mon[32186]: purged_snaps scrub starts 2024-05-22T22:54:19.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:19 smithi084 ceph-mon[32186]: purged_snaps scrub ok 2024-05-22T22:54:19.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:19 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-22T22:54:19.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:19 smithi084 ceph-mon[32186]: osd.0 [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826] boot 2024-05-22T22:54:19.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:19 smithi084 ceph-mon[32186]: osdmap e8: 1 total, 1 up, 1 in 2024-05-22T22:54:19.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:19 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-22T22:54:19.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:19 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:19.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:19 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:19.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:19 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:54:19.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:19 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:19.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:19 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:19.619 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:19 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:54:19.619 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:19 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:19 smithi204 ceph-mon[36211]: purged_snaps scrub starts 2024-05-22T22:54:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:19 smithi204 ceph-mon[36211]: purged_snaps scrub ok 2024-05-22T22:54:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:19 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-22T22:54:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:19 smithi204 ceph-mon[36211]: osd.0 [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826] boot 2024-05-22T22:54:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:19 smithi204 ceph-mon[36211]: osdmap e8: 1 total, 1 up, 1 in 2024-05-22T22:54:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:19 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-22T22:54:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:19 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:19 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:19.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:19 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:54:19.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:19 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:19.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:19 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:19.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:19 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:54:19.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:19 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:20.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.151+0000 7fef1e9e3700 1 -- 172.21.15.96:0/3824090326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef181057a0 msgr2=0x7fef18107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:54:20.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.151+0000 7fef1e9e3700 1 --2- 172.21.15.96:0/3824090326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef181057a0 0x7fef18107b90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fef0c009990 tx=0x7fef0c009ca0 comp rx=0 tx=0).stop 2024-05-22T22:54:20.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.151+0000 7fef1e9e3700 1 -- 172.21.15.96:0/3824090326 shutdown_connections 2024-05-22T22:54:20.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.151+0000 7fef1e9e3700 1 --2- 172.21.15.96:0/3824090326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef181080d0 0x7fef1810a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:54:20.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.151+0000 7fef1e9e3700 1 --2- 172.21.15.96:0/3824090326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef18102e70 0x7fef18105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:54:20.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.151+0000 7fef1e9e3700 1 --2- 172.21.15.96:0/3824090326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef181057a0 0x7fef18107b90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:54:20.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.151+0000 7fef1e9e3700 1 -- 172.21.15.96:0/3824090326 >> 172.21.15.96:0/3824090326 conn(0x7fef180fc8c0 msgr2=0x7fef180fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:54:20.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.151+0000 7fef1e9e3700 1 -- 172.21.15.96:0/3824090326 shutdown_connections 2024-05-22T22:54:20.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.151+0000 7fef1e9e3700 1 -- 172.21.15.96:0/3824090326 wait complete. 2024-05-22T22:54:20.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.151+0000 7fef1e9e3700 1 Processor -- start 2024-05-22T22:54:20.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.151+0000 7fef1e9e3700 1 -- start start 2024-05-22T22:54:20.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.152+0000 7fef1e9e3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef18102e70 0x7fef181027d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:54:20.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.152+0000 7fef1e9e3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef181057a0 0x7fef18100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:54:20.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.152+0000 7fef1e9e3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef181080d0 0x7fef18101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:54:20.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.152+0000 7fef1e9e3700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fef1810e0b0 con 0x7fef18102e70 2024-05-22T22:54:20.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.152+0000 7fef1e9e3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fef1810df30 con 0x7fef181080d0 2024-05-22T22:54:20.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.152+0000 7fef1e9e3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fef1810e230 con 0x7fef181057a0 2024-05-22T22:54:20.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.152+0000 7fef1cf80700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef181080d0 0x7fef18101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:54:20.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.152+0000 7fef1cf80700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef181080d0 0x7fef18101360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49672/0 (socket says 172.21.15.96:49672) 2024-05-22T22:54:20.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.152+0000 7fef1cf80700 1 -- 172.21.15.96:0/1841936168 learned_addr learned my addr 172.21.15.96:0/1841936168 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-05-22T22:54:20.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.152+0000 7fef177fe700 1 --2- 172.21.15.96:0/1841936168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef181057a0 0x7fef18100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:54:20.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.152+0000 7fef17fff700 1 --2- 172.21.15.96:0/1841936168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef18102e70 0x7fef181027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:54:20.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.152+0000 7fef1cf80700 1 -- 172.21.15.96:0/1841936168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef181057a0 msgr2=0x7fef18100e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:54:20.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.152+0000 7fef1cf80700 1 --2- 172.21.15.96:0/1841936168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef181057a0 0x7fef18100e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:54:20.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.152+0000 7fef1cf80700 1 -- 172.21.15.96:0/1841936168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef18102e70 msgr2=0x7fef181027d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:54:20.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.152+0000 7fef1cf80700 1 --2- 172.21.15.96:0/1841936168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef18102e70 0x7fef181027d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:54:20.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.152+0000 7fef1cf80700 1 -- 172.21.15.96:0/1841936168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef18101c10 con 0x7fef181080d0 2024-05-22T22:54:20.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.152+0000 7fef1cf80700 1 --2- 172.21.15.96:0/1841936168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef181080d0 0x7fef18101360 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fef1000b9e0 tx=0x7fef1000bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:54:20.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.152+0000 7fef157fa700 1 -- 172.21.15.96:0/1841936168 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef10019070 con 0x7fef181080d0 2024-05-22T22:54:20.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.152+0000 7fef157fa700 1 -- 172.21.15.96:0/1841936168 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef10014440 con 0x7fef181080d0 2024-05-22T22:54:20.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.152+0000 7fef1e9e3700 1 -- 172.21.15.96:0/1841936168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef181a28c0 con 0x7fef181080d0 2024-05-22T22:54:20.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.152+0000 7fef157fa700 1 -- 172.21.15.96:0/1841936168 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef100125a0 con 0x7fef181080d0 2024-05-22T22:54:20.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.152+0000 7fef1e9e3700 1 -- 172.21.15.96:0/1841936168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef181a2e50 con 0x7fef181080d0 2024-05-22T22:54:20.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.153+0000 7fef1e9e3700 1 -- 172.21.15.96:0/1841936168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef18069050 con 0x7fef181080d0 2024-05-22T22:54:20.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.154+0000 7fef157fa700 1 -- 172.21.15.96:0/1841936168 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fef1001c070 con 0x7fef181080d0 2024-05-22T22:54:20.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.155+0000 7fef157fa700 1 --2- 172.21.15.96:0/1841936168 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fef000a2b90 0x7fef000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:54:20.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.155+0000 7fef157fa700 1 -- 172.21.15.96:0/1841936168 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7fef100cc950 con 0x7fef181080d0 2024-05-22T22:54:20.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.155+0000 7fef17fff700 1 --2- 172.21.15.96:0/1841936168 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fef000a2b90 0x7fef000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:54:20.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.156+0000 7fef17fff700 1 --2- 172.21.15.96:0/1841936168 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fef000a2b90 0x7fef000a5050 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fef08006fd0 tx=0x7fef08008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:54:20.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.160+0000 7fef157fa700 1 -- 172.21.15.96:0/1841936168 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fef1009b320 con 0x7fef181080d0 2024-05-22T22:54:20.292 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:20 smithi096 ceph-mon[36392]: Detected new or changed devices on smithi084 2024-05-22T22:54:20.292 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:20 smithi096 ceph-mon[36392]: Adjusting osd_memory_target on smithi084 to 17088M 2024-05-22T22:54:20.292 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:20 smithi096 ceph-mon[36392]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:20.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:20.368+0000 7fef1e9e3700 1 -- 172.21.15.96:0/1841936168 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fef180627a0 con 0x7fef000a2b90 2024-05-22T22:54:20.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:20 smithi084 ceph-mon[32186]: Detected new or changed devices on smithi084 2024-05-22T22:54:20.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:20 smithi084 ceph-mon[32186]: Adjusting osd_memory_target on smithi084 to 17088M 2024-05-22T22:54:20.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:20 smithi084 ceph-mon[32186]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:20.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:20 smithi204 ceph-mon[36211]: Detected new or changed devices on smithi084 2024-05-22T22:54:20.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:20 smithi204 ceph-mon[36211]: Adjusting osd_memory_target on smithi084 to 17088M 2024-05-22T22:54:20.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:20 smithi204 ceph-mon[36211]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:21.286 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:21 smithi096 ceph-mon[36392]: osdmap e9: 1 total, 1 up, 1 in 2024-05-22T22:54:21.286 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:21 smithi096 ceph-mon[36392]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:54:21.286 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:21 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:54:21.286 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:21 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:54:21.286 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:21 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:21.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:21 smithi084 ceph-mon[32186]: osdmap e9: 1 total, 1 up, 1 in 2024-05-22T22:54:21.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:21 smithi084 ceph-mon[32186]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:54:21.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:21 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:54:21.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:21 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:54:21.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:21 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:21.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:21 smithi204 ceph-mon[36211]: osdmap e9: 1 total, 1 up, 1 in 2024-05-22T22:54:21.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:21 smithi204 ceph-mon[36211]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:54:21.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:21 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:54:21.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:21 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:54:21.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:21 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:22.258 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:22 smithi096 ceph-mon[36392]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:22.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:22 smithi084 ceph-mon[32186]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:22.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:22 smithi204 ceph-mon[36211]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:23.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:23 smithi096 ceph-mon[36392]: from='client.? 172.21.15.96:0/2414060626' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fb2385d-6a34-49e3-88e1-116237beb346"}]: dispatch 2024-05-22T22:54:23.543 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:23 smithi096 ceph-mon[36392]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fb2385d-6a34-49e3-88e1-116237beb346"}]: dispatch 2024-05-22T22:54:23.543 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:23 smithi096 ceph-mon[36392]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fb2385d-6a34-49e3-88e1-116237beb346"}]': finished 2024-05-22T22:54:23.543 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:23 smithi096 ceph-mon[36392]: osdmap e10: 2 total, 1 up, 2 in 2024-05-22T22:54:23.543 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:23 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-22T22:54:23.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:23 smithi084 ceph-mon[32186]: from='client.? 172.21.15.96:0/2414060626' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fb2385d-6a34-49e3-88e1-116237beb346"}]: dispatch 2024-05-22T22:54:23.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:23 smithi084 ceph-mon[32186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fb2385d-6a34-49e3-88e1-116237beb346"}]: dispatch 2024-05-22T22:54:23.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:23 smithi084 ceph-mon[32186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fb2385d-6a34-49e3-88e1-116237beb346"}]': finished 2024-05-22T22:54:23.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:23 smithi084 ceph-mon[32186]: osdmap e10: 2 total, 1 up, 2 in 2024-05-22T22:54:23.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:23 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-22T22:54:23.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:23 smithi204 ceph-mon[36211]: from='client.? 172.21.15.96:0/2414060626' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fb2385d-6a34-49e3-88e1-116237beb346"}]: dispatch 2024-05-22T22:54:23.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:23 smithi204 ceph-mon[36211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fb2385d-6a34-49e3-88e1-116237beb346"}]: dispatch 2024-05-22T22:54:23.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:23 smithi204 ceph-mon[36211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fb2385d-6a34-49e3-88e1-116237beb346"}]': finished 2024-05-22T22:54:23.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:23 smithi204 ceph-mon[36211]: osdmap e10: 2 total, 1 up, 2 in 2024-05-22T22:54:23.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:23 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-22T22:54:24.443 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:24 smithi096 ceph-mon[36392]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:24.444 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:24 smithi096 ceph-mon[36392]: from='client.? 172.21.15.96:0/692302674' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:54:24.555 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:24 smithi204 ceph-mon[36211]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:24.555 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:24 smithi204 ceph-mon[36211]: from='client.? 172.21.15.96:0/692302674' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:54:24.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:24 smithi084 ceph-mon[32186]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:24.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:24 smithi084 ceph-mon[32186]: from='client.? 172.21.15.96:0/692302674' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:54:26.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:26 smithi096 ceph-mon[36392]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:26.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:26 smithi084 ceph-mon[32186]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:26.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:26 smithi204 ceph-mon[36211]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:28.542 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:28 smithi096 ceph-mon[36392]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:28.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:28 smithi084 ceph-mon[32186]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:28.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:28 smithi204 ceph-mon[36211]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:30.677 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:30 smithi096 ceph-mon[36392]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:30.677 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:30 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-22T22:54:30.677 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:30 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:30.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:30 smithi084 ceph-mon[32186]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:30.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:30 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-22T22:54:30.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:30 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:30.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:30 smithi204 ceph-mon[36211]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:30.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:30 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-22T22:54:30.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:30 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:31.792 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:31 smithi096 ceph-mon[36392]: Deploying daemon osd.1 on smithi096 2024-05-22T22:54:31.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:31 smithi084 ceph-mon[32186]: Deploying daemon osd.1 on smithi096 2024-05-22T22:54:31.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:31 smithi204 ceph-mon[36211]: Deploying daemon osd.1 on smithi096 2024-05-22T22:54:32.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:32 smithi096 ceph-mon[36392]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:32.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:32 smithi084 ceph-mon[32186]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:32.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:32 smithi204 ceph-mon[36211]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:34.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:34 smithi096 ceph-mon[36392]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:34.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:34 smithi084 ceph-mon[32186]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:34.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:34 smithi204 ceph-mon[36211]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:36.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:35 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:36.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:35 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:36.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:35 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:54:36.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:35 smithi204 ceph-mon[36211]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:36.293 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:35 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:36.293 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:35 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:36.293 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:35 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:54:36.293 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:35 smithi096 ceph-mon[36392]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:36.368 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:35 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:36.368 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:35 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:36.369 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:35 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:54:36.369 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:35 smithi084 ceph-mon[32186]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:38.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:38.437+0000 7fef157fa700 1 -- 172.21.15.96:0/1841936168 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fef180627a0 con 0x7fef000a2b90 2024-05-22T22:54:38.432 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 1 on host 'smithi096' 2024-05-22T22:54:38.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:38.439+0000 7fef1e9e3700 1 -- 172.21.15.96:0/1841936168 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fef000a2b90 msgr2=0x7fef000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:54:38.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:38.439+0000 7fef1e9e3700 1 --2- 172.21.15.96:0/1841936168 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fef000a2b90 0x7fef000a5050 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fef08006fd0 tx=0x7fef08008040 comp rx=0 tx=0).stop 2024-05-22T22:54:38.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:38.439+0000 7fef1e9e3700 1 -- 172.21.15.96:0/1841936168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef181080d0 msgr2=0x7fef18101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:54:38.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:38.439+0000 7fef1e9e3700 1 --2- 172.21.15.96:0/1841936168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef181080d0 0x7fef18101360 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fef1000b9e0 tx=0x7fef1000bda0 comp rx=0 tx=0).stop 2024-05-22T22:54:38.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:38.439+0000 7fef1e9e3700 1 -- 172.21.15.96:0/1841936168 shutdown_connections 2024-05-22T22:54:38.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:38.439+0000 7fef1e9e3700 1 --2- 172.21.15.96:0/1841936168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef181080d0 0x7fef18101360 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:54:38.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:38.439+0000 7fef1e9e3700 1 --2- 172.21.15.96:0/1841936168 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fef000a2b90 0x7fef000a5050 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:54:38.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:38.439+0000 7fef1e9e3700 1 --2- 172.21.15.96:0/1841936168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef18102e70 0x7fef181027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:54:38.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:38.439+0000 7fef1e9e3700 1 --2- 172.21.15.96:0/1841936168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef181057a0 0x7fef18100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:54:38.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:38.439+0000 7fef1e9e3700 1 -- 172.21.15.96:0/1841936168 >> 172.21.15.96:0/1841936168 conn(0x7fef180fc8c0 msgr2=0x7fef181037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:54:38.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:38.439+0000 7fef1e9e3700 1 -- 172.21.15.96:0/1841936168 shutdown_connections 2024-05-22T22:54:38.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:54:38.439+0000 7fef1e9e3700 1 -- 172.21.15.96:0/1841936168 wait complete. 2024-05-22T22:54:38.755 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:38 smithi084 ceph-mon[32186]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:38.755 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:38 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:38.756 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:38 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:38.756 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:38 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:38.756 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:38 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:54:38.756 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:38 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:38.756 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:38 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:38.757 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:38 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:38.757 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:38 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:54:38.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:38 smithi096 ceph-mon[36392]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:38.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:38 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:38.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:38 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:38.783 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:38 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:38.783 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:38 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:54:38.783 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:38 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:38.783 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:38 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:38.783 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:38 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:38.783 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:38 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:54:38.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:38 smithi204 ceph-mon[36211]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:38.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:38 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:38.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:38 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:38.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:38 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:38.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:38 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:54:38.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:38 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:38.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:38 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:38.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:38 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:38.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:38 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:54:39.134 DEBUG:teuthology.orchestra.run.smithi096:osd.1> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.1.service 2024-05-22T22:54:39.177 INFO:tasks.cephadm:Deploying osd.2 on smithi204 with /dev/vg_nvme/lv_4... 2024-05-22T22:54:39.177 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-22T22:54:39.422 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.c/config 2024-05-22T22:54:39.694 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:39 smithi096 ceph-mon[36392]: from='osd.1 [v2:172.21.15.96:6800/2996505353,v1:172.21.15.96:6801/2996505353]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-22T22:54:39.694 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:39 smithi096 ceph-mon[36392]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-22T22:54:39.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:39 smithi204 ceph-mon[36211]: from='osd.1 [v2:172.21.15.96:6800/2996505353,v1:172.21.15.96:6801/2996505353]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-22T22:54:39.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:39 smithi204 ceph-mon[36211]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-22T22:54:39.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:39 smithi084 ceph-mon[32186]: from='osd.1 [v2:172.21.15.96:6800/2996505353,v1:172.21.15.96:6801/2996505353]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-22T22:54:39.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:39 smithi084 ceph-mon[32186]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-22T22:54:40.730 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:40 smithi204 ceph-mon[36211]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:40.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:40 smithi204 ceph-mon[36211]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-22T22:54:40.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:40 smithi204 ceph-mon[36211]: from='osd.1 [v2:172.21.15.96:6800/2996505353,v1:172.21.15.96:6801/2996505353]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-22T22:54:40.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:40 smithi204 ceph-mon[36211]: osdmap e11: 2 total, 1 up, 2 in 2024-05-22T22:54:40.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:40 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-22T22:54:40.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:40 smithi204 ceph-mon[36211]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-22T22:54:40.759 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:40 smithi096 ceph-mon[36392]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:40.759 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:40 smithi096 ceph-mon[36392]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-22T22:54:40.759 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:40 smithi096 ceph-mon[36392]: from='osd.1 [v2:172.21.15.96:6800/2996505353,v1:172.21.15.96:6801/2996505353]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-22T22:54:40.760 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:40 smithi096 ceph-mon[36392]: osdmap e11: 2 total, 1 up, 2 in 2024-05-22T22:54:40.760 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:40 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-22T22:54:40.760 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:40 smithi096 ceph-mon[36392]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-22T22:54:40.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:40 smithi084 ceph-mon[32186]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:40.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:40 smithi084 ceph-mon[32186]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-22T22:54:40.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:40 smithi084 ceph-mon[32186]: from='osd.1 [v2:172.21.15.96:6800/2996505353,v1:172.21.15.96:6801/2996505353]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-22T22:54:40.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:40 smithi084 ceph-mon[32186]: osdmap e11: 2 total, 1 up, 2 in 2024-05-22T22:54:40.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:40 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-22T22:54:40.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:40 smithi084 ceph-mon[32186]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-22T22:54:41.043 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 22 22:54:40 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-1[40931]: 2024-05-22T22:54:40.934+0000 7f4f2b20c700 -1 osd.1 0 waiting for initial osdmap 2024-05-22T22:54:41.043 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 22 22:54:40 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-1[40931]: 2024-05-22T22:54:40.937+0000 7f4f22ff7700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-22T22:54:41.281 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-22T22:54:41.281 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-22T22:54:41.281 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-05-22T22:54:41.282 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-05-22T22:54:41.282 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011501 s, 912 MB/s 2024-05-22T22:54:41.282 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-05-22T22:54:41.712 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:41 smithi096 ceph-mon[36392]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-22T22:54:41.712 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:41 smithi096 ceph-mon[36392]: osdmap e12: 2 total, 1 up, 2 in 2024-05-22T22:54:41.712 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:41 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-22T22:54:41.712 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:41 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-22T22:54:41.712 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:41 smithi096 ceph-mon[36392]: from='osd.1 ' entity='osd.1' 2024-05-22T22:54:41.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:41 smithi084 ceph-mon[32186]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-22T22:54:41.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:41 smithi084 ceph-mon[32186]: osdmap e12: 2 total, 1 up, 2 in 2024-05-22T22:54:41.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:41 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-22T22:54:41.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:41 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-22T22:54:41.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:41 smithi084 ceph-mon[32186]: from='osd.1 ' entity='osd.1' 2024-05-22T22:54:41.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:41 smithi204 ceph-mon[36211]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-22T22:54:41.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:41 smithi204 ceph-mon[36211]: osdmap e12: 2 total, 1 up, 2 in 2024-05-22T22:54:41.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:41 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-22T22:54:41.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:41 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-22T22:54:41.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:41 smithi204 ceph-mon[36211]: from='osd.1 ' entity='osd.1' 2024-05-22T22:54:41.952 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_4 2024-05-22T22:54:42.194 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.c/config 2024-05-22T22:54:42.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:42 smithi204 ceph-mon[36211]: purged_snaps scrub starts 2024-05-22T22:54:42.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:42 smithi204 ceph-mon[36211]: purged_snaps scrub ok 2024-05-22T22:54:42.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:42 smithi204 ceph-mon[36211]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:42.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:42 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-22T22:54:42.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:42 smithi204 ceph-mon[36211]: osd.1 [v2:172.21.15.96:6800/2996505353,v1:172.21.15.96:6801/2996505353] boot 2024-05-22T22:54:42.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:42 smithi204 ceph-mon[36211]: osdmap e13: 2 total, 2 up, 2 in 2024-05-22T22:54:42.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:42 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-22T22:54:42.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:42 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:42.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:42 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:42.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:42 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:54:42.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:42 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:42.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:42 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:42.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:42 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:54:42.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:42 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:42.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:42 smithi096 ceph-mon[36392]: purged_snaps scrub starts 2024-05-22T22:54:42.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:42 smithi096 ceph-mon[36392]: purged_snaps scrub ok 2024-05-22T22:54:42.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:42 smithi096 ceph-mon[36392]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:42.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:42 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-22T22:54:42.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:42 smithi096 ceph-mon[36392]: osd.1 [v2:172.21.15.96:6800/2996505353,v1:172.21.15.96:6801/2996505353] boot 2024-05-22T22:54:42.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:42 smithi096 ceph-mon[36392]: osdmap e13: 2 total, 2 up, 2 in 2024-05-22T22:54:42.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:42 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-22T22:54:42.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:42 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:42.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:42 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:42.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:42 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:54:42.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:42 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:42.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:42 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:42.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:42 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:54:42.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:42 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:42.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:42 smithi084 ceph-mon[32186]: purged_snaps scrub starts 2024-05-22T22:54:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:42 smithi084 ceph-mon[32186]: purged_snaps scrub ok 2024-05-22T22:54:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:42 smithi084 ceph-mon[32186]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-22T22:54:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:42 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-22T22:54:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:42 smithi084 ceph-mon[32186]: osd.1 [v2:172.21.15.96:6800/2996505353,v1:172.21.15.96:6801/2996505353] boot 2024-05-22T22:54:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:42 smithi084 ceph-mon[32186]: osdmap e13: 2 total, 2 up, 2 in 2024-05-22T22:54:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:42 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-22T22:54:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:42 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:42 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:42 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:54:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:42 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:42 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:42 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:54:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:42 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:54:43.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.683+0000 7efe499fa700 1 -- 172.21.15.204:0/1716034843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe44102e60 msgr2=0x7efe44109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:54:43.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.683+0000 7efe499fa700 1 --2- 172.21.15.204:0/1716034843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe44102e60 0x7efe44109710 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00b0a0 tx=0x7efe3c00b3b0 comp rx=0 tx=0).stop 2024-05-22T22:54:43.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.683+0000 7efe499fa700 1 -- 172.21.15.204:0/1716034843 shutdown_connections 2024-05-22T22:54:43.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.683+0000 7efe499fa700 1 --2- 172.21.15.204:0/1716034843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe44102e60 0x7efe44109710 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:54:43.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.683+0000 7efe499fa700 1 --2- 172.21.15.204:0/1716034843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe441012c0 0x7efe441016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:54:43.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.683+0000 7efe499fa700 1 --2- 172.21.15.204:0/1716034843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe441024c0 0x7efe44102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:54:43.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.683+0000 7efe499fa700 1 -- 172.21.15.204:0/1716034843 >> 172.21.15.204:0/1716034843 conn(0x7efe440fc860 msgr2=0x7efe440feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:54:43.680 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.683+0000 7efe499fa700 1 -- 172.21.15.204:0/1716034843 shutdown_connections 2024-05-22T22:54:43.680 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.683+0000 7efe499fa700 1 -- 172.21.15.204:0/1716034843 wait complete. 2024-05-22T22:54:43.680 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.683+0000 7efe499fa700 1 Processor -- start 2024-05-22T22:54:43.680 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.683+0000 7efe499fa700 1 -- start start 2024-05-22T22:54:43.680 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.684+0000 7efe499fa700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe441012c0 0x7efe44105250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:54:43.680 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.684+0000 7efe499fa700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe441024c0 0x7efe441038a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:54:43.680 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.684+0000 7efe499fa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe44102e60 0x7efe44103de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:54:43.680 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.684+0000 7efe499fa700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efe4410cf80 con 0x7efe441024c0 2024-05-22T22:54:43.680 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.684+0000 7efe499fa700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efe4410ce00 con 0x7efe44102e60 2024-05-22T22:54:43.680 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.684+0000 7efe499fa700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efe4410d100 con 0x7efe441012c0 2024-05-22T22:54:43.680 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.684+0000 7efe42ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe441012c0 0x7efe44105250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:54:43.680 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.684+0000 7efe42ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe441012c0 0x7efe44105250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36764/0 (socket says 172.21.15.204:36764) 2024-05-22T22:54:43.681 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.684+0000 7efe42ffd700 1 -- 172.21.15.204:0/2477511104 learned_addr learned my addr 172.21.15.204:0/2477511104 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-22T22:54:43.681 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.684+0000 7efe437fe700 1 --2- 172.21.15.204:0/2477511104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe44102e60 0x7efe44103de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:54:43.681 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.684+0000 7efe42ffd700 1 -- 172.21.15.204:0/2477511104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe44102e60 msgr2=0x7efe44103de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:54:43.681 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.684+0000 7efe42ffd700 1 --2- 172.21.15.204:0/2477511104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe44102e60 0x7efe44103de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:54:43.681 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.684+0000 7efe427fc700 1 --2- 172.21.15.204:0/2477511104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe441024c0 0x7efe441038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:54:43.681 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.684+0000 7efe42ffd700 1 -- 172.21.15.204:0/2477511104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe441024c0 msgr2=0x7efe441038a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:54:43.681 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.684+0000 7efe42ffd700 1 --2- 172.21.15.204:0/2477511104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe441024c0 0x7efe441038a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:54:43.681 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.684+0000 7efe42ffd700 1 -- 172.21.15.204:0/2477511104 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe44104690 con 0x7efe441012c0 2024-05-22T22:54:43.681 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.684+0000 7efe42ffd700 1 --2- 172.21.15.204:0/2477511104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe441012c0 0x7efe44105250 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efe3400b500 tx=0x7efe3400b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:54:43.681 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.684+0000 7efe489f8700 1 -- 172.21.15.204:0/2477511104 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe34015020 con 0x7efe441012c0 2024-05-22T22:54:43.681 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.684+0000 7efe489f8700 1 -- 172.21.15.204:0/2477511104 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe34011d10 con 0x7efe441012c0 2024-05-22T22:54:43.681 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.684+0000 7efe499fa700 1 -- 172.21.15.204:0/2477511104 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe441a27e0 con 0x7efe441012c0 2024-05-22T22:54:43.681 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.684+0000 7efe489f8700 1 -- 172.21.15.204:0/2477511104 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3400f2e0 con 0x7efe441012c0 2024-05-22T22:54:43.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.684+0000 7efe499fa700 1 -- 172.21.15.204:0/2477511104 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe441a2d70 con 0x7efe441012c0 2024-05-22T22:54:43.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.685+0000 7efe499fa700 1 -- 172.21.15.204:0/2477511104 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe44069050 con 0x7efe441012c0 2024-05-22T22:54:43.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.686+0000 7efe489f8700 1 -- 172.21.15.204:0/2477511104 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7efe34018020 con 0x7efe441012c0 2024-05-22T22:54:43.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.687+0000 7efe489f8700 1 --2- 172.21.15.204:0/2477511104 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7efe2c0a2c10 0x7efe2c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:54:43.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.687+0000 7efe489f8700 1 -- 172.21.15.204:0/2477511104 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7efe340ccf80 con 0x7efe441012c0 2024-05-22T22:54:43.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.687+0000 7efe427fc700 1 --2- 172.21.15.204:0/2477511104 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7efe2c0a2c10 0x7efe2c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:54:43.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.688+0000 7efe427fc700 1 --2- 172.21.15.204:0/2477511104 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7efe2c0a2c10 0x7efe2c0a50d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efe3800ba60 tx=0x7efe38005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:54:43.686 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.692+0000 7efe489f8700 1 -- 172.21.15.204:0/2477511104 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efe3401f020 con 0x7efe441012c0 2024-05-22T22:54:43.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:43 smithi096 ceph-mon[36392]: Detected new or changed devices on smithi096 2024-05-22T22:54:43.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:43 smithi096 ceph-mon[36392]: Adjusting osd_memory_target on smithi096 to 17088M 2024-05-22T22:54:43.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:43 smithi084 ceph-mon[32186]: Detected new or changed devices on smithi096 2024-05-22T22:54:43.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:43 smithi084 ceph-mon[32186]: Adjusting osd_memory_target on smithi096 to 17088M 2024-05-22T22:54:43.890 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:54:43.895+0000 7efe499fa700 1 -- 172.21.15.204:0/2477511104 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7efe441a2fd0 con 0x7efe2c0a2c10 2024-05-22T22:54:43.890 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:43 smithi204 ceph-mon[36211]: Detected new or changed devices on smithi096 2024-05-22T22:54:43.890 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:43 smithi204 ceph-mon[36211]: Adjusting osd_memory_target on smithi096 to 17088M 2024-05-22T22:54:44.785 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:44 smithi204 ceph-mon[36211]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:54:44.785 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:44 smithi204 ceph-mon[36211]: osdmap e14: 2 total, 2 up, 2 in 2024-05-22T22:54:44.785 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:44 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:54:44.785 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:44 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:54:44.785 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:44 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:44.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:44 smithi096 ceph-mon[36392]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:54:44.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:44 smithi096 ceph-mon[36392]: osdmap e14: 2 total, 2 up, 2 in 2024-05-22T22:54:44.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:54:44.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:54:44.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:44.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:44 smithi084 ceph-mon[32186]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:54:44.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:44 smithi084 ceph-mon[32186]: osdmap e14: 2 total, 2 up, 2 in 2024-05-22T22:54:44.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:44 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:54:44.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:44 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:54:44.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:44 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:45.532 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:45 smithi204 ceph-mon[36211]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:54:45.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:45 smithi096 ceph-mon[36392]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:54:45.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:45 smithi084 ceph-mon[32186]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:54:46.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:46 smithi096 ceph-mon[36392]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:54:46.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:46 smithi096 ceph-mon[36392]: from='client.? 172.21.15.204:0/2764203345' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e306e88e-a308-4585-ae11-c8af10b84038"}]: dispatch 2024-05-22T22:54:46.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:46 smithi096 ceph-mon[36392]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e306e88e-a308-4585-ae11-c8af10b84038"}]: dispatch 2024-05-22T22:54:46.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:46 smithi096 ceph-mon[36392]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e306e88e-a308-4585-ae11-c8af10b84038"}]': finished 2024-05-22T22:54:46.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:46 smithi096 ceph-mon[36392]: osdmap e15: 3 total, 2 up, 3 in 2024-05-22T22:54:46.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:46 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-22T22:54:46.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:46 smithi084 ceph-mon[32186]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:54:46.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:46 smithi084 ceph-mon[32186]: from='client.? 172.21.15.204:0/2764203345' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e306e88e-a308-4585-ae11-c8af10b84038"}]: dispatch 2024-05-22T22:54:46.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:46 smithi084 ceph-mon[32186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e306e88e-a308-4585-ae11-c8af10b84038"}]: dispatch 2024-05-22T22:54:46.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:46 smithi084 ceph-mon[32186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e306e88e-a308-4585-ae11-c8af10b84038"}]': finished 2024-05-22T22:54:46.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:46 smithi084 ceph-mon[32186]: osdmap e15: 3 total, 2 up, 3 in 2024-05-22T22:54:46.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:46 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-22T22:54:46.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:46 smithi204 ceph-mon[36211]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:54:46.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:46 smithi204 ceph-mon[36211]: from='client.? 172.21.15.204:0/2764203345' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e306e88e-a308-4585-ae11-c8af10b84038"}]: dispatch 2024-05-22T22:54:46.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:46 smithi204 ceph-mon[36211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e306e88e-a308-4585-ae11-c8af10b84038"}]: dispatch 2024-05-22T22:54:46.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:46 smithi204 ceph-mon[36211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e306e88e-a308-4585-ae11-c8af10b84038"}]': finished 2024-05-22T22:54:46.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:46 smithi204 ceph-mon[36211]: osdmap e15: 3 total, 2 up, 3 in 2024-05-22T22:54:46.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:46 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-22T22:54:47.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:47 smithi096 ceph-mon[36392]: from='client.? 172.21.15.204:0/2545137974' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:54:47.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:47 smithi084 ceph-mon[32186]: from='client.? 172.21.15.204:0/2545137974' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:54:47.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:47 smithi204 ceph-mon[36211]: from='client.? 172.21.15.204:0/2545137974' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:54:48.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:48 smithi096 ceph-mon[36392]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:54:48.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:48 smithi084 ceph-mon[32186]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:54:48.871 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:48 smithi204 ceph-mon[36211]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:54:50.572 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:50 smithi204 ceph-mon[36211]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:54:50.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:50 smithi096 ceph-mon[36392]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:54:50.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:50 smithi084 ceph-mon[32186]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:54:52.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:52 smithi096 ceph-mon[36392]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:54:52.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:52 smithi204 ceph-mon[36211]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:54:52.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:52 smithi084 ceph-mon[32186]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:54:53.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:53 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-22T22:54:53.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:53 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:53.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:53 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-22T22:54:53.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:53 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:53.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:53 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-22T22:54:53.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:53 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:54:54.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:54 smithi204 ceph-mon[36211]: Deploying daemon osd.2 on smithi204 2024-05-22T22:54:54.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:54 smithi204 ceph-mon[36211]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:54:54.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:54 smithi096 ceph-mon[36392]: Deploying daemon osd.2 on smithi204 2024-05-22T22:54:54.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:54 smithi096 ceph-mon[36392]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:54:54.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:54 smithi084 ceph-mon[32186]: Deploying daemon osd.2 on smithi204 2024-05-22T22:54:54.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:54 smithi084 ceph-mon[32186]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:54:56.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:56 smithi204 ceph-mon[36211]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:54:56.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:56 smithi096 ceph-mon[36392]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:54:56.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:56 smithi084 ceph-mon[32186]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:54:58.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:58 smithi096 ceph-mon[36392]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:54:58.868 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:58 smithi084 ceph-mon[32186]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:54:58.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:58 smithi204 ceph-mon[36211]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:55:00.043 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:59 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:00.043 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:59 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:00.043 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:59 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:55:00.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:54:59 smithi096 ceph-mon[36392]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:55:00.118 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:59 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:00.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:59 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:00.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:59 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:55:00.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:54:59 smithi084 ceph-mon[32186]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:55:00.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:59 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:00.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:59 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:00.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:59 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:55:00.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:54:59 smithi204 ceph-mon[36211]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:55:02.257 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:02.261+0000 7efe489f8700 1 -- 172.21.15.204:0/2477511104 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efe441a2fd0 con 0x7efe2c0a2c10 2024-05-22T22:55:02.257 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 2 on host 'smithi204' 2024-05-22T22:55:02.258 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:02.263+0000 7efe499fa700 1 -- 172.21.15.204:0/2477511104 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7efe2c0a2c10 msgr2=0x7efe2c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:55:02.258 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:02.263+0000 7efe499fa700 1 --2- 172.21.15.204:0/2477511104 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7efe2c0a2c10 0x7efe2c0a50d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efe3800ba60 tx=0x7efe38005d50 comp rx=0 tx=0).stop 2024-05-22T22:55:02.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:02.263+0000 7efe499fa700 1 -- 172.21.15.204:0/2477511104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe441012c0 msgr2=0x7efe44105250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:55:02.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:02.263+0000 7efe499fa700 1 --2- 172.21.15.204:0/2477511104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe441012c0 0x7efe44105250 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efe3400b500 tx=0x7efe3400b8c0 comp rx=0 tx=0).stop 2024-05-22T22:55:02.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:02.263+0000 7efe499fa700 1 -- 172.21.15.204:0/2477511104 shutdown_connections 2024-05-22T22:55:02.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:02.263+0000 7efe499fa700 1 --2- 172.21.15.204:0/2477511104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe44102e60 0x7efe44103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:02.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:02.263+0000 7efe499fa700 1 --2- 172.21.15.204:0/2477511104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe441012c0 0x7efe44105250 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:02.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:02.263+0000 7efe499fa700 1 --2- 172.21.15.204:0/2477511104 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7efe2c0a2c10 0x7efe2c0a50d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:02.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:02.263+0000 7efe499fa700 1 --2- 172.21.15.204:0/2477511104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe441024c0 0x7efe441038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:02.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:02.263+0000 7efe499fa700 1 -- 172.21.15.204:0/2477511104 >> 172.21.15.204:0/2477511104 conn(0x7efe440fc860 msgr2=0x7efe440fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:55:02.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:02.263+0000 7efe499fa700 1 -- 172.21.15.204:0/2477511104 shutdown_connections 2024-05-22T22:55:02.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:02.263+0000 7efe499fa700 1 -- 172.21.15.204:0/2477511104 wait complete. 2024-05-22T22:55:02.572 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:02 smithi204 ceph-mon[36211]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:55:02.573 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:02 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:02.573 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:02 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:02.573 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:02 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:02.573 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:02 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:55:02.573 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:02 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:02.573 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:02 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:02.573 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:02 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:02.573 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:02 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:55:02.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:02 smithi096 ceph-mon[36392]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:55:02.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:02 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:02.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:02 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:02.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:02 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:02.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:02 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:55:02.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:02 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:02.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:02 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:02.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:02 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:02.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:02 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:55:02.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:02 smithi084 ceph-mon[32186]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:55:02.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:02 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:02.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:02 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:02.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:02 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:02.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:02 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:55:02.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:02 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:02.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:02 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:02.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:02 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:02.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:02 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:55:02.898 DEBUG:teuthology.orchestra.run.smithi204:osd.2> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.2.service 2024-05-22T22:55:02.901 INFO:tasks.cephadm:Deploying osd.3 on smithi084 with /dev/vg_nvme/lv_3... 2024-05-22T22:55:02.901 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-22T22:55:03.146 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T22:55:03.524 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:03 smithi084 ceph-mon[32186]: from='osd.2 [v2:172.21.15.204:6800/1499577897,v1:172.21.15.204:6801/1499577897]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-22T22:55:03.524 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:03 smithi084 ceph-mon[32186]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-22T22:55:03.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:03 smithi096 ceph-mon[36392]: from='osd.2 [v2:172.21.15.204:6800/1499577897,v1:172.21.15.204:6801/1499577897]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-22T22:55:03.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:03 smithi096 ceph-mon[36392]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-22T22:55:03.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:03 smithi204 ceph-mon[36211]: from='osd.2 [v2:172.21.15.204:6800/1499577897,v1:172.21.15.204:6801/1499577897]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-22T22:55:03.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:03 smithi204 ceph-mon[36211]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-22T22:55:04.591 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:04 smithi084 ceph-mon[32186]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:55:04.591 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:04 smithi084 ceph-mon[32186]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-22T22:55:04.592 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:04 smithi084 ceph-mon[32186]: osdmap e16: 3 total, 2 up, 3 in 2024-05-22T22:55:04.592 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:04 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-22T22:55:04.592 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:04 smithi084 ceph-mon[32186]: from='osd.2 [v2:172.21.15.204:6800/1499577897,v1:172.21.15.204:6801/1499577897]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-22T22:55:04.592 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:04 smithi084 ceph-mon[32186]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-22T22:55:04.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:04 smithi096 ceph-mon[36392]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:55:04.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:04 smithi096 ceph-mon[36392]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-22T22:55:04.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:04 smithi096 ceph-mon[36392]: osdmap e16: 3 total, 2 up, 3 in 2024-05-22T22:55:04.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:04 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-22T22:55:04.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:04 smithi096 ceph-mon[36392]: from='osd.2 [v2:172.21.15.204:6800/1499577897,v1:172.21.15.204:6801/1499577897]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-22T22:55:04.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:04 smithi096 ceph-mon[36392]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-22T22:55:04.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:04 smithi204 ceph-mon[36211]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:55:04.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:04 smithi204 ceph-mon[36211]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-22T22:55:04.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:04 smithi204 ceph-mon[36211]: osdmap e16: 3 total, 2 up, 3 in 2024-05-22T22:55:04.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:04 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-22T22:55:04.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:04 smithi204 ceph-mon[36211]: from='osd.2 [v2:172.21.15.204:6800/1499577897,v1:172.21.15.204:6801/1499577897]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-22T22:55:04.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:04 smithi204 ceph-mon[36211]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-22T22:55:05.177 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-22T22:55:05.177 INFO:teuthology.orchestra.run.smithi084.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-22T22:55:05.177 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10+0 records in 2024-05-22T22:55:05.177 INFO:teuthology.orchestra.run.smithi084.stderr:10+0 records out 2024-05-22T22:55:05.177 INFO:teuthology.orchestra.run.smithi084.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111575 s, 940 MB/s 2024-05-22T22:55:05.177 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping successful for: 2024-05-22T22:55:05.405 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 22 22:55:04 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-2[41331]: 2024-05-22T22:55:04.948+0000 7f59dc10a700 -1 osd.2 0 waiting for initial osdmap 2024-05-22T22:55:05.405 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 22 22:55:04 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-2[41331]: 2024-05-22T22:55:04.953+0000 7f59d36f4700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-22T22:55:05.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:05 smithi096 ceph-mon[36392]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-22T22:55:05.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:05 smithi096 ceph-mon[36392]: osdmap e17: 3 total, 2 up, 3 in 2024-05-22T22:55:05.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:05 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-22T22:55:05.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:05 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-22T22:55:05.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:05 smithi096 ceph-mon[36392]: from='osd.2 ' entity='osd.2' 2024-05-22T22:55:05.818 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:05 smithi084 ceph-mon[32186]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-22T22:55:05.819 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:05 smithi084 ceph-mon[32186]: osdmap e17: 3 total, 2 up, 3 in 2024-05-22T22:55:05.819 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:05 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-22T22:55:05.819 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:05 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-22T22:55:05.819 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:05 smithi084 ceph-mon[32186]: from='osd.2 ' entity='osd.2' 2024-05-22T22:55:05.829 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:05 smithi204 ceph-mon[36211]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-22T22:55:05.829 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:05 smithi204 ceph-mon[36211]: osdmap e17: 3 total, 2 up, 3 in 2024-05-22T22:55:05.829 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:05 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-22T22:55:05.829 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:05 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-22T22:55:05.829 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:05 smithi204 ceph-mon[36211]: from='osd.2 ' entity='osd.2' 2024-05-22T22:55:05.849 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi084:vg_nvme/lv_3 2024-05-22T22:55:06.099 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T22:55:06.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:06 smithi096 ceph-mon[36392]: purged_snaps scrub starts 2024-05-22T22:55:06.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:06 smithi096 ceph-mon[36392]: purged_snaps scrub ok 2024-05-22T22:55:06.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:06 smithi096 ceph-mon[36392]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:55:06.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-22T22:55:06.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:06.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:06.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:55:06.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:06.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:06.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:55:06.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:06.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:06 smithi096 ceph-mon[36392]: osd.2 [v2:172.21.15.204:6800/1499577897,v1:172.21.15.204:6801/1499577897] boot 2024-05-22T22:55:06.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:06 smithi096 ceph-mon[36392]: osdmap e18: 3 total, 3 up, 3 in 2024-05-22T22:55:06.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-22T22:55:06.859 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:06 smithi084 ceph-mon[32186]: purged_snaps scrub starts 2024-05-22T22:55:06.859 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:06 smithi084 ceph-mon[32186]: purged_snaps scrub ok 2024-05-22T22:55:06.859 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:06 smithi084 ceph-mon[32186]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:55:06.860 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-22T22:55:06.860 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:06.860 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:06.860 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:55:06.860 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:06.860 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:06.860 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:55:06.860 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:06.860 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:06 smithi084 ceph-mon[32186]: osd.2 [v2:172.21.15.204:6800/1499577897,v1:172.21.15.204:6801/1499577897] boot 2024-05-22T22:55:06.860 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:06 smithi084 ceph-mon[32186]: osdmap e18: 3 total, 3 up, 3 in 2024-05-22T22:55:06.860 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-22T22:55:06.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:06 smithi204 ceph-mon[36211]: purged_snaps scrub starts 2024-05-22T22:55:06.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:06 smithi204 ceph-mon[36211]: purged_snaps scrub ok 2024-05-22T22:55:06.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:06 smithi204 ceph-mon[36211]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-22T22:55:06.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-22T22:55:06.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:55:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:55:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:06 smithi204 ceph-mon[36211]: osd.2 [v2:172.21.15.204:6800/1499577897,v1:172.21.15.204:6801/1499577897] boot 2024-05-22T22:55:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:06 smithi204 ceph-mon[36211]: osdmap e18: 3 total, 3 up, 3 in 2024-05-22T22:55:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-22T22:55:07.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.751+0000 7ff9053f0700 1 -- 172.21.15.84:0/1107961016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9001024c0 msgr2=0x7ff900102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:55:07.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.751+0000 7ff9053f0700 1 --2- 172.21.15.84:0/1107961016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9001024c0 0x7ff900102920 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40099c0 tx=0x7ff8f4009cd0 comp rx=0 tx=0).stop 2024-05-22T22:55:07.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.751+0000 7ff9053f0700 1 -- 172.21.15.84:0/1107961016 shutdown_connections 2024-05-22T22:55:07.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.751+0000 7ff9053f0700 1 --2- 172.21.15.84:0/1107961016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff900102e60 0x7ff900109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:07.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.751+0000 7ff9053f0700 1 --2- 172.21.15.84:0/1107961016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9001012c0 0x7ff9001016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:07.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.751+0000 7ff9053f0700 1 --2- 172.21.15.84:0/1107961016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9001024c0 0x7ff900102920 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:07.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.751+0000 7ff9053f0700 1 -- 172.21.15.84:0/1107961016 >> 172.21.15.84:0/1107961016 conn(0x7ff9000fc880 msgr2=0x7ff9000feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:55:07.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.752+0000 7ff9053f0700 1 -- 172.21.15.84:0/1107961016 shutdown_connections 2024-05-22T22:55:07.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.752+0000 7ff9053f0700 1 -- 172.21.15.84:0/1107961016 wait complete. 2024-05-22T22:55:07.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.752+0000 7ff9053f0700 1 Processor -- start 2024-05-22T22:55:07.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.752+0000 7ff9053f0700 1 -- start start 2024-05-22T22:55:07.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.752+0000 7ff9053f0700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9001012c0 0x7ff90006b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:55:07.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.752+0000 7ff9053f0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9001024c0 0x7ff90019d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:55:07.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.752+0000 7ff9053f0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff900102e60 0x7ff90019f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:55:07.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.753+0000 7ff9053f0700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff90010cea0 con 0x7ff9001012c0 2024-05-22T22:55:07.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.753+0000 7ff9053f0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff90010cd20 con 0x7ff9001024c0 2024-05-22T22:55:07.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.753+0000 7ff9053f0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff90010d020 con 0x7ff900102e60 2024-05-22T22:55:07.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.753+0000 7ff8feffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9001012c0 0x7ff90006b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:55:07.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.753+0000 7ff8feffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9001012c0 0x7ff90006b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44864/0 (socket says 172.21.15.84:44864) 2024-05-22T22:55:07.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.753+0000 7ff8feffd700 1 -- 172.21.15.84:0/4167373336 learned_addr learned my addr 172.21.15.84:0/4167373336 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:55:07.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.753+0000 7ff8ff7fe700 1 --2- 172.21.15.84:0/4167373336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff900102e60 0x7ff90019f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:55:07.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.753+0000 7ff8fe7fc700 1 --2- 172.21.15.84:0/4167373336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9001024c0 0x7ff90019d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:55:07.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.753+0000 7ff8feffd700 1 -- 172.21.15.84:0/4167373336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff900102e60 msgr2=0x7ff90019f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:55:07.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.753+0000 7ff8feffd700 1 --2- 172.21.15.84:0/4167373336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff900102e60 0x7ff90019f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:07.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.753+0000 7ff8feffd700 1 -- 172.21.15.84:0/4167373336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9001024c0 msgr2=0x7ff90019d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:55:07.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.753+0000 7ff8feffd700 1 --2- 172.21.15.84:0/4167373336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9001024c0 0x7ff90019d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:07.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.753+0000 7ff8feffd700 1 -- 172.21.15.84:0/4167373336 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9001a0200 con 0x7ff9001012c0 2024-05-22T22:55:07.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.753+0000 7ff8feffd700 1 --2- 172.21.15.84:0/4167373336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9001012c0 0x7ff90006b630 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff8f000c970 tx=0x7ff8f000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:55:07.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.753+0000 7ff8e7fff700 1 -- 172.21.15.84:0/4167373336 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0004220 con 0x7ff9001012c0 2024-05-22T22:55:07.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.753+0000 7ff9053f0700 1 -- 172.21.15.84:0/4167373336 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9001a04f0 con 0x7ff9001012c0 2024-05-22T22:55:07.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.753+0000 7ff8e7fff700 1 -- 172.21.15.84:0/4167373336 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8f0007d80 con 0x7ff9001012c0 2024-05-22T22:55:07.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.753+0000 7ff8e7fff700 1 -- 172.21.15.84:0/4167373336 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0003e50 con 0x7ff9001012c0 2024-05-22T22:55:07.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.753+0000 7ff9053f0700 1 -- 172.21.15.84:0/4167373336 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9001a0a00 con 0x7ff9001012c0 2024-05-22T22:55:07.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.754+0000 7ff9053f0700 1 -- 172.21.15.84:0/4167373336 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff900069050 con 0x7ff9001012c0 2024-05-22T22:55:07.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.756+0000 7ff8e7fff700 1 -- 172.21.15.84:0/4167373336 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7ff8f00078a0 con 0x7ff9001012c0 2024-05-22T22:55:07.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.756+0000 7ff8e7fff700 1 --2- 172.21.15.84:0/4167373336 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7ff8e80a2d30 0x7ff8e80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:55:07.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.756+0000 7ff8fe7fc700 1 --2- 172.21.15.84:0/4167373336 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7ff8e80a2d30 0x7ff8e80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:55:07.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.756+0000 7ff8e7fff700 1 -- 172.21.15.84:0/4167373336 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7ff8f00cdaa0 con 0x7ff9001012c0 2024-05-22T22:55:07.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.757+0000 7ff8fe7fc700 1 --2- 172.21.15.84:0/4167373336 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7ff8e80a2d30 0x7ff8e80a51f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4000c00 tx=0x7ff8f4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:55:07.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.760+0000 7ff8e7fff700 1 -- 172.21.15.84:0/4167373336 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff8f0014030 con 0x7ff9001012c0 2024-05-22T22:55:07.755 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:07 smithi084 ceph-mon[32186]: Detected new or changed devices on smithi204 2024-05-22T22:55:07.755 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:07 smithi084 ceph-mon[32186]: Adjusting osd_memory_target on smithi204 to 17088M 2024-05-22T22:55:07.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:07.964+0000 7ff9053f0700 1 -- 172.21.15.84:0/4167373336 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff9000627a0 con 0x7ff8e80a2d30 2024-05-22T22:55:08.043 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:07 smithi096 ceph-mon[36392]: Detected new or changed devices on smithi204 2024-05-22T22:55:08.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:07 smithi096 ceph-mon[36392]: Adjusting osd_memory_target on smithi204 to 17088M 2024-05-22T22:55:08.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:07 smithi204 ceph-mon[36211]: Detected new or changed devices on smithi204 2024-05-22T22:55:08.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:07 smithi204 ceph-mon[36211]: Adjusting osd_memory_target on smithi204 to 17088M 2024-05-22T22:55:08.833 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:08 smithi084 ceph-mon[32186]: pgmap v79: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:08.833 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:08 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-22T22:55:08.834 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:08 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-22T22:55:08.834 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:08 smithi084 ceph-mon[32186]: osdmap e19: 3 total, 3 up, 3 in 2024-05-22T22:55:08.834 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:08 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-22T22:55:08.834 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:08 smithi084 ceph-mon[32186]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:55:08.834 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:08 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:55:08.834 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:08 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:55:08.834 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:08 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:09.043 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:08 smithi096 ceph-mon[36392]: pgmap v79: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:09.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:08 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-22T22:55:09.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:08 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-22T22:55:09.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:08 smithi096 ceph-mon[36392]: osdmap e19: 3 total, 3 up, 3 in 2024-05-22T22:55:09.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:08 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-22T22:55:09.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:08 smithi096 ceph-mon[36392]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:55:09.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:08 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:55:09.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:08 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:55:09.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:08 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:09.090 INFO:journalctl@ceph.osd.0.smithi084.stdout:May 22 22:55:09 smithi084 sudo[49015]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-22T22:55:09.090 INFO:journalctl@ceph.osd.0.smithi084.stdout:May 22 22:55:09 smithi084 sudo[49015]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-22T22:55:09.090 INFO:journalctl@ceph.osd.0.smithi084.stdout:May 22 22:55:09 smithi084 sudo[49015]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-22T22:55:09.090 INFO:journalctl@ceph.osd.0.smithi084.stdout:May 22 22:55:09 smithi084 sudo[49015]: pam_unix(sudo:session): session closed for user root 2024-05-22T22:55:09.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:08 smithi204 ceph-mon[36211]: pgmap v79: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:09.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:08 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-22T22:55:09.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:08 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-22T22:55:09.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:08 smithi204 ceph-mon[36211]: osdmap e19: 3 total, 3 up, 3 in 2024-05-22T22:55:09.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:08 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-22T22:55:09.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:08 smithi204 ceph-mon[36211]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:55:09.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:08 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:55:09.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:08 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:55:09.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:08 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:09.369 INFO:journalctl@ceph.osd.0.smithi084.stdout:May 22 22:55:09 smithi084 sudo[49019]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-22T22:55:09.369 INFO:journalctl@ceph.osd.0.smithi084.stdout:May 22 22:55:09 smithi084 sudo[49019]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-22T22:55:09.369 INFO:journalctl@ceph.osd.0.smithi084.stdout:May 22 22:55:09 smithi084 sudo[49019]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-22T22:55:09.369 INFO:journalctl@ceph.osd.0.smithi084.stdout:May 22 22:55:09 smithi084 sudo[49019]: pam_unix(sudo:session): session closed for user root 2024-05-22T22:55:09.864 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 22 22:55:09 smithi096 sudo[45036]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-22T22:55:09.864 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 22 22:55:09 smithi096 sudo[45036]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-22T22:55:09.864 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 22 22:55:09 smithi096 sudo[45036]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-22T22:55:09.864 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 22 22:55:09 smithi096 sudo[45036]: pam_unix(sudo:session): session closed for user root 2024-05-22T22:55:09.865 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:09 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-22T22:55:09.865 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:09 smithi096 ceph-mon[36392]: osdmap e20: 3 total, 3 up, 3 in 2024-05-22T22:55:09.865 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:09 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:09.865 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:09 smithi096 ceph-mon[36392]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:10.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:09 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-22T22:55:10.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:09 smithi084 ceph-mon[32186]: osdmap e20: 3 total, 3 up, 3 in 2024-05-22T22:55:10.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:09 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:10.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:09 smithi084 ceph-mon[32186]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:10.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:09 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-22T22:55:10.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:09 smithi204 ceph-mon[36211]: osdmap e20: 3 total, 3 up, 3 in 2024-05-22T22:55:10.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:09 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:10.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:09 smithi204 ceph-mon[36211]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:10.293 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 22 22:55:09 smithi096 sudo[45039]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-22T22:55:10.293 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 22 22:55:09 smithi096 sudo[45039]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-22T22:55:10.294 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 22 22:55:09 smithi096 sudo[45039]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-22T22:55:10.294 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 22 22:55:09 smithi096 sudo[45039]: pam_unix(sudo:session): session closed for user root 2024-05-22T22:55:10.376 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:10.378+0000 7ff8e7fff700 1 -- 172.21.15.84:0/4167373336 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff8f0097e30 con 0x7ff9001012c0 2024-05-22T22:55:10.388 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 22 22:55:10 smithi204 sudo[44967]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-22T22:55:10.388 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 22 22:55:10 smithi204 sudo[44967]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-22T22:55:10.388 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 22 22:55:10 smithi204 sudo[44967]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-22T22:55:10.388 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 22 22:55:10 smithi204 sudo[44967]: pam_unix(sudo:session): session closed for user root 2024-05-22T22:55:10.655 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 22 22:55:10 smithi204 sudo[44970]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-22T22:55:10.655 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 22 22:55:10 smithi204 sudo[44970]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-22T22:55:10.655 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 22 22:55:10 smithi204 sudo[44970]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-22T22:55:10.655 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 22 22:55:10 smithi204 sudo[44970]: pam_unix(sudo:session): session closed for user root 2024-05-22T22:55:10.909 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:10 smithi084 sudo[49097]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-22T22:55:10.910 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:10 smithi084 sudo[49097]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-22T22:55:10.910 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:10 smithi084 sudo[49097]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-22T22:55:10.910 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:10 smithi084 sudo[49097]: pam_unix(sudo:session): session closed for user root 2024-05-22T22:55:11.171 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:10 smithi204 ceph-mon[36211]: osdmap e21: 3 total, 3 up, 3 in 2024-05-22T22:55:11.172 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:10 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:11.172 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:10 smithi204 ceph-mon[36211]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-22T22:55:11.172 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:10 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:11.172 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:10 smithi204 ceph-mon[36211]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-22T22:55:11.172 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 sudo[44973]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-22T22:55:11.172 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 sudo[44973]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-22T22:55:11.172 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 sudo[44973]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-22T22:55:11.172 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 sudo[44973]: pam_unix(sudo:session): session closed for user root 2024-05-22T22:55:11.285 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:10 smithi084 sudo[49100]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-22T22:55:11.285 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:10 smithi084 sudo[49100]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-22T22:55:11.285 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:10 smithi084 sudo[49100]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-22T22:55:11.285 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:10 smithi084 sudo[49100]: pam_unix(sudo:session): session closed for user root 2024-05-22T22:55:11.285 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:10 smithi084 ceph-mon[32186]: osdmap e21: 3 total, 3 up, 3 in 2024-05-22T22:55:11.285 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:10 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:11.285 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:10 smithi084 ceph-mon[32186]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-22T22:55:11.285 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:10 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:11.285 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:10 smithi084 ceph-mon[32186]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-22T22:55:11.293 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:10 smithi096 ceph-mon[36392]: osdmap e21: 3 total, 3 up, 3 in 2024-05-22T22:55:11.293 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:10 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:11.294 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:10 smithi096 ceph-mon[36392]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-22T22:55:11.294 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:10 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:11.294 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:10 smithi096 ceph-mon[36392]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-22T22:55:11.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 sudo[44976]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-22T22:55:11.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 sudo[44976]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-22T22:55:11.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 sudo[44976]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-22T22:55:11.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 sudo[44976]: pam_unix(sudo:session): session closed for user root 2024-05-22T22:55:11.909 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 sudo[45042]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-22T22:55:11.909 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 sudo[45042]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-22T22:55:11.909 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 sudo[45042]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-22T22:55:11.910 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 sudo[45042]: pam_unix(sudo:session): session closed for user root 2024-05-22T22:55:12.294 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 sudo[45045]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-22T22:55:12.294 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 sudo[45045]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-22T22:55:12.294 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 sudo[45045]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-22T22:55:12.294 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 sudo[45045]: pam_unix(sudo:session): session closed for user root 2024-05-22T22:55:12.294 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 ceph-mon[36392]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-22T22:55:12.294 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-22T22:55:12.294 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2882333421' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2a5fb97-6f60-4d45-9f18-447cae480974"}]: dispatch 2024-05-22T22:55:12.294 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:55:12.294 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-22T22:55:12.295 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2882333421' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2a5fb97-6f60-4d45-9f18-447cae480974"}]': finished 2024-05-22T22:55:12.295 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 ceph-mon[36392]: osdmap e22: 4 total, 3 up, 4 in 2024-05-22T22:55:12.295 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 ceph-mon[36392]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-22T22:55:12.295 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-22T22:55:12.295 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-22T22:55:12.295 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:55:12.295 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-22T22:55:12.295 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 ceph-mon[36392]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-22T22:55:12.295 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 ceph-mon[36392]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-22T22:55:12.295 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-22T22:55:12.295 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:55:12.295 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-22T22:55:12.295 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 ceph-mon[36392]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:12.295 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/27745737' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:55:12.296 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:11 smithi096 ceph-mon[36392]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-22T22:55:12.369 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:11 smithi084 ceph-mon[32186]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-22T22:55:12.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:11 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-22T22:55:12.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:11 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2882333421' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2a5fb97-6f60-4d45-9f18-447cae480974"}]: dispatch 2024-05-22T22:55:12.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:11 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:55:12.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:11 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-22T22:55:12.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:11 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2882333421' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2a5fb97-6f60-4d45-9f18-447cae480974"}]': finished 2024-05-22T22:55:12.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:11 smithi084 ceph-mon[32186]: osdmap e22: 4 total, 3 up, 4 in 2024-05-22T22:55:12.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:11 smithi084 ceph-mon[32186]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-22T22:55:12.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:11 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-22T22:55:12.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:11 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-22T22:55:12.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:11 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:55:12.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:11 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-22T22:55:12.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:11 smithi084 ceph-mon[32186]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-22T22:55:12.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:11 smithi084 ceph-mon[32186]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-22T22:55:12.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:11 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-22T22:55:12.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:11 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:55:12.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:11 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-22T22:55:12.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:11 smithi084 ceph-mon[32186]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:12.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:11 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/27745737' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:55:12.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:11 smithi084 ceph-mon[32186]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-22T22:55:12.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 ceph-mon[36211]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-22T22:55:12.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-22T22:55:12.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2882333421' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2a5fb97-6f60-4d45-9f18-447cae480974"}]: dispatch 2024-05-22T22:55:12.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:55:12.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-22T22:55:12.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2882333421' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2a5fb97-6f60-4d45-9f18-447cae480974"}]': finished 2024-05-22T22:55:12.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 ceph-mon[36211]: osdmap e22: 4 total, 3 up, 4 in 2024-05-22T22:55:12.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 ceph-mon[36211]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-22T22:55:12.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-22T22:55:12.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-22T22:55:12.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:55:12.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-22T22:55:12.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 ceph-mon[36211]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-22T22:55:12.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 ceph-mon[36211]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-22T22:55:12.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-22T22:55:12.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T22:55:12.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-22T22:55:12.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 ceph-mon[36211]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:12.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/27745737' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:55:12.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:11 smithi204 ceph-mon[36211]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-22T22:55:14.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:14 smithi096 ceph-mon[36392]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:14.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:14 smithi084 ceph-mon[32186]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:14.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:14 smithi204 ceph-mon[36211]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:16.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:16 smithi096 ceph-mon[36392]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:16.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:16 smithi084 ceph-mon[32186]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:16.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:16 smithi204 ceph-mon[36211]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:18.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:18 smithi096 ceph-mon[36392]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:18.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:18 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-22T22:55:18.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:18 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:18.831 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:18 smithi084 ceph-mon[32186]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:18.831 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:18 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-22T22:55:18.831 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:18 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:18.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:18 smithi204 ceph-mon[36211]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:18.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:18 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-22T22:55:18.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:18 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:19.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:19 smithi096 ceph-mon[36392]: Deploying daemon osd.3 on smithi084 2024-05-22T22:55:19.795 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:19 smithi084 ceph-mon[32186]: Deploying daemon osd.3 on smithi084 2024-05-22T22:55:19.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:19 smithi204 ceph-mon[36211]: Deploying daemon osd.3 on smithi084 2024-05-22T22:55:20.636 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:20 smithi084 ceph-mon[32186]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:20.793 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:20 smithi096 ceph-mon[36392]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:20.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:20 smithi204 ceph-mon[36211]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:22.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:22 smithi096 ceph-mon[36392]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:22.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:22 smithi084 ceph-mon[32186]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:22.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:22 smithi204 ceph-mon[36211]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:24.619 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:24 smithi084 ceph-mon[32186]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:24.619 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:24 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:24.619 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:24 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:24.619 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:24 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:55:24.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:24 smithi096 ceph-mon[36392]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:24.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:24 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:24.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:24 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:24.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:24 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:55:24.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:24 smithi204 ceph-mon[36211]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:24.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:24 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:24.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:24 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:24.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:24 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:55:26.619 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:26 smithi084 ceph-mon[32186]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:26.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:26 smithi096 ceph-mon[36392]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:26.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:26 smithi204 ceph-mon[36211]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:27.438 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 3 on host 'smithi084' 2024-05-22T22:55:27.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:27.441+0000 7ff8e7fff700 1 -- 172.21.15.84:0/4167373336 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff9000627a0 con 0x7ff8e80a2d30 2024-05-22T22:55:27.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:27.444+0000 7ff9053f0700 1 -- 172.21.15.84:0/4167373336 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7ff8e80a2d30 msgr2=0x7ff8e80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:55:27.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:27.444+0000 7ff9053f0700 1 --2- 172.21.15.84:0/4167373336 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7ff8e80a2d30 0x7ff8e80a51f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4000c00 tx=0x7ff8f4005fd0 comp rx=0 tx=0).stop 2024-05-22T22:55:27.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:27.444+0000 7ff9053f0700 1 -- 172.21.15.84:0/4167373336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9001012c0 msgr2=0x7ff90006b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:55:27.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:27.444+0000 7ff9053f0700 1 --2- 172.21.15.84:0/4167373336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9001012c0 0x7ff90006b630 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff8f000c970 tx=0x7ff8f000cc80 comp rx=0 tx=0).stop 2024-05-22T22:55:27.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:27.444+0000 7ff9053f0700 1 -- 172.21.15.84:0/4167373336 shutdown_connections 2024-05-22T22:55:27.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:27.444+0000 7ff9053f0700 1 --2- 172.21.15.84:0/4167373336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff900102e60 0x7ff90019f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:27.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:27.444+0000 7ff9053f0700 1 --2- 172.21.15.84:0/4167373336 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7ff8e80a2d30 0x7ff8e80a51f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:27.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:27.444+0000 7ff9053f0700 1 --2- 172.21.15.84:0/4167373336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9001012c0 0x7ff90006b630 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:27.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:27.444+0000 7ff9053f0700 1 --2- 172.21.15.84:0/4167373336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9001024c0 0x7ff90019d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:27.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:27.444+0000 7ff9053f0700 1 -- 172.21.15.84:0/4167373336 >> 172.21.15.84:0/4167373336 conn(0x7ff9000fc880 msgr2=0x7ff900105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:55:27.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:27.444+0000 7ff9053f0700 1 -- 172.21.15.84:0/4167373336 shutdown_connections 2024-05-22T22:55:27.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:55:27.444+0000 7ff9053f0700 1 -- 172.21.15.84:0/4167373336 wait complete. 2024-05-22T22:55:28.122 DEBUG:teuthology.orchestra.run.smithi084:osd.3> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.3.service 2024-05-22T22:55:28.124 INFO:tasks.cephadm:Deploying osd.4 on smithi096 with /dev/vg_nvme/lv_3... 2024-05-22T22:55:28.124 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-22T22:55:28.369 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.b/config 2024-05-22T22:55:28.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:28 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:28.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:28 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:28.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:28 smithi204 ceph-mon[36211]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:28.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:28 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:28.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:28 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:28.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:28 smithi204 ceph-mon[36211]: from='osd.3 [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-22T22:55:28.700 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:28 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:28.700 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:28 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:28.701 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:28 smithi096 ceph-mon[36392]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:28.701 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:28 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:28.701 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:28 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:28.701 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:28 smithi096 ceph-mon[36392]: from='osd.3 [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-22T22:55:28.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:28 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:28.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:28 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:28.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:28 smithi084 ceph-mon[32186]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:28.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:28 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:28.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:28 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:28.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:28 smithi084 ceph-mon[32186]: from='osd.3 [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-22T22:55:29.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:29 smithi096 ceph-mon[36392]: from='osd.3 [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-22T22:55:29.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:29 smithi096 ceph-mon[36392]: osdmap e23: 4 total, 3 up, 4 in 2024-05-22T22:55:29.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:29 smithi096 ceph-mon[36392]: from='osd.3 [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-22T22:55:29.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:29 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-22T22:55:29.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:29 smithi096 ceph-mon[36392]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:29.842 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:29 smithi084 ceph-mon[32186]: from='osd.3 [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-22T22:55:29.842 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:29 smithi084 ceph-mon[32186]: osdmap e23: 4 total, 3 up, 4 in 2024-05-22T22:55:29.842 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:29 smithi084 ceph-mon[32186]: from='osd.3 [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-22T22:55:29.842 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:29 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-22T22:55:29.843 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:29 smithi084 ceph-mon[32186]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:30.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:29 smithi204 ceph-mon[36211]: from='osd.3 [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-22T22:55:30.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:29 smithi204 ceph-mon[36211]: osdmap e23: 4 total, 3 up, 4 in 2024-05-22T22:55:30.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:29 smithi204 ceph-mon[36211]: from='osd.3 [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-22T22:55:30.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:29 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-22T22:55:30.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:29 smithi204 ceph-mon[36211]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:30.437 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-22T22:55:30.437 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-22T22:55:30.437 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-05-22T22:55:30.437 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-05-22T22:55:30.437 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111922 s, 937 MB/s 2024-05-22T22:55:30.437 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-05-22T22:55:30.476 INFO:journalctl@ceph.osd.3.smithi084.stdout:May 22 22:55:30 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-3[51121]: 2024-05-22T22:55:30.218+0000 7f7c7760f700 -1 osd.3 0 waiting for initial osdmap 2024-05-22T22:55:30.476 INFO:journalctl@ceph.osd.3.smithi084.stdout:May 22 22:55:30 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-3[51121]: 2024-05-22T22:55:30.226+0000 7f7c6f3fa700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-22T22:55:30.818 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:30 smithi084 ceph-mon[32186]: from='osd.3 [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-05-22T22:55:30.818 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:30 smithi084 ceph-mon[32186]: osdmap e24: 4 total, 3 up, 4 in 2024-05-22T22:55:30.818 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:30 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-22T22:55:30.818 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:30 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-22T22:55:30.819 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:30 smithi084 ceph-mon[32186]: from='osd.3 [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648]' entity='osd.3' 2024-05-22T22:55:30.863 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:30 smithi096 ceph-mon[36392]: from='osd.3 [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-05-22T22:55:30.863 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:30 smithi096 ceph-mon[36392]: osdmap e24: 4 total, 3 up, 4 in 2024-05-22T22:55:30.863 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:30 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-22T22:55:30.863 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:30 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-22T22:55:30.863 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:30 smithi096 ceph-mon[36392]: from='osd.3 [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648]' entity='osd.3' 2024-05-22T22:55:31.107 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_3 2024-05-22T22:55:31.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:30 smithi204 ceph-mon[36211]: from='osd.3 [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-05-22T22:55:31.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:30 smithi204 ceph-mon[36211]: osdmap e24: 4 total, 3 up, 4 in 2024-05-22T22:55:31.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:30 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-22T22:55:31.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:30 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-22T22:55:31.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:30 smithi204 ceph-mon[36211]: from='osd.3 [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648]' entity='osd.3' 2024-05-22T22:55:31.349 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.b/config 2024-05-22T22:55:32.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:31 smithi096 ceph-mon[36392]: purged_snaps scrub starts 2024-05-22T22:55:32.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:31 smithi096 ceph-mon[36392]: purged_snaps scrub ok 2024-05-22T22:55:32.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:31 smithi096 ceph-mon[36392]: osd.3 [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648] boot 2024-05-22T22:55:32.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:31 smithi096 ceph-mon[36392]: osdmap e25: 4 total, 4 up, 4 in 2024-05-22T22:55:32.045 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:31 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-22T22:55:32.045 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:31 smithi096 ceph-mon[36392]: Detected new or changed devices on smithi084 2024-05-22T22:55:32.045 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:31 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:32.045 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:31 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:32.045 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:31 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:55:32.045 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:31 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:55:32.045 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:31 smithi096 ceph-mon[36392]: Adjusting osd_memory_target on smithi084 to 8544M 2024-05-22T22:55:32.045 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:31 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:32.045 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:31 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:32.045 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:31 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:55:32.045 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:31 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:32.045 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:31 smithi096 ceph-mon[36392]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:32.046 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:31 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:55:32.046 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:31 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:32.046 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:31 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:55:32.046 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:31 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:32.046 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:31 smithi096 ceph-mon[36392]: osdmap e26: 4 total, 4 up, 4 in 2024-05-22T22:55:32.120 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:31 smithi084 ceph-mon[32186]: purged_snaps scrub starts 2024-05-22T22:55:32.120 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:31 smithi084 ceph-mon[32186]: purged_snaps scrub ok 2024-05-22T22:55:32.120 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:31 smithi084 ceph-mon[32186]: osd.3 [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648] boot 2024-05-22T22:55:32.120 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:31 smithi084 ceph-mon[32186]: osdmap e25: 4 total, 4 up, 4 in 2024-05-22T22:55:32.120 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:31 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-22T22:55:32.120 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:31 smithi084 ceph-mon[32186]: Detected new or changed devices on smithi084 2024-05-22T22:55:32.120 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:31 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:32.120 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:31 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:32.120 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:31 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:55:32.121 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:31 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:55:32.121 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:31 smithi084 ceph-mon[32186]: Adjusting osd_memory_target on smithi084 to 8544M 2024-05-22T22:55:32.121 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:31 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:32.121 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:31 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:32.121 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:31 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:55:32.121 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:31 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:32.121 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:31 smithi084 ceph-mon[32186]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:32.121 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:31 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:55:32.121 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:31 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:32.121 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:31 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:55:32.121 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:31 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:32.121 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:31 smithi084 ceph-mon[32186]: osdmap e26: 4 total, 4 up, 4 in 2024-05-22T22:55:32.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:31 smithi204 ceph-mon[36211]: purged_snaps scrub starts 2024-05-22T22:55:32.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:31 smithi204 ceph-mon[36211]: purged_snaps scrub ok 2024-05-22T22:55:32.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:31 smithi204 ceph-mon[36211]: osd.3 [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648] boot 2024-05-22T22:55:32.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:31 smithi204 ceph-mon[36211]: osdmap e25: 4 total, 4 up, 4 in 2024-05-22T22:55:32.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:31 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-22T22:55:32.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:31 smithi204 ceph-mon[36211]: Detected new or changed devices on smithi084 2024-05-22T22:55:32.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:31 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:32.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:31 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:32.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:31 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:55:32.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:31 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:55:32.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:31 smithi204 ceph-mon[36211]: Adjusting osd_memory_target on smithi084 to 8544M 2024-05-22T22:55:32.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:31 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:32.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:31 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:32.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:31 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:55:32.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:31 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:32.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:31 smithi204 ceph-mon[36211]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-22T22:55:32.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:31 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:55:32.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:31 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:32.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:31 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:55:32.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:31 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:32.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:31 smithi204 ceph-mon[36211]: osdmap e26: 4 total, 4 up, 4 in 2024-05-22T22:55:33.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.047+0000 7fcb03504700 1 -- 172.21.15.96:0/90516015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcafc1012c0 msgr2=0x7fcafc1016e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:55:33.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.047+0000 7fcb03504700 1 --2- 172.21.15.96:0/90516015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcafc1012c0 0x7fcafc1016e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0099c0 tx=0x7fcaec009cd0 comp rx=0 tx=0).stop 2024-05-22T22:55:33.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.047+0000 7fcb03504700 1 -- 172.21.15.96:0/90516015 shutdown_connections 2024-05-22T22:55:33.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.047+0000 7fcb03504700 1 --2- 172.21.15.96:0/90516015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcafc102e60 0x7fcafc109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:33.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.047+0000 7fcb03504700 1 --2- 172.21.15.96:0/90516015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcafc1012c0 0x7fcafc1016e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:33.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.047+0000 7fcb03504700 1 --2- 172.21.15.96:0/90516015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcafc1024c0 0x7fcafc102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:33.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.047+0000 7fcb03504700 1 -- 172.21.15.96:0/90516015 >> 172.21.15.96:0/90516015 conn(0x7fcafc0fc880 msgr2=0x7fcafc0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:55:33.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.047+0000 7fcb03504700 1 -- 172.21.15.96:0/90516015 shutdown_connections 2024-05-22T22:55:33.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.047+0000 7fcb03504700 1 -- 172.21.15.96:0/90516015 wait complete. 2024-05-22T22:55:33.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.048+0000 7fcb03504700 1 Processor -- start 2024-05-22T22:55:33.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.048+0000 7fcb03504700 1 -- start start 2024-05-22T22:55:33.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.048+0000 7fcb03504700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcafc1012c0 0x7fcafc06b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:55:33.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.048+0000 7fcb03504700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcafc1024c0 0x7fcafc19d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:55:33.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.048+0000 7fcb03504700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcafc102e60 0x7fcafc19f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:55:33.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.048+0000 7fcb03504700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc10cf00 con 0x7fcafc1024c0 2024-05-22T22:55:33.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.048+0000 7fcb03504700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc10cd80 con 0x7fcafc1012c0 2024-05-22T22:55:33.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.048+0000 7fcb03504700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc10d080 con 0x7fcafc102e60 2024-05-22T22:55:33.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.048+0000 7fcb012a0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcafc1012c0 0x7fcafc06b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:55:33.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.048+0000 7fcb012a0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcafc1012c0 0x7fcafc06b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52842/0 (socket says 172.21.15.96:52842) 2024-05-22T22:55:33.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.048+0000 7fcb012a0700 1 -- 172.21.15.96:0/2550295738 learned_addr learned my addr 172.21.15.96:0/2550295738 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-05-22T22:55:33.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.048+0000 7fcb00a9f700 1 --2- 172.21.15.96:0/2550295738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcafc1024c0 0x7fcafc19d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:55:33.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.048+0000 7fcb01aa1700 1 --2- 172.21.15.96:0/2550295738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcafc102e60 0x7fcafc19f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:55:33.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.048+0000 7fcb012a0700 1 -- 172.21.15.96:0/2550295738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcafc102e60 msgr2=0x7fcafc19f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:55:33.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.048+0000 7fcb012a0700 1 --2- 172.21.15.96:0/2550295738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcafc102e60 0x7fcafc19f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:33.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.048+0000 7fcb012a0700 1 -- 172.21.15.96:0/2550295738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcafc1024c0 msgr2=0x7fcafc19d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:55:33.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.048+0000 7fcb012a0700 1 --2- 172.21.15.96:0/2550295738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcafc1024c0 0x7fcafc19d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:33.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.048+0000 7fcb012a0700 1 -- 172.21.15.96:0/2550295738 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcafc197240 con 0x7fcafc1012c0 2024-05-22T22:55:33.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.048+0000 7fcb01aa1700 1 --2- 172.21.15.96:0/2550295738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcafc102e60 0x7fcafc19f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T22:55:33.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.048+0000 7fcb00a9f700 1 --2- 172.21.15.96:0/2550295738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcafc1024c0 0x7fcafc19d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T22:55:33.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.048+0000 7fcb012a0700 1 --2- 172.21.15.96:0/2550295738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcafc1012c0 0x7fcafc06b630 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00c010 tx=0x7fcaec009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:55:33.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.049+0000 7fcaf27fc700 1 -- 172.21.15.96:0/2550295738 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec030070 con 0x7fcafc1012c0 2024-05-22T22:55:33.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.049+0000 7fcb03504700 1 -- 172.21.15.96:0/2550295738 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcafc1974d0 con 0x7fcafc1012c0 2024-05-22T22:55:33.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.049+0000 7fcaf27fc700 1 -- 172.21.15.96:0/2550295738 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaec003f40 con 0x7fcafc1012c0 2024-05-22T22:55:33.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.049+0000 7fcaf27fc700 1 -- 172.21.15.96:0/2550295738 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec00fc60 con 0x7fcafc1012c0 2024-05-22T22:55:33.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.049+0000 7fcb03504700 1 -- 172.21.15.96:0/2550295738 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcafc197a80 con 0x7fcafc1012c0 2024-05-22T22:55:33.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.050+0000 7fcb03504700 1 -- 172.21.15.96:0/2550295738 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcafc069050 con 0x7fcafc1012c0 2024-05-22T22:55:33.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.051+0000 7fcaf27fc700 1 -- 172.21.15.96:0/2550295738 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcaec033070 con 0x7fcafc1012c0 2024-05-22T22:55:33.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.051+0000 7fcaf27fc700 1 --2- 172.21.15.96:0/2550295738 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fcae80a2c20 0x7fcae80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:55:33.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.051+0000 7fcaf27fc700 1 -- 172.21.15.96:0/2550295738 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7fcaec0d7850 con 0x7fcafc1012c0 2024-05-22T22:55:33.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.051+0000 7fcb00a9f700 1 --2- 172.21.15.96:0/2550295738 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fcae80a2c20 0x7fcae80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:55:33.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.052+0000 7fcb00a9f700 1 --2- 172.21.15.96:0/2550295738 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fcae80a2c20 0x7fcae80a50e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8005950 tx=0x7fcaf80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:55:33.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.056+0000 7fcaf27fc700 1 -- 172.21.15.96:0/2550295738 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcaec067850 con 0x7fcafc1012c0 2024-05-22T22:55:33.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:33.261+0000 7fcb03504700 1 -- 172.21.15.96:0/2550295738 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fcafc108330 con 0x7fcae80a2c20 2024-05-22T22:55:33.999 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:33 smithi096 ceph-mon[36392]: osdmap e27: 4 total, 4 up, 4 in 2024-05-22T22:55:33.999 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:33 smithi096 ceph-mon[36392]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:55:33.999 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:33 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:55:33.999 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:33 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:55:33.999 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:33 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:33.999 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:33 smithi096 ceph-mon[36392]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-22T22:55:34.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:33 smithi084 ceph-mon[32186]: osdmap e27: 4 total, 4 up, 4 in 2024-05-22T22:55:34.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:33 smithi084 ceph-mon[32186]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:55:34.120 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:33 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:55:34.120 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:33 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:55:34.120 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:33 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:34.120 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:33 smithi084 ceph-mon[32186]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-22T22:55:34.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:33 smithi204 ceph-mon[36211]: osdmap e27: 4 total, 4 up, 4 in 2024-05-22T22:55:34.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:33 smithi204 ceph-mon[36211]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:55:34.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:33 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:55:34.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:33 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:55:34.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:33 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:34.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:33 smithi204 ceph-mon[36211]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-22T22:55:35.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:34 smithi096 ceph-mon[36392]: osdmap e28: 4 total, 4 up, 4 in 2024-05-22T22:55:35.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:34 smithi084 ceph-mon[32186]: osdmap e28: 4 total, 4 up, 4 in 2024-05-22T22:55:35.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:34 smithi204 ceph-mon[36211]: osdmap e28: 4 total, 4 up, 4 in 2024-05-22T22:55:36.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:35 smithi096 ceph-mon[36392]: osdmap e29: 4 total, 4 up, 4 in 2024-05-22T22:55:36.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:35 smithi096 ceph-mon[36392]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-22T22:55:36.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:35 smithi084 ceph-mon[32186]: osdmap e29: 4 total, 4 up, 4 in 2024-05-22T22:55:36.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:35 smithi084 ceph-mon[32186]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-22T22:55:36.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:35 smithi204 ceph-mon[36211]: osdmap e29: 4 total, 4 up, 4 in 2024-05-22T22:55:36.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:35 smithi204 ceph-mon[36211]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-22T22:55:37.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:36 smithi096 ceph-mon[36392]: from='client.? 172.21.15.96:0/4186816983' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f"}]: dispatch 2024-05-22T22:55:37.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:36 smithi096 ceph-mon[36392]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f"}]: dispatch 2024-05-22T22:55:37.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:36 smithi096 ceph-mon[36392]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f"}]': finished 2024-05-22T22:55:37.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:36 smithi096 ceph-mon[36392]: osdmap e30: 5 total, 4 up, 5 in 2024-05-22T22:55:37.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:36 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-22T22:55:37.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:36 smithi096 ceph-mon[36392]: from='client.? 172.21.15.96:0/1845207300' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:55:37.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:36 smithi084 ceph-mon[32186]: from='client.? 172.21.15.96:0/4186816983' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f"}]: dispatch 2024-05-22T22:55:37.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:36 smithi084 ceph-mon[32186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f"}]: dispatch 2024-05-22T22:55:37.120 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:36 smithi084 ceph-mon[32186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f"}]': finished 2024-05-22T22:55:37.120 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:36 smithi084 ceph-mon[32186]: osdmap e30: 5 total, 4 up, 5 in 2024-05-22T22:55:37.120 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:36 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-22T22:55:37.120 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:36 smithi084 ceph-mon[32186]: from='client.? 172.21.15.96:0/1845207300' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:55:37.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:36 smithi204 ceph-mon[36211]: from='client.? 172.21.15.96:0/4186816983' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f"}]: dispatch 2024-05-22T22:55:37.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:36 smithi204 ceph-mon[36211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f"}]: dispatch 2024-05-22T22:55:37.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:36 smithi204 ceph-mon[36211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f"}]': finished 2024-05-22T22:55:37.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:36 smithi204 ceph-mon[36211]: osdmap e30: 5 total, 4 up, 5 in 2024-05-22T22:55:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:36 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-22T22:55:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:36 smithi204 ceph-mon[36211]: from='client.? 172.21.15.96:0/1845207300' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:55:38.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:37 smithi096 ceph-mon[36392]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-22T22:55:38.119 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:37 smithi084 ceph-mon[32186]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-22T22:55:38.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:37 smithi204 ceph-mon[36211]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-22T22:55:40.544 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:40 smithi096 ceph-mon[36392]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-22T22:55:40.619 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:40 smithi084 ceph-mon[32186]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-22T22:55:40.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:40 smithi204 ceph-mon[36211]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-22T22:55:42.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:42 smithi096 ceph-mon[36392]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-22T22:55:42.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:42 smithi084 ceph-mon[32186]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-22T22:55:42.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:42 smithi204 ceph-mon[36211]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-22T22:55:43.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:43 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-22T22:55:43.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:43 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:43.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:43 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-22T22:55:43.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:43 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:43.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:43 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-22T22:55:43.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:43 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:44.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:44 smithi096 ceph-mon[36392]: Deploying daemon osd.4 on smithi096 2024-05-22T22:55:44.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:44 smithi096 ceph-mon[36392]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-22T22:55:44.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:44 smithi084 ceph-mon[32186]: Deploying daemon osd.4 on smithi096 2024-05-22T22:55:44.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:44 smithi084 ceph-mon[32186]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-22T22:55:44.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:44 smithi204 ceph-mon[36211]: Deploying daemon osd.4 on smithi096 2024-05-22T22:55:44.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:44 smithi204 ceph-mon[36211]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-22T22:55:46.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:46 smithi096 ceph-mon[36392]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-22T22:55:46.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:46 smithi084 ceph-mon[32186]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-22T22:55:46.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:46 smithi204 ceph-mon[36211]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-22T22:55:48.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:48 smithi096 ceph-mon[36392]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-22T22:55:48.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:48 smithi084 ceph-mon[32186]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-22T22:55:48.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:48 smithi204 ceph-mon[36211]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-22T22:55:50.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:49 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:50.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:49 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:50.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:49 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:55:50.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:49 smithi096 ceph-mon[36392]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-22T22:55:50.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:49 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:50.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:49 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:50.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:49 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:55:50.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:49 smithi204 ceph-mon[36211]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-22T22:55:50.369 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:49 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:50.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:49 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:50.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:49 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:55:50.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:49 smithi084 ceph-mon[32186]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-22T22:55:52.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:52 smithi096 ceph-mon[36392]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-22T22:55:52.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:52 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:52.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:52 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:52.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:52 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:52.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:52 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:55:52.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:52 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:52.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:52 smithi084 ceph-mon[32186]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-22T22:55:52.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:52 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:52.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:52 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:52.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:52 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:52.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:52 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:55:52.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:52 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:52.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:52 smithi204 ceph-mon[36211]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-22T22:55:52.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:52 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:52.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:52 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:52.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:52 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:52.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:52 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:55:52.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:52 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:52.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:52.981+0000 7fcaf27fc700 1 -- 172.21.15.96:0/2550295738 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcafc108330 con 0x7fcae80a2c20 2024-05-22T22:55:52.977 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 4 on host 'smithi096' 2024-05-22T22:55:52.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:52.984+0000 7fcb03504700 1 -- 172.21.15.96:0/2550295738 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fcae80a2c20 msgr2=0x7fcae80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:55:52.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:52.984+0000 7fcb03504700 1 --2- 172.21.15.96:0/2550295738 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fcae80a2c20 0x7fcae80a50e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8005950 tx=0x7fcaf80058e0 comp rx=0 tx=0).stop 2024-05-22T22:55:52.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:52.984+0000 7fcb03504700 1 -- 172.21.15.96:0/2550295738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcafc1012c0 msgr2=0x7fcafc06b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:55:52.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:52.984+0000 7fcb03504700 1 --2- 172.21.15.96:0/2550295738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcafc1012c0 0x7fcafc06b630 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00c010 tx=0x7fcaec009f10 comp rx=0 tx=0).stop 2024-05-22T22:55:52.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:52.984+0000 7fcb03504700 1 -- 172.21.15.96:0/2550295738 shutdown_connections 2024-05-22T22:55:52.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:52.984+0000 7fcb03504700 1 --2- 172.21.15.96:0/2550295738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcafc102e60 0x7fcafc19f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:52.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:52.984+0000 7fcb03504700 1 --2- 172.21.15.96:0/2550295738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcafc1012c0 0x7fcafc06b630 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:52.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:52.984+0000 7fcb03504700 1 --2- 172.21.15.96:0/2550295738 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fcae80a2c20 0x7fcae80a50e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:52.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:52.984+0000 7fcb03504700 1 --2- 172.21.15.96:0/2550295738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcafc1024c0 0x7fcafc19d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:52.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:52.984+0000 7fcb03504700 1 -- 172.21.15.96:0/2550295738 >> 172.21.15.96:0/2550295738 conn(0x7fcafc0fc880 msgr2=0x7fcafc105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:55:52.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:52.984+0000 7fcb03504700 1 -- 172.21.15.96:0/2550295738 shutdown_connections 2024-05-22T22:55:52.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:55:52.984+0000 7fcb03504700 1 -- 172.21.15.96:0/2550295738 wait complete. 2024-05-22T22:55:53.588 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:53 smithi096 ceph-mon[36392]: from='osd.4 [v2:172.21.15.96:6808/2554265648,v1:172.21.15.96:6809/2554265648]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-22T22:55:53.589 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:53 smithi096 ceph-mon[36392]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-22T22:55:53.589 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:53 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:53.589 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:53 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:55:53.589 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:53 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:53.618 DEBUG:teuthology.orchestra.run.smithi096:osd.4> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.4.service 2024-05-22T22:55:53.620 INFO:tasks.cephadm:Deploying osd.5 on smithi204 with /dev/vg_nvme/lv_3... 2024-05-22T22:55:53.620 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-22T22:55:53.864 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.c/config 2024-05-22T22:55:53.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:53 smithi084 ceph-mon[32186]: from='osd.4 [v2:172.21.15.96:6808/2554265648,v1:172.21.15.96:6809/2554265648]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-22T22:55:53.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:53 smithi084 ceph-mon[32186]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-22T22:55:53.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:53 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:53.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:53 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:55:53.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:53 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:53.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:53 smithi204 ceph-mon[36211]: from='osd.4 [v2:172.21.15.96:6808/2554265648,v1:172.21.15.96:6809/2554265648]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-22T22:55:53.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:53 smithi204 ceph-mon[36211]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-22T22:55:53.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:53 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:53.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:53 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:55:53.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:53 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:54.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:54 smithi204 ceph-mon[36211]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-22T22:55:54.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:54 smithi204 ceph-mon[36211]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-22T22:55:54.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:54 smithi204 ceph-mon[36211]: from='osd.4 [v2:172.21.15.96:6808/2554265648,v1:172.21.15.96:6809/2554265648]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-22T22:55:54.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:54 smithi204 ceph-mon[36211]: osdmap e31: 5 total, 4 up, 5 in 2024-05-22T22:55:54.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:54 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-22T22:55:54.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:54 smithi204 ceph-mon[36211]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-22T22:55:54.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:54 smithi096 ceph-mon[36392]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-22T22:55:54.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:54 smithi096 ceph-mon[36392]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-22T22:55:54.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:54 smithi096 ceph-mon[36392]: from='osd.4 [v2:172.21.15.96:6808/2554265648,v1:172.21.15.96:6809/2554265648]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-22T22:55:54.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:54 smithi096 ceph-mon[36392]: osdmap e31: 5 total, 4 up, 5 in 2024-05-22T22:55:54.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:54 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-22T22:55:54.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:54 smithi096 ceph-mon[36392]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-22T22:55:54.795 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 22 22:55:54 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-4[47826]: 2024-05-22T22:55:54.516+0000 7f5fc119a700 -1 osd.4 0 waiting for initial osdmap 2024-05-22T22:55:54.795 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 22 22:55:54 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-4[47826]: 2024-05-22T22:55:54.521+0000 7f5fbcf8d700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-22T22:55:54.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:54 smithi084 ceph-mon[32186]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-22T22:55:54.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:54 smithi084 ceph-mon[32186]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-22T22:55:54.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:54 smithi084 ceph-mon[32186]: from='osd.4 [v2:172.21.15.96:6808/2554265648,v1:172.21.15.96:6809/2554265648]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-22T22:55:54.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:54 smithi084 ceph-mon[32186]: osdmap e31: 5 total, 4 up, 5 in 2024-05-22T22:55:54.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:54 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-22T22:55:54.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:54 smithi084 ceph-mon[32186]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-22T22:55:55.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:55 smithi204 ceph-mon[36211]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-22T22:55:55.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:55 smithi204 ceph-mon[36211]: osdmap e32: 5 total, 4 up, 5 in 2024-05-22T22:55:55.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:55 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-22T22:55:55.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:55 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-22T22:55:55.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:55 smithi096 ceph-mon[36392]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-22T22:55:55.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:55 smithi096 ceph-mon[36392]: osdmap e32: 5 total, 4 up, 5 in 2024-05-22T22:55:55.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:55 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-22T22:55:55.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:55 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-22T22:55:55.867 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-22T22:55:55.867 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-22T22:55:55.867 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-05-22T22:55:55.867 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-05-22T22:55:55.867 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132185 s, 793 MB/s 2024-05-22T22:55:55.867 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-05-22T22:55:55.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:55 smithi084 ceph-mon[32186]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-22T22:55:55.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:55 smithi084 ceph-mon[32186]: osdmap e32: 5 total, 4 up, 5 in 2024-05-22T22:55:55.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:55 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-22T22:55:55.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:55 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-22T22:55:56.453 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_3 2024-05-22T22:55:56.697 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.c/config 2024-05-22T22:55:56.739 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:56 smithi204 ceph-mon[36211]: purged_snaps scrub starts 2024-05-22T22:55:56.740 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:56 smithi204 ceph-mon[36211]: purged_snaps scrub ok 2024-05-22T22:55:56.740 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:56 smithi204 ceph-mon[36211]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-22T22:55:56.740 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:56 smithi204 ceph-mon[36211]: osd.4 [v2:172.21.15.96:6808/2554265648,v1:172.21.15.96:6809/2554265648] boot 2024-05-22T22:55:56.740 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:56 smithi204 ceph-mon[36211]: osdmap e33: 5 total, 5 up, 5 in 2024-05-22T22:55:56.740 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:56 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-22T22:55:56.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:56 smithi096 ceph-mon[36392]: purged_snaps scrub starts 2024-05-22T22:55:56.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:56 smithi096 ceph-mon[36392]: purged_snaps scrub ok 2024-05-22T22:55:56.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:56 smithi096 ceph-mon[36392]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-22T22:55:56.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:56 smithi096 ceph-mon[36392]: osd.4 [v2:172.21.15.96:6808/2554265648,v1:172.21.15.96:6809/2554265648] boot 2024-05-22T22:55:56.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:56 smithi096 ceph-mon[36392]: osdmap e33: 5 total, 5 up, 5 in 2024-05-22T22:55:56.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:56 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-22T22:55:56.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:56 smithi084 ceph-mon[32186]: purged_snaps scrub starts 2024-05-22T22:55:56.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:56 smithi084 ceph-mon[32186]: purged_snaps scrub ok 2024-05-22T22:55:56.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:56 smithi084 ceph-mon[32186]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-22T22:55:56.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:56 smithi084 ceph-mon[32186]: osd.4 [v2:172.21.15.96:6808/2554265648,v1:172.21.15.96:6809/2554265648] boot 2024-05-22T22:55:56.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:56 smithi084 ceph-mon[32186]: osdmap e33: 5 total, 5 up, 5 in 2024-05-22T22:55:56.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:56 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-22T22:55:57.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:57 smithi096 ceph-mon[36392]: osdmap e34: 5 total, 5 up, 5 in 2024-05-22T22:55:57.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:57 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:57.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:57 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:57.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:57 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:55:57.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:57 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:55:57.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:57 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:57.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:57 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:57.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:57 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:55:57.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:57 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:57.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:57 smithi084 ceph-mon[32186]: osdmap e34: 5 total, 5 up, 5 in 2024-05-22T22:55:57.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:57.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:57.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:55:57.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:55:57.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:57.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:57.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:55:57.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:57.899 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:57 smithi204 ceph-mon[36211]: osdmap e34: 5 total, 5 up, 5 in 2024-05-22T22:55:57.900 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:57 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:57.900 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:57 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:57.900 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:57 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:55:57.900 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:57 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:55:57.900 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:57 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:57.900 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:57 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:57.900 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:57 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:55:57.900 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:57 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:55:58.475 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.476+0000 7faf4cb9a700 1 -- 172.21.15.204:0/2338075478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf48100760 msgr2=0x7faf48100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:55:58.475 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.476+0000 7faf4cb9a700 1 --2- 172.21.15.204:0/2338075478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf48100760 0x7faf48100b80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faf38009990 tx=0x7faf38009ca0 comp rx=0 tx=0).stop 2024-05-22T22:55:58.475 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.477+0000 7faf4cb9a700 1 -- 172.21.15.204:0/2338075478 shutdown_connections 2024-05-22T22:55:58.475 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.477+0000 7faf4cb9a700 1 --2- 172.21.15.204:0/2338075478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf481040e0 0x7faf4810a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:58.475 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.477+0000 7faf4cb9a700 1 --2- 172.21.15.204:0/2338075478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf48100760 0x7faf48100b80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:58.475 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.477+0000 7faf4cb9a700 1 --2- 172.21.15.204:0/2338075478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf48103740 0x7faf48103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:58.475 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.477+0000 7faf4cb9a700 1 -- 172.21.15.204:0/2338075478 >> 172.21.15.204:0/2338075478 conn(0x7faf480fc0c0 msgr2=0x7faf480fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:55:58.475 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.477+0000 7faf4cb9a700 1 -- 172.21.15.204:0/2338075478 shutdown_connections 2024-05-22T22:55:58.475 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.477+0000 7faf4cb9a700 1 -- 172.21.15.204:0/2338075478 wait complete. 2024-05-22T22:55:58.476 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.477+0000 7faf4cb9a700 1 Processor -- start 2024-05-22T22:55:58.476 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.477+0000 7faf4cb9a700 1 -- start start 2024-05-22T22:55:58.476 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.478+0000 7faf4cb9a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf48100760 0x7faf48198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:55:58.476 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.478+0000 7faf4cb9a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf48103740 0x7faf48198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:55:58.476 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.478+0000 7faf4cb9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf481040e0 0x7faf4819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:55:58.476 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.478+0000 7faf4cb9a700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faf4810e080 con 0x7faf481040e0 2024-05-22T22:55:58.476 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.478+0000 7faf4cb9a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf4810df00 con 0x7faf48103740 2024-05-22T22:55:58.476 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.478+0000 7faf4cb9a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faf4810e200 con 0x7faf48100760 2024-05-22T22:55:58.476 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.478+0000 7faf4659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf48100760 0x7faf48198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:55:58.476 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.478+0000 7faf4659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf48100760 0x7faf48198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:58514/0 (socket says 172.21.15.204:58514) 2024-05-22T22:55:58.476 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.478+0000 7faf4659c700 1 -- 172.21.15.204:0/1248244354 learned_addr learned my addr 172.21.15.204:0/1248244354 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-22T22:55:58.476 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.478+0000 7faf45d9b700 1 --2- 172.21.15.204:0/1248244354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf48103740 0x7faf48198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:55:58.477 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.478+0000 7faf46d9d700 1 --2- 172.21.15.204:0/1248244354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf481040e0 0x7faf4819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:55:58.477 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.478+0000 7faf4659c700 1 -- 172.21.15.204:0/1248244354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf48103740 msgr2=0x7faf48198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:55:58.477 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.478+0000 7faf4659c700 1 --2- 172.21.15.204:0/1248244354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf48103740 0x7faf48198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:58.477 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.478+0000 7faf4659c700 1 -- 172.21.15.204:0/1248244354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf481040e0 msgr2=0x7faf4819fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:55:58.477 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.478+0000 7faf4659c700 1 --2- 172.21.15.204:0/1248244354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf481040e0 0x7faf4819fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:55:58.477 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.478+0000 7faf4659c700 1 -- 172.21.15.204:0/1248244354 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf481a0290 con 0x7faf48100760 2024-05-22T22:55:58.477 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.478+0000 7faf4659c700 1 --2- 172.21.15.204:0/1248244354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf48100760 0x7faf48198630 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faf38000c00 tx=0x7faf380047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:55:58.477 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.478+0000 7faf377fe700 1 -- 172.21.15.204:0/1248244354 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38003c70 con 0x7faf48100760 2024-05-22T22:55:58.477 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.478+0000 7faf377fe700 1 -- 172.21.15.204:0/1248244354 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf38004830 con 0x7faf48100760 2024-05-22T22:55:58.477 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.478+0000 7faf4cb9a700 1 -- 172.21.15.204:0/1248244354 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf481a0520 con 0x7faf48100760 2024-05-22T22:55:58.477 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.478+0000 7faf377fe700 1 -- 172.21.15.204:0/1248244354 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38005330 con 0x7faf48100760 2024-05-22T22:55:58.477 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.478+0000 7faf4cb9a700 1 -- 172.21.15.204:0/1248244354 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf481a0910 con 0x7faf48100760 2024-05-22T22:55:58.478 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.479+0000 7faf4cb9a700 1 -- 172.21.15.204:0/1248244354 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf48069050 con 0x7faf48100760 2024-05-22T22:55:58.479 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.480+0000 7faf377fe700 1 -- 172.21.15.204:0/1248244354 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7faf38005550 con 0x7faf48100760 2024-05-22T22:55:58.479 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.481+0000 7faf377fe700 1 --2- 172.21.15.204:0/1248244354 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7faf300a2c20 0x7faf300a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:55:58.479 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.481+0000 7faf377fe700 1 -- 172.21.15.204:0/1248244354 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7faf380cf040 con 0x7faf48100760 2024-05-22T22:55:58.479 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.481+0000 7faf45d9b700 1 --2- 172.21.15.204:0/1248244354 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7faf300a2c20 0x7faf300a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:55:58.479 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.482+0000 7faf45d9b700 1 --2- 172.21.15.204:0/1248244354 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7faf300a2c20 0x7faf300a50e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faf3c007c60 tx=0x7faf3c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:55:58.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.486+0000 7faf377fe700 1 -- 172.21.15.204:0/1248244354 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faf38013070 con 0x7faf48100760 2024-05-22T22:55:58.686 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:55:58.690+0000 7faf4cb9a700 1 -- 172.21.15.204:0/1248244354 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7faf480627a0 con 0x7faf300a2c20 2024-05-22T22:55:58.781 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:58 smithi084 ceph-mon[32186]: Detected new or changed devices on smithi096 2024-05-22T22:55:58.781 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:58 smithi084 ceph-mon[32186]: Adjusting osd_memory_target on smithi096 to 8544M 2024-05-22T22:55:58.781 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:58 smithi084 ceph-mon[32186]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:55:58.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:58 smithi096 ceph-mon[36392]: Detected new or changed devices on smithi096 2024-05-22T22:55:58.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:58 smithi096 ceph-mon[36392]: Adjusting osd_memory_target on smithi096 to 8544M 2024-05-22T22:55:58.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:58 smithi096 ceph-mon[36392]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:55:58.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:58 smithi204 ceph-mon[36211]: Detected new or changed devices on smithi096 2024-05-22T22:55:58.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:58 smithi204 ceph-mon[36211]: Adjusting osd_memory_target on smithi096 to 8544M 2024-05-22T22:55:58.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:58 smithi204 ceph-mon[36211]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:55:59.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:59 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:55:59.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:59 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:55:59.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:55:59 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:59.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:59 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:55:59.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:59 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:55:59.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:55:59 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:55:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:59 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:55:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:59 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:55:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:55:59 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:00.552 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:00 smithi204 ceph-mon[36211]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:56:00.552 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:00 smithi204 ceph-mon[36211]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:00.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:00 smithi096 ceph-mon[36392]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:56:00.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:00 smithi096 ceph-mon[36392]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:00.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:00 smithi084 ceph-mon[32186]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:56:00.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:00 smithi084 ceph-mon[32186]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:01.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:01 smithi096 ceph-mon[36392]: from='client.? 172.21.15.204:0/4008214381' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7996bf5e-93a5-41b0-a9b1-99e7ef4514f3"}]: dispatch 2024-05-22T22:56:01.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:01 smithi096 ceph-mon[36392]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7996bf5e-93a5-41b0-a9b1-99e7ef4514f3"}]: dispatch 2024-05-22T22:56:01.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:01 smithi096 ceph-mon[36392]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7996bf5e-93a5-41b0-a9b1-99e7ef4514f3"}]': finished 2024-05-22T22:56:01.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:01 smithi096 ceph-mon[36392]: osdmap e35: 6 total, 5 up, 6 in 2024-05-22T22:56:01.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:01 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-22T22:56:01.844 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:01 smithi204 ceph-mon[36211]: from='client.? 172.21.15.204:0/4008214381' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7996bf5e-93a5-41b0-a9b1-99e7ef4514f3"}]: dispatch 2024-05-22T22:56:01.844 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:01 smithi204 ceph-mon[36211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7996bf5e-93a5-41b0-a9b1-99e7ef4514f3"}]: dispatch 2024-05-22T22:56:01.844 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:01 smithi204 ceph-mon[36211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7996bf5e-93a5-41b0-a9b1-99e7ef4514f3"}]': finished 2024-05-22T22:56:01.844 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:01 smithi204 ceph-mon[36211]: osdmap e35: 6 total, 5 up, 6 in 2024-05-22T22:56:01.844 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:01 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-22T22:56:01.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:01 smithi084 ceph-mon[32186]: from='client.? 172.21.15.204:0/4008214381' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7996bf5e-93a5-41b0-a9b1-99e7ef4514f3"}]: dispatch 2024-05-22T22:56:01.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:01 smithi084 ceph-mon[32186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7996bf5e-93a5-41b0-a9b1-99e7ef4514f3"}]: dispatch 2024-05-22T22:56:01.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:01 smithi084 ceph-mon[32186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7996bf5e-93a5-41b0-a9b1-99e7ef4514f3"}]': finished 2024-05-22T22:56:01.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:01 smithi084 ceph-mon[32186]: osdmap e35: 6 total, 5 up, 6 in 2024-05-22T22:56:01.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:01 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-22T22:56:02.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:02 smithi096 ceph-mon[36392]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:02.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:02 smithi096 ceph-mon[36392]: from='client.? 172.21.15.204:0/3238233248' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:56:02.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:02 smithi084 ceph-mon[32186]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:02.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:02 smithi084 ceph-mon[32186]: from='client.? 172.21.15.204:0/3238233248' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:56:02.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:02 smithi204 ceph-mon[36211]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:02.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:02 smithi204 ceph-mon[36211]: from='client.? 172.21.15.204:0/3238233248' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:56:04.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:04 smithi096 ceph-mon[36392]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:04.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:04 smithi084 ceph-mon[32186]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:04.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:04 smithi204 ceph-mon[36211]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:06.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:06 smithi096 ceph-mon[36392]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:06.832 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:06 smithi204 ceph-mon[36211]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:06.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:06 smithi084 ceph-mon[32186]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:08.818 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:08 smithi204 ceph-mon[36211]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:08.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:08 smithi084 ceph-mon[32186]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:09.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:08 smithi096 ceph-mon[36392]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:09.729 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:09 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-22T22:56:09.730 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:09 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:09.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:09 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-22T22:56:09.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:09 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:10.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:09 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-22T22:56:10.045 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:09 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:10.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:10 smithi204 ceph-mon[36211]: Deploying daemon osd.5 on smithi204 2024-05-22T22:56:10.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:10 smithi204 ceph-mon[36211]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:10.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:10 smithi084 ceph-mon[32186]: Deploying daemon osd.5 on smithi204 2024-05-22T22:56:10.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:10 smithi084 ceph-mon[32186]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:11.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:10 smithi096 ceph-mon[36392]: Deploying daemon osd.5 on smithi204 2024-05-22T22:56:11.045 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:10 smithi096 ceph-mon[36392]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:11.867 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:11 smithi204 ceph-mon[36211]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:11.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:11 smithi084 ceph-mon[32186]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:12.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:11 smithi096 ceph-mon[36392]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:14.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:14 smithi096 ceph-mon[36392]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:14.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:14 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:14.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:14 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:14.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:14 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:56:14.801 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:14 smithi204 ceph-mon[36211]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:14.801 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:14 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:14.801 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:14 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:14.801 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:14 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:56:14.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:14 smithi084 ceph-mon[32186]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:14.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:14 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:14.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:14 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:14.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:14 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:56:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:16 smithi204 ceph-mon[36211]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:16.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:16 smithi096 ceph-mon[36392]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:16.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:16 smithi084 ceph-mon[32186]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:18.052 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 5 on host 'smithi204' 2024-05-22T22:56:18.053 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:56:18.054+0000 7faf377fe700 1 -- 172.21.15.204:0/1248244354 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7faf480627a0 con 0x7faf300a2c20 2024-05-22T22:56:18.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:56:18.057+0000 7faf4cb9a700 1 -- 172.21.15.204:0/1248244354 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7faf300a2c20 msgr2=0x7faf300a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:56:18.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:56:18.057+0000 7faf4cb9a700 1 --2- 172.21.15.204:0/1248244354 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7faf300a2c20 0x7faf300a50e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faf3c007c60 tx=0x7faf3c0073d0 comp rx=0 tx=0).stop 2024-05-22T22:56:18.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:56:18.057+0000 7faf4cb9a700 1 -- 172.21.15.204:0/1248244354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf48100760 msgr2=0x7faf48198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:56:18.055 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:56:18.057+0000 7faf4cb9a700 1 --2- 172.21.15.204:0/1248244354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf48100760 0x7faf48198630 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faf38000c00 tx=0x7faf380047b0 comp rx=0 tx=0).stop 2024-05-22T22:56:18.055 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:56:18.057+0000 7faf4cb9a700 1 -- 172.21.15.204:0/1248244354 shutdown_connections 2024-05-22T22:56:18.055 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:56:18.057+0000 7faf4cb9a700 1 --2- 172.21.15.204:0/1248244354 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7faf300a2c20 0x7faf300a50e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:56:18.055 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:56:18.057+0000 7faf4cb9a700 1 --2- 172.21.15.204:0/1248244354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf481040e0 0x7faf4819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:56:18.055 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:56:18.057+0000 7faf4cb9a700 1 --2- 172.21.15.204:0/1248244354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf48100760 0x7faf48198630 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:56:18.055 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:56:18.057+0000 7faf4cb9a700 1 --2- 172.21.15.204:0/1248244354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf48103740 0x7faf48198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:56:18.055 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:56:18.057+0000 7faf4cb9a700 1 -- 172.21.15.204:0/1248244354 >> 172.21.15.204:0/1248244354 conn(0x7faf480fc0c0 msgr2=0x7faf48106970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:56:18.055 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:56:18.057+0000 7faf4cb9a700 1 -- 172.21.15.204:0/1248244354 shutdown_connections 2024-05-22T22:56:18.055 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:56:18.057+0000 7faf4cb9a700 1 -- 172.21.15.204:0/1248244354 wait complete. 2024-05-22T22:56:18.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:18 smithi204 ceph-mon[36211]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:18.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:18 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:18.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:18 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:18.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:18 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:18.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:18 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:18.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:18 smithi204 ceph-mon[36211]: from='osd.5 [v2:172.21.15.204:6808/4075476986,v1:172.21.15.204:6809/4075476986]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-22T22:56:18.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:18 smithi204 ceph-mon[36211]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-22T22:56:18.782 DEBUG:teuthology.orchestra.run.smithi204:osd.5> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.5.service 2024-05-22T22:56:18.784 INFO:tasks.cephadm:Deploying osd.6 on smithi084 with /dev/vg_nvme/lv_2... 2024-05-22T22:56:18.785 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-22T22:56:18.794 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:18 smithi096 ceph-mon[36392]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:18.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:18 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:18.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:18 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:18.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:18 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:18.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:18 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:18.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:18 smithi096 ceph-mon[36392]: from='osd.5 [v2:172.21.15.204:6808/4075476986,v1:172.21.15.204:6809/4075476986]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-22T22:56:18.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:18 smithi096 ceph-mon[36392]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-22T22:56:18.812 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:18 smithi084 ceph-mon[32186]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:18.812 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:18 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:18.812 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:18 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:18.812 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:18 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:18.812 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:18 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:18.813 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:18 smithi084 ceph-mon[32186]: from='osd.5 [v2:172.21.15.204:6808/4075476986,v1:172.21.15.204:6809/4075476986]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-22T22:56:18.813 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:18 smithi084 ceph-mon[32186]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-22T22:56:19.034 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T22:56:20.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:20 smithi204 ceph-mon[36211]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-22T22:56:20.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:20 smithi204 ceph-mon[36211]: osdmap e36: 6 total, 5 up, 6 in 2024-05-22T22:56:20.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:20 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-22T22:56:20.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:20 smithi204 ceph-mon[36211]: from='osd.5 [v2:172.21.15.204:6808/4075476986,v1:172.21.15.204:6809/4075476986]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-22T22:56:20.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:20 smithi204 ceph-mon[36211]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-22T22:56:20.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:20 smithi204 ceph-mon[36211]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:20.402 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:20 smithi084 ceph-mon[32186]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-22T22:56:20.402 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:20 smithi084 ceph-mon[32186]: osdmap e36: 6 total, 5 up, 6 in 2024-05-22T22:56:20.402 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:20 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-22T22:56:20.402 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:20 smithi084 ceph-mon[32186]: from='osd.5 [v2:172.21.15.204:6808/4075476986,v1:172.21.15.204:6809/4075476986]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-22T22:56:20.403 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:20 smithi084 ceph-mon[32186]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-22T22:56:20.403 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:20 smithi084 ceph-mon[32186]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:20.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:20 smithi096 ceph-mon[36392]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-22T22:56:20.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:20 smithi096 ceph-mon[36392]: osdmap e36: 6 total, 5 up, 6 in 2024-05-22T22:56:20.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:20 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-22T22:56:20.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:20 smithi096 ceph-mon[36392]: from='osd.5 [v2:172.21.15.204:6808/4075476986,v1:172.21.15.204:6809/4075476986]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-22T22:56:20.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:20 smithi096 ceph-mon[36392]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-22T22:56:20.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:20 smithi096 ceph-mon[36392]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:20.656 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 22:56:20 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T22:56:20.606+0000 7f0976743700 -1 osd.5 0 waiting for initial osdmap 2024-05-22T22:56:20.656 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 22:56:20 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T22:56:20.613+0000 7f096e52e700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-22T22:56:21.103 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-22T22:56:21.104 INFO:teuthology.orchestra.run.smithi084.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-22T22:56:21.104 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10+0 records in 2024-05-22T22:56:21.104 INFO:teuthology.orchestra.run.smithi084.stderr:10+0 records out 2024-05-22T22:56:21.104 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111779 s, 938 MB/s 2024-05-22T22:56:21.104 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping successful for: 2024-05-22T22:56:21.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:21 smithi084 ceph-mon[32186]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-22T22:56:21.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:21 smithi084 ceph-mon[32186]: osdmap e37: 6 total, 5 up, 6 in 2024-05-22T22:56:21.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:21 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-22T22:56:21.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:21 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-22T22:56:21.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:21 smithi084 ceph-mon[32186]: from='osd.5 ' entity='osd.5' 2024-05-22T22:56:21.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:21 smithi204 ceph-mon[36211]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-22T22:56:21.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:21 smithi204 ceph-mon[36211]: osdmap e37: 6 total, 5 up, 6 in 2024-05-22T22:56:21.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:21 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-22T22:56:21.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:21 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-22T22:56:21.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:21 smithi204 ceph-mon[36211]: from='osd.5 ' entity='osd.5' 2024-05-22T22:56:21.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:21 smithi096 ceph-mon[36392]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-22T22:56:21.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:21 smithi096 ceph-mon[36392]: osdmap e37: 6 total, 5 up, 6 in 2024-05-22T22:56:21.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:21 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-22T22:56:21.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:21 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-22T22:56:21.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:21 smithi096 ceph-mon[36392]: from='osd.5 ' entity='osd.5' 2024-05-22T22:56:21.767 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi084:vg_nvme/lv_2 2024-05-22T22:56:22.050 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T22:56:22.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:22 smithi204 ceph-mon[36211]: purged_snaps scrub starts 2024-05-22T22:56:22.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:22 smithi204 ceph-mon[36211]: purged_snaps scrub ok 2024-05-22T22:56:22.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:22 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-22T22:56:22.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:22 smithi204 ceph-mon[36211]: osd.5 [v2:172.21.15.204:6808/4075476986,v1:172.21.15.204:6809/4075476986] boot 2024-05-22T22:56:22.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:22 smithi204 ceph-mon[36211]: osdmap e38: 6 total, 6 up, 6 in 2024-05-22T22:56:22.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:22 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-22T22:56:22.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:22 smithi204 ceph-mon[36211]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:22.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:22 smithi204 ceph-mon[36211]: osdmap e39: 6 total, 6 up, 6 in 2024-05-22T22:56:22.441 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:22 smithi084 ceph-mon[32186]: purged_snaps scrub starts 2024-05-22T22:56:22.441 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:22 smithi084 ceph-mon[32186]: purged_snaps scrub ok 2024-05-22T22:56:22.441 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:22 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-22T22:56:22.441 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:22 smithi084 ceph-mon[32186]: osd.5 [v2:172.21.15.204:6808/4075476986,v1:172.21.15.204:6809/4075476986] boot 2024-05-22T22:56:22.441 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:22 smithi084 ceph-mon[32186]: osdmap e38: 6 total, 6 up, 6 in 2024-05-22T22:56:22.441 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:22 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-22T22:56:22.441 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:22 smithi084 ceph-mon[32186]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:22.442 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:22 smithi084 ceph-mon[32186]: osdmap e39: 6 total, 6 up, 6 in 2024-05-22T22:56:22.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:22 smithi096 ceph-mon[36392]: purged_snaps scrub starts 2024-05-22T22:56:22.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:22 smithi096 ceph-mon[36392]: purged_snaps scrub ok 2024-05-22T22:56:22.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:22 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-22T22:56:22.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:22 smithi096 ceph-mon[36392]: osd.5 [v2:172.21.15.204:6808/4075476986,v1:172.21.15.204:6809/4075476986] boot 2024-05-22T22:56:22.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:22 smithi096 ceph-mon[36392]: osdmap e38: 6 total, 6 up, 6 in 2024-05-22T22:56:22.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:22 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-22T22:56:22.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:22 smithi096 ceph-mon[36392]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-22T22:56:22.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:22 smithi096 ceph-mon[36392]: osdmap e39: 6 total, 6 up, 6 in 2024-05-22T22:56:23.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:23 smithi096 ceph-mon[36392]: Detected new or changed devices on smithi204 2024-05-22T22:56:23.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:23 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:23.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:23 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:23.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:23 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:56:23.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:23 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:56:23.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:23 smithi096 ceph-mon[36392]: Adjusting osd_memory_target on smithi204 to 8544M 2024-05-22T22:56:23.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:23 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:23.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:23 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:23.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:23 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:56:23.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:23 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:23.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:23 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:56:23.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:23 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:23.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:23 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:56:23.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:23 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:23.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:23 smithi096 ceph-mon[36392]: osdmap e40: 6 total, 6 up, 6 in 2024-05-22T22:56:23.549 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:23 smithi084 ceph-mon[32186]: Detected new or changed devices on smithi204 2024-05-22T22:56:23.549 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:23 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:23.549 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:23 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:23.549 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:23 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:56:23.549 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:23 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:56:23.549 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:23 smithi084 ceph-mon[32186]: Adjusting osd_memory_target on smithi204 to 8544M 2024-05-22T22:56:23.549 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:23 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:23.549 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:23 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:23.549 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:23 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:56:23.550 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:23 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:23.550 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:23 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:56:23.550 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:23 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:23.550 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:23 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:56:23.550 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:23 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:23.550 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:23 smithi084 ceph-mon[32186]: osdmap e40: 6 total, 6 up, 6 in 2024-05-22T22:56:23.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:23 smithi204 ceph-mon[36211]: Detected new or changed devices on smithi204 2024-05-22T22:56:23.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:23 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:23.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:23 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:23.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:23 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:56:23.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:23 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:56:23.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:23 smithi204 ceph-mon[36211]: Adjusting osd_memory_target on smithi204 to 8544M 2024-05-22T22:56:23.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:23 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:23.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:23 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:23.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:23 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:56:23.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:23 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:23.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:23 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:56:23.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:23 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:23.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:23 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:56:23.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:23 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:23.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:23 smithi204 ceph-mon[36211]: osdmap e40: 6 total, 6 up, 6 in 2024-05-22T22:56:23.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.657+0000 7fc62ba8a700 1 -- 172.21.15.84:0/964849757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc624103720 msgr2=0x7fc624103b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:56:23.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.657+0000 7fc62ba8a700 1 --2- 172.21.15.84:0/964849757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc624103720 0x7fc624103b80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc614009990 tx=0x7fc614009ca0 comp rx=0 tx=0).stop 2024-05-22T22:56:23.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.657+0000 7fc62ba8a700 1 -- 172.21.15.84:0/964849757 shutdown_connections 2024-05-22T22:56:23.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.657+0000 7fc62ba8a700 1 --2- 172.21.15.84:0/964849757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6241040c0 0x7fc62410a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:56:23.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.657+0000 7fc62ba8a700 1 --2- 172.21.15.84:0/964849757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc624100740 0x7fc624100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:56:23.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.657+0000 7fc62ba8a700 1 --2- 172.21.15.84:0/964849757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc624103720 0x7fc624103b80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:56:23.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.657+0000 7fc62ba8a700 1 -- 172.21.15.84:0/964849757 >> 172.21.15.84:0/964849757 conn(0x7fc6240fc0a0 msgr2=0x7fc6240fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:56:23.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.658+0000 7fc62ba8a700 1 -- 172.21.15.84:0/964849757 shutdown_connections 2024-05-22T22:56:23.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.658+0000 7fc62ba8a700 1 -- 172.21.15.84:0/964849757 wait complete. 2024-05-22T22:56:23.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.658+0000 7fc62ba8a700 1 Processor -- start 2024-05-22T22:56:23.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.658+0000 7fc62ba8a700 1 -- start start 2024-05-22T22:56:23.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.658+0000 7fc62ba8a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc624100740 0x7fc6241986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:56:23.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.658+0000 7fc62ba8a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc624103720 0x7fc624198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:56:23.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.658+0000 7fc62ba8a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6241040c0 0x7fc62419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:56:23.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.658+0000 7fc62ba8a700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc62410e0b0 con 0x7fc624103720 2024-05-22T22:56:23.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.658+0000 7fc62ba8a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc62410df30 con 0x7fc624100740 2024-05-22T22:56:23.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.658+0000 7fc62ba8a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc62410e230 con 0x7fc6241040c0 2024-05-22T22:56:23.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.658+0000 7fc629025700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc624103720 0x7fc624198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:56:23.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.658+0000 7fc629025700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc624103720 0x7fc624198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49916/0 (socket says 172.21.15.84:49916) 2024-05-22T22:56:23.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.658+0000 7fc629025700 1 -- 172.21.15.84:0/2091976751 learned_addr learned my addr 172.21.15.84:0/2091976751 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:56:23.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.659+0000 7fc629826700 1 --2- 172.21.15.84:0/2091976751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc624100740 0x7fc6241986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:56:23.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.659+0000 7fc62a027700 1 --2- 172.21.15.84:0/2091976751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6241040c0 0x7fc62419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:56:23.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.659+0000 7fc629025700 1 -- 172.21.15.84:0/2091976751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6241040c0 msgr2=0x7fc62419fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:56:23.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.659+0000 7fc629025700 1 --2- 172.21.15.84:0/2091976751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6241040c0 0x7fc62419fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:56:23.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.659+0000 7fc629025700 1 -- 172.21.15.84:0/2091976751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc624100740 msgr2=0x7fc6241986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:56:23.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.659+0000 7fc629025700 1 --2- 172.21.15.84:0/2091976751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc624100740 0x7fc6241986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:56:23.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.659+0000 7fc629025700 1 -- 172.21.15.84:0/2091976751 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6241a0320 con 0x7fc624103720 2024-05-22T22:56:23.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.659+0000 7fc629025700 1 --2- 172.21.15.84:0/2091976751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc624103720 0x7fc624198c00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc614000c00 tx=0x7fc6140057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:56:23.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.659+0000 7fc61affd700 1 -- 172.21.15.84:0/2091976751 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc614027070 con 0x7fc624103720 2024-05-22T22:56:23.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.659+0000 7fc61affd700 1 -- 172.21.15.84:0/2091976751 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc614004150 con 0x7fc624103720 2024-05-22T22:56:23.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.659+0000 7fc61affd700 1 -- 172.21.15.84:0/2091976751 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61400f460 con 0x7fc624103720 2024-05-22T22:56:23.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.659+0000 7fc62ba8a700 1 -- 172.21.15.84:0/2091976751 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6241a05b0 con 0x7fc624103720 2024-05-22T22:56:23.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.659+0000 7fc62ba8a700 1 -- 172.21.15.84:0/2091976751 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6241a0a00 con 0x7fc624103720 2024-05-22T22:56:23.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.660+0000 7fc62ba8a700 1 -- 172.21.15.84:0/2091976751 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc624069050 con 0x7fc624103720 2024-05-22T22:56:23.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.661+0000 7fc61affd700 1 -- 172.21.15.84:0/2091976751 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc61402a070 con 0x7fc624103720 2024-05-22T22:56:23.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.662+0000 7fc61affd700 1 --2- 172.21.15.84:0/2091976751 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc6100a2c20 0x7fc6100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:56:23.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.662+0000 7fc629826700 1 --2- 172.21.15.84:0/2091976751 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc6100a2c20 0x7fc6100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:56:23.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.662+0000 7fc61affd700 1 -- 172.21.15.84:0/2091976751 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7fc6140cf6b0 con 0x7fc624103720 2024-05-22T22:56:23.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.662+0000 7fc629826700 1 --2- 172.21.15.84:0/2091976751 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc6100a2c20 0x7fc6100a50e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc62406a8f0 tx=0x7fc620007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:56:23.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.665+0000 7fc61affd700 1 -- 172.21.15.84:0/2091976751 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc614031020 con 0x7fc624103720 2024-05-22T22:56:23.872 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:23.875+0000 7fc62ba8a700 1 -- 172.21.15.84:0/2091976751 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc624062860 con 0x7fc6100a2c20 2024-05-22T22:56:24.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:24 smithi096 ceph-mon[36392]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-22T22:56:24.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:24 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:56:24.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:24 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:56:24.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:24 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:24.557 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:24 smithi204 ceph-mon[36211]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-22T22:56:24.558 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:24 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:56:24.558 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:24 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:56:24.558 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:24 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:24.621 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:24 smithi084 ceph-mon[32186]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-22T22:56:24.621 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:24 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:56:24.621 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:24 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:56:24.621 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:24 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:25.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:25 smithi096 ceph-mon[36392]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:56:25.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:25 smithi096 ceph-mon[36392]: osdmap e41: 6 total, 6 up, 6 in 2024-05-22T22:56:25.620 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:25 smithi084 ceph-mon[32186]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:56:25.620 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:25 smithi084 ceph-mon[32186]: osdmap e41: 6 total, 6 up, 6 in 2024-05-22T22:56:25.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:25 smithi204 ceph-mon[36211]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:56:25.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:25 smithi204 ceph-mon[36211]: osdmap e41: 6 total, 6 up, 6 in 2024-05-22T22:56:26.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:26 smithi096 ceph-mon[36392]: osdmap e42: 6 total, 6 up, 6 in 2024-05-22T22:56:26.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:26 smithi096 ceph-mon[36392]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-22T22:56:26.620 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:26 smithi084 ceph-mon[32186]: osdmap e42: 6 total, 6 up, 6 in 2024-05-22T22:56:26.620 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:26 smithi084 ceph-mon[32186]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-22T22:56:26.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:26 smithi204 ceph-mon[36211]: osdmap e42: 6 total, 6 up, 6 in 2024-05-22T22:56:26.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:26 smithi204 ceph-mon[36211]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-22T22:56:27.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:27 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/796805682' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4b7c5de-1836-4d16-8603-3c517524d9fc"}]: dispatch 2024-05-22T22:56:27.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:27 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/796805682' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4b7c5de-1836-4d16-8603-3c517524d9fc"}]': finished 2024-05-22T22:56:27.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:27 smithi096 ceph-mon[36392]: osdmap e43: 7 total, 6 up, 7 in 2024-05-22T22:56:27.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:27 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-22T22:56:27.620 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:27 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/796805682' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4b7c5de-1836-4d16-8603-3c517524d9fc"}]: dispatch 2024-05-22T22:56:27.620 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:27 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/796805682' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4b7c5de-1836-4d16-8603-3c517524d9fc"}]': finished 2024-05-22T22:56:27.621 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:27 smithi084 ceph-mon[32186]: osdmap e43: 7 total, 6 up, 7 in 2024-05-22T22:56:27.621 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:27 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-22T22:56:27.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:27 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/796805682' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4b7c5de-1836-4d16-8603-3c517524d9fc"}]: dispatch 2024-05-22T22:56:27.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:27 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/796805682' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4b7c5de-1836-4d16-8603-3c517524d9fc"}]': finished 2024-05-22T22:56:27.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:27 smithi204 ceph-mon[36211]: osdmap e43: 7 total, 6 up, 7 in 2024-05-22T22:56:27.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:27 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-22T22:56:28.620 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:28 smithi084 ceph-mon[32186]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-22T22:56:28.620 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:28 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2737635162' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:56:28.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:28 smithi204 ceph-mon[36211]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-22T22:56:28.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:28 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2737635162' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:56:28.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:28 smithi096 ceph-mon[36392]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-22T22:56:28.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:28 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2737635162' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:56:30.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:30 smithi096 ceph-mon[36392]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-22T22:56:30.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:30 smithi084 ceph-mon[32186]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-22T22:56:30.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:30 smithi204 ceph-mon[36211]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-22T22:56:32.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:32 smithi096 ceph-mon[36392]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-22T22:56:32.849 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:32 smithi084 ceph-mon[32186]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-22T22:56:32.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:32 smithi204 ceph-mon[36211]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-22T22:56:34.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:34 smithi096 ceph-mon[36392]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-22T22:56:34.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:34 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-22T22:56:34.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:34 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:34.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:34 smithi084 ceph-mon[32186]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-22T22:56:34.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:34 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-22T22:56:34.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:34 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:34.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:34 smithi204 ceph-mon[36211]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-22T22:56:34.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:34 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-22T22:56:34.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:34 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:35.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:35 smithi204 ceph-mon[36211]: Deploying daemon osd.6 on smithi084 2024-05-22T22:56:35.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:35 smithi096 ceph-mon[36392]: Deploying daemon osd.6 on smithi084 2024-05-22T22:56:35.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:35 smithi084 ceph-mon[32186]: Deploying daemon osd.6 on smithi084 2024-05-22T22:56:36.620 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:36 smithi084 ceph-mon[32186]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-22T22:56:36.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:36 smithi096 ceph-mon[36392]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-22T22:56:36.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:36 smithi204 ceph-mon[36211]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-22T22:56:38.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:38 smithi096 ceph-mon[36392]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-05-22T22:56:38.870 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:38 smithi084 ceph-mon[32186]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-05-22T22:56:38.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:38 smithi204 ceph-mon[36211]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-05-22T22:56:40.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:40 smithi096 ceph-mon[36392]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-22T22:56:40.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:40 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:40.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:40 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:40.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:40 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:56:40.865 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:40 smithi084 ceph-mon[32186]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-22T22:56:40.865 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:40 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:40.865 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:40 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:40.865 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:40 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:56:40.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:40 smithi204 ceph-mon[36211]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-22T22:56:40.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:40 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:40.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:40 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:40.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:40 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:56:42.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:42 smithi096 ceph-mon[36392]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-22T22:56:42.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:42 smithi084 ceph-mon[32186]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-22T22:56:42.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:42 smithi204 ceph-mon[36211]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-22T22:56:44.077 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 6 on host 'smithi084' 2024-05-22T22:56:44.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:44.079+0000 7fc61affd700 1 -- 172.21.15.84:0/2091976751 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc624062860 con 0x7fc6100a2c20 2024-05-22T22:56:44.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:44.082+0000 7fc62ba8a700 1 -- 172.21.15.84:0/2091976751 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc6100a2c20 msgr2=0x7fc6100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:56:44.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:44.082+0000 7fc62ba8a700 1 --2- 172.21.15.84:0/2091976751 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc6100a2c20 0x7fc6100a50e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc62406a8f0 tx=0x7fc620007400 comp rx=0 tx=0).stop 2024-05-22T22:56:44.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:44.082+0000 7fc62ba8a700 1 -- 172.21.15.84:0/2091976751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc624103720 msgr2=0x7fc624198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:56:44.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:44.082+0000 7fc62ba8a700 1 --2- 172.21.15.84:0/2091976751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc624103720 0x7fc624198c00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc614000c00 tx=0x7fc6140057c0 comp rx=0 tx=0).stop 2024-05-22T22:56:44.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:44.082+0000 7fc62ba8a700 1 -- 172.21.15.84:0/2091976751 shutdown_connections 2024-05-22T22:56:44.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:44.082+0000 7fc62ba8a700 1 --2- 172.21.15.84:0/2091976751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6241040c0 0x7fc62419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:56:44.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:44.082+0000 7fc62ba8a700 1 --2- 172.21.15.84:0/2091976751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc624100740 0x7fc6241986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:56:44.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:44.082+0000 7fc62ba8a700 1 --2- 172.21.15.84:0/2091976751 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc6100a2c20 0x7fc6100a50e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:56:44.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:44.082+0000 7fc62ba8a700 1 --2- 172.21.15.84:0/2091976751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc624103720 0x7fc624198c00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:56:44.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:44.082+0000 7fc62ba8a700 1 -- 172.21.15.84:0/2091976751 >> 172.21.15.84:0/2091976751 conn(0x7fc6240fc0a0 msgr2=0x7fc6241069c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:56:44.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:44.082+0000 7fc62ba8a700 1 -- 172.21.15.84:0/2091976751 shutdown_connections 2024-05-22T22:56:44.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:56:44.082+0000 7fc62ba8a700 1 -- 172.21.15.84:0/2091976751 wait complete. 2024-05-22T22:56:44.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:44 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:44.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:44 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:44.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:44 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:44.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:44 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:56:44.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:44 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:44.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:44 smithi084 ceph-mon[32186]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-22T22:56:44.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:44 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:44.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:44 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:44.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:44 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:56:44.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:44 smithi084 ceph-mon[32186]: from='osd.6 [v2:172.21.15.84:6818/1806202865,v1:172.21.15.84:6819/1806202865]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-22T22:56:44.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:44.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:44.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:44.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:56:44.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:44.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:44 smithi096 ceph-mon[36392]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-22T22:56:44.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:44.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:44.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:56:44.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:44 smithi096 ceph-mon[36392]: from='osd.6 [v2:172.21.15.84:6818/1806202865,v1:172.21.15.84:6819/1806202865]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-22T22:56:44.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:44 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:44.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:44 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:44.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:44 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:44.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:44 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:56:44.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:44 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:44.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:44 smithi204 ceph-mon[36211]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-22T22:56:44.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:44 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:44.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:44 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:44.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:44 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:56:44.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:44 smithi204 ceph-mon[36211]: from='osd.6 [v2:172.21.15.84:6818/1806202865,v1:172.21.15.84:6819/1806202865]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-22T22:56:44.880 DEBUG:teuthology.orchestra.run.smithi084:osd.6> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.6.service 2024-05-22T22:56:44.882 INFO:tasks.cephadm:Deploying osd.7 on smithi096 with /dev/vg_nvme/lv_2... 2024-05-22T22:56:44.883 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-22T22:56:45.129 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.b/config 2024-05-22T22:56:46.295 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:46 smithi096 ceph-mon[36392]: from='osd.6 [v2:172.21.15.84:6818/1806202865,v1:172.21.15.84:6819/1806202865]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-22T22:56:46.295 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:46 smithi096 ceph-mon[36392]: osdmap e44: 7 total, 6 up, 7 in 2024-05-22T22:56:46.295 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:46 smithi096 ceph-mon[36392]: from='osd.6 [v2:172.21.15.84:6818/1806202865,v1:172.21.15.84:6819/1806202865]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-22T22:56:46.296 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:46 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-22T22:56:46.296 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:46 smithi096 ceph-mon[36392]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-22T22:56:46.303 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:46 smithi084 ceph-mon[32186]: from='osd.6 [v2:172.21.15.84:6818/1806202865,v1:172.21.15.84:6819/1806202865]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-22T22:56:46.304 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:46 smithi084 ceph-mon[32186]: osdmap e44: 7 total, 6 up, 7 in 2024-05-22T22:56:46.304 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:46 smithi084 ceph-mon[32186]: from='osd.6 [v2:172.21.15.84:6818/1806202865,v1:172.21.15.84:6819/1806202865]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-22T22:56:46.304 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:46 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-22T22:56:46.304 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:46 smithi084 ceph-mon[32186]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-22T22:56:46.304 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 22:56:46 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T22:56:46.077+0000 7f19707a6700 -1 osd.6 0 waiting for initial osdmap 2024-05-22T22:56:46.304 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 22:56:46 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T22:56:46.087+0000 7f196c599700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-22T22:56:46.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:46 smithi204 ceph-mon[36211]: from='osd.6 [v2:172.21.15.84:6818/1806202865,v1:172.21.15.84:6819/1806202865]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-22T22:56:46.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:46 smithi204 ceph-mon[36211]: osdmap e44: 7 total, 6 up, 7 in 2024-05-22T22:56:46.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:46 smithi204 ceph-mon[36211]: from='osd.6 [v2:172.21.15.84:6818/1806202865,v1:172.21.15.84:6819/1806202865]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-22T22:56:46.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:46 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-22T22:56:46.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:46 smithi204 ceph-mon[36211]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-22T22:56:47.315 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:47 smithi096 ceph-mon[36392]: from='osd.6 [v2:172.21.15.84:6818/1806202865,v1:172.21.15.84:6819/1806202865]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-05-22T22:56:47.315 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:47 smithi096 ceph-mon[36392]: osdmap e45: 7 total, 6 up, 7 in 2024-05-22T22:56:47.315 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:47 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-22T22:56:47.315 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:47 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-22T22:56:47.330 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-22T22:56:47.330 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-22T22:56:47.330 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-05-22T22:56:47.330 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-05-22T22:56:47.331 INFO:teuthology.orchestra.run.smithi096.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0131123 s, 800 MB/s 2024-05-22T22:56:47.331 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-05-22T22:56:47.370 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:47 smithi084 ceph-mon[32186]: from='osd.6 [v2:172.21.15.84:6818/1806202865,v1:172.21.15.84:6819/1806202865]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-05-22T22:56:47.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:47 smithi084 ceph-mon[32186]: osdmap e45: 7 total, 6 up, 7 in 2024-05-22T22:56:47.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:47 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-22T22:56:47.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:47 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-22T22:56:47.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:47 smithi204 ceph-mon[36211]: from='osd.6 [v2:172.21.15.84:6818/1806202865,v1:172.21.15.84:6819/1806202865]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-05-22T22:56:47.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:47 smithi204 ceph-mon[36211]: osdmap e45: 7 total, 6 up, 7 in 2024-05-22T22:56:47.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:47 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-22T22:56:47.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:47 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-22T22:56:47.966 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_2 2024-05-22T22:56:48.209 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.b/config 2024-05-22T22:56:48.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:48 smithi096 ceph-mon[36392]: purged_snaps scrub starts 2024-05-22T22:56:48.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:48 smithi096 ceph-mon[36392]: purged_snaps scrub ok 2024-05-22T22:56:48.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:48 smithi096 ceph-mon[36392]: osd.6 [v2:172.21.15.84:6818/1806202865,v1:172.21.15.84:6819/1806202865] boot 2024-05-22T22:56:48.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:48 smithi096 ceph-mon[36392]: osdmap e46: 7 total, 7 up, 7 in 2024-05-22T22:56:48.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:48 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-22T22:56:48.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:48 smithi096 ceph-mon[36392]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:56:48.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:48 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:48.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:48 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:48.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:48 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:56:48.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:48 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:56:48.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:48 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:56:48.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:48 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:48.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:48 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:48.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:48 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:56:48.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:48 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:48.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:48 smithi084 ceph-mon[32186]: purged_snaps scrub starts 2024-05-22T22:56:48.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:48 smithi084 ceph-mon[32186]: purged_snaps scrub ok 2024-05-22T22:56:48.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:48 smithi084 ceph-mon[32186]: osd.6 [v2:172.21.15.84:6818/1806202865,v1:172.21.15.84:6819/1806202865] boot 2024-05-22T22:56:48.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:48 smithi084 ceph-mon[32186]: osdmap e46: 7 total, 7 up, 7 in 2024-05-22T22:56:48.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-22T22:56:48.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:48 smithi084 ceph-mon[32186]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:56:48.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:48.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:48.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:56:48.372 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:56:48.372 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:56:48.372 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:48.372 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:48.372 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:56:48.372 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:48.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:48 smithi204 ceph-mon[36211]: purged_snaps scrub starts 2024-05-22T22:56:48.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:48 smithi204 ceph-mon[36211]: purged_snaps scrub ok 2024-05-22T22:56:48.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:48 smithi204 ceph-mon[36211]: osd.6 [v2:172.21.15.84:6818/1806202865,v1:172.21.15.84:6819/1806202865] boot 2024-05-22T22:56:48.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:48 smithi204 ceph-mon[36211]: osdmap e46: 7 total, 7 up, 7 in 2024-05-22T22:56:48.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:48 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-22T22:56:48.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:48 smithi204 ceph-mon[36211]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:56:48.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:48 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:48.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:48 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:48.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:48 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:56:48.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:48 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:56:48.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:48 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:56:48.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:48 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:48.408 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:48 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:48.408 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:48 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:56:48.408 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:48 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:56:49.295 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:49 smithi096 ceph-mon[36392]: Detected new or changed devices on smithi084 2024-05-22T22:56:49.296 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:49 smithi096 ceph-mon[36392]: Adjusting osd_memory_target on smithi084 to 5696M 2024-05-22T22:56:49.296 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:49 smithi096 ceph-mon[36392]: osdmap e47: 7 total, 7 up, 7 in 2024-05-22T22:56:49.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:49 smithi084 ceph-mon[32186]: Detected new or changed devices on smithi084 2024-05-22T22:56:49.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:49 smithi084 ceph-mon[32186]: Adjusting osd_memory_target on smithi084 to 5696M 2024-05-22T22:56:49.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:49 smithi084 ceph-mon[32186]: osdmap e47: 7 total, 7 up, 7 in 2024-05-22T22:56:49.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:49 smithi204 ceph-mon[36211]: Detected new or changed devices on smithi084 2024-05-22T22:56:49.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:49 smithi204 ceph-mon[36211]: Adjusting osd_memory_target on smithi084 to 5696M 2024-05-22T22:56:49.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:49 smithi204 ceph-mon[36211]: osdmap e47: 7 total, 7 up, 7 in 2024-05-22T22:56:49.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.900+0000 7f064f1f0700 1 -- 172.21.15.96:0/2783648556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06481035b0 msgr2=0x7f0648103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:56:49.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.901+0000 7f064f1f0700 1 --2- 172.21.15.96:0/2783648556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06481035b0 0x7f0648103a10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f06340099c0 tx=0x7f0634009cd0 comp rx=0 tx=0).stop 2024-05-22T22:56:49.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.901+0000 7f064f1f0700 1 -- 172.21.15.96:0/2783648556 shutdown_connections 2024-05-22T22:56:49.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.901+0000 7f064f1f0700 1 --2- 172.21.15.96:0/2783648556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0648103f50 0x7f064810a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:56:49.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.901+0000 7f064f1f0700 1 --2- 172.21.15.96:0/2783648556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06481023b0 0x7f06481027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:56:49.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.901+0000 7f064f1f0700 1 --2- 172.21.15.96:0/2783648556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06481035b0 0x7f0648103a10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:56:49.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.901+0000 7f064f1f0700 1 -- 172.21.15.96:0/2783648556 >> 172.21.15.96:0/2783648556 conn(0x7f06480fd950 msgr2=0x7f06480ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:56:49.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.901+0000 7f064f1f0700 1 -- 172.21.15.96:0/2783648556 shutdown_connections 2024-05-22T22:56:49.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.901+0000 7f064f1f0700 1 -- 172.21.15.96:0/2783648556 wait complete. 2024-05-22T22:56:49.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.901+0000 7f064f1f0700 1 Processor -- start 2024-05-22T22:56:49.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.901+0000 7f064f1f0700 1 -- start start 2024-05-22T22:56:49.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.901+0000 7f064f1f0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06481023b0 0x7f0648198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:56:49.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.901+0000 7f064f1f0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06481035b0 0x7f0648198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:56:49.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.901+0000 7f064f1f0700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0648103f50 0x7f064819fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:56:49.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.901+0000 7f064f1f0700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f064810df90 con 0x7f0648103f50 2024-05-22T22:56:49.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.901+0000 7f064f1f0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f064810de10 con 0x7f06481023b0 2024-05-22T22:56:49.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.901+0000 7f064f1f0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f064810e110 con 0x7f06481035b0 2024-05-22T22:56:49.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.901+0000 7f064cf8c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06481023b0 0x7f0648198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:56:49.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.901+0000 7f064cf8c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06481023b0 0x7f0648198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48598/0 (socket says 172.21.15.96:48598) 2024-05-22T22:56:49.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.901+0000 7f064cf8c700 1 -- 172.21.15.96:0/1930457359 learned_addr learned my addr 172.21.15.96:0/1930457359 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-05-22T22:56:49.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.902+0000 7f0647fff700 1 --2- 172.21.15.96:0/1930457359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06481035b0 0x7f0648198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:56:49.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.902+0000 7f064cf8c700 1 -- 172.21.15.96:0/1930457359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06481035b0 msgr2=0x7f0648198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:56:49.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.902+0000 7f064cf8c700 1 --2- 172.21.15.96:0/1930457359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06481035b0 0x7f0648198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:56:49.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.902+0000 7f064cf8c700 1 -- 172.21.15.96:0/1930457359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0648103f50 msgr2=0x7f064819fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:56:49.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.902+0000 7f064cf8c700 1 --2- 172.21.15.96:0/1930457359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0648103f50 0x7f064819fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:56:49.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.902+0000 7f064cf8c700 1 -- 172.21.15.96:0/1930457359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06481a0170 con 0x7f06481023b0 2024-05-22T22:56:49.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.902+0000 7f064cf8c700 1 --2- 172.21.15.96:0/1930457359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06481023b0 0x7f0648198510 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f063c00c990 tx=0x7f063c00cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:56:49.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.902+0000 7f0645ffb700 1 -- 172.21.15.96:0/1930457359 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c007b00 con 0x7f06481023b0 2024-05-22T22:56:49.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.902+0000 7f064f1f0700 1 -- 172.21.15.96:0/1930457359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06481a0460 con 0x7f06481023b0 2024-05-22T22:56:49.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.902+0000 7f0645ffb700 1 -- 172.21.15.96:0/1930457359 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f063c007ca0 con 0x7f06481023b0 2024-05-22T22:56:49.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.902+0000 7f0645ffb700 1 -- 172.21.15.96:0/1930457359 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c0186f0 con 0x7f06481023b0 2024-05-22T22:56:49.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.902+0000 7f064f1f0700 1 -- 172.21.15.96:0/1930457359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06481a09f0 con 0x7f06481023b0 2024-05-22T22:56:49.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.903+0000 7f064f1f0700 1 -- 172.21.15.96:0/1930457359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0648069050 con 0x7f06481023b0 2024-05-22T22:56:49.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.904+0000 7f0645ffb700 1 -- 172.21.15.96:0/1930457359 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f063c018930 con 0x7f06481023b0 2024-05-22T22:56:49.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.905+0000 7f0645ffb700 1 --2- 172.21.15.96:0/1930457359 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f06380a2c20 0x7f06380a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:56:49.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.905+0000 7f0645ffb700 1 -- 172.21.15.96:0/1930457359 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f063c0ce310 con 0x7f06481023b0 2024-05-22T22:56:49.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.905+0000 7f0647fff700 1 --2- 172.21.15.96:0/1930457359 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f06380a2c20 0x7f06380a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:56:49.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.906+0000 7f0647fff700 1 --2- 172.21.15.96:0/1930457359 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f06380a2c20 0x7f06380a50e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0634000c00 tx=0x7f0634005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:56:49.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:49.909+0000 7f0645ffb700 1 -- 172.21.15.96:0/1930457359 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f063c014030 con 0x7f06481023b0 2024-05-22T22:56:50.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:56:50.119+0000 7f064f1f0700 1 -- 172.21.15.96:0/1930457359 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f06480627a0 con 0x7f06380a2c20 2024-05-22T22:56:50.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:50 smithi084 ceph-mon[32186]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:56:50.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:50 smithi204 ceph-mon[36211]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:56:50.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:50 smithi096 ceph-mon[36392]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:56:51.295 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:51 smithi096 ceph-mon[36392]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:56:51.295 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:51 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:56:51.295 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:51 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:56:51.295 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:51 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:51.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:51 smithi084 ceph-mon[32186]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:56:51.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:51 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:56:51.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:51 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:56:51.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:51 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:51.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:51 smithi204 ceph-mon[36211]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:56:51.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:51 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:56:51.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:51 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:56:51.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:51 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:56:52.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:52 smithi084 ceph-mon[32186]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:56:52.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:52 smithi204 ceph-mon[36211]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:56:52.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:52 smithi096 ceph-mon[36392]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:56:53.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:53 smithi084 ceph-mon[32186]: from='client.? 172.21.15.96:0/2217881780' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69a18c43-5447-480d-9466-190830070c3e"}]: dispatch 2024-05-22T22:56:53.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:53 smithi084 ceph-mon[32186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69a18c43-5447-480d-9466-190830070c3e"}]: dispatch 2024-05-22T22:56:53.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:53 smithi084 ceph-mon[32186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69a18c43-5447-480d-9466-190830070c3e"}]': finished 2024-05-22T22:56:53.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:53 smithi084 ceph-mon[32186]: osdmap e48: 8 total, 7 up, 8 in 2024-05-22T22:56:53.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:53 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-22T22:56:53.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:53 smithi204 ceph-mon[36211]: from='client.? 172.21.15.96:0/2217881780' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69a18c43-5447-480d-9466-190830070c3e"}]: dispatch 2024-05-22T22:56:53.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:53 smithi204 ceph-mon[36211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69a18c43-5447-480d-9466-190830070c3e"}]: dispatch 2024-05-22T22:56:53.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:53 smithi204 ceph-mon[36211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69a18c43-5447-480d-9466-190830070c3e"}]': finished 2024-05-22T22:56:53.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:53 smithi204 ceph-mon[36211]: osdmap e48: 8 total, 7 up, 8 in 2024-05-22T22:56:53.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:53 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-22T22:56:53.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:53 smithi096 ceph-mon[36392]: from='client.? 172.21.15.96:0/2217881780' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69a18c43-5447-480d-9466-190830070c3e"}]: dispatch 2024-05-22T22:56:53.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:53 smithi096 ceph-mon[36392]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69a18c43-5447-480d-9466-190830070c3e"}]: dispatch 2024-05-22T22:56:53.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:53 smithi096 ceph-mon[36392]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69a18c43-5447-480d-9466-190830070c3e"}]': finished 2024-05-22T22:56:53.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:53 smithi096 ceph-mon[36392]: osdmap e48: 8 total, 7 up, 8 in 2024-05-22T22:56:53.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:53 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-22T22:56:54.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:54 smithi084 ceph-mon[32186]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:56:54.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:54 smithi084 ceph-mon[32186]: from='client.? 172.21.15.96:0/2577844265' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:56:54.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:54 smithi204 ceph-mon[36211]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:56:54.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:54 smithi204 ceph-mon[36211]: from='client.? 172.21.15.96:0/2577844265' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:56:54.447 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:54 smithi096 ceph-mon[36392]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:56:54.447 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:54 smithi096 ceph-mon[36392]: from='client.? 172.21.15.96:0/2577844265' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:56:56.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:56 smithi096 ceph-mon[36392]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:56:56.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:56 smithi084 ceph-mon[32186]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:56:56.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:56 smithi204 ceph-mon[36211]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:56:58.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:56:58 smithi096 ceph-mon[36392]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:56:58.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:56:58 smithi084 ceph-mon[32186]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:56:58.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:56:58 smithi204 ceph-mon[36211]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:00.795 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:00 smithi096 ceph-mon[36392]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:00.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:00 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-22T22:57:00.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:00 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:00.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:00 smithi084 ceph-mon[32186]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:00.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:00 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-22T22:57:00.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:00 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:00.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:00 smithi204 ceph-mon[36211]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:00.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:00 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-22T22:57:00.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:00 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:01.771 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:01 smithi096 ceph-mon[36392]: Deploying daemon osd.7 on smithi096 2024-05-22T22:57:01.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:01 smithi084 ceph-mon[32186]: Deploying daemon osd.7 on smithi096 2024-05-22T22:57:01.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:01 smithi204 ceph-mon[36211]: Deploying daemon osd.7 on smithi096 2024-05-22T22:57:02.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:02 smithi096 ceph-mon[36392]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:02.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:02 smithi084 ceph-mon[32186]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:02.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:02 smithi204 ceph-mon[36211]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:04.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:04 smithi096 ceph-mon[36392]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:04.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:04 smithi084 ceph-mon[32186]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:04.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:04 smithi204 ceph-mon[36211]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:06.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:06 smithi096 ceph-mon[36392]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:06.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:06.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:06.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:57:06.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:06 smithi084 ceph-mon[32186]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:06.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:06.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:06.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:57:06.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:06 smithi204 ceph-mon[36211]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:06.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:06.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:06.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:57:08.672 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:08 smithi096 ceph-mon[36392]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:08.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:08 smithi084 ceph-mon[32186]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:08.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:08 smithi204 ceph-mon[36211]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:09.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:57:09.531+0000 7f0645ffb700 1 -- 172.21.15.96:0/1930457359 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f06480627a0 con 0x7f06380a2c20 2024-05-22T22:57:09.529 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 7 on host 'smithi096' 2024-05-22T22:57:09.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:57:09.534+0000 7f064f1f0700 1 -- 172.21.15.96:0/1930457359 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f06380a2c20 msgr2=0x7f06380a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:57:09.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:57:09.534+0000 7f064f1f0700 1 --2- 172.21.15.96:0/1930457359 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f06380a2c20 0x7f06380a50e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0634000c00 tx=0x7f0634005fd0 comp rx=0 tx=0).stop 2024-05-22T22:57:09.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:57:09.534+0000 7f064f1f0700 1 -- 172.21.15.96:0/1930457359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06481023b0 msgr2=0x7f0648198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:57:09.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:57:09.534+0000 7f064f1f0700 1 --2- 172.21.15.96:0/1930457359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06481023b0 0x7f0648198510 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f063c00c990 tx=0x7f063c00cd50 comp rx=0 tx=0).stop 2024-05-22T22:57:09.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:57:09.534+0000 7f064f1f0700 1 -- 172.21.15.96:0/1930457359 shutdown_connections 2024-05-22T22:57:09.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:57:09.534+0000 7f064f1f0700 1 --2- 172.21.15.96:0/1930457359 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f06380a2c20 0x7f06380a50e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:57:09.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:57:09.534+0000 7f064f1f0700 1 --2- 172.21.15.96:0/1930457359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0648103f50 0x7f064819fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:57:09.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:57:09.534+0000 7f064f1f0700 1 --2- 172.21.15.96:0/1930457359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06481023b0 0x7f0648198510 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:57:09.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:57:09.534+0000 7f064f1f0700 1 --2- 172.21.15.96:0/1930457359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06481035b0 0x7f0648198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:57:09.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:57:09.534+0000 7f064f1f0700 1 -- 172.21.15.96:0/1930457359 >> 172.21.15.96:0/1930457359 conn(0x7f06480fd950 msgr2=0x7f06481068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:57:09.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:57:09.534+0000 7f064f1f0700 1 -- 172.21.15.96:0/1930457359 shutdown_connections 2024-05-22T22:57:09.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:57:09.534+0000 7f064f1f0700 1 -- 172.21.15.96:0/1930457359 wait complete. 2024-05-22T22:57:10.188 DEBUG:teuthology.orchestra.run.smithi096:osd.7> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.7.service 2024-05-22T22:57:10.191 INFO:tasks.cephadm:Deploying osd.8 on smithi204 with /dev/vg_nvme/lv_2... 2024-05-22T22:57:10.191 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-22T22:57:10.434 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.c/config 2024-05-22T22:57:10.739 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:10 smithi096 ceph-mon[36392]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:10.739 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:10 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:10.739 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:10 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:10.739 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:10 smithi096 ceph-mon[36392]: from='osd.7 [v2:172.21.15.96:6816/3195792774,v1:172.21.15.96:6817/3195792774]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-22T22:57:10.739 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:10 smithi096 ceph-mon[36392]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-22T22:57:10.739 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:10 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:10.739 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:10 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:10.794 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:10 smithi204 ceph-mon[36211]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:10.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:10 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:10.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:10 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:10.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:10 smithi204 ceph-mon[36211]: from='osd.7 [v2:172.21.15.96:6816/3195792774,v1:172.21.15.96:6817/3195792774]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-22T22:57:10.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:10 smithi204 ceph-mon[36211]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-22T22:57:10.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:10 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:10.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:10 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:10.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:10 smithi084 ceph-mon[32186]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:10.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:10 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:10.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:10 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:10.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:10 smithi084 ceph-mon[32186]: from='osd.7 [v2:172.21.15.96:6816/3195792774,v1:172.21.15.96:6817/3195792774]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-22T22:57:10.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:10 smithi084 ceph-mon[32186]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-22T22:57:10.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:10 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:10.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:10 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:11.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:11 smithi096 ceph-mon[36392]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-22T22:57:11.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:11 smithi096 ceph-mon[36392]: osdmap e49: 8 total, 7 up, 8 in 2024-05-22T22:57:11.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:11 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-22T22:57:11.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:11 smithi096 ceph-mon[36392]: from='osd.7 [v2:172.21.15.96:6816/3195792774,v1:172.21.15.96:6817/3195792774]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-22T22:57:11.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:11 smithi096 ceph-mon[36392]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-22T22:57:11.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:11 smithi084 ceph-mon[32186]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-22T22:57:11.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:11 smithi084 ceph-mon[32186]: osdmap e49: 8 total, 7 up, 8 in 2024-05-22T22:57:11.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:11 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-22T22:57:11.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:11 smithi084 ceph-mon[32186]: from='osd.7 [v2:172.21.15.96:6816/3195792774,v1:172.21.15.96:6817/3195792774]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-22T22:57:11.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:11 smithi084 ceph-mon[32186]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-22T22:57:11.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:11 smithi204 ceph-mon[36211]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-22T22:57:11.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:11 smithi204 ceph-mon[36211]: osdmap e49: 8 total, 7 up, 8 in 2024-05-22T22:57:11.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:11 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-22T22:57:11.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:11 smithi204 ceph-mon[36211]: from='osd.7 [v2:172.21.15.96:6816/3195792774,v1:172.21.15.96:6817/3195792774]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-22T22:57:11.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:11 smithi204 ceph-mon[36211]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-22T22:57:12.296 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 22:57:12 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T22:57:11.999+0000 7f6b6bcaf700 -1 osd.7 0 waiting for initial osdmap 2024-05-22T22:57:12.296 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 22:57:12 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T22:57:12.011+0000 7f6b63a9a700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-22T22:57:12.680 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-22T22:57:12.680 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-22T22:57:12.680 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-05-22T22:57:12.680 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-05-22T22:57:12.680 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0119037 s, 881 MB/s 2024-05-22T22:57:12.680 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-05-22T22:57:12.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:12 smithi096 ceph-mon[36392]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:12.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:12 smithi096 ceph-mon[36392]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-22T22:57:12.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:12 smithi096 ceph-mon[36392]: osdmap e50: 8 total, 7 up, 8 in 2024-05-22T22:57:12.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:12 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-22T22:57:12.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:12 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-22T22:57:12.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:12 smithi096 ceph-mon[36392]: osdmap e51: 8 total, 7 up, 8 in 2024-05-22T22:57:12.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:12 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-22T22:57:12.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:12 smithi096 ceph-mon[36392]: from='osd.7 ' entity='osd.7' 2024-05-22T22:57:12.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:12 smithi084 ceph-mon[32186]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:12.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:12 smithi084 ceph-mon[32186]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-22T22:57:12.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:12 smithi084 ceph-mon[32186]: osdmap e50: 8 total, 7 up, 8 in 2024-05-22T22:57:12.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:12 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-22T22:57:12.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:12 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-22T22:57:12.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:12 smithi084 ceph-mon[32186]: osdmap e51: 8 total, 7 up, 8 in 2024-05-22T22:57:12.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:12 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-22T22:57:12.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:12 smithi084 ceph-mon[32186]: from='osd.7 ' entity='osd.7' 2024-05-22T22:57:12.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:12 smithi204 ceph-mon[36211]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:12.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:12 smithi204 ceph-mon[36211]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-22T22:57:12.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:12 smithi204 ceph-mon[36211]: osdmap e50: 8 total, 7 up, 8 in 2024-05-22T22:57:12.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:12 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-22T22:57:12.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:12 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-22T22:57:12.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:12 smithi204 ceph-mon[36211]: osdmap e51: 8 total, 7 up, 8 in 2024-05-22T22:57:12.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:12 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-22T22:57:12.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:12 smithi204 ceph-mon[36211]: from='osd.7 ' entity='osd.7' 2024-05-22T22:57:13.343 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_2 2024-05-22T22:57:13.592 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.c/config 2024-05-22T22:57:13.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:13 smithi204 ceph-mon[36211]: purged_snaps scrub starts 2024-05-22T22:57:13.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:13 smithi204 ceph-mon[36211]: purged_snaps scrub ok 2024-05-22T22:57:13.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:13 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-22T22:57:13.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:13 smithi204 ceph-mon[36211]: osd.7 [v2:172.21.15.96:6816/3195792774,v1:172.21.15.96:6817/3195792774] boot 2024-05-22T22:57:13.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:13 smithi204 ceph-mon[36211]: osdmap e52: 8 total, 8 up, 8 in 2024-05-22T22:57:13.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:13 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-22T22:57:13.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:13 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:13.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:13 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:13.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:13 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:57:13.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:13 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:57:13.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:13 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:57:13.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:13 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:13.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:13 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:13.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:13 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:57:13.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:13 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:13.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:13 smithi096 ceph-mon[36392]: purged_snaps scrub starts 2024-05-22T22:57:13.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:13 smithi096 ceph-mon[36392]: purged_snaps scrub ok 2024-05-22T22:57:13.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:13 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-22T22:57:13.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:13 smithi096 ceph-mon[36392]: osd.7 [v2:172.21.15.96:6816/3195792774,v1:172.21.15.96:6817/3195792774] boot 2024-05-22T22:57:13.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:13 smithi096 ceph-mon[36392]: osdmap e52: 8 total, 8 up, 8 in 2024-05-22T22:57:13.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:13 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-22T22:57:13.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:13 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:13.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:13 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:13.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:13 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:57:13.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:13 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:57:13.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:13 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:57:13.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:13 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:13.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:13 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:13.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:13 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:57:13.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:13 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:13.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:13 smithi084 ceph-mon[32186]: purged_snaps scrub starts 2024-05-22T22:57:13.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:13 smithi084 ceph-mon[32186]: purged_snaps scrub ok 2024-05-22T22:57:13.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:13 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-22T22:57:13.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:13 smithi084 ceph-mon[32186]: osd.7 [v2:172.21.15.96:6816/3195792774,v1:172.21.15.96:6817/3195792774] boot 2024-05-22T22:57:13.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:13 smithi084 ceph-mon[32186]: osdmap e52: 8 total, 8 up, 8 in 2024-05-22T22:57:13.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:13 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-22T22:57:13.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:13 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:13.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:13 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:13.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:13 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:57:13.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:13 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:57:13.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:13 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:57:13.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:13 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:13.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:13 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:13.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:13 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:57:13.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:13 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:14.789 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:14 smithi204 ceph-mon[36211]: Detected new or changed devices on smithi096 2024-05-22T22:57:14.789 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:14 smithi204 ceph-mon[36211]: Adjusting osd_memory_target on smithi096 to 5696M 2024-05-22T22:57:14.789 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:14 smithi204 ceph-mon[36211]: pgmap v176: 1 pgs: 1 remapped+peering; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:14.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:14 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:57:14.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:14 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:14.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:14 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:57:14.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:14 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:14.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:14 smithi204 ceph-mon[36211]: osdmap e53: 8 total, 8 up, 8 in 2024-05-22T22:57:14.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:14 smithi096 ceph-mon[36392]: Detected new or changed devices on smithi096 2024-05-22T22:57:14.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:14 smithi096 ceph-mon[36392]: Adjusting osd_memory_target on smithi096 to 5696M 2024-05-22T22:57:14.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:14 smithi096 ceph-mon[36392]: pgmap v176: 1 pgs: 1 remapped+peering; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:14.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:14 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:57:14.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:14 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:14.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:14 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:57:14.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:14 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:14.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:14 smithi096 ceph-mon[36392]: osdmap e53: 8 total, 8 up, 8 in 2024-05-22T22:57:14.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:14 smithi084 ceph-mon[32186]: Detected new or changed devices on smithi096 2024-05-22T22:57:14.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:14 smithi084 ceph-mon[32186]: Adjusting osd_memory_target on smithi096 to 5696M 2024-05-22T22:57:14.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:14 smithi084 ceph-mon[32186]: pgmap v176: 1 pgs: 1 remapped+peering; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-22T22:57:14.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:14 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:57:14.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:14 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:14.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:14 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:57:14.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:14 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:14.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:14 smithi084 ceph-mon[32186]: osdmap e53: 8 total, 8 up, 8 in 2024-05-22T22:57:15.376 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.350+0000 7f1707ebc700 1 -- 172.21.15.204:0/2166202118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17001023f0 msgr2=0x7f1700102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:57:15.376 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.350+0000 7f1707ebc700 1 --2- 172.21.15.204:0/2166202118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17001023f0 0x7f1700102810 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009990 tx=0x7f16fc009ca0 comp rx=0 tx=0).stop 2024-05-22T22:57:15.377 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.350+0000 7f1707ebc700 1 -- 172.21.15.204:0/2166202118 shutdown_connections 2024-05-22T22:57:15.377 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.350+0000 7f1707ebc700 1 --2- 172.21.15.204:0/2166202118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1700103f90 0x7f170010a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:57:15.377 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.350+0000 7f1707ebc700 1 --2- 172.21.15.204:0/2166202118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17001023f0 0x7f1700102810 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:57:15.377 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.350+0000 7f1707ebc700 1 --2- 172.21.15.204:0/2166202118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17001035f0 0x7f1700103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:57:15.377 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.350+0000 7f1707ebc700 1 -- 172.21.15.204:0/2166202118 >> 172.21.15.204:0/2166202118 conn(0x7f17000fd9b0 msgr2=0x7f17000ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:57:15.377 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.350+0000 7f1707ebc700 1 -- 172.21.15.204:0/2166202118 shutdown_connections 2024-05-22T22:57:15.377 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.350+0000 7f1707ebc700 1 -- 172.21.15.204:0/2166202118 wait complete. 2024-05-22T22:57:15.377 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.351+0000 7f1707ebc700 1 Processor -- start 2024-05-22T22:57:15.377 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.351+0000 7f1707ebc700 1 -- start start 2024-05-22T22:57:15.377 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.351+0000 7f1707ebc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17001023f0 0x7f1700198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:57:15.377 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.351+0000 7f1707ebc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17001035f0 0x7f1700198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:57:15.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.351+0000 7f1707ebc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1700103f90 0x7f170019fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:57:15.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.351+0000 7f1707ebc700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f170010dfd0 con 0x7f17001035f0 2024-05-22T22:57:15.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.351+0000 7f1707ebc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f170010de50 con 0x7f17001023f0 2024-05-22T22:57:15.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.351+0000 7f1707ebc700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f170010e150 con 0x7f1700103f90 2024-05-22T22:57:15.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.351+0000 7f1706459700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1700103f90 0x7f170019fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:57:15.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.351+0000 7f1706459700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1700103f90 0x7f170019fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33464/0 (socket says 172.21.15.204:33464) 2024-05-22T22:57:15.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.351+0000 7f1706459700 1 -- 172.21.15.204:0/4245303635 learned_addr learned my addr 172.21.15.204:0/4245303635 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-22T22:57:15.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.351+0000 7f1705c58700 1 --2- 172.21.15.204:0/4245303635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17001023f0 0x7f1700198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:57:15.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.351+0000 7f1705457700 1 --2- 172.21.15.204:0/4245303635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17001035f0 0x7f1700198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:57:15.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.351+0000 7f1706459700 1 -- 172.21.15.204:0/4245303635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17001023f0 msgr2=0x7f1700198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:57:15.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.351+0000 7f1706459700 1 --2- 172.21.15.204:0/4245303635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17001023f0 0x7f1700198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:57:15.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.351+0000 7f1706459700 1 -- 172.21.15.204:0/4245303635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17001035f0 msgr2=0x7f1700198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:57:15.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.351+0000 7f1706459700 1 --2- 172.21.15.204:0/4245303635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17001035f0 0x7f1700198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:57:15.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.351+0000 7f1706459700 1 -- 172.21.15.204:0/4245303635 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17001a01f0 con 0x7f1700103f90 2024-05-22T22:57:15.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.352+0000 7f1705457700 1 --2- 172.21.15.204:0/4245303635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17001035f0 0x7f1700198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T22:57:15.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.352+0000 7f1706459700 1 --2- 172.21.15.204:0/4245303635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1700103f90 0x7f170019fb50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f16f8007f10 tx=0x7f16f800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:57:15.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.352+0000 7f16f6ffd700 1 -- 172.21.15.204:0/4245303635 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f8004bb0 con 0x7f1700103f90 2024-05-22T22:57:15.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.352+0000 7f1707ebc700 1 -- 172.21.15.204:0/4245303635 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17001a04e0 con 0x7f1700103f90 2024-05-22T22:57:15.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.352+0000 7f16f6ffd700 1 -- 172.21.15.204:0/4245303635 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16f8004d50 con 0x7f1700103f90 2024-05-22T22:57:15.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.352+0000 7f16f6ffd700 1 -- 172.21.15.204:0/4245303635 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f8005a60 con 0x7f1700103f90 2024-05-22T22:57:15.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.352+0000 7f1707ebc700 1 -- 172.21.15.204:0/4245303635 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17001a0a70 con 0x7f1700103f90 2024-05-22T22:57:15.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.353+0000 7f1707ebc700 1 -- 172.21.15.204:0/4245303635 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1700069050 con 0x7f1700103f90 2024-05-22T22:57:15.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.354+0000 7f16f6ffd700 1 -- 172.21.15.204:0/4245303635 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f16f801a020 con 0x7f1700103f90 2024-05-22T22:57:15.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.354+0000 7f16f6ffd700 1 --2- 172.21.15.204:0/4245303635 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f16ec0a2c20 0x7f16ec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:57:15.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.355+0000 7f16f6ffd700 1 -- 172.21.15.204:0/4245303635 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7f16f80ce2d0 con 0x7f1700103f90 2024-05-22T22:57:15.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.355+0000 7f1705c58700 1 --2- 172.21.15.204:0/4245303635 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f16ec0a2c20 0x7f16ec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:57:15.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.355+0000 7f1705c58700 1 --2- 172.21.15.204:0/4245303635 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f16ec0a2c20 0x7f16ec0a50e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f16fc003890 tx=0x7f16fc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:57:15.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.358+0000 7f16f6ffd700 1 -- 172.21.15.204:0/4245303635 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f16f8010070 con 0x7f1700103f90 2024-05-22T22:57:15.560 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:15.562+0000 7f1707ebc700 1 -- 172.21.15.204:0/4245303635 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f17000627a0 con 0x7f16ec0a2c20 2024-05-22T22:57:16.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:15 smithi204 ceph-mon[36211]: osdmap e54: 8 total, 8 up, 8 in 2024-05-22T22:57:16.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:15 smithi204 ceph-mon[36211]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:16.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:15 smithi204 ceph-mon[36211]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:57:16.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:15 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:57:16.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:15 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:57:16.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:15 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:16.296 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:15 smithi096 ceph-mon[36392]: osdmap e54: 8 total, 8 up, 8 in 2024-05-22T22:57:16.296 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:15 smithi096 ceph-mon[36392]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:16.296 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:15 smithi096 ceph-mon[36392]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:57:16.296 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:15 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:57:16.296 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:15 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:57:16.296 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:15 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:16.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:15 smithi084 ceph-mon[32186]: osdmap e54: 8 total, 8 up, 8 in 2024-05-22T22:57:16.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:15 smithi084 ceph-mon[32186]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:16.372 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:15 smithi084 ceph-mon[32186]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:57:16.372 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:15 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:57:16.372 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:15 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:57:16.372 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:15 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:17.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:16 smithi204 ceph-mon[36211]: osdmap e55: 8 total, 8 up, 8 in 2024-05-22T22:57:17.296 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:16 smithi096 ceph-mon[36392]: osdmap e55: 8 total, 8 up, 8 in 2024-05-22T22:57:17.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:16 smithi084 ceph-mon[32186]: osdmap e55: 8 total, 8 up, 8 in 2024-05-22T22:57:18.296 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:17 smithi096 ceph-mon[36392]: osdmap e56: 8 total, 8 up, 8 in 2024-05-22T22:57:18.296 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:17 smithi096 ceph-mon[36392]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:18.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:17 smithi084 ceph-mon[32186]: osdmap e56: 8 total, 8 up, 8 in 2024-05-22T22:57:18.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:17 smithi084 ceph-mon[32186]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:18.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:17 smithi204 ceph-mon[36211]: osdmap e56: 8 total, 8 up, 8 in 2024-05-22T22:57:18.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:17 smithi204 ceph-mon[36211]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:19.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:18 smithi204 ceph-mon[36211]: from='client.? 172.21.15.204:0/1687423132' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c5cadca-689a-441e-847c-ba370d400ef1"}]: dispatch 2024-05-22T22:57:19.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:18 smithi204 ceph-mon[36211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c5cadca-689a-441e-847c-ba370d400ef1"}]: dispatch 2024-05-22T22:57:19.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:18 smithi204 ceph-mon[36211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c5cadca-689a-441e-847c-ba370d400ef1"}]': finished 2024-05-22T22:57:19.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:18 smithi204 ceph-mon[36211]: osdmap e57: 9 total, 8 up, 9 in 2024-05-22T22:57:19.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:18 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-22T22:57:19.296 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:18 smithi096 ceph-mon[36392]: from='client.? 172.21.15.204:0/1687423132' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c5cadca-689a-441e-847c-ba370d400ef1"}]: dispatch 2024-05-22T22:57:19.296 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:18 smithi096 ceph-mon[36392]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c5cadca-689a-441e-847c-ba370d400ef1"}]: dispatch 2024-05-22T22:57:19.296 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:18 smithi096 ceph-mon[36392]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c5cadca-689a-441e-847c-ba370d400ef1"}]': finished 2024-05-22T22:57:19.296 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:18 smithi096 ceph-mon[36392]: osdmap e57: 9 total, 8 up, 9 in 2024-05-22T22:57:19.296 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:18 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-22T22:57:19.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:18 smithi084 ceph-mon[32186]: from='client.? 172.21.15.204:0/1687423132' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c5cadca-689a-441e-847c-ba370d400ef1"}]: dispatch 2024-05-22T22:57:19.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:18 smithi084 ceph-mon[32186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c5cadca-689a-441e-847c-ba370d400ef1"}]: dispatch 2024-05-22T22:57:19.372 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:18 smithi084 ceph-mon[32186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c5cadca-689a-441e-847c-ba370d400ef1"}]': finished 2024-05-22T22:57:19.372 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:18 smithi084 ceph-mon[32186]: osdmap e57: 9 total, 8 up, 9 in 2024-05-22T22:57:19.372 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:18 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-22T22:57:20.296 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:19 smithi096 ceph-mon[36392]: from='client.? 172.21.15.204:0/475123617' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:57:20.296 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:19 smithi096 ceph-mon[36392]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:20.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:19 smithi084 ceph-mon[32186]: from='client.? 172.21.15.204:0/475123617' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:57:20.371 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:19 smithi084 ceph-mon[32186]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:20.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:19 smithi204 ceph-mon[36211]: from='client.? 172.21.15.204:0/475123617' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:57:20.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:19 smithi204 ceph-mon[36211]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:22.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:22 smithi096 ceph-mon[36392]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:22.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:22 smithi084 ceph-mon[32186]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:22.903 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:22 smithi204 ceph-mon[36211]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:24.770 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:24 smithi204 ceph-mon[36211]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:24.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:24 smithi096 ceph-mon[36392]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:24.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:24 smithi084 ceph-mon[32186]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:26.784 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:26 smithi096 ceph-mon[36392]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:26.785 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:26 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-22T22:57:26.785 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:26 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:26.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:26 smithi084 ceph-mon[32186]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:26.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:26 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-22T22:57:26.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:26 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:26.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:26 smithi204 ceph-mon[36211]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:26.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:26 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-22T22:57:26.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:26 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:27.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:27 smithi204 ceph-mon[36211]: Deploying daemon osd.8 on smithi204 2024-05-22T22:57:27.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:27 smithi096 ceph-mon[36392]: Deploying daemon osd.8 on smithi204 2024-05-22T22:57:27.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:27 smithi084 ceph-mon[32186]: Deploying daemon osd.8 on smithi204 2024-05-22T22:57:28.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:28 smithi204 ceph-mon[36211]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:28.786 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:28 smithi084 ceph-mon[32186]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:28.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:28 smithi096 ceph-mon[36392]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:30.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:30 smithi096 ceph-mon[36392]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:30.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:30 smithi084 ceph-mon[32186]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:30.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:30 smithi204 ceph-mon[36211]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:32.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:32 smithi204 ceph-mon[36211]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:32.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:32 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:57:32.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:32 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:32.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:32 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:32.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:32 smithi096 ceph-mon[36392]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:32.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:32 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:57:32.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:32 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:32.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:32 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:32.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:32 smithi084 ceph-mon[32186]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:32.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:32 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:57:32.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:32 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:32.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:32 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:34.770 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:34 smithi204 ceph-mon[36211]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:34.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:34 smithi096 ceph-mon[36392]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:34.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:34 smithi084 ceph-mon[32186]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:35.581 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:35.583+0000 7f16f6ffd700 1 -- 172.21.15.204:0/4245303635 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f17000627a0 con 0x7f16ec0a2c20 2024-05-22T22:57:35.582 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 8 on host 'smithi204' 2024-05-22T22:57:35.584 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:35.586+0000 7f1707ebc700 1 -- 172.21.15.204:0/4245303635 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f16ec0a2c20 msgr2=0x7f16ec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:57:35.584 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:35.586+0000 7f1707ebc700 1 --2- 172.21.15.204:0/4245303635 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f16ec0a2c20 0x7f16ec0a50e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f16fc003890 tx=0x7f16fc005fd0 comp rx=0 tx=0).stop 2024-05-22T22:57:35.585 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:35.586+0000 7f1707ebc700 1 -- 172.21.15.204:0/4245303635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1700103f90 msgr2=0x7f170019fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:57:35.585 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:35.586+0000 7f1707ebc700 1 --2- 172.21.15.204:0/4245303635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1700103f90 0x7f170019fb50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f16f8007f10 tx=0x7f16f800d3b0 comp rx=0 tx=0).stop 2024-05-22T22:57:35.585 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:35.586+0000 7f1707ebc700 1 -- 172.21.15.204:0/4245303635 shutdown_connections 2024-05-22T22:57:35.585 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:35.586+0000 7f1707ebc700 1 --2- 172.21.15.204:0/4245303635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1700103f90 0x7f170019fb50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:57:35.585 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:35.586+0000 7f1707ebc700 1 --2- 172.21.15.204:0/4245303635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17001023f0 0x7f1700198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:57:35.585 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:35.586+0000 7f1707ebc700 1 --2- 172.21.15.204:0/4245303635 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f16ec0a2c20 0x7f16ec0a50e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:57:35.585 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:35.586+0000 7f1707ebc700 1 --2- 172.21.15.204:0/4245303635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17001035f0 0x7f1700198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:57:35.585 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:35.586+0000 7f1707ebc700 1 -- 172.21.15.204:0/4245303635 >> 172.21.15.204:0/4245303635 conn(0x7f17000fd9b0 msgr2=0x7f17001068f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:57:35.585 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:35.587+0000 7f1707ebc700 1 -- 172.21.15.204:0/4245303635 shutdown_connections 2024-05-22T22:57:35.585 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:57:35.587+0000 7f1707ebc700 1 -- 172.21.15.204:0/4245303635 wait complete. 2024-05-22T22:57:36.285 DEBUG:teuthology.orchestra.run.smithi204:osd.8> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.8.service 2024-05-22T22:57:36.287 INFO:tasks.cephadm:Deploying osd.9 on smithi084 with /dev/vg_nvme/lv_1... 2024-05-22T22:57:36.288 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-22T22:57:36.540 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T22:57:36.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:36 smithi204 ceph-mon[36211]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:36.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:36 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:36.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:36 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:36.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:36 smithi204 ceph-mon[36211]: from='osd.8 [v2:172.21.15.204:6816/606997927,v1:172.21.15.204:6817/606997927]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-22T22:57:36.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:36 smithi204 ceph-mon[36211]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-22T22:57:36.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:36 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:36.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:36 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:36.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:36 smithi096 ceph-mon[36392]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:36.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:36 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:36.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:36 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:36.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:36 smithi096 ceph-mon[36392]: from='osd.8 [v2:172.21.15.204:6816/606997927,v1:172.21.15.204:6817/606997927]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-22T22:57:36.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:36 smithi096 ceph-mon[36392]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-22T22:57:36.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:36 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:36.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:36 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:36.621 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:36 smithi084 ceph-mon[32186]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:36.622 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:36 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:36.622 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:36 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:36.622 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:36 smithi084 ceph-mon[32186]: from='osd.8 [v2:172.21.15.204:6816/606997927,v1:172.21.15.204:6817/606997927]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-22T22:57:36.622 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:36 smithi084 ceph-mon[32186]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-22T22:57:36.622 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:36 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:36.622 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:36 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:37.554 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:37 smithi204 ceph-mon[36211]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-22T22:57:37.554 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:37 smithi204 ceph-mon[36211]: from='osd.8 [v2:172.21.15.204:6816/606997927,v1:172.21.15.204:6817/606997927]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-22T22:57:37.554 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:37 smithi204 ceph-mon[36211]: osdmap e58: 9 total, 8 up, 9 in 2024-05-22T22:57:37.554 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:37 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-22T22:57:37.554 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:37 smithi204 ceph-mon[36211]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-22T22:57:37.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:37 smithi084 ceph-mon[32186]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-22T22:57:37.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:37 smithi084 ceph-mon[32186]: from='osd.8 [v2:172.21.15.204:6816/606997927,v1:172.21.15.204:6817/606997927]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-22T22:57:37.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:37 smithi084 ceph-mon[32186]: osdmap e58: 9 total, 8 up, 9 in 2024-05-22T22:57:37.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:37 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-22T22:57:37.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:37 smithi084 ceph-mon[32186]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-22T22:57:38.046 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:37 smithi096 ceph-mon[36392]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-22T22:57:38.046 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:37 smithi096 ceph-mon[36392]: from='osd.8 [v2:172.21.15.204:6816/606997927,v1:172.21.15.204:6817/606997927]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-22T22:57:38.046 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:37 smithi096 ceph-mon[36392]: osdmap e58: 9 total, 8 up, 9 in 2024-05-22T22:57:38.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:37 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-22T22:57:38.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:37 smithi096 ceph-mon[36392]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-22T22:57:38.222 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 22:57:37 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T22:57:37.967+0000 7f34d5aed700 -1 osd.8 0 waiting for initial osdmap 2024-05-22T22:57:38.222 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 22:57:37 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T22:57:37.976+0000 7f34cd0d7700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-22T22:57:38.834 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:38 smithi084 ceph-mon[32186]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:38.834 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:38 smithi084 ceph-mon[32186]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-22T22:57:38.834 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:38 smithi084 ceph-mon[32186]: osdmap e59: 9 total, 8 up, 9 in 2024-05-22T22:57:38.834 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:38 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-22T22:57:38.834 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:38 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-22T22:57:38.834 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:38 smithi084 ceph-mon[32186]: from='osd.8 ' entity='osd.8' 2024-05-22T22:57:38.834 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:38 smithi084 ceph-mon[32186]: osd.8 [v2:172.21.15.204:6816/606997927,v1:172.21.15.204:6817/606997927] boot 2024-05-22T22:57:38.835 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:38 smithi084 ceph-mon[32186]: osdmap e60: 9 total, 9 up, 9 in 2024-05-22T22:57:38.835 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:38 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-22T22:57:38.835 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-22T22:57:38.835 INFO:teuthology.orchestra.run.smithi084.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-22T22:57:38.835 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10+0 records in 2024-05-22T22:57:38.835 INFO:teuthology.orchestra.run.smithi084.stderr:10+0 records out 2024-05-22T22:57:38.835 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0119267 s, 879 MB/s 2024-05-22T22:57:38.835 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping successful for: 2024-05-22T22:57:38.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:38 smithi204 ceph-mon[36211]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:38.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:38 smithi204 ceph-mon[36211]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-22T22:57:38.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:38 smithi204 ceph-mon[36211]: osdmap e59: 9 total, 8 up, 9 in 2024-05-22T22:57:38.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:38 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-22T22:57:38.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:38 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-22T22:57:38.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:38 smithi204 ceph-mon[36211]: from='osd.8 ' entity='osd.8' 2024-05-22T22:57:38.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:38 smithi204 ceph-mon[36211]: osd.8 [v2:172.21.15.204:6816/606997927,v1:172.21.15.204:6817/606997927] boot 2024-05-22T22:57:38.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:38 smithi204 ceph-mon[36211]: osdmap e60: 9 total, 9 up, 9 in 2024-05-22T22:57:38.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:38 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-22T22:57:39.046 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:38 smithi096 ceph-mon[36392]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-22T22:57:39.046 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:38 smithi096 ceph-mon[36392]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-22T22:57:39.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:38 smithi096 ceph-mon[36392]: osdmap e59: 9 total, 8 up, 9 in 2024-05-22T22:57:39.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:38 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-22T22:57:39.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:38 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-22T22:57:39.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:38 smithi096 ceph-mon[36392]: from='osd.8 ' entity='osd.8' 2024-05-22T22:57:39.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:38 smithi096 ceph-mon[36392]: osd.8 [v2:172.21.15.204:6816/606997927,v1:172.21.15.204:6817/606997927] boot 2024-05-22T22:57:39.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:38 smithi096 ceph-mon[36392]: osdmap e60: 9 total, 9 up, 9 in 2024-05-22T22:57:39.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:38 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-22T22:57:39.647 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi084:vg_nvme/lv_1 2024-05-22T22:57:39.871 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:39 smithi084 ceph-mon[32186]: purged_snaps scrub starts 2024-05-22T22:57:39.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:39 smithi084 ceph-mon[32186]: purged_snaps scrub ok 2024-05-22T22:57:39.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:39 smithi084 ceph-mon[32186]: osdmap e61: 9 total, 9 up, 9 in 2024-05-22T22:57:39.887 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T22:57:39.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:39 smithi204 ceph-mon[36211]: purged_snaps scrub starts 2024-05-22T22:57:39.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:39 smithi204 ceph-mon[36211]: purged_snaps scrub ok 2024-05-22T22:57:39.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:39 smithi204 ceph-mon[36211]: osdmap e61: 9 total, 9 up, 9 in 2024-05-22T22:57:40.046 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:39 smithi096 ceph-mon[36392]: purged_snaps scrub starts 2024-05-22T22:57:40.046 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:39 smithi096 ceph-mon[36392]: purged_snaps scrub ok 2024-05-22T22:57:40.046 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:39 smithi096 ceph-mon[36392]: osdmap e61: 9 total, 9 up, 9 in 2024-05-22T22:57:40.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:40 smithi084 ceph-mon[32186]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 297 MiB used, 804 GiB / 805 GiB avail 2024-05-22T22:57:40.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:40 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:40.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:40 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:40.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:40 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:57:40.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:40 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:57:40.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:40 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:57:40.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:40 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:40.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:40 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:40.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:40 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:57:40.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:40 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:40.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:40 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:57:40.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:40 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:40.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:40 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:57:40.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:40 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:40.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:40 smithi084 ceph-mon[32186]: osdmap e62: 9 total, 9 up, 9 in 2024-05-22T22:57:40.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:40 smithi204 ceph-mon[36211]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 297 MiB used, 804 GiB / 805 GiB avail 2024-05-22T22:57:40.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:40 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:40.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:40 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:40.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:40 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:57:40.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:40 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:57:40.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:40 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:57:40.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:40 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:40.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:40 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:40.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:40 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:57:40.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:40 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:40.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:40 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:57:40.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:40 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:40.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:40 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:57:40.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:40 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:40.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:40 smithi204 ceph-mon[36211]: osdmap e62: 9 total, 9 up, 9 in 2024-05-22T22:57:41.046 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:40 smithi096 ceph-mon[36392]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 297 MiB used, 804 GiB / 805 GiB avail 2024-05-22T22:57:41.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:40 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:41.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:40 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:41.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:40 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:57:41.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:40 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:57:41.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:40 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:57:41.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:40 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:41.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:40 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:41.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:40 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:57:41.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:40 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:41.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:40 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:57:41.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:40 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:41.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:40 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:57:41.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:40 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:41.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:40 smithi096 ceph-mon[36392]: osdmap e62: 9 total, 9 up, 9 in 2024-05-22T22:57:41.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.570+0000 7f7856618700 1 -- 172.21.15.84:0/3402214926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7850101f20 msgr2=0x7f78501097c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:57:41.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.570+0000 7f7856618700 1 --2- 172.21.15.84:0/3402214926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7850101f20 0x7f78501097c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f784800b0a0 tx=0x7f784800b3b0 comp rx=0 tx=0).stop 2024-05-22T22:57:41.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.570+0000 7f7856618700 1 -- 172.21.15.84:0/3402214926 shutdown_connections 2024-05-22T22:57:41.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.570+0000 7f7856618700 1 --2- 172.21.15.84:0/3402214926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7850101f20 0x7f78501097c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:57:41.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.570+0000 7f7856618700 1 --2- 172.21.15.84:0/3402214926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78501028b0 0x7f7850102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:57:41.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.570+0000 7f7856618700 1 --2- 172.21.15.84:0/3402214926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7850101560 0x7f78501019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:57:41.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.570+0000 7f7856618700 1 -- 172.21.15.84:0/3402214926 >> 172.21.15.84:0/3402214926 conn(0x7f78500fbff0 msgr2=0x7f78500fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:57:41.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.570+0000 7f7856618700 1 -- 172.21.15.84:0/3402214926 shutdown_connections 2024-05-22T22:57:41.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.570+0000 7f7856618700 1 -- 172.21.15.84:0/3402214926 wait complete. 2024-05-22T22:57:41.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.571+0000 7f7856618700 1 Processor -- start 2024-05-22T22:57:41.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.571+0000 7f7856618700 1 -- start start 2024-05-22T22:57:41.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.571+0000 7f7856618700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7850101560 0x7f7850197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:57:41.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.571+0000 7f7856618700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7850101f20 0x7f7850197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:57:41.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.571+0000 7f7856618700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78501028b0 0x7f7850100450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:57:41.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.571+0000 7f7856618700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f785010cef0 con 0x7f7850101560 2024-05-22T22:57:41.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.571+0000 7f7856618700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f785010cd70 con 0x7f78501028b0 2024-05-22T22:57:41.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.571+0000 7f7856618700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f785010d070 con 0x7f7850101f20 2024-05-22T22:57:41.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.571+0000 7f7847fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7850101f20 0x7f7850197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:57:41.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.571+0000 7f7854bb5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78501028b0 0x7f7850100450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:57:41.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.571+0000 7f7847fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7850101f20 0x7f7850197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:50776/0 (socket says 172.21.15.84:50776) 2024-05-22T22:57:41.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.571+0000 7f7847fff700 1 -- 172.21.15.84:0/2678547708 learned_addr learned my addr 172.21.15.84:0/2678547708 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:57:41.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.572+0000 7f7847fff700 1 -- 172.21.15.84:0/2678547708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78501028b0 msgr2=0x7f7850100450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:57:41.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.572+0000 7f7847fff700 1 --2- 172.21.15.84:0/2678547708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78501028b0 0x7f7850100450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:57:41.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.572+0000 7f7847fff700 1 -- 172.21.15.84:0/2678547708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7850101560 msgr2=0x7f7850197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:57:41.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.572+0000 7f7847fff700 1 --2- 172.21.15.84:0/2678547708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7850101560 0x7f7850197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:57:41.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.572+0000 7f7847fff700 1 -- 172.21.15.84:0/2678547708 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7850100b20 con 0x7f7850101f20 2024-05-22T22:57:41.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.572+0000 7f7847fff700 1 --2- 172.21.15.84:0/2678547708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7850101f20 0x7f7850197980 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f783800d980 tx=0x7f783800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:57:41.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.578+0000 7f784dffb700 1 -- 172.21.15.84:0/2678547708 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7838016070 con 0x7f7850101f20 2024-05-22T22:57:41.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.579+0000 7f784dffb700 1 -- 172.21.15.84:0/2678547708 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f783800be30 con 0x7f7850101f20 2024-05-22T22:57:41.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.579+0000 7f7856618700 1 -- 172.21.15.84:0/2678547708 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7850100e10 con 0x7f7850101f20 2024-05-22T22:57:41.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.579+0000 7f784dffb700 1 -- 172.21.15.84:0/2678547708 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783800f3c0 con 0x7f7850101f20 2024-05-22T22:57:41.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.579+0000 7f7856618700 1 -- 172.21.15.84:0/2678547708 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f785019c9f0 con 0x7f7850101f20 2024-05-22T22:57:41.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.580+0000 7f7856618700 1 -- 172.21.15.84:0/2678547708 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7850069050 con 0x7f7850101f20 2024-05-22T22:57:41.579 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.581+0000 7f784dffb700 1 -- 172.21.15.84:0/2678547708 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7838019070 con 0x7f7850101f20 2024-05-22T22:57:41.580 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.581+0000 7f784dffb700 1 --2- 172.21.15.84:0/2678547708 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f783c0a2ca0 0x7f783c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:57:41.580 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.582+0000 7f784ffff700 1 --2- 172.21.15.84:0/2678547708 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f783c0a2ca0 0x7f783c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:57:41.580 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.582+0000 7f784dffb700 1 -- 172.21.15.84:0/2678547708 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7f78380ceb70 con 0x7f7850101f20 2024-05-22T22:57:41.580 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.582+0000 7f784ffff700 1 --2- 172.21.15.84:0/2678547708 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f783c0a2ca0 0x7f783c0a5160 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7840005950 tx=0x7f78400058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:57:41.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.585+0000 7f784dffb700 1 -- 172.21.15.84:0/2678547708 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7838020030 con 0x7f7850101f20 2024-05-22T22:57:41.621 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:41 smithi084 ceph-mon[32186]: Detected new or changed devices on smithi204 2024-05-22T22:57:41.621 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:41 smithi084 ceph-mon[32186]: Adjusting osd_memory_target on smithi204 to 5696M 2024-05-22T22:57:41.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:57:41.790+0000 7f7856618700 1 -- 172.21.15.84:0/2678547708 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f785002e010 con 0x7f783c0a2ca0 2024-05-22T22:57:41.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:41 smithi204 ceph-mon[36211]: Detected new or changed devices on smithi204 2024-05-22T22:57:41.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:41 smithi204 ceph-mon[36211]: Adjusting osd_memory_target on smithi204 to 5696M 2024-05-22T22:57:42.046 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:41 smithi096 ceph-mon[36392]: Detected new or changed devices on smithi204 2024-05-22T22:57:42.046 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:41 smithi096 ceph-mon[36392]: Adjusting osd_memory_target on smithi204 to 5696M 2024-05-22T22:57:42.657 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:42 smithi084 ceph-mon[32186]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 297 MiB used, 804 GiB / 805 GiB avail 2024-05-22T22:57:42.657 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:42 smithi084 ceph-mon[32186]: osdmap e63: 9 total, 9 up, 9 in 2024-05-22T22:57:42.657 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:42 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:57:42.658 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:42 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:57:42.658 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:42 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:42.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:42 smithi204 ceph-mon[36211]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 297 MiB used, 804 GiB / 805 GiB avail 2024-05-22T22:57:42.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:42 smithi204 ceph-mon[36211]: osdmap e63: 9 total, 9 up, 9 in 2024-05-22T22:57:42.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:42 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:57:42.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:42 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:57:42.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:42 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:43.046 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:42 smithi096 ceph-mon[36392]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 297 MiB used, 804 GiB / 805 GiB avail 2024-05-22T22:57:43.046 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:42 smithi096 ceph-mon[36392]: osdmap e63: 9 total, 9 up, 9 in 2024-05-22T22:57:43.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:42 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:57:43.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:42 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:57:43.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:42 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:43.766 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:43 smithi084 ceph-mon[32186]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:57:43.767 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:43 smithi084 ceph-mon[32186]: osdmap e64: 9 total, 9 up, 9 in 2024-05-22T22:57:43.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:43 smithi204 ceph-mon[36211]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:57:43.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:43 smithi204 ceph-mon[36211]: osdmap e64: 9 total, 9 up, 9 in 2024-05-22T22:57:44.046 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:43 smithi096 ceph-mon[36392]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:57:44.046 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:43 smithi096 ceph-mon[36392]: osdmap e64: 9 total, 9 up, 9 in 2024-05-22T22:57:44.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:44 smithi084 ceph-mon[32186]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 804 GiB / 805 GiB avail 2024-05-22T22:57:44.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:44 smithi204 ceph-mon[36211]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 804 GiB / 805 GiB avail 2024-05-22T22:57:45.046 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:44 smithi096 ceph-mon[36392]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 804 GiB / 805 GiB avail 2024-05-22T22:57:45.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:45 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2588939952' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3984fb2-7c00-4275-a10e-bca7c859f714"}]: dispatch 2024-05-22T22:57:45.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:45 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2588939952' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3984fb2-7c00-4275-a10e-bca7c859f714"}]': finished 2024-05-22T22:57:45.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:45 smithi084 ceph-mon[32186]: osdmap e65: 10 total, 9 up, 10 in 2024-05-22T22:57:45.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-22T22:57:45.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:45 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1107900826' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:57:45.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:45 smithi084 ceph-mon[32186]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-05-22T22:57:45.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:45 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2588939952' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3984fb2-7c00-4275-a10e-bca7c859f714"}]: dispatch 2024-05-22T22:57:45.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:45 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2588939952' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3984fb2-7c00-4275-a10e-bca7c859f714"}]': finished 2024-05-22T22:57:45.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:45 smithi204 ceph-mon[36211]: osdmap e65: 10 total, 9 up, 10 in 2024-05-22T22:57:45.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-22T22:57:45.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:45 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1107900826' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:57:45.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:45 smithi204 ceph-mon[36211]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-05-22T22:57:46.046 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:45 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2588939952' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3984fb2-7c00-4275-a10e-bca7c859f714"}]: dispatch 2024-05-22T22:57:46.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:45 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2588939952' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3984fb2-7c00-4275-a10e-bca7c859f714"}]': finished 2024-05-22T22:57:46.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:45 smithi096 ceph-mon[36392]: osdmap e65: 10 total, 9 up, 10 in 2024-05-22T22:57:46.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-22T22:57:46.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:45 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1107900826' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:57:46.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:45 smithi096 ceph-mon[36392]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-05-22T22:57:48.599 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:48 smithi084 ceph-mon[32186]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-22T22:57:48.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:48 smithi096 ceph-mon[36392]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-22T22:57:48.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:48 smithi204 ceph-mon[36211]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-22T22:57:50.788 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:50 smithi084 ceph-mon[32186]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-22T22:57:50.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:50 smithi096 ceph-mon[36392]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-22T22:57:50.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:50 smithi204 ceph-mon[36211]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-22T22:57:52.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:52 smithi096 ceph-mon[36392]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-22T22:57:52.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:52 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-22T22:57:52.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:52 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:52.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:52 smithi084 ceph-mon[32186]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-22T22:57:52.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:52 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-22T22:57:52.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:52 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:52.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:52 smithi204 ceph-mon[36211]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-22T22:57:52.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:52 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-22T22:57:52.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:52 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:57:53.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:53 smithi096 ceph-mon[36392]: Deploying daemon osd.9 on smithi084 2024-05-22T22:57:53.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:53 smithi084 ceph-mon[32186]: Deploying daemon osd.9 on smithi084 2024-05-22T22:57:53.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:53 smithi204 ceph-mon[36211]: Deploying daemon osd.9 on smithi084 2024-05-22T22:57:54.736 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:54 smithi084 ceph-mon[32186]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-22T22:57:54.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:54 smithi096 ceph-mon[36392]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-22T22:57:54.811 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:54 smithi204 ceph-mon[36211]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-22T22:57:56.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:56 smithi096 ceph-mon[36392]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-22T22:57:56.828 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:56 smithi084 ceph-mon[32186]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-22T22:57:56.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:56 smithi204 ceph-mon[36211]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-22T22:57:58.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:58 smithi096 ceph-mon[36392]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-22T22:57:58.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:58 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:58.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:58 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:58.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:57:58 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:57:58.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:58 smithi084 ceph-mon[32186]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-22T22:57:58.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:58 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:58.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:58 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:58.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:57:58 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:57:58.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:58 smithi204 ceph-mon[36211]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-22T22:57:58.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:58 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:58.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:58 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:57:58.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:57:58 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:58:00.622 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:00 smithi084 ceph-mon[32186]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-22T22:58:00.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:00 smithi096 ceph-mon[36392]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-22T22:58:00.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:00 smithi204 ceph-mon[36211]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-22T22:58:02.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:02.453+0000 7f784dffb700 1 -- 172.21.15.84:0/2678547708 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f785002e010 con 0x7f783c0a2ca0 2024-05-22T22:58:02.452 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 9 on host 'smithi084' 2024-05-22T22:58:02.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:02.456+0000 7f7856618700 1 -- 172.21.15.84:0/2678547708 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f783c0a2ca0 msgr2=0x7f783c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:58:02.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:02.456+0000 7f7856618700 1 --2- 172.21.15.84:0/2678547708 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f783c0a2ca0 0x7f783c0a5160 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7840005950 tx=0x7f78400058e0 comp rx=0 tx=0).stop 2024-05-22T22:58:02.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:02.456+0000 7f7856618700 1 -- 172.21.15.84:0/2678547708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7850101f20 msgr2=0x7f7850197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:58:02.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:02.456+0000 7f7856618700 1 --2- 172.21.15.84:0/2678547708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7850101f20 0x7f7850197980 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f783800d980 tx=0x7f783800dc90 comp rx=0 tx=0).stop 2024-05-22T22:58:02.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:02.456+0000 7f7856618700 1 -- 172.21.15.84:0/2678547708 shutdown_connections 2024-05-22T22:58:02.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:02.456+0000 7f7856618700 1 --2- 172.21.15.84:0/2678547708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78501028b0 0x7f7850100450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:02.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:02.456+0000 7f7856618700 1 --2- 172.21.15.84:0/2678547708 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f783c0a2ca0 0x7f783c0a5160 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:02.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:02.456+0000 7f7856618700 1 --2- 172.21.15.84:0/2678547708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7850101560 0x7f7850197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:02.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:02.456+0000 7f7856618700 1 --2- 172.21.15.84:0/2678547708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7850101f20 0x7f7850197980 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:02.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:02.456+0000 7f7856618700 1 -- 172.21.15.84:0/2678547708 >> 172.21.15.84:0/2678547708 conn(0x7f78500fbff0 msgr2=0x7f7850105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:58:02.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:02.456+0000 7f7856618700 1 -- 172.21.15.84:0/2678547708 shutdown_connections 2024-05-22T22:58:02.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:02.456+0000 7f7856618700 1 -- 172.21.15.84:0/2678547708 wait complete. 2024-05-22T22:58:02.776 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:02 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:02.776 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:02 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:02.776 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:02 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:58:02.776 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:02 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:58:02.776 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:02 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:02.776 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:02 smithi084 ceph-mon[32186]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-22T22:58:02.776 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:02 smithi084 ceph-mon[32186]: from='osd.9 [v2:172.21.15.84:6826/2786189012,v1:172.21.15.84:6827/2786189012]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-22T22:58:02.776 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:02 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:02.776 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:02 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:02.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:02 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:02.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:02 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:02.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:02 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:58:02.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:02 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:58:02.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:02 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:02.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:02 smithi096 ceph-mon[36392]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-22T22:58:02.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:02 smithi096 ceph-mon[36392]: from='osd.9 [v2:172.21.15.84:6826/2786189012,v1:172.21.15.84:6827/2786189012]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-22T22:58:02.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:02 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:02.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:02 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:02.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:02 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:02.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:02 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:02.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:02 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:58:02.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:02 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:58:02.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:02 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:02.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:02 smithi204 ceph-mon[36211]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-22T22:58:02.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:02 smithi204 ceph-mon[36211]: from='osd.9 [v2:172.21.15.84:6826/2786189012,v1:172.21.15.84:6827/2786189012]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-22T22:58:02.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:02 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:02.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:02 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:03.118 DEBUG:teuthology.orchestra.run.smithi084:osd.9> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.9.service 2024-05-22T22:58:03.120 INFO:tasks.cephadm:Deploying osd.10 on smithi096 with /dev/vg_nvme/lv_1... 2024-05-22T22:58:03.121 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-22T22:58:03.365 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.b/config 2024-05-22T22:58:03.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:03 smithi096 ceph-mon[36392]: from='osd.9 [v2:172.21.15.84:6826/2786189012,v1:172.21.15.84:6827/2786189012]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-22T22:58:03.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:03 smithi096 ceph-mon[36392]: osdmap e66: 10 total, 9 up, 10 in 2024-05-22T22:58:03.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:03 smithi096 ceph-mon[36392]: from='osd.9 [v2:172.21.15.84:6826/2786189012,v1:172.21.15.84:6827/2786189012]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-22T22:58:03.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:03 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-22T22:58:03.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:03 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:58:03.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:03 smithi084 ceph-mon[32186]: from='osd.9 [v2:172.21.15.84:6826/2786189012,v1:172.21.15.84:6827/2786189012]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-22T22:58:03.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:03 smithi084 ceph-mon[32186]: osdmap e66: 10 total, 9 up, 10 in 2024-05-22T22:58:03.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:03 smithi084 ceph-mon[32186]: from='osd.9 [v2:172.21.15.84:6826/2786189012,v1:172.21.15.84:6827/2786189012]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-22T22:58:03.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:03 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-22T22:58:03.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:03 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:58:03.873 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 22:58:03 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T22:58:03.465+0000 7f8f89000700 -1 osd.9 0 waiting for initial osdmap 2024-05-22T22:58:03.873 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 22:58:03 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T22:58:03.478+0000 7f8f84df3700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-22T22:58:03.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:03 smithi204 ceph-mon[36211]: from='osd.9 [v2:172.21.15.84:6826/2786189012,v1:172.21.15.84:6827/2786189012]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-22T22:58:03.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:03 smithi204 ceph-mon[36211]: osdmap e66: 10 total, 9 up, 10 in 2024-05-22T22:58:03.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:03 smithi204 ceph-mon[36211]: from='osd.9 [v2:172.21.15.84:6826/2786189012,v1:172.21.15.84:6827/2786189012]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-22T22:58:03.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:03 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-22T22:58:03.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:03 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:58:04.602 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:04 smithi084 ceph-mon[32186]: from='osd.9 [v2:172.21.15.84:6826/2786189012,v1:172.21.15.84:6827/2786189012]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-05-22T22:58:04.602 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:04 smithi084 ceph-mon[32186]: osdmap e67: 10 total, 9 up, 10 in 2024-05-22T22:58:04.602 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:04 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-22T22:58:04.602 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:04 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-22T22:58:04.602 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:04 smithi084 ceph-mon[32186]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-22T22:58:04.760 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:04 smithi096 ceph-mon[36392]: from='osd.9 [v2:172.21.15.84:6826/2786189012,v1:172.21.15.84:6827/2786189012]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-05-22T22:58:04.760 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:04 smithi096 ceph-mon[36392]: osdmap e67: 10 total, 9 up, 10 in 2024-05-22T22:58:04.760 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:04 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-22T22:58:04.760 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:04 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-22T22:58:04.760 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:04 smithi096 ceph-mon[36392]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-22T22:58:04.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:04 smithi204 ceph-mon[36211]: from='osd.9 [v2:172.21.15.84:6826/2786189012,v1:172.21.15.84:6827/2786189012]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-05-22T22:58:04.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:04 smithi204 ceph-mon[36211]: osdmap e67: 10 total, 9 up, 10 in 2024-05-22T22:58:04.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:04 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-22T22:58:04.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:04 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-22T22:58:04.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:04 smithi204 ceph-mon[36211]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-22T22:58:05.499 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-22T22:58:05.499 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-22T22:58:05.499 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-05-22T22:58:05.499 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-05-22T22:58:05.499 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0138713 s, 756 MB/s 2024-05-22T22:58:05.499 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-05-22T22:58:05.768 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:05 smithi084 ceph-mon[32186]: purged_snaps scrub starts 2024-05-22T22:58:05.768 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:05 smithi084 ceph-mon[32186]: purged_snaps scrub ok 2024-05-22T22:58:05.768 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:05 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-22T22:58:05.768 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:05 smithi084 ceph-mon[32186]: osd.9 [v2:172.21.15.84:6826/2786189012,v1:172.21.15.84:6827/2786189012] boot 2024-05-22T22:58:05.768 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:05 smithi084 ceph-mon[32186]: osdmap e68: 10 total, 10 up, 10 in 2024-05-22T22:58:05.768 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:05 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-22T22:58:05.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:05 smithi096 ceph-mon[36392]: purged_snaps scrub starts 2024-05-22T22:58:05.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:05 smithi096 ceph-mon[36392]: purged_snaps scrub ok 2024-05-22T22:58:05.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:05 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-22T22:58:05.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:05 smithi096 ceph-mon[36392]: osd.9 [v2:172.21.15.84:6826/2786189012,v1:172.21.15.84:6827/2786189012] boot 2024-05-22T22:58:05.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:05 smithi096 ceph-mon[36392]: osdmap e68: 10 total, 10 up, 10 in 2024-05-22T22:58:05.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:05 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-22T22:58:05.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:05 smithi204 ceph-mon[36211]: purged_snaps scrub starts 2024-05-22T22:58:05.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:05 smithi204 ceph-mon[36211]: purged_snaps scrub ok 2024-05-22T22:58:05.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:05 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-22T22:58:05.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:05 smithi204 ceph-mon[36211]: osd.9 [v2:172.21.15.84:6826/2786189012,v1:172.21.15.84:6827/2786189012] boot 2024-05-22T22:58:05.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:05 smithi204 ceph-mon[36211]: osdmap e68: 10 total, 10 up, 10 in 2024-05-22T22:58:05.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:05 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-22T22:58:06.213 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_1 2024-05-22T22:58:06.451 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.b/config 2024-05-22T22:58:06.728 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:06 smithi096 ceph-mon[36392]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:06.728 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:06 smithi096 ceph-mon[36392]: osdmap e69: 10 total, 10 up, 10 in 2024-05-22T22:58:06.728 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:06.728 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:06.729 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:58:06.729 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:58:06.729 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:58:06.729 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:58:06.729 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:06.729 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:58:06.729 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:58:06.729 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:06.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:06 smithi084 ceph-mon[32186]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:06.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:06 smithi084 ceph-mon[32186]: osdmap e69: 10 total, 10 up, 10 in 2024-05-22T22:58:06.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:06.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:06.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:58:06.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:58:06.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:58:06.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:58:06.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:06.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:58:06.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:58:06.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:06.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:06 smithi204 ceph-mon[36211]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:06.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:06 smithi204 ceph-mon[36211]: osdmap e69: 10 total, 10 up, 10 in 2024-05-22T22:58:06.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:06.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:06.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:58:06.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:58:06.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:58:06.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:58:06.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:06.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:58:06.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:58:06.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:07.677 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:07 smithi096 ceph-mon[36392]: Detected new or changed devices on smithi084 2024-05-22T22:58:07.677 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:07 smithi096 ceph-mon[36392]: Adjusting osd_memory_target on smithi084 to 4272M 2024-05-22T22:58:07.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:07 smithi084 ceph-mon[32186]: Detected new or changed devices on smithi084 2024-05-22T22:58:07.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:07 smithi084 ceph-mon[32186]: Adjusting osd_memory_target on smithi084 to 4272M 2024-05-22T22:58:07.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:07 smithi204 ceph-mon[36211]: Detected new or changed devices on smithi084 2024-05-22T22:58:07.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:07 smithi204 ceph-mon[36211]: Adjusting osd_memory_target on smithi084 to 4272M 2024-05-22T22:58:08.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.162+0000 7ff5c7a1c700 1 -- 172.21.15.96:0/2280969310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5c0104010 msgr2=0x7ff5c010a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:58:08.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.162+0000 7ff5c7a1c700 1 --2- 172.21.15.96:0/2280969310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5c0104010 0x7ff5c010a8c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff5b800b0a0 tx=0x7ff5b800b3b0 comp rx=0 tx=0).stop 2024-05-22T22:58:08.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.162+0000 7ff5c7a1c700 1 -- 172.21.15.96:0/2280969310 shutdown_connections 2024-05-22T22:58:08.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.162+0000 7ff5c7a1c700 1 --2- 172.21.15.96:0/2280969310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5c0104010 0x7ff5c010a8c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:08.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.162+0000 7ff5c7a1c700 1 --2- 172.21.15.96:0/2280969310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5c0100690 0x7ff5c0100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:08.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.162+0000 7ff5c7a1c700 1 --2- 172.21.15.96:0/2280969310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5c0103670 0x7ff5c0103ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:08.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.162+0000 7ff5c7a1c700 1 -- 172.21.15.96:0/2280969310 >> 172.21.15.96:0/2280969310 conn(0x7ff5c00fbfd0 msgr2=0x7ff5c00fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:58:08.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.162+0000 7ff5c7a1c700 1 -- 172.21.15.96:0/2280969310 shutdown_connections 2024-05-22T22:58:08.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.162+0000 7ff5c7a1c700 1 -- 172.21.15.96:0/2280969310 wait complete. 2024-05-22T22:58:08.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.162+0000 7ff5c7a1c700 1 Processor -- start 2024-05-22T22:58:08.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.162+0000 7ff5c7a1c700 1 -- start start 2024-05-22T22:58:08.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.162+0000 7ff5c7a1c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5c0100690 0x7ff5c0198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:58:08.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.162+0000 7ff5c7a1c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5c0103670 0x7ff5c0198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:58:08.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.163+0000 7ff5c7a1c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5c0104010 0x7ff5c019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:58:08.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.163+0000 7ff5c7a1c700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c010e0a0 con 0x7ff5c0100690 2024-05-22T22:58:08.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.163+0000 7ff5c7a1c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c010df20 con 0x7ff5c0103670 2024-05-22T22:58:08.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.163+0000 7ff5c7a1c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c010e220 con 0x7ff5c0104010 2024-05-22T22:58:08.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.163+0000 7ff5c4fb7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5c0103670 0x7ff5c0198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:58:08.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.163+0000 7ff5c4fb7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5c0103670 0x7ff5c0198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34556/0 (socket says 172.21.15.96:34556) 2024-05-22T22:58:08.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.163+0000 7ff5c4fb7700 1 -- 172.21.15.96:0/4055140587 learned_addr learned my addr 172.21.15.96:0/4055140587 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-05-22T22:58:08.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.163+0000 7ff5c5fb9700 1 --2- 172.21.15.96:0/4055140587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5c0104010 0x7ff5c019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:58:08.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.163+0000 7ff5c57b8700 1 --2- 172.21.15.96:0/4055140587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5c0100690 0x7ff5c0198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:58:08.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.163+0000 7ff5c4fb7700 1 -- 172.21.15.96:0/4055140587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5c0104010 msgr2=0x7ff5c019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:58:08.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.163+0000 7ff5c4fb7700 1 --2- 172.21.15.96:0/4055140587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5c0104010 0x7ff5c019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:08.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.163+0000 7ff5c4fb7700 1 -- 172.21.15.96:0/4055140587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5c0100690 msgr2=0x7ff5c0198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:58:08.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.163+0000 7ff5c4fb7700 1 --2- 172.21.15.96:0/4055140587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5c0100690 0x7ff5c0198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:08.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.163+0000 7ff5c4fb7700 1 -- 172.21.15.96:0/4055140587 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c01a02b0 con 0x7ff5c0103670 2024-05-22T22:58:08.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.163+0000 7ff5c5fb9700 1 --2- 172.21.15.96:0/4055140587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5c0104010 0x7ff5c019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T22:58:08.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.163+0000 7ff5c4fb7700 1 --2- 172.21.15.96:0/4055140587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5c0103670 0x7ff5c0198b90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000b530 tx=0x7ff5b000b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:58:08.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.163+0000 7ff5b67fc700 1 -- 172.21.15.96:0/4055140587 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0015020 con 0x7ff5c0103670 2024-05-22T22:58:08.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.163+0000 7ff5b67fc700 1 -- 172.21.15.96:0/4055140587 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5b0011d10 con 0x7ff5c0103670 2024-05-22T22:58:08.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.163+0000 7ff5c7a1c700 1 -- 172.21.15.96:0/4055140587 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c01a05a0 con 0x7ff5c0103670 2024-05-22T22:58:08.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.163+0000 7ff5b67fc700 1 -- 172.21.15.96:0/4055140587 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b000f2e0 con 0x7ff5c0103670 2024-05-22T22:58:08.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.163+0000 7ff5c7a1c700 1 -- 172.21.15.96:0/4055140587 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c01a0b10 con 0x7ff5c0103670 2024-05-22T22:58:08.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.164+0000 7ff5c7a1c700 1 -- 172.21.15.96:0/4055140587 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c0069050 con 0x7ff5c0103670 2024-05-22T22:58:08.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.165+0000 7ff5b67fc700 1 -- 172.21.15.96:0/4055140587 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff5b0018020 con 0x7ff5c0103670 2024-05-22T22:58:08.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.166+0000 7ff5b67fc700 1 --2- 172.21.15.96:0/4055140587 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7ff5ac0a2c20 0x7ff5ac0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:58:08.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.166+0000 7ff5b67fc700 1 -- 172.21.15.96:0/4055140587 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7ff5b00ce650 con 0x7ff5c0103670 2024-05-22T22:58:08.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.166+0000 7ff5c57b8700 1 --2- 172.21.15.96:0/4055140587 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7ff5ac0a2c20 0x7ff5ac0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:58:08.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.167+0000 7ff5c57b8700 1 --2- 172.21.15.96:0/4055140587 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7ff5ac0a2c20 0x7ff5ac0a50e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc006fd0 tx=0x7ff5bc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:58:08.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.170+0000 7ff5b67fc700 1 -- 172.21.15.96:0/4055140587 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff5b001f020 con 0x7ff5c0103670 2024-05-22T22:58:08.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:08.373+0000 7ff5c7a1c700 1 -- 172.21.15.96:0/4055140587 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff5c00627a0 con 0x7ff5ac0a2c20 2024-05-22T22:58:08.796 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:08 smithi096 ceph-mon[36392]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:08.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:08 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:58:08.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:08 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:58:08.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:08 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:58:08.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:08 smithi084 ceph-mon[32186]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:08.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:08 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:58:08.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:08 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:58:08.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:08 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:58:08.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:08 smithi204 ceph-mon[36211]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:08.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:08 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:58:08.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:08 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:58:08.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:08 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:58:09.547 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:09 smithi096 ceph-mon[36392]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:58:09.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:09 smithi084 ceph-mon[32186]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:58:09.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:09 smithi204 ceph-mon[36211]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:58:10.547 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:10 smithi096 ceph-mon[36392]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:10.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:10 smithi084 ceph-mon[32186]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:10.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:10 smithi204 ceph-mon[36211]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:11.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:11 smithi096 ceph-mon[36392]: from='client.? 172.21.15.96:0/2158024252' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4938160-a808-4ded-91c2-d245c99fff8c"}]: dispatch 2024-05-22T22:58:11.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:11 smithi096 ceph-mon[36392]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4938160-a808-4ded-91c2-d245c99fff8c"}]: dispatch 2024-05-22T22:58:11.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:11 smithi096 ceph-mon[36392]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b4938160-a808-4ded-91c2-d245c99fff8c"}]': finished 2024-05-22T22:58:11.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:11 smithi096 ceph-mon[36392]: osdmap e70: 11 total, 10 up, 11 in 2024-05-22T22:58:11.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:11 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-22T22:58:11.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:11 smithi084 ceph-mon[32186]: from='client.? 172.21.15.96:0/2158024252' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4938160-a808-4ded-91c2-d245c99fff8c"}]: dispatch 2024-05-22T22:58:11.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:11 smithi084 ceph-mon[32186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4938160-a808-4ded-91c2-d245c99fff8c"}]: dispatch 2024-05-22T22:58:11.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:11 smithi084 ceph-mon[32186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b4938160-a808-4ded-91c2-d245c99fff8c"}]': finished 2024-05-22T22:58:11.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:11 smithi084 ceph-mon[32186]: osdmap e70: 11 total, 10 up, 11 in 2024-05-22T22:58:11.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:11 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-22T22:58:11.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:11 smithi204 ceph-mon[36211]: from='client.? 172.21.15.96:0/2158024252' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4938160-a808-4ded-91c2-d245c99fff8c"}]: dispatch 2024-05-22T22:58:11.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:11 smithi204 ceph-mon[36211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4938160-a808-4ded-91c2-d245c99fff8c"}]: dispatch 2024-05-22T22:58:11.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:11 smithi204 ceph-mon[36211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b4938160-a808-4ded-91c2-d245c99fff8c"}]': finished 2024-05-22T22:58:11.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:11 smithi204 ceph-mon[36211]: osdmap e70: 11 total, 10 up, 11 in 2024-05-22T22:58:11.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:11 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-22T22:58:12.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:12 smithi096 ceph-mon[36392]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:12.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:12 smithi096 ceph-mon[36392]: from='client.? 172.21.15.96:0/4149424135' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:58:12.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:12 smithi084 ceph-mon[32186]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:12.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:12 smithi084 ceph-mon[32186]: from='client.? 172.21.15.96:0/4149424135' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:58:12.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:12 smithi204 ceph-mon[36211]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:12.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:12 smithi204 ceph-mon[36211]: from='client.? 172.21.15.96:0/4149424135' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:58:14.767 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:14 smithi096 ceph-mon[36392]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:14.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:14 smithi084 ceph-mon[32186]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:14.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:14 smithi204 ceph-mon[36211]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:16.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:16 smithi096 ceph-mon[36392]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:16.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:16 smithi084 ceph-mon[32186]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:16.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:16 smithi204 ceph-mon[36211]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:18.647 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:18 smithi096 ceph-mon[36392]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:18.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:18 smithi084 ceph-mon[32186]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:18.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:18 smithi204 ceph-mon[36211]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:19.547 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:19 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-22T22:58:19.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:19 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:58:19.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:19 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-22T22:58:19.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:19 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:58:19.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:19 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-22T22:58:19.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:19 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:58:20.547 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:20 smithi096 ceph-mon[36392]: Deploying daemon osd.10 on smithi096 2024-05-22T22:58:20.547 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:20 smithi096 ceph-mon[36392]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:20.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:20 smithi084 ceph-mon[32186]: Deploying daemon osd.10 on smithi096 2024-05-22T22:58:20.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:20 smithi084 ceph-mon[32186]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:20.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:20 smithi204 ceph-mon[36211]: Deploying daemon osd.10 on smithi096 2024-05-22T22:58:20.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:20 smithi204 ceph-mon[36211]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:22.698 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:22 smithi096 ceph-mon[36392]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:22.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:22 smithi084 ceph-mon[32186]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:22.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:22 smithi204 ceph-mon[36211]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:24.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:24 smithi096 ceph-mon[36392]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:24.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:24 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:24.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:24 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:24.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:24 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:58:24.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:24 smithi084 ceph-mon[32186]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:24.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:24 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:24.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:24 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:24.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:24 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:58:24.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:24 smithi204 ceph-mon[36211]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:24.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:24 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:24.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:24 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:24.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:24 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:58:26.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:26 smithi096 ceph-mon[36392]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:26.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:26 smithi084 ceph-mon[32186]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:26.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:26 smithi204 ceph-mon[36211]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:28.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:28.391+0000 7ff5b67fc700 1 -- 172.21.15.96:0/4055140587 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff5c00627a0 con 0x7ff5ac0a2c20 2024-05-22T22:58:28.391 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 10 on host 'smithi096' 2024-05-22T22:58:28.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:28.393+0000 7ff5c7a1c700 1 -- 172.21.15.96:0/4055140587 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7ff5ac0a2c20 msgr2=0x7ff5ac0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:58:28.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:28.393+0000 7ff5c7a1c700 1 --2- 172.21.15.96:0/4055140587 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7ff5ac0a2c20 0x7ff5ac0a50e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc006fd0 tx=0x7ff5bc008040 comp rx=0 tx=0).stop 2024-05-22T22:58:28.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:28.394+0000 7ff5c7a1c700 1 -- 172.21.15.96:0/4055140587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5c0103670 msgr2=0x7ff5c0198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:58:28.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:28.394+0000 7ff5c7a1c700 1 --2- 172.21.15.96:0/4055140587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5c0103670 0x7ff5c0198b90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000b530 tx=0x7ff5b000b8f0 comp rx=0 tx=0).stop 2024-05-22T22:58:28.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:28.394+0000 7ff5c7a1c700 1 -- 172.21.15.96:0/4055140587 shutdown_connections 2024-05-22T22:58:28.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:28.394+0000 7ff5c7a1c700 1 --2- 172.21.15.96:0/4055140587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5c0104010 0x7ff5c019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:28.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:28.394+0000 7ff5c7a1c700 1 --2- 172.21.15.96:0/4055140587 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7ff5ac0a2c20 0x7ff5ac0a50e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:28.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:28.394+0000 7ff5c7a1c700 1 --2- 172.21.15.96:0/4055140587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5c0100690 0x7ff5c0198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:28.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:28.394+0000 7ff5c7a1c700 1 --2- 172.21.15.96:0/4055140587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5c0103670 0x7ff5c0198b90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:28.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:28.394+0000 7ff5c7a1c700 1 -- 172.21.15.96:0/4055140587 >> 172.21.15.96:0/4055140587 conn(0x7ff5c00fbfd0 msgr2=0x7ff5c0106970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:58:28.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:28.394+0000 7ff5c7a1c700 1 -- 172.21.15.96:0/4055140587 shutdown_connections 2024-05-22T22:58:28.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-22T22:58:28.394+0000 7ff5c7a1c700 1 -- 172.21.15.96:0/4055140587 wait complete. 2024-05-22T22:58:28.771 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:28 smithi096 ceph-mon[36392]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:28.771 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:28 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:28.771 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:28 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:28.771 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:28 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:58:28.771 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:28 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:58:28.771 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:28 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:28.771 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:28 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:28.771 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:28 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:28.771 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:28 smithi096 ceph-mon[36392]: from='osd.10 [v2:172.21.15.96:6824/1985398526,v1:172.21.15.96:6825/1985398526]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-22T22:58:28.771 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:28 smithi096 ceph-mon[36392]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-22T22:58:28.772 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:28 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:58:28.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:28 smithi084 ceph-mon[32186]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:28.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:28 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:28.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:28 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:28.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:28 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:58:28.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:28 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:58:28.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:28 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:28.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:28 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:28.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:28 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:28.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:28 smithi084 ceph-mon[32186]: from='osd.10 [v2:172.21.15.96:6824/1985398526,v1:172.21.15.96:6825/1985398526]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-22T22:58:28.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:28 smithi084 ceph-mon[32186]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-22T22:58:28.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:28 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:58:28.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:28 smithi204 ceph-mon[36211]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:28.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:28 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:28.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:28 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:28.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:28 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:58:28.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:28 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:58:28.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:28 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:28.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:28 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:28.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:28 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:28.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:28 smithi204 ceph-mon[36211]: from='osd.10 [v2:172.21.15.96:6824/1985398526,v1:172.21.15.96:6825/1985398526]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-22T22:58:28.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:28 smithi204 ceph-mon[36211]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-22T22:58:28.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:28 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:58:29.131 DEBUG:teuthology.orchestra.run.smithi096:osd.10> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.10.service 2024-05-22T22:58:29.133 INFO:tasks.cephadm:Deploying osd.11 on smithi204 with /dev/vg_nvme/lv_1... 2024-05-22T22:58:29.133 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-22T22:58:29.380 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.c/config 2024-05-22T22:58:30.592 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:30 smithi096 ceph-mon[36392]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-22T22:58:30.592 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:30 smithi096 ceph-mon[36392]: osdmap e71: 11 total, 10 up, 11 in 2024-05-22T22:58:30.592 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:30 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-22T22:58:30.592 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:30 smithi096 ceph-mon[36392]: from='osd.10 [v2:172.21.15.96:6824/1985398526,v1:172.21.15.96:6825/1985398526]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-22T22:58:30.592 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:30 smithi096 ceph-mon[36392]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-22T22:58:30.593 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:30 smithi096 ceph-mon[36392]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:30.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:30 smithi204 ceph-mon[36211]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-22T22:58:30.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:30 smithi204 ceph-mon[36211]: osdmap e71: 11 total, 10 up, 11 in 2024-05-22T22:58:30.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:30 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-22T22:58:30.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:30 smithi204 ceph-mon[36211]: from='osd.10 [v2:172.21.15.96:6824/1985398526,v1:172.21.15.96:6825/1985398526]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-22T22:58:30.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:30 smithi204 ceph-mon[36211]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-22T22:58:30.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:30 smithi204 ceph-mon[36211]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:30.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:30 smithi084 ceph-mon[32186]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-22T22:58:30.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:30 smithi084 ceph-mon[32186]: osdmap e71: 11 total, 10 up, 11 in 2024-05-22T22:58:30.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:30 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-22T22:58:30.876 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:30 smithi084 ceph-mon[32186]: from='osd.10 [v2:172.21.15.96:6824/1985398526,v1:172.21.15.96:6825/1985398526]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-22T22:58:30.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:30 smithi084 ceph-mon[32186]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-22T22:58:30.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:30 smithi084 ceph-mon[32186]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:31.047 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 22:58:30 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T22:58:30.786+0000 7f6957606700 -1 osd.10 0 waiting for initial osdmap 2024-05-22T22:58:31.047 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 22:58:30 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T22:58:30.800+0000 7f694f3f1700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-22T22:58:31.568 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-22T22:58:31.568 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-22T22:58:31.568 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-05-22T22:58:31.568 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-05-22T22:58:31.568 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113063 s, 927 MB/s 2024-05-22T22:58:31.568 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-05-22T22:58:31.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:31 smithi204 ceph-mon[36211]: purged_snaps scrub starts 2024-05-22T22:58:31.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:31 smithi204 ceph-mon[36211]: purged_snaps scrub ok 2024-05-22T22:58:31.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:31 smithi204 ceph-mon[36211]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-22T22:58:31.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:31 smithi204 ceph-mon[36211]: osdmap e72: 11 total, 10 up, 11 in 2024-05-22T22:58:31.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:31 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-22T22:58:31.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:31 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-22T22:58:31.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:31 smithi204 ceph-mon[36211]: from='osd.10 ' entity='osd.10' 2024-05-22T22:58:31.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:31 smithi096 ceph-mon[36392]: purged_snaps scrub starts 2024-05-22T22:58:31.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:31 smithi096 ceph-mon[36392]: purged_snaps scrub ok 2024-05-22T22:58:31.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:31 smithi096 ceph-mon[36392]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-22T22:58:31.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:31 smithi096 ceph-mon[36392]: osdmap e72: 11 total, 10 up, 11 in 2024-05-22T22:58:31.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:31 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-22T22:58:31.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:31 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-22T22:58:31.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:31 smithi096 ceph-mon[36392]: from='osd.10 ' entity='osd.10' 2024-05-22T22:58:31.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:31 smithi084 ceph-mon[32186]: purged_snaps scrub starts 2024-05-22T22:58:31.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:31 smithi084 ceph-mon[32186]: purged_snaps scrub ok 2024-05-22T22:58:31.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:31 smithi084 ceph-mon[32186]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-22T22:58:31.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:31 smithi084 ceph-mon[32186]: osdmap e72: 11 total, 10 up, 11 in 2024-05-22T22:58:31.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:31 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-22T22:58:31.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:31 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-22T22:58:31.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:31 smithi084 ceph-mon[32186]: from='osd.10 ' entity='osd.10' 2024-05-22T22:58:32.308 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_1 2024-05-22T22:58:32.547 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:32 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-22T22:58:32.547 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:32 smithi096 ceph-mon[36392]: osd.10 [v2:172.21.15.96:6824/1985398526,v1:172.21.15.96:6825/1985398526] boot 2024-05-22T22:58:32.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:32 smithi096 ceph-mon[36392]: osdmap e73: 11 total, 11 up, 11 in 2024-05-22T22:58:32.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:32 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-22T22:58:32.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:32 smithi096 ceph-mon[36392]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:32.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:32 smithi096 ceph-mon[36392]: osdmap e74: 11 total, 11 up, 11 in 2024-05-22T22:58:32.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:32 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:32.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:32 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:32.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:32 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:58:32.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:32 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:58:32.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:32 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:58:32.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:32 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:58:32.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:32 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:32.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:32 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:58:32.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:32 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:58:32.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:32 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:32.549 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.c/config 2024-05-22T22:58:32.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:32 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-22T22:58:32.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:32 smithi204 ceph-mon[36211]: osd.10 [v2:172.21.15.96:6824/1985398526,v1:172.21.15.96:6825/1985398526] boot 2024-05-22T22:58:32.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:32 smithi204 ceph-mon[36211]: osdmap e73: 11 total, 11 up, 11 in 2024-05-22T22:58:32.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:32 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-22T22:58:32.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:32 smithi204 ceph-mon[36211]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:32.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:32 smithi204 ceph-mon[36211]: osdmap e74: 11 total, 11 up, 11 in 2024-05-22T22:58:32.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:32 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:32.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:32 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:32.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:32 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:58:32.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:32 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:58:32.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:32 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:58:32.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:32 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:58:32.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:32 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:32.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:32 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:58:32.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:32 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:58:32.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:32 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:32.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:32 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-22T22:58:32.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:32 smithi084 ceph-mon[32186]: osd.10 [v2:172.21.15.96:6824/1985398526,v1:172.21.15.96:6825/1985398526] boot 2024-05-22T22:58:32.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:32 smithi084 ceph-mon[32186]: osdmap e73: 11 total, 11 up, 11 in 2024-05-22T22:58:32.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:32 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-22T22:58:32.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:32 smithi084 ceph-mon[32186]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-22T22:58:32.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:32 smithi084 ceph-mon[32186]: osdmap e74: 11 total, 11 up, 11 in 2024-05-22T22:58:32.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:32 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:32.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:32 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:32.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:32 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:58:32.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:32 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:58:32.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:32 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:58:32.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:32 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:58:32.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:32 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:32.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:32 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:58:32.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:32 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:58:32.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:32 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:33.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:33 smithi204 ceph-mon[36211]: Detected new or changed devices on smithi096 2024-05-22T22:58:33.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:33 smithi204 ceph-mon[36211]: Adjusting osd_memory_target on smithi096 to 4272M 2024-05-22T22:58:33.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:33 smithi204 ceph-mon[36211]: osdmap e75: 11 total, 11 up, 11 in 2024-05-22T22:58:33.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:33 smithi096 ceph-mon[36392]: Detected new or changed devices on smithi096 2024-05-22T22:58:33.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:33 smithi096 ceph-mon[36392]: Adjusting osd_memory_target on smithi096 to 4272M 2024-05-22T22:58:33.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:33 smithi096 ceph-mon[36392]: osdmap e75: 11 total, 11 up, 11 in 2024-05-22T22:58:33.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:33 smithi084 ceph-mon[32186]: Detected new or changed devices on smithi096 2024-05-22T22:58:33.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:33 smithi084 ceph-mon[32186]: Adjusting osd_memory_target on smithi096 to 4272M 2024-05-22T22:58:33.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:33 smithi084 ceph-mon[32186]: osdmap e75: 11 total, 11 up, 11 in 2024-05-22T22:58:34.223 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.222+0000 7f833e681700 1 -- 172.21.15.204:0/1226723421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8338104050 msgr2=0x7f833810a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:58:34.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.222+0000 7f833e681700 1 --2- 172.21.15.204:0/1226723421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8338104050 0x7f833810a900 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f833000b0a0 tx=0x7f833000b3b0 comp rx=0 tx=0).stop 2024-05-22T22:58:34.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.222+0000 7f833e681700 1 -- 172.21.15.204:0/1226723421 shutdown_connections 2024-05-22T22:58:34.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.222+0000 7f833e681700 1 --2- 172.21.15.204:0/1226723421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8338104050 0x7f833810a900 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:34.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.222+0000 7f833e681700 1 --2- 172.21.15.204:0/1226723421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83381006d0 0x7f8338100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:34.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.222+0000 7f833e681700 1 --2- 172.21.15.204:0/1226723421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83381036b0 0x7f8338103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:34.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.222+0000 7f833e681700 1 -- 172.21.15.204:0/1226723421 >> 172.21.15.204:0/1226723421 conn(0x7f83380fc030 msgr2=0x7f83380fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:58:34.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.222+0000 7f833e681700 1 -- 172.21.15.204:0/1226723421 shutdown_connections 2024-05-22T22:58:34.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.222+0000 7f833e681700 1 -- 172.21.15.204:0/1226723421 wait complete. 2024-05-22T22:58:34.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.223+0000 7f833e681700 1 Processor -- start 2024-05-22T22:58:34.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.223+0000 7f833e681700 1 -- start start 2024-05-22T22:58:34.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.223+0000 7f833e681700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83381006d0 0x7f8338103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:58:34.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.223+0000 7f833e681700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83381036b0 0x7f8338101660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:58:34.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.223+0000 7f833e681700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8338104050 0x7f8338101ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:58:34.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.223+0000 7f833e681700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f833810dfc0 con 0x7f8338104050 2024-05-22T22:58:34.225 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.223+0000 7f833e681700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f833810de40 con 0x7f83381036b0 2024-05-22T22:58:34.225 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.223+0000 7f833e681700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f833810e140 con 0x7f83381006d0 2024-05-22T22:58:34.225 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.223+0000 7f8337fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83381006d0 0x7f8338103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:58:34.225 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.223+0000 7f8337fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83381006d0 0x7f8338103010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:34654/0 (socket says 172.21.15.204:34654) 2024-05-22T22:58:34.225 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.223+0000 7f8337fff700 1 -- 172.21.15.204:0/3171132631 learned_addr learned my addr 172.21.15.204:0/3171132631 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-22T22:58:34.225 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.223+0000 7f833cc1e700 1 --2- 172.21.15.204:0/3171132631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8338104050 0x7f8338101ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:58:34.225 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.224+0000 7f83377fe700 1 --2- 172.21.15.204:0/3171132631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83381036b0 0x7f8338101660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:58:34.225 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.224+0000 7f8337fff700 1 -- 172.21.15.204:0/3171132631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83381036b0 msgr2=0x7f8338101660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:58:34.225 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.224+0000 7f8337fff700 1 --2- 172.21.15.204:0/3171132631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83381036b0 0x7f8338101660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:34.225 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.224+0000 7f8337fff700 1 -- 172.21.15.204:0/3171132631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8338104050 msgr2=0x7f8338101ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:58:34.225 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.224+0000 7f8337fff700 1 --2- 172.21.15.204:0/3171132631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8338104050 0x7f8338101ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:34.225 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.224+0000 7f8337fff700 1 -- 172.21.15.204:0/3171132631 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8338102450 con 0x7f83381006d0 2024-05-22T22:58:34.226 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.224+0000 7f8337fff700 1 --2- 172.21.15.204:0/3171132631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83381006d0 0x7f8338103010 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f832800d980 tx=0x7f832800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:58:34.226 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.224+0000 7f83357fa700 1 -- 172.21.15.204:0/3171132631 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8328017030 con 0x7f83381006d0 2024-05-22T22:58:34.226 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.224+0000 7f83357fa700 1 -- 172.21.15.204:0/3171132631 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8328004500 con 0x7f83381006d0 2024-05-22T22:58:34.226 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.224+0000 7f833e681700 1 -- 172.21.15.204:0/3171132631 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83381a26b0 con 0x7f83381006d0 2024-05-22T22:58:34.226 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.224+0000 7f83357fa700 1 -- 172.21.15.204:0/3171132631 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8328003aa0 con 0x7f83381006d0 2024-05-22T22:58:34.226 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.224+0000 7f833e681700 1 -- 172.21.15.204:0/3171132631 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83381a2bf0 con 0x7f83381006d0 2024-05-22T22:58:34.227 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.225+0000 7f833e681700 1 -- 172.21.15.204:0/3171132631 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8338069050 con 0x7f83381006d0 2024-05-22T22:58:34.227 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.226+0000 7f83357fa700 1 -- 172.21.15.204:0/3171132631 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f832801a070 con 0x7f83381006d0 2024-05-22T22:58:34.227 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.227+0000 7f83357fa700 1 --2- 172.21.15.204:0/3171132631 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f83200a2ca0 0x7f83200a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:58:34.227 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.227+0000 7f83357fa700 1 -- 172.21.15.204:0/3171132631 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7f83280ce2e0 con 0x7f83381006d0 2024-05-22T22:58:34.227 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.227+0000 7f83377fe700 1 --2- 172.21.15.204:0/3171132631 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f83200a2ca0 0x7f83200a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:58:34.227 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.227+0000 7f83377fe700 1 --2- 172.21.15.204:0/3171132631 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f83200a2ca0 0x7f83200a5160 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f832c005fd0 tx=0x7f832c005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:58:34.230 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.231+0000 7f83357fa700 1 -- 172.21.15.204:0/3171132631 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f83280232f0 con 0x7f83381006d0 2024-05-22T22:58:34.438 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:34.439+0000 7f833e681700 1 -- 172.21.15.204:0/3171132631 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f83380627a0 con 0x7f83200a2ca0 2024-05-22T22:58:34.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:34 smithi204 ceph-mon[36211]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:34.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:34 smithi204 ceph-mon[36211]: osdmap e76: 11 total, 11 up, 11 in 2024-05-22T22:58:34.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:34 smithi096 ceph-mon[36392]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:34.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:34 smithi096 ceph-mon[36392]: osdmap e76: 11 total, 11 up, 11 in 2024-05-22T22:58:34.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:34 smithi084 ceph-mon[32186]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:34.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:34 smithi084 ceph-mon[32186]: osdmap e76: 11 total, 11 up, 11 in 2024-05-22T22:58:35.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:35 smithi204 ceph-mon[36211]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:58:35.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:35 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:58:35.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:35 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:58:35.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:35 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:58:35.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:35 smithi204 ceph-mon[36211]: osdmap e77: 11 total, 11 up, 11 in 2024-05-22T22:58:35.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:35 smithi096 ceph-mon[36392]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:58:35.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:35 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:58:35.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:35 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:58:35.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:35 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:58:35.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:35 smithi096 ceph-mon[36392]: osdmap e77: 11 total, 11 up, 11 in 2024-05-22T22:58:35.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:35 smithi084 ceph-mon[32186]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:58:35.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:35 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-22T22:58:35.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:35 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-22T22:58:35.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:35 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:58:35.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:35 smithi084 ceph-mon[32186]: osdmap e77: 11 total, 11 up, 11 in 2024-05-22T22:58:36.478 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:36 smithi204 ceph-mon[36211]: pgmap v242: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-22T22:58:36.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:36 smithi096 ceph-mon[36392]: pgmap v242: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-22T22:58:36.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:36 smithi084 ceph-mon[32186]: pgmap v242: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-22T22:58:37.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:37 smithi204 ceph-mon[36211]: from='client.? 172.21.15.204:0/1076555852' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "238922ef-d046-40cc-833d-25b7e7a523cb"}]: dispatch 2024-05-22T22:58:37.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:37 smithi204 ceph-mon[36211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "238922ef-d046-40cc-833d-25b7e7a523cb"}]: dispatch 2024-05-22T22:58:37.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:37 smithi204 ceph-mon[36211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "238922ef-d046-40cc-833d-25b7e7a523cb"}]': finished 2024-05-22T22:58:37.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:37 smithi204 ceph-mon[36211]: osdmap e78: 12 total, 11 up, 12 in 2024-05-22T22:58:37.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:37 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-22T22:58:37.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:37 smithi096 ceph-mon[36392]: from='client.? 172.21.15.204:0/1076555852' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "238922ef-d046-40cc-833d-25b7e7a523cb"}]: dispatch 2024-05-22T22:58:37.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:37 smithi096 ceph-mon[36392]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "238922ef-d046-40cc-833d-25b7e7a523cb"}]: dispatch 2024-05-22T22:58:37.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:37 smithi096 ceph-mon[36392]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "238922ef-d046-40cc-833d-25b7e7a523cb"}]': finished 2024-05-22T22:58:37.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:37 smithi096 ceph-mon[36392]: osdmap e78: 12 total, 11 up, 12 in 2024-05-22T22:58:37.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:37 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-22T22:58:37.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:37 smithi084 ceph-mon[32186]: from='client.? 172.21.15.204:0/1076555852' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "238922ef-d046-40cc-833d-25b7e7a523cb"}]: dispatch 2024-05-22T22:58:37.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:37 smithi084 ceph-mon[32186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "238922ef-d046-40cc-833d-25b7e7a523cb"}]: dispatch 2024-05-22T22:58:37.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:37 smithi084 ceph-mon[32186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "238922ef-d046-40cc-833d-25b7e7a523cb"}]': finished 2024-05-22T22:58:37.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:37 smithi084 ceph-mon[32186]: osdmap e78: 12 total, 11 up, 12 in 2024-05-22T22:58:37.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:37 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-22T22:58:38.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:38 smithi096 ceph-mon[36392]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-05-22T22:58:38.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:38 smithi096 ceph-mon[36392]: from='client.? 172.21.15.204:0/2916954837' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:58:38.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:38 smithi084 ceph-mon[32186]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-05-22T22:58:38.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:38 smithi084 ceph-mon[32186]: from='client.? 172.21.15.204:0/2916954837' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:58:38.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:38 smithi204 ceph-mon[36211]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-05-22T22:58:38.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:38 smithi204 ceph-mon[36211]: from='client.? 172.21.15.204:0/2916954837' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-22T22:58:40.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:40 smithi096 ceph-mon[36392]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-22T22:58:40.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:40 smithi084 ceph-mon[32186]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-22T22:58:40.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:40 smithi204 ceph-mon[36211]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-22T22:58:42.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:42 smithi096 ceph-mon[36392]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:42.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:42 smithi084 ceph-mon[32186]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:42.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:42 smithi204 ceph-mon[36211]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:44.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:44 smithi096 ceph-mon[36392]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:44.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:44 smithi084 ceph-mon[32186]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:44.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:44 smithi204 ceph-mon[36211]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:45.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-22T22:58:45.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:58:45.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-22T22:58:45.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:58:45.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-22T22:58:45.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:58:46.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:46 smithi096 ceph-mon[36392]: Deploying daemon osd.11 on smithi204 2024-05-22T22:58:46.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:46 smithi096 ceph-mon[36392]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:46.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:46 smithi084 ceph-mon[32186]: Deploying daemon osd.11 on smithi204 2024-05-22T22:58:46.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:46 smithi084 ceph-mon[32186]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:46.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:46 smithi204 ceph-mon[36211]: Deploying daemon osd.11 on smithi204 2024-05-22T22:58:46.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:46 smithi204 ceph-mon[36211]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:48.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:48 smithi096 ceph-mon[36392]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:48.847 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:48 smithi204 ceph-mon[36211]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:48.872 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:48 smithi084 ceph-mon[32186]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:50.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:50 smithi204 ceph-mon[36211]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:50.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:50 smithi084 ceph-mon[32186]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:51.047 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:50 smithi096 ceph-mon[36392]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:52.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:52 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:52.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:52 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:52.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:52 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:58:52.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:52 smithi204 ceph-mon[36211]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:52.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:52 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:52.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:52 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:52.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:52 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:58:52.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:52 smithi096 ceph-mon[36392]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:52.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:52 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:52.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:52 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:52.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:52 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:58:52.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:52 smithi084 ceph-mon[32186]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:54.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:54 smithi204 ceph-mon[36211]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:54.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:54 smithi096 ceph-mon[36392]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:54.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:54 smithi084 ceph-mon[32186]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:55.159 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 11 on host 'smithi204' 2024-05-22T22:58:55.160 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:55.160+0000 7f83357fa700 1 -- 172.21.15.204:0/3171132631 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f83380627a0 con 0x7f83200a2ca0 2024-05-22T22:58:55.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:55.162+0000 7f833e681700 1 -- 172.21.15.204:0/3171132631 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f83200a2ca0 msgr2=0x7f83200a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:58:55.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:55.162+0000 7f833e681700 1 --2- 172.21.15.204:0/3171132631 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f83200a2ca0 0x7f83200a5160 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f832c005fd0 tx=0x7f832c005f60 comp rx=0 tx=0).stop 2024-05-22T22:58:55.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:55.163+0000 7f833e681700 1 -- 172.21.15.204:0/3171132631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83381006d0 msgr2=0x7f8338103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:58:55.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:55.163+0000 7f833e681700 1 --2- 172.21.15.204:0/3171132631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83381006d0 0x7f8338103010 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f832800d980 tx=0x7f832800dc90 comp rx=0 tx=0).stop 2024-05-22T22:58:55.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:55.163+0000 7f833e681700 1 -- 172.21.15.204:0/3171132631 shutdown_connections 2024-05-22T22:58:55.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:55.163+0000 7f833e681700 1 --2- 172.21.15.204:0/3171132631 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f83200a2ca0 0x7f83200a5160 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:55.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:55.163+0000 7f833e681700 1 --2- 172.21.15.204:0/3171132631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8338104050 0x7f8338101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:55.163 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:55.163+0000 7f833e681700 1 --2- 172.21.15.204:0/3171132631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83381006d0 0x7f8338103010 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:55.163 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:55.163+0000 7f833e681700 1 --2- 172.21.15.204:0/3171132631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83381036b0 0x7f8338101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:55.163 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:55.163+0000 7f833e681700 1 -- 172.21.15.204:0/3171132631 >> 172.21.15.204:0/3171132631 conn(0x7f83380fc030 msgr2=0x7f8338106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:58:55.163 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:55.163+0000 7f833e681700 1 -- 172.21.15.204:0/3171132631 shutdown_connections 2024-05-22T22:58:55.163 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:58:55.163+0000 7f833e681700 1 -- 172.21.15.204:0/3171132631 wait complete. 2024-05-22T22:58:55.891 DEBUG:teuthology.orchestra.run.smithi204:osd.11> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.11.service 2024-05-22T22:58:55.894 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-22T22:58:55.894 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph osd stat -f json 2024-05-22T22:58:56.152 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T22:58:56.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:56 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:56.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:56 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:56.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:56 smithi204 ceph-mon[36211]: from='osd.11 [v2:172.21.15.204:6824/3106116519,v1:172.21.15.204:6825/3106116519]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-22T22:58:56.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:56 smithi204 ceph-mon[36211]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-22T22:58:56.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:56 smithi204 ceph-mon[36211]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:56.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:56 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:56.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:56 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:56.274 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:56 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:56.274 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:56 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:56.274 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:56 smithi084 ceph-mon[32186]: from='osd.11 [v2:172.21.15.204:6824/3106116519,v1:172.21.15.204:6825/3106116519]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-22T22:58:56.274 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:56 smithi084 ceph-mon[32186]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-22T22:58:56.275 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:56 smithi084 ceph-mon[32186]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:56.275 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:56 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:56.275 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:56 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:56.547 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:56 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:56.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:56 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:56.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:56 smithi096 ceph-mon[36392]: from='osd.11 [v2:172.21.15.204:6824/3106116519,v1:172.21.15.204:6825/3106116519]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-22T22:58:56.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:56 smithi096 ceph-mon[36392]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-22T22:58:56.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:56 smithi096 ceph-mon[36392]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:56.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:56 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:56.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:56 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:58:57.373 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:57 smithi084 ceph-mon[32186]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-22T22:58:57.373 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:57 smithi084 ceph-mon[32186]: osdmap e79: 12 total, 11 up, 12 in 2024-05-22T22:58:57.373 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:57 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-22T22:58:57.373 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:57 smithi084 ceph-mon[32186]: from='osd.11 [v2:172.21.15.204:6824/3106116519,v1:172.21.15.204:6825/3106116519]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-22T22:58:57.373 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:57 smithi084 ceph-mon[32186]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-22T22:58:57.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:57 smithi204 ceph-mon[36211]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-22T22:58:57.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:57 smithi204 ceph-mon[36211]: osdmap e79: 12 total, 11 up, 12 in 2024-05-22T22:58:57.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:57 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-22T22:58:57.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:57 smithi204 ceph-mon[36211]: from='osd.11 [v2:172.21.15.204:6824/3106116519,v1:172.21.15.204:6825/3106116519]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-22T22:58:57.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:57 smithi204 ceph-mon[36211]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-22T22:58:57.547 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:57 smithi096 ceph-mon[36392]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-22T22:58:57.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:57 smithi096 ceph-mon[36392]: osdmap e79: 12 total, 11 up, 12 in 2024-05-22T22:58:57.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:57 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-22T22:58:57.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:57 smithi096 ceph-mon[36392]: from='osd.11 [v2:172.21.15.204:6824/3106116519,v1:172.21.15.204:6825/3106116519]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-22T22:58:57.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:57 smithi096 ceph-mon[36392]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-22T22:58:57.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 22:58:57 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T22:58:57.546+0000 7f78cf8e1700 -1 osd.11 0 waiting for initial osdmap 2024-05-22T22:58:57.904 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 22:58:57 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T22:58:57.561+0000 7f78c76cc700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-22T22:58:57.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.962+0000 7f8c32932700 1 -- 172.21.15.84:0/3726499575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c2c1012c0 msgr2=0x7f8c2c1016e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:58:57.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.962+0000 7f8c32932700 1 --2- 172.21.15.84:0/3726499575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c2c1012c0 0x7f8c2c1016e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c0099c0 tx=0x7f8c1c009cd0 comp rx=0 tx=0).stop 2024-05-22T22:58:57.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.962+0000 7f8c32932700 1 -- 172.21.15.84:0/3726499575 shutdown_connections 2024-05-22T22:58:57.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.962+0000 7f8c32932700 1 --2- 172.21.15.84:0/3726499575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c2c102e60 0x7f8c2c109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:57.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.962+0000 7f8c32932700 1 --2- 172.21.15.84:0/3726499575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c2c1012c0 0x7f8c2c1016e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:57.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.962+0000 7f8c32932700 1 --2- 172.21.15.84:0/3726499575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c2c1024c0 0x7f8c2c102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:57.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.962+0000 7f8c32932700 1 -- 172.21.15.84:0/3726499575 >> 172.21.15.84:0/3726499575 conn(0x7f8c2c0fc860 msgr2=0x7f8c2c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:58:57.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.962+0000 7f8c32932700 1 -- 172.21.15.84:0/3726499575 shutdown_connections 2024-05-22T22:58:57.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.962+0000 7f8c32932700 1 -- 172.21.15.84:0/3726499575 wait complete. 2024-05-22T22:58:57.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.962+0000 7f8c32932700 1 Processor -- start 2024-05-22T22:58:57.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.962+0000 7f8c32932700 1 -- start start 2024-05-22T22:58:57.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.963+0000 7f8c32932700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c2c1012c0 0x7f8c2c197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:58:57.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.963+0000 7f8c32932700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c2c1024c0 0x7f8c2c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:58:57.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.963+0000 7f8c32932700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c2c102e60 0x7f8c2c19ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:58:57.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.963+0000 7f8c32932700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c10cf80 con 0x7f8c2c102e60 2024-05-22T22:58:57.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.963+0000 7f8c32932700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c10ce00 con 0x7f8c2c1024c0 2024-05-22T22:58:57.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.963+0000 7f8c32932700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c10d100 con 0x7f8c2c1012c0 2024-05-22T22:58:57.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.963+0000 7f8c2bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c2c1012c0 0x7f8c2c197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:58:57.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.963+0000 7f8c30ecf700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c2c102e60 0x7f8c2c19ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:58:57.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.963+0000 7f8c2bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c2c1012c0 0x7f8c2c197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:58234/0 (socket says 172.21.15.84:58234) 2024-05-22T22:58:57.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.963+0000 7f8c2bfff700 1 -- 172.21.15.84:0/1633619640 learned_addr learned my addr 172.21.15.84:0/1633619640 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:58:57.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.963+0000 7f8c2b7fe700 1 --2- 172.21.15.84:0/1633619640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c2c1024c0 0x7f8c2c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:58:57.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.963+0000 7f8c30ecf700 1 -- 172.21.15.84:0/1633619640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c2c1012c0 msgr2=0x7f8c2c197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:58:57.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.963+0000 7f8c30ecf700 1 --2- 172.21.15.84:0/1633619640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c2c1012c0 0x7f8c2c197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:57.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.963+0000 7f8c30ecf700 1 -- 172.21.15.84:0/1633619640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c2c1024c0 msgr2=0x7f8c2c197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:58:57.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.963+0000 7f8c30ecf700 1 --2- 172.21.15.84:0/1633619640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c2c1024c0 0x7f8c2c197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:57.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.963+0000 7f8c30ecf700 1 -- 172.21.15.84:0/1633619640 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c2c19f2e0 con 0x7f8c2c102e60 2024-05-22T22:58:57.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.963+0000 7f8c30ecf700 1 --2- 172.21.15.84:0/1633619640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c2c102e60 0x7f8c2c19ec40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400be50 tx=0x7f8c24009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:58:57.964 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.963+0000 7f8c297fa700 1 -- 172.21.15.84:0/1633619640 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c24019070 con 0x7f8c2c102e60 2024-05-22T22:58:57.964 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.964+0000 7f8c297fa700 1 -- 172.21.15.84:0/1633619640 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c2400cb90 con 0x7f8c2c102e60 2024-05-22T22:58:57.964 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.964+0000 7f8c297fa700 1 -- 172.21.15.84:0/1633619640 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c240125a0 con 0x7f8c2c102e60 2024-05-22T22:58:57.964 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.964+0000 7f8c32932700 1 -- 172.21.15.84:0/1633619640 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c2c19f5d0 con 0x7f8c2c102e60 2024-05-22T22:58:57.964 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.964+0000 7f8c32932700 1 -- 172.21.15.84:0/1633619640 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c2c19fb40 con 0x7f8c2c102e60 2024-05-22T22:58:57.964 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.964+0000 7f8c32932700 1 -- 172.21.15.84:0/1633619640 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c2c069050 con 0x7f8c2c102e60 2024-05-22T22:58:57.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.966+0000 7f8c297fa700 1 -- 172.21.15.84:0/1633619640 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8c2401c070 con 0x7f8c2c102e60 2024-05-22T22:58:57.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.966+0000 7f8c297fa700 1 --2- 172.21.15.84:0/1633619640 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8c140a2c20 0x7f8c140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:58:57.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.966+0000 7f8c2bfff700 1 --2- 172.21.15.84:0/1633619640 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8c140a2c20 0x7f8c140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:58:57.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.966+0000 7f8c297fa700 1 -- 172.21.15.84:0/1633619640 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6847+0+0 (secure 0 0 0) 0x7f8c240ce460 con 0x7f8c2c102e60 2024-05-22T22:58:57.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.967+0000 7f8c2bfff700 1 --2- 172.21.15.84:0/1633619640 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8c140a2c20 0x7f8c140a50e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c009750 tx=0x7f8c1c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:58:57.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:57.970+0000 7f8c297fa700 1 -- 172.21.15.84:0/1633619640 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8c2409b980 con 0x7f8c2c102e60 2024-05-22T22:58:58.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:58 smithi204 ceph-mon[36211]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-22T22:58:58.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:58 smithi204 ceph-mon[36211]: osdmap e80: 12 total, 11 up, 12 in 2024-05-22T22:58:58.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:58 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-22T22:58:58.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:58 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-22T22:58:58.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:58 smithi204 ceph-mon[36211]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:58.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:58 smithi204 ceph-mon[36211]: from='osd.11 ' entity='osd.11' 2024-05-22T22:58:58.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:58.172+0000 7f8c32932700 1 -- 172.21.15.84:0/1633619640 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f8c2c02dbc0 con 0x7f8c2c102e60 2024-05-22T22:58:58.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:58.173+0000 7f8c297fa700 1 -- 172.21.15.84:0/1633619640 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f8c2409b280 con 0x7f8c2c102e60 2024-05-22T22:58:58.171 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:58:58.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:58.175+0000 7f8c32932700 1 -- 172.21.15.84:0/1633619640 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8c140a2c20 msgr2=0x7f8c140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:58:58.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:58.175+0000 7f8c32932700 1 --2- 172.21.15.84:0/1633619640 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8c140a2c20 0x7f8c140a50e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c009750 tx=0x7f8c1c005fb0 comp rx=0 tx=0).stop 2024-05-22T22:58:58.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:58.175+0000 7f8c32932700 1 -- 172.21.15.84:0/1633619640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c2c102e60 msgr2=0x7f8c2c19ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:58:58.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:58.175+0000 7f8c32932700 1 --2- 172.21.15.84:0/1633619640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c2c102e60 0x7f8c2c19ec40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400be50 tx=0x7f8c24009ed0 comp rx=0 tx=0).stop 2024-05-22T22:58:58.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:58.175+0000 7f8c32932700 1 -- 172.21.15.84:0/1633619640 shutdown_connections 2024-05-22T22:58:58.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:58.175+0000 7f8c32932700 1 --2- 172.21.15.84:0/1633619640 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8c140a2c20 0x7f8c140a50e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:58.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:58.175+0000 7f8c32932700 1 --2- 172.21.15.84:0/1633619640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c2c102e60 0x7f8c2c19ec40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:58.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:58.175+0000 7f8c32932700 1 --2- 172.21.15.84:0/1633619640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c2c1012c0 0x7f8c2c197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:58.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:58.175+0000 7f8c32932700 1 --2- 172.21.15.84:0/1633619640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c2c1024c0 0x7f8c2c197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:58:58.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:58.175+0000 7f8c32932700 1 -- 172.21.15.84:0/1633619640 >> 172.21.15.84:0/1633619640 conn(0x7f8c2c0fc860 msgr2=0x7f8c2c105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:58:58.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:58.175+0000 7f8c32932700 1 -- 172.21.15.84:0/1633619640 shutdown_connections 2024-05-22T22:58:58.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:58:58.175+0000 7f8c32932700 1 -- 172.21.15.84:0/1633619640 wait complete. 2024-05-22T22:58:58.373 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:58 smithi084 ceph-mon[32186]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-22T22:58:58.373 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:58 smithi084 ceph-mon[32186]: osdmap e80: 12 total, 11 up, 12 in 2024-05-22T22:58:58.373 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:58 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-22T22:58:58.373 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:58 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-22T22:58:58.373 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:58 smithi084 ceph-mon[32186]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:58.373 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:58 smithi084 ceph-mon[32186]: from='osd.11 ' entity='osd.11' 2024-05-22T22:58:58.547 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:58 smithi096 ceph-mon[36392]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-22T22:58:58.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:58 smithi096 ceph-mon[36392]: osdmap e80: 12 total, 11 up, 12 in 2024-05-22T22:58:58.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:58 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-22T22:58:58.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:58 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-22T22:58:58.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:58 smithi096 ceph-mon[36392]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-22T22:58:58.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:58 smithi096 ceph-mon[36392]: from='osd.11 ' entity='osd.11' 2024-05-22T22:58:58.850 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":80,"num_osds":12,"num_up_osds":11,"osd_up_since":1716418711,"num_in_osds":12,"osd_in_since":1716418717,"num_remapped_pgs":0} 2024-05-22T22:58:59.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:59 smithi204 ceph-mon[36211]: purged_snaps scrub starts 2024-05-22T22:58:59.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:59 smithi204 ceph-mon[36211]: purged_snaps scrub ok 2024-05-22T22:58:59.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:59 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-22T22:58:59.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:59 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1633619640' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-22T22:58:59.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:59 smithi204 ceph-mon[36211]: osd.11 [v2:172.21.15.204:6824/3106116519,v1:172.21.15.204:6825/3106116519] boot 2024-05-22T22:58:59.410 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:59 smithi204 ceph-mon[36211]: osdmap e81: 12 total, 12 up, 12 in 2024-05-22T22:58:59.410 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:58:59 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-22T22:58:59.424 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:59 smithi084 ceph-mon[32186]: purged_snaps scrub starts 2024-05-22T22:58:59.424 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:59 smithi084 ceph-mon[32186]: purged_snaps scrub ok 2024-05-22T22:58:59.424 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:59 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-22T22:58:59.424 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:59 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1633619640' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-22T22:58:59.425 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:59 smithi084 ceph-mon[32186]: osd.11 [v2:172.21.15.204:6824/3106116519,v1:172.21.15.204:6825/3106116519] boot 2024-05-22T22:58:59.425 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:59 smithi084 ceph-mon[32186]: osdmap e81: 12 total, 12 up, 12 in 2024-05-22T22:58:59.425 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:58:59 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-22T22:58:59.547 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:59 smithi096 ceph-mon[36392]: purged_snaps scrub starts 2024-05-22T22:58:59.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:59 smithi096 ceph-mon[36392]: purged_snaps scrub ok 2024-05-22T22:58:59.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:59 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-22T22:58:59.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:59 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1633619640' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-22T22:58:59.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:59 smithi096 ceph-mon[36392]: osd.11 [v2:172.21.15.204:6824/3106116519,v1:172.21.15.204:6825/3106116519] boot 2024-05-22T22:58:59.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:59 smithi096 ceph-mon[36392]: osdmap e81: 12 total, 12 up, 12 in 2024-05-22T22:58:59.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:58:59 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-22T22:58:59.851 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph osd stat -f json 2024-05-22T22:59:00.091 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T22:59:00.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:00 smithi204 ceph-mon[36211]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:00.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:00 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:00.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:00 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:00.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:00 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:59:00.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:00 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:59:00.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:00 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:59:00.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:00 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:59:00.410 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:00 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:00.410 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:00 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:00.410 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:00 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:59:00.410 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:00 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:00.410 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:00 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:59:00.483 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:00 smithi084 ceph-mon[32186]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:00.484 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:00 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:00.484 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:00 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:00.484 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:00 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:59:00.484 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:00 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:59:00.484 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:00 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:59:00.484 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:00 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:59:00.484 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:00 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:00.484 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:00 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:00.484 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:00 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:59:00.484 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:00 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:00.484 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:00 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:59:00.550 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:00 smithi096 ceph-mon[36392]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:00.550 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:00 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:00.550 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:00 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:00.550 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:00 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:59:00.550 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:00 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:59:00.550 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:00 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:59:00.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:00 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-22T22:59:00.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:00 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:00.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:00 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:00.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:00 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:59:00.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:00 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:00.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:00 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:59:01.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:01 smithi204 ceph-mon[36211]: Detected new or changed devices on smithi204 2024-05-22T22:59:01.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:01 smithi204 ceph-mon[36211]: Adjusting osd_memory_target on smithi204 to 4272M 2024-05-22T22:59:01.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:01 smithi204 ceph-mon[36211]: osdmap e82: 12 total, 12 up, 12 in 2024-05-22T22:59:01.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:01 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:01.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:01 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:59:01.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:01 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:01.501 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:01 smithi084 ceph-mon[32186]: Detected new or changed devices on smithi204 2024-05-22T22:59:01.502 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:01 smithi084 ceph-mon[32186]: Adjusting osd_memory_target on smithi204 to 4272M 2024-05-22T22:59:01.502 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:01 smithi084 ceph-mon[32186]: osdmap e82: 12 total, 12 up, 12 in 2024-05-22T22:59:01.502 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:01 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:01.502 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:01 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:59:01.502 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:01 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:01.547 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:01 smithi096 ceph-mon[36392]: Detected new or changed devices on smithi204 2024-05-22T22:59:01.547 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:01 smithi096 ceph-mon[36392]: Adjusting osd_memory_target on smithi204 to 4272M 2024-05-22T22:59:01.547 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:01 smithi096 ceph-mon[36392]: osdmap e82: 12 total, 12 up, 12 in 2024-05-22T22:59:01.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:01 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:01.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:01 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:59:01.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:01 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:01.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.921+0000 7faecd281700 1 -- 172.21.15.84:0/3494715311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faec8103f70 msgr2=0x7faec810a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:01.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.921+0000 7faecd281700 1 --2- 172.21.15.84:0/3494715311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faec8103f70 0x7faec810a820 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faec000b0d0 tx=0x7faec000b3e0 comp rx=0 tx=0).stop 2024-05-22T22:59:01.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.921+0000 7faecd281700 1 -- 172.21.15.84:0/3494715311 shutdown_connections 2024-05-22T22:59:01.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.921+0000 7faecd281700 1 --2- 172.21.15.84:0/3494715311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faec8103f70 0x7faec810a820 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:01.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.921+0000 7faecd281700 1 --2- 172.21.15.84:0/3494715311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faec81023d0 0x7faec81027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:01.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.921+0000 7faecd281700 1 --2- 172.21.15.84:0/3494715311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faec81035d0 0x7faec8103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:01.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.921+0000 7faecd281700 1 -- 172.21.15.84:0/3494715311 >> 172.21.15.84:0/3494715311 conn(0x7faec80fd970 msgr2=0x7faec80ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:01.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.921+0000 7faecd281700 1 -- 172.21.15.84:0/3494715311 shutdown_connections 2024-05-22T22:59:01.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.921+0000 7faecd281700 1 -- 172.21.15.84:0/3494715311 wait complete. 2024-05-22T22:59:01.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.922+0000 7faecd281700 1 Processor -- start 2024-05-22T22:59:01.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.922+0000 7faecd281700 1 -- start start 2024-05-22T22:59:01.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.922+0000 7faecd281700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faec81023d0 0x7faec8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:01.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.922+0000 7faecd281700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faec81035d0 0x7faec81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:01.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.922+0000 7faecd281700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faec8103f70 0x7faec819fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:01.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.922+0000 7faecd281700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faec810df90 con 0x7faec8103f70 2024-05-22T22:59:01.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.922+0000 7faecd281700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faec810de10 con 0x7faec81023d0 2024-05-22T22:59:01.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.922+0000 7faecd281700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faec810e110 con 0x7faec81035d0 2024-05-22T22:59:01.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.922+0000 7faec77fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faec8103f70 0x7faec819fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:01.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.922+0000 7faec77fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faec8103f70 0x7faec819fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39128/0 (socket says 172.21.15.84:39128) 2024-05-22T22:59:01.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.922+0000 7faec77fe700 1 -- 172.21.15.84:0/555840705 learned_addr learned my addr 172.21.15.84:0/555840705 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:01.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.922+0000 7faec6ffd700 1 --2- 172.21.15.84:0/555840705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faec81023d0 0x7faec8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:01.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.922+0000 7faebffff700 1 --2- 172.21.15.84:0/555840705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faec81035d0 0x7faec81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:01.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.923+0000 7faec77fe700 1 -- 172.21.15.84:0/555840705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faec81035d0 msgr2=0x7faec81989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:01.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.923+0000 7faec77fe700 1 --2- 172.21.15.84:0/555840705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faec81035d0 0x7faec81989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:01.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.923+0000 7faec77fe700 1 -- 172.21.15.84:0/555840705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faec81023d0 msgr2=0x7faec8198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:01.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.923+0000 7faec77fe700 1 --2- 172.21.15.84:0/555840705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faec81023d0 0x7faec8198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:01.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.923+0000 7faec77fe700 1 -- 172.21.15.84:0/555840705 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec81a00f0 con 0x7faec8103f70 2024-05-22T22:59:01.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.923+0000 7faec77fe700 1 --2- 172.21.15.84:0/555840705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faec8103f70 0x7faec819fa50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faec0007d10 tx=0x7faec0007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:01.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.923+0000 7faec4ff9700 1 -- 172.21.15.84:0/555840705 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec0021020 con 0x7faec8103f70 2024-05-22T22:59:01.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.923+0000 7faec4ff9700 1 -- 172.21.15.84:0/555840705 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faec001d950 con 0x7faec8103f70 2024-05-22T22:59:01.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.923+0000 7faec4ff9700 1 -- 172.21.15.84:0/555840705 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec0005450 con 0x7faec8103f70 2024-05-22T22:59:01.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.923+0000 7faecd281700 1 -- 172.21.15.84:0/555840705 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec81a0380 con 0x7faec8103f70 2024-05-22T22:59:01.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.923+0000 7faecd281700 1 -- 172.21.15.84:0/555840705 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec81a0840 con 0x7faec8103f70 2024-05-22T22:59:01.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.924+0000 7faecd281700 1 -- 172.21.15.84:0/555840705 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faec8069050 con 0x7faec8103f70 2024-05-22T22:59:01.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.925+0000 7faec4ff9700 1 -- 172.21.15.84:0/555840705 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7faec0024020 con 0x7faec8103f70 2024-05-22T22:59:01.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.926+0000 7faec4ff9700 1 --2- 172.21.15.84:0/555840705 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7faeb40a2c20 0x7faeb40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:01.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.926+0000 7faec6ffd700 1 --2- 172.21.15.84:0/555840705 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7faeb40a2c20 0x7faeb40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:01.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.926+0000 7faec4ff9700 1 -- 172.21.15.84:0/555840705 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7faec00cfd90 con 0x7faec8103f70 2024-05-22T22:59:01.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.926+0000 7faec6ffd700 1 --2- 172.21.15.84:0/555840705 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7faeb40a2c20 0x7faeb40a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faec8103430 tx=0x7faeb800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:01.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:01.930+0000 7faec4ff9700 1 -- 172.21.15.84:0/555840705 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faec009d220 con 0x7faec8103f70 2024-05-22T22:59:02.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:02.124+0000 7faecd281700 1 -- 172.21.15.84:0/555840705 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7faec8062f70 con 0x7faec8103f70 2024-05-22T22:59:02.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:02.125+0000 7faec4ff9700 1 -- 172.21.15.84:0/555840705 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7faec009cb20 con 0x7faec8103f70 2024-05-22T22:59:02.124 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:02.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:02.128+0000 7faecd281700 1 -- 172.21.15.84:0/555840705 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7faeb40a2c20 msgr2=0x7faeb40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:02.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:02.128+0000 7faecd281700 1 --2- 172.21.15.84:0/555840705 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7faeb40a2c20 0x7faeb40a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faec8103430 tx=0x7faeb800a3b0 comp rx=0 tx=0).stop 2024-05-22T22:59:02.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:02.128+0000 7faecd281700 1 -- 172.21.15.84:0/555840705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faec8103f70 msgr2=0x7faec819fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:02.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:02.128+0000 7faecd281700 1 --2- 172.21.15.84:0/555840705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faec8103f70 0x7faec819fa50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faec0007d10 tx=0x7faec0007d40 comp rx=0 tx=0).stop 2024-05-22T22:59:02.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:02.128+0000 7faecd281700 1 -- 172.21.15.84:0/555840705 shutdown_connections 2024-05-22T22:59:02.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:02.128+0000 7faecd281700 1 --2- 172.21.15.84:0/555840705 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7faeb40a2c20 0x7faeb40a50e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:02.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:02.128+0000 7faecd281700 1 --2- 172.21.15.84:0/555840705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faec8103f70 0x7faec819fa50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:02.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:02.128+0000 7faecd281700 1 --2- 172.21.15.84:0/555840705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faec81023d0 0x7faec8198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:02.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:02.128+0000 7faecd281700 1 --2- 172.21.15.84:0/555840705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faec81035d0 0x7faec81989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:02.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:02.128+0000 7faecd281700 1 -- 172.21.15.84:0/555840705 >> 172.21.15.84:0/555840705 conn(0x7faec80fd970 msgr2=0x7faec8106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:02.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:02.128+0000 7faecd281700 1 -- 172.21.15.84:0/555840705 shutdown_connections 2024-05-22T22:59:02.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:02.128+0000 7faecd281700 1 -- 172.21.15.84:0/555840705 wait complete. 2024-05-22T22:59:02.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:02 smithi204 ceph-mon[36211]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:02.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:02 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/555840705' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-22T22:59:02.495 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:02 smithi084 ceph-mon[32186]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:02.496 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:02 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/555840705' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-22T22:59:02.547 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:02 smithi096 ceph-mon[36392]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:02.547 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:02 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/555840705' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-22T22:59:02.943 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1716418738,"num_in_osds":12,"osd_in_since":1716418717,"num_remapped_pgs":0} 2024-05-22T22:59:02.944 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-22T22:59:03.181 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T22:59:04.747 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:04 smithi084 ceph-mon[32186]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:04.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:04 smithi096 ceph-mon[36392]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:04.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:04 smithi204 ceph-mon[36211]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:05.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.979+0000 7fccde978700 1 -- 172.21.15.84:0/3092068363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccd81072c0 msgr2=0x7fccd81076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:05.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.979+0000 7fccde978700 1 --2- 172.21.15.84:0/3092068363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccd81072c0 0x7fccd81076a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009990 tx=0x7fccc8009ca0 comp rx=0 tx=0).stop 2024-05-22T22:59:05.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.979+0000 7fccde978700 1 -- 172.21.15.84:0/3092068363 shutdown_connections 2024-05-22T22:59:05.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.979+0000 7fccde978700 1 --2- 172.21.15.84:0/3092068363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccd8101c10 0x7fccd810e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:05.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.979+0000 7fccde978700 1 --2- 172.21.15.84:0/3092068363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccd81072c0 0x7fccd81076a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:05.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.979+0000 7fccde978700 1 --2- 172.21.15.84:0/3092068363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccd8101250 0x7fccd81016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:05.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.979+0000 7fccde978700 1 -- 172.21.15.84:0/3092068363 >> 172.21.15.84:0/3092068363 conn(0x7fccd80781d0 msgr2=0x7fccd80785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:05.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.979+0000 7fccde978700 1 -- 172.21.15.84:0/3092068363 shutdown_connections 2024-05-22T22:59:05.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.979+0000 7fccde978700 1 -- 172.21.15.84:0/3092068363 wait complete. 2024-05-22T22:59:05.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.980+0000 7fccde978700 1 Processor -- start 2024-05-22T22:59:05.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.980+0000 7fccde978700 1 -- start start 2024-05-22T22:59:05.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.980+0000 7fccde978700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccd8101250 0x7fccd819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:05.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.980+0000 7fccde978700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccd8101c10 0x7fccd819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:05.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.980+0000 7fccde978700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccd81072c0 0x7fccd8198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:05.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.980+0000 7fccde978700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81115b0 con 0x7fccd8101250 2024-05-22T22:59:05.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.980+0000 7fccde978700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8111430 con 0x7fccd81072c0 2024-05-22T22:59:05.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.980+0000 7fccde978700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8111730 con 0x7fccd8101c10 2024-05-22T22:59:05.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.980+0000 7fccd7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccd8101250 0x7fccd819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:05.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.980+0000 7fccd7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccd8101250 0x7fccd819e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39132/0 (socket says 172.21.15.84:39132) 2024-05-22T22:59:05.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.980+0000 7fccd7fff700 1 -- 172.21.15.84:0/1966467485 learned_addr learned my addr 172.21.15.84:0/1966467485 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:05.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.980+0000 7fccdcf15700 1 --2- 172.21.15.84:0/1966467485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccd81072c0 0x7fccd8198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:05.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.980+0000 7fcccffff700 1 --2- 172.21.15.84:0/1966467485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccd8101c10 0x7fccd819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:05.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.981+0000 7fccd7fff700 1 -- 172.21.15.84:0/1966467485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccd8101c10 msgr2=0x7fccd819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:05.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.981+0000 7fccd7fff700 1 --2- 172.21.15.84:0/1966467485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccd8101c10 0x7fccd819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:05.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.981+0000 7fccd7fff700 1 -- 172.21.15.84:0/1966467485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccd81072c0 msgr2=0x7fccd8198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:05.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.981+0000 7fccd7fff700 1 --2- 172.21.15.84:0/1966467485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccd81072c0 0x7fccd8198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:05.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.981+0000 7fccd7fff700 1 -- 172.21.15.84:0/1966467485 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd81990c0 con 0x7fccd8101250 2024-05-22T22:59:05.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.981+0000 7fcccffff700 1 --2- 172.21.15.84:0/1966467485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccd8101c10 0x7fccd819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T22:59:05.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.981+0000 7fccd7fff700 1 --2- 172.21.15.84:0/1966467485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccd8101250 0x7fccd819e7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fccc8000c00 tx=0x7fccc8003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:05.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.981+0000 7fccd5ffb700 1 -- 172.21.15.84:0/1966467485 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8027070 con 0x7fccd8101250 2024-05-22T22:59:05.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.981+0000 7fccde978700 1 -- 172.21.15.84:0/1966467485 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd8199350 con 0x7fccd8101250 2024-05-22T22:59:05.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.981+0000 7fccd5ffb700 1 -- 172.21.15.84:0/1966467485 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc8004110 con 0x7fccd8101250 2024-05-22T22:59:05.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.981+0000 7fccd5ffb700 1 -- 172.21.15.84:0/1966467485 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc800f460 con 0x7fccd8101250 2024-05-22T22:59:05.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.981+0000 7fccde978700 1 -- 172.21.15.84:0/1966467485 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd81a5670 con 0x7fccd8101250 2024-05-22T22:59:05.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.982+0000 7fccde978700 1 -- 172.21.15.84:0/1966467485 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd8102970 con 0x7fccd8101250 2024-05-22T22:59:05.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.983+0000 7fccd5ffb700 1 -- 172.21.15.84:0/1966467485 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fccc802a070 con 0x7fccd8101250 2024-05-22T22:59:05.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.984+0000 7fccd5ffb700 1 --2- 172.21.15.84:0/1966467485 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fccc40a2c20 0x7fccc40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:05.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.984+0000 7fcccffff700 1 --2- 172.21.15.84:0/1966467485 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fccc40a2c20 0x7fccc40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:05.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.984+0000 7fccd5ffb700 1 -- 172.21.15.84:0/1966467485 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fccc80cfb30 con 0x7fccd8101250 2024-05-22T22:59:05.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.984+0000 7fcccffff700 1 --2- 172.21.15.84:0/1966467485 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fccc40a2c20 0x7fccc40a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fccd80ff910 tx=0x7fccc0005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:05.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:04.988+0000 7fccd5ffb700 1 -- 172.21.15.84:0/1966467485 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fccc805e850 con 0x7fccd8101250 2024-05-22T22:59:05.186 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:05.188+0000 7fccde978700 1 -- 172.21.15.84:0/1966467485 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fccd8199e90 con 0x7fccd8101250 2024-05-22T22:59:05.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:05.189+0000 7fccd5ffb700 1 -- 172.21.15.84:0/1966467485 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18431 (secure 0 0 0) 0x7fccc8013070 con 0x7fccd8101250 2024-05-22T22:59:05.188 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:05.188 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":82,"fsid":"c1135a12-188d-11ef-bc9a-c7b262605968","created":"2024-05-22T22:51:49.878218+0000","modified":"2024-05-22T22:59:00.132491+0000","last_up_change":"2024-05-22T22:58:58.550278+0000","last_in_change":"2024-05-22T22:58:37.322139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T22:55:07.719394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762df065-fb4f-4c71-aab8-a62d2282c253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6803","nonce":1559945826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6805","nonce":1559945826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6809","nonce":1559945826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6807","nonce":1559945826}]},"public_addr":"172.21.15.84:6803/1559945826","cluster_addr":"172.21.15.84:6805/1559945826","heartbeat_back_addr":"172.21.15.84:6809/1559945826","heartbeat_front_addr":"172.21.15.84:6807/1559945826","state":["exists","up"]},{"osd":1,"uuid":"3fb2385d-6a34-49e3-88e1-116237beb346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6801","nonce":2996505353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6803","nonce":2996505353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6807","nonce":2996505353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6805","nonce":2996505353}]},"public_addr":"172.21.15.96:6801/2996505353","cluster_addr":"172.21.15.96:6803/2996505353","heartbeat_back_addr":"172.21.15.96:6807/2996505353","heartbeat_front_addr":"172.21.15.96:6805/2996505353","state":["exists","up"]},{"osd":2,"uuid":"e306e88e-a308-4585-ae11-c8af10b84038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6801","nonce":1499577897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6803","nonce":1499577897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6807","nonce":1499577897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6805","nonce":1499577897}]},"public_addr":"172.21.15.204:6801/1499577897","cluster_addr":"172.21.15.204:6803/1499577897","heartbeat_back_addr":"172.21.15.204:6807/1499577897","heartbeat_front_addr":"172.21.15.204:6805/1499577897","state":["exists","up"]},{"osd":3,"uuid":"d2a5fb97-6f60-4d45-9f18-447cae480974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6811","nonce":2869340648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6813","nonce":2869340648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6817","nonce":2869340648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6815","nonce":2869340648}]},"public_addr":"172.21.15.84:6811/2869340648","cluster_addr":"172.21.15.84:6813/2869340648","heartbeat_back_addr":"172.21.15.84:6817/2869340648","heartbeat_front_addr":"172.21.15.84:6815/2869340648","state":["exists","up"]},{"osd":4,"uuid":"4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6809","nonce":2554265648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6811","nonce":2554265648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6815","nonce":2554265648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6813","nonce":2554265648}]},"public_addr":"172.21.15.96:6809/2554265648","cluster_addr":"172.21.15.96:6811/2554265648","heartbeat_back_addr":"172.21.15.96:6815/2554265648","heartbeat_front_addr":"172.21.15.96:6813/2554265648","state":["exists","up"]},{"osd":5,"uuid":"7996bf5e-93a5-41b0-a9b1-99e7ef4514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6809","nonce":4075476986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6811","nonce":4075476986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6815","nonce":4075476986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6813","nonce":4075476986}]},"public_addr":"172.21.15.204:6809/4075476986","cluster_addr":"172.21.15.204:6811/4075476986","heartbeat_back_addr":"172.21.15.204:6815/4075476986","heartbeat_front_addr":"172.21.15.204:6813/4075476986","state":["exists","up"]},{"osd":6,"uuid":"f4b7c5de-1836-4d16-8603-3c517524d9fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6819","nonce":1806202865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6821","nonce":1806202865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6825","nonce":1806202865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6823","nonce":1806202865}]},"public_addr":"172.21.15.84:6819/1806202865","cluster_addr":"172.21.15.84:6821/1806202865","heartbeat_back_addr":"172.21.15.84:6825/1806202865","heartbeat_front_addr":"172.21.15.84:6823/1806202865","state":["exists","up"]},{"osd":7,"uuid":"69a18c43-5447-480d-9466-190830070c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6817","nonce":3195792774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6819","nonce":3195792774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6823","nonce":3195792774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6821","nonce":3195792774}]},"public_addr":"172.21.15.96:6817/3195792774","cluster_addr":"172.21.15.96:6819/3195792774","heartbeat_back_addr":"172.21.15.96:6823/3195792774","heartbeat_front_addr":"172.21.15.96:6821/3195792774","state":["exists","up"]},{"osd":8,"uuid":"2c5cadca-689a-441e-847c-ba370d400ef1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6817","nonce":606997927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6819","nonce":606997927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6823","nonce":606997927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6821","nonce":606997927}]},"public_addr":"172.21.15.204:6817/606997927","cluster_addr":"172.21.15.204:6819/606997927","heartbeat_back_addr":"172.21.15.204:6823/606997927","heartbeat_front_addr":"172.21.15.204:6821/606997927","state":["exists","up"]},{"osd":9,"uuid":"e3984fb2-7c00-4275-a10e-bca7c859f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6827","nonce":2786189012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6829","nonce":2786189012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6833","nonce":2786189012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6831","nonce":2786189012}]},"public_addr":"172.21.15.84:6827/2786189012","cluster_addr":"172.21.15.84:6829/2786189012","heartbeat_back_addr":"172.21.15.84:6833/2786189012","heartbeat_front_addr":"172.21.15.84:6831/2786189012","state":["exists","up"]},{"osd":10,"uuid":"b4938160-a808-4ded-91c2-d245c99fff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6825","nonce":1985398526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6827","nonce":1985398526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6831","nonce":1985398526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6829","nonce":1985398526}]},"public_addr":"172.21.15.96:6825/1985398526","cluster_addr":"172.21.15.96:6827/1985398526","heartbeat_back_addr":"172.21.15.96:6831/1985398526","heartbeat_front_addr":"172.21.15.96:6829/1985398526","state":["exists","up"]},{"osd":11,"uuid":"238922ef-d046-40cc-833d-25b7e7a523cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6825","nonce":3106116519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6827","nonce":3106116519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6831","nonce":3106116519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6829","nonce":3106116519}]},"public_addr":"172.21.15.204:6825/3106116519","cluster_addr":"172.21.15.204:6827/3106116519","heartbeat_back_addr":"172.21.15.204:6831/3106116519","heartbeat_front_addr":"172.21.15.204:6829/3106116519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:16.443480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:39.898133+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:03.664544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:28.838687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:53.829733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:19.269307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:45.166910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:10.771614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:36.678516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:03.150592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:29.327769+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:56.315938+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6801/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:6800/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/2766288608":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/950455999":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/4000289608":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1650467363":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1052746114":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6800/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6801/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/14610813":"2024-05-23T22:52:45.395791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T22:59:05.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:05.191+0000 7fccde978700 1 -- 172.21.15.84:0/1966467485 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fccc40a2c20 msgr2=0x7fccc40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:05.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:05.191+0000 7fccde978700 1 --2- 172.21.15.84:0/1966467485 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fccc40a2c20 0x7fccc40a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fccd80ff910 tx=0x7fccc0005f50 comp rx=0 tx=0).stop 2024-05-22T22:59:05.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:05.191+0000 7fccde978700 1 -- 172.21.15.84:0/1966467485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccd8101250 msgr2=0x7fccd819e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:05.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:05.191+0000 7fccde978700 1 --2- 172.21.15.84:0/1966467485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccd8101250 0x7fccd819e7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fccc8000c00 tx=0x7fccc8003680 comp rx=0 tx=0).stop 2024-05-22T22:59:05.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:05.191+0000 7fccde978700 1 -- 172.21.15.84:0/1966467485 shutdown_connections 2024-05-22T22:59:05.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:05.191+0000 7fccde978700 1 --2- 172.21.15.84:0/1966467485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccd81072c0 0x7fccd8198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:05.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:05.191+0000 7fccde978700 1 --2- 172.21.15.84:0/1966467485 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fccc40a2c20 0x7fccc40a50e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:05.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:05.191+0000 7fccde978700 1 --2- 172.21.15.84:0/1966467485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccd8101250 0x7fccd819e7a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:05.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:05.191+0000 7fccde978700 1 --2- 172.21.15.84:0/1966467485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccd8101c10 0x7fccd819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:05.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:05.191+0000 7fccde978700 1 -- 172.21.15.84:0/1966467485 >> 172.21.15.84:0/1966467485 conn(0x7fccd80781d0 msgr2=0x7fccd8100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:05.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:05.191+0000 7fccde978700 1 -- 172.21.15.84:0/1966467485 shutdown_connections 2024-05-22T22:59:05.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:05.191+0000 7fccde978700 1 -- 172.21.15.84:0/1966467485 wait complete. 2024-05-22T22:59:05.635 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:05 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1966467485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T22:59:05.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:05 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1966467485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T22:59:05.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:05 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1966467485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T22:59:05.960 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T22:55:07.719394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T22:59:05.961 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-22T22:59:06.195 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T22:59:06.623 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:06 smithi084 ceph-mon[32186]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:06.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:06 smithi096 ceph-mon[36392]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:06.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:06 smithi204 ceph-mon[36211]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:07.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.943+0000 7f0fbe293700 1 -- 172.21.15.84:0/3591208415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fb81076a0 msgr2=0x7f0fb810f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:07.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.943+0000 7f0fbe293700 1 --2- 172.21.15.84:0/3591208415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fb81076a0 0x7f0fb810f270 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000b0a0 tx=0x7f0fb000b3b0 comp rx=0 tx=0).stop 2024-05-22T22:59:07.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.943+0000 7f0fbe293700 1 -- 172.21.15.84:0/3591208415 shutdown_connections 2024-05-22T22:59:07.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.943+0000 7f0fbe293700 1 --2- 172.21.15.84:0/3591208415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fb81076a0 0x7f0fb810f270 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:07.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.943+0000 7f0fbe293700 1 --2- 172.21.15.84:0/3591208415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fb8102610 0x7f0fb81029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:07.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.943+0000 7f0fbe293700 1 --2- 172.21.15.84:0/3591208415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fb8102fc0 0x7f0fb8106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:07.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.943+0000 7f0fbe293700 1 -- 172.21.15.84:0/3591208415 >> 172.21.15.84:0/3591208415 conn(0x7f0fb80794c0 msgr2=0x7f0fb80798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:07.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.943+0000 7f0fbe293700 1 -- 172.21.15.84:0/3591208415 shutdown_connections 2024-05-22T22:59:07.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.943+0000 7f0fbe293700 1 -- 172.21.15.84:0/3591208415 wait complete. 2024-05-22T22:59:07.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.943+0000 7f0fbe293700 1 Processor -- start 2024-05-22T22:59:07.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.943+0000 7f0fbe293700 1 -- start start 2024-05-22T22:59:07.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.944+0000 7f0fbe293700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fb8102610 0x7f0fb819e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:07.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.944+0000 7f0fbe293700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fb8102fc0 0x7f0fb819ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:07.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.944+0000 7f0fbe293700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fb81076a0 0x7f0fb8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:07.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.944+0000 7f0fbe293700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb8112680 con 0x7f0fb8102fc0 2024-05-22T22:59:07.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.944+0000 7f0fbe293700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb8112500 con 0x7f0fb8102610 2024-05-22T22:59:07.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.944+0000 7f0fbe293700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb8112800 con 0x7f0fb81076a0 2024-05-22T22:59:07.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.944+0000 7f0fb77fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fb8102fc0 0x7f0fb819ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:07.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.944+0000 7f0fb77fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fb8102fc0 0x7f0fb819ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39156/0 (socket says 172.21.15.84:39156) 2024-05-22T22:59:07.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.944+0000 7f0fb77fe700 1 -- 172.21.15.84:0/1358462118 learned_addr learned my addr 172.21.15.84:0/1358462118 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:07.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.944+0000 7f0fb7fff700 1 --2- 172.21.15.84:0/1358462118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fb8102610 0x7f0fb819e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:07.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.944+0000 7f0fbc830700 1 --2- 172.21.15.84:0/1358462118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fb81076a0 0x7f0fb8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:07.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.944+0000 7f0fb77fe700 1 -- 172.21.15.84:0/1358462118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fb81076a0 msgr2=0x7f0fb8198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:07.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.944+0000 7f0fb77fe700 1 --2- 172.21.15.84:0/1358462118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fb81076a0 0x7f0fb8198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:07.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.944+0000 7f0fb77fe700 1 -- 172.21.15.84:0/1358462118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fb8102610 msgr2=0x7f0fb819e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:07.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.944+0000 7f0fb77fe700 1 --2- 172.21.15.84:0/1358462118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fb8102610 0x7f0fb819e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:07.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.944+0000 7f0fb77fe700 1 -- 172.21.15.84:0/1358462118 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb81990a0 con 0x7f0fb8102fc0 2024-05-22T22:59:07.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.944+0000 7f0fbc830700 1 --2- 172.21.15.84:0/1358462118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fb81076a0 0x7f0fb8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T22:59:07.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.944+0000 7f0fb77fe700 1 --2- 172.21.15.84:0/1358462118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fb8102fc0 0x7f0fb819ec70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00e970 tx=0x7f0fac00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:07.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.944+0000 7f0fb57fa700 1 -- 172.21.15.84:0/1358462118 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac00cbd0 con 0x7f0fb8102fc0 2024-05-22T22:59:07.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.944+0000 7f0fbe293700 1 -- 172.21.15.84:0/1358462118 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb8199390 con 0x7f0fb8102fc0 2024-05-22T22:59:07.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.944+0000 7f0fb57fa700 1 -- 172.21.15.84:0/1358462118 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fac00cd70 con 0x7f0fb8102fc0 2024-05-22T22:59:07.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.944+0000 7f0fb57fa700 1 -- 172.21.15.84:0/1358462118 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac010430 con 0x7f0fb8102fc0 2024-05-22T22:59:07.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.944+0000 7f0fbe293700 1 -- 172.21.15.84:0/1358462118 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb81a5600 con 0x7f0fb8102fc0 2024-05-22T22:59:07.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.945+0000 7f0fbe293700 1 -- 172.21.15.84:0/1358462118 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fb8069050 con 0x7f0fb8102fc0 2024-05-22T22:59:07.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.947+0000 7f0fb57fa700 1 -- 172.21.15.84:0/1358462118 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0fac004750 con 0x7f0fb8102fc0 2024-05-22T22:59:07.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.948+0000 7f0fb57fa700 1 --2- 172.21.15.84:0/1358462118 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0fa00a2c20 0x7f0fa00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:07.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.948+0000 7f0fb7fff700 1 --2- 172.21.15.84:0/1358462118 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0fa00a2c20 0x7f0fa00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:07.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.948+0000 7f0fb57fa700 1 -- 172.21.15.84:0/1358462118 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0fac0cf0c0 con 0x7f0fb8102fc0 2024-05-22T22:59:07.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.948+0000 7f0fb7fff700 1 --2- 172.21.15.84:0/1358462118 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0fa00a2c20 0x7f0fa00a50e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8005fd0 tx=0x7f0fa8005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:07.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:07.951+0000 7f0fb57fa700 1 -- 172.21.15.84:0/1358462118 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0fac053850 con 0x7f0fb8102fc0 2024-05-22T22:59:08.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:08.145+0000 7f0fbe293700 1 -- 172.21.15.84:0/1358462118 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0fb802dbc0 con 0x7f0fb8102fc0 2024-05-22T22:59:08.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:08.146+0000 7f0fb57fa700 1 -- 172.21.15.84:0/1358462118 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f0fac09c4a0 con 0x7f0fb8102fc0 2024-05-22T22:59:08.144 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2024-05-22T22:59:08.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:08.148+0000 7f0fbe293700 1 -- 172.21.15.84:0/1358462118 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0fa00a2c20 msgr2=0x7f0fa00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:08.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:08.148+0000 7f0fbe293700 1 --2- 172.21.15.84:0/1358462118 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0fa00a2c20 0x7f0fa00a50e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8005fd0 tx=0x7f0fa8005f60 comp rx=0 tx=0).stop 2024-05-22T22:59:08.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:08.148+0000 7f0fbe293700 1 -- 172.21.15.84:0/1358462118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fb8102fc0 msgr2=0x7f0fb819ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:08.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:08.148+0000 7f0fbe293700 1 --2- 172.21.15.84:0/1358462118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fb8102fc0 0x7f0fb819ec70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00e970 tx=0x7f0fac00ed30 comp rx=0 tx=0).stop 2024-05-22T22:59:08.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:08.148+0000 7f0fbe293700 1 -- 172.21.15.84:0/1358462118 shutdown_connections 2024-05-22T22:59:08.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:08.148+0000 7f0fbe293700 1 --2- 172.21.15.84:0/1358462118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fb81076a0 0x7f0fb8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:08.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:08.148+0000 7f0fbe293700 1 --2- 172.21.15.84:0/1358462118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fb8102610 0x7f0fb819e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:08.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:08.148+0000 7f0fbe293700 1 --2- 172.21.15.84:0/1358462118 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0fa00a2c20 0x7f0fa00a50e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:08.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:08.148+0000 7f0fbe293700 1 --2- 172.21.15.84:0/1358462118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fb8102fc0 0x7f0fb819ec70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:08.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:08.148+0000 7f0fbe293700 1 -- 172.21.15.84:0/1358462118 >> 172.21.15.84:0/1358462118 conn(0x7f0fb80794c0 msgr2=0x7f0fb81004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:08.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:08.148+0000 7f0fbe293700 1 -- 172.21.15.84:0/1358462118 shutdown_connections 2024-05-22T22:59:08.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:08.148+0000 7f0fbe293700 1 -- 172.21.15.84:0/1358462118 wait complete. 2024-05-22T22:59:08.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:08 smithi096 ceph-mon[36392]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:08.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:08 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1358462118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T22:59:08.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:08 smithi084 ceph-mon[32186]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:08.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:08 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1358462118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T22:59:08.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:08 smithi204 ceph-mon[36211]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:08.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:08 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1358462118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T22:59:08.942 INFO:tasks.cephadm:Adding mds.a on smithi084 2024-05-22T22:59:08.943 INFO:tasks.cephadm:Adding mds.d on smithi084 2024-05-22T22:59:08.943 INFO:tasks.cephadm:Adding mds.g on smithi084 2024-05-22T22:59:08.943 INFO:tasks.cephadm:Adding mds.j on smithi084 2024-05-22T22:59:08.943 INFO:tasks.cephadm:Adding mds.b on smithi096 2024-05-22T22:59:08.943 INFO:tasks.cephadm:Adding mds.e on smithi096 2024-05-22T22:59:08.943 INFO:tasks.cephadm:Adding mds.h on smithi096 2024-05-22T22:59:08.943 INFO:tasks.cephadm:Adding mds.k on smithi096 2024-05-22T22:59:08.943 INFO:tasks.cephadm:Adding mds.c on smithi204 2024-05-22T22:59:08.943 INFO:tasks.cephadm:Adding mds.f on smithi204 2024-05-22T22:59:08.943 INFO:tasks.cephadm:Adding mds.i on smithi204 2024-05-22T22:59:08.943 INFO:tasks.cephadm:Adding mds.l on smithi204 2024-05-22T22:59:08.943 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph orch apply mds all '12;smithi084=a;smithi084=d;smithi084=g;smithi084=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l' 2024-05-22T22:59:09.190 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.c/config 2024-05-22T22:59:10.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:10 smithi204 ceph-mon[36211]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:10.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:10 smithi084 ceph-mon[32186]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:10.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.937+0000 7f0de0814700 1 -- 172.21.15.204:0/4081068975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ddc102e60 msgr2=0x7f0ddc109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:10.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.937+0000 7f0de0814700 1 --2- 172.21.15.204:0/4081068975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ddc102e60 0x7f0ddc109710 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0dd400b0a0 tx=0x7f0dd400b3b0 comp rx=0 tx=0).stop 2024-05-22T22:59:10.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.937+0000 7f0de0814700 1 -- 172.21.15.204:0/4081068975 shutdown_connections 2024-05-22T22:59:10.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.937+0000 7f0de0814700 1 --2- 172.21.15.204:0/4081068975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ddc102e60 0x7f0ddc109710 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:10.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.937+0000 7f0de0814700 1 --2- 172.21.15.204:0/4081068975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ddc1012c0 0x7f0ddc1016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:10.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.937+0000 7f0de0814700 1 --2- 172.21.15.204:0/4081068975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ddc1024c0 0x7f0ddc102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:10.938 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.937+0000 7f0de0814700 1 -- 172.21.15.204:0/4081068975 >> 172.21.15.204:0/4081068975 conn(0x7f0ddc0fc860 msgr2=0x7f0ddc0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:10.938 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.937+0000 7f0de0814700 1 -- 172.21.15.204:0/4081068975 shutdown_connections 2024-05-22T22:59:10.938 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.937+0000 7f0de0814700 1 -- 172.21.15.204:0/4081068975 wait complete. 2024-05-22T22:59:10.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.938+0000 7f0de0814700 1 Processor -- start 2024-05-22T22:59:10.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.938+0000 7f0de0814700 1 -- start start 2024-05-22T22:59:10.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.938+0000 7f0de0814700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ddc1012c0 0x7f0ddc105250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:10.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.938+0000 7f0de0814700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ddc1024c0 0x7f0ddc1038a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:10.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.938+0000 7f0de0814700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ddc102e60 0x7f0ddc103de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:10.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.938+0000 7f0de0814700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc10cea0 con 0x7f0ddc1012c0 2024-05-22T22:59:10.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.938+0000 7f0de0814700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc10cd20 con 0x7f0ddc102e60 2024-05-22T22:59:10.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.938+0000 7f0de0814700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc10d020 con 0x7f0ddc1024c0 2024-05-22T22:59:10.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.939+0000 7f0dd9d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ddc1024c0 0x7f0ddc1038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:10.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.939+0000 7f0dd9d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ddc1024c0 0x7f0ddc1038a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:45454/0 (socket says 172.21.15.204:45454) 2024-05-22T22:59:10.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.939+0000 7f0dd9d9b700 1 -- 172.21.15.204:0/2515971921 learned_addr learned my addr 172.21.15.204:0/2515971921 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-22T22:59:10.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.939+0000 7f0ddad9d700 1 --2- 172.21.15.204:0/2515971921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ddc102e60 0x7f0ddc103de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:10.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.939+0000 7f0dda59c700 1 --2- 172.21.15.204:0/2515971921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ddc1012c0 0x7f0ddc105250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:10.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.939+0000 7f0dd9d9b700 1 -- 172.21.15.204:0/2515971921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ddc102e60 msgr2=0x7f0ddc103de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:10.941 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.939+0000 7f0dd9d9b700 1 --2- 172.21.15.204:0/2515971921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ddc102e60 0x7f0ddc103de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:10.941 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.939+0000 7f0dd9d9b700 1 -- 172.21.15.204:0/2515971921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ddc1012c0 msgr2=0x7f0ddc105250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:10.941 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.939+0000 7f0dd9d9b700 1 --2- 172.21.15.204:0/2515971921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ddc1012c0 0x7f0ddc105250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:10.941 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.939+0000 7f0dd9d9b700 1 -- 172.21.15.204:0/2515971921 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ddc104690 con 0x7f0ddc1024c0 2024-05-22T22:59:10.941 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.939+0000 7f0dd9d9b700 1 --2- 172.21.15.204:0/2515971921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ddc1024c0 0x7f0ddc1038a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc00c970 tx=0x7f0dcc00cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:10.941 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.939+0000 7f0dcb7fe700 1 -- 172.21.15.204:0/2515971921 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc007aa0 con 0x7f0ddc1024c0 2024-05-22T22:59:10.941 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.939+0000 7f0de0814700 1 -- 172.21.15.204:0/2515971921 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ddc1a2660 con 0x7f0ddc1024c0 2024-05-22T22:59:10.941 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.939+0000 7f0dcb7fe700 1 -- 172.21.15.204:0/2515971921 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dcc004d10 con 0x7f0ddc1024c0 2024-05-22T22:59:10.941 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.939+0000 7f0dcb7fe700 1 -- 172.21.15.204:0/2515971921 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc005690 con 0x7f0ddc1024c0 2024-05-22T22:59:10.941 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.939+0000 7f0de0814700 1 -- 172.21.15.204:0/2515971921 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ddc1a2b70 con 0x7f0ddc1024c0 2024-05-22T22:59:10.941 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.940+0000 7f0de0814700 1 -- 172.21.15.204:0/2515971921 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ddc069050 con 0x7f0ddc1024c0 2024-05-22T22:59:10.942 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.941+0000 7f0dcb7fe700 1 -- 172.21.15.204:0/2515971921 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0dcc004750 con 0x7f0ddc1024c0 2024-05-22T22:59:10.943 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.942+0000 7f0dcb7fe700 1 --2- 172.21.15.204:0/2515971921 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0dc40a2c20 0x7f0dc40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:10.943 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.942+0000 7f0dcb7fe700 1 -- 172.21.15.204:0/2515971921 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0dcc0cede0 con 0x7f0ddc1024c0 2024-05-22T22:59:10.943 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.942+0000 7f0dda59c700 1 --2- 172.21.15.204:0/2515971921 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0dc40a2c20 0x7f0dc40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:10.943 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.942+0000 7f0dda59c700 1 --2- 172.21.15.204:0/2515971921 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0dc40a2c20 0x7f0dc40a50e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc102320 tx=0x7f0dd0005d70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:10.945 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:10.945+0000 7f0dcb7fe700 1 -- 172.21.15.204:0/2515971921 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0dcc09c270 con 0x7f0ddc1024c0 2024-05-22T22:59:11.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:10 smithi096 ceph-mon[36392]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:11.147 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:11.148+0000 7f0de0814700 1 -- 172.21.15.204:0/2515971921 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi084=a;smithi084=d;smithi084=g;smithi084=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}) -- 0x7f0ddc0fe910 con 0x7f0dc40a2c20 2024-05-22T22:59:11.156 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:11.156+0000 7f0dcb7fe700 1 -- 172.21.15.204:0/2515971921 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f0ddc0fe910 con 0x7f0dc40a2c20 2024-05-22T22:59:11.156 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mds.all update... 2024-05-22T22:59:11.158 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:11.159+0000 7f0de0814700 1 -- 172.21.15.204:0/2515971921 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0dc40a2c20 msgr2=0x7f0dc40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:11.159 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:11.159+0000 7f0de0814700 1 --2- 172.21.15.204:0/2515971921 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0dc40a2c20 0x7f0dc40a50e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc102320 tx=0x7f0dd0005d70 comp rx=0 tx=0).stop 2024-05-22T22:59:11.159 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:11.159+0000 7f0de0814700 1 -- 172.21.15.204:0/2515971921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ddc1024c0 msgr2=0x7f0ddc1038a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:11.159 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:11.159+0000 7f0de0814700 1 --2- 172.21.15.204:0/2515971921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ddc1024c0 0x7f0ddc1038a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc00c970 tx=0x7f0dcc00cd30 comp rx=0 tx=0).stop 2024-05-22T22:59:11.159 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:11.159+0000 7f0de0814700 1 -- 172.21.15.204:0/2515971921 shutdown_connections 2024-05-22T22:59:11.159 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:11.159+0000 7f0de0814700 1 --2- 172.21.15.204:0/2515971921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ddc102e60 0x7f0ddc103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:11.159 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:11.159+0000 7f0de0814700 1 --2- 172.21.15.204:0/2515971921 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0dc40a2c20 0x7f0dc40a50e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:11.159 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:11.159+0000 7f0de0814700 1 --2- 172.21.15.204:0/2515971921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ddc1012c0 0x7f0ddc105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:11.159 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:11.159+0000 7f0de0814700 1 --2- 172.21.15.204:0/2515971921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ddc1024c0 0x7f0ddc1038a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:11.159 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:11.159+0000 7f0de0814700 1 -- 172.21.15.204:0/2515971921 >> 172.21.15.204:0/2515971921 conn(0x7f0ddc0fc860 msgr2=0x7f0ddc0fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:11.159 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:11.159+0000 7f0de0814700 1 -- 172.21.15.204:0/2515971921 shutdown_connections 2024-05-22T22:59:11.159 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-22T22:59:11.159+0000 7f0de0814700 1 -- 172.21.15.204:0/2515971921 wait complete. 2024-05-22T22:59:11.991 DEBUG:teuthology.orchestra.run.smithi084:mds.a> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.a.service 2024-05-22T22:59:11.993 DEBUG:teuthology.orchestra.run.smithi084:mds.d> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.d.service 2024-05-22T22:59:12.037 DEBUG:teuthology.orchestra.run.smithi084:mds.g> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.g.service 2024-05-22T22:59:12.079 DEBUG:teuthology.orchestra.run.smithi084:mds.j> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.j.service 2024-05-22T22:59:12.122 DEBUG:teuthology.orchestra.run.smithi096:mds.b> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.b.service 2024-05-22T22:59:12.125 DEBUG:teuthology.orchestra.run.smithi096:mds.e> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.e.service 2024-05-22T22:59:12.168 DEBUG:teuthology.orchestra.run.smithi096:mds.h> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.h.service 2024-05-22T22:59:12.211 DEBUG:teuthology.orchestra.run.smithi096:mds.k> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.k.service 2024-05-22T22:59:12.232 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:12 smithi204 ceph-mon[36211]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi084=a;smithi084=d;smithi084=g;smithi084=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:59:12.232 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:12 smithi204 ceph-mon[36211]: Saving service mds.all spec with placement smithi084=a;smithi084=d;smithi084=g;smithi084=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-05-22T22:59:12.232 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:12 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:12.232 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:12 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:59:12.232 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:12 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:12.232 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:12 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:59:12.232 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:12 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:12.232 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:12 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:12.233 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:12 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:12.233 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:12 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:12.233 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:12 smithi204 ceph-mon[36211]: Deploying daemon mds.l on smithi204 2024-05-22T22:59:12.233 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:12 smithi204 ceph-mon[36211]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:12.254 DEBUG:teuthology.orchestra.run.smithi204:mds.c> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.c.service 2024-05-22T22:59:12.256 DEBUG:teuthology.orchestra.run.smithi204:mds.f> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.f.service 2024-05-22T22:59:12.299 DEBUG:teuthology.orchestra.run.smithi204:mds.i> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.i.service 2024-05-22T22:59:12.342 DEBUG:teuthology.orchestra.run.smithi204:mds.l> sudo journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.l.service 2024-05-22T22:59:12.373 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:12 smithi084 ceph-mon[32186]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi084=a;smithi084=d;smithi084=g;smithi084=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:59:12.373 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:12 smithi084 ceph-mon[32186]: Saving service mds.all spec with placement smithi084=a;smithi084=d;smithi084=g;smithi084=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-05-22T22:59:12.373 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:12 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:12.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:12 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:59:12.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:12 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:12.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:12 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:59:12.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:12 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:12.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:12 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:12.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:12 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:12.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:12 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:12.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:12 smithi084 ceph-mon[32186]: Deploying daemon mds.l on smithi204 2024-05-22T22:59:12.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:12 smithi084 ceph-mon[32186]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:12.385 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-22T22:59:12.385 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T22:59:12.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.513+0000 7fdfca590640 1 -- 172.21.15.84:0/884179633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfc4107640 msgr2=0x7fdfc4109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:12.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.513+0000 7fdfca590640 1 --2- 172.21.15.84:0/884179633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfc4107640 0x7fdfc4109a30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009ca0 tx=0x7fdfbc02a8c0 comp rx=0 tx=0).stop 2024-05-22T22:59:12.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.513+0000 7fdfca590640 1 -- 172.21.15.84:0/884179633 shutdown_connections 2024-05-22T22:59:12.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.513+0000 7fdfca590640 1 --2- 172.21.15.84:0/884179633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfc4107640 0x7fdfc4109a30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:12.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.513+0000 7fdfca590640 1 --2- 172.21.15.84:0/884179633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfc4104d10 0x7fdfc4107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:12.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.513+0000 7fdfca590640 1 --2- 172.21.15.84:0/884179633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfc41023e0 0x7fdfc41047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:12.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.513+0000 7fdfca590640 1 -- 172.21.15.84:0/884179633 >> 172.21.15.84:0/884179633 conn(0x7fdfc40fbd70 msgr2=0x7fdfc40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:12.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.513+0000 7fdfca590640 1 -- 172.21.15.84:0/884179633 shutdown_connections 2024-05-22T22:59:12.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.513+0000 7fdfca590640 1 -- 172.21.15.84:0/884179633 wait complete. 2024-05-22T22:59:12.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.513+0000 7fdfca590640 1 Processor -- start 2024-05-22T22:59:12.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.513+0000 7fdfca590640 1 -- start start 2024-05-22T22:59:12.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.513+0000 7fdfca590640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfc41023e0 0x7fdfc4197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:12.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.513+0000 7fdfca590640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfc4104d10 0x7fdfc4198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:12.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.513+0000 7fdfca590640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfc4107640 0x7fdfc419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.513+0000 7fdfca590640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc410d5b0 con 0x7fdfc4104d10 2024-05-22T22:59:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.513+0000 7fdfca590640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc410d430 con 0x7fdfc4107640 2024-05-22T22:59:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.513+0000 7fdfca590640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc410d730 con 0x7fdfc41023e0 2024-05-22T22:59:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.513+0000 7fdfbbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfc4104d10 0x7fdfc4198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.514+0000 7fdfbbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfc4104d10 0x7fdfc4198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35214/0 (socket says 172.21.15.84:35214) 2024-05-22T22:59:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.514+0000 7fdfbbfff640 1 -- 172.21.15.84:0/1917223427 learned_addr learned my addr 172.21.15.84:0/1917223427 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.514+0000 7fdfc3fff640 1 --2- 172.21.15.84:0/1917223427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfc41023e0 0x7fdfc4197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.514+0000 7fdfc8b2d640 1 --2- 172.21.15.84:0/1917223427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfc4107640 0x7fdfc419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.514+0000 7fdfbbfff640 1 -- 172.21.15.84:0/1917223427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfc41023e0 msgr2=0x7fdfc4197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.514+0000 7fdfbbfff640 1 --2- 172.21.15.84:0/1917223427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfc41023e0 0x7fdfc4197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.514+0000 7fdfbbfff640 1 -- 172.21.15.84:0/1917223427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfc4107640 msgr2=0x7fdfc419f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.514+0000 7fdfbbfff640 1 --2- 172.21.15.84:0/1917223427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfc4107640 0x7fdfc419f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.514+0000 7fdfbbfff640 1 -- 172.21.15.84:0/1917223427 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc419f810 con 0x7fdfc4104d10 2024-05-22T22:59:12.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.514+0000 7fdfc8b2d640 1 --2- 172.21.15.84:0/1917223427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfc4107640 0x7fdfc419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T22:59:12.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.514+0000 7fdfbbfff640 1 --2- 172.21.15.84:0/1917223427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfc4104d10 0x7fdfc4198130 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00b450 tx=0x7fdfac00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:12.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.514+0000 7fdfc1ffb640 1 -- 172.21.15.84:0/1917223427 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac015020 con 0x7fdfc4104d10 2024-05-22T22:59:12.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.514+0000 7fdfc1ffb640 1 -- 172.21.15.84:0/1917223427 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfac011d70 con 0x7fdfc4104d10 2024-05-22T22:59:12.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.514+0000 7fdfca590640 1 -- 172.21.15.84:0/1917223427 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc419fb00 con 0x7fdfc4104d10 2024-05-22T22:59:12.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.514+0000 7fdfc1ffb640 1 -- 172.21.15.84:0/1917223427 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac00f340 con 0x7fdfc4104d10 2024-05-22T22:59:12.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.514+0000 7fdfca590640 1 -- 172.21.15.84:0/1917223427 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc41a0050 con 0x7fdfc4104d10 2024-05-22T22:59:12.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.515+0000 7fdfca590640 1 -- 172.21.15.84:0/1917223427 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc40fd860 con 0x7fdfc4104d10 2024-05-22T22:59:12.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.516+0000 7fdfc1ffb640 1 -- 172.21.15.84:0/1917223427 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdfac018020 con 0x7fdfc4104d10 2024-05-22T22:59:12.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.517+0000 7fdfc1ffb640 1 --2- 172.21.15.84:0/1917223427 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fdfb00a2c10 0x7fdfb00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:12.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.517+0000 7fdfc3fff640 1 --2- 172.21.15.84:0/1917223427 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fdfb00a2c10 0x7fdfb00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:12.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.517+0000 7fdfc1ffb640 1 -- 172.21.15.84:0/1917223427 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fdfac0ce6a0 con 0x7fdfc4104d10 2024-05-22T22:59:12.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.517+0000 7fdfc3fff640 1 --2- 172.21.15.84:0/1917223427 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fdfb00a2c10 0x7fdfb00a50d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4005d00 tx=0x7fdfb4005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:12.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.521+0000 7fdfc1ffb640 1 -- 172.21.15.84:0/1917223427 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdfac01f020 con 0x7fdfc4104d10 2024-05-22T22:59:12.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:12 smithi096 ceph-mon[36392]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi084=a;smithi084=d;smithi084=g;smithi084=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:59:12.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:12 smithi096 ceph-mon[36392]: Saving service mds.all spec with placement smithi084=a;smithi084=d;smithi084=g;smithi084=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-05-22T22:59:12.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:12 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:12.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:12 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:59:12.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:12 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:12.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:12 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:59:12.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:12 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:12.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:12 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:12.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:12 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:12.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:12 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:12.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:12 smithi096 ceph-mon[36392]: Deploying daemon mds.l on smithi204 2024-05-22T22:59:12.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:12 smithi096 ceph-mon[36392]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:12.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.697+0000 7fdfca590640 1 -- 172.21.15.84:0/1917223427 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfc402d470 con 0x7fdfc4104d10 2024-05-22T22:59:12.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.698+0000 7fdfc1ffb640 1 -- 172.21.15.84:0/1917223427 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18431 (secure 0 0 0) 0x7fdfac00d040 con 0x7fdfc4104d10 2024-05-22T22:59:12.698 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:12.698 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":82,"fsid":"c1135a12-188d-11ef-bc9a-c7b262605968","created":"2024-05-22T22:51:49.878218+0000","modified":"2024-05-22T22:59:00.132491+0000","last_up_change":"2024-05-22T22:58:58.550278+0000","last_in_change":"2024-05-22T22:58:37.322139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T22:55:07.719394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762df065-fb4f-4c71-aab8-a62d2282c253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6803","nonce":1559945826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6805","nonce":1559945826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6809","nonce":1559945826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6807","nonce":1559945826}]},"public_addr":"172.21.15.84:6803/1559945826","cluster_addr":"172.21.15.84:6805/1559945826","heartbeat_back_addr":"172.21.15.84:6809/1559945826","heartbeat_front_addr":"172.21.15.84:6807/1559945826","state":["exists","up"]},{"osd":1,"uuid":"3fb2385d-6a34-49e3-88e1-116237beb346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6801","nonce":2996505353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6803","nonce":2996505353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6807","nonce":2996505353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6805","nonce":2996505353}]},"public_addr":"172.21.15.96:6801/2996505353","cluster_addr":"172.21.15.96:6803/2996505353","heartbeat_back_addr":"172.21.15.96:6807/2996505353","heartbeat_front_addr":"172.21.15.96:6805/2996505353","state":["exists","up"]},{"osd":2,"uuid":"e306e88e-a308-4585-ae11-c8af10b84038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6801","nonce":1499577897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6803","nonce":1499577897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6807","nonce":1499577897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6805","nonce":1499577897}]},"public_addr":"172.21.15.204:6801/1499577897","cluster_addr":"172.21.15.204:6803/1499577897","heartbeat_back_addr":"172.21.15.204:6807/1499577897","heartbeat_front_addr":"172.21.15.204:6805/1499577897","state":["exists","up"]},{"osd":3,"uuid":"d2a5fb97-6f60-4d45-9f18-447cae480974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6811","nonce":2869340648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6813","nonce":2869340648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6817","nonce":2869340648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6815","nonce":2869340648}]},"public_addr":"172.21.15.84:6811/2869340648","cluster_addr":"172.21.15.84:6813/2869340648","heartbeat_back_addr":"172.21.15.84:6817/2869340648","heartbeat_front_addr":"172.21.15.84:6815/2869340648","state":["exists","up"]},{"osd":4,"uuid":"4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6809","nonce":2554265648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6811","nonce":2554265648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6815","nonce":2554265648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6813","nonce":2554265648}]},"public_addr":"172.21.15.96:6809/2554265648","cluster_addr":"172.21.15.96:6811/2554265648","heartbeat_back_addr":"172.21.15.96:6815/2554265648","heartbeat_front_addr":"172.21.15.96:6813/2554265648","state":["exists","up"]},{"osd":5,"uuid":"7996bf5e-93a5-41b0-a9b1-99e7ef4514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6809","nonce":4075476986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6811","nonce":4075476986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6815","nonce":4075476986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6813","nonce":4075476986}]},"public_addr":"172.21.15.204:6809/4075476986","cluster_addr":"172.21.15.204:6811/4075476986","heartbeat_back_addr":"172.21.15.204:6815/4075476986","heartbeat_front_addr":"172.21.15.204:6813/4075476986","state":["exists","up"]},{"osd":6,"uuid":"f4b7c5de-1836-4d16-8603-3c517524d9fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6819","nonce":1806202865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6821","nonce":1806202865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6825","nonce":1806202865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6823","nonce":1806202865}]},"public_addr":"172.21.15.84:6819/1806202865","cluster_addr":"172.21.15.84:6821/1806202865","heartbeat_back_addr":"172.21.15.84:6825/1806202865","heartbeat_front_addr":"172.21.15.84:6823/1806202865","state":["exists","up"]},{"osd":7,"uuid":"69a18c43-5447-480d-9466-190830070c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6817","nonce":3195792774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6819","nonce":3195792774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6823","nonce":3195792774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6821","nonce":3195792774}]},"public_addr":"172.21.15.96:6817/3195792774","cluster_addr":"172.21.15.96:6819/3195792774","heartbeat_back_addr":"172.21.15.96:6823/3195792774","heartbeat_front_addr":"172.21.15.96:6821/3195792774","state":["exists","up"]},{"osd":8,"uuid":"2c5cadca-689a-441e-847c-ba370d400ef1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6817","nonce":606997927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6819","nonce":606997927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6823","nonce":606997927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6821","nonce":606997927}]},"public_addr":"172.21.15.204:6817/606997927","cluster_addr":"172.21.15.204:6819/606997927","heartbeat_back_addr":"172.21.15.204:6823/606997927","heartbeat_front_addr":"172.21.15.204:6821/606997927","state":["exists","up"]},{"osd":9,"uuid":"e3984fb2-7c00-4275-a10e-bca7c859f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6827","nonce":2786189012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6829","nonce":2786189012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6833","nonce":2786189012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6831","nonce":2786189012}]},"public_addr":"172.21.15.84:6827/2786189012","cluster_addr":"172.21.15.84:6829/2786189012","heartbeat_back_addr":"172.21.15.84:6833/2786189012","heartbeat_front_addr":"172.21.15.84:6831/2786189012","state":["exists","up"]},{"osd":10,"uuid":"b4938160-a808-4ded-91c2-d245c99fff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6825","nonce":1985398526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6827","nonce":1985398526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6831","nonce":1985398526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6829","nonce":1985398526}]},"public_addr":"172.21.15.96:6825/1985398526","cluster_addr":"172.21.15.96:6827/1985398526","heartbeat_back_addr":"172.21.15.96:6831/1985398526","heartbeat_front_addr":"172.21.15.96:6829/1985398526","state":["exists","up"]},{"osd":11,"uuid":"238922ef-d046-40cc-833d-25b7e7a523cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6825","nonce":3106116519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6827","nonce":3106116519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6831","nonce":3106116519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6829","nonce":3106116519}]},"public_addr":"172.21.15.204:6825/3106116519","cluster_addr":"172.21.15.204:6827/3106116519","heartbeat_back_addr":"172.21.15.204:6831/3106116519","heartbeat_front_addr":"172.21.15.204:6829/3106116519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:16.443480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:39.898133+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:03.664544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:28.838687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:53.829733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:19.269307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:45.166910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:10.771614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:36.678516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:03.150592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:29.327769+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:56.315938+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6801/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:6800/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/2766288608":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/950455999":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/4000289608":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1650467363":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1052746114":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6800/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6801/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/14610813":"2024-05-23T22:52:45.395791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T22:59:12.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.699+0000 7fdfca590640 1 -- 172.21.15.84:0/1917223427 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fdfb00a2c10 msgr2=0x7fdfb00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:12.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.699+0000 7fdfca590640 1 --2- 172.21.15.84:0/1917223427 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fdfb00a2c10 0x7fdfb00a50d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4005d00 tx=0x7fdfb4005c50 comp rx=0 tx=0).stop 2024-05-22T22:59:12.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.699+0000 7fdfca590640 1 -- 172.21.15.84:0/1917223427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfc4104d10 msgr2=0x7fdfc4198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:12.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.699+0000 7fdfca590640 1 --2- 172.21.15.84:0/1917223427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfc4104d10 0x7fdfc4198130 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00b450 tx=0x7fdfac00b910 comp rx=0 tx=0).stop 2024-05-22T22:59:12.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.700+0000 7fdfca590640 1 -- 172.21.15.84:0/1917223427 shutdown_connections 2024-05-22T22:59:12.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.700+0000 7fdfca590640 1 --2- 172.21.15.84:0/1917223427 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fdfb00a2c10 0x7fdfb00a50d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:12.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.700+0000 7fdfca590640 1 --2- 172.21.15.84:0/1917223427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfc4107640 0x7fdfc419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:12.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.700+0000 7fdfca590640 1 --2- 172.21.15.84:0/1917223427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfc4104d10 0x7fdfc4198130 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:12.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.700+0000 7fdfca590640 1 --2- 172.21.15.84:0/1917223427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfc41023e0 0x7fdfc4197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:12.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.700+0000 7fdfca590640 1 -- 172.21.15.84:0/1917223427 >> 172.21.15.84:0/1917223427 conn(0x7fdfc40fbd70 msgr2=0x7fdfc4102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:12.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.700+0000 7fdfca590640 1 -- 172.21.15.84:0/1917223427 shutdown_connections 2024-05-22T22:59:12.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.700+0000 7fdfca590640 1 -- 172.21.15.84:0/1917223427 wait complete. 2024-05-22T22:59:12.711 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T22:55:07.719394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T22:59:12.712 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T22:59:12.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.816+0000 7fda9237e640 1 -- 172.21.15.84:0/3082273960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda8c100a00 msgr2=0x7fda8c100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:12.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.816+0000 7fda9237e640 1 --2- 172.21.15.84:0/3082273960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda8c100a00 0x7fda8c100e80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fda74009900 tx=0x7fda7402a400 comp rx=0 tx=0).stop 2024-05-22T22:59:12.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.816+0000 7fda9237e640 1 -- 172.21.15.84:0/3082273960 shutdown_connections 2024-05-22T22:59:12.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.816+0000 7fda9237e640 1 --2- 172.21.15.84:0/3082273960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda8c1013c0 0x7fda8c108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:12.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.816+0000 7fda9237e640 1 --2- 172.21.15.84:0/3082273960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda8c100a00 0x7fda8c100e80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:12.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.816+0000 7fda9237e640 1 --2- 172.21.15.84:0/3082273960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda8c101e40 0x7fda8c102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:12.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.816+0000 7fda9237e640 1 -- 172.21.15.84:0/3082273960 >> 172.21.15.84:0/3082273960 conn(0x7fda8c0fb4b0 msgr2=0x7fda8c0fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:12.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.816+0000 7fda9237e640 1 -- 172.21.15.84:0/3082273960 shutdown_connections 2024-05-22T22:59:12.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.816+0000 7fda9237e640 1 -- 172.21.15.84:0/3082273960 wait complete. 2024-05-22T22:59:12.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.816+0000 7fda9237e640 1 Processor -- start 2024-05-22T22:59:12.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.816+0000 7fda9237e640 1 -- start start 2024-05-22T22:59:12.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.817+0000 7fda9237e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda8c100a00 0x7fda8c197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:12.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.817+0000 7fda9237e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda8c1013c0 0x7fda8c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:12.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.817+0000 7fda9237e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda8c101e40 0x7fda8c19f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:12.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.817+0000 7fda9237e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c10d5d0 con 0x7fda8c101e40 2024-05-22T22:59:12.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.817+0000 7fda9237e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c10d450 con 0x7fda8c100a00 2024-05-22T22:59:12.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.817+0000 7fda9237e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c10d750 con 0x7fda8c1013c0 2024-05-22T22:59:12.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.817+0000 7fda9091b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda8c101e40 0x7fda8c19f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:12.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.817+0000 7fda9091b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda8c101e40 0x7fda8c19f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35234/0 (socket says 172.21.15.84:35234) 2024-05-22T22:59:12.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.817+0000 7fda9091b640 1 -- 172.21.15.84:0/130323162 learned_addr learned my addr 172.21.15.84:0/130323162 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:12.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.817+0000 7fda83fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda8c1013c0 0x7fda8c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:12.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.817+0000 7fda8bfff640 1 --2- 172.21.15.84:0/130323162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda8c100a00 0x7fda8c197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:12.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.817+0000 7fda9091b640 1 -- 172.21.15.84:0/130323162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda8c1013c0 msgr2=0x7fda8c1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:12.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.817+0000 7fda9091b640 1 --2- 172.21.15.84:0/130323162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda8c1013c0 0x7fda8c1981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:12.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.817+0000 7fda9091b640 1 -- 172.21.15.84:0/130323162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda8c100a00 msgr2=0x7fda8c197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:12.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.817+0000 7fda9091b640 1 --2- 172.21.15.84:0/130323162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda8c100a00 0x7fda8c197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:12.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.817+0000 7fda9091b640 1 -- 172.21.15.84:0/130323162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8c19f8b0 con 0x7fda8c101e40 2024-05-22T22:59:12.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.817+0000 7fda9091b640 1 --2- 172.21.15.84:0/130323162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda8c101e40 0x7fda8c19f210 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fda8400b970 tx=0x7fda8400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:12.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.817+0000 7fda89ffb640 1 -- 172.21.15.84:0/130323162 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda84019070 con 0x7fda8c101e40 2024-05-22T22:59:12.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.817+0000 7fda89ffb640 1 -- 172.21.15.84:0/130323162 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda8400cb40 con 0x7fda8c101e40 2024-05-22T22:59:12.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.817+0000 7fda9237e640 1 -- 172.21.15.84:0/130323162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda8c19fba0 con 0x7fda8c101e40 2024-05-22T22:59:12.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.817+0000 7fda89ffb640 1 -- 172.21.15.84:0/130323162 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda84007650 con 0x7fda8c101e40 2024-05-22T22:59:12.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.817+0000 7fda9237e640 1 -- 172.21.15.84:0/130323162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda8c1a0140 con 0x7fda8c101e40 2024-05-22T22:59:12.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.818+0000 7fda9237e640 1 -- 172.21.15.84:0/130323162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda8c107b40 con 0x7fda8c101e40 2024-05-22T22:59:12.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.820+0000 7fda89ffb640 1 -- 172.21.15.84:0/130323162 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fda8401c070 con 0x7fda8c101e40 2024-05-22T22:59:12.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.821+0000 7fda89ffb640 1 --2- 172.21.15.84:0/130323162 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fda780a2c10 0x7fda780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:12.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.821+0000 7fda8bfff640 1 --2- 172.21.15.84:0/130323162 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fda780a2c10 0x7fda780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:12.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.821+0000 7fda89ffb640 1 -- 172.21.15.84:0/130323162 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fda840ce580 con 0x7fda8c101e40 2024-05-22T22:59:12.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.821+0000 7fda8bfff640 1 --2- 172.21.15.84:0/130323162 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fda780a2c10 0x7fda780a50d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fda7c005fd0 tx=0x7fda7c005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:12.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.824+0000 7fda89ffb640 1 -- 172.21.15.84:0/130323162 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fda8409b330 con 0x7fda8c101e40 2024-05-22T22:59:12.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.997+0000 7fda9237e640 1 -- 172.21.15.84:0/130323162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fda8c02d450 con 0x7fda8c101e40 2024-05-22T22:59:12.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:12.998+0000 7fda89ffb640 1 -- 172.21.15.84:0/130323162 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fda8409f630 con 0x7fda8c101e40 2024-05-22T22:59:12.998 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2024-05-22T22:59:12.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.000+0000 7fda9237e640 1 -- 172.21.15.84:0/130323162 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fda780a2c10 msgr2=0x7fda780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:13.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.000+0000 7fda9237e640 1 --2- 172.21.15.84:0/130323162 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fda780a2c10 0x7fda780a50d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fda7c005fd0 tx=0x7fda7c005e20 comp rx=0 tx=0).stop 2024-05-22T22:59:13.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.000+0000 7fda9237e640 1 -- 172.21.15.84:0/130323162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda8c101e40 msgr2=0x7fda8c19f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:13.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.000+0000 7fda9237e640 1 --2- 172.21.15.84:0/130323162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda8c101e40 0x7fda8c19f210 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fda8400b970 tx=0x7fda8400be30 comp rx=0 tx=0).stop 2024-05-22T22:59:13.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.000+0000 7fda9237e640 1 -- 172.21.15.84:0/130323162 shutdown_connections 2024-05-22T22:59:13.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.000+0000 7fda9237e640 1 --2- 172.21.15.84:0/130323162 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fda780a2c10 0x7fda780a50d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:13.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.000+0000 7fda9237e640 1 --2- 172.21.15.84:0/130323162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda8c101e40 0x7fda8c19f210 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:13.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.000+0000 7fda9237e640 1 --2- 172.21.15.84:0/130323162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda8c1013c0 0x7fda8c1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:13.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.000+0000 7fda9237e640 1 --2- 172.21.15.84:0/130323162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda8c100a00 0x7fda8c197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:13.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.000+0000 7fda9237e640 1 -- 172.21.15.84:0/130323162 >> 172.21.15.84:0/130323162 conn(0x7fda8c0fb4b0 msgr2=0x7fda8c104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:13.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.000+0000 7fda9237e640 1 -- 172.21.15.84:0/130323162 shutdown_connections 2024-05-22T22:59:13.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.000+0000 7fda9237e640 1 -- 172.21.15.84:0/130323162 wait complete. 2024-05-22T22:59:13.011 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T22:59:13.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.112+0000 7f18c22ee640 1 -- 172.21.15.84:0/2047926741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18bc1007b0 msgr2=0x7f18bc100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:13.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.112+0000 7f18c22ee640 1 --2- 172.21.15.84:0/2047926741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18bc1007b0 0x7f18bc100bd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009900 tx=0x7f18ac02a410 comp rx=0 tx=0).stop 2024-05-22T22:59:13.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.112+0000 7f18c22ee640 1 -- 172.21.15.84:0/2047926741 shutdown_connections 2024-05-22T22:59:13.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.112+0000 7f18c22ee640 1 --2- 172.21.15.84:0/2047926741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18bc102360 0x7f18bc108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:13.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.112+0000 7f18c22ee640 1 --2- 172.21.15.84:0/2047926741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18bc1019a0 0x7f18bc101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:13.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.112+0000 7f18c22ee640 1 --2- 172.21.15.84:0/2047926741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18bc1007b0 0x7f18bc100bd0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:13.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.112+0000 7f18c22ee640 1 -- 172.21.15.84:0/2047926741 >> 172.21.15.84:0/2047926741 conn(0x7f18bc0fbd20 msgr2=0x7f18bc0fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:13.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.112+0000 7f18c22ee640 1 -- 172.21.15.84:0/2047926741 shutdown_connections 2024-05-22T22:59:13.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.112+0000 7f18c22ee640 1 -- 172.21.15.84:0/2047926741 wait complete. 2024-05-22T22:59:13.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.112+0000 7f18c22ee640 1 Processor -- start 2024-05-22T22:59:13.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.112+0000 7f18c22ee640 1 -- start start 2024-05-22T22:59:13.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.113+0000 7f18c22ee640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18bc1007b0 0x7f18bc197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:13.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.113+0000 7f18c22ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18bc1019a0 0x7f18bc1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:13.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.113+0000 7f18c22ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18bc102360 0x7f18bc19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:13.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.113+0000 7f18c22ee640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc10d550 con 0x7f18bc102360 2024-05-22T22:59:13.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.113+0000 7f18c22ee640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc10d3d0 con 0x7f18bc1019a0 2024-05-22T22:59:13.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.113+0000 7f18c22ee640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc10d6d0 con 0x7f18bc1007b0 2024-05-22T22:59:13.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.113+0000 7f18c088b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18bc102360 0x7f18bc19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:13.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.113+0000 7f18c088b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18bc102360 0x7f18bc19f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35256/0 (socket says 172.21.15.84:35256) 2024-05-22T22:59:13.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.113+0000 7f18c088b640 1 -- 172.21.15.84:0/4095474028 learned_addr learned my addr 172.21.15.84:0/4095474028 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:13.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.113+0000 7f18b3fff640 1 --2- 172.21.15.84:0/4095474028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18bc1019a0 0x7f18bc1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:13.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.113+0000 7f18bbfff640 1 --2- 172.21.15.84:0/4095474028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18bc1007b0 0x7f18bc197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:13.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.113+0000 7f18c088b640 1 -- 172.21.15.84:0/4095474028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18bc1007b0 msgr2=0x7f18bc197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:13.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.113+0000 7f18c088b640 1 --2- 172.21.15.84:0/4095474028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18bc1007b0 0x7f18bc197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:13.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.113+0000 7f18c088b640 1 -- 172.21.15.84:0/4095474028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18bc1019a0 msgr2=0x7f18bc1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:13.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.113+0000 7f18c088b640 1 --2- 172.21.15.84:0/4095474028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18bc1019a0 0x7f18bc1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:13.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.113+0000 7f18c088b640 1 -- 172.21.15.84:0/4095474028 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18bc19f7a0 con 0x7f18bc102360 2024-05-22T22:59:13.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.113+0000 7f18b3fff640 1 --2- 172.21.15.84:0/4095474028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18bc1019a0 0x7f18bc1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T22:59:13.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.113+0000 7f18c088b640 1 --2- 172.21.15.84:0/4095474028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18bc102360 0x7f18bc19f130 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f18b400cce0 tx=0x7f18b400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:13.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.113+0000 7f18b9ffb640 1 -- 172.21.15.84:0/4095474028 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b400f450 con 0x7f18bc102360 2024-05-22T22:59:13.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.113+0000 7f18b9ffb640 1 -- 172.21.15.84:0/4095474028 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18b400fad0 con 0x7f18bc102360 2024-05-22T22:59:13.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.113+0000 7f18b9ffb640 1 -- 172.21.15.84:0/4095474028 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b4018c10 con 0x7f18bc102360 2024-05-22T22:59:13.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.113+0000 7f18c22ee640 1 -- 172.21.15.84:0/4095474028 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18bc19fa90 con 0x7f18bc102360 2024-05-22T22:59:13.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.113+0000 7f18c22ee640 1 -- 172.21.15.84:0/4095474028 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18bc19ff90 con 0x7f18bc102360 2024-05-22T22:59:13.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.114+0000 7f18c22ee640 1 -- 172.21.15.84:0/4095474028 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18bc107ad0 con 0x7f18bc102360 2024-05-22T22:59:13.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.115+0000 7f18b9ffb640 1 -- 172.21.15.84:0/4095474028 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f18b400f5f0 con 0x7f18bc102360 2024-05-22T22:59:13.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.116+0000 7f18b9ffb640 1 --2- 172.21.15.84:0/4095474028 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f18a80a2c10 0x7f18a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:13.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.116+0000 7f18bbfff640 1 --2- 172.21.15.84:0/4095474028 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f18a80a2c10 0x7f18a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:13.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.116+0000 7f18b9ffb640 1 -- 172.21.15.84:0/4095474028 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f18b40cf210 con 0x7f18bc102360 2024-05-22T22:59:13.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.117+0000 7f18bbfff640 1 --2- 172.21.15.84:0/4095474028 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f18a80a2c10 0x7f18a80a50d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f18ac006010 tx=0x7f18ac000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:13.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.120+0000 7f18b9ffb640 1 -- 172.21.15.84:0/4095474028 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f18b4014030 con 0x7f18bc102360 2024-05-22T22:59:13.335 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.336+0000 7f18c22ee640 1 -- 172.21.15.84:0/4095474028 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18bc02d200 con 0x7f18bc102360 2024-05-22T22:59:13.336 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.336+0000 7f18b9ffb640 1 -- 172.21.15.84:0/4095474028 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f18b409c610 con 0x7f18bc102360 2024-05-22T22:59:13.336 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:13.336 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-22T22:59:13.336 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 1 2024-05-22T22:59:13.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.338+0000 7f18c22ee640 1 -- 172.21.15.84:0/4095474028 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f18a80a2c10 msgr2=0x7f18a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:13.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.338+0000 7f18c22ee640 1 --2- 172.21.15.84:0/4095474028 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f18a80a2c10 0x7f18a80a50d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f18ac006010 tx=0x7f18ac000bc0 comp rx=0 tx=0).stop 2024-05-22T22:59:13.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.338+0000 7f18c22ee640 1 -- 172.21.15.84:0/4095474028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18bc102360 msgr2=0x7f18bc19f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:13.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.338+0000 7f18c22ee640 1 --2- 172.21.15.84:0/4095474028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18bc102360 0x7f18bc19f130 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f18b400cce0 tx=0x7f18b400a620 comp rx=0 tx=0).stop 2024-05-22T22:59:13.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.338+0000 7f18c22ee640 1 -- 172.21.15.84:0/4095474028 shutdown_connections 2024-05-22T22:59:13.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.338+0000 7f18c22ee640 1 --2- 172.21.15.84:0/4095474028 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f18a80a2c10 0x7f18a80a50d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:13.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.338+0000 7f18c22ee640 1 --2- 172.21.15.84:0/4095474028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18bc102360 0x7f18bc19f130 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:13.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.338+0000 7f18c22ee640 1 --2- 172.21.15.84:0/4095474028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18bc1019a0 0x7f18bc1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:13.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.338+0000 7f18c22ee640 1 --2- 172.21.15.84:0/4095474028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18bc1007b0 0x7f18bc197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:13.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.338+0000 7f18c22ee640 1 -- 172.21.15.84:0/4095474028 >> 172.21.15.84:0/4095474028 conn(0x7f18bc0fbd20 msgr2=0x7f18bc104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:13.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.338+0000 7f18c22ee640 1 -- 172.21.15.84:0/4095474028 shutdown_connections 2024-05-22T22:59:13.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:13.338+0000 7f18c22ee640 1 -- 172.21.15.84:0/4095474028 wait complete. 2024-05-22T22:59:13.348 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:13 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1917223427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T22:59:13.348 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:13 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/130323162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T22:59:13.520 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:13 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1917223427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T22:59:13.520 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:13 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/130323162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T22:59:13.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:13 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1917223427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T22:59:13.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:13 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/130323162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T22:59:13.909 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 22 22:59:13 smithi204 systemd[1]: Starting Ceph mds.l for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T22:59:14.197 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 22 22:59:13 smithi204 podman[66013]: 2024-05-22 22:59:13.835884418 +0000 UTC m=+0.016840870 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T22:59:14.197 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 22 22:59:13 smithi204 podman[66013]: 2024-05-22 22:59:13.953672255 +0000 UTC m=+0.134628686 container create ccde4371bca298a22ee0d67457a41509ed67e91a1b261d890259a252b3293fde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-2e1f762, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=) 2024-05-22T22:59:14.197 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 22 22:59:14 smithi204 podman[66013]: 2024-05-22 22:59:14.198553775 +0000 UTC m=+0.379510213 container init ccde4371bca298a22ee0d67457a41509ed67e91a1b261d890259a252b3293fde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-l, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-2e1f762, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521) 2024-05-22T22:59:14.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:14 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/4095474028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:14.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:14 smithi096 ceph-mon[36392]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:14.623 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:14 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/4095474028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:14.623 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:14 smithi084 ceph-mon[32186]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:14.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:14 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/4095474028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:14.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:14 smithi204 ceph-mon[36211]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:14.659 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 22 22:59:14 smithi204 podman[66013]: 2024-05-22 22:59:14.201314266 +0000 UTC m=+0.382270713 container start ccde4371bca298a22ee0d67457a41509ed67e91a1b261d890259a252b3293fde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-l, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-2e1f762, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-22T22:59:14.660 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 22 22:59:14 smithi204 ceph-mds[66033]: -- 172.21.15.204:0/3770742589 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562026353a40 con 0x562026326c00 2024-05-22T22:59:14.660 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 22 22:59:14 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-l[66029]: starting mds.l at 2024-05-22T22:59:14.660 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 22 22:59:14 smithi204 bash[66013]: ccde4371bca298a22ee0d67457a41509ed67e91a1b261d890259a252b3293fde 2024-05-22T22:59:14.660 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 22 22:59:14 smithi204 systemd[1]: Started Ceph mds.l for c1135a12-188d-11ef-bc9a-c7b262605968. 2024-05-22T22:59:15.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:15 smithi096 ceph-mon[36392]: mds.? [v2:172.21.15.204:6832/3932120262,v1:172.21.15.204:6833/3932120262] up:boot 2024-05-22T22:59:15.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:15 smithi096 ceph-mon[36392]: fsmap 1 up:standby 2024-05-22T22:59:15.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:15 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-22T22:59:15.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:15 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:15.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:15 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:15.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:15 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:15.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:15 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:15.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:15 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:15.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:15 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:15.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:15 smithi096 ceph-mon[36392]: Deploying daemon mds.k on smithi096 2024-05-22T22:59:15.350 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T22:59:15.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.465+0000 7f6a14142640 1 -- 172.21.15.84:0/1570669960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a0c101a90 msgr2=0x7f6a0c1083b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:15.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.465+0000 7f6a14142640 1 --2- 172.21.15.84:0/1570669960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a0c101a90 0x7f6a0c1083b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009ca0 tx=0x7f69fc02a8c0 comp rx=0 tx=0).stop 2024-05-22T22:59:15.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.465+0000 7f6a14142640 1 -- 172.21.15.84:0/1570669960 shutdown_connections 2024-05-22T22:59:15.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.465+0000 7f6a14142640 1 --2- 172.21.15.84:0/1570669960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a0c101a90 0x7f6a0c1083b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:15.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.465+0000 7f6a14142640 1 --2- 172.21.15.84:0/1570669960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a0c1010d0 0x7f6a0c101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:15.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.465+0000 7f6a14142640 1 --2- 172.21.15.84:0/1570669960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a0c0fff00 0x7f6a0c100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:15.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.465+0000 7f6a14142640 1 -- 172.21.15.84:0/1570669960 >> 172.21.15.84:0/1570669960 conn(0x7f6a0c0fb4e0 msgr2=0x7f6a0c0fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:15.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.465+0000 7f6a14142640 1 -- 172.21.15.84:0/1570669960 shutdown_connections 2024-05-22T22:59:15.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.465+0000 7f6a14142640 1 -- 172.21.15.84:0/1570669960 wait complete. 2024-05-22T22:59:15.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.466+0000 7f6a14142640 1 Processor -- start 2024-05-22T22:59:15.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.466+0000 7f6a14142640 1 -- start start 2024-05-22T22:59:15.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.466+0000 7f6a14142640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a0c0fff00 0x7f6a0c195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:15.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.466+0000 7f6a14142640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a0c1010d0 0x7f6a0c195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:15.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.466+0000 7f6a14142640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a0c101a90 0x7f6a0c19cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:15.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.466+0000 7f6a14142640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c10b340 con 0x7f6a0c101a90 2024-05-22T22:59:15.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.466+0000 7f6a14142640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c10b1c0 con 0x7f6a0c1010d0 2024-05-22T22:59:15.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.466+0000 7f6a14142640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c10b4c0 con 0x7f6a0c0fff00 2024-05-22T22:59:15.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.466+0000 7f6a126df640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a0c101a90 0x7f6a0c19cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:15.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.466+0000 7f6a116dd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a0c1010d0 0x7f6a0c195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:15.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.466+0000 7f6a126df640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a0c101a90 0x7f6a0c19cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35276/0 (socket says 172.21.15.84:35276) 2024-05-22T22:59:15.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.466+0000 7f6a126df640 1 -- 172.21.15.84:0/1929777559 learned_addr learned my addr 172.21.15.84:0/1929777559 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:15.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.466+0000 7f6a116dd640 1 --2- 172.21.15.84:0/1929777559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a0c1010d0 0x7f6a0c195ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:47472/0 (socket says 172.21.15.84:47472) 2024-05-22T22:59:15.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.466+0000 7f6a11ede640 1 --2- 172.21.15.84:0/1929777559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a0c0fff00 0x7f6a0c195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:15.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.466+0000 7f6a126df640 1 -- 172.21.15.84:0/1929777559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a0c0fff00 msgr2=0x7f6a0c195980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:15.470 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.466+0000 7f6a126df640 1 --2- 172.21.15.84:0/1929777559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a0c0fff00 0x7f6a0c195980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:15.470 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.466+0000 7f6a126df640 1 -- 172.21.15.84:0/1929777559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a0c1010d0 msgr2=0x7f6a0c195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:15.470 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.466+0000 7f6a126df640 1 --2- 172.21.15.84:0/1929777559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a0c1010d0 0x7f6a0c195ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:15.470 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.466+0000 7f6a126df640 1 -- 172.21.15.84:0/1929777559 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0c19d5a0 con 0x7f6a0c101a90 2024-05-22T22:59:15.470 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.466+0000 7f6a126df640 1 --2- 172.21.15.84:0/1929777559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a0c101a90 0x7f6a0c19cf30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f69fc005e30 tx=0x7f69fc031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:15.470 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.466+0000 7f6a0affd640 1 -- 172.21.15.84:0/1929777559 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc003c30 con 0x7f6a0c101a90 2024-05-22T22:59:15.470 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.467+0000 7f6a0affd640 1 -- 172.21.15.84:0/1929777559 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69fc003dd0 con 0x7f6a0c101a90 2024-05-22T22:59:15.470 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.467+0000 7f6a14142640 1 -- 172.21.15.84:0/1929777559 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a0c19d830 con 0x7f6a0c101a90 2024-05-22T22:59:15.470 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.467+0000 7f6a0affd640 1 -- 172.21.15.84:0/1929777559 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc02fde0 con 0x7f6a0c101a90 2024-05-22T22:59:15.470 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.467+0000 7f6a14142640 1 -- 172.21.15.84:0/1929777559 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a0c19dd80 con 0x7f6a0c101a90 2024-05-22T22:59:15.471 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.468+0000 7f6a14142640 1 -- 172.21.15.84:0/1929777559 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a0c107310 con 0x7f6a0c101a90 2024-05-22T22:59:15.471 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.469+0000 7f6a0affd640 1 -- 172.21.15.84:0/1929777559 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f69fc004050 con 0x7f6a0c101a90 2024-05-22T22:59:15.471 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.469+0000 7f6a0affd640 1 --2- 172.21.15.84:0/1929777559 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f69f40a2b90 0x7f69f40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:15.471 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.469+0000 7f6a11ede640 1 --2- 172.21.15.84:0/1929777559 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f69f40a2b90 0x7f69f40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:15.471 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.469+0000 7f6a0affd640 1 -- 172.21.15.84:0/1929777559 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f69fc0eff10 con 0x7f6a0c101a90 2024-05-22T22:59:15.471 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.470+0000 7f6a11ede640 1 --2- 172.21.15.84:0/1929777559 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f69f40a2b90 0x7f69f40a5050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6a04005fd0 tx=0x7f6a04010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:15.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.474+0000 7f6a0affd640 1 -- 172.21.15.84:0/1929777559 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f69fc0bd310 con 0x7f6a0c101a90 2024-05-22T22:59:15.623 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:15 smithi084 ceph-mon[32186]: mds.? [v2:172.21.15.204:6832/3932120262,v1:172.21.15.204:6833/3932120262] up:boot 2024-05-22T22:59:15.623 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:15 smithi084 ceph-mon[32186]: fsmap 1 up:standby 2024-05-22T22:59:15.623 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:15 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-22T22:59:15.623 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:15 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:15.623 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:15 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:15.623 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:15 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:15.623 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:15 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:15.624 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:15 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:15.624 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:15 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:15.624 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:15 smithi084 ceph-mon[32186]: Deploying daemon mds.k on smithi096 2024-05-22T22:59:15.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:15 smithi204 ceph-mon[36211]: mds.? [v2:172.21.15.204:6832/3932120262,v1:172.21.15.204:6833/3932120262] up:boot 2024-05-22T22:59:15.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:15 smithi204 ceph-mon[36211]: fsmap 1 up:standby 2024-05-22T22:59:15.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:15 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-22T22:59:15.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:15 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:15.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:15 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:15.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:15 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:15.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:15 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:15.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:15 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:15.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:15 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:15.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:15 smithi204 ceph-mon[36211]: Deploying daemon mds.k on smithi096 2024-05-22T22:59:15.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.685+0000 7f6a14142640 1 -- 172.21.15.84:0/1929777559 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a0c062470 con 0x7f6a0c101a90 2024-05-22T22:59:15.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.686+0000 7f6a0affd640 1 -- 172.21.15.84:0/1929777559 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f69fc0bcc10 con 0x7f6a0c101a90 2024-05-22T22:59:15.686 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:15.686 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-22T22:59:15.687 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 2 2024-05-22T22:59:15.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.688+0000 7f6a14142640 1 -- 172.21.15.84:0/1929777559 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f69f40a2b90 msgr2=0x7f69f40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:15.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.688+0000 7f6a14142640 1 --2- 172.21.15.84:0/1929777559 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f69f40a2b90 0x7f69f40a5050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6a04005fd0 tx=0x7f6a04010040 comp rx=0 tx=0).stop 2024-05-22T22:59:15.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.688+0000 7f6a14142640 1 -- 172.21.15.84:0/1929777559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a0c101a90 msgr2=0x7f6a0c19cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:15.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.688+0000 7f6a14142640 1 --2- 172.21.15.84:0/1929777559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a0c101a90 0x7f6a0c19cf30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f69fc005e30 tx=0x7f69fc031ee0 comp rx=0 tx=0).stop 2024-05-22T22:59:15.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.688+0000 7f6a14142640 1 -- 172.21.15.84:0/1929777559 shutdown_connections 2024-05-22T22:59:15.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.688+0000 7f6a14142640 1 --2- 172.21.15.84:0/1929777559 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f69f40a2b90 0x7f69f40a5050 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:15.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.688+0000 7f6a14142640 1 --2- 172.21.15.84:0/1929777559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a0c101a90 0x7f6a0c19cf30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:15.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.688+0000 7f6a14142640 1 --2- 172.21.15.84:0/1929777559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a0c1010d0 0x7f6a0c195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:15.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.688+0000 7f6a14142640 1 --2- 172.21.15.84:0/1929777559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a0c0fff00 0x7f6a0c195980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:15.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.688+0000 7f6a14142640 1 -- 172.21.15.84:0/1929777559 >> 172.21.15.84:0/1929777559 conn(0x7f6a0c0fb4e0 msgr2=0x7f6a0c1044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:15.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.688+0000 7f6a14142640 1 -- 172.21.15.84:0/1929777559 shutdown_connections 2024-05-22T22:59:15.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:15.688+0000 7f6a14142640 1 -- 172.21.15.84:0/1929777559 wait complete. 2024-05-22T22:59:16.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:16 smithi096 ceph-mon[36392]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:16.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:16 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1929777559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:16.623 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:16 smithi084 ceph-mon[32186]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:16.623 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:16 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1929777559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:16.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:16 smithi204 ceph-mon[36211]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:16.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:16 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1929777559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:16.798 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 22 22:59:16 smithi096 systemd[1]: Starting Ceph mds.k for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T22:59:17.179 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 22 22:59:16 smithi096 podman[65915]: 2024-05-22 22:59:16.831636152 +0000 UTC m=+0.016681094 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T22:59:17.179 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 22 22:59:16 smithi096 podman[65915]: 2024-05-22 22:59:16.935921614 +0000 UTC m=+0.120966542 container create a9999061cbc9aff3b5d9d944ddf7cc0ee37a9959bb6ef2b1df889ad3d71ba71b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-k, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-2e1f762, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240521) 2024-05-22T22:59:17.179 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 22 22:59:17 smithi096 podman[65915]: 2024-05-22 22:59:17.180538124 +0000 UTC m=+0.365583041 container init a9999061cbc9aff3b5d9d944ddf7cc0ee37a9959bb6ef2b1df889ad3d71ba71b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, RELEASE=squid-2e1f762) 2024-05-22T22:59:17.548 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 22 22:59:17 smithi096 podman[65915]: 2024-05-22 22:59:17.183425076 +0000 UTC m=+0.368470011 container start a9999061cbc9aff3b5d9d944ddf7cc0ee37a9959bb6ef2b1df889ad3d71ba71b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-2e1f762, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, io.buildah.version=1.33.5) 2024-05-22T22:59:17.548 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 22 22:59:17 smithi096 ceph-mds[65934]: -- 172.21.15.96:0/397526670 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56145ff17a40 con 0x56145feea800 2024-05-22T22:59:17.548 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 22 22:59:17 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-k[65930]: starting mds.k at 2024-05-22T22:59:17.548 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 22 22:59:17 smithi096 bash[65915]: a9999061cbc9aff3b5d9d944ddf7cc0ee37a9959bb6ef2b1df889ad3d71ba71b 2024-05-22T22:59:17.548 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 22 22:59:17 smithi096 systemd[1]: Started Ceph mds.k for c1135a12-188d-11ef-bc9a-c7b262605968. 2024-05-22T22:59:17.700 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T22:59:17.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.803+0000 7f0caa361640 1 -- 172.21.15.84:0/637047256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ca4104cb0 msgr2=0x7f0ca41070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:17.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.803+0000 7f0caa361640 1 --2- 172.21.15.84:0/637047256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ca4104cb0 0x7f0ca41070a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009900 tx=0x7f0c9802a410 comp rx=0 tx=0).stop 2024-05-22T22:59:17.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.803+0000 7f0caa361640 1 -- 172.21.15.84:0/637047256 shutdown_connections 2024-05-22T22:59:17.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.803+0000 7f0caa361640 1 --2- 172.21.15.84:0/637047256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ca41075e0 0x7f0ca41099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:17.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.803+0000 7f0caa361640 1 --2- 172.21.15.84:0/637047256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ca4104cb0 0x7f0ca41070a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:17.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.803+0000 7f0caa361640 1 --2- 172.21.15.84:0/637047256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ca4102380 0x7f0ca4104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:17.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.803+0000 7f0caa361640 1 -- 172.21.15.84:0/637047256 >> 172.21.15.84:0/637047256 conn(0x7f0ca40fbd50 msgr2=0x7f0ca40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:17.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.803+0000 7f0caa361640 1 -- 172.21.15.84:0/637047256 shutdown_connections 2024-05-22T22:59:17.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.803+0000 7f0caa361640 1 -- 172.21.15.84:0/637047256 wait complete. 2024-05-22T22:59:17.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.804+0000 7f0caa361640 1 Processor -- start 2024-05-22T22:59:17.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.804+0000 7f0caa361640 1 -- start start 2024-05-22T22:59:17.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.804+0000 7f0caa361640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ca4102380 0x7f0ca4197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:17.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.804+0000 7f0caa361640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ca4104cb0 0x7f0ca41981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:17.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.804+0000 7f0caa361640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ca41075e0 0x7f0ca419f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:17.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.804+0000 7f0caa361640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca410d540 con 0x7f0ca41075e0 2024-05-22T22:59:17.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.804+0000 7f0caa361640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca410d3c0 con 0x7f0ca4102380 2024-05-22T22:59:17.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.804+0000 7f0caa361640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca410d6c0 con 0x7f0ca4104cb0 2024-05-22T22:59:17.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.804+0000 7f0ca88fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ca41075e0 0x7f0ca419f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:17.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.804+0000 7f0ca3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ca4102380 0x7f0ca4197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:17.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.804+0000 7f0ca88fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ca41075e0 0x7f0ca419f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35280/0 (socket says 172.21.15.84:35280) 2024-05-22T22:59:17.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.804+0000 7f0ca88fe640 1 -- 172.21.15.84:0/693409415 learned_addr learned my addr 172.21.15.84:0/693409415 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:17.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.804+0000 7f0ca37fe640 1 --2- 172.21.15.84:0/693409415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ca4104cb0 0x7f0ca41981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:17.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.805+0000 7f0ca88fe640 1 -- 172.21.15.84:0/693409415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ca4104cb0 msgr2=0x7f0ca41981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:17.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.805+0000 7f0ca88fe640 1 --2- 172.21.15.84:0/693409415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ca4104cb0 0x7f0ca41981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:17.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.805+0000 7f0ca88fe640 1 -- 172.21.15.84:0/693409415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ca4102380 msgr2=0x7f0ca4197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:17.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.805+0000 7f0ca88fe640 1 --2- 172.21.15.84:0/693409415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ca4102380 0x7f0ca4197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:17.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.805+0000 7f0ca88fe640 1 -- 172.21.15.84:0/693409415 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca419f870 con 0x7f0ca41075e0 2024-05-22T22:59:17.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.805+0000 7f0ca3fff640 1 --2- 172.21.15.84:0/693409415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ca4102380 0x7f0ca4197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T22:59:17.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.805+0000 7f0ca88fe640 1 --2- 172.21.15.84:0/693409415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ca41075e0 0x7f0ca419f1d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c00eb10 tx=0x7f0c9c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:17.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.805+0000 7f0ca17fa640 1 -- 172.21.15.84:0/693409415 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c004b20 con 0x7f0ca41075e0 2024-05-22T22:59:17.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.805+0000 7f0ca17fa640 1 -- 172.21.15.84:0/693409415 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c9c004cc0 con 0x7f0ca41075e0 2024-05-22T22:59:17.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.805+0000 7f0caa361640 1 -- 172.21.15.84:0/693409415 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca419fb60 con 0x7f0ca41075e0 2024-05-22T22:59:17.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.805+0000 7f0ca17fa640 1 -- 172.21.15.84:0/693409415 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c010760 con 0x7f0ca41075e0 2024-05-22T22:59:17.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.805+0000 7f0caa361640 1 -- 172.21.15.84:0/693409415 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca41a00b0 con 0x7f0ca41075e0 2024-05-22T22:59:17.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.806+0000 7f0caa361640 1 -- 172.21.15.84:0/693409415 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ca40fd840 con 0x7f0ca41075e0 2024-05-22T22:59:17.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.808+0000 7f0ca17fa640 1 -- 172.21.15.84:0/693409415 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0c9c005000 con 0x7f0ca41075e0 2024-05-22T22:59:17.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.808+0000 7f0ca17fa640 1 --2- 172.21.15.84:0/693409415 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0c8c0a2b90 0x7f0c8c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:17.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.809+0000 7f0ca17fa640 1 -- 172.21.15.84:0/693409415 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0c9c0cf630 con 0x7f0ca41075e0 2024-05-22T22:59:17.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.809+0000 7f0ca3fff640 1 --2- 172.21.15.84:0/693409415 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0c8c0a2b90 0x7f0c8c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:17.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.809+0000 7f0ca3fff640 1 --2- 172.21.15.84:0/693409415 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0c8c0a2b90 0x7f0c8c0a5050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4199130 tx=0x7f0c9400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:17.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:17.813+0000 7f0ca17fa640 1 -- 172.21.15.84:0/693409415 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0c9c09ca30 con 0x7f0ca41075e0 2024-05-22T22:59:18.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:18.044+0000 7f0caa361640 1 -- 172.21.15.84:0/693409415 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ca402d2a0 con 0x7f0ca41075e0 2024-05-22T22:59:18.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:18.045+0000 7f0ca17fa640 1 -- 172.21.15.84:0/693409415 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f0c9c09c330 con 0x7f0ca41075e0 2024-05-22T22:59:18.045 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:18.045 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-22T22:59:18.045 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 3 2024-05-22T22:59:18.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:18.047+0000 7f0caa361640 1 -- 172.21.15.84:0/693409415 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0c8c0a2b90 msgr2=0x7f0c8c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:18.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:18.047+0000 7f0caa361640 1 --2- 172.21.15.84:0/693409415 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0c8c0a2b90 0x7f0c8c0a5050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4199130 tx=0x7f0c9400b390 comp rx=0 tx=0).stop 2024-05-22T22:59:18.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:18.047+0000 7f0caa361640 1 -- 172.21.15.84:0/693409415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ca41075e0 msgr2=0x7f0ca419f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:18.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:18.047+0000 7f0caa361640 1 --2- 172.21.15.84:0/693409415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ca41075e0 0x7f0ca419f1d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c00eb10 tx=0x7f0c9c00c2d0 comp rx=0 tx=0).stop 2024-05-22T22:59:18.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:18.047+0000 7f0caa361640 1 -- 172.21.15.84:0/693409415 shutdown_connections 2024-05-22T22:59:18.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:18.047+0000 7f0caa361640 1 --2- 172.21.15.84:0/693409415 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0c8c0a2b90 0x7f0c8c0a5050 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:18.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:18.047+0000 7f0caa361640 1 --2- 172.21.15.84:0/693409415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ca41075e0 0x7f0ca419f1d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:18.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:18.047+0000 7f0caa361640 1 --2- 172.21.15.84:0/693409415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ca4104cb0 0x7f0ca41981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:18.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:18.047+0000 7f0caa361640 1 --2- 172.21.15.84:0/693409415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ca4102380 0x7f0ca4197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:18.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:18.047+0000 7f0caa361640 1 -- 172.21.15.84:0/693409415 >> 172.21.15.84:0/693409415 conn(0x7f0ca40fbd50 msgr2=0x7f0ca4102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:18.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:18.047+0000 7f0caa361640 1 -- 172.21.15.84:0/693409415 shutdown_connections 2024-05-22T22:59:18.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:18.047+0000 7f0caa361640 1 -- 172.21.15.84:0/693409415 wait complete. 2024-05-22T22:59:18.270 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:18 smithi084 ceph-mon[32186]: mds.? [v2:172.21.15.96:6832/3792500684,v1:172.21.15.96:6833/3792500684] up:boot 2024-05-22T22:59:18.270 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:18 smithi084 ceph-mon[32186]: fsmap 2 up:standby 2024-05-22T22:59:18.270 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:18 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-22T22:59:18.270 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:18 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:18.270 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:18 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:18.270 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:18 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:18.270 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:18 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:18.271 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:18 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:18.271 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:18 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:18.271 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:18 smithi084 ceph-mon[32186]: Deploying daemon mds.j on smithi084 2024-05-22T22:59:18.271 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:18 smithi084 ceph-mon[32186]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:18.271 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:18 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/693409415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:18.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:18 smithi096 ceph-mon[36392]: mds.? [v2:172.21.15.96:6832/3792500684,v1:172.21.15.96:6833/3792500684] up:boot 2024-05-22T22:59:18.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:18 smithi096 ceph-mon[36392]: fsmap 2 up:standby 2024-05-22T22:59:18.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:18 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-22T22:59:18.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:18 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:18.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:18 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:18.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:18 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:18.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:18 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:18.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:18 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:18.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:18 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:18.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:18 smithi096 ceph-mon[36392]: Deploying daemon mds.j on smithi084 2024-05-22T22:59:18.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:18 smithi096 ceph-mon[36392]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:18.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:18 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/693409415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:18.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:18 smithi204 ceph-mon[36211]: mds.? [v2:172.21.15.96:6832/3792500684,v1:172.21.15.96:6833/3792500684] up:boot 2024-05-22T22:59:18.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:18 smithi204 ceph-mon[36211]: fsmap 2 up:standby 2024-05-22T22:59:18.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:18 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-22T22:59:18.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:18 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:18.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:18 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:18.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:18 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:18.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:18 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:18.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:18 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:18.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:18 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:18.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:18 smithi204 ceph-mon[36211]: Deploying daemon mds.j on smithi084 2024-05-22T22:59:18.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:18 smithi204 ceph-mon[36211]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:18.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:18 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/693409415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:19.824 INFO:journalctl@ceph.mds.j.smithi084.stdout:May 22 22:59:19 smithi084 systemd[1]: Starting Ceph mds.j for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T22:59:20.059 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T22:59:20.083 INFO:journalctl@ceph.mds.j.smithi084.stdout:May 22 22:59:19 smithi084 podman[69981]: 2024-05-22 22:59:19.727097972 +0000 UTC m=+0.017283386 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T22:59:20.083 INFO:journalctl@ceph.mds.j.smithi084.stdout:May 22 22:59:19 smithi084 podman[69981]: 2024-05-22 22:59:19.844739636 +0000 UTC m=+0.134925036 container create af78cff964907f6a461b2ae3ae823a71158c78a064d7f0d06817fafc008fc509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-j, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, RELEASE=squid-2e1f762, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-22T22:59:20.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.166+0000 7f9c3edc4640 1 -- 172.21.15.84:0/1958421614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c38102ab0 msgr2=0x7f9c38102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.166+0000 7f9c3edc4640 1 --2- 172.21.15.84:0/1958421614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c38102ab0 0x7f9c38102f30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9c28009900 tx=0x7f9c2802a410 comp rx=0 tx=0).stop 2024-05-22T22:59:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.166+0000 7f9c3edc4640 1 -- 172.21.15.84:0/1958421614 shutdown_connections 2024-05-22T22:59:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.166+0000 7f9c3edc4640 1 --2- 172.21.15.84:0/1958421614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c38103470 0x7f9c38109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.166+0000 7f9c3edc4640 1 --2- 172.21.15.84:0/1958421614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c38102ab0 0x7f9c38102f30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.166+0000 7f9c3edc4640 1 --2- 172.21.15.84:0/1958421614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c381018c0 0x7f9c38101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.166+0000 7f9c3edc4640 1 -- 172.21.15.84:0/1958421614 >> 172.21.15.84:0/1958421614 conn(0x7f9c380fce50 msgr2=0x7f9c380ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.167+0000 7f9c3edc4640 1 -- 172.21.15.84:0/1958421614 shutdown_connections 2024-05-22T22:59:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.167+0000 7f9c3edc4640 1 -- 172.21.15.84:0/1958421614 wait complete. 2024-05-22T22:59:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.167+0000 7f9c3edc4640 1 Processor -- start 2024-05-22T22:59:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.167+0000 7f9c3edc4640 1 -- start start 2024-05-22T22:59:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.167+0000 7f9c3edc4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c381018c0 0x7f9c38075620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.167+0000 7f9c3edc4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c38102ab0 0x7f9c38073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.167+0000 7f9c3edc4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c38103470 0x7f9c380741a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:20.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.167+0000 7f9c3edc4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3810d540 con 0x7f9c381018c0 2024-05-22T22:59:20.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.167+0000 7f9c3edc4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3810d3c0 con 0x7f9c38103470 2024-05-22T22:59:20.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.167+0000 7f9c3edc4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3810d6c0 con 0x7f9c38102ab0 2024-05-22T22:59:20.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.167+0000 7f9c3cb60640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c381018c0 0x7f9c38075620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:20.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.167+0000 7f9c3cb60640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c381018c0 0x7f9c38075620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35298/0 (socket says 172.21.15.84:35298) 2024-05-22T22:59:20.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.167+0000 7f9c3cb60640 1 -- 172.21.15.84:0/132526562 learned_addr learned my addr 172.21.15.84:0/132526562 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:20.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.167+0000 7f9c3d361640 1 --2- 172.21.15.84:0/132526562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c38103470 0x7f9c380741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:20.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.168+0000 7f9c3cb60640 1 -- 172.21.15.84:0/132526562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c38102ab0 msgr2=0x7f9c38073c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:20.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.168+0000 7f9c3cb60640 1 --2- 172.21.15.84:0/132526562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c38102ab0 0x7f9c38073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:20.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.168+0000 7f9c3cb60640 1 -- 172.21.15.84:0/132526562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c38103470 msgr2=0x7f9c380741a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:20.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.168+0000 7f9c3cb60640 1 --2- 172.21.15.84:0/132526562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c38103470 0x7f9c380741a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:20.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.168+0000 7f9c3cb60640 1 -- 172.21.15.84:0/132526562 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c38074a30 con 0x7f9c381018c0 2024-05-22T22:59:20.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.168+0000 7f9c3cb60640 1 --2- 172.21.15.84:0/132526562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c381018c0 0x7f9c38075620 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400d950 tx=0x7f9c3400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:20.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.168+0000 7f9c2dffb640 1 -- 172.21.15.84:0/132526562 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c34017020 con 0x7f9c381018c0 2024-05-22T22:59:20.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.168+0000 7f9c3edc4640 1 -- 172.21.15.84:0/132526562 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c381a1ce0 con 0x7f9c381018c0 2024-05-22T22:59:20.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.168+0000 7f9c2dffb640 1 -- 172.21.15.84:0/132526562 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c34009c30 con 0x7f9c381018c0 2024-05-22T22:59:20.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.168+0000 7f9c2dffb640 1 -- 172.21.15.84:0/132526562 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3400f740 con 0x7f9c381018c0 2024-05-22T22:59:20.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.168+0000 7f9c3edc4640 1 -- 172.21.15.84:0/132526562 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c381a21d0 con 0x7f9c381018c0 2024-05-22T22:59:20.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.169+0000 7f9c3edc4640 1 -- 172.21.15.84:0/132526562 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c38108be0 con 0x7f9c381018c0 2024-05-22T22:59:20.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.170+0000 7f9c2dffb640 1 -- 172.21.15.84:0/132526562 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9c3401a020 con 0x7f9c381018c0 2024-05-22T22:59:20.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.171+0000 7f9c2dffb640 1 --2- 172.21.15.84:0/132526562 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9c200a2b90 0x7f9c200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:20.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.171+0000 7f9c2ffff640 1 --2- 172.21.15.84:0/132526562 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9c200a2b90 0x7f9c200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:20.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.171+0000 7f9c2dffb640 1 -- 172.21.15.84:0/132526562 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9c340ce710 con 0x7f9c381018c0 2024-05-22T22:59:20.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.171+0000 7f9c2ffff640 1 --2- 172.21.15.84:0/132526562 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9c200a2b90 0x7f9c200a5050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9c280096c0 tx=0x7f9c28031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:20.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.175+0000 7f9c2dffb640 1 -- 172.21.15.84:0/132526562 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9c34021070 con 0x7f9c381018c0 2024-05-22T22:59:20.373 INFO:journalctl@ceph.mds.j.smithi084.stdout:May 22 22:59:20 smithi084 podman[69981]: 2024-05-22 22:59:20.08892575 +0000 UTC m=+0.379111159 container init af78cff964907f6a461b2ae3ae823a71158c78a064d7f0d06817fafc008fc509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-j, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240521, RELEASE=squid-2e1f762, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-22T22:59:20.373 INFO:journalctl@ceph.mds.j.smithi084.stdout:May 22 22:59:20 smithi084 podman[69981]: 2024-05-22 22:59:20.092588111 +0000 UTC m=+0.382773514 container start af78cff964907f6a461b2ae3ae823a71158c78a064d7f0d06817fafc008fc509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-2e1f762, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-05-22T22:59:20.374 INFO:journalctl@ceph.mds.j.smithi084.stdout:May 22 22:59:20 smithi084 ceph-mds[70020]: -- 172.21.15.84:0/1265017951 <== mon.0 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5591dffc5a40 con 0x5591dff98c00 2024-05-22T22:59:20.374 INFO:journalctl@ceph.mds.j.smithi084.stdout:May 22 22:59:20 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-j[69996]: starting mds.j at 2024-05-22T22:59:20.374 INFO:journalctl@ceph.mds.j.smithi084.stdout:May 22 22:59:20 smithi084 bash[69981]: af78cff964907f6a461b2ae3ae823a71158c78a064d7f0d06817fafc008fc509 2024-05-22T22:59:20.374 INFO:journalctl@ceph.mds.j.smithi084.stdout:May 22 22:59:20 smithi084 systemd[1]: Started Ceph mds.j for c1135a12-188d-11ef-bc9a-c7b262605968. 2024-05-22T22:59:20.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.399+0000 7f9c3edc4640 1 -- 172.21.15.84:0/132526562 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c3802d200 con 0x7f9c381018c0 2024-05-22T22:59:20.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.400+0000 7f9c2dffb640 1 -- 172.21.15.84:0/132526562 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f9c3802d200 con 0x7f9c381018c0 2024-05-22T22:59:20.400 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:20.400 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-22T22:59:20.400 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 3 2024-05-22T22:59:20.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.402+0000 7f9c3edc4640 1 -- 172.21.15.84:0/132526562 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9c200a2b90 msgr2=0x7f9c200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:20.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.402+0000 7f9c3edc4640 1 --2- 172.21.15.84:0/132526562 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9c200a2b90 0x7f9c200a5050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9c280096c0 tx=0x7f9c28031400 comp rx=0 tx=0).stop 2024-05-22T22:59:20.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.402+0000 7f9c3edc4640 1 -- 172.21.15.84:0/132526562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c381018c0 msgr2=0x7f9c38075620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:20.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.402+0000 7f9c3edc4640 1 --2- 172.21.15.84:0/132526562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c381018c0 0x7f9c38075620 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400d950 tx=0x7f9c3400de10 comp rx=0 tx=0).stop 2024-05-22T22:59:20.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.402+0000 7f9c3edc4640 1 -- 172.21.15.84:0/132526562 shutdown_connections 2024-05-22T22:59:20.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.402+0000 7f9c3edc4640 1 --2- 172.21.15.84:0/132526562 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9c200a2b90 0x7f9c200a5050 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:20.403 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.402+0000 7f9c3edc4640 1 --2- 172.21.15.84:0/132526562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c38103470 0x7f9c380741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:20.403 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.402+0000 7f9c3edc4640 1 --2- 172.21.15.84:0/132526562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c38102ab0 0x7f9c38073c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:20.403 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.402+0000 7f9c3edc4640 1 --2- 172.21.15.84:0/132526562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c381018c0 0x7f9c38075620 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:20.403 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.402+0000 7f9c3edc4640 1 -- 172.21.15.84:0/132526562 >> 172.21.15.84:0/132526562 conn(0x7f9c380fce50 msgr2=0x7f9c38105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:20.403 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.402+0000 7f9c3edc4640 1 -- 172.21.15.84:0/132526562 shutdown_connections 2024-05-22T22:59:20.403 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:20.402+0000 7f9c3edc4640 1 -- 172.21.15.84:0/132526562 wait complete. 2024-05-22T22:59:20.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:20 smithi096 ceph-mon[36392]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:20.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:20 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:20.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:20 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:20.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:20 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:20.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:20 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:20.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:20 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:20.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:20 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:20.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:20 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/132526562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:20.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:20 smithi084 ceph-mon[32186]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:20.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:20 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:20.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:20 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:20.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:20 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:20.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:20 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:20.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:20 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:20.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:20 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:20.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:20 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/132526562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:20.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:20 smithi204 ceph-mon[36211]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:20.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:20 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:20.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:20 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:20.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:20 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:20.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:20 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:20.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:20 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:20.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:20 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:20.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:20 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/132526562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:21.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:21 smithi096 ceph-mon[36392]: Deploying daemon mds.d on smithi084 2024-05-22T22:59:21.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:21 smithi096 ceph-mon[36392]: mds.? [v2:172.21.15.84:6834/2391815590,v1:172.21.15.84:6835/2391815590] up:boot 2024-05-22T22:59:21.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:21 smithi096 ceph-mon[36392]: fsmap 3 up:standby 2024-05-22T22:59:21.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:21 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-22T22:59:21.870 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:21 smithi204 ceph-mon[36211]: Deploying daemon mds.d on smithi084 2024-05-22T22:59:21.870 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:21 smithi204 ceph-mon[36211]: mds.? [v2:172.21.15.84:6834/2391815590,v1:172.21.15.84:6835/2391815590] up:boot 2024-05-22T22:59:21.870 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:21 smithi204 ceph-mon[36211]: fsmap 3 up:standby 2024-05-22T22:59:21.870 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:21 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-22T22:59:21.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:21 smithi084 ceph-mon[32186]: Deploying daemon mds.d on smithi084 2024-05-22T22:59:21.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:21 smithi084 ceph-mon[32186]: mds.? [v2:172.21.15.84:6834/2391815590,v1:172.21.15.84:6835/2391815590] up:boot 2024-05-22T22:59:21.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:21 smithi084 ceph-mon[32186]: fsmap 3 up:standby 2024-05-22T22:59:21.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:21 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-22T22:59:22.415 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T22:59:22.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.528+0000 7f32af0da640 1 -- 172.21.15.84:0/23121078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32a8102ad0 msgr2=0x7f32a8102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:22.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.528+0000 7f32af0da640 1 --2- 172.21.15.84:0/23121078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32a8102ad0 0x7f32a8102f50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3298009900 tx=0x7f329802a410 comp rx=0 tx=0).stop 2024-05-22T22:59:22.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.528+0000 7f32af0da640 1 -- 172.21.15.84:0/23121078 shutdown_connections 2024-05-22T22:59:22.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.528+0000 7f32af0da640 1 --2- 172.21.15.84:0/23121078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32a8103490 0x7f32a8109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:22.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.528+0000 7f32af0da640 1 --2- 172.21.15.84:0/23121078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32a8102ad0 0x7f32a8102f50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:22.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.528+0000 7f32af0da640 1 --2- 172.21.15.84:0/23121078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32a81018e0 0x7f32a8101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:22.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.528+0000 7f32af0da640 1 -- 172.21.15.84:0/23121078 >> 172.21.15.84:0/23121078 conn(0x7f32a80fce70 msgr2=0x7f32a80ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:22.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.528+0000 7f32af0da640 1 -- 172.21.15.84:0/23121078 shutdown_connections 2024-05-22T22:59:22.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.528+0000 7f32af0da640 1 -- 172.21.15.84:0/23121078 wait complete. 2024-05-22T22:59:22.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.528+0000 7f32af0da640 1 Processor -- start 2024-05-22T22:59:22.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.528+0000 7f32af0da640 1 -- start start 2024-05-22T22:59:22.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.529+0000 7f32af0da640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32a81018e0 0x7f32a8199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:22.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.529+0000 7f32af0da640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32a8102ad0 0x7f32a819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:22.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.529+0000 7f32af0da640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32a8103490 0x7f32a81a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:22.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.529+0000 7f32af0da640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f32a810d5c0 con 0x7f32a8102ad0 2024-05-22T22:59:22.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.529+0000 7f32af0da640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f32a810d440 con 0x7f32a8103490 2024-05-22T22:59:22.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.529+0000 7f32af0da640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f32a810d740 con 0x7f32a81018e0 2024-05-22T22:59:22.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.529+0000 7f329ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32a8102ad0 0x7f32a819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:22.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.529+0000 7f329ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32a8102ad0 0x7f32a819a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44558/0 (socket says 172.21.15.84:44558) 2024-05-22T22:59:22.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.529+0000 7f329ffff640 1 -- 172.21.15.84:0/2175856169 learned_addr learned my addr 172.21.15.84:0/2175856169 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:22.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.529+0000 7f32ace76640 1 --2- 172.21.15.84:0/2175856169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32a81018e0 0x7f32a8199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:22.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.529+0000 7f32ad677640 1 --2- 172.21.15.84:0/2175856169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32a8103490 0x7f32a81a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:22.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.529+0000 7f329ffff640 1 -- 172.21.15.84:0/2175856169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32a81018e0 msgr2=0x7f32a8199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:22.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.529+0000 7f329ffff640 1 --2- 172.21.15.84:0/2175856169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32a81018e0 0x7f32a8199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:22.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.529+0000 7f329ffff640 1 -- 172.21.15.84:0/2175856169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32a8103490 msgr2=0x7f32a81a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:22.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.529+0000 7f329ffff640 1 --2- 172.21.15.84:0/2175856169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32a8103490 0x7f32a81a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:22.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.529+0000 7f329ffff640 1 -- 172.21.15.84:0/2175856169 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a81a1a40 con 0x7f32a8102ad0 2024-05-22T22:59:22.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.529+0000 7f32ad677640 1 --2- 172.21.15.84:0/2175856169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32a8103490 0x7f32a81a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T22:59:22.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.529+0000 7f329ffff640 1 --2- 172.21.15.84:0/2175856169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32a8102ad0 0x7f32a819a360 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3298009230 tx=0x7f3298004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:22.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.529+0000 7f329dffb640 1 -- 172.21.15.84:0/2175856169 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329803f030 con 0x7f32a8102ad0 2024-05-22T22:59:22.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.529+0000 7f329dffb640 1 -- 172.21.15.84:0/2175856169 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3298003ce0 con 0x7f32a8102ad0 2024-05-22T22:59:22.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.529+0000 7f32af0da640 1 -- 172.21.15.84:0/2175856169 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a81a1cd0 con 0x7f32a8102ad0 2024-05-22T22:59:22.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.529+0000 7f329dffb640 1 -- 172.21.15.84:0/2175856169 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298035070 con 0x7f32a8102ad0 2024-05-22T22:59:22.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.530+0000 7f32af0da640 1 -- 172.21.15.84:0/2175856169 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a81a2240 con 0x7f32a8102ad0 2024-05-22T22:59:22.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.531+0000 7f32af0da640 1 -- 172.21.15.84:0/2175856169 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32a81a2530 con 0x7f32a8102ad0 2024-05-22T22:59:22.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.533+0000 7f329dffb640 1 -- 172.21.15.84:0/2175856169 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3298042030 con 0x7f32a8102ad0 2024-05-22T22:59:22.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.533+0000 7f329dffb640 1 --2- 172.21.15.84:0/2175856169 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f32900a2c10 0x7f32900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:22.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.533+0000 7f329dffb640 1 -- 172.21.15.84:0/2175856169 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f32980ef610 con 0x7f32a8102ad0 2024-05-22T22:59:22.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.533+0000 7f32ace76640 1 --2- 172.21.15.84:0/2175856169 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f32900a2c10 0x7f32900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:22.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.534+0000 7f32ace76640 1 --2- 172.21.15.84:0/2175856169 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f32900a2c10 0x7f32900a50d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f32a8102930 tx=0x7f32a4009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:22.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.537+0000 7f329dffb640 1 -- 172.21.15.84:0/2175856169 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f32980bcb10 con 0x7f32a8102ad0 2024-05-22T22:59:22.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.758+0000 7f32af0da640 1 -- 172.21.15.84:0/2175856169 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32a81a2700 con 0x7f32a8102ad0 2024-05-22T22:59:22.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.759+0000 7f329dffb640 1 -- 172.21.15.84:0/2175856169 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f32980bc410 con 0x7f32a8102ad0 2024-05-22T22:59:22.759 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:22.759 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-22T22:59:22.759 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 4 2024-05-22T22:59:22.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.761+0000 7f32af0da640 1 -- 172.21.15.84:0/2175856169 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f32900a2c10 msgr2=0x7f32900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:22.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.761+0000 7f32af0da640 1 --2- 172.21.15.84:0/2175856169 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f32900a2c10 0x7f32900a50d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f32a8102930 tx=0x7f32a4009870 comp rx=0 tx=0).stop 2024-05-22T22:59:22.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.761+0000 7f32af0da640 1 -- 172.21.15.84:0/2175856169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32a8102ad0 msgr2=0x7f32a819a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:22.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.761+0000 7f32af0da640 1 --2- 172.21.15.84:0/2175856169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32a8102ad0 0x7f32a819a360 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3298009230 tx=0x7f3298004360 comp rx=0 tx=0).stop 2024-05-22T22:59:22.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.761+0000 7f32af0da640 1 -- 172.21.15.84:0/2175856169 shutdown_connections 2024-05-22T22:59:22.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.761+0000 7f32af0da640 1 --2- 172.21.15.84:0/2175856169 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f32900a2c10 0x7f32900a50d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:22.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.761+0000 7f32af0da640 1 --2- 172.21.15.84:0/2175856169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32a8103490 0x7f32a81a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:22.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.761+0000 7f32af0da640 1 --2- 172.21.15.84:0/2175856169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32a8102ad0 0x7f32a819a360 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:22.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.761+0000 7f32af0da640 1 --2- 172.21.15.84:0/2175856169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32a81018e0 0x7f32a8199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:22.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.761+0000 7f32af0da640 1 -- 172.21.15.84:0/2175856169 >> 172.21.15.84:0/2175856169 conn(0x7f32a80fce70 msgr2=0x7f32a8073aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:22.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.761+0000 7f32af0da640 1 -- 172.21.15.84:0/2175856169 shutdown_connections 2024-05-22T22:59:22.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:22.761+0000 7f32af0da640 1 -- 172.21.15.84:0/2175856169 wait complete. 2024-05-22T22:59:22.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:22 smithi084 ceph-mon[32186]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:22.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:22 smithi204 ceph-mon[36211]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:23.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:22 smithi096 ceph-mon[36392]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:23.373 INFO:journalctl@ceph.mds.d.smithi084.stdout:May 22 22:59:23 smithi084 systemd[1]: Started Ceph mds.d for c1135a12-188d-11ef-bc9a-c7b262605968. 2024-05-22T22:59:23.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:23 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2175856169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:23.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:23 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:23.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:23 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:23.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:23 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:23.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:23 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:23.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:23 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:23.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:23 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:23.903 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:23 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2175856169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:23.903 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:23 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:23.903 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:23 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:23.903 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:23 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:23.903 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:23 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:23.903 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:23 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:23.903 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:23 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:23.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:23 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2175856169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:23.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:23 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:23.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:23 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:23.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:23 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:23.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:23 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:23.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:23 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:23.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:23 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:24.772 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T22:59:24.810 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:24 smithi204 ceph-mon[36211]: Deploying daemon mds.e on smithi096 2024-05-22T22:59:24.810 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:24 smithi204 ceph-mon[36211]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:24.811 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:24 smithi204 ceph-mon[36211]: mds.? [v2:172.21.15.84:6836/4284308638,v1:172.21.15.84:6837/4284308638] up:boot 2024-05-22T22:59:24.811 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:24 smithi204 ceph-mon[36211]: fsmap 4 up:standby 2024-05-22T22:59:24.811 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:24 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-22T22:59:24.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:24 smithi084 ceph-mon[32186]: Deploying daemon mds.e on smithi096 2024-05-22T22:59:24.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:24 smithi084 ceph-mon[32186]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:24.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:24 smithi084 ceph-mon[32186]: mds.? [v2:172.21.15.84:6836/4284308638,v1:172.21.15.84:6837/4284308638] up:boot 2024-05-22T22:59:24.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:24 smithi084 ceph-mon[32186]: fsmap 4 up:standby 2024-05-22T22:59:24.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:24 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-22T22:59:24.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.886+0000 7f7c592bd640 1 -- 172.21.15.84:0/4043634833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c54102360 msgr2=0x7f7c54108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:24.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.886+0000 7f7c592bd640 1 --2- 172.21.15.84:0/4043634833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c54102360 0x7f7c54108bf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c009900 tx=0x7f7c4c02a410 comp rx=0 tx=0).stop 2024-05-22T22:59:24.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.886+0000 7f7c592bd640 1 -- 172.21.15.84:0/4043634833 shutdown_connections 2024-05-22T22:59:24.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.886+0000 7f7c592bd640 1 --2- 172.21.15.84:0/4043634833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c54102360 0x7f7c54108bf0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:24.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.886+0000 7f7c592bd640 1 --2- 172.21.15.84:0/4043634833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c541019a0 0x7f7c54101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:24.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.886+0000 7f7c592bd640 1 --2- 172.21.15.84:0/4043634833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c541007b0 0x7f7c54100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:24.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.886+0000 7f7c592bd640 1 -- 172.21.15.84:0/4043634833 >> 172.21.15.84:0/4043634833 conn(0x7f7c540fbd20 msgr2=0x7f7c540fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:24.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.886+0000 7f7c592bd640 1 -- 172.21.15.84:0/4043634833 shutdown_connections 2024-05-22T22:59:24.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.886+0000 7f7c592bd640 1 -- 172.21.15.84:0/4043634833 wait complete. 2024-05-22T22:59:24.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.887+0000 7f7c592bd640 1 Processor -- start 2024-05-22T22:59:24.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.887+0000 7f7c592bd640 1 -- start start 2024-05-22T22:59:24.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.887+0000 7f7c592bd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c541007b0 0x7f7c54197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:24.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.887+0000 7f7c592bd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c541019a0 0x7f7c541980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:24.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.887+0000 7f7c592bd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c54102360 0x7f7c5419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:24.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.887+0000 7f7c592bd640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5410c430 con 0x7f7c541019a0 2024-05-22T22:59:24.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.887+0000 7f7c592bd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5410c2b0 con 0x7f7c54102360 2024-05-22T22:59:24.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.887+0000 7f7c592bd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5410c5b0 con 0x7f7c541007b0 2024-05-22T22:59:24.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.887+0000 7f7c527fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c541019a0 0x7f7c541980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:24.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.887+0000 7f7c527fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c541019a0 0x7f7c541980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44596/0 (socket says 172.21.15.84:44596) 2024-05-22T22:59:24.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.887+0000 7f7c527fc640 1 -- 172.21.15.84:0/1633720766 learned_addr learned my addr 172.21.15.84:0/1633720766 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:24.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.887+0000 7f7c52ffd640 1 --2- 172.21.15.84:0/1633720766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c541007b0 0x7f7c54197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:24.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.887+0000 7f7c537fe640 1 --2- 172.21.15.84:0/1633720766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c54102360 0x7f7c5419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:24.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.888+0000 7f7c527fc640 1 -- 172.21.15.84:0/1633720766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c541007b0 msgr2=0x7f7c54197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:24.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.888+0000 7f7c527fc640 1 --2- 172.21.15.84:0/1633720766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c541007b0 0x7f7c54197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:24.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.888+0000 7f7c527fc640 1 -- 172.21.15.84:0/1633720766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c54102360 msgr2=0x7f7c5419f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:24.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.888+0000 7f7c527fc640 1 --2- 172.21.15.84:0/1633720766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c54102360 0x7f7c5419f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:24.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.888+0000 7f7c527fc640 1 -- 172.21.15.84:0/1633720766 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5419f7a0 con 0x7f7c541019a0 2024-05-22T22:59:24.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.888+0000 7f7c527fc640 1 --2- 172.21.15.84:0/1633720766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c541019a0 0x7f7c541980c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00d770 tx=0x7f7c3c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:24.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.888+0000 7f7c4bfff640 1 -- 172.21.15.84:0/1633720766 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c017070 con 0x7f7c541019a0 2024-05-22T22:59:24.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.888+0000 7f7c592bd640 1 -- 172.21.15.84:0/1633720766 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c5419fa90 con 0x7f7c541019a0 2024-05-22T22:59:24.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.888+0000 7f7c4bfff640 1 -- 172.21.15.84:0/1633720766 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c3c005500 con 0x7f7c541019a0 2024-05-22T22:59:24.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.888+0000 7f7c4bfff640 1 -- 172.21.15.84:0/1633720766 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c004e00 con 0x7f7c541019a0 2024-05-22T22:59:24.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.888+0000 7f7c592bd640 1 -- 172.21.15.84:0/1633720766 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c5419ff90 con 0x7f7c541019a0 2024-05-22T22:59:24.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.889+0000 7f7c592bd640 1 -- 172.21.15.84:0/1633720766 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c541a01c0 con 0x7f7c541019a0 2024-05-22T22:59:24.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.890+0000 7f7c4bfff640 1 -- 172.21.15.84:0/1633720766 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7c3c004fa0 con 0x7f7c541019a0 2024-05-22T22:59:24.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.891+0000 7f7c4bfff640 1 --2- 172.21.15.84:0/1633720766 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f7c400a2b90 0x7f7c400a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:24.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.891+0000 7f7c52ffd640 1 --2- 172.21.15.84:0/1633720766 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f7c400a2b90 0x7f7c400a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:24.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.891+0000 7f7c4bfff640 1 -- 172.21.15.84:0/1633720766 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f7c3c0ce9d0 con 0x7f7c541019a0 2024-05-22T22:59:24.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.891+0000 7f7c52ffd640 1 --2- 172.21.15.84:0/1633720766 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f7c400a2b90 0x7f7c400a5050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7c54101800 tx=0x7f7c44009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:24.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:24.894+0000 7f7c4bfff640 1 -- 172.21.15.84:0/1633720766 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7c3c09bdd0 con 0x7f7c541019a0 2024-05-22T22:59:24.933 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:24 smithi096 ceph-mon[36392]: Deploying daemon mds.e on smithi096 2024-05-22T22:59:24.933 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:24 smithi096 ceph-mon[36392]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:24.933 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:24 smithi096 ceph-mon[36392]: mds.? [v2:172.21.15.84:6836/4284308638,v1:172.21.15.84:6837/4284308638] up:boot 2024-05-22T22:59:24.933 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:24 smithi096 ceph-mon[36392]: fsmap 4 up:standby 2024-05-22T22:59:24.933 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:24 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-22T22:59:25.107 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:25.107+0000 7f7c592bd640 1 -- 172.21.15.84:0/1633720766 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c541a0390 con 0x7f7c541019a0 2024-05-22T22:59:25.108 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:25.108+0000 7f7c4bfff640 1 -- 172.21.15.84:0/1633720766 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f7c3c09b6d0 con 0x7f7c541019a0 2024-05-22T22:59:25.108 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:25.108 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-22T22:59:25.108 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 5 2024-05-22T22:59:25.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:25.110+0000 7f7c592bd640 1 -- 172.21.15.84:0/1633720766 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f7c400a2b90 msgr2=0x7f7c400a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:25.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:25.110+0000 7f7c592bd640 1 --2- 172.21.15.84:0/1633720766 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f7c400a2b90 0x7f7c400a5050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7c54101800 tx=0x7f7c44009670 comp rx=0 tx=0).stop 2024-05-22T22:59:25.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:25.110+0000 7f7c592bd640 1 -- 172.21.15.84:0/1633720766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c541019a0 msgr2=0x7f7c541980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:25.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:25.110+0000 7f7c592bd640 1 --2- 172.21.15.84:0/1633720766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c541019a0 0x7f7c541980c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00d770 tx=0x7f7c3c00dc30 comp rx=0 tx=0).stop 2024-05-22T22:59:25.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:25.110+0000 7f7c592bd640 1 -- 172.21.15.84:0/1633720766 shutdown_connections 2024-05-22T22:59:25.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:25.110+0000 7f7c592bd640 1 --2- 172.21.15.84:0/1633720766 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f7c400a2b90 0x7f7c400a5050 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:25.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:25.110+0000 7f7c592bd640 1 --2- 172.21.15.84:0/1633720766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c54102360 0x7f7c5419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:25.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:25.110+0000 7f7c592bd640 1 --2- 172.21.15.84:0/1633720766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c541019a0 0x7f7c541980c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:25.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:25.110+0000 7f7c592bd640 1 --2- 172.21.15.84:0/1633720766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c541007b0 0x7f7c54197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:25.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:25.110+0000 7f7c592bd640 1 -- 172.21.15.84:0/1633720766 >> 172.21.15.84:0/1633720766 conn(0x7f7c540fbd20 msgr2=0x7f7c54068a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:25.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:25.110+0000 7f7c592bd640 1 -- 172.21.15.84:0/1633720766 shutdown_connections 2024-05-22T22:59:25.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:25.110+0000 7f7c592bd640 1 -- 172.21.15.84:0/1633720766 wait complete. 2024-05-22T22:59:25.548 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 22 22:59:25 smithi096 systemd[1]: Starting Ceph mds.e for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T22:59:25.822 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:25 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1633720766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:25.823 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 22 22:59:25 smithi096 podman[66228]: 2024-05-22 22:59:25.585200877 +0000 UTC m=+0.017349106 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T22:59:25.823 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 22 22:59:25 smithi096 podman[66228]: 2024-05-22 22:59:25.699486245 +0000 UTC m=+0.131634480 container create 61c5842649bc6a7a1d2ce53e223d35547ed7ec9d578bbc84347473a77e9c0c5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-e, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, RELEASE=squid-2e1f762, org.label-schema.vendor=CentOS) 2024-05-22T22:59:25.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:25 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1633720766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:25.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:25 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1633720766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:26.298 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 22 22:59:25 smithi096 podman[66228]: 2024-05-22 22:59:25.943763976 +0000 UTC m=+0.375912189 container init 61c5842649bc6a7a1d2ce53e223d35547ed7ec9d578bbc84347473a77e9c0c5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-e, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-2e1f762, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240521) 2024-05-22T22:59:26.298 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 22 22:59:25 smithi096 podman[66228]: 2024-05-22 22:59:25.946341405 +0000 UTC m=+0.378489621 container start 61c5842649bc6a7a1d2ce53e223d35547ed7ec9d578bbc84347473a77e9c0c5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-2e1f762, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-22T22:59:26.298 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 22 22:59:25 smithi096 ceph-mds[66247]: -- 172.21.15.96:0/2861882061 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5608c00fba40 con 0x5608c00ce800 2024-05-22T22:59:26.298 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 22 22:59:25 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-e[66243]: starting mds.e at 2024-05-22T22:59:26.299 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 22 22:59:26 smithi096 bash[66228]: 61c5842649bc6a7a1d2ce53e223d35547ed7ec9d578bbc84347473a77e9c0c5a 2024-05-22T22:59:26.299 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 22 22:59:26 smithi096 systemd[1]: Started Ceph mds.e for c1135a12-188d-11ef-bc9a-c7b262605968. 2024-05-22T22:59:26.841 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:26 smithi204 ceph-mon[36211]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:26.841 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:26 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:26.841 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:26 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:26.841 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:26 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:26.842 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:26 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:26.842 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:26 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:26.842 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:26 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:26.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:26 smithi084 ceph-mon[32186]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:26.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:26 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:26.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:26 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:26.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:26 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:26.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:26 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:26.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:26 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:26.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:26 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:27.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:26 smithi096 ceph-mon[36392]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:27.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:26 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:27.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:26 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:27.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:26 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:27.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:26 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:27.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:26 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:27.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:26 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:27.122 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T22:59:27.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.236+0000 7f573ff3b640 1 -- 172.21.15.84:0/1266834409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57381019a0 msgr2=0x7f5738101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:27.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.236+0000 7f573ff3b640 1 --2- 172.21.15.84:0/1266834409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57381019a0 0x7f5738101e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f572c009900 tx=0x7f572c02a410 comp rx=0 tx=0).stop 2024-05-22T22:59:27.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.236+0000 7f573ff3b640 1 -- 172.21.15.84:0/1266834409 shutdown_connections 2024-05-22T22:59:27.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.236+0000 7f573ff3b640 1 --2- 172.21.15.84:0/1266834409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5738102360 0x7f5738108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:27.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.236+0000 7f573ff3b640 1 --2- 172.21.15.84:0/1266834409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57381019a0 0x7f5738101e20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:27.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.236+0000 7f573ff3b640 1 --2- 172.21.15.84:0/1266834409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57381007b0 0x7f5738100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:27.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.236+0000 7f573ff3b640 1 -- 172.21.15.84:0/1266834409 >> 172.21.15.84:0/1266834409 conn(0x7f57380fbd20 msgr2=0x7f57380fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:27.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.236+0000 7f573ff3b640 1 -- 172.21.15.84:0/1266834409 shutdown_connections 2024-05-22T22:59:27.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.236+0000 7f573ff3b640 1 -- 172.21.15.84:0/1266834409 wait complete. 2024-05-22T22:59:27.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.236+0000 7f573ff3b640 1 Processor -- start 2024-05-22T22:59:27.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.236+0000 7f573ff3b640 1 -- start start 2024-05-22T22:59:27.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.237+0000 7f573ff3b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57381007b0 0x7f5738197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:27.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.237+0000 7f573ff3b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57381019a0 0x7f5738198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:27.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.237+0000 7f573ff3b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5738102360 0x7f573819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:27.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.237+0000 7f573ff3b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f573810c430 con 0x7f57381019a0 2024-05-22T22:59:27.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.237+0000 7f573ff3b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f573810c2b0 con 0x7f5738102360 2024-05-22T22:59:27.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.237+0000 7f573ff3b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f573810c5b0 con 0x7f57381007b0 2024-05-22T22:59:27.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.237+0000 7f573d4d6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57381019a0 0x7f5738198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:27.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.237+0000 7f573d4d6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57381019a0 0x7f5738198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44606/0 (socket says 172.21.15.84:44606) 2024-05-22T22:59:27.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.237+0000 7f573d4d6640 1 -- 172.21.15.84:0/2152149899 learned_addr learned my addr 172.21.15.84:0/2152149899 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:27.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.237+0000 7f573dcd7640 1 --2- 172.21.15.84:0/2152149899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57381007b0 0x7f5738197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:27.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.237+0000 7f573e4d8640 1 --2- 172.21.15.84:0/2152149899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5738102360 0x7f573819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:27.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.237+0000 7f573d4d6640 1 -- 172.21.15.84:0/2152149899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57381007b0 msgr2=0x7f5738197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:27.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.237+0000 7f573d4d6640 1 --2- 172.21.15.84:0/2152149899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57381007b0 0x7f5738197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:27.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.237+0000 7f573d4d6640 1 -- 172.21.15.84:0/2152149899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5738102360 msgr2=0x7f573819f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:27.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.237+0000 7f573d4d6640 1 --2- 172.21.15.84:0/2152149899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5738102360 0x7f573819f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:27.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.237+0000 7f573d4d6640 1 -- 172.21.15.84:0/2152149899 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573819f800 con 0x7f57381019a0 2024-05-22T22:59:27.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.237+0000 7f573dcd7640 1 --2- 172.21.15.84:0/2152149899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57381007b0 0x7f5738197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T22:59:27.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.237+0000 7f573d4d6640 1 --2- 172.21.15.84:0/2152149899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57381019a0 0x7f5738198120 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f572c009230 tx=0x7f572c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:27.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.237+0000 7f5736ffd640 1 -- 172.21.15.84:0/2152149899 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c03f030 con 0x7f57381019a0 2024-05-22T22:59:27.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.237+0000 7f5736ffd640 1 -- 172.21.15.84:0/2152149899 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f572c003ce0 con 0x7f57381019a0 2024-05-22T22:59:27.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.237+0000 7f573ff3b640 1 -- 172.21.15.84:0/2152149899 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f573819fa90 con 0x7f57381019a0 2024-05-22T22:59:27.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.237+0000 7f5736ffd640 1 -- 172.21.15.84:0/2152149899 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c035070 con 0x7f57381019a0 2024-05-22T22:59:27.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.237+0000 7f573ff3b640 1 -- 172.21.15.84:0/2152149899 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f573819ff80 con 0x7f57381019a0 2024-05-22T22:59:27.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.238+0000 7f573ff3b640 1 -- 172.21.15.84:0/2152149899 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57381a0150 con 0x7f57381019a0 2024-05-22T22:59:27.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.239+0000 7f5736ffd640 1 -- 172.21.15.84:0/2152149899 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f572c042030 con 0x7f57381019a0 2024-05-22T22:59:27.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.240+0000 7f5736ffd640 1 --2- 172.21.15.84:0/2152149899 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f57200a2b90 0x7f57200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:27.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.240+0000 7f573dcd7640 1 --2- 172.21.15.84:0/2152149899 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f57200a2b90 0x7f57200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:27.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.240+0000 7f5736ffd640 1 -- 172.21.15.84:0/2152149899 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f572c0ef640 con 0x7f57381019a0 2024-05-22T22:59:27.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.240+0000 7f573dcd7640 1 --2- 172.21.15.84:0/2152149899 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f57200a2b90 0x7f57200a5050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f57300058c0 tx=0x7f5730005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:27.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.244+0000 7f5736ffd640 1 -- 172.21.15.84:0/2152149899 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f572c0bca40 con 0x7f57381019a0 2024-05-22T22:59:27.459 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.459+0000 7f573ff3b640 1 -- 172.21.15.84:0/2152149899 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57381a04e0 con 0x7f57381019a0 2024-05-22T22:59:27.459 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.459+0000 7f5736ffd640 1 -- 172.21.15.84:0/2152149899 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4696 (secure 0 0 0) 0x7f572c0bc340 con 0x7f57381019a0 2024-05-22T22:59:27.459 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:27.460 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-22T22:59:27.460 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 6 2024-05-22T22:59:27.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.461+0000 7f573ff3b640 1 -- 172.21.15.84:0/2152149899 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f57200a2b90 msgr2=0x7f57200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:27.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.461+0000 7f573ff3b640 1 --2- 172.21.15.84:0/2152149899 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f57200a2b90 0x7f57200a5050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f57300058c0 tx=0x7f5730005d50 comp rx=0 tx=0).stop 2024-05-22T22:59:27.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.461+0000 7f573ff3b640 1 -- 172.21.15.84:0/2152149899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57381019a0 msgr2=0x7f5738198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:27.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.461+0000 7f573ff3b640 1 --2- 172.21.15.84:0/2152149899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57381019a0 0x7f5738198120 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f572c009230 tx=0x7f572c004360 comp rx=0 tx=0).stop 2024-05-22T22:59:27.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.461+0000 7f573ff3b640 1 -- 172.21.15.84:0/2152149899 shutdown_connections 2024-05-22T22:59:27.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.461+0000 7f573ff3b640 1 --2- 172.21.15.84:0/2152149899 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f57200a2b90 0x7f57200a5050 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:27.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.461+0000 7f573ff3b640 1 --2- 172.21.15.84:0/2152149899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5738102360 0x7f573819f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:27.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.461+0000 7f573ff3b640 1 --2- 172.21.15.84:0/2152149899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57381019a0 0x7f5738198120 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:27.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.461+0000 7f573ff3b640 1 --2- 172.21.15.84:0/2152149899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57381007b0 0x7f5738197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:27.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.461+0000 7f573ff3b640 1 -- 172.21.15.84:0/2152149899 >> 172.21.15.84:0/2152149899 conn(0x7f57380fbd20 msgr2=0x7f5738068a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:27.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.461+0000 7f573ff3b640 1 -- 172.21.15.84:0/2152149899 shutdown_connections 2024-05-22T22:59:27.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:27.461+0000 7f573ff3b640 1 -- 172.21.15.84:0/2152149899 wait complete. 2024-05-22T22:59:27.775 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:27 smithi204 ceph-mon[36211]: Deploying daemon mds.f on smithi204 2024-05-22T22:59:27.775 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:27 smithi204 ceph-mon[36211]: mds.? [v2:172.21.15.96:6834/1115134082,v1:172.21.15.96:6835/1115134082] up:boot 2024-05-22T22:59:27.775 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:27 smithi204 ceph-mon[36211]: fsmap 5 up:standby 2024-05-22T22:59:27.775 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:27 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-22T22:59:27.775 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:27 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2152149899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:27.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:27 smithi084 ceph-mon[32186]: Deploying daemon mds.f on smithi204 2024-05-22T22:59:27.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:27 smithi084 ceph-mon[32186]: mds.? [v2:172.21.15.96:6834/1115134082,v1:172.21.15.96:6835/1115134082] up:boot 2024-05-22T22:59:27.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:27 smithi084 ceph-mon[32186]: fsmap 5 up:standby 2024-05-22T22:59:27.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:27 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-22T22:59:27.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:27 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2152149899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:28.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:27 smithi096 ceph-mon[36392]: Deploying daemon mds.f on smithi204 2024-05-22T22:59:28.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:27 smithi096 ceph-mon[36392]: mds.? [v2:172.21.15.96:6834/1115134082,v1:172.21.15.96:6835/1115134082] up:boot 2024-05-22T22:59:28.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:27 smithi096 ceph-mon[36392]: fsmap 5 up:standby 2024-05-22T22:59:28.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:27 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-22T22:59:28.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:27 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2152149899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:28.409 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 22 22:59:28 smithi204 systemd[1]: Starting Ceph mds.f for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T22:59:28.748 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:28 smithi204 ceph-mon[36211]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:28.748 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 22 22:59:28 smithi204 podman[66328]: 2024-05-22 22:59:28.358815376 +0000 UTC m=+0.018148246 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T22:59:28.748 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 22 22:59:28 smithi204 podman[66328]: 2024-05-22 22:59:28.504225371 +0000 UTC m=+0.163558233 container create a1ab7dd8ec58b743f7121085e5fd62878a0ee2ad85f12c83b562cef07c2f5fcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-f, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2e1f762, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-22T22:59:28.748 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 22 22:59:28 smithi204 podman[66328]: 2024-05-22 22:59:28.749461797 +0000 UTC m=+0.408794661 container init a1ab7dd8ec58b743f7121085e5fd62878a0ee2ad85f12c83b562cef07c2f5fcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-f, org.label-schema.license=GPLv2, RELEASE=squid-2e1f762, ceph=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-22T22:59:28.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:28 smithi084 ceph-mon[32186]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:29.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:28 smithi096 ceph-mon[36392]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:29.159 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 22 22:59:28 smithi204 podman[66328]: 2024-05-22 22:59:28.752059405 +0000 UTC m=+0.411392281 container start a1ab7dd8ec58b743f7121085e5fd62878a0ee2ad85f12c83b562cef07c2f5fcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-f, GIT_CLEAN=True, RELEASE=squid-2e1f762, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-22T22:59:29.159 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 22 22:59:28 smithi204 ceph-mds[66347]: -- 172.21.15.204:0/2013489740 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5585918bfa40 con 0x558591892c00 2024-05-22T22:59:29.160 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 22 22:59:28 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-f[66343]: starting mds.f at 2024-05-22T22:59:29.160 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 22 22:59:28 smithi204 bash[66328]: a1ab7dd8ec58b743f7121085e5fd62878a0ee2ad85f12c83b562cef07c2f5fcf 2024-05-22T22:59:29.160 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 22 22:59:28 smithi204 systemd[1]: Started Ceph mds.f for c1135a12-188d-11ef-bc9a-c7b262605968. 2024-05-22T22:59:29.473 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T22:59:29.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.589+0000 7fd5ff11d640 1 -- 172.21.15.84:0/3513648533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5f8101ee0 msgr2=0x7fd5f8102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:29.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.589+0000 7fd5ff11d640 1 --2- 172.21.15.84:0/3513648533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5f8101ee0 0x7fd5f8102300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec009900 tx=0x7fd5ec02a400 comp rx=0 tx=0).stop 2024-05-22T22:59:29.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.589+0000 7fd5ff11d640 1 -- 172.21.15.84:0/3513648533 shutdown_connections 2024-05-22T22:59:29.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.589+0000 7fd5ff11d640 1 --2- 172.21.15.84:0/3513648533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5f8101460 0x7fd5f8108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:29.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.589+0000 7fd5ff11d640 1 --2- 172.21.15.84:0/3513648533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5f8100aa0 0x7fd5f8100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:29.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.589+0000 7fd5ff11d640 1 --2- 172.21.15.84:0/3513648533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5f8101ee0 0x7fd5f8102300 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:29.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.589+0000 7fd5ff11d640 1 -- 172.21.15.84:0/3513648533 >> 172.21.15.84:0/3513648533 conn(0x7fd5f80fb550 msgr2=0x7fd5f80fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:29.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.589+0000 7fd5ff11d640 1 -- 172.21.15.84:0/3513648533 shutdown_connections 2024-05-22T22:59:29.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.589+0000 7fd5ff11d640 1 -- 172.21.15.84:0/3513648533 wait complete. 2024-05-22T22:59:29.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.590+0000 7fd5ff11d640 1 Processor -- start 2024-05-22T22:59:29.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.590+0000 7fd5ff11d640 1 -- start start 2024-05-22T22:59:29.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.590+0000 7fd5ff11d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5f8100aa0 0x7fd5f8196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:29.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.590+0000 7fd5ff11d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5f8101460 0x7fd5f81970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:29.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.590+0000 7fd5ff11d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5f8101ee0 0x7fd5f819e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:29.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.590+0000 7fd5ff11d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f810c4f0 con 0x7fd5f8101ee0 2024-05-22T22:59:29.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.590+0000 7fd5ff11d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f810c370 con 0x7fd5f8100aa0 2024-05-22T22:59:29.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.590+0000 7fd5ff11d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f810c670 con 0x7fd5f8101460 2024-05-22T22:59:29.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.590+0000 7fd5fd6ba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5f8101ee0 0x7fd5f819e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:29.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.590+0000 7fd5fceb9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5f8100aa0 0x7fd5f8196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:29.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.590+0000 7fd5fd6ba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5f8101ee0 0x7fd5f819e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44626/0 (socket says 172.21.15.84:44626) 2024-05-22T22:59:29.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.590+0000 7fd5fceb9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5f8100aa0 0x7fd5f8196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:34746/0 (socket says 172.21.15.84:34746) 2024-05-22T22:59:29.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.590+0000 7fd5fd6ba640 1 -- 172.21.15.84:0/999244531 learned_addr learned my addr 172.21.15.84:0/999244531 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:29.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.590+0000 7fd5f7fff640 1 --2- 172.21.15.84:0/999244531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5f8101460 0x7fd5f81970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:29.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.591+0000 7fd5fd6ba640 1 -- 172.21.15.84:0/999244531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5f8101460 msgr2=0x7fd5f81970d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:29.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.591+0000 7fd5fd6ba640 1 --2- 172.21.15.84:0/999244531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5f8101460 0x7fd5f81970d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:29.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.591+0000 7fd5fd6ba640 1 -- 172.21.15.84:0/999244531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5f8100aa0 msgr2=0x7fd5f8196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:29.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.591+0000 7fd5fd6ba640 1 --2- 172.21.15.84:0/999244531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5f8100aa0 0x7fd5f8196b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:29.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.591+0000 7fd5fd6ba640 1 -- 172.21.15.84:0/999244531 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f819e7b0 con 0x7fd5f8101ee0 2024-05-22T22:59:29.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.591+0000 7fd5fceb9640 1 --2- 172.21.15.84:0/999244531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5f8100aa0 0x7fd5f8196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T22:59:29.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.591+0000 7fd5fd6ba640 1 --2- 172.21.15.84:0/999244531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5f8101ee0 0x7fd5f819e140 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000eaa0 tx=0x7fd5f000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:29.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.591+0000 7fd5f5ffb640 1 -- 172.21.15.84:0/999244531 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f000cdb0 con 0x7fd5f8101ee0 2024-05-22T22:59:29.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.591+0000 7fd5f5ffb640 1 -- 172.21.15.84:0/999244531 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5f0005020 con 0x7fd5f8101ee0 2024-05-22T22:59:29.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.591+0000 7fd5ff11d640 1 -- 172.21.15.84:0/999244531 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f819eaa0 con 0x7fd5f8101ee0 2024-05-22T22:59:29.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.591+0000 7fd5f5ffb640 1 -- 172.21.15.84:0/999244531 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f00116b0 con 0x7fd5f8101ee0 2024-05-22T22:59:29.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.591+0000 7fd5ff11d640 1 -- 172.21.15.84:0/999244531 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f819efa0 con 0x7fd5f8101ee0 2024-05-22T22:59:29.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.592+0000 7fd5ff11d640 1 -- 172.21.15.84:0/999244531 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5f8107be0 con 0x7fd5f8101ee0 2024-05-22T22:59:29.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.594+0000 7fd5f5ffb640 1 -- 172.21.15.84:0/999244531 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd5f0005560 con 0x7fd5f8101ee0 2024-05-22T22:59:29.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.595+0000 7fd5f5ffb640 1 --2- 172.21.15.84:0/999244531 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fd5e80a2b90 0x7fd5e80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:29.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.595+0000 7fd5fceb9640 1 --2- 172.21.15.84:0/999244531 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fd5e80a2b90 0x7fd5e80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:29.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.595+0000 7fd5f5ffb640 1 -- 172.21.15.84:0/999244531 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd5f00cee10 con 0x7fd5f8101ee0 2024-05-22T22:59:29.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.595+0000 7fd5fceb9640 1 --2- 172.21.15.84:0/999244531 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fd5e80a2b90 0x7fd5e80a5050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0314e0 tx=0x7fd5ec000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:29.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.599+0000 7fd5f5ffb640 1 -- 172.21.15.84:0/999244531 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd5f00d2030 con 0x7fd5f8101ee0 2024-05-22T22:59:29.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.822+0000 7fd5ff11d640 1 -- 172.21.15.84:0/999244531 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5f802d200 con 0x7fd5f8101ee0 2024-05-22T22:59:29.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.822+0000 7fd5f5ffb640 1 -- 172.21.15.84:0/999244531 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7fd5f0016030 con 0x7fd5f8101ee0 2024-05-22T22:59:29.823 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:29.823 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-22T22:59:29.824 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 7 2024-05-22T22:59:29.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.824+0000 7fd5ff11d640 1 -- 172.21.15.84:0/999244531 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fd5e80a2b90 msgr2=0x7fd5e80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:29.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.824+0000 7fd5ff11d640 1 --2- 172.21.15.84:0/999244531 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fd5e80a2b90 0x7fd5e80a5050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0314e0 tx=0x7fd5ec000bc0 comp rx=0 tx=0).stop 2024-05-22T22:59:29.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.825+0000 7fd5ff11d640 1 -- 172.21.15.84:0/999244531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5f8101ee0 msgr2=0x7fd5f819e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:29.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.825+0000 7fd5ff11d640 1 --2- 172.21.15.84:0/999244531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5f8101ee0 0x7fd5f819e140 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000eaa0 tx=0x7fd5f000ef60 comp rx=0 tx=0).stop 2024-05-22T22:59:29.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.825+0000 7fd5ff11d640 1 -- 172.21.15.84:0/999244531 shutdown_connections 2024-05-22T22:59:29.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.825+0000 7fd5ff11d640 1 --2- 172.21.15.84:0/999244531 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fd5e80a2b90 0x7fd5e80a5050 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:29.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.825+0000 7fd5ff11d640 1 --2- 172.21.15.84:0/999244531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5f8101ee0 0x7fd5f819e140 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:29.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.825+0000 7fd5ff11d640 1 --2- 172.21.15.84:0/999244531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5f8101460 0x7fd5f81970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:29.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.825+0000 7fd5ff11d640 1 --2- 172.21.15.84:0/999244531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5f8100aa0 0x7fd5f8196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:29.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.825+0000 7fd5ff11d640 1 -- 172.21.15.84:0/999244531 >> 172.21.15.84:0/999244531 conn(0x7fd5f80fb550 msgr2=0x7fd5f8104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:29.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.825+0000 7fd5ff11d640 1 -- 172.21.15.84:0/999244531 shutdown_connections 2024-05-22T22:59:29.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:29.825+0000 7fd5ff11d640 1 -- 172.21.15.84:0/999244531 wait complete. 2024-05-22T22:59:30.123 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:29 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:30.124 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:29 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:30.124 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:29 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:30.124 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:29 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:30.124 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:29 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:30.124 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:29 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:30.124 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:29 smithi084 ceph-mon[32186]: Deploying daemon mds.c on smithi204 2024-05-22T22:59:30.124 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:29 smithi084 ceph-mon[32186]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:30.124 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:29 smithi084 ceph-mon[32186]: mds.? [v2:172.21.15.204:6834/2890595273,v1:172.21.15.204:6835/2890595273] up:boot 2024-05-22T22:59:30.124 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:29 smithi084 ceph-mon[32186]: fsmap 6 up:standby 2024-05-22T22:59:30.124 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:29 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-22T22:59:30.124 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:29 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/999244531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:30.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:29 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:30.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:29 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:30.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:29 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:30.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:29 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:30.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:29 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:30.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:29 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:30.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:29 smithi204 ceph-mon[36211]: Deploying daemon mds.c on smithi204 2024-05-22T22:59:30.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:29 smithi204 ceph-mon[36211]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:30.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:29 smithi204 ceph-mon[36211]: mds.? [v2:172.21.15.204:6834/2890595273,v1:172.21.15.204:6835/2890595273] up:boot 2024-05-22T22:59:30.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:29 smithi204 ceph-mon[36211]: fsmap 6 up:standby 2024-05-22T22:59:30.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:29 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-22T22:59:30.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:29 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/999244531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:30.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:29 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:30.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:29 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:30.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:29 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:30.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:29 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:30.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:29 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:30.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:29 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:30.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:29 smithi096 ceph-mon[36392]: Deploying daemon mds.c on smithi204 2024-05-22T22:59:30.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:29 smithi096 ceph-mon[36392]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:30.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:29 smithi096 ceph-mon[36392]: mds.? [v2:172.21.15.204:6834/2890595273,v1:172.21.15.204:6835/2890595273] up:boot 2024-05-22T22:59:30.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:29 smithi096 ceph-mon[36392]: fsmap 6 up:standby 2024-05-22T22:59:30.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:29 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-22T22:59:30.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:29 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/999244531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:31.322 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 22 22:59:31 smithi204 systemd[1]: Starting Ceph mds.c for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T22:59:31.638 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 22 22:59:31 smithi204 podman[66631]: 2024-05-22 22:59:31.225445424 +0000 UTC m=+0.017374712 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T22:59:31.639 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 22 22:59:31 smithi204 podman[66631]: 2024-05-22 22:59:31.361167035 +0000 UTC m=+0.153096299 container create c208e68ab1edc37670ce9bc65e707262cd199913cd45d765f479797316793510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-2e1f762, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-22T22:59:31.639 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 22 22:59:31 smithi204 podman[66631]: 2024-05-22 22:59:31.639629853 +0000 UTC m=+0.431559122 container init c208e68ab1edc37670ce9bc65e707262cd199913cd45d765f479797316793510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-c, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-2e1f762, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-22T22:59:31.836 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T22:59:31.909 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 22 22:59:31 smithi204 podman[66631]: 2024-05-22 22:59:31.642476549 +0000 UTC m=+0.434405823 container start c208e68ab1edc37670ce9bc65e707262cd199913cd45d765f479797316793510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-2e1f762, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-22T22:59:31.909 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 22 22:59:31 smithi204 ceph-mds[66650]: -- 172.21.15.204:0/2867275414 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559d3f6b7a40 con 0x559d3f68b000 2024-05-22T22:59:31.909 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 22 22:59:31 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-c[66646]: starting mds.c at 2024-05-22T22:59:31.909 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 22 22:59:31 smithi204 bash[66631]: c208e68ab1edc37670ce9bc65e707262cd199913cd45d765f479797316793510 2024-05-22T22:59:31.909 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 22 22:59:31 smithi204 systemd[1]: Started Ceph mds.c for c1135a12-188d-11ef-bc9a-c7b262605968. 2024-05-22T22:59:31.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.948+0000 7f5dcda12640 1 -- 172.21.15.84:0/2867345632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dc8102330 msgr2=0x7f5dc8104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:31.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.948+0000 7f5dcda12640 1 --2- 172.21.15.84:0/2867345632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dc8102330 0x7f5dc8104720 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009900 tx=0x7f5db802a410 comp rx=0 tx=0).stop 2024-05-22T22:59:31.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.948+0000 7f5dcda12640 1 -- 172.21.15.84:0/2867345632 shutdown_connections 2024-05-22T22:59:31.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.948+0000 7f5dcda12640 1 --2- 172.21.15.84:0/2867345632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5dc8107590 0x7f5dc81099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:31.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.948+0000 7f5dcda12640 1 --2- 172.21.15.84:0/2867345632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dc8104c60 0x7f5dc8107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:31.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.948+0000 7f5dcda12640 1 --2- 172.21.15.84:0/2867345632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dc8102330 0x7f5dc8104720 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:31.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.948+0000 7f5dcda12640 1 -- 172.21.15.84:0/2867345632 >> 172.21.15.84:0/2867345632 conn(0x7f5dc80fbd20 msgr2=0x7f5dc80fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:31.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.948+0000 7f5dcda12640 1 -- 172.21.15.84:0/2867345632 shutdown_connections 2024-05-22T22:59:31.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.948+0000 7f5dcda12640 1 -- 172.21.15.84:0/2867345632 wait complete. 2024-05-22T22:59:31.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.949+0000 7f5dcda12640 1 Processor -- start 2024-05-22T22:59:31.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.949+0000 7f5dcda12640 1 -- start start 2024-05-22T22:59:31.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.949+0000 7f5dcda12640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dc8102330 0x7f5dc8197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:31.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.949+0000 7f5dcda12640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dc8104c60 0x7f5dc8198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:31.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.949+0000 7f5dcda12640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5dc8107590 0x7f5dc819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:31.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.949+0000 7f5dcda12640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc810d530 con 0x7f5dc8104c60 2024-05-22T22:59:31.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.949+0000 7f5dcda12640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc810d3b0 con 0x7f5dc8107590 2024-05-22T22:59:31.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.949+0000 7f5dcda12640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc810d6b0 con 0x7f5dc8102330 2024-05-22T22:59:31.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.949+0000 7f5dbffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dc8104c60 0x7f5dc8198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:31.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.949+0000 7f5dbffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dc8104c60 0x7f5dc8198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50976/0 (socket says 172.21.15.84:50976) 2024-05-22T22:59:31.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.949+0000 7f5dbffff640 1 -- 172.21.15.84:0/2243298601 learned_addr learned my addr 172.21.15.84:0/2243298601 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:31.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.949+0000 7f5dc77fe640 1 --2- 172.21.15.84:0/2243298601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5dc8107590 0x7f5dc819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:31.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.949+0000 7f5dc6ffd640 1 --2- 172.21.15.84:0/2243298601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dc8102330 0x7f5dc8197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:31.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.949+0000 7f5dbffff640 1 -- 172.21.15.84:0/2243298601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dc8102330 msgr2=0x7f5dc8197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:31.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.949+0000 7f5dbffff640 1 --2- 172.21.15.84:0/2243298601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dc8102330 0x7f5dc8197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:31.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.949+0000 7f5dbffff640 1 -- 172.21.15.84:0/2243298601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5dc8107590 msgr2=0x7f5dc819f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:31.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.949+0000 7f5dbffff640 1 --2- 172.21.15.84:0/2243298601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5dc8107590 0x7f5dc819f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:31.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.949+0000 7f5dbffff640 1 -- 172.21.15.84:0/2243298601 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc819f830 con 0x7f5dc8104c60 2024-05-22T22:59:31.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.949+0000 7f5dc77fe640 1 --2- 172.21.15.84:0/2243298601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5dc8107590 0x7f5dc819f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T22:59:31.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.949+0000 7f5dbffff640 1 --2- 172.21.15.84:0/2243298601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dc8104c60 0x7f5dc8198160 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5db000d770 tx=0x7f5db000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:31.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.950+0000 7f5dcca10640 1 -- 172.21.15.84:0/2243298601 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0017070 con 0x7f5dc8104c60 2024-05-22T22:59:31.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.950+0000 7f5dcca10640 1 -- 172.21.15.84:0/2243298601 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5db0009c30 con 0x7f5dc8104c60 2024-05-22T22:59:31.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.950+0000 7f5dcca10640 1 -- 172.21.15.84:0/2243298601 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db000f620 con 0x7f5dc8104c60 2024-05-22T22:59:31.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.950+0000 7f5dcda12640 1 -- 172.21.15.84:0/2243298601 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc819fb20 con 0x7f5dc8104c60 2024-05-22T22:59:31.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.950+0000 7f5dcda12640 1 -- 172.21.15.84:0/2243298601 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc81a00c0 con 0x7f5dc8104c60 2024-05-22T22:59:31.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.951+0000 7f5dcda12640 1 -- 172.21.15.84:0/2243298601 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dc80fd810 con 0x7f5dc8104c60 2024-05-22T22:59:31.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.952+0000 7f5dcca10640 1 -- 172.21.15.84:0/2243298601 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5db001a070 con 0x7f5dc8104c60 2024-05-22T22:59:31.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.953+0000 7f5dcca10640 1 --2- 172.21.15.84:0/2243298601 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f5db40a2c10 0x7f5db40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:31.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.953+0000 7f5dc6ffd640 1 --2- 172.21.15.84:0/2243298601 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f5db40a2c10 0x7f5db40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:31.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.953+0000 7f5dcca10640 1 -- 172.21.15.84:0/2243298601 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5db00ce770 con 0x7f5dc8104c60 2024-05-22T22:59:31.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.953+0000 7f5dc6ffd640 1 --2- 172.21.15.84:0/2243298601 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f5db40a2c10 0x7f5db40a50d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5db8006010 tx=0x7f5db80315d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:31.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:31.957+0000 7f5dcca10640 1 -- 172.21.15.84:0/2243298601 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5db0009de0 con 0x7f5dc8104c60 2024-05-22T22:59:32.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:32.171+0000 7f5dcda12640 1 -- 172.21.15.84:0/2243298601 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5dc802d3b0 con 0x7f5dc8104c60 2024-05-22T22:59:32.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:32.172+0000 7f5dcca10640 1 -- 172.21.15.84:0/2243298601 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7f5db009bb70 con 0x7f5dc8104c60 2024-05-22T22:59:32.172 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:32.172 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-22T22:59:32.172 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 7 2024-05-22T22:59:32.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:32.174+0000 7f5dcda12640 1 -- 172.21.15.84:0/2243298601 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f5db40a2c10 msgr2=0x7f5db40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:32.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:32.174+0000 7f5dcda12640 1 --2- 172.21.15.84:0/2243298601 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f5db40a2c10 0x7f5db40a50d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5db8006010 tx=0x7f5db80315d0 comp rx=0 tx=0).stop 2024-05-22T22:59:32.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:32.174+0000 7f5dcda12640 1 -- 172.21.15.84:0/2243298601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dc8104c60 msgr2=0x7f5dc8198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:32.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:32.174+0000 7f5dcda12640 1 --2- 172.21.15.84:0/2243298601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dc8104c60 0x7f5dc8198160 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5db000d770 tx=0x7f5db000dc30 comp rx=0 tx=0).stop 2024-05-22T22:59:32.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:32.174+0000 7f5dcda12640 1 -- 172.21.15.84:0/2243298601 shutdown_connections 2024-05-22T22:59:32.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:32.174+0000 7f5dcda12640 1 --2- 172.21.15.84:0/2243298601 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f5db40a2c10 0x7f5db40a50d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:32.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:32.174+0000 7f5dcda12640 1 --2- 172.21.15.84:0/2243298601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5dc8107590 0x7f5dc819f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:32.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:32.174+0000 7f5dcda12640 1 --2- 172.21.15.84:0/2243298601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dc8104c60 0x7f5dc8198160 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:32.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:32.174+0000 7f5dcda12640 1 --2- 172.21.15.84:0/2243298601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dc8102330 0x7f5dc8197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:32.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:32.174+0000 7f5dcda12640 1 -- 172.21.15.84:0/2243298601 >> 172.21.15.84:0/2243298601 conn(0x7f5dc80fbd20 msgr2=0x7f5dc8102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:32.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:32.174+0000 7f5dcda12640 1 -- 172.21.15.84:0/2243298601 shutdown_connections 2024-05-22T22:59:32.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:32.174+0000 7f5dcda12640 1 -- 172.21.15.84:0/2243298601 wait complete. 2024-05-22T22:59:32.699 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:32 smithi096 ceph-mon[36392]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:32.699 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:32 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:32.699 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:32 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:32.699 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:32 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:32.699 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:32 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:32.699 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:32 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:32.700 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:32 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:32.700 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:32 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2243298601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:32.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:32 smithi084 ceph-mon[32186]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:32.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:32 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:32.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:32 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:32.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:32 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:32.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:32 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:32.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:32 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:32.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:32 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:32.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:32 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2243298601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:32.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:32 smithi204 ceph-mon[36211]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:32.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:32 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:32.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:32 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:32.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:32 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:32.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:32 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:32.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:32 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:32.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:32 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:32.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:32 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2243298601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:33.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:33 smithi084 ceph-mon[32186]: Deploying daemon mds.h on smithi096 2024-05-22T22:59:33.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:33 smithi084 ceph-mon[32186]: mds.? [v2:172.21.15.204:6836/3662981631,v1:172.21.15.204:6837/3662981631] up:boot 2024-05-22T22:59:33.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:33 smithi084 ceph-mon[32186]: fsmap 7 up:standby 2024-05-22T22:59:33.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:33 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-22T22:59:33.876 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:33 smithi096 ceph-mon[36392]: Deploying daemon mds.h on smithi096 2024-05-22T22:59:33.876 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:33 smithi096 ceph-mon[36392]: mds.? [v2:172.21.15.204:6836/3662981631,v1:172.21.15.204:6837/3662981631] up:boot 2024-05-22T22:59:33.876 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:33 smithi096 ceph-mon[36392]: fsmap 7 up:standby 2024-05-22T22:59:33.876 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:33 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-22T22:59:33.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:33 smithi204 ceph-mon[36211]: Deploying daemon mds.h on smithi096 2024-05-22T22:59:33.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:33 smithi204 ceph-mon[36211]: mds.? [v2:172.21.15.204:6836/3662981631,v1:172.21.15.204:6837/3662981631] up:boot 2024-05-22T22:59:33.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:33 smithi204 ceph-mon[36211]: fsmap 7 up:standby 2024-05-22T22:59:33.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:33 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-22T22:59:34.185 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T22:59:34.255 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 22 22:59:34 smithi096 systemd[1]: Starting Ceph mds.h for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T22:59:34.300 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.298+0000 7f8c47058640 1 -- 172.21.15.84:0/195124632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c40102ad0 msgr2=0x7f8c40102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:34.300 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.298+0000 7f8c47058640 1 --2- 172.21.15.84:0/195124632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c40102ad0 0x7f8c40102f50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009900 tx=0x7f8c2c02a410 comp rx=0 tx=0).stop 2024-05-22T22:59:34.300 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.298+0000 7f8c47058640 1 -- 172.21.15.84:0/195124632 shutdown_connections 2024-05-22T22:59:34.300 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.298+0000 7f8c47058640 1 --2- 172.21.15.84:0/195124632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c40103490 0x7f8c40109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:34.300 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.298+0000 7f8c47058640 1 --2- 172.21.15.84:0/195124632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c40102ad0 0x7f8c40102f50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:34.300 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.298+0000 7f8c47058640 1 --2- 172.21.15.84:0/195124632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c401018e0 0x7f8c40101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:34.300 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.298+0000 7f8c47058640 1 -- 172.21.15.84:0/195124632 >> 172.21.15.84:0/195124632 conn(0x7f8c400fce50 msgr2=0x7f8c400ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:34.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.298+0000 7f8c47058640 1 -- 172.21.15.84:0/195124632 shutdown_connections 2024-05-22T22:59:34.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.298+0000 7f8c47058640 1 -- 172.21.15.84:0/195124632 wait complete. 2024-05-22T22:59:34.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.298+0000 7f8c47058640 1 Processor -- start 2024-05-22T22:59:34.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.298+0000 7f8c47058640 1 -- start start 2024-05-22T22:59:34.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.299+0000 7f8c47058640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c401018e0 0x7f8c40075620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:34.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.299+0000 7f8c47058640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c40102ad0 0x7f8c40073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:34.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.299+0000 7f8c47058640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c40103490 0x7f8c400741a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:34.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.299+0000 7f8c47058640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4010d540 con 0x7f8c40102ad0 2024-05-22T22:59:34.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.299+0000 7f8c47058640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4010d3c0 con 0x7f8c401018e0 2024-05-22T22:59:34.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.299+0000 7f8c47058640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4010d6c0 con 0x7f8c40103490 2024-05-22T22:59:34.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.299+0000 7f8c3ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c40102ad0 0x7f8c40073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:34.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.299+0000 7f8c3ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c40102ad0 0x7f8c40073c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50980/0 (socket says 172.21.15.84:50980) 2024-05-22T22:59:34.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.299+0000 7f8c3ffff640 1 -- 172.21.15.84:0/3548761147 learned_addr learned my addr 172.21.15.84:0/3548761147 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:34.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.299+0000 7f8c44df4640 1 --2- 172.21.15.84:0/3548761147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c401018e0 0x7f8c40075620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:34.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.299+0000 7f8c455f5640 1 --2- 172.21.15.84:0/3548761147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c40103490 0x7f8c400741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:34.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.299+0000 7f8c3ffff640 1 -- 172.21.15.84:0/3548761147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c40103490 msgr2=0x7f8c400741a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:34.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.299+0000 7f8c3ffff640 1 --2- 172.21.15.84:0/3548761147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c40103490 0x7f8c400741a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:34.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.299+0000 7f8c3ffff640 1 -- 172.21.15.84:0/3548761147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c401018e0 msgr2=0x7f8c40075620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:34.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.299+0000 7f8c3ffff640 1 --2- 172.21.15.84:0/3548761147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c401018e0 0x7f8c40075620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:34.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.299+0000 7f8c3ffff640 1 -- 172.21.15.84:0/3548761147 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c40074a30 con 0x7f8c40102ad0 2024-05-22T22:59:34.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.299+0000 7f8c455f5640 1 --2- 172.21.15.84:0/3548761147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c40103490 0x7f8c400741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T22:59:34.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.299+0000 7f8c3ffff640 1 --2- 172.21.15.84:0/3548761147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c40102ad0 0x7f8c40073c60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009230 tx=0x7f8c2c031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:34.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.299+0000 7f8c3dffb640 1 -- 172.21.15.84:0/3548761147 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c03f030 con 0x7f8c40102ad0 2024-05-22T22:59:34.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.299+0000 7f8c47058640 1 -- 172.21.15.84:0/3548761147 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c401a1c50 con 0x7f8c40102ad0 2024-05-22T22:59:34.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.299+0000 7f8c3dffb640 1 -- 172.21.15.84:0/3548761147 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c2c003ad0 con 0x7f8c40102ad0 2024-05-22T22:59:34.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.299+0000 7f8c3dffb640 1 -- 172.21.15.84:0/3548761147 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c035070 con 0x7f8c40102ad0 2024-05-22T22:59:34.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.300+0000 7f8c47058640 1 -- 172.21.15.84:0/3548761147 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c401a21a0 con 0x7f8c40102ad0 2024-05-22T22:59:34.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.301+0000 7f8c47058640 1 -- 172.21.15.84:0/3548761147 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c40108c00 con 0x7f8c40102ad0 2024-05-22T22:59:34.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.302+0000 7f8c3dffb640 1 -- 172.21.15.84:0/3548761147 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8c2c042030 con 0x7f8c40102ad0 2024-05-22T22:59:34.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.302+0000 7f8c3dffb640 1 --2- 172.21.15.84:0/3548761147 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8c300a2b90 0x7f8c300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:34.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.303+0000 7f8c44df4640 1 --2- 172.21.15.84:0/3548761147 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8c300a2b90 0x7f8c300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:34.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.303+0000 7f8c3dffb640 1 -- 172.21.15.84:0/3548761147 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8c2c0ef520 con 0x7f8c40102ad0 2024-05-22T22:59:34.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.303+0000 7f8c44df4640 1 --2- 172.21.15.84:0/3548761147 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8c300a2b90 0x7f8c300a5050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8c40102930 tx=0x7f8c34005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:34.305 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.306+0000 7f8c3dffb640 1 -- 172.21.15.84:0/3548761147 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8c2c0bc920 con 0x7f8c40102ad0 2024-05-22T22:59:34.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.519+0000 7f8c47058640 1 -- 172.21.15.84:0/3548761147 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c4002d200 con 0x7f8c40102ad0 2024-05-22T22:59:34.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.519+0000 7f8c3dffb640 1 -- 172.21.15.84:0/3548761147 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7f8c2c0bc220 con 0x7f8c40102ad0 2024-05-22T22:59:34.520 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:34.520 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-22T22:59:34.520 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2024-05-22T22:59:34.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.521+0000 7f8c47058640 1 -- 172.21.15.84:0/3548761147 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8c300a2b90 msgr2=0x7f8c300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:34.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.521+0000 7f8c47058640 1 --2- 172.21.15.84:0/3548761147 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8c300a2b90 0x7f8c300a5050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8c40102930 tx=0x7f8c34005f90 comp rx=0 tx=0).stop 2024-05-22T22:59:34.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.521+0000 7f8c47058640 1 -- 172.21.15.84:0/3548761147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c40102ad0 msgr2=0x7f8c40073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:34.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.521+0000 7f8c47058640 1 --2- 172.21.15.84:0/3548761147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c40102ad0 0x7f8c40073c60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009230 tx=0x7f8c2c031b60 comp rx=0 tx=0).stop 2024-05-22T22:59:34.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.521+0000 7f8c47058640 1 -- 172.21.15.84:0/3548761147 shutdown_connections 2024-05-22T22:59:34.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.521+0000 7f8c47058640 1 --2- 172.21.15.84:0/3548761147 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8c300a2b90 0x7f8c300a5050 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:34.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.521+0000 7f8c47058640 1 --2- 172.21.15.84:0/3548761147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c40103490 0x7f8c400741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:34.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.521+0000 7f8c47058640 1 --2- 172.21.15.84:0/3548761147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c40102ad0 0x7f8c40073c60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:34.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.521+0000 7f8c47058640 1 --2- 172.21.15.84:0/3548761147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c401018e0 0x7f8c40075620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:34.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.521+0000 7f8c47058640 1 -- 172.21.15.84:0/3548761147 >> 172.21.15.84:0/3548761147 conn(0x7f8c400fce50 msgr2=0x7f8c40105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:34.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.521+0000 7f8c47058640 1 -- 172.21.15.84:0/3548761147 shutdown_connections 2024-05-22T22:59:34.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:34.521+0000 7f8c47058640 1 -- 172.21.15.84:0/3548761147 wait complete. 2024-05-22T22:59:34.548 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 22 22:59:34 smithi096 podman[66534]: 2024-05-22 22:59:34.374451404 +0000 UTC m=+0.017102186 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T22:59:34.548 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 22 22:59:34 smithi096 podman[66534]: 2024-05-22 22:59:34.488341539 +0000 UTC m=+0.130992345 container create 1c2d2eeaf388409b6f06ff21b6b9a354760400e5f6e3ffe9285d02560b0125b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-h, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-2e1f762, org.label-schema.build-date=20240521) 2024-05-22T22:59:34.856 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:34 smithi096 ceph-mon[36392]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:34.857 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:34 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3548761147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:34.857 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 22 22:59:34 smithi096 podman[66534]: 2024-05-22 22:59:34.757555445 +0000 UTC m=+0.400206228 container init 1c2d2eeaf388409b6f06ff21b6b9a354760400e5f6e3ffe9285d02560b0125b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-h, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-2e1f762, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-22T22:59:34.857 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 22 22:59:34 smithi096 podman[66534]: 2024-05-22 22:59:34.760228096 +0000 UTC m=+0.402878886 container start 1c2d2eeaf388409b6f06ff21b6b9a354760400e5f6e3ffe9285d02560b0125b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-h, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-2e1f762, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-22T22:59:34.857 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 22 22:59:34 smithi096 ceph-mds[66553]: -- 172.21.15.96:0/2918650392 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ac8ed7fa40 con 0x55ac8ed52c00 2024-05-22T22:59:34.857 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 22 22:59:34 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-h[66549]: starting mds.h at 2024-05-22T22:59:34.857 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 22 22:59:34 smithi096 bash[66534]: 1c2d2eeaf388409b6f06ff21b6b9a354760400e5f6e3ffe9285d02560b0125b3 2024-05-22T22:59:34.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:34 smithi084 ceph-mon[32186]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:34.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:34 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3548761147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:34.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:34 smithi204 ceph-mon[36211]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:34.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:34 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3548761147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:35.298 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 22 22:59:34 smithi096 systemd[1]: Started Ceph mds.h for c1135a12-188d-11ef-bc9a-c7b262605968. 2024-05-22T22:59:36.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:35 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:36.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:35 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:36.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:35 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:36.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:35 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:36.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:35 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:36.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:35 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:36.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:35 smithi204 ceph-mon[36211]: Deploying daemon mds.i on smithi204 2024-05-22T22:59:36.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:35 smithi204 ceph-mon[36211]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:36.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:35 smithi204 ceph-mon[36211]: mds.? [v2:172.21.15.96:6836/1191862158,v1:172.21.15.96:6837/1191862158] up:boot 2024-05-22T22:59:36.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:35 smithi204 ceph-mon[36211]: fsmap 8 up:standby 2024-05-22T22:59:36.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:35 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-22T22:59:36.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:35 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:36.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:35 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:36.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:35 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:36.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:35 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:36.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:35 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:36.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:35 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:36.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:35 smithi096 ceph-mon[36392]: Deploying daemon mds.i on smithi204 2024-05-22T22:59:36.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:35 smithi096 ceph-mon[36392]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:36.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:35 smithi096 ceph-mon[36392]: mds.? [v2:172.21.15.96:6836/1191862158,v1:172.21.15.96:6837/1191862158] up:boot 2024-05-22T22:59:36.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:35 smithi096 ceph-mon[36392]: fsmap 8 up:standby 2024-05-22T22:59:36.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:35 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-22T22:59:36.373 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:35 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:36.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:35 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:36.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:35 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:36.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:35 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:36.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:35 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:36.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:35 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:36.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:35 smithi084 ceph-mon[32186]: Deploying daemon mds.i on smithi204 2024-05-22T22:59:36.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:35 smithi084 ceph-mon[32186]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:36.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:35 smithi084 ceph-mon[32186]: mds.? [v2:172.21.15.96:6836/1191862158,v1:172.21.15.96:6837/1191862158] up:boot 2024-05-22T22:59:36.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:35 smithi084 ceph-mon[32186]: fsmap 8 up:standby 2024-05-22T22:59:36.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:35 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-22T22:59:36.533 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T22:59:36.649 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.646+0000 7f8830542640 1 -- 172.21.15.84:0/3291573662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88280ffb90 msgr2=0x7f88280fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:36.649 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.646+0000 7f8830542640 1 --2- 172.21.15.84:0/3291573662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88280ffb90 0x7f88280fffb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8824009cd0 tx=0x7f882402a8e0 comp rx=0 tx=0).stop 2024-05-22T22:59:36.649 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.647+0000 7f8830542640 1 -- 172.21.15.84:0/3291573662 shutdown_connections 2024-05-22T22:59:36.649 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.647+0000 7f8830542640 1 --2- 172.21.15.84:0/3291573662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8828103520 0x7f8828109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:36.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.647+0000 7f8830542640 1 --2- 172.21.15.84:0/3291573662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8828102b60 0x7f8828102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:36.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.647+0000 7f8830542640 1 --2- 172.21.15.84:0/3291573662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88280ffb90 0x7f88280fffb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:36.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.647+0000 7f8830542640 1 -- 172.21.15.84:0/3291573662 >> 172.21.15.84:0/3291573662 conn(0x7f88280fb4e0 msgr2=0x7f88280fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:36.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.647+0000 7f8830542640 1 -- 172.21.15.84:0/3291573662 shutdown_connections 2024-05-22T22:59:36.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.647+0000 7f8830542640 1 -- 172.21.15.84:0/3291573662 wait complete. 2024-05-22T22:59:36.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.647+0000 7f8830542640 1 Processor -- start 2024-05-22T22:59:36.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.647+0000 7f8830542640 1 -- start start 2024-05-22T22:59:36.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.647+0000 7f8830542640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88280ffb90 0x7f8828197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:36.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.647+0000 7f8830542640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8828102b60 0x7f8828198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:36.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.647+0000 7f8830542640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8828103520 0x7f882819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:36.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.647+0000 7f8830542640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f882810d560 con 0x7f88280ffb90 2024-05-22T22:59:36.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.647+0000 7f8830542640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f882810d3e0 con 0x7f8828102b60 2024-05-22T22:59:36.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.647+0000 7f8830542640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f882810d6e0 con 0x7f8828103520 2024-05-22T22:59:36.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.647+0000 7f882e2de640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88280ffb90 0x7f8828197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:36.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.647+0000 7f882e2de640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88280ffb90 0x7f8828197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50990/0 (socket says 172.21.15.84:50990) 2024-05-22T22:59:36.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.647+0000 7f882e2de640 1 -- 172.21.15.84:0/2823350163 learned_addr learned my addr 172.21.15.84:0/2823350163 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:36.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.648+0000 7f882eadf640 1 --2- 172.21.15.84:0/2823350163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8828103520 0x7f882819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:36.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.648+0000 7f882dadd640 1 --2- 172.21.15.84:0/2823350163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8828102b60 0x7f8828198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:36.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.648+0000 7f882e2de640 1 -- 172.21.15.84:0/2823350163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8828103520 msgr2=0x7f882819f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:36.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.648+0000 7f882e2de640 1 --2- 172.21.15.84:0/2823350163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8828103520 0x7f882819f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:36.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.648+0000 7f882e2de640 1 -- 172.21.15.84:0/2823350163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8828102b60 msgr2=0x7f8828198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:36.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.648+0000 7f882e2de640 1 --2- 172.21.15.84:0/2823350163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8828102b60 0x7f8828198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:36.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.648+0000 7f882e2de640 1 -- 172.21.15.84:0/2823350163 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f882819f840 con 0x7f88280ffb90 2024-05-22T22:59:36.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.648+0000 7f882eadf640 1 --2- 172.21.15.84:0/2823350163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8828103520 0x7f882819f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T22:59:36.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.648+0000 7f882e2de640 1 --2- 172.21.15.84:0/2823350163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88280ffb90 0x7f8828197c30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8824004af0 tx=0x7f8824003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:36.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.648+0000 7f881f7fe640 1 -- 172.21.15.84:0/2823350163 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824004450 con 0x7f88280ffb90 2024-05-22T22:59:36.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.648+0000 7f881f7fe640 1 -- 172.21.15.84:0/2823350163 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8824003e30 con 0x7f88280ffb90 2024-05-22T22:59:36.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.648+0000 7f8830542640 1 -- 172.21.15.84:0/2823350163 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f882819fad0 con 0x7f88280ffb90 2024-05-22T22:59:36.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.648+0000 7f881f7fe640 1 -- 172.21.15.84:0/2823350163 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882403b660 con 0x7f88280ffb90 2024-05-22T22:59:36.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.648+0000 7f8830542640 1 -- 172.21.15.84:0/2823350163 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88281a0020 con 0x7f88280ffb90 2024-05-22T22:59:36.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.649+0000 7f8830542640 1 -- 172.21.15.84:0/2823350163 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8828108db0 con 0x7f88280ffb90 2024-05-22T22:59:36.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.650+0000 7f881f7fe640 1 -- 172.21.15.84:0/2823350163 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f88240305a0 con 0x7f88280ffb90 2024-05-22T22:59:36.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.651+0000 7f881f7fe640 1 --2- 172.21.15.84:0/2823350163 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f88140a2b90 0x7f88140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:36.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.651+0000 7f882dadd640 1 --2- 172.21.15.84:0/2823350163 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f88140a2b90 0x7f88140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:36.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.651+0000 7f881f7fe640 1 -- 172.21.15.84:0/2823350163 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f882402e030 con 0x7f88280ffb90 2024-05-22T22:59:36.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.651+0000 7f882dadd640 1 --2- 172.21.15.84:0/2823350163 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f88140a2b90 0x7f88140a5050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8818005d00 tx=0x7f8818005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:36.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.654+0000 7f881f7fe640 1 -- 172.21.15.84:0/2823350163 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f88240bd7f0 con 0x7f88280ffb90 2024-05-22T22:59:36.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.867+0000 7f8830542640 1 -- 172.21.15.84:0/2823350163 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f882802d3c0 con 0x7f88280ffb90 2024-05-22T22:59:36.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.867+0000 7f881f7fe640 1 -- 172.21.15.84:0/2823350163 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7177 (secure 0 0 0) 0x7f88240bd0f0 con 0x7f88280ffb90 2024-05-22T22:59:36.867 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:36.867 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-22T22:59:36.868 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 9 2024-05-22T22:59:36.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.869+0000 7f8830542640 1 -- 172.21.15.84:0/2823350163 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f88140a2b90 msgr2=0x7f88140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:36.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.869+0000 7f8830542640 1 --2- 172.21.15.84:0/2823350163 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f88140a2b90 0x7f88140a5050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8818005d00 tx=0x7f8818005c50 comp rx=0 tx=0).stop 2024-05-22T22:59:36.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.869+0000 7f8830542640 1 -- 172.21.15.84:0/2823350163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88280ffb90 msgr2=0x7f8828197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:36.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.869+0000 7f8830542640 1 --2- 172.21.15.84:0/2823350163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88280ffb90 0x7f8828197c30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8824004af0 tx=0x7f8824003b80 comp rx=0 tx=0).stop 2024-05-22T22:59:36.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.869+0000 7f8830542640 1 -- 172.21.15.84:0/2823350163 shutdown_connections 2024-05-22T22:59:36.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.869+0000 7f8830542640 1 --2- 172.21.15.84:0/2823350163 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f88140a2b90 0x7f88140a5050 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:36.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.869+0000 7f8830542640 1 --2- 172.21.15.84:0/2823350163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8828103520 0x7f882819f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:36.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.869+0000 7f8830542640 1 --2- 172.21.15.84:0/2823350163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8828102b60 0x7f8828198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:36.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.869+0000 7f8830542640 1 --2- 172.21.15.84:0/2823350163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88280ffb90 0x7f8828197c30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:36.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.869+0000 7f8830542640 1 -- 172.21.15.84:0/2823350163 >> 172.21.15.84:0/2823350163 conn(0x7f88280fb4e0 msgr2=0x7f8828105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:36.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.869+0000 7f8830542640 1 -- 172.21.15.84:0/2823350163 shutdown_connections 2024-05-22T22:59:36.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:36.869+0000 7f8830542640 1 -- 172.21.15.84:0/2823350163 wait complete. 2024-05-22T22:59:37.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:36 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2823350163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:37.359 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:36 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2823350163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:37.360 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 22 22:59:37 smithi204 systemd[1]: Starting Ceph mds.i for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T22:59:37.373 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:36 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2823350163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:37.631 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 22 22:59:37 smithi204 podman[66935]: 2024-05-22 22:59:37.262640913 +0000 UTC m=+0.017771903 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T22:59:37.631 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 22 22:59:37 smithi204 podman[66935]: 2024-05-22 22:59:37.378932653 +0000 UTC m=+0.134063636 container create 81b84a23d555ed38fe039a3a884dc62d1e90cf5d153a81c760264d744ef1b3b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-i, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-2e1f762, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521) 2024-05-22T22:59:37.631 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 22 22:59:37 smithi204 podman[66935]: 2024-05-22 22:59:37.632032246 +0000 UTC m=+0.387163227 container init 81b84a23d555ed38fe039a3a884dc62d1e90cf5d153a81c760264d744ef1b3b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-2e1f762, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, io.buildah.version=1.33.5) 2024-05-22T22:59:37.909 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 22 22:59:37 smithi204 podman[66935]: 2024-05-22 22:59:37.63500656 +0000 UTC m=+0.390137542 container start 81b84a23d555ed38fe039a3a884dc62d1e90cf5d153a81c760264d744ef1b3b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-i, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, RELEASE=squid-2e1f762, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-05-22T22:59:37.909 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 22 22:59:37 smithi204 ceph-mds[66955]: -- 172.21.15.204:0/3120478939 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560263365a40 con 0x560263339000 2024-05-22T22:59:37.909 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 22 22:59:37 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-i[66951]: starting mds.i at 2024-05-22T22:59:37.910 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 22 22:59:37 smithi204 bash[66935]: 81b84a23d555ed38fe039a3a884dc62d1e90cf5d153a81c760264d744ef1b3b6 2024-05-22T22:59:37.910 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 22 22:59:37 smithi204 systemd[1]: Started Ceph mds.i for c1135a12-188d-11ef-bc9a-c7b262605968. 2024-05-22T22:59:38.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:37 smithi096 ceph-mon[36392]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:38.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:37 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:38.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:37 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:38.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:37 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:38.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:37 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:38.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:37 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:38.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:37 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:38.373 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:37 smithi084 ceph-mon[32186]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:38.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:37 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:38.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:37 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:38.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:37 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:38.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:37 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:38.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:37 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:38.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:37 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:38.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:37 smithi204 ceph-mon[36211]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:38.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:37 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:38.410 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:37 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:38.410 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:37 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:38.410 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:37 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:38.410 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:37 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:38.410 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:37 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:38.881 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T22:59:38.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.986+0000 7f0759110640 1 -- 172.21.15.84:0/1885367203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07541007b0 msgr2=0x7f0754100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:38.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.986+0000 7f0759110640 1 --2- 172.21.15.84:0/1885367203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07541007b0 0x7f0754100bd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0744009900 tx=0x7f074402a410 comp rx=0 tx=0).stop 2024-05-22T22:59:38.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.986+0000 7f0759110640 1 -- 172.21.15.84:0/1885367203 shutdown_connections 2024-05-22T22:59:38.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.986+0000 7f0759110640 1 --2- 172.21.15.84:0/1885367203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0754102360 0x7f0754108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:38.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.986+0000 7f0759110640 1 --2- 172.21.15.84:0/1885367203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07541019a0 0x7f0754101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:38.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.986+0000 7f0759110640 1 --2- 172.21.15.84:0/1885367203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07541007b0 0x7f0754100bd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:38.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.986+0000 7f0759110640 1 -- 172.21.15.84:0/1885367203 >> 172.21.15.84:0/1885367203 conn(0x7f07540fbd20 msgr2=0x7f07540fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:38.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.986+0000 7f0759110640 1 -- 172.21.15.84:0/1885367203 shutdown_connections 2024-05-22T22:59:38.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.986+0000 7f0759110640 1 -- 172.21.15.84:0/1885367203 wait complete. 2024-05-22T22:59:38.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.987+0000 7f0759110640 1 Processor -- start 2024-05-22T22:59:38.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.987+0000 7f0759110640 1 -- start start 2024-05-22T22:59:38.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.987+0000 7f0759110640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07541007b0 0x7f0754197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:38.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.987+0000 7f0759110640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07541019a0 0x7f0754198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:38.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.987+0000 7f0759110640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0754102360 0x7f075419f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:38.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.987+0000 7f0759110640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f075410d610 con 0x7f0754102360 2024-05-22T22:59:38.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.987+0000 7f0759110640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f075410d490 con 0x7f07541007b0 2024-05-22T22:59:38.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.987+0000 7f0759110640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f075410d790 con 0x7f07541019a0 2024-05-22T22:59:38.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.987+0000 7f075359e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0754102360 0x7f075419f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:38.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.987+0000 7f075359e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0754102360 0x7f075419f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51014/0 (socket says 172.21.15.84:51014) 2024-05-22T22:59:38.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.987+0000 7f075359e640 1 -- 172.21.15.84:0/1314147577 learned_addr learned my addr 172.21.15.84:0/1314147577 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:38.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.987+0000 7f0752d9d640 1 --2- 172.21.15.84:0/1314147577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07541007b0 0x7f0754197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:38.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.987+0000 7f075259c640 1 --2- 172.21.15.84:0/1314147577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07541019a0 0x7f0754198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:38.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.987+0000 7f075359e640 1 -- 172.21.15.84:0/1314147577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07541019a0 msgr2=0x7f0754198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:38.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.987+0000 7f075359e640 1 --2- 172.21.15.84:0/1314147577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07541019a0 0x7f0754198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:38.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.988+0000 7f075359e640 1 -- 172.21.15.84:0/1314147577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07541007b0 msgr2=0x7f0754197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:38.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.988+0000 7f075359e640 1 --2- 172.21.15.84:0/1314147577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07541007b0 0x7f0754197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:38.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.988+0000 7f075359e640 1 -- 172.21.15.84:0/1314147577 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075419f960 con 0x7f0754102360 2024-05-22T22:59:38.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.988+0000 7f075259c640 1 --2- 172.21.15.84:0/1314147577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07541019a0 0x7f0754198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T22:59:38.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.988+0000 7f075359e640 1 --2- 172.21.15.84:0/1314147577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0754102360 0x7f075419f2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f074c00b970 tx=0x7f074c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:38.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.988+0000 7f073bfff640 1 -- 172.21.15.84:0/1314147577 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c019070 con 0x7f0754102360 2024-05-22T22:59:38.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.988+0000 7f073bfff640 1 -- 172.21.15.84:0/1314147577 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f074c00cb40 con 0x7f0754102360 2024-05-22T22:59:38.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.988+0000 7f0759110640 1 -- 172.21.15.84:0/1314147577 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075419fc50 con 0x7f0754102360 2024-05-22T22:59:38.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.988+0000 7f073bfff640 1 -- 172.21.15.84:0/1314147577 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c007650 con 0x7f0754102360 2024-05-22T22:59:38.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.988+0000 7f0759110640 1 -- 172.21.15.84:0/1314147577 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07541a01a0 con 0x7f0754102360 2024-05-22T22:59:38.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.989+0000 7f0759110640 1 -- 172.21.15.84:0/1314147577 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0754107ba0 con 0x7f0754102360 2024-05-22T22:59:38.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.990+0000 7f073bfff640 1 -- 172.21.15.84:0/1314147577 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f074c01c070 con 0x7f0754102360 2024-05-22T22:59:38.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.991+0000 7f073bfff640 1 --2- 172.21.15.84:0/1314147577 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f073c0a2b90 0x7f073c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:38.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.991+0000 7f0752d9d640 1 --2- 172.21.15.84:0/1314147577 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f073c0a2b90 0x7f073c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:38.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.991+0000 7f073bfff640 1 -- 172.21.15.84:0/1314147577 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f074c0ce130 con 0x7f0754102360 2024-05-22T22:59:38.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.991+0000 7f0752d9d640 1 --2- 172.21.15.84:0/1314147577 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f073c0a2b90 0x7f073c0a5050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0744005ab0 tx=0x7f0744000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:38.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:38.995+0000 7f073bfff640 1 -- 172.21.15.84:0/1314147577 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f074c09aee0 con 0x7f0754102360 2024-05-22T22:59:39.123 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:38 smithi084 ceph-mon[32186]: Deploying daemon mds.g on smithi084 2024-05-22T22:59:39.124 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:38 smithi084 ceph-mon[32186]: mds.? [v2:172.21.15.204:6838/3328293038,v1:172.21.15.204:6839/3328293038] up:boot 2024-05-22T22:59:39.124 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:38 smithi084 ceph-mon[32186]: fsmap 9 up:standby 2024-05-22T22:59:39.124 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:38 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-22T22:59:39.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:39.213+0000 7f0759110640 1 -- 172.21.15.84:0/1314147577 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f075402d3d0 con 0x7f0754102360 2024-05-22T22:59:39.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:39.213+0000 7f073bfff640 1 -- 172.21.15.84:0/1314147577 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7f074c09f1e0 con 0x7f0754102360 2024-05-22T22:59:39.214 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:39.214 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-22T22:59:39.214 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 10 2024-05-22T22:59:39.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:39.215+0000 7f0759110640 1 -- 172.21.15.84:0/1314147577 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f073c0a2b90 msgr2=0x7f073c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:39.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:39.215+0000 7f0759110640 1 --2- 172.21.15.84:0/1314147577 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f073c0a2b90 0x7f073c0a5050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0744005ab0 tx=0x7f0744000bc0 comp rx=0 tx=0).stop 2024-05-22T22:59:39.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:39.215+0000 7f0759110640 1 -- 172.21.15.84:0/1314147577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0754102360 msgr2=0x7f075419f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:39.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:39.215+0000 7f0759110640 1 --2- 172.21.15.84:0/1314147577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0754102360 0x7f075419f2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f074c00b970 tx=0x7f074c00be30 comp rx=0 tx=0).stop 2024-05-22T22:59:39.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:39.215+0000 7f0759110640 1 -- 172.21.15.84:0/1314147577 shutdown_connections 2024-05-22T22:59:39.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:39.215+0000 7f0759110640 1 --2- 172.21.15.84:0/1314147577 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f073c0a2b90 0x7f073c0a5050 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:39.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:39.215+0000 7f0759110640 1 --2- 172.21.15.84:0/1314147577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0754102360 0x7f075419f2f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:39.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:39.215+0000 7f0759110640 1 --2- 172.21.15.84:0/1314147577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07541019a0 0x7f0754198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:39.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:39.215+0000 7f0759110640 1 --2- 172.21.15.84:0/1314147577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07541007b0 0x7f0754197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:39.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:39.215+0000 7f0759110640 1 -- 172.21.15.84:0/1314147577 >> 172.21.15.84:0/1314147577 conn(0x7f07540fbd20 msgr2=0x7f0754104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:39.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:39.216+0000 7f0759110640 1 -- 172.21.15.84:0/1314147577 shutdown_connections 2024-05-22T22:59:39.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:39.216+0000 7f0759110640 1 -- 172.21.15.84:0/1314147577 wait complete. 2024-05-22T22:59:39.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:38 smithi096 ceph-mon[36392]: Deploying daemon mds.g on smithi084 2024-05-22T22:59:39.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:38 smithi096 ceph-mon[36392]: mds.? [v2:172.21.15.204:6838/3328293038,v1:172.21.15.204:6839/3328293038] up:boot 2024-05-22T22:59:39.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:38 smithi096 ceph-mon[36392]: fsmap 9 up:standby 2024-05-22T22:59:39.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:38 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-22T22:59:39.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:38 smithi204 ceph-mon[36211]: Deploying daemon mds.g on smithi084 2024-05-22T22:59:39.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:38 smithi204 ceph-mon[36211]: mds.? [v2:172.21.15.204:6838/3328293038,v1:172.21.15.204:6839/3328293038] up:boot 2024-05-22T22:59:39.410 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:38 smithi204 ceph-mon[36211]: fsmap 9 up:standby 2024-05-22T22:59:39.410 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:38 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-22T22:59:40.123 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:39 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1314147577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:40.124 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:39 smithi084 ceph-mon[32186]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:40.124 INFO:journalctl@ceph.mds.g.smithi084.stdout:May 22 22:59:39 smithi084 systemd[1]: Starting Ceph mds.g for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T22:59:40.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:39 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1314147577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:40.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:39 smithi096 ceph-mon[36392]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:40.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:39 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1314147577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:40.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:39 smithi204 ceph-mon[36211]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:40.474 INFO:journalctl@ceph.mds.g.smithi084.stdout:May 22 22:59:40 smithi084 podman[70966]: 2024-05-22 22:59:40.100780568 +0000 UTC m=+0.017186287 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T22:59:40.475 INFO:journalctl@ceph.mds.g.smithi084.stdout:May 22 22:59:40 smithi084 podman[70966]: 2024-05-22 22:59:40.212411113 +0000 UTC m=+0.128816849 container create dfa016465ba0dedaee560b8bf48747f3e208d696e262d619ba7c82eef82cd365 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-g, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-2e1f762, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-22T22:59:40.777 INFO:journalctl@ceph.mds.g.smithi084.stdout:May 22 22:59:40 smithi084 podman[70966]: 2024-05-22 22:59:40.475638287 +0000 UTC m=+0.392044011 container init dfa016465ba0dedaee560b8bf48747f3e208d696e262d619ba7c82eef82cd365 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-2e1f762, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-05-22T22:59:40.777 INFO:journalctl@ceph.mds.g.smithi084.stdout:May 22 22:59:40 smithi084 podman[70966]: 2024-05-22 22:59:40.479418114 +0000 UTC m=+0.395823840 container start dfa016465ba0dedaee560b8bf48747f3e208d696e262d619ba7c82eef82cd365 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-g, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, RELEASE=squid-2e1f762, org.label-schema.schema-version=1.0) 2024-05-22T22:59:40.777 INFO:journalctl@ceph.mds.g.smithi084.stdout:May 22 22:59:40 smithi084 ceph-mds[70985]: -- 172.21.15.84:0/3744152998 <== mon.0 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5641ddae7a40 con 0x5641ddabac00 2024-05-22T22:59:40.778 INFO:journalctl@ceph.mds.g.smithi084.stdout:May 22 22:59:40 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-g[70981]: starting mds.g at 2024-05-22T22:59:40.778 INFO:journalctl@ceph.mds.g.smithi084.stdout:May 22 22:59:40 smithi084 bash[70966]: dfa016465ba0dedaee560b8bf48747f3e208d696e262d619ba7c82eef82cd365 2024-05-22T22:59:40.778 INFO:journalctl@ceph.mds.g.smithi084.stdout:May 22 22:59:40 smithi084 systemd[1]: Started Ceph mds.g for c1135a12-188d-11ef-bc9a-c7b262605968. 2024-05-22T22:59:41.228 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T22:59:41.337 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.335+0000 7f77059d5640 1 -- 172.21.15.84:0/1941530794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7700102360 msgr2=0x7f7700108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:41.337 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.335+0000 7f77059d5640 1 --2- 172.21.15.84:0/1941530794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7700102360 0x7f7700108bf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009ca0 tx=0x7f76f802a8c0 comp rx=0 tx=0).stop 2024-05-22T22:59:41.337 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.335+0000 7f77059d5640 1 -- 172.21.15.84:0/1941530794 shutdown_connections 2024-05-22T22:59:41.337 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.335+0000 7f77059d5640 1 --2- 172.21.15.84:0/1941530794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7700102360 0x7f7700108bf0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:41.337 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.335+0000 7f77059d5640 1 --2- 172.21.15.84:0/1941530794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77001019a0 0x7f7700101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:41.337 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.335+0000 7f77059d5640 1 --2- 172.21.15.84:0/1941530794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77001007b0 0x7f7700100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.335+0000 7f77059d5640 1 -- 172.21.15.84:0/1941530794 >> 172.21.15.84:0/1941530794 conn(0x7f77000fbd20 msgr2=0x7f77000fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.335+0000 7f77059d5640 1 -- 172.21.15.84:0/1941530794 shutdown_connections 2024-05-22T22:59:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.335+0000 7f77059d5640 1 -- 172.21.15.84:0/1941530794 wait complete. 2024-05-22T22:59:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.336+0000 7f77059d5640 1 Processor -- start 2024-05-22T22:59:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.336+0000 7f77059d5640 1 -- start start 2024-05-22T22:59:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.336+0000 7f77059d5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77001007b0 0x7f7700069570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.336+0000 7f77059d5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77001019a0 0x7f770019c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.336+0000 7f77059d5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7700102360 0x7f770019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.336+0000 7f77059d5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f770010c430 con 0x7f77001007b0 2024-05-22T22:59:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.336+0000 7f77059d5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f770010c2b0 con 0x7f77001019a0 2024-05-22T22:59:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.336+0000 7f77059d5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f770010c5b0 con 0x7f7700102360 2024-05-22T22:59:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.336+0000 7f76feffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77001007b0 0x7f7700069570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.336+0000 7f76feffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77001007b0 0x7f7700069570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51054/0 (socket says 172.21.15.84:51054) 2024-05-22T22:59:41.339 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.336+0000 7f76feffd640 1 -- 172.21.15.84:0/3427159418 learned_addr learned my addr 172.21.15.84:0/3427159418 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:41.339 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.336+0000 7f76ff7fe640 1 --2- 172.21.15.84:0/3427159418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7700102360 0x7f770019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:41.339 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.336+0000 7f76f7fff640 1 --2- 172.21.15.84:0/3427159418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77001019a0 0x7f770019c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:41.339 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.336+0000 7f76feffd640 1 -- 172.21.15.84:0/3427159418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7700102360 msgr2=0x7f770019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:41.339 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.336+0000 7f76feffd640 1 --2- 172.21.15.84:0/3427159418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7700102360 0x7f770019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:41.339 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.336+0000 7f76feffd640 1 -- 172.21.15.84:0/3427159418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77001019a0 msgr2=0x7f770019c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:41.339 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.336+0000 7f76feffd640 1 --2- 172.21.15.84:0/3427159418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77001019a0 0x7f770019c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:41.339 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.336+0000 7f76feffd640 1 -- 172.21.15.84:0/3427159418 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770019f7f0 con 0x7f77001007b0 2024-05-22T22:59:41.339 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.337+0000 7f76feffd640 1 --2- 172.21.15.84:0/3427159418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77001007b0 0x7f7700069570 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f76f000e9c0 tx=0x7f76f000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:41.339 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.337+0000 7f77049d3640 1 -- 172.21.15.84:0/3427159418 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f000cd80 con 0x7f77001007b0 2024-05-22T22:59:41.339 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.337+0000 7f77059d5640 1 -- 172.21.15.84:0/3427159418 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f770019fa80 con 0x7f77001007b0 2024-05-22T22:59:41.339 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.337+0000 7f77049d3640 1 -- 172.21.15.84:0/3427159418 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76f00035f0 con 0x7f77001007b0 2024-05-22T22:59:41.339 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.337+0000 7f77049d3640 1 -- 172.21.15.84:0/3427159418 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f0010690 con 0x7f77001007b0 2024-05-22T22:59:41.339 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.337+0000 7f77059d5640 1 -- 172.21.15.84:0/3427159418 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f770019ffd0 con 0x7f77001007b0 2024-05-22T22:59:41.340 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.338+0000 7f77059d5640 1 -- 172.21.15.84:0/3427159418 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7700107ad0 con 0x7f77001007b0 2024-05-22T22:59:41.340 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.339+0000 7f77049d3640 1 -- 172.21.15.84:0/3427159418 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f76f00037a0 con 0x7f77001007b0 2024-05-22T22:59:41.340 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.339+0000 7f77049d3640 1 --2- 172.21.15.84:0/3427159418 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f76ec0a2b90 0x7f76ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:41.341 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.340+0000 7f76f7fff640 1 --2- 172.21.15.84:0/3427159418 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f76ec0a2b90 0x7f76ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:41.341 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.340+0000 7f77049d3640 1 -- 172.21.15.84:0/3427159418 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f76f00cf0f0 con 0x7f77001007b0 2024-05-22T22:59:41.341 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.340+0000 7f76f7fff640 1 --2- 172.21.15.84:0/3427159418 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f76ec0a2b90 0x7f76ec0a5050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f76e8005fd0 tx=0x7f76e8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:41.344 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.344+0000 7f77049d3640 1 -- 172.21.15.84:0/3427159418 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f76f009c4f0 con 0x7f77001007b0 2024-05-22T22:59:41.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.571+0000 7f77059d5640 1 -- 172.21.15.84:0/3427159418 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f770002d200 con 0x7f77001007b0 2024-05-22T22:59:41.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.572+0000 7f77049d3640 1 -- 172.21.15.84:0/3427159418 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8830 (secure 0 0 0) 0x7f76f009bdf0 con 0x7f77001007b0 2024-05-22T22:59:41.572 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:41.572 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-22T22:59:41.572 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 11 2024-05-22T22:59:41.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.574+0000 7f77059d5640 1 -- 172.21.15.84:0/3427159418 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f76ec0a2b90 msgr2=0x7f76ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:41.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.574+0000 7f77059d5640 1 --2- 172.21.15.84:0/3427159418 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f76ec0a2b90 0x7f76ec0a5050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f76e8005fd0 tx=0x7f76e8005850 comp rx=0 tx=0).stop 2024-05-22T22:59:41.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.574+0000 7f77059d5640 1 -- 172.21.15.84:0/3427159418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77001007b0 msgr2=0x7f7700069570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:41.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.574+0000 7f77059d5640 1 --2- 172.21.15.84:0/3427159418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77001007b0 0x7f7700069570 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f76f000e9c0 tx=0x7f76f000ee80 comp rx=0 tx=0).stop 2024-05-22T22:59:41.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.574+0000 7f77059d5640 1 -- 172.21.15.84:0/3427159418 shutdown_connections 2024-05-22T22:59:41.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.574+0000 7f77059d5640 1 --2- 172.21.15.84:0/3427159418 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f76ec0a2b90 0x7f76ec0a5050 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:41.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.574+0000 7f77059d5640 1 --2- 172.21.15.84:0/3427159418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7700102360 0x7f770019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:41.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.574+0000 7f77059d5640 1 --2- 172.21.15.84:0/3427159418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77001019a0 0x7f770019c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:41.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.574+0000 7f77059d5640 1 --2- 172.21.15.84:0/3427159418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77001007b0 0x7f7700069570 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:41.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.574+0000 7f77059d5640 1 -- 172.21.15.84:0/3427159418 >> 172.21.15.84:0/3427159418 conn(0x7f77000fbd20 msgr2=0x7f7700104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:41.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.574+0000 7f77059d5640 1 -- 172.21.15.84:0/3427159418 shutdown_connections 2024-05-22T22:59:41.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:41.574+0000 7f77059d5640 1 -- 172.21.15.84:0/3427159418 wait complete. 2024-05-22T22:59:41.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:41 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:41.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:41 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:41.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:41 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:41.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:41 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:41.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:41 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:41.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:41 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:41.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:41 smithi084 ceph-mon[32186]: Deploying daemon mds.a on smithi084 2024-05-22T22:59:41.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:41 smithi084 ceph-mon[32186]: mds.? [v2:172.21.15.84:6838/669165452,v1:172.21.15.84:6839/669165452] up:boot 2024-05-22T22:59:41.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:41 smithi084 ceph-mon[32186]: fsmap 10 up:standby 2024-05-22T22:59:41.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:41 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-22T22:59:41.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:41 smithi084 ceph-mon[32186]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:41.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:41 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3427159418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:41.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:41 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:41.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:41 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:41.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:41 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:41.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:41 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:41.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:41 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:41.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:41 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:41.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:41 smithi204 ceph-mon[36211]: Deploying daemon mds.a on smithi084 2024-05-22T22:59:41.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:41 smithi204 ceph-mon[36211]: mds.? [v2:172.21.15.84:6838/669165452,v1:172.21.15.84:6839/669165452] up:boot 2024-05-22T22:59:41.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:41 smithi204 ceph-mon[36211]: fsmap 10 up:standby 2024-05-22T22:59:41.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:41 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-22T22:59:41.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:41 smithi204 ceph-mon[36211]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:41.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:41 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3427159418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:42.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:41 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:42.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:41 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:42.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:41 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:42.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:41 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:42.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:41 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:42.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:41 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:42.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:41 smithi096 ceph-mon[36392]: Deploying daemon mds.a on smithi084 2024-05-22T22:59:42.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:41 smithi096 ceph-mon[36392]: mds.? [v2:172.21.15.84:6838/669165452,v1:172.21.15.84:6839/669165452] up:boot 2024-05-22T22:59:42.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:41 smithi096 ceph-mon[36392]: fsmap 10 up:standby 2024-05-22T22:59:42.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:41 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-22T22:59:42.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:41 smithi096 ceph-mon[36392]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:42.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:41 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3427159418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:42.850 INFO:journalctl@ceph.mds.a.smithi084.stdout:May 22 22:59:42 smithi084 systemd[1]: Starting Ceph mds.a for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T22:59:43.121 INFO:journalctl@ceph.mds.a.smithi084.stdout:May 22 22:59:42 smithi084 podman[71322]: 2024-05-22 22:59:42.753805667 +0000 UTC m=+0.016999526 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T22:59:43.122 INFO:journalctl@ceph.mds.a.smithi084.stdout:May 22 22:59:42 smithi084 podman[71322]: 2024-05-22 22:59:42.867169058 +0000 UTC m=+0.130362942 container create d06a7e93c3c95de78db4b6b266e161e4ed9facaaa6bfb952fbe18b733b3d5c29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-a, RELEASE=squid-2e1f762, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-22T22:59:43.122 INFO:journalctl@ceph.mds.a.smithi084.stdout:May 22 22:59:43 smithi084 podman[71322]: 2024-05-22 22:59:43.095032739 +0000 UTC m=+0.358226597 container init d06a7e93c3c95de78db4b6b266e161e4ed9facaaa6bfb952fbe18b733b3d5c29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-2e1f762, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-22T22:59:43.122 INFO:journalctl@ceph.mds.a.smithi084.stdout:May 22 22:59:43 smithi084 podman[71322]: 2024-05-22 22:59:43.097808749 +0000 UTC m=+0.361002595 container start d06a7e93c3c95de78db4b6b266e161e4ed9facaaa6bfb952fbe18b733b3d5c29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-2e1f762, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-22T22:59:43.123 INFO:journalctl@ceph.mds.a.smithi084.stdout:May 22 22:59:43 smithi084 ceph-mds[71341]: -- 172.21.15.84:0/2519342654 <== mon.0 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5646af1f9a40 con 0x5646af1cd000 2024-05-22T22:59:43.373 INFO:journalctl@ceph.mds.a.smithi084.stdout:May 22 22:59:43 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-a[71337]: starting mds.a at 2024-05-22T22:59:43.374 INFO:journalctl@ceph.mds.a.smithi084.stdout:May 22 22:59:43 smithi084 bash[71322]: d06a7e93c3c95de78db4b6b266e161e4ed9facaaa6bfb952fbe18b733b3d5c29 2024-05-22T22:59:43.374 INFO:journalctl@ceph.mds.a.smithi084.stdout:May 22 22:59:43 smithi084 systemd[1]: Started Ceph mds.a for c1135a12-188d-11ef-bc9a-c7b262605968. 2024-05-22T22:59:43.587 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T22:59:43.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.692+0000 7f6c641e7640 1 -- 172.21.15.84:0/3557975431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c5c1019a0 msgr2=0x7f6c5c101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:43.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.692+0000 7f6c641e7640 1 --2- 172.21.15.84:0/3557975431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c5c1019a0 0x7f6c5c101e20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c009900 tx=0x7f6c4c02a410 comp rx=0 tx=0).stop 2024-05-22T22:59:43.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.692+0000 7f6c641e7640 1 -- 172.21.15.84:0/3557975431 shutdown_connections 2024-05-22T22:59:43.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.692+0000 7f6c641e7640 1 --2- 172.21.15.84:0/3557975431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c5c102360 0x7f6c5c108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:43.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.692+0000 7f6c641e7640 1 --2- 172.21.15.84:0/3557975431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c5c1019a0 0x7f6c5c101e20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:43.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.692+0000 7f6c641e7640 1 --2- 172.21.15.84:0/3557975431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c5c1007b0 0x7f6c5c100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:43.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.692+0000 7f6c641e7640 1 -- 172.21.15.84:0/3557975431 >> 172.21.15.84:0/3557975431 conn(0x7f6c5c0fbd20 msgr2=0x7f6c5c0fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:43.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.693+0000 7f6c641e7640 1 -- 172.21.15.84:0/3557975431 shutdown_connections 2024-05-22T22:59:43.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.693+0000 7f6c641e7640 1 -- 172.21.15.84:0/3557975431 wait complete. 2024-05-22T22:59:43.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.693+0000 7f6c641e7640 1 Processor -- start 2024-05-22T22:59:43.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.693+0000 7f6c641e7640 1 -- start start 2024-05-22T22:59:43.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.693+0000 7f6c641e7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c5c1007b0 0x7f6c5c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:43.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.693+0000 7f6c641e7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c5c1019a0 0x7f6c5c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:43.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.693+0000 7f6c641e7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c5c102360 0x7f6c5c19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:43.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.693+0000 7f6c641e7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c10d550 con 0x7f6c5c1019a0 2024-05-22T22:59:43.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.693+0000 7f6c641e7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c10d3d0 con 0x7f6c5c102360 2024-05-22T22:59:43.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.693+0000 7f6c641e7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c10d6d0 con 0x7f6c5c1007b0 2024-05-22T22:59:43.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.693+0000 7f6c61782640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c5c1019a0 0x7f6c5c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:43.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.693+0000 7f6c61782640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c5c1019a0 0x7f6c5c1980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37174/0 (socket says 172.21.15.84:37174) 2024-05-22T22:59:43.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.693+0000 7f6c61782640 1 -- 172.21.15.84:0/3989862775 learned_addr learned my addr 172.21.15.84:0/3989862775 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:43.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.694+0000 7f6c61f83640 1 --2- 172.21.15.84:0/3989862775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c5c1007b0 0x7f6c5c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:43.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.694+0000 7f6c62784640 1 --2- 172.21.15.84:0/3989862775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c5c102360 0x7f6c5c19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:43.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.694+0000 7f6c61782640 1 -- 172.21.15.84:0/3989862775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c5c1007b0 msgr2=0x7f6c5c197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:43.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.694+0000 7f6c61782640 1 --2- 172.21.15.84:0/3989862775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c5c1007b0 0x7f6c5c197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:43.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.694+0000 7f6c61782640 1 -- 172.21.15.84:0/3989862775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c5c102360 msgr2=0x7f6c5c19f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:43.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.694+0000 7f6c61782640 1 --2- 172.21.15.84:0/3989862775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c5c102360 0x7f6c5c19f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:43.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.694+0000 7f6c61782640 1 -- 172.21.15.84:0/3989862775 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5c19f7a0 con 0x7f6c5c1019a0 2024-05-22T22:59:43.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.694+0000 7f6c61782640 1 --2- 172.21.15.84:0/3989862775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c5c1019a0 0x7f6c5c1980c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c009230 tx=0x7f6c4c031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:43.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.694+0000 7f6c52ffd640 1 -- 172.21.15.84:0/3989862775 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c03f030 con 0x7f6c5c1019a0 2024-05-22T22:59:43.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.694+0000 7f6c52ffd640 1 -- 172.21.15.84:0/3989862775 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c4c003ad0 con 0x7f6c5c1019a0 2024-05-22T22:59:43.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.694+0000 7f6c52ffd640 1 -- 172.21.15.84:0/3989862775 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c035070 con 0x7f6c5c1019a0 2024-05-22T22:59:43.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.694+0000 7f6c641e7640 1 -- 172.21.15.84:0/3989862775 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5c19fa30 con 0x7f6c5c1019a0 2024-05-22T22:59:43.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.694+0000 7f6c641e7640 1 -- 172.21.15.84:0/3989862775 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c5c19ff20 con 0x7f6c5c1019a0 2024-05-22T22:59:43.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.695+0000 7f6c641e7640 1 -- 172.21.15.84:0/3989862775 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c5c107ad0 con 0x7f6c5c1019a0 2024-05-22T22:59:43.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.696+0000 7f6c52ffd640 1 -- 172.21.15.84:0/3989862775 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6c4c042030 con 0x7f6c5c1019a0 2024-05-22T22:59:43.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.697+0000 7f6c52ffd640 1 --2- 172.21.15.84:0/3989862775 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f6c480a2b90 0x7f6c480a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:43.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.697+0000 7f6c61f83640 1 --2- 172.21.15.84:0/3989862775 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f6c480a2b90 0x7f6c480a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:43.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.698+0000 7f6c52ffd640 1 -- 172.21.15.84:0/3989862775 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f6c4c0ef0a0 con 0x7f6c5c1019a0 2024-05-22T22:59:43.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.698+0000 7f6c61f83640 1 --2- 172.21.15.84:0/3989862775 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f6c480a2b90 0x7f6c480a5050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c101800 tx=0x7f6c58005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:43.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.702+0000 7f6c52ffd640 1 -- 172.21.15.84:0/3989862775 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6c4c0bbe50 con 0x7f6c5c1019a0 2024-05-22T22:59:43.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.917+0000 7f6c641e7640 1 -- 172.21.15.84:0/3989862775 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c5c02d200 con 0x7f6c5c1019a0 2024-05-22T22:59:43.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.918+0000 7f6c52ffd640 1 -- 172.21.15.84:0/3989862775 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9656 (secure 0 0 0) 0x7f6c4c0c0150 con 0x7f6c5c1019a0 2024-05-22T22:59:43.918 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:43.918 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-22T22:59:43.918 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 12 2024-05-22T22:59:43.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.920+0000 7f6c641e7640 1 -- 172.21.15.84:0/3989862775 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f6c480a2b90 msgr2=0x7f6c480a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:43.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.920+0000 7f6c641e7640 1 --2- 172.21.15.84:0/3989862775 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f6c480a2b90 0x7f6c480a5050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c101800 tx=0x7f6c58005f90 comp rx=0 tx=0).stop 2024-05-22T22:59:43.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.920+0000 7f6c641e7640 1 -- 172.21.15.84:0/3989862775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c5c1019a0 msgr2=0x7f6c5c1980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:43.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.920+0000 7f6c641e7640 1 --2- 172.21.15.84:0/3989862775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c5c1019a0 0x7f6c5c1980c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c009230 tx=0x7f6c4c031b60 comp rx=0 tx=0).stop 2024-05-22T22:59:43.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.920+0000 7f6c641e7640 1 -- 172.21.15.84:0/3989862775 shutdown_connections 2024-05-22T22:59:43.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.920+0000 7f6c641e7640 1 --2- 172.21.15.84:0/3989862775 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f6c480a2b90 0x7f6c480a5050 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:43.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.920+0000 7f6c641e7640 1 --2- 172.21.15.84:0/3989862775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c5c102360 0x7f6c5c19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:43.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.920+0000 7f6c641e7640 1 --2- 172.21.15.84:0/3989862775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c5c1019a0 0x7f6c5c1980c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:43.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.920+0000 7f6c641e7640 1 --2- 172.21.15.84:0/3989862775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c5c1007b0 0x7f6c5c197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:43.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.920+0000 7f6c641e7640 1 -- 172.21.15.84:0/3989862775 >> 172.21.15.84:0/3989862775 conn(0x7f6c5c0fbd20 msgr2=0x7f6c5c104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:43.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.920+0000 7f6c641e7640 1 -- 172.21.15.84:0/3989862775 shutdown_connections 2024-05-22T22:59:43.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:43.920+0000 7f6c641e7640 1 -- 172.21.15.84:0/3989862775 wait complete. 2024-05-22T22:59:44.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:44 smithi084 ceph-mon[32186]: mds.? [v2:172.21.15.84:6840/2709540650,v1:172.21.15.84:6841/2709540650] up:boot 2024-05-22T22:59:44.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:44 smithi084 ceph-mon[32186]: fsmap 11 up:standby 2024-05-22T22:59:44.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:44 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-22T22:59:44.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:44 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:44.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:44 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:44.374 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:44 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:44.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:44 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:44.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:44 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:44.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:44 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:44.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:44 smithi084 ceph-mon[32186]: Deploying daemon mds.b on smithi096 2024-05-22T22:59:44.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:44 smithi084 ceph-mon[32186]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:44.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:44 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3989862775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:44.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:44 smithi096 ceph-mon[36392]: mds.? [v2:172.21.15.84:6840/2709540650,v1:172.21.15.84:6841/2709540650] up:boot 2024-05-22T22:59:44.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:44 smithi096 ceph-mon[36392]: fsmap 11 up:standby 2024-05-22T22:59:44.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-22T22:59:44.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:44.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:44.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:44.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:44.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:44.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:44 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:44.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:44 smithi096 ceph-mon[36392]: Deploying daemon mds.b on smithi096 2024-05-22T22:59:44.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:44 smithi096 ceph-mon[36392]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:44.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:44 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3989862775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:44.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:44 smithi204 ceph-mon[36211]: mds.? [v2:172.21.15.84:6840/2709540650,v1:172.21.15.84:6841/2709540650] up:boot 2024-05-22T22:59:44.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:44 smithi204 ceph-mon[36211]: fsmap 11 up:standby 2024-05-22T22:59:44.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:44 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-22T22:59:44.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:44 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:44.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:44 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:44.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:44 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:44.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:44 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-22T22:59:44.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:44 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-22T22:59:44.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:44 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:44.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:44 smithi204 ceph-mon[36211]: Deploying daemon mds.b on smithi096 2024-05-22T22:59:44.661 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:44 smithi204 ceph-mon[36211]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:44.661 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:44 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3989862775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:45.548 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 22 22:59:45 smithi096 systemd[1]: Starting Ceph mds.b for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T22:59:45.932 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T22:59:45.963 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 22 22:59:45 smithi096 podman[66839]: 2024-05-22 22:59:45.596415963 +0000 UTC m=+0.017229672 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T22:59:45.963 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 22 22:59:45 smithi096 podman[66839]: 2024-05-22 22:59:45.726034945 +0000 UTC m=+0.146848650 container create e4324ef0d993e90e1b81b6f1309395cbe5339294436e4a29b465bfe7c87f878b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-2e1f762, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521) 2024-05-22T22:59:45.963 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 22 22:59:45 smithi096 podman[66839]: 2024-05-22 22:59:45.936952025 +0000 UTC m=+0.357765731 container init e4324ef0d993e90e1b81b6f1309395cbe5339294436e4a29b465bfe7c87f878b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-b, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-2e1f762, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-22T22:59:45.963 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 22 22:59:45 smithi096 podman[66839]: 2024-05-22 22:59:45.93955148 +0000 UTC m=+0.360365185 container start e4324ef0d993e90e1b81b6f1309395cbe5339294436e4a29b465bfe7c87f878b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-2e1f762, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-05-22T22:59:45.963 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 22 22:59:45 smithi096 ceph-mds[66858]: -- 172.21.15.96:0/2007769165 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559314e05a40 con 0x559314dd8c00 2024-05-22T22:59:46.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.045+0000 7f5664109640 1 -- 172.21.15.84:0/903524232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f565c103520 msgr2=0x7f565c109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:46.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.045+0000 7f5664109640 1 --2- 172.21.15.84:0/903524232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f565c103520 0x7f565c109db0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5654009cd0 tx=0x7f565402a8e0 comp rx=0 tx=0).stop 2024-05-22T22:59:46.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.045+0000 7f5664109640 1 -- 172.21.15.84:0/903524232 shutdown_connections 2024-05-22T22:59:46.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.045+0000 7f5664109640 1 --2- 172.21.15.84:0/903524232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f565c103520 0x7f565c109db0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:46.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.045+0000 7f5664109640 1 --2- 172.21.15.84:0/903524232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565c102b60 0x7f565c102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:46.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.045+0000 7f5664109640 1 --2- 172.21.15.84:0/903524232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f565c0ffb90 0x7f565c0fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:46.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.045+0000 7f5664109640 1 -- 172.21.15.84:0/903524232 >> 172.21.15.84:0/903524232 conn(0x7f565c0fb4e0 msgr2=0x7f565c0fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:46.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.045+0000 7f5664109640 1 -- 172.21.15.84:0/903524232 shutdown_connections 2024-05-22T22:59:46.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.045+0000 7f5664109640 1 -- 172.21.15.84:0/903524232 wait complete. 2024-05-22T22:59:46.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.046+0000 7f5664109640 1 Processor -- start 2024-05-22T22:59:46.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.046+0000 7f5664109640 1 -- start start 2024-05-22T22:59:46.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.046+0000 7f5664109640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f565c0ffb90 0x7f565c197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:46.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.046+0000 7f5664109640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565c102b60 0x7f565c1980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:46.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.046+0000 7f5664109640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f565c103520 0x7f565c19f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:46.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.046+0000 7f5664109640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f565c10d560 con 0x7f565c0ffb90 2024-05-22T22:59:46.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.046+0000 7f5664109640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f565c10d3e0 con 0x7f565c102b60 2024-05-22T22:59:46.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.046+0000 7f5664109640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f565c10d6e0 con 0x7f565c103520 2024-05-22T22:59:46.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.046+0000 7f5661ea5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f565c0ffb90 0x7f565c197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:46.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.046+0000 7f5661ea5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f565c0ffb90 0x7f565c197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37190/0 (socket says 172.21.15.84:37190) 2024-05-22T22:59:46.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.046+0000 7f5661ea5640 1 -- 172.21.15.84:0/1831600495 learned_addr learned my addr 172.21.15.84:0/1831600495 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:46.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.046+0000 7f56616a4640 1 --2- 172.21.15.84:0/1831600495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565c102b60 0x7f565c1980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:46.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.046+0000 7f5661ea5640 1 -- 172.21.15.84:0/1831600495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f565c103520 msgr2=0x7f565c19f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:46.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.046+0000 7f5661ea5640 1 --2- 172.21.15.84:0/1831600495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f565c103520 0x7f565c19f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:46.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.046+0000 7f5661ea5640 1 -- 172.21.15.84:0/1831600495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565c102b60 msgr2=0x7f565c1980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:46.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.046+0000 7f5661ea5640 1 --2- 172.21.15.84:0/1831600495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565c102b60 0x7f565c1980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:46.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.046+0000 7f5661ea5640 1 -- 172.21.15.84:0/1831600495 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565c19f7c0 con 0x7f565c0ffb90 2024-05-22T22:59:46.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.046+0000 7f56616a4640 1 --2- 172.21.15.84:0/1831600495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565c102b60 0x7f565c1980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T22:59:46.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.046+0000 7f5661ea5640 1 --2- 172.21.15.84:0/1831600495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f565c0ffb90 0x7f565c197ba0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f565800d950 tx=0x7f565800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:46.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.047+0000 7f5652ffd640 1 -- 172.21.15.84:0/1831600495 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5658017070 con 0x7f565c0ffb90 2024-05-22T22:59:46.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.047+0000 7f5652ffd640 1 -- 172.21.15.84:0/1831600495 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5658009c30 con 0x7f565c0ffb90 2024-05-22T22:59:46.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.047+0000 7f5664109640 1 -- 172.21.15.84:0/1831600495 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f565c19fab0 con 0x7f565c0ffb90 2024-05-22T22:59:46.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.047+0000 7f5652ffd640 1 -- 172.21.15.84:0/1831600495 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565800f600 con 0x7f565c0ffb90 2024-05-22T22:59:46.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.047+0000 7f5664109640 1 -- 172.21.15.84:0/1831600495 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f565c1a0000 con 0x7f565c0ffb90 2024-05-22T22:59:46.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.048+0000 7f5664109640 1 -- 172.21.15.84:0/1831600495 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f565c108c90 con 0x7f565c0ffb90 2024-05-22T22:59:46.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.049+0000 7f5652ffd640 1 -- 172.21.15.84:0/1831600495 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f565801a070 con 0x7f565c0ffb90 2024-05-22T22:59:46.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.049+0000 7f5652ffd640 1 --2- 172.21.15.84:0/1831600495 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f56480a2c10 0x7f56480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:46.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.049+0000 7f56616a4640 1 --2- 172.21.15.84:0/1831600495 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f56480a2c10 0x7f56480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:46.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.049+0000 7f5652ffd640 1 -- 172.21.15.84:0/1831600495 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f56580ce670 con 0x7f565c0ffb90 2024-05-22T22:59:46.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.050+0000 7f56616a4640 1 --2- 172.21.15.84:0/1831600495 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f56480a2c10 0x7f56480a50d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f565c1990b0 tx=0x7f564c00b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:46.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.054+0000 7f5652ffd640 1 -- 172.21.15.84:0/1831600495 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5658009de0 con 0x7f565c0ffb90 2024-05-22T22:59:46.275 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.275+0000 7f5664109640 1 -- 172.21.15.84:0/1831600495 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f565c02d200 con 0x7f565c0ffb90 2024-05-22T22:59:46.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.276+0000 7f5652ffd640 1 -- 172.21.15.84:0/1831600495 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9656 (secure 0 0 0) 0x7f565809ba70 con 0x7f565c0ffb90 2024-05-22T22:59:46.276 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:46.276 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-22T22:59:46.277 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 12 2024-05-22T22:59:46.278 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.278+0000 7f5664109640 1 -- 172.21.15.84:0/1831600495 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f56480a2c10 msgr2=0x7f56480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:46.278 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.278+0000 7f5664109640 1 --2- 172.21.15.84:0/1831600495 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f56480a2c10 0x7f56480a50d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f565c1990b0 tx=0x7f564c00b5f0 comp rx=0 tx=0).stop 2024-05-22T22:59:46.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.278+0000 7f5664109640 1 -- 172.21.15.84:0/1831600495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f565c0ffb90 msgr2=0x7f565c197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:46.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.278+0000 7f5664109640 1 --2- 172.21.15.84:0/1831600495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f565c0ffb90 0x7f565c197ba0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f565800d950 tx=0x7f565800de10 comp rx=0 tx=0).stop 2024-05-22T22:59:46.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.278+0000 7f5664109640 1 -- 172.21.15.84:0/1831600495 shutdown_connections 2024-05-22T22:59:46.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.278+0000 7f5664109640 1 --2- 172.21.15.84:0/1831600495 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f56480a2c10 0x7f56480a50d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:46.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.278+0000 7f5664109640 1 --2- 172.21.15.84:0/1831600495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f565c103520 0x7f565c19f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:46.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.278+0000 7f5664109640 1 --2- 172.21.15.84:0/1831600495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565c102b60 0x7f565c1980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:46.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.278+0000 7f5664109640 1 --2- 172.21.15.84:0/1831600495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f565c0ffb90 0x7f565c197ba0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:46.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.278+0000 7f5664109640 1 -- 172.21.15.84:0/1831600495 >> 172.21.15.84:0/1831600495 conn(0x7f565c0fb4e0 msgr2=0x7f565c105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:46.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.278+0000 7f5664109640 1 -- 172.21.15.84:0/1831600495 shutdown_connections 2024-05-22T22:59:46.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:46.278+0000 7f5664109640 1 -- 172.21.15.84:0/1831600495 wait complete. 2024-05-22T22:59:46.298 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 22 22:59:45 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-b[66854]: starting mds.b at 2024-05-22T22:59:46.298 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 22 22:59:46 smithi096 bash[66839]: e4324ef0d993e90e1b81b6f1309395cbe5339294436e4a29b465bfe7c87f878b 2024-05-22T22:59:46.298 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 22 22:59:46 smithi096 systemd[1]: Started Ceph mds.b for c1135a12-188d-11ef-bc9a-c7b262605968. 2024-05-22T22:59:46.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:46 smithi096 ceph-mon[36392]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:46.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:46 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:46.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:46 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:46.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:46 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:46.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:46 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:46.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:46 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:46.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:46 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:59:46.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:46 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1831600495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:46.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:46 smithi084 ceph-mon[32186]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:46.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:46 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:46.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:46 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:46.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:46 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:46.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:46 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:46.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:46 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:46.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:46 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:59:46.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:46 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1831600495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:46.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:46 smithi204 ceph-mon[36211]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:46.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:46 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:46.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:46 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:46.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:46 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:46.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:46 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:46.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:46 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:46.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:46 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:59:46.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:46 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1831600495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:47.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:47 smithi084 ceph-mon[32186]: mds.? [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] up:boot 2024-05-22T22:59:47.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:47 smithi084 ceph-mon[32186]: fsmap 12 up:standby 2024-05-22T22:59:47.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:47 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-22T22:59:47.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:47 smithi204 ceph-mon[36211]: mds.? [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] up:boot 2024-05-22T22:59:47.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:47 smithi204 ceph-mon[36211]: fsmap 12 up:standby 2024-05-22T22:59:47.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:47 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-22T22:59:48.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:47 smithi096 ceph-mon[36392]: mds.? [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] up:boot 2024-05-22T22:59:48.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:47 smithi096 ceph-mon[36392]: fsmap 12 up:standby 2024-05-22T22:59:48.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:47 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-22T22:59:48.291 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T22:59:48.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.401+0000 7f9e73389640 1 -- 172.21.15.84:0/1959115292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e6c102380 msgr2=0x7f9e6c108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:48.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.401+0000 7f9e73389640 1 --2- 172.21.15.84:0/1959115292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e6c102380 0x7f9e6c108c10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009ca0 tx=0x7f9e6402a8c0 comp rx=0 tx=0).stop 2024-05-22T22:59:48.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.401+0000 7f9e73389640 1 -- 172.21.15.84:0/1959115292 shutdown_connections 2024-05-22T22:59:48.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.401+0000 7f9e73389640 1 --2- 172.21.15.84:0/1959115292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e6c102380 0x7f9e6c108c10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:48.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.401+0000 7f9e73389640 1 --2- 172.21.15.84:0/1959115292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e6c1019c0 0x7f9e6c101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:48.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.401+0000 7f9e73389640 1 --2- 172.21.15.84:0/1959115292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e6c1007d0 0x7f9e6c100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:48.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.401+0000 7f9e73389640 1 -- 172.21.15.84:0/1959115292 >> 172.21.15.84:0/1959115292 conn(0x7f9e6c0fbd40 msgr2=0x7f9e6c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:48.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.401+0000 7f9e73389640 1 -- 172.21.15.84:0/1959115292 shutdown_connections 2024-05-22T22:59:48.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.401+0000 7f9e73389640 1 -- 172.21.15.84:0/1959115292 wait complete. 2024-05-22T22:59:48.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.402+0000 7f9e73389640 1 Processor -- start 2024-05-22T22:59:48.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.402+0000 7f9e73389640 1 -- start start 2024-05-22T22:59:48.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.402+0000 7f9e73389640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e6c1007d0 0x7f9e6c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:48.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.402+0000 7f9e73389640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e6c1019c0 0x7f9e6c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:48.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.402+0000 7f9e73389640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e6c102380 0x7f9e6c19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:48.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.402+0000 7f9e73389640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c10d5d0 con 0x7f9e6c102380 2024-05-22T22:59:48.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.402+0000 7f9e73389640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c10d450 con 0x7f9e6c1007d0 2024-05-22T22:59:48.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.402+0000 7f9e73389640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c10d750 con 0x7f9e6c1019c0 2024-05-22T22:59:48.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.402+0000 7f9e71926640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e6c102380 0x7f9e6c19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:48.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.402+0000 7f9e71926640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e6c102380 0x7f9e6c19f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37198/0 (socket says 172.21.15.84:37198) 2024-05-22T22:59:48.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.402+0000 7f9e71926640 1 -- 172.21.15.84:0/2351058165 learned_addr learned my addr 172.21.15.84:0/2351058165 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:48.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.402+0000 7f9e70924640 1 --2- 172.21.15.84:0/2351058165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e6c1019c0 0x7f9e6c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:48.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.402+0000 7f9e71125640 1 --2- 172.21.15.84:0/2351058165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e6c1007d0 0x7f9e6c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:48.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.402+0000 7f9e71926640 1 -- 172.21.15.84:0/2351058165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e6c1019c0 msgr2=0x7f9e6c1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:48.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.402+0000 7f9e71926640 1 --2- 172.21.15.84:0/2351058165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e6c1019c0 0x7f9e6c1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:48.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.402+0000 7f9e71926640 1 -- 172.21.15.84:0/2351058165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e6c1007d0 msgr2=0x7f9e6c197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:48.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.402+0000 7f9e71926640 1 --2- 172.21.15.84:0/2351058165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e6c1007d0 0x7f9e6c197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:48.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.402+0000 7f9e71926640 1 -- 172.21.15.84:0/2351058165 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6c19f7a0 con 0x7f9e6c102380 2024-05-22T22:59:48.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.402+0000 7f9e71125640 1 --2- 172.21.15.84:0/2351058165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e6c1007d0 0x7f9e6c197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T22:59:48.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.402+0000 7f9e71926640 1 --2- 172.21.15.84:0/2351058165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e6c102380 0x7f9e6c19f130 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009430 tx=0x7f9e640319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:48.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.402+0000 7f9e5e7fc640 1 -- 172.21.15.84:0/2351058165 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64003b70 con 0x7f9e6c102380 2024-05-22T22:59:48.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.402+0000 7f9e5e7fc640 1 -- 172.21.15.84:0/2351058165 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e64003d10 con 0x7f9e6c102380 2024-05-22T22:59:48.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.402+0000 7f9e73389640 1 -- 172.21.15.84:0/2351058165 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e6c19fa30 con 0x7f9e6c102380 2024-05-22T22:59:48.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.402+0000 7f9e5e7fc640 1 -- 172.21.15.84:0/2351058165 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6402fc50 con 0x7f9e6c102380 2024-05-22T22:59:48.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.403+0000 7f9e73389640 1 -- 172.21.15.84:0/2351058165 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e6c19ff80 con 0x7f9e6c102380 2024-05-22T22:59:48.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.403+0000 7f9e73389640 1 -- 172.21.15.84:0/2351058165 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e6c107a20 con 0x7f9e6c102380 2024-05-22T22:59:48.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.404+0000 7f9e5e7fc640 1 -- 172.21.15.84:0/2351058165 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9e64004040 con 0x7f9e6c102380 2024-05-22T22:59:48.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.405+0000 7f9e5e7fc640 1 --2- 172.21.15.84:0/2351058165 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9e580a2b90 0x7f9e580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:48.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.405+0000 7f9e71125640 1 --2- 172.21.15.84:0/2351058165 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9e580a2b90 0x7f9e580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:48.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.406+0000 7f9e5e7fc640 1 -- 172.21.15.84:0/2351058165 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9e640f0100 con 0x7f9e6c102380 2024-05-22T22:59:48.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.406+0000 7f9e71125640 1 --2- 172.21.15.84:0/2351058165 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9e580a2b90 0x7f9e580a5050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c101820 tx=0x7f9e60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:48.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.410+0000 7f9e5e7fc640 1 -- 172.21.15.84:0/2351058165 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9e640bd500 con 0x7f9e6c102380 2024-05-22T22:59:48.623 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:48 smithi084 ceph-mon[32186]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:48.624 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:48.624 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:48.624 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:48.624 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:48.624 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:48.624 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:48.624 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:48.624 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:59:48.624 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:48 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:48.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.629+0000 7f9e73389640 1 -- 172.21.15.84:0/2351058165 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e6c02d3b0 con 0x7f9e6c102380 2024-05-22T22:59:48.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.630+0000 7f9e5e7fc640 1 -- 172.21.15.84:0/2351058165 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10482 (secure 0 0 0) 0x7f9e640bce00 con 0x7f9e6c102380 2024-05-22T22:59:48.630 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:48.630 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-22T22:59:48.630 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 13 2024-05-22T22:59:48.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.632+0000 7f9e73389640 1 -- 172.21.15.84:0/2351058165 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9e580a2b90 msgr2=0x7f9e580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:48.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.632+0000 7f9e73389640 1 --2- 172.21.15.84:0/2351058165 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9e580a2b90 0x7f9e580a5050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c101820 tx=0x7f9e60005f90 comp rx=0 tx=0).stop 2024-05-22T22:59:48.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.632+0000 7f9e73389640 1 -- 172.21.15.84:0/2351058165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e6c102380 msgr2=0x7f9e6c19f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:48.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.632+0000 7f9e73389640 1 --2- 172.21.15.84:0/2351058165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e6c102380 0x7f9e6c19f130 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009430 tx=0x7f9e640319f0 comp rx=0 tx=0).stop 2024-05-22T22:59:48.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.632+0000 7f9e73389640 1 -- 172.21.15.84:0/2351058165 shutdown_connections 2024-05-22T22:59:48.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.632+0000 7f9e73389640 1 --2- 172.21.15.84:0/2351058165 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9e580a2b90 0x7f9e580a5050 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:48.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.632+0000 7f9e73389640 1 --2- 172.21.15.84:0/2351058165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e6c102380 0x7f9e6c19f130 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:48.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.632+0000 7f9e73389640 1 --2- 172.21.15.84:0/2351058165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e6c1019c0 0x7f9e6c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:48.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.632+0000 7f9e73389640 1 --2- 172.21.15.84:0/2351058165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e6c1007d0 0x7f9e6c197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:48.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.632+0000 7f9e73389640 1 -- 172.21.15.84:0/2351058165 >> 172.21.15.84:0/2351058165 conn(0x7f9e6c0fbd40 msgr2=0x7f9e6c104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:48.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.632+0000 7f9e73389640 1 -- 172.21.15.84:0/2351058165 shutdown_connections 2024-05-22T22:59:48.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.632+0000 7f9e73389640 1 -- 172.21.15.84:0/2351058165 wait complete. 2024-05-22T22:59:48.644 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T22:59:48.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.750+0000 7fc11a21c640 1 -- 172.21.15.84:0/939224850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc114102330 msgr2=0x7fc114104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:48.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.750+0000 7fc11a21c640 1 --2- 172.21.15.84:0/939224850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc114102330 0x7fc114104720 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc104009900 tx=0x7fc10402a410 comp rx=0 tx=0).stop 2024-05-22T22:59:48.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.750+0000 7fc11a21c640 1 -- 172.21.15.84:0/939224850 shutdown_connections 2024-05-22T22:59:48.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.750+0000 7fc11a21c640 1 --2- 172.21.15.84:0/939224850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc114107590 0x7fc1141099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:48.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.750+0000 7fc11a21c640 1 --2- 172.21.15.84:0/939224850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc114104c60 0x7fc114107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:48.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.750+0000 7fc11a21c640 1 --2- 172.21.15.84:0/939224850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc114102330 0x7fc114104720 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:48.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.750+0000 7fc11a21c640 1 -- 172.21.15.84:0/939224850 >> 172.21.15.84:0/939224850 conn(0x7fc1140fbd20 msgr2=0x7fc1140fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:48.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.750+0000 7fc11a21c640 1 -- 172.21.15.84:0/939224850 shutdown_connections 2024-05-22T22:59:48.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.750+0000 7fc11a21c640 1 -- 172.21.15.84:0/939224850 wait complete. 2024-05-22T22:59:48.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.751+0000 7fc11a21c640 1 Processor -- start 2024-05-22T22:59:48.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.751+0000 7fc11a21c640 1 -- start start 2024-05-22T22:59:48.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.751+0000 7fc11a21c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc114102330 0x7fc114197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:48.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.751+0000 7fc11a21c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc114104c60 0x7fc114198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:48.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.751+0000 7fc11a21c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc114107590 0x7fc11419f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:48.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.751+0000 7fc11a21c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc11410d530 con 0x7fc114104c60 2024-05-22T22:59:48.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.751+0000 7fc11a21c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc11410d3b0 con 0x7fc114107590 2024-05-22T22:59:48.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.751+0000 7fc11a21c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc11410d6b0 con 0x7fc114102330 2024-05-22T22:59:48.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.751+0000 7fc112ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc114104c60 0x7fc114198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:48.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.752+0000 7fc1137fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc114102330 0x7fc114197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:48.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.752+0000 7fc112ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc114104c60 0x7fc114198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37218/0 (socket says 172.21.15.84:37218) 2024-05-22T22:59:48.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.752+0000 7fc112ffd640 1 -- 172.21.15.84:0/281354270 learned_addr learned my addr 172.21.15.84:0/281354270 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:48.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.752+0000 7fc113fff640 1 --2- 172.21.15.84:0/281354270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc114107590 0x7fc11419f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:48.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.752+0000 7fc112ffd640 1 -- 172.21.15.84:0/281354270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc114102330 msgr2=0x7fc114197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:48.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.752+0000 7fc112ffd640 1 --2- 172.21.15.84:0/281354270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc114102330 0x7fc114197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:48.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.752+0000 7fc112ffd640 1 -- 172.21.15.84:0/281354270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc114107590 msgr2=0x7fc11419f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:48.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.752+0000 7fc112ffd640 1 --2- 172.21.15.84:0/281354270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc114107590 0x7fc11419f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:48.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.752+0000 7fc112ffd640 1 -- 172.21.15.84:0/281354270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11419f7f0 con 0x7fc114104c60 2024-05-22T22:59:48.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.752+0000 7fc112ffd640 1 --2- 172.21.15.84:0/281354270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc114104c60 0x7fc114198110 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00e9c0 tx=0x7fc0fc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:48.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.752+0000 7fc110ff9640 1 -- 172.21.15.84:0/281354270 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc00cd80 con 0x7fc114104c60 2024-05-22T22:59:48.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.752+0000 7fc11a21c640 1 -- 172.21.15.84:0/281354270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11419fae0 con 0x7fc114104c60 2024-05-22T22:59:48.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.752+0000 7fc110ff9640 1 -- 172.21.15.84:0/281354270 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0fc005500 con 0x7fc114104c60 2024-05-22T22:59:48.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.752+0000 7fc110ff9640 1 -- 172.21.15.84:0/281354270 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc010690 con 0x7fc114104c60 2024-05-22T22:59:48.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.752+0000 7fc11a21c640 1 -- 172.21.15.84:0/281354270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1141a0080 con 0x7fc114104c60 2024-05-22T22:59:48.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.753+0000 7fc11a21c640 1 -- 172.21.15.84:0/281354270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1140fd810 con 0x7fc114104c60 2024-05-22T22:59:48.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.754+0000 7fc110ff9640 1 -- 172.21.15.84:0/281354270 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc0fc0046c0 con 0x7fc114104c60 2024-05-22T22:59:48.757 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.755+0000 7fc110ff9640 1 --2- 172.21.15.84:0/281354270 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc1000a2b90 0x7fc1000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:48.757 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.755+0000 7fc110ff9640 1 -- 172.21.15.84:0/281354270 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc0fc0cef20 con 0x7fc114104c60 2024-05-22T22:59:48.757 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.755+0000 7fc1137fe640 1 --2- 172.21.15.84:0/281354270 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc1000a2b90 0x7fc1000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:48.757 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.756+0000 7fc1137fe640 1 --2- 172.21.15.84:0/281354270 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc1000a2b90 0x7fc1000a5050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc1040314e0 tx=0x7fc104000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:48.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.760+0000 7fc110ff9640 1 -- 172.21.15.84:0/281354270 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc0fc09c320 con 0x7fc114104c60 2024-05-22T22:59:48.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:48 smithi204 ceph-mon[36211]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:48.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:48 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:48.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:48 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:48.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:48 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:48.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:48 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:48.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:48 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:48.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:48 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:48.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:48 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:48.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:48 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:59:48.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:48 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:48.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.944+0000 7fc11a21c640 1 -- 172.21.15.84:0/281354270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc11402d3b0 con 0x7fc114104c60 2024-05-22T22:59:48.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.945+0000 7fc110ff9640 1 -- 172.21.15.84:0/281354270 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18431 (secure 0 0 0) 0x7fc0fc0b4e20 con 0x7fc114104c60 2024-05-22T22:59:48.946 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:48.946 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":82,"fsid":"c1135a12-188d-11ef-bc9a-c7b262605968","created":"2024-05-22T22:51:49.878218+0000","modified":"2024-05-22T22:59:00.132491+0000","last_up_change":"2024-05-22T22:58:58.550278+0000","last_in_change":"2024-05-22T22:58:37.322139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T22:55:07.719394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762df065-fb4f-4c71-aab8-a62d2282c253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6803","nonce":1559945826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6805","nonce":1559945826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6809","nonce":1559945826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6807","nonce":1559945826}]},"public_addr":"172.21.15.84:6803/1559945826","cluster_addr":"172.21.15.84:6805/1559945826","heartbeat_back_addr":"172.21.15.84:6809/1559945826","heartbeat_front_addr":"172.21.15.84:6807/1559945826","state":["exists","up"]},{"osd":1,"uuid":"3fb2385d-6a34-49e3-88e1-116237beb346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6801","nonce":2996505353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6803","nonce":2996505353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6807","nonce":2996505353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6805","nonce":2996505353}]},"public_addr":"172.21.15.96:6801/2996505353","cluster_addr":"172.21.15.96:6803/2996505353","heartbeat_back_addr":"172.21.15.96:6807/2996505353","heartbeat_front_addr":"172.21.15.96:6805/2996505353","state":["exists","up"]},{"osd":2,"uuid":"e306e88e-a308-4585-ae11-c8af10b84038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6801","nonce":1499577897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6803","nonce":1499577897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6807","nonce":1499577897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6805","nonce":1499577897}]},"public_addr":"172.21.15.204:6801/1499577897","cluster_addr":"172.21.15.204:6803/1499577897","heartbeat_back_addr":"172.21.15.204:6807/1499577897","heartbeat_front_addr":"172.21.15.204:6805/1499577897","state":["exists","up"]},{"osd":3,"uuid":"d2a5fb97-6f60-4d45-9f18-447cae480974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6811","nonce":2869340648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6813","nonce":2869340648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6817","nonce":2869340648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6815","nonce":2869340648}]},"public_addr":"172.21.15.84:6811/2869340648","cluster_addr":"172.21.15.84:6813/2869340648","heartbeat_back_addr":"172.21.15.84:6817/2869340648","heartbeat_front_addr":"172.21.15.84:6815/2869340648","state":["exists","up"]},{"osd":4,"uuid":"4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6809","nonce":2554265648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6811","nonce":2554265648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6815","nonce":2554265648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6813","nonce":2554265648}]},"public_addr":"172.21.15.96:6809/2554265648","cluster_addr":"172.21.15.96:6811/2554265648","heartbeat_back_addr":"172.21.15.96:6815/2554265648","heartbeat_front_addr":"172.21.15.96:6813/2554265648","state":["exists","up"]},{"osd":5,"uuid":"7996bf5e-93a5-41b0-a9b1-99e7ef4514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6809","nonce":4075476986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6811","nonce":4075476986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6815","nonce":4075476986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6813","nonce":4075476986}]},"public_addr":"172.21.15.204:6809/4075476986","cluster_addr":"172.21.15.204:6811/4075476986","heartbeat_back_addr":"172.21.15.204:6815/4075476986","heartbeat_front_addr":"172.21.15.204:6813/4075476986","state":["exists","up"]},{"osd":6,"uuid":"f4b7c5de-1836-4d16-8603-3c517524d9fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6819","nonce":1806202865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6821","nonce":1806202865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6825","nonce":1806202865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6823","nonce":1806202865}]},"public_addr":"172.21.15.84:6819/1806202865","cluster_addr":"172.21.15.84:6821/1806202865","heartbeat_back_addr":"172.21.15.84:6825/1806202865","heartbeat_front_addr":"172.21.15.84:6823/1806202865","state":["exists","up"]},{"osd":7,"uuid":"69a18c43-5447-480d-9466-190830070c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6817","nonce":3195792774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6819","nonce":3195792774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6823","nonce":3195792774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6821","nonce":3195792774}]},"public_addr":"172.21.15.96:6817/3195792774","cluster_addr":"172.21.15.96:6819/3195792774","heartbeat_back_addr":"172.21.15.96:6823/3195792774","heartbeat_front_addr":"172.21.15.96:6821/3195792774","state":["exists","up"]},{"osd":8,"uuid":"2c5cadca-689a-441e-847c-ba370d400ef1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6817","nonce":606997927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6819","nonce":606997927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6823","nonce":606997927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6821","nonce":606997927}]},"public_addr":"172.21.15.204:6817/606997927","cluster_addr":"172.21.15.204:6819/606997927","heartbeat_back_addr":"172.21.15.204:6823/606997927","heartbeat_front_addr":"172.21.15.204:6821/606997927","state":["exists","up"]},{"osd":9,"uuid":"e3984fb2-7c00-4275-a10e-bca7c859f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6827","nonce":2786189012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6829","nonce":2786189012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6833","nonce":2786189012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6831","nonce":2786189012}]},"public_addr":"172.21.15.84:6827/2786189012","cluster_addr":"172.21.15.84:6829/2786189012","heartbeat_back_addr":"172.21.15.84:6833/2786189012","heartbeat_front_addr":"172.21.15.84:6831/2786189012","state":["exists","up"]},{"osd":10,"uuid":"b4938160-a808-4ded-91c2-d245c99fff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6825","nonce":1985398526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6827","nonce":1985398526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6831","nonce":1985398526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6829","nonce":1985398526}]},"public_addr":"172.21.15.96:6825/1985398526","cluster_addr":"172.21.15.96:6827/1985398526","heartbeat_back_addr":"172.21.15.96:6831/1985398526","heartbeat_front_addr":"172.21.15.96:6829/1985398526","state":["exists","up"]},{"osd":11,"uuid":"238922ef-d046-40cc-833d-25b7e7a523cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6825","nonce":3106116519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6827","nonce":3106116519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6831","nonce":3106116519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6829","nonce":3106116519}]},"public_addr":"172.21.15.204:6825/3106116519","cluster_addr":"172.21.15.204:6827/3106116519","heartbeat_back_addr":"172.21.15.204:6831/3106116519","heartbeat_front_addr":"172.21.15.204:6829/3106116519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:16.443480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:39.898133+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:03.664544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:28.838687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:53.829733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:19.269307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:45.166910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:10.771614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:36.678516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:03.150592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:29.327769+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:56.315938+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6801/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:6800/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/2766288608":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/950455999":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/4000289608":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1650467363":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1052746114":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6800/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6801/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/14610813":"2024-05-23T22:52:45.395791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T22:59:48.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.947+0000 7fc11a21c640 1 -- 172.21.15.84:0/281354270 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc1000a2b90 msgr2=0x7fc1000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:48.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.947+0000 7fc11a21c640 1 --2- 172.21.15.84:0/281354270 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc1000a2b90 0x7fc1000a5050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc1040314e0 tx=0x7fc104000bc0 comp rx=0 tx=0).stop 2024-05-22T22:59:48.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.947+0000 7fc11a21c640 1 -- 172.21.15.84:0/281354270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc114104c60 msgr2=0x7fc114198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:48.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.947+0000 7fc11a21c640 1 --2- 172.21.15.84:0/281354270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc114104c60 0x7fc114198110 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00e9c0 tx=0x7fc0fc00ee80 comp rx=0 tx=0).stop 2024-05-22T22:59:48.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.947+0000 7fc11a21c640 1 -- 172.21.15.84:0/281354270 shutdown_connections 2024-05-22T22:59:48.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.947+0000 7fc11a21c640 1 --2- 172.21.15.84:0/281354270 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc1000a2b90 0x7fc1000a5050 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:48.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.947+0000 7fc11a21c640 1 --2- 172.21.15.84:0/281354270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc114107590 0x7fc11419f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:48.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.947+0000 7fc11a21c640 1 --2- 172.21.15.84:0/281354270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc114104c60 0x7fc114198110 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:48.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.947+0000 7fc11a21c640 1 --2- 172.21.15.84:0/281354270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc114102330 0x7fc114197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:48.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.947+0000 7fc11a21c640 1 -- 172.21.15.84:0/281354270 >> 172.21.15.84:0/281354270 conn(0x7fc1140fbd20 msgr2=0x7fc114102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:48.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.947+0000 7fc11a21c640 1 -- 172.21.15.84:0/281354270 shutdown_connections 2024-05-22T22:59:48.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:48.947+0000 7fc11a21c640 1 -- 172.21.15.84:0/281354270 wait complete. 2024-05-22T22:59:48.959 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T22:55:07.719394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T22:59:48.960 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T22:59:49.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:48 smithi096 ceph-mon[36392]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:49.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:48 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:49.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:48 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:49.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:48 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:49.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:48 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:49.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:48 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:49.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:48 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:49.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:48 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:49.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:48 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:59:49.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:48 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:49.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.065+0000 7fa51787d640 1 -- 172.21.15.84:0/2115701806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5101007b0 msgr2=0x7fa510100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:49.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.065+0000 7fa51787d640 1 --2- 172.21.15.84:0/2115701806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5101007b0 0x7fa510100bd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa50c009900 tx=0x7fa50c02a410 comp rx=0 tx=0).stop 2024-05-22T22:59:49.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.065+0000 7fa51787d640 1 -- 172.21.15.84:0/2115701806 shutdown_connections 2024-05-22T22:59:49.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.065+0000 7fa51787d640 1 --2- 172.21.15.84:0/2115701806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa510102360 0x7fa510108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.065+0000 7fa51787d640 1 --2- 172.21.15.84:0/2115701806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5101019a0 0x7fa510101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.065+0000 7fa51787d640 1 --2- 172.21.15.84:0/2115701806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5101007b0 0x7fa510100bd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.065+0000 7fa51787d640 1 -- 172.21.15.84:0/2115701806 >> 172.21.15.84:0/2115701806 conn(0x7fa5100fbd20 msgr2=0x7fa5100fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:49.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.065+0000 7fa51787d640 1 -- 172.21.15.84:0/2115701806 shutdown_connections 2024-05-22T22:59:49.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.065+0000 7fa51787d640 1 -- 172.21.15.84:0/2115701806 wait complete. 2024-05-22T22:59:49.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.066+0000 7fa51787d640 1 Processor -- start 2024-05-22T22:59:49.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.066+0000 7fa51787d640 1 -- start start 2024-05-22T22:59:49.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.066+0000 7fa51787d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5101007b0 0x7fa510197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:49.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.066+0000 7fa51787d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5101019a0 0x7fa5101981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:49.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.066+0000 7fa51787d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa510102360 0x7fa51019f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:49.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.066+0000 7fa51787d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa51010c400 con 0x7fa5101019a0 2024-05-22T22:59:49.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.066+0000 7fa51787d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa51010c280 con 0x7fa5101007b0 2024-05-22T22:59:49.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.066+0000 7fa51787d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa51010c580 con 0x7fa510102360 2024-05-22T22:59:49.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.066+0000 7fa514e18640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5101019a0 0x7fa5101981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:49.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.066+0000 7fa514e18640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5101019a0 0x7fa5101981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37238/0 (socket says 172.21.15.84:37238) 2024-05-22T22:59:49.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.066+0000 7fa514e18640 1 -- 172.21.15.84:0/2700700674 learned_addr learned my addr 172.21.15.84:0/2700700674 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:49.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.066+0000 7fa515619640 1 --2- 172.21.15.84:0/2700700674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5101007b0 0x7fa510197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:49.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.066+0000 7fa515e1a640 1 --2- 172.21.15.84:0/2700700674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa510102360 0x7fa51019f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:49.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.066+0000 7fa514e18640 1 -- 172.21.15.84:0/2700700674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa510102360 msgr2=0x7fa51019f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:49.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.066+0000 7fa514e18640 1 --2- 172.21.15.84:0/2700700674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa510102360 0x7fa51019f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.066+0000 7fa514e18640 1 -- 172.21.15.84:0/2700700674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5101007b0 msgr2=0x7fa510197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:49.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.066+0000 7fa514e18640 1 --2- 172.21.15.84:0/2700700674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5101007b0 0x7fa510197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.066+0000 7fa514e18640 1 -- 172.21.15.84:0/2700700674 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51019f880 con 0x7fa5101019a0 2024-05-22T22:59:49.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.067+0000 7fa514e18640 1 --2- 172.21.15.84:0/2700700674 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5101019a0 0x7fa5101981b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa50000e9f0 tx=0x7fa50000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:49.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.067+0000 7fa5067fc640 1 -- 172.21.15.84:0/2700700674 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50000cd80 con 0x7fa5101019a0 2024-05-22T22:59:49.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.067+0000 7fa5067fc640 1 -- 172.21.15.84:0/2700700674 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa500010910 con 0x7fa5101019a0 2024-05-22T22:59:49.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.067+0000 7fa51787d640 1 -- 172.21.15.84:0/2700700674 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa51019fb70 con 0x7fa5101019a0 2024-05-22T22:59:49.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.067+0000 7fa5067fc640 1 -- 172.21.15.84:0/2700700674 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa500018990 con 0x7fa5101019a0 2024-05-22T22:59:49.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.067+0000 7fa51787d640 1 -- 172.21.15.84:0/2700700674 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5101a00e0 con 0x7fa5101019a0 2024-05-22T22:59:49.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.068+0000 7fa51787d640 1 -- 172.21.15.84:0/2700700674 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5101a02b0 con 0x7fa5101019a0 2024-05-22T22:59:49.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.069+0000 7fa5067fc640 1 -- 172.21.15.84:0/2700700674 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa500010ac0 con 0x7fa5101019a0 2024-05-22T22:59:49.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.069+0000 7fa5067fc640 1 --2- 172.21.15.84:0/2700700674 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa4fc0a2c10 0x7fa4fc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:49.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.070+0000 7fa515619640 1 --2- 172.21.15.84:0/2700700674 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa4fc0a2c10 0x7fa4fc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:49.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.070+0000 7fa5067fc640 1 -- 172.21.15.84:0/2700700674 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa5000cf2e0 con 0x7fa5101019a0 2024-05-22T22:59:49.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.070+0000 7fa515619640 1 --2- 172.21.15.84:0/2700700674 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa4fc0a2c10 0x7fa4fc0a50d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa50c005ab0 tx=0x7fa50c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:49.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.073+0000 7fa5067fc640 1 -- 172.21.15.84:0/2700700674 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa50009c070 con 0x7fa5101019a0 2024-05-22T22:59:49.263 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.263+0000 7fa51787d640 1 -- 172.21.15.84:0/2700700674 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa5101a0480 con 0x7fa5101019a0 2024-05-22T22:59:49.263 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.263+0000 7fa5067fc640 1 -- 172.21.15.84:0/2700700674 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fa5000a0370 con 0x7fa5101019a0 2024-05-22T22:59:49.264 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2024-05-22T22:59:49.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.265+0000 7fa51787d640 1 -- 172.21.15.84:0/2700700674 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa4fc0a2c10 msgr2=0x7fa4fc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:49.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.265+0000 7fa51787d640 1 --2- 172.21.15.84:0/2700700674 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa4fc0a2c10 0x7fa4fc0a50d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa50c005ab0 tx=0x7fa50c000bc0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.265+0000 7fa51787d640 1 -- 172.21.15.84:0/2700700674 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5101019a0 msgr2=0x7fa5101981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:49.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.265+0000 7fa51787d640 1 --2- 172.21.15.84:0/2700700674 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5101019a0 0x7fa5101981b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa50000e9f0 tx=0x7fa50000eeb0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.265+0000 7fa51787d640 1 -- 172.21.15.84:0/2700700674 shutdown_connections 2024-05-22T22:59:49.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.265+0000 7fa51787d640 1 --2- 172.21.15.84:0/2700700674 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa4fc0a2c10 0x7fa4fc0a50d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.265+0000 7fa51787d640 1 --2- 172.21.15.84:0/2700700674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa510102360 0x7fa51019f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.265+0000 7fa51787d640 1 --2- 172.21.15.84:0/2700700674 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5101019a0 0x7fa5101981b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.265+0000 7fa51787d640 1 --2- 172.21.15.84:0/2700700674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5101007b0 0x7fa510197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.265+0000 7fa51787d640 1 -- 172.21.15.84:0/2700700674 >> 172.21.15.84:0/2700700674 conn(0x7fa5100fbd20 msgr2=0x7fa510068a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:49.267 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.265+0000 7fa51787d640 1 -- 172.21.15.84:0/2700700674 shutdown_connections 2024-05-22T22:59:49.267 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.265+0000 7fa51787d640 1 -- 172.21.15.84:0/2700700674 wait complete. 2024-05-22T22:59:49.280 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-22T22:59:49.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.380+0000 7f7b5c84d640 1 -- 172.21.15.84:0/3741346870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b58100a80 msgr2=0x7f7b58100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:49.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.380+0000 7f7b5c84d640 1 --2- 172.21.15.84:0/3741346870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b58100a80 0x7f7b58100f00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009ca0 tx=0x7f7b4c02a8c0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.380+0000 7f7b5c84d640 1 -- 172.21.15.84:0/3741346870 shutdown_connections 2024-05-22T22:59:49.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.380+0000 7f7b5c84d640 1 --2- 172.21.15.84:0/3741346870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b58101440 0x7f7b58108ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.380+0000 7f7b5c84d640 1 --2- 172.21.15.84:0/3741346870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b58100a80 0x7f7b58100f00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.380+0000 7f7b5c84d640 1 --2- 172.21.15.84:0/3741346870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b58101e30 0x7f7b58102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.380+0000 7f7b5c84d640 1 -- 172.21.15.84:0/3741346870 >> 172.21.15.84:0/3741346870 conn(0x7f7b580fb550 msgr2=0x7f7b580fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:49.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.380+0000 7f7b5c84d640 1 -- 172.21.15.84:0/3741346870 shutdown_connections 2024-05-22T22:59:49.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.380+0000 7f7b5c84d640 1 -- 172.21.15.84:0/3741346870 wait complete. 2024-05-22T22:59:49.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.381+0000 7f7b5c84d640 1 Processor -- start 2024-05-22T22:59:49.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.381+0000 7f7b5c84d640 1 -- start start 2024-05-22T22:59:49.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.381+0000 7f7b5c84d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b58100a80 0x7f7b58197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:49.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.381+0000 7f7b5c84d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b58101440 0x7f7b58198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:49.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.381+0000 7f7b5c84d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b58101e30 0x7f7b5819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:49.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.381+0000 7f7b5c84d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5810d680 con 0x7f7b58101440 2024-05-22T22:59:49.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.381+0000 7f7b5c84d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5810d500 con 0x7f7b58100a80 2024-05-22T22:59:49.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.381+0000 7f7b5c84d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5810d800 con 0x7f7b58101e30 2024-05-22T22:59:49.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.381+0000 7f7b55d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b58101440 0x7f7b58198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:49.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.381+0000 7f7b55d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b58101440 0x7f7b58198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37268/0 (socket says 172.21.15.84:37268) 2024-05-22T22:59:49.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.381+0000 7f7b55d9b640 1 -- 172.21.15.84:0/1371207942 learned_addr learned my addr 172.21.15.84:0/1371207942 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:49.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.381+0000 7f7b5659c640 1 --2- 172.21.15.84:0/1371207942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b58100a80 0x7f7b58197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:49.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.381+0000 7f7b56d9d640 1 --2- 172.21.15.84:0/1371207942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b58101e30 0x7f7b5819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:49.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.381+0000 7f7b55d9b640 1 -- 172.21.15.84:0/1371207942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b58101e30 msgr2=0x7f7b5819f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:49.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.381+0000 7f7b55d9b640 1 --2- 172.21.15.84:0/1371207942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b58101e30 0x7f7b5819f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.381+0000 7f7b55d9b640 1 -- 172.21.15.84:0/1371207942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b58100a80 msgr2=0x7f7b58197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:49.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.381+0000 7f7b55d9b640 1 --2- 172.21.15.84:0/1371207942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b58100a80 0x7f7b58197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.381+0000 7f7b55d9b640 1 -- 172.21.15.84:0/1371207942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5819f900 con 0x7f7b58101440 2024-05-22T22:59:49.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.381+0000 7f7b5659c640 1 --2- 172.21.15.84:0/1371207942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b58100a80 0x7f7b58197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T22:59:49.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.381+0000 7f7b55d9b640 1 --2- 172.21.15.84:0/1371207942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b58101440 0x7f7b58198230 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c031a40 tx=0x7f7b4c031ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:49.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.381+0000 7f7b477fe640 1 -- 172.21.15.84:0/1371207942 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c02faa0 con 0x7f7b58101440 2024-05-22T22:59:49.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.382+0000 7f7b5c84d640 1 -- 172.21.15.84:0/1371207942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b5819fb90 con 0x7f7b58101440 2024-05-22T22:59:49.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.382+0000 7f7b477fe640 1 -- 172.21.15.84:0/1371207942 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b4c02fc40 con 0x7f7b58101440 2024-05-22T22:59:49.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.382+0000 7f7b477fe640 1 -- 172.21.15.84:0/1371207942 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c03a790 con 0x7f7b58101440 2024-05-22T22:59:49.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.382+0000 7f7b5c84d640 1 -- 172.21.15.84:0/1371207942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b581a00e0 con 0x7f7b58101440 2024-05-22T22:59:49.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.383+0000 7f7b5c84d640 1 -- 172.21.15.84:0/1371207942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b58107bc0 con 0x7f7b58101440 2024-05-22T22:59:49.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.384+0000 7f7b477fe640 1 -- 172.21.15.84:0/1371207942 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7b4c030b30 con 0x7f7b58101440 2024-05-22T22:59:49.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.384+0000 7f7b477fe640 1 --2- 172.21.15.84:0/1371207942 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f7b400a2b90 0x7f7b400a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:49.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.384+0000 7f7b5659c640 1 --2- 172.21.15.84:0/1371207942 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f7b400a2b90 0x7f7b400a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:49.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.384+0000 7f7b477fe640 1 -- 172.21.15.84:0/1371207942 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f7b4c0f0410 con 0x7f7b58101440 2024-05-22T22:59:49.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.385+0000 7f7b5659c640 1 --2- 172.21.15.84:0/1371207942 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f7b400a2b90 0x7f7b400a5050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7b50005fd0 tx=0x7f7b50005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:49.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.389+0000 7f7b477fe640 1 -- 172.21.15.84:0/1371207942 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7b4c0bd810 con 0x7f7b58101440 2024-05-22T22:59:49.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.576+0000 7f7b5c84d640 1 -- 172.21.15.84:0/1371207942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7b5802d380 con 0x7f7b58101440 2024-05-22T22:59:49.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.577+0000 7f7b477fe640 1 -- 172.21.15.84:0/1371207942 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f7b4c0bd110 con 0x7f7b58101440 2024-05-22T22:59:49.578 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:49.578 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2024-05-22T22:59:49.578 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T22:59:49.578 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2024-05-22T22:59:49.578 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2024-05-22T22:59:49.579 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T22:59:49.579 INFO:teuthology.orchestra.run.smithi084.stdout:] 2024-05-22T22:59:49.580 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.579+0000 7f7b5c84d640 1 -- 172.21.15.84:0/1371207942 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f7b400a2b90 msgr2=0x7f7b400a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:49.580 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.579+0000 7f7b5c84d640 1 --2- 172.21.15.84:0/1371207942 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f7b400a2b90 0x7f7b400a5050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7b50005fd0 tx=0x7f7b50005850 comp rx=0 tx=0).stop 2024-05-22T22:59:49.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.579+0000 7f7b5c84d640 1 -- 172.21.15.84:0/1371207942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b58101440 msgr2=0x7f7b58198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:49.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.579+0000 7f7b5c84d640 1 --2- 172.21.15.84:0/1371207942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b58101440 0x7f7b58198230 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c031a40 tx=0x7f7b4c031ec0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.579+0000 7f7b5c84d640 1 -- 172.21.15.84:0/1371207942 shutdown_connections 2024-05-22T22:59:49.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.579+0000 7f7b5c84d640 1 --2- 172.21.15.84:0/1371207942 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f7b400a2b90 0x7f7b400a5050 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.579+0000 7f7b5c84d640 1 --2- 172.21.15.84:0/1371207942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b58101e30 0x7f7b5819f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.579+0000 7f7b5c84d640 1 --2- 172.21.15.84:0/1371207942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b58101440 0x7f7b58198230 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.579+0000 7f7b5c84d640 1 --2- 172.21.15.84:0/1371207942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b58100a80 0x7f7b58197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.579+0000 7f7b5c84d640 1 -- 172.21.15.84:0/1371207942 >> 172.21.15.84:0/1371207942 conn(0x7f7b580fb550 msgr2=0x7f7b58104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:49.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.579+0000 7f7b5c84d640 1 -- 172.21.15.84:0/1371207942 shutdown_connections 2024-05-22T22:59:49.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.579+0000 7f7b5c84d640 1 -- 172.21.15.84:0/1371207942 wait complete. 2024-05-22T22:59:49.591 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-22T22:59:49.591 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-22T22:59:49.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.697+0000 7f0992d16640 1 -- 172.21.15.84:0/1887646231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f098c1019a0 msgr2=0x7f098c101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:49.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.697+0000 7f0992d16640 1 --2- 172.21.15.84:0/1887646231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f098c1019a0 0x7f098c101e20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0974009900 tx=0x7f097402a410 comp rx=0 tx=0).stop 2024-05-22T22:59:49.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.697+0000 7f0992d16640 1 -- 172.21.15.84:0/1887646231 shutdown_connections 2024-05-22T22:59:49.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.697+0000 7f0992d16640 1 --2- 172.21.15.84:0/1887646231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f098c102360 0x7f098c108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.697+0000 7f0992d16640 1 --2- 172.21.15.84:0/1887646231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f098c1019a0 0x7f098c101e20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.697+0000 7f0992d16640 1 --2- 172.21.15.84:0/1887646231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f098c1007b0 0x7f098c100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.697+0000 7f0992d16640 1 -- 172.21.15.84:0/1887646231 >> 172.21.15.84:0/1887646231 conn(0x7f098c0fbd20 msgr2=0x7f098c0fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:49.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.697+0000 7f0992d16640 1 -- 172.21.15.84:0/1887646231 shutdown_connections 2024-05-22T22:59:49.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.698+0000 7f0992d16640 1 -- 172.21.15.84:0/1887646231 wait complete. 2024-05-22T22:59:49.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.698+0000 7f0992d16640 1 Processor -- start 2024-05-22T22:59:49.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.698+0000 7f0992d16640 1 -- start start 2024-05-22T22:59:49.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.698+0000 7f0992d16640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f098c1007b0 0x7f098c197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:49.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.698+0000 7f0992d16640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f098c1019a0 0x7f098c198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:49.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.698+0000 7f0992d16640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f098c102360 0x7f098c19f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:49.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.698+0000 7f0992d16640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f098c10d550 con 0x7f098c1019a0 2024-05-22T22:59:49.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.698+0000 7f0992d16640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f098c10d3d0 con 0x7f098c102360 2024-05-22T22:59:49.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.698+0000 7f0992d16640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f098c10d6d0 con 0x7f098c1007b0 2024-05-22T22:59:49.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.698+0000 7f098bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f098c1019a0 0x7f098c198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:49.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.698+0000 7f098bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f098c1019a0 0x7f098c198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37286/0 (socket says 172.21.15.84:37286) 2024-05-22T22:59:49.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.698+0000 7f098bfff640 1 -- 172.21.15.84:0/2166686226 learned_addr learned my addr 172.21.15.84:0/2166686226 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:49.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.698+0000 7f0990ab2640 1 --2- 172.21.15.84:0/2166686226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f098c1007b0 0x7f098c197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:49.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.698+0000 7f09912b3640 1 --2- 172.21.15.84:0/2166686226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f098c102360 0x7f098c19f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:49.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.699+0000 7f098bfff640 1 -- 172.21.15.84:0/2166686226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f098c1007b0 msgr2=0x7f098c197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:49.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.699+0000 7f098bfff640 1 --2- 172.21.15.84:0/2166686226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f098c1007b0 0x7f098c197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.699+0000 7f098bfff640 1 -- 172.21.15.84:0/2166686226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f098c102360 msgr2=0x7f098c19f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:49.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.699+0000 7f098bfff640 1 --2- 172.21.15.84:0/2166686226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f098c102360 0x7f098c19f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:49.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.699+0000 7f098bfff640 1 -- 172.21.15.84:0/2166686226 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f098c19f810 con 0x7f098c1019a0 2024-05-22T22:59:49.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.699+0000 7f09912b3640 1 --2- 172.21.15.84:0/2166686226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f098c102360 0x7f098c19f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T22:59:49.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.699+0000 7f098bfff640 1 --2- 172.21.15.84:0/2166686226 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f098c1019a0 0x7f098c198130 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0974009230 tx=0x7f0974031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:49.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.699+0000 7f0989ffb640 1 -- 172.21.15.84:0/2166686226 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097403f030 con 0x7f098c1019a0 2024-05-22T22:59:49.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.699+0000 7f0989ffb640 1 -- 172.21.15.84:0/2166686226 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0974003ad0 con 0x7f098c1019a0 2024-05-22T22:59:49.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.699+0000 7f0992d16640 1 -- 172.21.15.84:0/2166686226 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f098c19faa0 con 0x7f098c1019a0 2024-05-22T22:59:49.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.699+0000 7f0989ffb640 1 -- 172.21.15.84:0/2166686226 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974035070 con 0x7f098c1019a0 2024-05-22T22:59:49.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.699+0000 7f0992d16640 1 -- 172.21.15.84:0/2166686226 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f098c19ff90 con 0x7f098c1019a0 2024-05-22T22:59:49.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.700+0000 7f0992d16640 1 -- 172.21.15.84:0/2166686226 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f098c107af0 con 0x7f098c1019a0 2024-05-22T22:59:49.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.701+0000 7f0989ffb640 1 -- 172.21.15.84:0/2166686226 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0974042030 con 0x7f098c1019a0 2024-05-22T22:59:49.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.702+0000 7f0989ffb640 1 --2- 172.21.15.84:0/2166686226 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f09780a2b90 0x7f09780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:49.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.702+0000 7f0990ab2640 1 --2- 172.21.15.84:0/2166686226 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f09780a2b90 0x7f09780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:49.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.702+0000 7f0989ffb640 1 -- 172.21.15.84:0/2166686226 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f09740ef0b0 con 0x7f098c1019a0 2024-05-22T22:59:49.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.702+0000 7f0990ab2640 1 --2- 172.21.15.84:0/2166686226 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f09780a2b90 0x7f09780a5050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f098c101800 tx=0x7f097c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:49.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.706+0000 7f0989ffb640 1 -- 172.21.15.84:0/2166686226 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f09740bbe60 con 0x7f098c1019a0 2024-05-22T22:59:49.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:49 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2351058165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:49.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:49 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:59:49.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:49 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/281354270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T22:59:49.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:49 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2700700674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T22:59:49.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:49 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2351058165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:49.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:49 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:59:49.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:49 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/281354270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T22:59:49.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:49 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2700700674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T22:59:49.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:49.885+0000 7f0992d16640 1 -- 172.21.15.84:0/2166686226 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f098c02d200 con 0x7f098c1019a0 2024-05-22T22:59:49.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:49 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2351058165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:49.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:49 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T22:59:49.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:49 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/281354270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T22:59:49.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:49 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2700700674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T22:59:50.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.566+0000 7f0989ffb640 1 -- 172.21.15.84:0/2166686226 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f09740c0160 con 0x7f098c1019a0 2024-05-22T22:59:50.566 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' created 2024-05-22T22:59:50.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.568+0000 7f0992d16640 1 -- 172.21.15.84:0/2166686226 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f09780a2b90 msgr2=0x7f09780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:50.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.568+0000 7f0992d16640 1 --2- 172.21.15.84:0/2166686226 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f09780a2b90 0x7f09780a5050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f098c101800 tx=0x7f097c009500 comp rx=0 tx=0).stop 2024-05-22T22:59:50.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.568+0000 7f0992d16640 1 -- 172.21.15.84:0/2166686226 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f098c1019a0 msgr2=0x7f098c198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:50.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.568+0000 7f0992d16640 1 --2- 172.21.15.84:0/2166686226 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f098c1019a0 0x7f098c198130 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0974009230 tx=0x7f0974031b60 comp rx=0 tx=0).stop 2024-05-22T22:59:50.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.568+0000 7f0992d16640 1 -- 172.21.15.84:0/2166686226 shutdown_connections 2024-05-22T22:59:50.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.568+0000 7f0992d16640 1 --2- 172.21.15.84:0/2166686226 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f09780a2b90 0x7f09780a5050 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:50.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.569+0000 7f0992d16640 1 --2- 172.21.15.84:0/2166686226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f098c102360 0x7f098c19f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:50.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.569+0000 7f0992d16640 1 --2- 172.21.15.84:0/2166686226 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f098c1019a0 0x7f098c198130 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:50.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.569+0000 7f0992d16640 1 --2- 172.21.15.84:0/2166686226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f098c1007b0 0x7f098c197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:50.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.569+0000 7f0992d16640 1 -- 172.21.15.84:0/2166686226 >> 172.21.15.84:0/2166686226 conn(0x7f098c0fbd20 msgr2=0x7f098c104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:50.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.569+0000 7f0992d16640 1 -- 172.21.15.84:0/2166686226 shutdown_connections 2024-05-22T22:59:50.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.569+0000 7f0992d16640 1 -- 172.21.15.84:0/2166686226 wait complete. 2024-05-22T22:59:50.583 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-22T22:59:50.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.691+0000 7fb7e2c7d640 1 -- 172.21.15.84:0/3590183224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7dc1007b0 msgr2=0x7fb7dc100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:50.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.691+0000 7fb7e2c7d640 1 --2- 172.21.15.84:0/3590183224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7dc1007b0 0x7fb7dc100bd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009900 tx=0x7fb7d002a410 comp rx=0 tx=0).stop 2024-05-22T22:59:50.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.691+0000 7fb7e2c7d640 1 -- 172.21.15.84:0/3590183224 shutdown_connections 2024-05-22T22:59:50.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.691+0000 7fb7e2c7d640 1 --2- 172.21.15.84:0/3590183224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7dc102360 0x7fb7dc108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:50.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.691+0000 7fb7e2c7d640 1 --2- 172.21.15.84:0/3590183224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7dc1019a0 0x7fb7dc101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:50.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.691+0000 7fb7e2c7d640 1 --2- 172.21.15.84:0/3590183224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7dc1007b0 0x7fb7dc100bd0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:50.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.691+0000 7fb7e2c7d640 1 -- 172.21.15.84:0/3590183224 >> 172.21.15.84:0/3590183224 conn(0x7fb7dc0fbd60 msgr2=0x7fb7dc0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:50.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.691+0000 7fb7e2c7d640 1 -- 172.21.15.84:0/3590183224 shutdown_connections 2024-05-22T22:59:50.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.691+0000 7fb7e2c7d640 1 -- 172.21.15.84:0/3590183224 wait complete. 2024-05-22T22:59:50.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.691+0000 7fb7e2c7d640 1 Processor -- start 2024-05-22T22:59:50.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.692+0000 7fb7e2c7d640 1 -- start start 2024-05-22T22:59:50.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.692+0000 7fb7e2c7d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7dc1007b0 0x7fb7dc06aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:50.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.692+0000 7fb7e2c7d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7dc1019a0 0x7fb7dc19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:50.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.692+0000 7fb7e2c7d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7dc102360 0x7fb7dc19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:50.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.692+0000 7fb7e2c7d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb7dc10c400 con 0x7fb7dc102360 2024-05-22T22:59:50.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.692+0000 7fb7e2c7d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb7dc10c280 con 0x7fb7dc1019a0 2024-05-22T22:59:50.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.692+0000 7fb7e2c7d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb7dc10c580 con 0x7fb7dc1007b0 2024-05-22T22:59:50.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.692+0000 7fb7e121a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7dc102360 0x7fb7dc19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:50.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.692+0000 7fb7e121a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7dc102360 0x7fb7dc19efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37310/0 (socket says 172.21.15.84:37310) 2024-05-22T22:59:50.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.692+0000 7fb7e121a640 1 -- 172.21.15.84:0/2996331749 learned_addr learned my addr 172.21.15.84:0/2996331749 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:50.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.692+0000 7fb7e0a19640 1 --2- 172.21.15.84:0/2996331749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7dc1007b0 0x7fb7dc06aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:50.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.692+0000 7fb7dbfff640 1 --2- 172.21.15.84:0/2996331749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7dc1019a0 0x7fb7dc19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:50.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.692+0000 7fb7e121a640 1 -- 172.21.15.84:0/2996331749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7dc1007b0 msgr2=0x7fb7dc06aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:50.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.692+0000 7fb7e121a640 1 --2- 172.21.15.84:0/2996331749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7dc1007b0 0x7fb7dc06aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:50.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.692+0000 7fb7e121a640 1 -- 172.21.15.84:0/2996331749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7dc1019a0 msgr2=0x7fb7dc19c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:50.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.692+0000 7fb7e121a640 1 --2- 172.21.15.84:0/2996331749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7dc1019a0 0x7fb7dc19c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:50.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.692+0000 7fb7e121a640 1 -- 172.21.15.84:0/2996331749 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7dc19f8b0 con 0x7fb7dc102360 2024-05-22T22:59:50.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.692+0000 7fb7dbfff640 1 --2- 172.21.15.84:0/2996331749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7dc1019a0 0x7fb7dc19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T22:59:50.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.692+0000 7fb7e121a640 1 --2- 172.21.15.84:0/2996331749 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7dc102360 0x7fb7dc19efa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb7d400cad0 tx=0x7fb7d400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:50.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.692+0000 7fb7d9ffb640 1 -- 172.21.15.84:0/2996331749 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d400ae20 con 0x7fb7dc102360 2024-05-22T22:59:50.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.692+0000 7fb7d9ffb640 1 -- 172.21.15.84:0/2996331749 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7d4005020 con 0x7fb7dc102360 2024-05-22T22:59:50.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.692+0000 7fb7e2c7d640 1 -- 172.21.15.84:0/2996331749 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7dc19fb40 con 0x7fb7dc102360 2024-05-22T22:59:50.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.692+0000 7fb7d9ffb640 1 -- 172.21.15.84:0/2996331749 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d400f6f0 con 0x7fb7dc102360 2024-05-22T22:59:50.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.693+0000 7fb7e2c7d640 1 -- 172.21.15.84:0/2996331749 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7dc1a0040 con 0x7fb7dc102360 2024-05-22T22:59:50.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.694+0000 7fb7e2c7d640 1 -- 172.21.15.84:0/2996331749 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7dc107b40 con 0x7fb7dc102360 2024-05-22T22:59:50.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.695+0000 7fb7d9ffb640 1 -- 172.21.15.84:0/2996331749 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb7d40051d0 con 0x7fb7dc102360 2024-05-22T22:59:50.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.695+0000 7fb7d9ffb640 1 --2- 172.21.15.84:0/2996331749 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb7cc0a2b90 0x7fb7cc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:50.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.695+0000 7fb7e0a19640 1 --2- 172.21.15.84:0/2996331749 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb7cc0a2b90 0x7fb7cc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:50.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.695+0000 7fb7d9ffb640 1 -- 172.21.15.84:0/2996331749 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7fb7d40cedd0 con 0x7fb7dc102360 2024-05-22T22:59:50.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.696+0000 7fb7e0a19640 1 --2- 172.21.15.84:0/2996331749 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb7cc0a2b90 0x7fb7cc0a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0005ab0 tx=0x7fb7d0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:50.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.700+0000 7fb7d9ffb640 1 -- 172.21.15.84:0/2996331749 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb7d4014030 con 0x7fb7dc102360 2024-05-22T22:59:50.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:50 smithi084 ceph-mon[32186]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:50.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:50 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1371207942' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-22T22:59:50.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:50 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2166686226' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-22T22:59:50.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:50 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:50.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:50 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:50.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:50 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:50.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:50 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:50.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:50 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:50.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:50 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:50.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:50 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:50.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:50 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:59:50.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:50 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:50.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:50 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:50.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:50.877+0000 7fb7e2c7d640 1 -- 172.21.15.84:0/2996331749 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb7dc062470 con 0x7fb7dc102360 2024-05-22T22:59:50.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:50 smithi204 ceph-mon[36211]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:50.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:50 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1371207942' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-22T22:59:50.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:50 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2166686226' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-22T22:59:50.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:50 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:50.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:50 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:50.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:50 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:50.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:50 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:50.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:50 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:50.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:50 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:50.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:50 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:50.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:50 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:59:50.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:50 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:50.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:50 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:51.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:50 smithi096 ceph-mon[36392]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:51.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:50 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1371207942' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-22T22:59:51.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:50 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2166686226' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-22T22:59:51.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:50 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:51.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:50 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:51.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:50 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:51.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:50 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:51.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:50 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:51.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:50 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:51.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:50 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T22:59:51.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:50 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T22:59:51.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:50 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:51.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:50 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T22:59:51.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.574+0000 7fb7d9ffb640 1 -- 172.21.15.84:0/2996331749 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7fb7d409c060 con 0x7fb7dc102360 2024-05-22T22:59:51.574 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' created 2024-05-22T22:59:51.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.576+0000 7fb7e2c7d640 1 -- 172.21.15.84:0/2996331749 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb7cc0a2b90 msgr2=0x7fb7cc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:51.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.576+0000 7fb7e2c7d640 1 --2- 172.21.15.84:0/2996331749 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb7cc0a2b90 0x7fb7cc0a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0005ab0 tx=0x7fb7d0000bc0 comp rx=0 tx=0).stop 2024-05-22T22:59:51.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.576+0000 7fb7e2c7d640 1 -- 172.21.15.84:0/2996331749 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7dc102360 msgr2=0x7fb7dc19efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:51.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.576+0000 7fb7e2c7d640 1 --2- 172.21.15.84:0/2996331749 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7dc102360 0x7fb7dc19efa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb7d400cad0 tx=0x7fb7d400cf90 comp rx=0 tx=0).stop 2024-05-22T22:59:51.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.576+0000 7fb7e2c7d640 1 -- 172.21.15.84:0/2996331749 shutdown_connections 2024-05-22T22:59:51.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.576+0000 7fb7e2c7d640 1 --2- 172.21.15.84:0/2996331749 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb7cc0a2b90 0x7fb7cc0a5050 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:51.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.576+0000 7fb7e2c7d640 1 --2- 172.21.15.84:0/2996331749 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7dc102360 0x7fb7dc19efa0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:51.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.576+0000 7fb7e2c7d640 1 --2- 172.21.15.84:0/2996331749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7dc1019a0 0x7fb7dc19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:51.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.576+0000 7fb7e2c7d640 1 --2- 172.21.15.84:0/2996331749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7dc1007b0 0x7fb7dc06aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:51.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.576+0000 7fb7e2c7d640 1 -- 172.21.15.84:0/2996331749 >> 172.21.15.84:0/2996331749 conn(0x7fb7dc0fbd60 msgr2=0x7fb7dc104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:51.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.576+0000 7fb7e2c7d640 1 -- 172.21.15.84:0/2996331749 shutdown_connections 2024-05-22T22:59:51.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.576+0000 7fb7e2c7d640 1 -- 172.21.15.84:0/2996331749 wait complete. 2024-05-22T22:59:51.597 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-22T22:59:51.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.702+0000 7fe8bdad4640 1 -- 172.21.15.84:0/3932849035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8b81019a0 msgr2=0x7fe8b8101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:51.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.702+0000 7fe8bdad4640 1 --2- 172.21.15.84:0/3932849035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8b81019a0 0x7fe8b8101e20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009900 tx=0x7fe8a002a410 comp rx=0 tx=0).stop 2024-05-22T22:59:51.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.702+0000 7fe8bdad4640 1 -- 172.21.15.84:0/3932849035 shutdown_connections 2024-05-22T22:59:51.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.702+0000 7fe8bdad4640 1 --2- 172.21.15.84:0/3932849035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8b8102360 0x7fe8b8108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:51.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.702+0000 7fe8bdad4640 1 --2- 172.21.15.84:0/3932849035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8b81019a0 0x7fe8b8101e20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:51.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.702+0000 7fe8bdad4640 1 --2- 172.21.15.84:0/3932849035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8b81007b0 0x7fe8b8100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:51.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.702+0000 7fe8bdad4640 1 -- 172.21.15.84:0/3932849035 >> 172.21.15.84:0/3932849035 conn(0x7fe8b80fbd20 msgr2=0x7fe8b80fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:51.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.702+0000 7fe8bdad4640 1 -- 172.21.15.84:0/3932849035 shutdown_connections 2024-05-22T22:59:51.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.702+0000 7fe8bdad4640 1 -- 172.21.15.84:0/3932849035 wait complete. 2024-05-22T22:59:51.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.702+0000 7fe8bdad4640 1 Processor -- start 2024-05-22T22:59:51.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.702+0000 7fe8bdad4640 1 -- start start 2024-05-22T22:59:51.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.703+0000 7fe8bdad4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8b81007b0 0x7fe8b8197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:51.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.703+0000 7fe8bdad4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8b81019a0 0x7fe8b8198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:51.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.703+0000 7fe8bdad4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8b8102360 0x7fe8b819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:51.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.703+0000 7fe8bdad4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b810d5b0 con 0x7fe8b81019a0 2024-05-22T22:59:51.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.703+0000 7fe8bdad4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b810d430 con 0x7fe8b81007b0 2024-05-22T22:59:51.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.703+0000 7fe8bdad4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b810d730 con 0x7fe8b8102360 2024-05-22T22:59:51.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.703+0000 7fe8affff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8b81019a0 0x7fe8b8198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:51.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.703+0000 7fe8affff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8b81019a0 0x7fe8b8198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41698/0 (socket says 172.21.15.84:41698) 2024-05-22T22:59:51.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.703+0000 7fe8affff640 1 -- 172.21.15.84:0/4131633920 learned_addr learned my addr 172.21.15.84:0/4131633920 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:51.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.703+0000 7fe8b77fe640 1 --2- 172.21.15.84:0/4131633920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8b81007b0 0x7fe8b8197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:51.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.703+0000 7fe8affff640 1 -- 172.21.15.84:0/4131633920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8b8102360 msgr2=0x7fe8b819f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:51.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.703+0000 7fe8b7fff640 1 --2- 172.21.15.84:0/4131633920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8b8102360 0x7fe8b819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:51.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.703+0000 7fe8affff640 1 --2- 172.21.15.84:0/4131633920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8b8102360 0x7fe8b819f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:51.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.703+0000 7fe8affff640 1 -- 172.21.15.84:0/4131633920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8b81007b0 msgr2=0x7fe8b8197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:51.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.703+0000 7fe8affff640 1 --2- 172.21.15.84:0/4131633920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8b81007b0 0x7fe8b8197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:51.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.703+0000 7fe8affff640 1 -- 172.21.15.84:0/4131633920 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b819f850 con 0x7fe8b81019a0 2024-05-22T22:59:51.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.703+0000 7fe8b77fe640 1 --2- 172.21.15.84:0/4131633920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8b81007b0 0x7fe8b8197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T22:59:51.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.703+0000 7fe8b7fff640 1 --2- 172.21.15.84:0/4131633920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8b8102360 0x7fe8b819f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T22:59:51.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.703+0000 7fe8affff640 1 --2- 172.21.15.84:0/4131633920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8b81019a0 0x7fe8b8198180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009230 tx=0x7fe8a0004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:51.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.703+0000 7fe8b57fa640 1 -- 172.21.15.84:0/4131633920 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a003f030 con 0x7fe8b81019a0 2024-05-22T22:59:51.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.703+0000 7fe8b57fa640 1 -- 172.21.15.84:0/4131633920 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8a0003ce0 con 0x7fe8b81019a0 2024-05-22T22:59:51.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.703+0000 7fe8b57fa640 1 -- 172.21.15.84:0/4131633920 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0035070 con 0x7fe8b81019a0 2024-05-22T22:59:51.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.703+0000 7fe8bdad4640 1 -- 172.21.15.84:0/4131633920 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b819fae0 con 0x7fe8b81019a0 2024-05-22T22:59:51.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.703+0000 7fe8bdad4640 1 -- 172.21.15.84:0/4131633920 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b81a0020 con 0x7fe8b81019a0 2024-05-22T22:59:51.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.705+0000 7fe8bdad4640 1 -- 172.21.15.84:0/4131633920 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8b8107ad0 con 0x7fe8b81019a0 2024-05-22T22:59:51.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.706+0000 7fe8b57fa640 1 -- 172.21.15.84:0/4131633920 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe8a0042030 con 0x7fe8b81019a0 2024-05-22T22:59:51.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.707+0000 7fe8b57fa640 1 --2- 172.21.15.84:0/4131633920 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe8a40a2c10 0x7fe8a40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:51.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.707+0000 7fe8b77fe640 1 --2- 172.21.15.84:0/4131633920 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe8a40a2c10 0x7fe8a40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:51.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.707+0000 7fe8b57fa640 1 -- 172.21.15.84:0/4131633920 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7fe8a00efad0 con 0x7fe8b81019a0 2024-05-22T22:59:51.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.707+0000 7fe8b77fe640 1 --2- 172.21.15.84:0/4131633920 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe8a40a2c10 0x7fe8a40a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8006fd0 tx=0x7fe8a8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:51.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.711+0000 7fe8b57fa640 1 -- 172.21.15.84:0/4131633920 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe8a00bcbe0 con 0x7fe8b81019a0 2024-05-22T22:59:51.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:51 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2166686226' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-22T22:59:51.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:51 smithi084 ceph-mon[32186]: osdmap e83: 12 total, 12 up, 12 in 2024-05-22T22:59:51.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:51 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2996331749' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-22T22:59:51.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:51 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2166686226' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-22T22:59:51.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:51 smithi204 ceph-mon[36211]: osdmap e83: 12 total, 12 up, 12 in 2024-05-22T22:59:51.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:51 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2996331749' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-22T22:59:51.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:51.924+0000 7fe8bdad4640 1 -- 172.21.15.84:0/4131633920 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe8b802d200 con 0x7fe8b81019a0 2024-05-22T22:59:52.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:51 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2166686226' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-22T22:59:52.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:51 smithi096 ceph-mon[36392]: osdmap e83: 12 total, 12 up, 12 in 2024-05-22T22:59:52.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:51 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2996331749' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-22T22:59:52.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.584+0000 7fe8b57fa640 1 -- 172.21.15.84:0/4131633920 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fe8a00bc4e0 con 0x7fe8b81019a0 2024-05-22T22:59:52.584 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 2 and data pool 3 2024-05-22T22:59:52.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.586+0000 7fe8bdad4640 1 -- 172.21.15.84:0/4131633920 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe8a40a2c10 msgr2=0x7fe8a40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:52.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.586+0000 7fe8bdad4640 1 --2- 172.21.15.84:0/4131633920 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe8a40a2c10 0x7fe8a40a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8006fd0 tx=0x7fe8a8008040 comp rx=0 tx=0).stop 2024-05-22T22:59:52.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.586+0000 7fe8bdad4640 1 -- 172.21.15.84:0/4131633920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8b81019a0 msgr2=0x7fe8b8198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:52.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.586+0000 7fe8bdad4640 1 --2- 172.21.15.84:0/4131633920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8b81019a0 0x7fe8b8198180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009230 tx=0x7fe8a0004430 comp rx=0 tx=0).stop 2024-05-22T22:59:52.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.586+0000 7fe8bdad4640 1 -- 172.21.15.84:0/4131633920 shutdown_connections 2024-05-22T22:59:52.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.586+0000 7fe8bdad4640 1 --2- 172.21.15.84:0/4131633920 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe8a40a2c10 0x7fe8a40a50d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:52.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.586+0000 7fe8bdad4640 1 --2- 172.21.15.84:0/4131633920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8b8102360 0x7fe8b819f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:52.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.586+0000 7fe8bdad4640 1 --2- 172.21.15.84:0/4131633920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8b81019a0 0x7fe8b8198180 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:52.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.586+0000 7fe8bdad4640 1 --2- 172.21.15.84:0/4131633920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8b81007b0 0x7fe8b8197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:52.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.586+0000 7fe8bdad4640 1 -- 172.21.15.84:0/4131633920 >> 172.21.15.84:0/4131633920 conn(0x7fe8b80fbd20 msgr2=0x7fe8b8104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:52.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.586+0000 7fe8bdad4640 1 -- 172.21.15.84:0/4131633920 shutdown_connections 2024-05-22T22:59:52.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.586+0000 7fe8bdad4640 1 -- 172.21.15.84:0/4131633920 wait complete. 2024-05-22T22:59:52.600 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T22:59:52.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.700+0000 7f16abbdc640 1 -- 172.21.15.84:0/1798698771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16a4101a20 msgr2=0x7f16a4101ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:52.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.700+0000 7f16abbdc640 1 --2- 172.21.15.84:0/1798698771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16a4101a20 0x7f16a4101ea0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1694009900 tx=0x7f169402a410 comp rx=0 tx=0).stop 2024-05-22T22:59:52.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.700+0000 7f16abbdc640 1 -- 172.21.15.84:0/1798698771 shutdown_connections 2024-05-22T22:59:52.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.700+0000 7f16abbdc640 1 --2- 172.21.15.84:0/1798698771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a41023e0 0x7f16a4108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:52.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.700+0000 7f16abbdc640 1 --2- 172.21.15.84:0/1798698771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16a4101a20 0x7f16a4101ea0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:52.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.700+0000 7f16abbdc640 1 --2- 172.21.15.84:0/1798698771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16a4100830 0x7f16a4100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:52.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.700+0000 7f16abbdc640 1 -- 172.21.15.84:0/1798698771 >> 172.21.15.84:0/1798698771 conn(0x7f16a40fbdc0 msgr2=0x7f16a40fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:52.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.700+0000 7f16abbdc640 1 -- 172.21.15.84:0/1798698771 shutdown_connections 2024-05-22T22:59:52.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.700+0000 7f16abbdc640 1 -- 172.21.15.84:0/1798698771 wait complete. 2024-05-22T22:59:52.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.701+0000 7f16abbdc640 1 Processor -- start 2024-05-22T22:59:52.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.701+0000 7f16abbdc640 1 -- start start 2024-05-22T22:59:52.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.701+0000 7f16abbdc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a4100830 0x7f16a406aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:52.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.701+0000 7f16abbdc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16a4101a20 0x7f16a419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:52.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.701+0000 7f16abbdc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16a41023e0 0x7f16a41a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:52.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.701+0000 7f16abbdc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f16a410c440 con 0x7f16a4101a20 2024-05-22T22:59:52.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.701+0000 7f16abbdc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f16a410c2c0 con 0x7f16a4100830 2024-05-22T22:59:52.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.701+0000 7f16abbdc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f16a410c5c0 con 0x7f16a41023e0 2024-05-22T22:59:52.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.701+0000 7f16a9177640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16a4101a20 0x7f16a419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:52.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.701+0000 7f16a9177640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16a4101a20 0x7f16a419ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41718/0 (socket says 172.21.15.84:41718) 2024-05-22T22:59:52.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.701+0000 7f16a9177640 1 -- 172.21.15.84:0/4041681581 learned_addr learned my addr 172.21.15.84:0/4041681581 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:52.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.701+0000 7f16aa179640 1 --2- 172.21.15.84:0/4041681581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16a41023e0 0x7f16a41a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:52.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.701+0000 7f16a9978640 1 --2- 172.21.15.84:0/4041681581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a4100830 0x7f16a406aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:52.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.701+0000 7f16a9177640 1 -- 172.21.15.84:0/4041681581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16a41023e0 msgr2=0x7f16a41a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:52.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.701+0000 7f16a9177640 1 --2- 172.21.15.84:0/4041681581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16a41023e0 0x7f16a41a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:52.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.701+0000 7f16a9177640 1 -- 172.21.15.84:0/4041681581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a4100830 msgr2=0x7f16a406aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:52.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.701+0000 7f16a9177640 1 --2- 172.21.15.84:0/4041681581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a4100830 0x7f16a406aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:52.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.701+0000 7f16a9177640 1 -- 172.21.15.84:0/4041681581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a41a1aa0 con 0x7f16a4101a20 2024-05-22T22:59:52.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.701+0000 7f16a9978640 1 --2- 172.21.15.84:0/4041681581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a4100830 0x7f16a406aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T22:59:52.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.701+0000 7f16aa179640 1 --2- 172.21.15.84:0/4041681581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16a41023e0 0x7f16a41a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T22:59:52.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.701+0000 7f16a9177640 1 --2- 172.21.15.84:0/4041681581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16a4101a20 0x7f16a419ed60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1694009230 tx=0x7f1694004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:52.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.702+0000 7f169affd640 1 -- 172.21.15.84:0/4041681581 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169403f030 con 0x7f16a4101a20 2024-05-22T22:59:52.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.702+0000 7f169affd640 1 -- 172.21.15.84:0/4041681581 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1694003ce0 con 0x7f16a4101a20 2024-05-22T22:59:52.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.702+0000 7f16abbdc640 1 -- 172.21.15.84:0/4041681581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a41a1cd0 con 0x7f16a4101a20 2024-05-22T22:59:52.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.702+0000 7f169affd640 1 -- 172.21.15.84:0/4041681581 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1694035070 con 0x7f16a4101a20 2024-05-22T22:59:52.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.702+0000 7f16abbdc640 1 -- 172.21.15.84:0/4041681581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a41a21c0 con 0x7f16a4101a20 2024-05-22T22:59:52.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.703+0000 7f16abbdc640 1 -- 172.21.15.84:0/4041681581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16a4107bd0 con 0x7f16a4101a20 2024-05-22T22:59:52.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.704+0000 7f169affd640 1 -- 172.21.15.84:0/4041681581 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1694042030 con 0x7f16a4101a20 2024-05-22T22:59:52.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.704+0000 7f169affd640 1 --2- 172.21.15.84:0/4041681581 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f16900a2c40 0x7f16900a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:52.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.705+0000 7f16a9978640 1 --2- 172.21.15.84:0/4041681581 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f16900a2c40 0x7f16900a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:52.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.705+0000 7f169affd640 1 -- 172.21.15.84:0/4041681581 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f16940ef580 con 0x7f16a4101a20 2024-05-22T22:59:52.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.705+0000 7f16a9978640 1 --2- 172.21.15.84:0/4041681581 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f16900a2c40 0x7f16900a5100 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f16a4101880 tx=0x7f16a0005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:52.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.709+0000 7f169affd640 1 -- 172.21.15.84:0/4041681581 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f16940bc630 con 0x7f16a4101a20 2024-05-22T22:59:52.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:52 smithi084 ceph-mon[32186]: pgmap v286: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:52.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:52 smithi084 ceph-mon[32186]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-22T22:59:52.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:52 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2996331749' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-22T22:59:52.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:52 smithi084 ceph-mon[32186]: osdmap e84: 12 total, 12 up, 12 in 2024-05-22T22:59:52.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:52 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/4131633920' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-22T22:59:52.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:52 smithi084 ceph-mon[32186]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-22T22:59:52.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:52 smithi084 ceph-mon[32186]: osdmap e85: 12 total, 12 up, 12 in 2024-05-22T22:59:52.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.888+0000 7f16abbdc640 1 -- 172.21.15.84:0/4041681581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16a402d380 con 0x7f16a4101a20 2024-05-22T22:59:52.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.889+0000 7f169affd640 1 -- 172.21.15.84:0/4041681581 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22207 (secure 0 0 0) 0x7f16940bbf30 con 0x7f16a4101a20 2024-05-22T22:59:52.890 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:52.890 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":85,"fsid":"c1135a12-188d-11ef-bc9a-c7b262605968","created":"2024-05-22T22:51:49.878218+0000","modified":"2024-05-22T22:59:51.926494+0000","last_up_change":"2024-05-22T22:58:58.550278+0000","last_in_change":"2024-05-22T22:58:37.322139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T22:55:07.719394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T22:59:50.150511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T22:59:51.111182+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762df065-fb4f-4c71-aab8-a62d2282c253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6803","nonce":1559945826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6805","nonce":1559945826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6809","nonce":1559945826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6807","nonce":1559945826}]},"public_addr":"172.21.15.84:6803/1559945826","cluster_addr":"172.21.15.84:6805/1559945826","heartbeat_back_addr":"172.21.15.84:6809/1559945826","heartbeat_front_addr":"172.21.15.84:6807/1559945826","state":["exists","up"]},{"osd":1,"uuid":"3fb2385d-6a34-49e3-88e1-116237beb346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6801","nonce":2996505353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6803","nonce":2996505353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6807","nonce":2996505353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6805","nonce":2996505353}]},"public_addr":"172.21.15.96:6801/2996505353","cluster_addr":"172.21.15.96:6803/2996505353","heartbeat_back_addr":"172.21.15.96:6807/2996505353","heartbeat_front_addr":"172.21.15.96:6805/2996505353","state":["exists","up"]},{"osd":2,"uuid":"e306e88e-a308-4585-ae11-c8af10b84038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6801","nonce":1499577897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6803","nonce":1499577897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6807","nonce":1499577897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6805","nonce":1499577897}]},"public_addr":"172.21.15.204:6801/1499577897","cluster_addr":"172.21.15.204:6803/1499577897","heartbeat_back_addr":"172.21.15.204:6807/1499577897","heartbeat_front_addr":"172.21.15.204:6805/1499577897","state":["exists","up"]},{"osd":3,"uuid":"d2a5fb97-6f60-4d45-9f18-447cae480974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6811","nonce":2869340648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6813","nonce":2869340648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6817","nonce":2869340648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6815","nonce":2869340648}]},"public_addr":"172.21.15.84:6811/2869340648","cluster_addr":"172.21.15.84:6813/2869340648","heartbeat_back_addr":"172.21.15.84:6817/2869340648","heartbeat_front_addr":"172.21.15.84:6815/2869340648","state":["exists","up"]},{"osd":4,"uuid":"4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6809","nonce":2554265648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6811","nonce":2554265648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6815","nonce":2554265648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6813","nonce":2554265648}]},"public_addr":"172.21.15.96:6809/2554265648","cluster_addr":"172.21.15.96:6811/2554265648","heartbeat_back_addr":"172.21.15.96:6815/2554265648","heartbeat_front_addr":"172.21.15.96:6813/2554265648","state":["exists","up"]},{"osd":5,"uuid":"7996bf5e-93a5-41b0-a9b1-99e7ef4514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6809","nonce":4075476986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6811","nonce":4075476986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6815","nonce":4075476986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6813","nonce":4075476986}]},"public_addr":"172.21.15.204:6809/4075476986","cluster_addr":"172.21.15.204:6811/4075476986","heartbeat_back_addr":"172.21.15.204:6815/4075476986","heartbeat_front_addr":"172.21.15.204:6813/4075476986","state":["exists","up"]},{"osd":6,"uuid":"f4b7c5de-1836-4d16-8603-3c517524d9fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6819","nonce":1806202865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6821","nonce":1806202865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6825","nonce":1806202865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6823","nonce":1806202865}]},"public_addr":"172.21.15.84:6819/1806202865","cluster_addr":"172.21.15.84:6821/1806202865","heartbeat_back_addr":"172.21.15.84:6825/1806202865","heartbeat_front_addr":"172.21.15.84:6823/1806202865","state":["exists","up"]},{"osd":7,"uuid":"69a18c43-5447-480d-9466-190830070c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6817","nonce":3195792774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6819","nonce":3195792774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6823","nonce":3195792774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6821","nonce":3195792774}]},"public_addr":"172.21.15.96:6817/3195792774","cluster_addr":"172.21.15.96:6819/3195792774","heartbeat_back_addr":"172.21.15.96:6823/3195792774","heartbeat_front_addr":"172.21.15.96:6821/3195792774","state":["exists","up"]},{"osd":8,"uuid":"2c5cadca-689a-441e-847c-ba370d400ef1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6817","nonce":606997927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6819","nonce":606997927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6823","nonce":606997927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6821","nonce":606997927}]},"public_addr":"172.21.15.204:6817/606997927","cluster_addr":"172.21.15.204:6819/606997927","heartbeat_back_addr":"172.21.15.204:6823/606997927","heartbeat_front_addr":"172.21.15.204:6821/606997927","state":["exists","up"]},{"osd":9,"uuid":"e3984fb2-7c00-4275-a10e-bca7c859f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6827","nonce":2786189012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6829","nonce":2786189012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6833","nonce":2786189012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6831","nonce":2786189012}]},"public_addr":"172.21.15.84:6827/2786189012","cluster_addr":"172.21.15.84:6829/2786189012","heartbeat_back_addr":"172.21.15.84:6833/2786189012","heartbeat_front_addr":"172.21.15.84:6831/2786189012","state":["exists","up"]},{"osd":10,"uuid":"b4938160-a808-4ded-91c2-d245c99fff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6825","nonce":1985398526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6827","nonce":1985398526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6831","nonce":1985398526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6829","nonce":1985398526}]},"public_addr":"172.21.15.96:6825/1985398526","cluster_addr":"172.21.15.96:6827/1985398526","heartbeat_back_addr":"172.21.15.96:6831/1985398526","heartbeat_front_addr":"172.21.15.96:6829/1985398526","state":["exists","up"]},{"osd":11,"uuid":"238922ef-d046-40cc-833d-25b7e7a523cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6825","nonce":3106116519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6827","nonce":3106116519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6831","nonce":3106116519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6829","nonce":3106116519}]},"public_addr":"172.21.15.204:6825/3106116519","cluster_addr":"172.21.15.204:6827/3106116519","heartbeat_back_addr":"172.21.15.204:6831/3106116519","heartbeat_front_addr":"172.21.15.204:6829/3106116519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:16.443480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:39.898133+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:03.664544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:28.838687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:53.829733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:19.269307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:45.166910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:10.771614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:36.678516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:03.150592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:29.327769+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:56.315938+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6801/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:6800/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/2766288608":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/950455999":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/4000289608":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1650467363":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1052746114":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6800/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6801/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/14610813":"2024-05-23T22:52:45.395791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T22:59:52.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.891+0000 7f16abbdc640 1 -- 172.21.15.84:0/4041681581 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f16900a2c40 msgr2=0x7f16900a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:52.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.891+0000 7f16abbdc640 1 --2- 172.21.15.84:0/4041681581 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f16900a2c40 0x7f16900a5100 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f16a4101880 tx=0x7f16a0005e70 comp rx=0 tx=0).stop 2024-05-22T22:59:52.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.891+0000 7f16abbdc640 1 -- 172.21.15.84:0/4041681581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16a4101a20 msgr2=0x7f16a419ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:52.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.891+0000 7f16abbdc640 1 --2- 172.21.15.84:0/4041681581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16a4101a20 0x7f16a419ed60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1694009230 tx=0x7f1694004360 comp rx=0 tx=0).stop 2024-05-22T22:59:52.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.892+0000 7f16abbdc640 1 -- 172.21.15.84:0/4041681581 shutdown_connections 2024-05-22T22:59:52.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.892+0000 7f16abbdc640 1 --2- 172.21.15.84:0/4041681581 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f16900a2c40 0x7f16900a5100 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:52.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.892+0000 7f16abbdc640 1 --2- 172.21.15.84:0/4041681581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16a41023e0 0x7f16a41a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:52.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.892+0000 7f16abbdc640 1 --2- 172.21.15.84:0/4041681581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16a4101a20 0x7f16a419ed60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:52.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.892+0000 7f16abbdc640 1 --2- 172.21.15.84:0/4041681581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a4100830 0x7f16a406aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:52.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.892+0000 7f16abbdc640 1 -- 172.21.15.84:0/4041681581 >> 172.21.15.84:0/4041681581 conn(0x7f16a40fbdc0 msgr2=0x7f16a4104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:52.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.892+0000 7f16abbdc640 1 -- 172.21.15.84:0/4041681581 shutdown_connections 2024-05-22T22:59:52.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:52.892+0000 7f16abbdc640 1 -- 172.21.15.84:0/4041681581 wait complete. 2024-05-22T22:59:52.903 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T22:59:52.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:52 smithi204 ceph-mon[36211]: pgmap v286: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:52.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:52 smithi204 ceph-mon[36211]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-22T22:59:52.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:52 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2996331749' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-22T22:59:52.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:52 smithi204 ceph-mon[36211]: osdmap e84: 12 total, 12 up, 12 in 2024-05-22T22:59:52.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:52 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/4131633920' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-22T22:59:52.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:52 smithi204 ceph-mon[36211]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-22T22:59:52.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:52 smithi204 ceph-mon[36211]: osdmap e85: 12 total, 12 up, 12 in 2024-05-22T22:59:53.007 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.004+0000 7fc90a914640 1 -- 172.21.15.84:0/1164975324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9041018c0 msgr2=0x7fc904101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:53.007 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.004+0000 7fc90a914640 1 --2- 172.21.15.84:0/1164975324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9041018c0 0x7fc904101ce0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009900 tx=0x7fc8f402a410 comp rx=0 tx=0).stop 2024-05-22T22:59:53.007 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.004+0000 7fc90a914640 1 -- 172.21.15.84:0/1164975324 shutdown_connections 2024-05-22T22:59:53.007 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.004+0000 7fc90a914640 1 --2- 172.21.15.84:0/1164975324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc904103470 0x7fc904109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:53.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.004+0000 7fc90a914640 1 --2- 172.21.15.84:0/1164975324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc904102ab0 0x7fc904102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:53.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.004+0000 7fc90a914640 1 --2- 172.21.15.84:0/1164975324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9041018c0 0x7fc904101ce0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:53.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.004+0000 7fc90a914640 1 -- 172.21.15.84:0/1164975324 >> 172.21.15.84:0/1164975324 conn(0x7fc9040fce50 msgr2=0x7fc9040ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:53.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.004+0000 7fc90a914640 1 -- 172.21.15.84:0/1164975324 shutdown_connections 2024-05-22T22:59:53.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.004+0000 7fc90a914640 1 -- 172.21.15.84:0/1164975324 wait complete. 2024-05-22T22:59:53.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.005+0000 7fc90a914640 1 Processor -- start 2024-05-22T22:59:53.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.005+0000 7fc90a914640 1 -- start start 2024-05-22T22:59:53.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.005+0000 7fc90a914640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9041018c0 0x7fc904197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:53.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.005+0000 7fc90a914640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc904102ab0 0x7fc9041980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:53.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.005+0000 7fc90a914640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc904103470 0x7fc90419f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:53.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.005+0000 7fc90a914640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc90410d540 con 0x7fc9041018c0 2024-05-22T22:59:53.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.005+0000 7fc90a914640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc90410d3c0 con 0x7fc904102ab0 2024-05-22T22:59:53.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.005+0000 7fc90a914640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc90410d6c0 con 0x7fc904103470 2024-05-22T22:59:53.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.005+0000 7fc903fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9041018c0 0x7fc904197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:53.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.005+0000 7fc903fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9041018c0 0x7fc904197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41748/0 (socket says 172.21.15.84:41748) 2024-05-22T22:59:53.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.005+0000 7fc903fff640 1 -- 172.21.15.84:0/3139210315 learned_addr learned my addr 172.21.15.84:0/3139210315 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:53.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.005+0000 7fc8fbfff640 1 --2- 172.21.15.84:0/3139210315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc904102ab0 0x7fc9041980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:53.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.005+0000 7fc908eb1640 1 --2- 172.21.15.84:0/3139210315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc904103470 0x7fc90419f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:53.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.005+0000 7fc903fff640 1 -- 172.21.15.84:0/3139210315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc904103470 msgr2=0x7fc90419f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:53.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.005+0000 7fc903fff640 1 --2- 172.21.15.84:0/3139210315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc904103470 0x7fc90419f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:53.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.005+0000 7fc903fff640 1 -- 172.21.15.84:0/3139210315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc904102ab0 msgr2=0x7fc9041980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:53.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.005+0000 7fc903fff640 1 --2- 172.21.15.84:0/3139210315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc904102ab0 0x7fc9041980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:53.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.005+0000 7fc903fff640 1 -- 172.21.15.84:0/3139210315 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90419f7d0 con 0x7fc9041018c0 2024-05-22T22:59:53.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.005+0000 7fc908eb1640 1 --2- 172.21.15.84:0/3139210315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc904103470 0x7fc90419f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T22:59:53.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.006+0000 7fc903fff640 1 --2- 172.21.15.84:0/3139210315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9041018c0 0x7fc904197bb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4031610 tx=0x7fc8f40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:53.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.006+0000 7fc901ffb640 1 -- 172.21.15.84:0/3139210315 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f403f030 con 0x7fc9041018c0 2024-05-22T22:59:53.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.006+0000 7fc901ffb640 1 -- 172.21.15.84:0/3139210315 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f4003ce0 con 0x7fc9041018c0 2024-05-22T22:59:53.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.006+0000 7fc90a914640 1 -- 172.21.15.84:0/3139210315 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90419fa60 con 0x7fc9041018c0 2024-05-22T22:59:53.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.006+0000 7fc901ffb640 1 -- 172.21.15.84:0/3139210315 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4035070 con 0x7fc9041018c0 2024-05-22T22:59:53.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.006+0000 7fc90a914640 1 -- 172.21.15.84:0/3139210315 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc90419ff50 con 0x7fc9041018c0 2024-05-22T22:59:53.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.007+0000 7fc90a914640 1 -- 172.21.15.84:0/3139210315 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc904108be0 con 0x7fc9041018c0 2024-05-22T22:59:53.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.008+0000 7fc901ffb640 1 -- 172.21.15.84:0/3139210315 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc8f4042030 con 0x7fc9041018c0 2024-05-22T22:59:53.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.009+0000 7fc901ffb640 1 --2- 172.21.15.84:0/3139210315 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc8f00a2b90 0x7fc8f00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:53.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.009+0000 7fc8fbfff640 1 --2- 172.21.15.84:0/3139210315 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc8f00a2b90 0x7fc8f00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:53.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.009+0000 7fc901ffb640 1 -- 172.21.15.84:0/3139210315 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc8f40ef490 con 0x7fc9041018c0 2024-05-22T22:59:53.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.009+0000 7fc8fbfff640 1 --2- 172.21.15.84:0/3139210315 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc8f00a2b90 0x7fc8f00a5050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc9041990c0 tx=0x7fc8ec00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:53.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.013+0000 7fc901ffb640 1 -- 172.21.15.84:0/3139210315 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc8f40bc540 con 0x7fc9041018c0 2024-05-22T22:59:53.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:52 smithi096 ceph-mon[36392]: pgmap v286: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:53.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:52 smithi096 ceph-mon[36392]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-22T22:59:53.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:52 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2996331749' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-22T22:59:53.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:52 smithi096 ceph-mon[36392]: osdmap e84: 12 total, 12 up, 12 in 2024-05-22T22:59:53.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:52 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/4131633920' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-22T22:59:53.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:52 smithi096 ceph-mon[36392]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-22T22:59:53.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:52 smithi096 ceph-mon[36392]: osdmap e85: 12 total, 12 up, 12 in 2024-05-22T22:59:53.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.190+0000 7fc90a914640 1 -- 172.21.15.84:0/3139210315 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc90402d420 con 0x7fc9041018c0 2024-05-22T22:59:53.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.192+0000 7fc901ffb640 1 -- 172.21.15.84:0/3139210315 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22194 (secure 0 0 0) 0x7fc8f40bbe40 con 0x7fc9041018c0 2024-05-22T22:59:53.193 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:53.193 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":86,"fsid":"c1135a12-188d-11ef-bc9a-c7b262605968","created":"2024-05-22T22:51:49.878218+0000","modified":"2024-05-22T22:59:52.940885+0000","last_up_change":"2024-05-22T22:58:58.550278+0000","last_in_change":"2024-05-22T22:58:37.322139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T22:55:07.719394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T22:59:50.150511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T22:59:51.111182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762df065-fb4f-4c71-aab8-a62d2282c253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6803","nonce":1559945826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6805","nonce":1559945826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6809","nonce":1559945826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6807","nonce":1559945826}]},"public_addr":"172.21.15.84:6803/1559945826","cluster_addr":"172.21.15.84:6805/1559945826","heartbeat_back_addr":"172.21.15.84:6809/1559945826","heartbeat_front_addr":"172.21.15.84:6807/1559945826","state":["exists","up"]},{"osd":1,"uuid":"3fb2385d-6a34-49e3-88e1-116237beb346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6801","nonce":2996505353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6803","nonce":2996505353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6807","nonce":2996505353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6805","nonce":2996505353}]},"public_addr":"172.21.15.96:6801/2996505353","cluster_addr":"172.21.15.96:6803/2996505353","heartbeat_back_addr":"172.21.15.96:6807/2996505353","heartbeat_front_addr":"172.21.15.96:6805/2996505353","state":["exists","up"]},{"osd":2,"uuid":"e306e88e-a308-4585-ae11-c8af10b84038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6801","nonce":1499577897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6803","nonce":1499577897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6807","nonce":1499577897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6805","nonce":1499577897}]},"public_addr":"172.21.15.204:6801/1499577897","cluster_addr":"172.21.15.204:6803/1499577897","heartbeat_back_addr":"172.21.15.204:6807/1499577897","heartbeat_front_addr":"172.21.15.204:6805/1499577897","state":["exists","up"]},{"osd":3,"uuid":"d2a5fb97-6f60-4d45-9f18-447cae480974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6811","nonce":2869340648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6813","nonce":2869340648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6817","nonce":2869340648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6815","nonce":2869340648}]},"public_addr":"172.21.15.84:6811/2869340648","cluster_addr":"172.21.15.84:6813/2869340648","heartbeat_back_addr":"172.21.15.84:6817/2869340648","heartbeat_front_addr":"172.21.15.84:6815/2869340648","state":["exists","up"]},{"osd":4,"uuid":"4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6809","nonce":2554265648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6811","nonce":2554265648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6815","nonce":2554265648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6813","nonce":2554265648}]},"public_addr":"172.21.15.96:6809/2554265648","cluster_addr":"172.21.15.96:6811/2554265648","heartbeat_back_addr":"172.21.15.96:6815/2554265648","heartbeat_front_addr":"172.21.15.96:6813/2554265648","state":["exists","up"]},{"osd":5,"uuid":"7996bf5e-93a5-41b0-a9b1-99e7ef4514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6809","nonce":4075476986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6811","nonce":4075476986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6815","nonce":4075476986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6813","nonce":4075476986}]},"public_addr":"172.21.15.204:6809/4075476986","cluster_addr":"172.21.15.204:6811/4075476986","heartbeat_back_addr":"172.21.15.204:6815/4075476986","heartbeat_front_addr":"172.21.15.204:6813/4075476986","state":["exists","up"]},{"osd":6,"uuid":"f4b7c5de-1836-4d16-8603-3c517524d9fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6819","nonce":1806202865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6821","nonce":1806202865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6825","nonce":1806202865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6823","nonce":1806202865}]},"public_addr":"172.21.15.84:6819/1806202865","cluster_addr":"172.21.15.84:6821/1806202865","heartbeat_back_addr":"172.21.15.84:6825/1806202865","heartbeat_front_addr":"172.21.15.84:6823/1806202865","state":["exists","up"]},{"osd":7,"uuid":"69a18c43-5447-480d-9466-190830070c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6817","nonce":3195792774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6819","nonce":3195792774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6823","nonce":3195792774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6821","nonce":3195792774}]},"public_addr":"172.21.15.96:6817/3195792774","cluster_addr":"172.21.15.96:6819/3195792774","heartbeat_back_addr":"172.21.15.96:6823/3195792774","heartbeat_front_addr":"172.21.15.96:6821/3195792774","state":["exists","up"]},{"osd":8,"uuid":"2c5cadca-689a-441e-847c-ba370d400ef1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6817","nonce":606997927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6819","nonce":606997927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6823","nonce":606997927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6821","nonce":606997927}]},"public_addr":"172.21.15.204:6817/606997927","cluster_addr":"172.21.15.204:6819/606997927","heartbeat_back_addr":"172.21.15.204:6823/606997927","heartbeat_front_addr":"172.21.15.204:6821/606997927","state":["exists","up"]},{"osd":9,"uuid":"e3984fb2-7c00-4275-a10e-bca7c859f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6827","nonce":2786189012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6829","nonce":2786189012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6833","nonce":2786189012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6831","nonce":2786189012}]},"public_addr":"172.21.15.84:6827/2786189012","cluster_addr":"172.21.15.84:6829/2786189012","heartbeat_back_addr":"172.21.15.84:6833/2786189012","heartbeat_front_addr":"172.21.15.84:6831/2786189012","state":["exists","up"]},{"osd":10,"uuid":"b4938160-a808-4ded-91c2-d245c99fff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6825","nonce":1985398526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6827","nonce":1985398526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6831","nonce":1985398526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6829","nonce":1985398526}]},"public_addr":"172.21.15.96:6825/1985398526","cluster_addr":"172.21.15.96:6827/1985398526","heartbeat_back_addr":"172.21.15.96:6831/1985398526","heartbeat_front_addr":"172.21.15.96:6829/1985398526","state":["exists","up"]},{"osd":11,"uuid":"238922ef-d046-40cc-833d-25b7e7a523cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6825","nonce":3106116519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6827","nonce":3106116519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6831","nonce":3106116519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6829","nonce":3106116519}]},"public_addr":"172.21.15.204:6825/3106116519","cluster_addr":"172.21.15.204:6827/3106116519","heartbeat_back_addr":"172.21.15.204:6831/3106116519","heartbeat_front_addr":"172.21.15.204:6829/3106116519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:16.443480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:39.898133+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:03.664544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:28.838687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:53.829733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:19.269307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:45.166910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:10.771614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:36.678516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:03.150592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:29.327769+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:56.315938+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6801/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:6800/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/2766288608":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/950455999":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/4000289608":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1650467363":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1052746114":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6800/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6801/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/14610813":"2024-05-23T22:52:45.395791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T22:59:53.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.194+0000 7fc90a914640 1 -- 172.21.15.84:0/3139210315 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc8f00a2b90 msgr2=0x7fc8f00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:53.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.194+0000 7fc90a914640 1 --2- 172.21.15.84:0/3139210315 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc8f00a2b90 0x7fc8f00a5050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc9041990c0 tx=0x7fc8ec00b390 comp rx=0 tx=0).stop 2024-05-22T22:59:53.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.194+0000 7fc90a914640 1 -- 172.21.15.84:0/3139210315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9041018c0 msgr2=0x7fc904197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:53.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.194+0000 7fc90a914640 1 --2- 172.21.15.84:0/3139210315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9041018c0 0x7fc904197bb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4031610 tx=0x7fc8f40043b0 comp rx=0 tx=0).stop 2024-05-22T22:59:53.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.194+0000 7fc90a914640 1 -- 172.21.15.84:0/3139210315 shutdown_connections 2024-05-22T22:59:53.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.194+0000 7fc90a914640 1 --2- 172.21.15.84:0/3139210315 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc8f00a2b90 0x7fc8f00a5050 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:53.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.194+0000 7fc90a914640 1 --2- 172.21.15.84:0/3139210315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc904103470 0x7fc90419f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:53.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.194+0000 7fc90a914640 1 --2- 172.21.15.84:0/3139210315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc904102ab0 0x7fc9041980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:53.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.194+0000 7fc90a914640 1 --2- 172.21.15.84:0/3139210315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9041018c0 0x7fc904197bb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:53.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.194+0000 7fc90a914640 1 -- 172.21.15.84:0/3139210315 >> 172.21.15.84:0/3139210315 conn(0x7fc9040fce50 msgr2=0x7fc904105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:53.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.194+0000 7fc90a914640 1 -- 172.21.15.84:0/3139210315 shutdown_connections 2024-05-22T22:59:53.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.194+0000 7fc90a914640 1 -- 172.21.15.84:0/3139210315 wait complete. 2024-05-22T22:59:53.206 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-22T22:59:53.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.308+0000 7f5605f02640 1 -- 172.21.15.84:0/2972825848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56001007d0 msgr2=0x7f5600100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:53.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.308+0000 7f5605f02640 1 --2- 172.21.15.84:0/2972825848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56001007d0 0x7f5600100bf0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009900 tx=0x7f55f002a410 comp rx=0 tx=0).stop 2024-05-22T22:59:53.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.308+0000 7f5605f02640 1 -- 172.21.15.84:0/2972825848 shutdown_connections 2024-05-22T22:59:53.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.308+0000 7f5605f02640 1 --2- 172.21.15.84:0/2972825848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5600102380 0x7f5600108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:53.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.308+0000 7f5605f02640 1 --2- 172.21.15.84:0/2972825848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56001019c0 0x7f5600101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:53.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.308+0000 7f5605f02640 1 --2- 172.21.15.84:0/2972825848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56001007d0 0x7f5600100bf0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.308+0000 7f5605f02640 1 -- 172.21.15.84:0/2972825848 >> 172.21.15.84:0/2972825848 conn(0x7f56000fbd60 msgr2=0x7f56000fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.308+0000 7f5605f02640 1 -- 172.21.15.84:0/2972825848 shutdown_connections 2024-05-22T22:59:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.308+0000 7f5605f02640 1 -- 172.21.15.84:0/2972825848 wait complete. 2024-05-22T22:59:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.308+0000 7f5605f02640 1 Processor -- start 2024-05-22T22:59:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.309+0000 7f5605f02640 1 -- start start 2024-05-22T22:59:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.309+0000 7f5605f02640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56001007d0 0x7f5600104900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.309+0000 7f5605f02640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56001019c0 0x7f5600102f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.309+0000 7f5605f02640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5600102380 0x7f5600103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.309+0000 7f5605f02640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f560010c520 con 0x7f5600102380 2024-05-22T22:59:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.309+0000 7f5605f02640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f560010c3a0 con 0x7f56001019c0 2024-05-22T22:59:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.309+0000 7f5605f02640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f560010c6a0 con 0x7f56001007d0 2024-05-22T22:59:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.309+0000 7f55fffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5600102380 0x7f5600103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.309+0000 7f55fffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5600102380 0x7f5600103480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41756/0 (socket says 172.21.15.84:41756) 2024-05-22T22:59:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.309+0000 7f55fffff640 1 -- 172.21.15.84:0/946926282 learned_addr learned my addr 172.21.15.84:0/946926282 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.309+0000 7f55feffd640 1 --2- 172.21.15.84:0/946926282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56001019c0 0x7f5600102f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.309+0000 7f55ff7fe640 1 --2- 172.21.15.84:0/946926282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56001007d0 0x7f5600104900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.309+0000 7f55fffff640 1 -- 172.21.15.84:0/946926282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56001007d0 msgr2=0x7f5600104900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.309+0000 7f55fffff640 1 --2- 172.21.15.84:0/946926282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56001007d0 0x7f5600104900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.309+0000 7f55fffff640 1 -- 172.21.15.84:0/946926282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56001019c0 msgr2=0x7f5600102f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.309+0000 7f55fffff640 1 --2- 172.21.15.84:0/946926282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56001019c0 0x7f5600102f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.309+0000 7f55fffff640 1 -- 172.21.15.84:0/946926282 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5600103d10 con 0x7f5600102380 2024-05-22T22:59:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.309+0000 7f55ff7fe640 1 --2- 172.21.15.84:0/946926282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56001007d0 0x7f5600104900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T22:59:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.309+0000 7f55feffd640 1 --2- 172.21.15.84:0/946926282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56001019c0 0x7f5600102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T22:59:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.310+0000 7f55fffff640 1 --2- 172.21.15.84:0/946926282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5600102380 0x7f5600103480 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f55f800b970 tx=0x7f55f800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.310+0000 7f55fcff9640 1 -- 172.21.15.84:0/946926282 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8019070 con 0x7f5600102380 2024-05-22T22:59:53.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.310+0000 7f55fcff9640 1 -- 172.21.15.84:0/946926282 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55f800cb40 con 0x7f5600102380 2024-05-22T22:59:53.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.310+0000 7f5605f02640 1 -- 172.21.15.84:0/946926282 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56001a1e90 con 0x7f5600102380 2024-05-22T22:59:53.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.310+0000 7f55fcff9640 1 -- 172.21.15.84:0/946926282 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8007650 con 0x7f5600102380 2024-05-22T22:59:53.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.310+0000 7f5605f02640 1 -- 172.21.15.84:0/946926282 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56001a2370 con 0x7f5600102380 2024-05-22T22:59:53.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.311+0000 7f5605f02640 1 -- 172.21.15.84:0/946926282 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56001a2690 con 0x7f5600102380 2024-05-22T22:59:53.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.312+0000 7f55fcff9640 1 -- 172.21.15.84:0/946926282 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f55f801c070 con 0x7f5600102380 2024-05-22T22:59:53.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.313+0000 7f55fcff9640 1 --2- 172.21.15.84:0/946926282 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f55ec0a2b90 0x7f55ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:53.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.313+0000 7f55ff7fe640 1 --2- 172.21.15.84:0/946926282 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f55ec0a2b90 0x7f55ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:53.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.313+0000 7f55fcff9640 1 -- 172.21.15.84:0/946926282 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f55f80ce510 con 0x7f5600102380 2024-05-22T22:59:53.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.314+0000 7f55ff7fe640 1 --2- 172.21.15.84:0/946926282 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f55ec0a2b90 0x7f55ec0a5050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f55f0000c00 tx=0x7f55f0031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:53.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.318+0000 7f55fcff9640 1 -- 172.21.15.84:0/946926282 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f55f809b5c0 con 0x7f5600102380 2024-05-22T22:59:53.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.531+0000 7f5605f02640 1 -- 172.21.15.84:0/946926282 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f56001a2a90 con 0x7f5600102380 2024-05-22T22:59:53.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.585+0000 7f55fcff9640 1 -- 172.21.15.84:0/946926282 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f55f809aec0 con 0x7f5600102380 2024-05-22T22:59:53.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.587+0000 7f5605f02640 1 -- 172.21.15.84:0/946926282 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f55ec0a2b90 msgr2=0x7f55ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:53.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.587+0000 7f5605f02640 1 --2- 172.21.15.84:0/946926282 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f55ec0a2b90 0x7f55ec0a5050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f55f0000c00 tx=0x7f55f0031400 comp rx=0 tx=0).stop 2024-05-22T22:59:53.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.587+0000 7f5605f02640 1 -- 172.21.15.84:0/946926282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5600102380 msgr2=0x7f5600103480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:53.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.587+0000 7f5605f02640 1 --2- 172.21.15.84:0/946926282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5600102380 0x7f5600103480 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f55f800b970 tx=0x7f55f800be30 comp rx=0 tx=0).stop 2024-05-22T22:59:53.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.587+0000 7f5605f02640 1 -- 172.21.15.84:0/946926282 shutdown_connections 2024-05-22T22:59:53.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.587+0000 7f5605f02640 1 --2- 172.21.15.84:0/946926282 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f55ec0a2b90 0x7f55ec0a5050 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:53.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.587+0000 7f5605f02640 1 --2- 172.21.15.84:0/946926282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5600102380 0x7f5600103480 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:53.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.587+0000 7f5605f02640 1 --2- 172.21.15.84:0/946926282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56001019c0 0x7f5600102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:53.589 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.587+0000 7f5605f02640 1 --2- 172.21.15.84:0/946926282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56001007d0 0x7f5600104900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:53.589 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.587+0000 7f5605f02640 1 -- 172.21.15.84:0/946926282 >> 172.21.15.84:0/946926282 conn(0x7f56000fbd60 msgr2=0x7f5600069e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:53.589 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.587+0000 7f5605f02640 1 -- 172.21.15.84:0/946926282 shutdown_connections 2024-05-22T22:59:53.589 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.587+0000 7f5605f02640 1 -- 172.21.15.84:0/946926282 wait complete. 2024-05-22T22:59:53.591 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:53 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/4131633920' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-22T22:59:53.591 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:53 smithi084 ceph-mon[32186]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-22T22:59:53.591 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:53 smithi084 ceph-mon[32186]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-22T22:59:53.591 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:53 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/4041681581' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T22:59:53.591 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:53 smithi084 ceph-mon[32186]: osdmap e86: 12 total, 12 up, 12 in 2024-05-22T22:59:53.591 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:53 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3139210315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T22:59:53.591 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:53 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/946926282' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-22T22:59:53.600 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-05-22T22:59:53.600 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-05-22T22:59:53.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.704+0000 7fc12af9e640 1 -- 172.21.15.84:0/1638628614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1241007d0 msgr2=0x7fc124100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:53.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.704+0000 7fc12af9e640 1 --2- 172.21.15.84:0/1638628614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1241007d0 0x7fc124100bf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc120009900 tx=0x7fc12002a410 comp rx=0 tx=0).stop 2024-05-22T22:59:53.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.704+0000 7fc12af9e640 1 -- 172.21.15.84:0/1638628614 shutdown_connections 2024-05-22T22:59:53.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.704+0000 7fc12af9e640 1 --2- 172.21.15.84:0/1638628614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc124102380 0x7fc124108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:53.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.704+0000 7fc12af9e640 1 --2- 172.21.15.84:0/1638628614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1241019c0 0x7fc124101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:53.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.704+0000 7fc12af9e640 1 --2- 172.21.15.84:0/1638628614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1241007d0 0x7fc124100bf0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:53.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.704+0000 7fc12af9e640 1 -- 172.21.15.84:0/1638628614 >> 172.21.15.84:0/1638628614 conn(0x7fc1240fbd40 msgr2=0x7fc1240fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:53.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.704+0000 7fc12af9e640 1 -- 172.21.15.84:0/1638628614 shutdown_connections 2024-05-22T22:59:53.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.704+0000 7fc12af9e640 1 -- 172.21.15.84:0/1638628614 wait complete. 2024-05-22T22:59:53.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.705+0000 7fc12af9e640 1 Processor -- start 2024-05-22T22:59:53.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.705+0000 7fc12af9e640 1 -- start start 2024-05-22T22:59:53.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.705+0000 7fc12af9e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1241007d0 0x7fc12406a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:53.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.705+0000 7fc12af9e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1241019c0 0x7fc12419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:53.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.705+0000 7fc12af9e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc124102380 0x7fc12419f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:53.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.705+0000 7fc12af9e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc12410c520 con 0x7fc1241007d0 2024-05-22T22:59:53.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.705+0000 7fc12af9e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc12410c3a0 con 0x7fc124102380 2024-05-22T22:59:53.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.705+0000 7fc12af9e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc12410c6a0 con 0x7fc1241019c0 2024-05-22T22:59:53.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.705+0000 7fc128d3a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1241007d0 0x7fc12406a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:53.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.705+0000 7fc128d3a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1241007d0 0x7fc12406a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41772/0 (socket says 172.21.15.84:41772) 2024-05-22T22:59:53.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.705+0000 7fc128d3a640 1 -- 172.21.15.84:0/2756118123 learned_addr learned my addr 172.21.15.84:0/2756118123 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:53.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.705+0000 7fc128d3a640 1 -- 172.21.15.84:0/2756118123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1241019c0 msgr2=0x7fc12419cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:53.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.705+0000 7fc128d3a640 1 --2- 172.21.15.84:0/2756118123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1241019c0 0x7fc12419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:53.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.705+0000 7fc128d3a640 1 -- 172.21.15.84:0/2756118123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc124102380 msgr2=0x7fc12419f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:53.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.705+0000 7fc128d3a640 1 --2- 172.21.15.84:0/2756118123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc124102380 0x7fc12419f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:53.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.705+0000 7fc128d3a640 1 -- 172.21.15.84:0/2756118123 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12419fa00 con 0x7fc1241007d0 2024-05-22T22:59:53.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.705+0000 7fc128d3a640 1 --2- 172.21.15.84:0/2756118123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1241007d0 0x7fc12406a9e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc120031610 tx=0x7fc1200043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:53.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.705+0000 7fc119ffb640 1 -- 172.21.15.84:0/2756118123 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12003f030 con 0x7fc1241007d0 2024-05-22T22:59:53.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.705+0000 7fc119ffb640 1 -- 172.21.15.84:0/2756118123 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc120003ce0 con 0x7fc1241007d0 2024-05-22T22:59:53.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.705+0000 7fc12af9e640 1 -- 172.21.15.84:0/2756118123 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc12419fc30 con 0x7fc1241007d0 2024-05-22T22:59:53.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.705+0000 7fc119ffb640 1 -- 172.21.15.84:0/2756118123 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120035070 con 0x7fc1241007d0 2024-05-22T22:59:53.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.706+0000 7fc12af9e640 1 -- 172.21.15.84:0/2756118123 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1241a01f0 con 0x7fc1241007d0 2024-05-22T22:59:53.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.707+0000 7fc12af9e640 1 -- 172.21.15.84:0/2756118123 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc124107c60 con 0x7fc1241007d0 2024-05-22T22:59:53.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.708+0000 7fc119ffb640 1 -- 172.21.15.84:0/2756118123 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc120042030 con 0x7fc1241007d0 2024-05-22T22:59:53.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.708+0000 7fc119ffb640 1 --2- 172.21.15.84:0/2756118123 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc10c0a2c10 0x7fc10c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:53.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.708+0000 7fc11bfff640 1 --2- 172.21.15.84:0/2756118123 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc10c0a2c10 0x7fc10c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:53.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.708+0000 7fc119ffb640 1 -- 172.21.15.84:0/2756118123 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc1200ef7f0 con 0x7fc1241007d0 2024-05-22T22:59:53.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.709+0000 7fc11bfff640 1 --2- 172.21.15.84:0/2756118123 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc10c0a2c10 0x7fc10c0a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc12406a750 tx=0x7fc114006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:53.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.712+0000 7fc119ffb640 1 -- 172.21.15.84:0/2756118123 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc1200bc8a0 con 0x7fc1241007d0 2024-05-22T22:59:53.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:53 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/4131633920' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-22T22:59:53.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:53 smithi204 ceph-mon[36211]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-22T22:59:53.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:53 smithi204 ceph-mon[36211]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-22T22:59:53.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:53 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/4041681581' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T22:59:53.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:53 smithi204 ceph-mon[36211]: osdmap e86: 12 total, 12 up, 12 in 2024-05-22T22:59:53.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:53 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3139210315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T22:59:53.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:53 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/946926282' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-22T22:59:53.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:53.926+0000 7fc12af9e640 1 -- 172.21.15.84:0/2756118123 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fc12402d400 con 0x7fc1241007d0 2024-05-22T22:59:54.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:53 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/4131633920' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-22T22:59:54.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:53 smithi096 ceph-mon[36392]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-22T22:59:54.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:53 smithi096 ceph-mon[36392]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-22T22:59:54.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:53 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/4041681581' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T22:59:54.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:53 smithi096 ceph-mon[36392]: osdmap e86: 12 total, 12 up, 12 in 2024-05-22T22:59:54.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:53 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3139210315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T22:59:54.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:53 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/946926282' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-22T22:59:54.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.596+0000 7fc119ffb640 1 -- 172.21.15.84:0/2756118123 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v16) ==== 121+0+0 (secure 0 0 0) 0x7fc1200bc1a0 con 0x7fc1241007d0 2024-05-22T22:59:54.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.598+0000 7fc12af9e640 1 -- 172.21.15.84:0/2756118123 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc10c0a2c10 msgr2=0x7fc10c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:54.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.598+0000 7fc12af9e640 1 --2- 172.21.15.84:0/2756118123 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc10c0a2c10 0x7fc10c0a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc12406a750 tx=0x7fc114006c90 comp rx=0 tx=0).stop 2024-05-22T22:59:54.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.599+0000 7fc12af9e640 1 -- 172.21.15.84:0/2756118123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1241007d0 msgr2=0x7fc12406a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:54.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.599+0000 7fc12af9e640 1 --2- 172.21.15.84:0/2756118123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1241007d0 0x7fc12406a9e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc120031610 tx=0x7fc1200043b0 comp rx=0 tx=0).stop 2024-05-22T22:59:54.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.599+0000 7fc12af9e640 1 -- 172.21.15.84:0/2756118123 shutdown_connections 2024-05-22T22:59:54.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.599+0000 7fc12af9e640 1 --2- 172.21.15.84:0/2756118123 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc10c0a2c10 0x7fc10c0a50d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:54.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.599+0000 7fc12af9e640 1 --2- 172.21.15.84:0/2756118123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc124102380 0x7fc12419f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:54.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.599+0000 7fc12af9e640 1 --2- 172.21.15.84:0/2756118123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1241019c0 0x7fc12419cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:54.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.599+0000 7fc12af9e640 1 --2- 172.21.15.84:0/2756118123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1241007d0 0x7fc12406a9e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:54.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.599+0000 7fc12af9e640 1 -- 172.21.15.84:0/2756118123 >> 172.21.15.84:0/2756118123 conn(0x7fc1240fbd40 msgr2=0x7fc124104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:54.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.599+0000 7fc12af9e640 1 -- 172.21.15.84:0/2756118123 shutdown_connections 2024-05-22T22:59:54.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.599+0000 7fc12af9e640 1 -- 172.21.15.84:0/2756118123 wait complete. 2024-05-22T22:59:54.613 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-05-22T22:59:54.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.720+0000 7f1105437640 1 -- 172.21.15.84:0/2800163395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1100101440 msgr2=0x7f1100108ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:54.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.720+0000 7f1105437640 1 --2- 172.21.15.84:0/2800163395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1100101440 0x7f1100108ce0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f10f8009ca0 tx=0x7f10f802a8c0 comp rx=0 tx=0).stop 2024-05-22T22:59:54.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.720+0000 7f1105437640 1 -- 172.21.15.84:0/2800163395 shutdown_connections 2024-05-22T22:59:54.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.720+0000 7f1105437640 1 --2- 172.21.15.84:0/2800163395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1100101440 0x7f1100108ce0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:54.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.720+0000 7f1105437640 1 --2- 172.21.15.84:0/2800163395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1100100a80 0x7f1100100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:54.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.720+0000 7f1105437640 1 --2- 172.21.15.84:0/2800163395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1100101d50 0x7f1100102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:54.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.720+0000 7f1105437640 1 -- 172.21.15.84:0/2800163395 >> 172.21.15.84:0/2800163395 conn(0x7f11000fb4b0 msgr2=0x7f11000fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:54.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.720+0000 7f1105437640 1 -- 172.21.15.84:0/2800163395 shutdown_connections 2024-05-22T22:59:54.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.720+0000 7f1105437640 1 -- 172.21.15.84:0/2800163395 wait complete. 2024-05-22T22:59:54.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.720+0000 7f1105437640 1 Processor -- start 2024-05-22T22:59:54.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.721+0000 7f1105437640 1 -- start start 2024-05-22T22:59:54.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.721+0000 7f1105437640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1100100a80 0x7f1100199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:54.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.721+0000 7f1105437640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1100101440 0x7f110019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:54.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.721+0000 7f1105437640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1100101d50 0x7f11001a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:54.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.721+0000 7f1105437640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f110010c450 con 0x7f1100100a80 2024-05-22T22:59:54.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.721+0000 7f1105437640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f110010c2d0 con 0x7f1100101d50 2024-05-22T22:59:54.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.721+0000 7f1105437640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f110010c5d0 con 0x7f1100101440 2024-05-22T22:59:54.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.721+0000 7f10feffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1100100a80 0x7f1100199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:54.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.721+0000 7f10feffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1100100a80 0x7f1100199de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41786/0 (socket says 172.21.15.84:41786) 2024-05-22T22:59:54.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.721+0000 7f10feffd640 1 -- 172.21.15.84:0/1646051490 learned_addr learned my addr 172.21.15.84:0/1646051490 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:54.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.721+0000 7f10ff7fe640 1 --2- 172.21.15.84:0/1646051490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1100101d50 0x7f11001a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:54.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.721+0000 7f10feffd640 1 -- 172.21.15.84:0/1646051490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1100101440 msgr2=0x7f110019a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:54.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.721+0000 7f10feffd640 1 --2- 172.21.15.84:0/1646051490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1100101440 0x7f110019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:54.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.721+0000 7f10feffd640 1 -- 172.21.15.84:0/1646051490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1100101d50 msgr2=0x7f11001a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:54.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.721+0000 7f10feffd640 1 --2- 172.21.15.84:0/1646051490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1100101d50 0x7f11001a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:54.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.721+0000 7f10feffd640 1 -- 172.21.15.84:0/1646051490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11001a1a00 con 0x7f1100100a80 2024-05-22T22:59:54.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.721+0000 7f10ff7fe640 1 --2- 172.21.15.84:0/1646051490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1100101d50 0x7f11001a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T22:59:54.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.721+0000 7f10feffd640 1 --2- 172.21.15.84:0/1646051490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1100100a80 0x7f1100199de0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f10f000d770 tx=0x7f10f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:54.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.721+0000 7f10fcff9640 1 -- 172.21.15.84:0/1646051490 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0017070 con 0x7f1100100a80 2024-05-22T22:59:54.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.721+0000 7f10fcff9640 1 -- 172.21.15.84:0/1646051490 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10f0009c30 con 0x7f1100100a80 2024-05-22T22:59:54.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.721+0000 7f1105437640 1 -- 172.21.15.84:0/1646051490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11001a1cf0 con 0x7f1100100a80 2024-05-22T22:59:54.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.721+0000 7f10fcff9640 1 -- 172.21.15.84:0/1646051490 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0003dd0 con 0x7f1100100a80 2024-05-22T22:59:54.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.722+0000 7f1105437640 1 -- 172.21.15.84:0/1646051490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11001a21e0 con 0x7f1100100a80 2024-05-22T22:59:54.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.723+0000 7f1105437640 1 -- 172.21.15.84:0/1646051490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11001a2410 con 0x7f1100100a80 2024-05-22T22:59:54.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.724+0000 7f10fcff9640 1 -- 172.21.15.84:0/1646051490 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f10f0003fb0 con 0x7f1100100a80 2024-05-22T22:59:54.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.724+0000 7f10fcff9640 1 --2- 172.21.15.84:0/1646051490 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f10ec0a2b90 0x7f10ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:54.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.724+0000 7f10f67fc640 1 --2- 172.21.15.84:0/1646051490 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f10ec0a2b90 0x7f10ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:54.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.725+0000 7f10fcff9640 1 -- 172.21.15.84:0/1646051490 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f10f00ce930 con 0x7f1100100a80 2024-05-22T22:59:54.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.725+0000 7f10f67fc640 1 --2- 172.21.15.84:0/1646051490 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f10ec0a2b90 0x7f10ec0a5050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f110019b2f0 tx=0x7f10e8005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:54.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.729+0000 7f10fcff9640 1 -- 172.21.15.84:0/1646051490 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f10f009b9e0 con 0x7f1100100a80 2024-05-22T22:59:54.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:54 smithi084 ceph-mon[32186]: pgmap v290: 97 pgs: 14 creating+peering, 36 active+clean, 47 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:54.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:54 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/946926282' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-22T22:59:54.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:54 smithi084 ceph-mon[32186]: mds.? [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] up:active 2024-05-22T22:59:54.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:54 smithi084 ceph-mon[32186]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-22T22:59:54.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:54 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2756118123' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-22T22:59:54.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:54 smithi084 ceph-mon[32186]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-22T22:59:54.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:54 smithi084 ceph-mon[32186]: Cluster is now healthy 2024-05-22T22:59:54.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:54 smithi204 ceph-mon[36211]: pgmap v290: 97 pgs: 14 creating+peering, 36 active+clean, 47 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:54.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:54 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/946926282' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-22T22:59:54.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:54 smithi204 ceph-mon[36211]: mds.? [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] up:active 2024-05-22T22:59:54.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:54 smithi204 ceph-mon[36211]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-22T22:59:54.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:54 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2756118123' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-22T22:59:54.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:54 smithi204 ceph-mon[36211]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-22T22:59:54.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:54 smithi204 ceph-mon[36211]: Cluster is now healthy 2024-05-22T22:59:54.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:54.944+0000 7f1105437640 1 -- 172.21.15.84:0/1646051490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f11001a27a0 con 0x7f1100100a80 2024-05-22T22:59:54.950 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:54 smithi096 ceph-mon[36392]: pgmap v290: 97 pgs: 14 creating+peering, 36 active+clean, 47 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T22:59:54.950 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:54 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/946926282' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-22T22:59:54.950 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:54 smithi096 ceph-mon[36392]: mds.? [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] up:active 2024-05-22T22:59:54.950 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:54 smithi096 ceph-mon[36392]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-22T22:59:54.950 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:54 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2756118123' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-22T22:59:54.950 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:54 smithi096 ceph-mon[36392]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-22T22:59:54.950 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:54 smithi096 ceph-mon[36392]: Cluster is now healthy 2024-05-22T22:59:55.602 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.602+0000 7f10fcff9640 1 -- 172.21.15.84:0/1646051490 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v18) ==== 115+0+0 (secure 0 0 0) 0x7f10f0009de0 con 0x7f1100100a80 2024-05-22T22:59:55.605 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.605+0000 7f1105437640 1 -- 172.21.15.84:0/1646051490 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f10ec0a2b90 msgr2=0x7f10ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:55.605 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.605+0000 7f1105437640 1 --2- 172.21.15.84:0/1646051490 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f10ec0a2b90 0x7f10ec0a5050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f110019b2f0 tx=0x7f10e8005bf0 comp rx=0 tx=0).stop 2024-05-22T22:59:55.605 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.605+0000 7f1105437640 1 -- 172.21.15.84:0/1646051490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1100100a80 msgr2=0x7f1100199de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:55.605 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.605+0000 7f1105437640 1 --2- 172.21.15.84:0/1646051490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1100100a80 0x7f1100199de0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f10f000d770 tx=0x7f10f000dc30 comp rx=0 tx=0).stop 2024-05-22T22:59:55.605 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.605+0000 7f1105437640 1 -- 172.21.15.84:0/1646051490 shutdown_connections 2024-05-22T22:59:55.605 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.605+0000 7f1105437640 1 --2- 172.21.15.84:0/1646051490 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f10ec0a2b90 0x7f10ec0a5050 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:55.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.605+0000 7f1105437640 1 --2- 172.21.15.84:0/1646051490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1100101d50 0x7f11001a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:55.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.605+0000 7f1105437640 1 --2- 172.21.15.84:0/1646051490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1100101440 0x7f110019a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:55.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.605+0000 7f1105437640 1 --2- 172.21.15.84:0/1646051490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1100100a80 0x7f1100199de0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:55.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.605+0000 7f1105437640 1 -- 172.21.15.84:0/1646051490 >> 172.21.15.84:0/1646051490 conn(0x7f11000fb4b0 msgr2=0x7f11000fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:55.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.605+0000 7f1105437640 1 -- 172.21.15.84:0/1646051490 shutdown_connections 2024-05-22T22:59:55.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.605+0000 7f1105437640 1 -- 172.21.15.84:0/1646051490 wait complete. 2024-05-22T22:59:55.618 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-05-22T22:59:55.618 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-05-22T22:59:55.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.725+0000 7f371c963640 1 -- 172.21.15.84:0/1285652034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3718102ab0 msgr2=0x7f3718102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:55.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.725+0000 7f371c963640 1 --2- 172.21.15.84:0/1285652034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3718102ab0 0x7f3718102f30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f370c009900 tx=0x7f370c02a410 comp rx=0 tx=0).stop 2024-05-22T22:59:55.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.725+0000 7f371c963640 1 -- 172.21.15.84:0/1285652034 shutdown_connections 2024-05-22T22:59:55.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.725+0000 7f371c963640 1 --2- 172.21.15.84:0/1285652034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3718103470 0x7f3718109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:55.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.725+0000 7f371c963640 1 --2- 172.21.15.84:0/1285652034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3718102ab0 0x7f3718102f30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:55.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.725+0000 7f371c963640 1 --2- 172.21.15.84:0/1285652034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37181018c0 0x7f3718101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:55.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.725+0000 7f371c963640 1 -- 172.21.15.84:0/1285652034 >> 172.21.15.84:0/1285652034 conn(0x7f37180fce50 msgr2=0x7f37180ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:55.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.725+0000 7f371c963640 1 -- 172.21.15.84:0/1285652034 shutdown_connections 2024-05-22T22:59:55.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.725+0000 7f371c963640 1 -- 172.21.15.84:0/1285652034 wait complete. 2024-05-22T22:59:55.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.726+0000 7f371c963640 1 Processor -- start 2024-05-22T22:59:55.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.726+0000 7f371c963640 1 -- start start 2024-05-22T22:59:55.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.726+0000 7f371c963640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37181018c0 0x7f3718197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:55.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.726+0000 7f371c963640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3718102ab0 0x7f3718198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:55.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.726+0000 7f371c963640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3718103470 0x7f371819f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:55.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.726+0000 7f371c963640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f371810d540 con 0x7f3718103470 2024-05-22T22:59:55.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.726+0000 7f371c963640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f371810d3c0 con 0x7f37181018c0 2024-05-22T22:59:55.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.726+0000 7f371c963640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f371810d6c0 con 0x7f3718102ab0 2024-05-22T22:59:55.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.726+0000 7f3716d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3718103470 0x7f371819f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:55.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.726+0000 7f3716d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3718103470 0x7f371819f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41806/0 (socket says 172.21.15.84:41806) 2024-05-22T22:59:55.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.726+0000 7f3716d9d640 1 -- 172.21.15.84:0/3871473434 learned_addr learned my addr 172.21.15.84:0/3871473434 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:55.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.726+0000 7f371659c640 1 --2- 172.21.15.84:0/3871473434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37181018c0 0x7f3718197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:55.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.726+0000 7f3715d9b640 1 --2- 172.21.15.84:0/3871473434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3718102ab0 0x7f3718198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:55.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.726+0000 7f3716d9d640 1 -- 172.21.15.84:0/3871473434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3718102ab0 msgr2=0x7f3718198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:55.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.726+0000 7f3716d9d640 1 --2- 172.21.15.84:0/3871473434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3718102ab0 0x7f3718198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:55.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.727+0000 7f3716d9d640 1 -- 172.21.15.84:0/3871473434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37181018c0 msgr2=0x7f3718197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:55.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.727+0000 7f3716d9d640 1 --2- 172.21.15.84:0/3871473434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37181018c0 0x7f3718197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:55.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.727+0000 7f3716d9d640 1 -- 172.21.15.84:0/3871473434 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371819f7e0 con 0x7f3718103470 2024-05-22T22:59:55.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.727+0000 7f3716d9d640 1 --2- 172.21.15.84:0/3871473434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3718103470 0x7f371819f170 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f371000eb10 tx=0x7f371000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:55.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.727+0000 7f37077fe640 1 -- 172.21.15.84:0/3871473434 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710004b20 con 0x7f3718103470 2024-05-22T22:59:55.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.727+0000 7f371c963640 1 -- 172.21.15.84:0/3871473434 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371819fad0 con 0x7f3718103470 2024-05-22T22:59:55.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.727+0000 7f37077fe640 1 -- 172.21.15.84:0/3871473434 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3710004cc0 con 0x7f3718103470 2024-05-22T22:59:55.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.727+0000 7f37077fe640 1 -- 172.21.15.84:0/3871473434 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710010760 con 0x7f3718103470 2024-05-22T22:59:55.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.727+0000 7f371c963640 1 -- 172.21.15.84:0/3871473434 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f371819fff0 con 0x7f3718103470 2024-05-22T22:59:55.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.728+0000 7f371c963640 1 -- 172.21.15.84:0/3871473434 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3718108cb0 con 0x7f3718103470 2024-05-22T22:59:55.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.729+0000 7f37077fe640 1 -- 172.21.15.84:0/3871473434 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3710005020 con 0x7f3718103470 2024-05-22T22:59:55.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.730+0000 7f37077fe640 1 --2- 172.21.15.84:0/3871473434 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f37000a2b90 0x7f37000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:55.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.730+0000 7f371659c640 1 --2- 172.21.15.84:0/3871473434 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f37000a2b90 0x7f37000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:55.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.730+0000 7f37077fe640 1 -- 172.21.15.84:0/3871473434 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f37100cf590 con 0x7f3718103470 2024-05-22T22:59:55.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.730+0000 7f371659c640 1 --2- 172.21.15.84:0/3871473434 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f37000a2b90 0x7f37000a5050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3718102910 tx=0x7f370800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:55.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.734+0000 7f37077fe640 1 -- 172.21.15.84:0/3871473434 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f371009c640 con 0x7f3718103470 2024-05-22T22:59:55.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:55 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2756118123' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-22T22:59:55.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:55 smithi084 ceph-mon[32186]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-22T22:59:55.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:55 smithi084 ceph-mon[32186]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-05-22T22:59:55.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:55 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1646051490' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-22T22:59:55.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:55 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2756118123' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-22T22:59:55.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:55 smithi204 ceph-mon[36211]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-22T22:59:55.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:55 smithi204 ceph-mon[36211]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-05-22T22:59:55.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:55 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1646051490' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-22T22:59:55.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.933+0000 7f371c963640 1 -- 172.21.15.84:0/3871473434 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3718061cb0 con 0x7f37000a2b90 2024-05-22T22:59:55.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.951+0000 7f37077fe640 1 -- 172.21.15.84:0/3871473434 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3718061cb0 con 0x7f37000a2b90 2024-05-22T22:59:55.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.954+0000 7f371c963640 1 -- 172.21.15.84:0/3871473434 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f37000a2b90 msgr2=0x7f37000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:55.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.954+0000 7f371c963640 1 --2- 172.21.15.84:0/3871473434 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f37000a2b90 0x7f37000a5050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3718102910 tx=0x7f370800b390 comp rx=0 tx=0).stop 2024-05-22T22:59:55.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.954+0000 7f371c963640 1 -- 172.21.15.84:0/3871473434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3718103470 msgr2=0x7f371819f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:55.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.954+0000 7f371c963640 1 --2- 172.21.15.84:0/3871473434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3718103470 0x7f371819f170 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f371000eb10 tx=0x7f371000c2d0 comp rx=0 tx=0).stop 2024-05-22T22:59:55.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.954+0000 7f371c963640 1 -- 172.21.15.84:0/3871473434 shutdown_connections 2024-05-22T22:59:55.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.954+0000 7f371c963640 1 --2- 172.21.15.84:0/3871473434 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f37000a2b90 0x7f37000a5050 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:55.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.954+0000 7f371c963640 1 --2- 172.21.15.84:0/3871473434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3718103470 0x7f371819f170 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:55.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.954+0000 7f371c963640 1 --2- 172.21.15.84:0/3871473434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3718102ab0 0x7f3718198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:55.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.954+0000 7f371c963640 1 --2- 172.21.15.84:0/3871473434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37181018c0 0x7f3718197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:55.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.954+0000 7f371c963640 1 -- 172.21.15.84:0/3871473434 >> 172.21.15.84:0/3871473434 conn(0x7f37180fce50 msgr2=0x7f3718105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:55.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.954+0000 7f371c963640 1 -- 172.21.15.84:0/3871473434 shutdown_connections 2024-05-22T22:59:55.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:55.954+0000 7f371c963640 1 -- 172.21.15.84:0/3871473434 wait complete. 2024-05-22T22:59:55.965 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-22T22:59:55.970 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:55 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2756118123' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-22T22:59:55.970 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:55 smithi096 ceph-mon[36392]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-22T22:59:55.970 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:55 smithi096 ceph-mon[36392]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-05-22T22:59:55.971 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:55 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1646051490' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-22T22:59:56.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.070+0000 7f25c7f1b640 1 -- 172.21.15.84:0/677670786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25c0102330 msgr2=0x7f25c0104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:56.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.070+0000 7f25c7f1b640 1 --2- 172.21.15.84:0/677670786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25c0102330 0x7f25c0104720 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009900 tx=0x7f25b802a410 comp rx=0 tx=0).stop 2024-05-22T22:59:56.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.070+0000 7f25c7f1b640 1 -- 172.21.15.84:0/677670786 shutdown_connections 2024-05-22T22:59:56.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.070+0000 7f25c7f1b640 1 --2- 172.21.15.84:0/677670786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25c0107590 0x7f25c01099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:56.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.070+0000 7f25c7f1b640 1 --2- 172.21.15.84:0/677670786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25c0104c60 0x7f25c0107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:56.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.070+0000 7f25c7f1b640 1 --2- 172.21.15.84:0/677670786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25c0102330 0x7f25c0104720 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:56.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.070+0000 7f25c7f1b640 1 -- 172.21.15.84:0/677670786 >> 172.21.15.84:0/677670786 conn(0x7f25c00fbd60 msgr2=0x7f25c00fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:56.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.071+0000 7f25c7f1b640 1 -- 172.21.15.84:0/677670786 shutdown_connections 2024-05-22T22:59:56.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.071+0000 7f25c7f1b640 1 -- 172.21.15.84:0/677670786 wait complete. 2024-05-22T22:59:56.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.071+0000 7f25c7f1b640 1 Processor -- start 2024-05-22T22:59:56.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.071+0000 7f25c7f1b640 1 -- start start 2024-05-22T22:59:56.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.071+0000 7f25c7f1b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25c0102330 0x7f25c0197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:56.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.071+0000 7f25c7f1b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25c0104c60 0x7f25c01981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:56.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.071+0000 7f25c7f1b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25c0107590 0x7f25c019f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:56.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.071+0000 7f25c7f1b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f25c010d5b0 con 0x7f25c0102330 2024-05-22T22:59:56.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.071+0000 7f25c7f1b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f25c010d430 con 0x7f25c0104c60 2024-05-22T22:59:56.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.071+0000 7f25c7f1b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f25c010d730 con 0x7f25c0107590 2024-05-22T22:59:56.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.071+0000 7f25c5cb7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25c0102330 0x7f25c0197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:56.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.071+0000 7f25c5cb7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25c0102330 0x7f25c0197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41852/0 (socket says 172.21.15.84:41852) 2024-05-22T22:59:56.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.071+0000 7f25c5cb7640 1 -- 172.21.15.84:0/2586603149 learned_addr learned my addr 172.21.15.84:0/2586603149 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:56.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.071+0000 7f25c54b6640 1 --2- 172.21.15.84:0/2586603149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25c0104c60 0x7f25c01981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:56.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.071+0000 7f25c64b8640 1 --2- 172.21.15.84:0/2586603149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25c0107590 0x7f25c019f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:56.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.072+0000 7f25c5cb7640 1 -- 172.21.15.84:0/2586603149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25c0107590 msgr2=0x7f25c019f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:56.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.072+0000 7f25c5cb7640 1 --2- 172.21.15.84:0/2586603149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25c0107590 0x7f25c019f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:56.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.072+0000 7f25c5cb7640 1 -- 172.21.15.84:0/2586603149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25c0104c60 msgr2=0x7f25c01981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:56.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.072+0000 7f25c5cb7640 1 --2- 172.21.15.84:0/2586603149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25c0104c60 0x7f25c01981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:56.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.072+0000 7f25c5cb7640 1 -- 172.21.15.84:0/2586603149 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c019f890 con 0x7f25c0102330 2024-05-22T22:59:56.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.072+0000 7f25c5cb7640 1 --2- 172.21.15.84:0/2586603149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25c0102330 0x7f25c0197c80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009230 tx=0x7f25b8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:56.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.072+0000 7f25beffd640 1 -- 172.21.15.84:0/2586603149 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b803f030 con 0x7f25c0102330 2024-05-22T22:59:56.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.072+0000 7f25beffd640 1 -- 172.21.15.84:0/2586603149 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25b8003ce0 con 0x7f25c0102330 2024-05-22T22:59:56.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.072+0000 7f25beffd640 1 -- 172.21.15.84:0/2586603149 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b8035070 con 0x7f25c0102330 2024-05-22T22:59:56.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.072+0000 7f25c7f1b640 1 -- 172.21.15.84:0/2586603149 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25c019fb20 con 0x7f25c0102330 2024-05-22T22:59:56.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.072+0000 7f25c7f1b640 1 -- 172.21.15.84:0/2586603149 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25c01a0060 con 0x7f25c0102330 2024-05-22T22:59:56.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.073+0000 7f25c7f1b640 1 -- 172.21.15.84:0/2586603149 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25c00fd850 con 0x7f25c0102330 2024-05-22T22:59:56.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.074+0000 7f25beffd640 1 -- 172.21.15.84:0/2586603149 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f25b8042030 con 0x7f25c0102330 2024-05-22T22:59:56.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.075+0000 7f25beffd640 1 --2- 172.21.15.84:0/2586603149 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f25a80a2c10 0x7f25a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:56.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.075+0000 7f25c54b6640 1 --2- 172.21.15.84:0/2586603149 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f25a80a2c10 0x7f25a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:56.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.075+0000 7f25beffd640 1 -- 172.21.15.84:0/2586603149 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f25b80ef8c0 con 0x7f25c0102330 2024-05-22T22:59:56.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.075+0000 7f25c54b6640 1 --2- 172.21.15.84:0/2586603149 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f25a80a2c10 0x7f25a80a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f25c006aa10 tx=0x7f25b4009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:56.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.078+0000 7f25beffd640 1 -- 172.21.15.84:0/2586603149 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f25b80bc970 con 0x7f25c0102330 2024-05-22T22:59:56.282 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.282+0000 7f25c7f1b640 1 -- 172.21.15.84:0/2586603149 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f25c0061d40 con 0x7f25a80a2c10 2024-05-22T22:59:56.305 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.305+0000 7f25beffd640 1 -- 172.21.15.84:0/2586603149 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f25c0061d40 con 0x7f25a80a2c10 2024-05-22T22:59:56.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.307+0000 7f25c7f1b640 1 -- 172.21.15.84:0/2586603149 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f25a80a2c10 msgr2=0x7f25a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:56.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.307+0000 7f25c7f1b640 1 --2- 172.21.15.84:0/2586603149 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f25a80a2c10 0x7f25a80a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f25c006aa10 tx=0x7f25b4009110 comp rx=0 tx=0).stop 2024-05-22T22:59:56.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.307+0000 7f25c7f1b640 1 -- 172.21.15.84:0/2586603149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25c0102330 msgr2=0x7f25c0197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:56.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.307+0000 7f25c7f1b640 1 --2- 172.21.15.84:0/2586603149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25c0102330 0x7f25c0197c80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009230 tx=0x7f25b8004360 comp rx=0 tx=0).stop 2024-05-22T22:59:56.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.307+0000 7f25c7f1b640 1 -- 172.21.15.84:0/2586603149 shutdown_connections 2024-05-22T22:59:56.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.307+0000 7f25c7f1b640 1 --2- 172.21.15.84:0/2586603149 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f25a80a2c10 0x7f25a80a50d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:56.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.307+0000 7f25c7f1b640 1 --2- 172.21.15.84:0/2586603149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25c0107590 0x7f25c019f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:56.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.307+0000 7f25c7f1b640 1 --2- 172.21.15.84:0/2586603149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25c0104c60 0x7f25c01981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:56.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.307+0000 7f25c7f1b640 1 --2- 172.21.15.84:0/2586603149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25c0102330 0x7f25c0197c80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:56.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.307+0000 7f25c7f1b640 1 -- 172.21.15.84:0/2586603149 >> 172.21.15.84:0/2586603149 conn(0x7f25c00fbd60 msgr2=0x7f25c0102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:56.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.307+0000 7f25c7f1b640 1 -- 172.21.15.84:0/2586603149 shutdown_connections 2024-05-22T22:59:56.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.307+0000 7f25c7f1b640 1 -- 172.21.15.84:0/2586603149 wait complete. 2024-05-22T22:59:56.317 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:59:55 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:59:55.938+0000 7f735dc31700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-22T22:59:56.317 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:59:55 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:59:55.938+0000 7f735dc31700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-22T22:59:56.317 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:59:55 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:59:55.938+0000 7f735dc31700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-22T22:59:56.317 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:59:55 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:59:55.938+0000 7f735dc31700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-22T22:59:56.317 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 22:59:55 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T22:59:55.938+0000 7f735dc31700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-22T22:59:56.318 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-22T22:59:56.425 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.423+0000 7f91802a5640 1 -- 172.21.15.84:0/3681117152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9178103520 msgr2=0x7f9178109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:56.426 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.423+0000 7f91802a5640 1 --2- 172.21.15.84:0/3681117152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9178103520 0x7f9178109db0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9170009cd0 tx=0x7f917002a8e0 comp rx=0 tx=0).stop 2024-05-22T22:59:56.426 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.423+0000 7f91802a5640 1 -- 172.21.15.84:0/3681117152 shutdown_connections 2024-05-22T22:59:56.426 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.423+0000 7f91802a5640 1 --2- 172.21.15.84:0/3681117152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9178103520 0x7f9178109db0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:56.426 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.423+0000 7f91802a5640 1 --2- 172.21.15.84:0/3681117152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9178102b60 0x7f9178102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:56.426 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.423+0000 7f91802a5640 1 --2- 172.21.15.84:0/3681117152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91780ffb90 0x7f91780fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:56.426 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.423+0000 7f91802a5640 1 -- 172.21.15.84:0/3681117152 >> 172.21.15.84:0/3681117152 conn(0x7f91780fb4a0 msgr2=0x7f91780fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:56.426 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.423+0000 7f91802a5640 1 -- 172.21.15.84:0/3681117152 shutdown_connections 2024-05-22T22:59:56.426 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.423+0000 7f91802a5640 1 -- 172.21.15.84:0/3681117152 wait complete. 2024-05-22T22:59:56.426 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.424+0000 7f91802a5640 1 Processor -- start 2024-05-22T22:59:56.426 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.424+0000 7f91802a5640 1 -- start start 2024-05-22T22:59:56.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.424+0000 7f91802a5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91780ffb90 0x7f9178197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:56.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.424+0000 7f91802a5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9178102b60 0x7f91781980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:56.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.424+0000 7f91802a5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9178103520 0x7f917819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:56.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.424+0000 7f91802a5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f917810d560 con 0x7f9178103520 2024-05-22T22:59:56.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.424+0000 7f91802a5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f917810d3e0 con 0x7f91780ffb90 2024-05-22T22:59:56.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.424+0000 7f91802a5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f917810d6e0 con 0x7f9178102b60 2024-05-22T22:59:56.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.424+0000 7f917e842640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9178103520 0x7f917819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:56.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.424+0000 7f917e842640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9178103520 0x7f917819f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41858/0 (socket says 172.21.15.84:41858) 2024-05-22T22:59:56.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.424+0000 7f917e842640 1 -- 172.21.15.84:0/1256144678 learned_addr learned my addr 172.21.15.84:0/1256144678 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:56.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.424+0000 7f917e041640 1 --2- 172.21.15.84:0/1256144678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91780ffb90 0x7f9178197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:56.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.424+0000 7f917d840640 1 --2- 172.21.15.84:0/1256144678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9178102b60 0x7f91781980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:56.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.424+0000 7f917e842640 1 -- 172.21.15.84:0/1256144678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9178102b60 msgr2=0x7f91781980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:56.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.424+0000 7f917e842640 1 --2- 172.21.15.84:0/1256144678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9178102b60 0x7f91781980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:56.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.424+0000 7f917e842640 1 -- 172.21.15.84:0/1256144678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91780ffb90 msgr2=0x7f9178197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:56.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.424+0000 7f917e842640 1 --2- 172.21.15.84:0/1256144678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91780ffb90 0x7f9178197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:56.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.424+0000 7f917e842640 1 -- 172.21.15.84:0/1256144678 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f917819f7d0 con 0x7f9178103520 2024-05-22T22:59:56.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.424+0000 7f917e842640 1 --2- 172.21.15.84:0/1256144678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9178103520 0x7f917819f160 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9170004af0 tx=0x7f9170003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:56.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.425+0000 7f916f7fe640 1 -- 172.21.15.84:0/1256144678 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9170004450 con 0x7f9178103520 2024-05-22T22:59:56.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.425+0000 7f916f7fe640 1 -- 172.21.15.84:0/1256144678 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91700045f0 con 0x7f9178103520 2024-05-22T22:59:56.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.425+0000 7f91802a5640 1 -- 172.21.15.84:0/1256144678 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f917819fa60 con 0x7f9178103520 2024-05-22T22:59:56.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.425+0000 7f916f7fe640 1 -- 172.21.15.84:0/1256144678 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91700466c0 con 0x7f9178103520 2024-05-22T22:59:56.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.425+0000 7f91802a5640 1 -- 172.21.15.84:0/1256144678 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91781a0000 con 0x7f9178103520 2024-05-22T22:59:56.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.426+0000 7f91802a5640 1 -- 172.21.15.84:0/1256144678 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9178108c90 con 0x7f9178103520 2024-05-22T22:59:56.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.427+0000 7f916f7fe640 1 -- 172.21.15.84:0/1256144678 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9170046860 con 0x7f9178103520 2024-05-22T22:59:56.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.428+0000 7f916f7fe640 1 --2- 172.21.15.84:0/1256144678 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f91640a2c10 0x7f91640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:56.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.428+0000 7f917e041640 1 --2- 172.21.15.84:0/1256144678 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f91640a2c10 0x7f91640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:56.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.428+0000 7f916f7fe640 1 -- 172.21.15.84:0/1256144678 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f917002e030 con 0x7f9178103520 2024-05-22T22:59:56.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.428+0000 7f917e041640 1 --2- 172.21.15.84:0/1256144678 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f91640a2c10 0x7f91640a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9174005fd0 tx=0x7f917400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:56.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.433+0000 7f916f7fe640 1 -- 172.21.15.84:0/1256144678 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f917004b060 con 0x7f9178103520 2024-05-22T22:59:56.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.639+0000 7f91802a5640 1 -- 172.21.15.84:0/1256144678 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f9178061cb0 con 0x7f91640a2c10 2024-05-22T22:59:56.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.644+0000 7f916f7fe640 1 -- 172.21.15.84:0/1256144678 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f9178061cb0 con 0x7f91640a2c10 2024-05-22T22:59:56.645 INFO:teuthology.orchestra.run.smithi084.stdout:/volumes/qa/sv_0/999a6055-a3a7-4706-bb7b-143a0196aaa2 2024-05-22T22:59:56.647 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.647+0000 7f91802a5640 1 -- 172.21.15.84:0/1256144678 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f91640a2c10 msgr2=0x7f91640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:56.647 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.647+0000 7f91802a5640 1 --2- 172.21.15.84:0/1256144678 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f91640a2c10 0x7f91640a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9174005fd0 tx=0x7f917400b390 comp rx=0 tx=0).stop 2024-05-22T22:59:56.648 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.647+0000 7f91802a5640 1 -- 172.21.15.84:0/1256144678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9178103520 msgr2=0x7f917819f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:56.648 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.647+0000 7f91802a5640 1 --2- 172.21.15.84:0/1256144678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9178103520 0x7f917819f160 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9170004af0 tx=0x7f9170003c10 comp rx=0 tx=0).stop 2024-05-22T22:59:56.648 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.647+0000 7f91802a5640 1 -- 172.21.15.84:0/1256144678 shutdown_connections 2024-05-22T22:59:56.648 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.647+0000 7f91802a5640 1 --2- 172.21.15.84:0/1256144678 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f91640a2c10 0x7f91640a50d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:56.648 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.647+0000 7f91802a5640 1 --2- 172.21.15.84:0/1256144678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9178103520 0x7f917819f160 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:56.648 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.647+0000 7f91802a5640 1 --2- 172.21.15.84:0/1256144678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9178102b60 0x7f91781980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:56.648 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.647+0000 7f91802a5640 1 --2- 172.21.15.84:0/1256144678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91780ffb90 0x7f9178197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:56.648 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.647+0000 7f91802a5640 1 -- 172.21.15.84:0/1256144678 >> 172.21.15.84:0/1256144678 conn(0x7f91780fb4a0 msgr2=0x7f9178105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:56.648 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.647+0000 7f91802a5640 1 -- 172.21.15.84:0/1256144678 shutdown_connections 2024-05-22T22:59:56.648 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.647+0000 7f91802a5640 1 -- 172.21.15.84:0/1256144678 wait complete. 2024-05-22T22:59:56.658 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-22T22:59:56.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.764+0000 7fc4ee888640 1 -- 172.21.15.84:0/1144677639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4e8102330 msgr2=0x7fc4e8104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:56.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.764+0000 7fc4ee888640 1 --2- 172.21.15.84:0/1144677639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4e8102330 0x7fc4e8104720 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009900 tx=0x7fc4d802a410 comp rx=0 tx=0).stop 2024-05-22T22:59:56.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.764+0000 7fc4ee888640 1 -- 172.21.15.84:0/1144677639 shutdown_connections 2024-05-22T22:59:56.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.764+0000 7fc4ee888640 1 --2- 172.21.15.84:0/1144677639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4e8107590 0x7fc4e81099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:56.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.764+0000 7fc4ee888640 1 --2- 172.21.15.84:0/1144677639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4e8104c60 0x7fc4e8107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:56.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.764+0000 7fc4ee888640 1 --2- 172.21.15.84:0/1144677639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4e8102330 0x7fc4e8104720 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:56.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.764+0000 7fc4ee888640 1 -- 172.21.15.84:0/1144677639 >> 172.21.15.84:0/1144677639 conn(0x7fc4e80fbd20 msgr2=0x7fc4e80fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:56.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.764+0000 7fc4ee888640 1 -- 172.21.15.84:0/1144677639 shutdown_connections 2024-05-22T22:59:56.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.764+0000 7fc4ee888640 1 -- 172.21.15.84:0/1144677639 wait complete. 2024-05-22T22:59:56.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.764+0000 7fc4ee888640 1 Processor -- start 2024-05-22T22:59:56.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.764+0000 7fc4ee888640 1 -- start start 2024-05-22T22:59:56.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.764+0000 7fc4ee888640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4e8102330 0x7fc4e8197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:56.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.764+0000 7fc4ee888640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4e8104c60 0x7fc4e81981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:56.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.764+0000 7fc4ee888640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4e8107590 0x7fc4e819f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:56.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.764+0000 7fc4ee888640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e810d550 con 0x7fc4e8102330 2024-05-22T22:59:56.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.764+0000 7fc4ee888640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e810d3d0 con 0x7fc4e8107590 2024-05-22T22:59:56.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.764+0000 7fc4ee888640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e810d6d0 con 0x7fc4e8104c60 2024-05-22T22:59:56.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.765+0000 7fc4e7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4e8102330 0x7fc4e8197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:56.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.765+0000 7fc4e7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4e8102330 0x7fc4e8197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41882/0 (socket says 172.21.15.84:41882) 2024-05-22T22:59:56.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.765+0000 7fc4e7fff640 1 -- 172.21.15.84:0/3711160281 learned_addr learned my addr 172.21.15.84:0/3711160281 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:56.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.765+0000 7fc4ece25640 1 --2- 172.21.15.84:0/3711160281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4e8107590 0x7fc4e819f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:56.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.765+0000 7fc4dffff640 1 --2- 172.21.15.84:0/3711160281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4e8104c60 0x7fc4e81981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:56.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.765+0000 7fc4e7fff640 1 -- 172.21.15.84:0/3711160281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4e8104c60 msgr2=0x7fc4e81981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:56.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.765+0000 7fc4e7fff640 1 --2- 172.21.15.84:0/3711160281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4e8104c60 0x7fc4e81981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:56.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.765+0000 7fc4e7fff640 1 -- 172.21.15.84:0/3711160281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4e8107590 msgr2=0x7fc4e819f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:56.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.765+0000 7fc4e7fff640 1 --2- 172.21.15.84:0/3711160281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4e8107590 0x7fc4e819f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:56.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.765+0000 7fc4e7fff640 1 -- 172.21.15.84:0/3711160281 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e819f880 con 0x7fc4e8102330 2024-05-22T22:59:56.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.765+0000 7fc4ece25640 1 --2- 172.21.15.84:0/3711160281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4e8107590 0x7fc4e819f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T22:59:56.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.765+0000 7fc4e7fff640 1 --2- 172.21.15.84:0/3711160281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4e8102330 0x7fc4e8197c70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009230 tx=0x7fc4d8031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:56.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.765+0000 7fc4e5ffb640 1 -- 172.21.15.84:0/3711160281 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d803f030 con 0x7fc4e8102330 2024-05-22T22:59:56.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.765+0000 7fc4e5ffb640 1 -- 172.21.15.84:0/3711160281 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4d8003ad0 con 0x7fc4e8102330 2024-05-22T22:59:56.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.765+0000 7fc4ee888640 1 -- 172.21.15.84:0/3711160281 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e819fb10 con 0x7fc4e8102330 2024-05-22T22:59:56.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.765+0000 7fc4e5ffb640 1 -- 172.21.15.84:0/3711160281 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8035070 con 0x7fc4e8102330 2024-05-22T22:59:56.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.765+0000 7fc4ee888640 1 -- 172.21.15.84:0/3711160281 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e81a0000 con 0x7fc4e8102330 2024-05-22T22:59:56.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.766+0000 7fc4ee888640 1 -- 172.21.15.84:0/3711160281 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e80fd810 con 0x7fc4e8102330 2024-05-22T22:59:56.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.767+0000 7fc4e5ffb640 1 -- 172.21.15.84:0/3711160281 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc4d8042030 con 0x7fc4e8102330 2024-05-22T22:59:56.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.768+0000 7fc4e5ffb640 1 --2- 172.21.15.84:0/3711160281 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc4d40a2b90 0x7fc4d40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:56.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.768+0000 7fc4dffff640 1 --2- 172.21.15.84:0/3711160281 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc4d40a2b90 0x7fc4d40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:56.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.768+0000 7fc4e5ffb640 1 -- 172.21.15.84:0/3711160281 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc4d80ef510 con 0x7fc4e8102330 2024-05-22T22:59:56.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.768+0000 7fc4dffff640 1 --2- 172.21.15.84:0/3711160281 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc4d40a2b90 0x7fc4d40a5050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8069560 tx=0x7fc4d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:56.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.773+0000 7fc4e5ffb640 1 -- 172.21.15.84:0/3711160281 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc4d80bc5c0 con 0x7fc4e8102330 2024-05-22T22:59:56.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:56 smithi084 ceph-mon[32186]: pgmap v291: 97 pgs: 14 creating+peering, 77 active+clean, 6 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 207 B/s rd, 415 B/s wr, 5 op/s 2024-05-22T22:59:56.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:56 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1646051490' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-22T22:59:56.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:56 smithi084 ceph-mon[32186]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-05-22T22:59:56.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:56 smithi204 ceph-mon[36211]: pgmap v291: 97 pgs: 14 creating+peering, 77 active+clean, 6 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 207 B/s rd, 415 B/s wr, 5 op/s 2024-05-22T22:59:56.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:56 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1646051490' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-22T22:59:56.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:56 smithi204 ceph-mon[36211]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-05-22T22:59:56.978 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:56.978+0000 7fc4ee888640 1 -- 172.21.15.84:0/3711160281 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc4e8061d40 con 0x7fc4d40a2b90 2024-05-22T22:59:57.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.001+0000 7fc4e5ffb640 1 -- 172.21.15.84:0/3711160281 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc4e8061d40 con 0x7fc4d40a2b90 2024-05-22T22:59:57.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.002+0000 7fc4ee888640 1 -- 172.21.15.84:0/3711160281 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc4d40a2b90 msgr2=0x7fc4d40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:57.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.002+0000 7fc4ee888640 1 --2- 172.21.15.84:0/3711160281 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc4d40a2b90 0x7fc4d40a5050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8069560 tx=0x7fc4d0005f90 comp rx=0 tx=0).stop 2024-05-22T22:59:57.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.002+0000 7fc4ee888640 1 -- 172.21.15.84:0/3711160281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4e8102330 msgr2=0x7fc4e8197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:57.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.002+0000 7fc4ee888640 1 --2- 172.21.15.84:0/3711160281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4e8102330 0x7fc4e8197c70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009230 tx=0x7fc4d8031b60 comp rx=0 tx=0).stop 2024-05-22T22:59:57.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.003+0000 7fc4ee888640 1 -- 172.21.15.84:0/3711160281 shutdown_connections 2024-05-22T22:59:57.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.003+0000 7fc4ee888640 1 --2- 172.21.15.84:0/3711160281 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc4d40a2b90 0x7fc4d40a5050 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.003+0000 7fc4ee888640 1 --2- 172.21.15.84:0/3711160281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4e8107590 0x7fc4e819f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.003+0000 7fc4ee888640 1 --2- 172.21.15.84:0/3711160281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4e8104c60 0x7fc4e81981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.003+0000 7fc4ee888640 1 --2- 172.21.15.84:0/3711160281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4e8102330 0x7fc4e8197c70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.003+0000 7fc4ee888640 1 -- 172.21.15.84:0/3711160281 >> 172.21.15.84:0/3711160281 conn(0x7fc4e80fbd20 msgr2=0x7fc4e8102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:57.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.003+0000 7fc4ee888640 1 -- 172.21.15.84:0/3711160281 shutdown_connections 2024-05-22T22:59:57.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.003+0000 7fc4ee888640 1 -- 172.21.15.84:0/3711160281 wait complete. 2024-05-22T22:59:57.014 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-22T22:59:57.048 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 22 22:59:56 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-b[66854]: 2024-05-22T22:59:56.555+0000 7faba6381700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-22T22:59:57.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:56 smithi096 ceph-mon[36392]: pgmap v291: 97 pgs: 14 creating+peering, 77 active+clean, 6 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 207 B/s rd, 415 B/s wr, 5 op/s 2024-05-22T22:59:57.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:56 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1646051490' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-22T22:59:57.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:56 smithi096 ceph-mon[36392]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-05-22T22:59:57.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.118+0000 7f26e33b7640 1 -- 172.21.15.84:0/119594460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26dc101440 msgr2=0x7f26dc108ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:57.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.118+0000 7f26e33b7640 1 --2- 172.21.15.84:0/119594460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26dc101440 0x7f26dc108ce0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f26d8009900 tx=0x7f26d802a410 comp rx=0 tx=0).stop 2024-05-22T22:59:57.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.118+0000 7f26e33b7640 1 -- 172.21.15.84:0/119594460 shutdown_connections 2024-05-22T22:59:57.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.118+0000 7f26e33b7640 1 --2- 172.21.15.84:0/119594460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26dc101440 0x7f26dc108ce0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.118+0000 7f26e33b7640 1 --2- 172.21.15.84:0/119594460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26dc100a80 0x7f26dc100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.118+0000 7f26e33b7640 1 --2- 172.21.15.84:0/119594460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26dc101d50 0x7f26dc102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.118+0000 7f26e33b7640 1 -- 172.21.15.84:0/119594460 >> 172.21.15.84:0/119594460 conn(0x7f26dc0fb4b0 msgr2=0x7f26dc0fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:57.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.118+0000 7f26e33b7640 1 -- 172.21.15.84:0/119594460 shutdown_connections 2024-05-22T22:59:57.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.118+0000 7f26e33b7640 1 -- 172.21.15.84:0/119594460 wait complete. 2024-05-22T22:59:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.119+0000 7f26e33b7640 1 Processor -- start 2024-05-22T22:59:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.119+0000 7f26e33b7640 1 -- start start 2024-05-22T22:59:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.119+0000 7f26e33b7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26dc100a80 0x7f26dc197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.119+0000 7f26e33b7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26dc101440 0x7f26dc1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.119+0000 7f26e33b7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26dc101d50 0x7f26dc19f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.119+0000 7f26e33b7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc10d530 con 0x7f26dc100a80 2024-05-22T22:59:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.119+0000 7f26e33b7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc10d3b0 con 0x7f26dc101d50 2024-05-22T22:59:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.119+0000 7f26e33b7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc10d6b0 con 0x7f26dc101440 2024-05-22T22:59:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.119+0000 7f26e1153640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26dc100a80 0x7f26dc197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.119+0000 7f26e1153640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26dc100a80 0x7f26dc197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41900/0 (socket says 172.21.15.84:41900) 2024-05-22T22:59:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.119+0000 7f26e1153640 1 -- 172.21.15.84:0/1938246819 learned_addr learned my addr 172.21.15.84:0/1938246819 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.119+0000 7f26e1954640 1 --2- 172.21.15.84:0/1938246819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26dc101d50 0x7f26dc19f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.119+0000 7f26e1153640 1 -- 172.21.15.84:0/1938246819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26dc101440 msgr2=0x7f26dc1980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.119+0000 7f26e0952640 1 --2- 172.21.15.84:0/1938246819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26dc101440 0x7f26dc1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.119+0000 7f26e1153640 1 --2- 172.21.15.84:0/1938246819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26dc101440 0x7f26dc1980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.119+0000 7f26e1153640 1 -- 172.21.15.84:0/1938246819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26dc101d50 msgr2=0x7f26dc19f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.119+0000 7f26e1153640 1 --2- 172.21.15.84:0/1938246819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26dc101d50 0x7f26dc19f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.119+0000 7f26e1153640 1 -- 172.21.15.84:0/1938246819 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26dc19f7d0 con 0x7f26dc100a80 2024-05-22T22:59:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.120+0000 7f26e1153640 1 --2- 172.21.15.84:0/1938246819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26dc100a80 0x7f26dc197bb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009fd0 tx=0x7f26d400c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.120+0000 7f26d27fc640 1 -- 172.21.15.84:0/1938246819 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d4011470 con 0x7f26dc100a80 2024-05-22T22:59:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.120+0000 7f26d27fc640 1 -- 172.21.15.84:0/1938246819 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d4011af0 con 0x7f26dc100a80 2024-05-22T22:59:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.120+0000 7f26e33b7640 1 -- 172.21.15.84:0/1938246819 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26dc19fac0 con 0x7f26dc100a80 2024-05-22T22:59:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.120+0000 7f26d27fc640 1 -- 172.21.15.84:0/1938246819 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d401ab90 con 0x7f26dc100a80 2024-05-22T22:59:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.120+0000 7f26e33b7640 1 -- 172.21.15.84:0/1938246819 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26dc1a0010 con 0x7f26dc100a80 2024-05-22T22:59:57.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.121+0000 7f26e33b7640 1 -- 172.21.15.84:0/1938246819 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26dc107bc0 con 0x7f26dc100a80 2024-05-22T22:59:57.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.122+0000 7f26d27fc640 1 -- 172.21.15.84:0/1938246819 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f26d4011610 con 0x7f26dc100a80 2024-05-22T22:59:57.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.123+0000 7f26d27fc640 1 --2- 172.21.15.84:0/1938246819 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f26c80a2b90 0x7f26c80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:57.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.123+0000 7f26e0952640 1 --2- 172.21.15.84:0/1938246819 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f26c80a2b90 0x7f26c80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:57.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.123+0000 7f26d27fc640 1 -- 172.21.15.84:0/1938246819 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f26d40cf4c0 con 0x7f26dc100a80 2024-05-22T22:59:57.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.123+0000 7f26e0952640 1 --2- 172.21.15.84:0/1938246819 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f26c80a2b90 0x7f26c80a5050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f26dc1990c0 tx=0x7f26cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:57.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.127+0000 7f26d27fc640 1 -- 172.21.15.84:0/1938246819 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f26d4016030 con 0x7f26dc100a80 2024-05-22T22:59:57.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.329+0000 7f26e33b7640 1 -- 172.21.15.84:0/1938246819 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f26dc061cb0 con 0x7f26c80a2b90 2024-05-22T22:59:57.334 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.334+0000 7f26d27fc640 1 -- 172.21.15.84:0/1938246819 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f26dc061cb0 con 0x7f26c80a2b90 2024-05-22T22:59:57.334 INFO:teuthology.orchestra.run.smithi084.stdout:/volumes/qa/sv_1/3ffdedc9-b28a-44c9-9c20-ef01dee53af2 2024-05-22T22:59:57.337 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.336+0000 7f26e33b7640 1 -- 172.21.15.84:0/1938246819 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f26c80a2b90 msgr2=0x7f26c80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:57.337 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.336+0000 7f26e33b7640 1 --2- 172.21.15.84:0/1938246819 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f26c80a2b90 0x7f26c80a5050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f26dc1990c0 tx=0x7f26cc005f90 comp rx=0 tx=0).stop 2024-05-22T22:59:57.337 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.336+0000 7f26e33b7640 1 -- 172.21.15.84:0/1938246819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26dc100a80 msgr2=0x7f26dc197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:57.337 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.336+0000 7f26e33b7640 1 --2- 172.21.15.84:0/1938246819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26dc100a80 0x7f26dc197bb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009fd0 tx=0x7f26d400c270 comp rx=0 tx=0).stop 2024-05-22T22:59:57.337 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.336+0000 7f26e33b7640 1 -- 172.21.15.84:0/1938246819 shutdown_connections 2024-05-22T22:59:57.337 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.336+0000 7f26e33b7640 1 --2- 172.21.15.84:0/1938246819 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f26c80a2b90 0x7f26c80a5050 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.337 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.336+0000 7f26e33b7640 1 --2- 172.21.15.84:0/1938246819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26dc101d50 0x7f26dc19f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.337 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.336+0000 7f26e33b7640 1 --2- 172.21.15.84:0/1938246819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26dc101440 0x7f26dc1980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.337 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.336+0000 7f26e33b7640 1 --2- 172.21.15.84:0/1938246819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26dc100a80 0x7f26dc197bb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.337 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.336+0000 7f26e33b7640 1 -- 172.21.15.84:0/1938246819 >> 172.21.15.84:0/1938246819 conn(0x7f26dc0fb4b0 msgr2=0x7f26dc104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:57.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.336+0000 7f26e33b7640 1 -- 172.21.15.84:0/1938246819 shutdown_connections 2024-05-22T22:59:57.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.336+0000 7f26e33b7640 1 -- 172.21.15.84:0/1938246819 wait complete. 2024-05-22T22:59:57.348 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T22:59:57.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.448+0000 7fb331370640 1 -- 172.21.15.84:0/4206799788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb32c103490 msgr2=0x7fb32c109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:57.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.448+0000 7fb331370640 1 --2- 172.21.15.84:0/4206799788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb32c103490 0x7fb32c109d20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb324009ca0 tx=0x7fb32402a8c0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.448+0000 7fb331370640 1 -- 172.21.15.84:0/4206799788 shutdown_connections 2024-05-22T22:59:57.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.448+0000 7fb331370640 1 --2- 172.21.15.84:0/4206799788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb32c103490 0x7fb32c109d20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.448+0000 7fb331370640 1 --2- 172.21.15.84:0/4206799788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb32c102ad0 0x7fb32c102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.448+0000 7fb331370640 1 --2- 172.21.15.84:0/4206799788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb32c1018e0 0x7fb32c101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.448+0000 7fb331370640 1 -- 172.21.15.84:0/4206799788 >> 172.21.15.84:0/4206799788 conn(0x7fb32c0fce50 msgr2=0x7fb32c0ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:57.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.448+0000 7fb331370640 1 -- 172.21.15.84:0/4206799788 shutdown_connections 2024-05-22T22:59:57.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.448+0000 7fb331370640 1 -- 172.21.15.84:0/4206799788 wait complete. 2024-05-22T22:59:57.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.449+0000 7fb331370640 1 Processor -- start 2024-05-22T22:59:57.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.449+0000 7fb331370640 1 -- start start 2024-05-22T22:59:57.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.449+0000 7fb331370640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb32c1018e0 0x7fb32c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:57.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.449+0000 7fb331370640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb32c102ad0 0x7fb32c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:57.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.449+0000 7fb331370640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb32c103490 0x7fb32c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:57.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.449+0000 7fb331370640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb32c10d540 con 0x7fb32c1018e0 2024-05-22T22:59:57.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.449+0000 7fb331370640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb32c10d3c0 con 0x7fb32c103490 2024-05-22T22:59:57.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.449+0000 7fb331370640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb32c10d6c0 con 0x7fb32c102ad0 2024-05-22T22:59:57.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.449+0000 7fb32affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb32c1018e0 0x7fb32c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:57.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.449+0000 7fb32affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb32c1018e0 0x7fb32c197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41914/0 (socket says 172.21.15.84:41914) 2024-05-22T22:59:57.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.449+0000 7fb32affd640 1 -- 172.21.15.84:0/2002528979 learned_addr learned my addr 172.21.15.84:0/2002528979 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:57.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.449+0000 7fb32b7fe640 1 --2- 172.21.15.84:0/2002528979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb32c103490 0x7fb32c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:57.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.449+0000 7fb32affd640 1 -- 172.21.15.84:0/2002528979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb32c102ad0 msgr2=0x7fb32c198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:57.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.449+0000 7fb32affd640 1 --2- 172.21.15.84:0/2002528979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb32c102ad0 0x7fb32c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.449+0000 7fb32affd640 1 -- 172.21.15.84:0/2002528979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb32c103490 msgr2=0x7fb32c19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:57.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.449+0000 7fb32affd640 1 --2- 172.21.15.84:0/2002528979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb32c103490 0x7fb32c19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.449+0000 7fb32affd640 1 -- 172.21.15.84:0/2002528979 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb32c19f710 con 0x7fb32c1018e0 2024-05-22T22:59:57.453 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.449+0000 7fb32affd640 1 --2- 172.21.15.84:0/2002528979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb32c1018e0 0x7fb32c197af0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb31c00d770 tx=0x7fb31c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:57.453 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.450+0000 7fb313fff640 1 -- 172.21.15.84:0/2002528979 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c017070 con 0x7fb32c1018e0 2024-05-22T22:59:57.453 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.450+0000 7fb313fff640 1 -- 172.21.15.84:0/2002528979 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb31c009c30 con 0x7fb32c1018e0 2024-05-22T22:59:57.453 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.450+0000 7fb313fff640 1 -- 172.21.15.84:0/2002528979 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c003dd0 con 0x7fb32c1018e0 2024-05-22T22:59:57.453 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.450+0000 7fb331370640 1 -- 172.21.15.84:0/2002528979 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb32c19fa00 con 0x7fb32c1018e0 2024-05-22T22:59:57.453 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.450+0000 7fb331370640 1 -- 172.21.15.84:0/2002528979 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb32c19ff00 con 0x7fb32c1018e0 2024-05-22T22:59:57.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.451+0000 7fb331370640 1 -- 172.21.15.84:0/2002528979 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb32c108ba0 con 0x7fb32c1018e0 2024-05-22T22:59:57.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.452+0000 7fb313fff640 1 -- 172.21.15.84:0/2002528979 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb31c003fb0 con 0x7fb32c1018e0 2024-05-22T22:59:57.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.453+0000 7fb313fff640 1 --2- 172.21.15.84:0/2002528979 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb3140a2b90 0x7fb3140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:57.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.453+0000 7fb32a7fc640 1 --2- 172.21.15.84:0/2002528979 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb3140a2b90 0x7fb3140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:57.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.453+0000 7fb313fff640 1 -- 172.21.15.84:0/2002528979 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb31c0ce930 con 0x7fb32c1018e0 2024-05-22T22:59:57.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.453+0000 7fb32a7fc640 1 --2- 172.21.15.84:0/2002528979 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb3140a2b90 0x7fb3140a5050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb32c199000 tx=0x7fb320005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:57.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.457+0000 7fb313fff640 1 -- 172.21.15.84:0/2002528979 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb31c09b9e0 con 0x7fb32c1018e0 2024-05-22T22:59:57.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.614+0000 7fb313fff640 1 -- 172.21.15.84:0/2002528979 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb31c097950 con 0x7fb32c1018e0 2024-05-22T22:59:57.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.677+0000 7fb331370640 1 -- 172.21.15.84:0/2002528979 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb32c02d200 con 0x7fb32c1018e0 2024-05-22T22:59:57.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.678+0000 7fb313fff640 1 -- 172.21.15.84:0/2002528979 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7fb31c009de0 con 0x7fb32c1018e0 2024-05-22T22:59:57.679 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:57.679 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T22:59:57.679 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T22:59:57.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.680+0000 7fb331370640 1 -- 172.21.15.84:0/2002528979 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb3140a2b90 msgr2=0x7fb3140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:57.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.680+0000 7fb331370640 1 --2- 172.21.15.84:0/2002528979 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb3140a2b90 0x7fb3140a5050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb32c199000 tx=0x7fb320005c70 comp rx=0 tx=0).stop 2024-05-22T22:59:57.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.680+0000 7fb331370640 1 -- 172.21.15.84:0/2002528979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb32c1018e0 msgr2=0x7fb32c197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:57.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.680+0000 7fb331370640 1 --2- 172.21.15.84:0/2002528979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb32c1018e0 0x7fb32c197af0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb31c00d770 tx=0x7fb31c00dc30 comp rx=0 tx=0).stop 2024-05-22T22:59:57.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.680+0000 7fb331370640 1 -- 172.21.15.84:0/2002528979 shutdown_connections 2024-05-22T22:59:57.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.680+0000 7fb331370640 1 --2- 172.21.15.84:0/2002528979 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb3140a2b90 0x7fb3140a5050 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.680+0000 7fb331370640 1 --2- 172.21.15.84:0/2002528979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb32c103490 0x7fb32c19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.680+0000 7fb331370640 1 --2- 172.21.15.84:0/2002528979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb32c102ad0 0x7fb32c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.680+0000 7fb331370640 1 --2- 172.21.15.84:0/2002528979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb32c1018e0 0x7fb32c197af0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.680+0000 7fb331370640 1 -- 172.21.15.84:0/2002528979 >> 172.21.15.84:0/2002528979 conn(0x7fb32c0fce50 msgr2=0x7fb32c105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:57.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.680+0000 7fb331370640 1 -- 172.21.15.84:0/2002528979 shutdown_connections 2024-05-22T22:59:57.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.680+0000 7fb331370640 1 -- 172.21.15.84:0/2002528979 wait complete. 2024-05-22T22:59:57.692 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T22:59:57.794 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.791+0000 7fb18ad3b640 1 -- 172.21.15.84:0/1860025207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb184101d60 msgr2=0x7fb184102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:57.794 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.791+0000 7fb18ad3b640 1 --2- 172.21.15.84:0/1860025207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb184101d60 0x7fb184102180 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb174009900 tx=0x7fb17402a410 comp rx=0 tx=0).stop 2024-05-22T22:59:57.794 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.791+0000 7fb18ad3b640 1 -- 172.21.15.84:0/1860025207 shutdown_connections 2024-05-22T22:59:57.794 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.791+0000 7fb18ad3b640 1 --2- 172.21.15.84:0/1860025207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb184101450 0x7fb184108cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.794 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.791+0000 7fb18ad3b640 1 --2- 172.21.15.84:0/1860025207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb184100a90 0x7fb184100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.794 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.791+0000 7fb18ad3b640 1 --2- 172.21.15.84:0/1860025207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb184101d60 0x7fb184102180 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.794 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.791+0000 7fb18ad3b640 1 -- 172.21.15.84:0/1860025207 >> 172.21.15.84:0/1860025207 conn(0x7fb1840fb4e0 msgr2=0x7fb1840fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:57.795 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.791+0000 7fb18ad3b640 1 -- 172.21.15.84:0/1860025207 shutdown_connections 2024-05-22T22:59:57.795 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.791+0000 7fb18ad3b640 1 -- 172.21.15.84:0/1860025207 wait complete. 2024-05-22T22:59:57.795 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.792+0000 7fb18ad3b640 1 Processor -- start 2024-05-22T22:59:57.795 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.792+0000 7fb18ad3b640 1 -- start start 2024-05-22T22:59:57.795 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.792+0000 7fb18ad3b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb184100a90 0x7fb184100550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:57.795 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.792+0000 7fb18ad3b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb184101450 0x7fb18419c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:57.795 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.792+0000 7fb18ad3b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb184101d60 0x7fb18419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:57.795 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.792+0000 7fb18ad3b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb18410c450 con 0x7fb184101d60 2024-05-22T22:59:57.795 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.792+0000 7fb18ad3b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb18410c2d0 con 0x7fb184101450 2024-05-22T22:59:57.795 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.792+0000 7fb18ad3b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb18410c5d0 con 0x7fb184100a90 2024-05-22T22:59:57.795 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.792+0000 7fb1892d8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb184101d60 0x7fb18419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:57.795 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.792+0000 7fb1892d8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb184101d60 0x7fb18419ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41934/0 (socket says 172.21.15.84:41934) 2024-05-22T22:59:57.795 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.792+0000 7fb1892d8640 1 -- 172.21.15.84:0/1056644724 learned_addr learned my addr 172.21.15.84:0/1056644724 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:57.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.792+0000 7fb183fff640 1 --2- 172.21.15.84:0/1056644724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb184101450 0x7fb18419c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:57.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.792+0000 7fb188ad7640 1 --2- 172.21.15.84:0/1056644724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb184100a90 0x7fb184100550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:57.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.792+0000 7fb1892d8640 1 -- 172.21.15.84:0/1056644724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb184100a90 msgr2=0x7fb184100550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:57.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.792+0000 7fb1892d8640 1 --2- 172.21.15.84:0/1056644724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb184100a90 0x7fb184100550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.793+0000 7fb1892d8640 1 -- 172.21.15.84:0/1056644724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb184101450 msgr2=0x7fb18419c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:57.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.793+0000 7fb1892d8640 1 --2- 172.21.15.84:0/1056644724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb184101450 0x7fb18419c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.793+0000 7fb1892d8640 1 -- 172.21.15.84:0/1056644724 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb18419f7e0 con 0x7fb184101d60 2024-05-22T22:59:57.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.793+0000 7fb1892d8640 1 --2- 172.21.15.84:0/1056644724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb184101d60 0x7fb18419ef20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb17c00eaa0 tx=0x7fb17c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:57.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.793+0000 7fb181ffb640 1 -- 172.21.15.84:0/1056644724 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c00cdb0 con 0x7fb184101d60 2024-05-22T22:59:57.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.793+0000 7fb181ffb640 1 -- 172.21.15.84:0/1056644724 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb17c005020 con 0x7fb184101d60 2024-05-22T22:59:57.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.793+0000 7fb18ad3b640 1 -- 172.21.15.84:0/1056644724 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb18419fad0 con 0x7fb184101d60 2024-05-22T22:59:57.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.793+0000 7fb181ffb640 1 -- 172.21.15.84:0/1056644724 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c0116b0 con 0x7fb184101d60 2024-05-22T22:59:57.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.793+0000 7fb18ad3b640 1 -- 172.21.15.84:0/1056644724 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1841a0020 con 0x7fb184101d60 2024-05-22T22:59:57.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.795+0000 7fb18ad3b640 1 -- 172.21.15.84:0/1056644724 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb184107c20 con 0x7fb184101d60 2024-05-22T22:59:57.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.795+0000 7fb181ffb640 1 -- 172.21.15.84:0/1056644724 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb17c005560 con 0x7fb184101d60 2024-05-22T22:59:57.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.796+0000 7fb181ffb640 1 --2- 172.21.15.84:0/1056644724 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb16c0a2c20 0x7fb16c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:57.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.796+0000 7fb188ad7640 1 --2- 172.21.15.84:0/1056644724 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb16c0a2c20 0x7fb16c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:57.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.796+0000 7fb181ffb640 1 -- 172.21.15.84:0/1056644724 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb17c0cf270 con 0x7fb184101d60 2024-05-22T22:59:57.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.796+0000 7fb188ad7640 1 --2- 172.21.15.84:0/1056644724 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb16c0a2c20 0x7fb16c0a50e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb174006010 tx=0x7fb174000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:57.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.800+0000 7fb181ffb640 1 -- 172.21.15.84:0/1056644724 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb17c016030 con 0x7fb184101d60 2024-05-22T22:59:57.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:57 smithi084 ceph-mon[32186]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:59:57.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:57 smithi084 ceph-mon[32186]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:59:57.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:57 smithi204 ceph-mon[36211]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:59:57.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:57 smithi204 ceph-mon[36211]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:59:57.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.979+0000 7fb18ad3b640 1 -- 172.21.15.84:0/1056644724 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb18402d3b0 con 0x7fb184101d60 2024-05-22T22:59:57.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.981+0000 7fb181ffb640 1 -- 172.21.15.84:0/1056644724 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22194 (secure 0 0 0) 0x7fb17c09c220 con 0x7fb184101d60 2024-05-22T22:59:57.980 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:57.981 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":86,"fsid":"c1135a12-188d-11ef-bc9a-c7b262605968","created":"2024-05-22T22:51:49.878218+0000","modified":"2024-05-22T22:59:52.940885+0000","last_up_change":"2024-05-22T22:58:58.550278+0000","last_in_change":"2024-05-22T22:58:37.322139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T22:55:07.719394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T22:59:50.150511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T22:59:51.111182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762df065-fb4f-4c71-aab8-a62d2282c253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6803","nonce":1559945826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6805","nonce":1559945826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6809","nonce":1559945826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6807","nonce":1559945826}]},"public_addr":"172.21.15.84:6803/1559945826","cluster_addr":"172.21.15.84:6805/1559945826","heartbeat_back_addr":"172.21.15.84:6809/1559945826","heartbeat_front_addr":"172.21.15.84:6807/1559945826","state":["exists","up"]},{"osd":1,"uuid":"3fb2385d-6a34-49e3-88e1-116237beb346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6801","nonce":2996505353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6803","nonce":2996505353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6807","nonce":2996505353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6805","nonce":2996505353}]},"public_addr":"172.21.15.96:6801/2996505353","cluster_addr":"172.21.15.96:6803/2996505353","heartbeat_back_addr":"172.21.15.96:6807/2996505353","heartbeat_front_addr":"172.21.15.96:6805/2996505353","state":["exists","up"]},{"osd":2,"uuid":"e306e88e-a308-4585-ae11-c8af10b84038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6801","nonce":1499577897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6803","nonce":1499577897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6807","nonce":1499577897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6805","nonce":1499577897}]},"public_addr":"172.21.15.204:6801/1499577897","cluster_addr":"172.21.15.204:6803/1499577897","heartbeat_back_addr":"172.21.15.204:6807/1499577897","heartbeat_front_addr":"172.21.15.204:6805/1499577897","state":["exists","up"]},{"osd":3,"uuid":"d2a5fb97-6f60-4d45-9f18-447cae480974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6811","nonce":2869340648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6813","nonce":2869340648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6817","nonce":2869340648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6815","nonce":2869340648}]},"public_addr":"172.21.15.84:6811/2869340648","cluster_addr":"172.21.15.84:6813/2869340648","heartbeat_back_addr":"172.21.15.84:6817/2869340648","heartbeat_front_addr":"172.21.15.84:6815/2869340648","state":["exists","up"]},{"osd":4,"uuid":"4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6809","nonce":2554265648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6811","nonce":2554265648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6815","nonce":2554265648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6813","nonce":2554265648}]},"public_addr":"172.21.15.96:6809/2554265648","cluster_addr":"172.21.15.96:6811/2554265648","heartbeat_back_addr":"172.21.15.96:6815/2554265648","heartbeat_front_addr":"172.21.15.96:6813/2554265648","state":["exists","up"]},{"osd":5,"uuid":"7996bf5e-93a5-41b0-a9b1-99e7ef4514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6809","nonce":4075476986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6811","nonce":4075476986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6815","nonce":4075476986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6813","nonce":4075476986}]},"public_addr":"172.21.15.204:6809/4075476986","cluster_addr":"172.21.15.204:6811/4075476986","heartbeat_back_addr":"172.21.15.204:6815/4075476986","heartbeat_front_addr":"172.21.15.204:6813/4075476986","state":["exists","up"]},{"osd":6,"uuid":"f4b7c5de-1836-4d16-8603-3c517524d9fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6819","nonce":1806202865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6821","nonce":1806202865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6825","nonce":1806202865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6823","nonce":1806202865}]},"public_addr":"172.21.15.84:6819/1806202865","cluster_addr":"172.21.15.84:6821/1806202865","heartbeat_back_addr":"172.21.15.84:6825/1806202865","heartbeat_front_addr":"172.21.15.84:6823/1806202865","state":["exists","up"]},{"osd":7,"uuid":"69a18c43-5447-480d-9466-190830070c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6817","nonce":3195792774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6819","nonce":3195792774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6823","nonce":3195792774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6821","nonce":3195792774}]},"public_addr":"172.21.15.96:6817/3195792774","cluster_addr":"172.21.15.96:6819/3195792774","heartbeat_back_addr":"172.21.15.96:6823/3195792774","heartbeat_front_addr":"172.21.15.96:6821/3195792774","state":["exists","up"]},{"osd":8,"uuid":"2c5cadca-689a-441e-847c-ba370d400ef1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6817","nonce":606997927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6819","nonce":606997927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6823","nonce":606997927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6821","nonce":606997927}]},"public_addr":"172.21.15.204:6817/606997927","cluster_addr":"172.21.15.204:6819/606997927","heartbeat_back_addr":"172.21.15.204:6823/606997927","heartbeat_front_addr":"172.21.15.204:6821/606997927","state":["exists","up"]},{"osd":9,"uuid":"e3984fb2-7c00-4275-a10e-bca7c859f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6827","nonce":2786189012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6829","nonce":2786189012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6833","nonce":2786189012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6831","nonce":2786189012}]},"public_addr":"172.21.15.84:6827/2786189012","cluster_addr":"172.21.15.84:6829/2786189012","heartbeat_back_addr":"172.21.15.84:6833/2786189012","heartbeat_front_addr":"172.21.15.84:6831/2786189012","state":["exists","up"]},{"osd":10,"uuid":"b4938160-a808-4ded-91c2-d245c99fff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6825","nonce":1985398526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6827","nonce":1985398526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6831","nonce":1985398526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6829","nonce":1985398526}]},"public_addr":"172.21.15.96:6825/1985398526","cluster_addr":"172.21.15.96:6827/1985398526","heartbeat_back_addr":"172.21.15.96:6831/1985398526","heartbeat_front_addr":"172.21.15.96:6829/1985398526","state":["exists","up"]},{"osd":11,"uuid":"238922ef-d046-40cc-833d-25b7e7a523cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6825","nonce":3106116519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6827","nonce":3106116519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6831","nonce":3106116519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6829","nonce":3106116519}]},"public_addr":"172.21.15.204:6825/3106116519","cluster_addr":"172.21.15.204:6827/3106116519","heartbeat_back_addr":"172.21.15.204:6831/3106116519","heartbeat_front_addr":"172.21.15.204:6829/3106116519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:16.443480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:39.898133+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:03.664544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:28.838687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:53.829733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:19.269307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:45.166910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:10.771614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:36.678516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:03.150592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:29.327769+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:56.315938+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6801/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:6800/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/2766288608":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/950455999":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/4000289608":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1650467363":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1052746114":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6800/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6801/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/14610813":"2024-05-23T22:52:45.395791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T22:59:57.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.983+0000 7fb18ad3b640 1 -- 172.21.15.84:0/1056644724 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb16c0a2c20 msgr2=0x7fb16c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:57.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.983+0000 7fb18ad3b640 1 --2- 172.21.15.84:0/1056644724 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb16c0a2c20 0x7fb16c0a50e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb174006010 tx=0x7fb174000bc0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.983+0000 7fb18ad3b640 1 -- 172.21.15.84:0/1056644724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb184101d60 msgr2=0x7fb18419ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:57.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.983+0000 7fb18ad3b640 1 --2- 172.21.15.84:0/1056644724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb184101d60 0x7fb18419ef20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb17c00eaa0 tx=0x7fb17c00ef60 comp rx=0 tx=0).stop 2024-05-22T22:59:57.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.983+0000 7fb18ad3b640 1 -- 172.21.15.84:0/1056644724 shutdown_connections 2024-05-22T22:59:57.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.983+0000 7fb18ad3b640 1 --2- 172.21.15.84:0/1056644724 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb16c0a2c20 0x7fb16c0a50e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.983+0000 7fb18ad3b640 1 --2- 172.21.15.84:0/1056644724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb184101d60 0x7fb18419ef20 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.983+0000 7fb18ad3b640 1 --2- 172.21.15.84:0/1056644724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb184101450 0x7fb18419c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.983+0000 7fb18ad3b640 1 --2- 172.21.15.84:0/1056644724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb184100a90 0x7fb184100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:57.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.983+0000 7fb18ad3b640 1 -- 172.21.15.84:0/1056644724 >> 172.21.15.84:0/1056644724 conn(0x7fb1840fb4e0 msgr2=0x7fb184104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:57.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.983+0000 7fb18ad3b640 1 -- 172.21.15.84:0/1056644724 shutdown_connections 2024-05-22T22:59:57.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:57.983+0000 7fb18ad3b640 1 -- 172.21.15.84:0/1056644724 wait complete. 2024-05-22T22:59:57.994 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-22T22:59:57.995 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T22:59:58.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:57 smithi096 ceph-mon[36392]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:59:58.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:57 smithi096 ceph-mon[36392]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:59:58.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.096+0000 7fb119ba0640 1 -- 172.21.15.84:0/2621565614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb114104aa0 msgr2=0x7fb114106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:58.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.096+0000 7fb119ba0640 1 --2- 172.21.15.84:0/2621565614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb114104aa0 0x7fb114106e90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb100009900 tx=0x7fb10002a410 comp rx=0 tx=0).stop 2024-05-22T22:59:58.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.096+0000 7fb119ba0640 1 -- 172.21.15.84:0/2621565614 shutdown_connections 2024-05-22T22:59:58.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.096+0000 7fb119ba0640 1 --2- 172.21.15.84:0/2621565614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1141073d0 0x7fb114109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:58.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.096+0000 7fb119ba0640 1 --2- 172.21.15.84:0/2621565614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb114104aa0 0x7fb114106e90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:58.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.096+0000 7fb119ba0640 1 --2- 172.21.15.84:0/2621565614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb114102170 0x7fb114104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:58.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.096+0000 7fb119ba0640 1 -- 172.21.15.84:0/2621565614 >> 172.21.15.84:0/2621565614 conn(0x7fb1140fbb80 msgr2=0x7fb1140fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:58.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.096+0000 7fb119ba0640 1 -- 172.21.15.84:0/2621565614 shutdown_connections 2024-05-22T22:59:58.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.096+0000 7fb119ba0640 1 -- 172.21.15.84:0/2621565614 wait complete. 2024-05-22T22:59:58.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.097+0000 7fb119ba0640 1 Processor -- start 2024-05-22T22:59:58.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.097+0000 7fb119ba0640 1 -- start start 2024-05-22T22:59:58.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.097+0000 7fb119ba0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb114102170 0x7fb114197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:58.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.097+0000 7fb119ba0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb114104aa0 0x7fb1141980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.097+0000 7fb119ba0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1141073d0 0x7fb11419f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.097+0000 7fb119ba0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb11410d440 con 0x7fb114102170 2024-05-22T22:59:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.097+0000 7fb119ba0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb11410d2c0 con 0x7fb1141073d0 2024-05-22T22:59:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.097+0000 7fb119ba0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb11410d5c0 con 0x7fb114104aa0 2024-05-22T22:59:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.097+0000 7fb118b9e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb114102170 0x7fb114197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.097+0000 7fb118b9e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb114102170 0x7fb114197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41944/0 (socket says 172.21.15.84:41944) 2024-05-22T22:59:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.097+0000 7fb118b9e640 1 -- 172.21.15.84:0/1811221212 learned_addr learned my addr 172.21.15.84:0/1811221212 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.097+0000 7fb118b9e640 1 -- 172.21.15.84:0/1811221212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb114104aa0 msgr2=0x7fb1141980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.097+0000 7fb118b9e640 1 --2- 172.21.15.84:0/1811221212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb114104aa0 0x7fb1141980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.097+0000 7fb118b9e640 1 -- 172.21.15.84:0/1811221212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1141073d0 msgr2=0x7fb11419f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.097+0000 7fb118b9e640 1 --2- 172.21.15.84:0/1811221212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1141073d0 0x7fb11419f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.097+0000 7fb118b9e640 1 -- 172.21.15.84:0/1811221212 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11419f790 con 0x7fb114102170 2024-05-22T22:59:58.100 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.097+0000 7fb118b9e640 1 --2- 172.21.15.84:0/1811221212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb114102170 0x7fb114197b70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb10800b4a0 tx=0x7fb10800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:58.100 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.097+0000 7fb111ffb640 1 -- 172.21.15.84:0/1811221212 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb108014070 con 0x7fb114102170 2024-05-22T22:59:58.100 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.097+0000 7fb119ba0640 1 -- 172.21.15.84:0/1811221212 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11419fa80 con 0x7fb114102170 2024-05-22T22:59:58.100 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.097+0000 7fb111ffb640 1 -- 172.21.15.84:0/1811221212 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb108010af0 con 0x7fb114102170 2024-05-22T22:59:58.100 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.097+0000 7fb111ffb640 1 -- 172.21.15.84:0/1811221212 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb108017d10 con 0x7fb114102170 2024-05-22T22:59:58.100 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.098+0000 7fb119ba0640 1 -- 172.21.15.84:0/1811221212 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1141a0050 con 0x7fb114102170 2024-05-22T22:59:58.101 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.099+0000 7fb119ba0640 1 -- 172.21.15.84:0/1811221212 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1140fd670 con 0x7fb114102170 2024-05-22T22:59:58.101 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.099+0000 7fb111ffb640 1 -- 172.21.15.84:0/1811221212 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb108017070 con 0x7fb114102170 2024-05-22T22:59:58.101 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.100+0000 7fb111ffb640 1 --2- 172.21.15.84:0/1811221212 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb1040a2c20 0x7fb1040a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:58.101 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.100+0000 7fb113fff640 1 --2- 172.21.15.84:0/1811221212 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb1040a2c20 0x7fb1040a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:58.101 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.100+0000 7fb111ffb640 1 -- 172.21.15.84:0/1811221212 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb1080ce390 con 0x7fb114102170 2024-05-22T22:59:58.101 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.101+0000 7fb113fff640 1 --2- 172.21.15.84:0/1811221212 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb1040a2c20 0x7fb1040a50e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb1000096c0 tx=0x7fb100031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:58.104 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.105+0000 7fb111ffb640 1 -- 172.21.15.84:0/1811221212 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb10809b450 con 0x7fb114102170 2024-05-22T22:59:58.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.277+0000 7fb119ba0640 1 -- 172.21.15.84:0/1811221212 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb114061cb0 con 0x7fb1040a2c20 2024-05-22T22:59:58.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.285+0000 7fb111ffb640 1 -- 172.21.15.84:0/1811221212 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258892 (secure 0 0 0) 0x7fb114061cb0 con 0x7fb1040a2c20 2024-05-22T22:59:58.288 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:58.289 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2024-05-22T22:59:58.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.288+0000 7fb119ba0640 1 -- 172.21.15.84:0/1811221212 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb1040a2c20 msgr2=0x7fb1040a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:58.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.288+0000 7fb119ba0640 1 --2- 172.21.15.84:0/1811221212 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb1040a2c20 0x7fb1040a50e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb1000096c0 tx=0x7fb100031400 comp rx=0 tx=0).stop 2024-05-22T22:59:58.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.288+0000 7fb119ba0640 1 -- 172.21.15.84:0/1811221212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb114102170 msgr2=0x7fb114197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:58.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.288+0000 7fb119ba0640 1 --2- 172.21.15.84:0/1811221212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb114102170 0x7fb114197b70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb10800b4a0 tx=0x7fb10800b960 comp rx=0 tx=0).stop 2024-05-22T22:59:58.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.288+0000 7fb119ba0640 1 -- 172.21.15.84:0/1811221212 shutdown_connections 2024-05-22T22:59:58.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.288+0000 7fb119ba0640 1 --2- 172.21.15.84:0/1811221212 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb1040a2c20 0x7fb1040a50e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:58.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.288+0000 7fb119ba0640 1 --2- 172.21.15.84:0/1811221212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1141073d0 0x7fb11419f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:58.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.288+0000 7fb119ba0640 1 --2- 172.21.15.84:0/1811221212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb114104aa0 0x7fb1141980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:58.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.288+0000 7fb119ba0640 1 --2- 172.21.15.84:0/1811221212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb114102170 0x7fb114197b70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:58.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.288+0000 7fb119ba0640 1 -- 172.21.15.84:0/1811221212 >> 172.21.15.84:0/1811221212 conn(0x7fb1140fbb80 msgr2=0x7fb114102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:58.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.288+0000 7fb119ba0640 1 -- 172.21.15.84:0/1811221212 shutdown_connections 2024-05-22T22:59:58.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.288+0000 7fb119ba0640 1 -- 172.21.15.84:0/1811221212 wait complete. 2024-05-22T22:59:58.299 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-05-22T22:59:57.494429+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":629475,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":148,"num_read_kb":146,"num_write":238,"num_write_kb":2991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":188,"ondisk_log_size":188,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":161,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376892,"kb_used_data":11028,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564932,"statfs":{"total":1151940427776,"available":1151554490368,"internally_reserved":0,"allocated":11292672,"data_stored":7076876,"data_compressed":107052,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":39107,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":12,"num_read_kb":27,"num_write":57,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001356"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.636409+0000","last_change":"2024-05-22T22:59:51.951625+0000","last_active":"2024-05-22T22:59:53.636409+0000","last_peered":"2024-05-22T22:59:53.636409+0000","last_clean":"2024-05-22T22:59:53.636409+0000","last_became_active":"2024-05-22T22:59:51.951225+0000","last_became_peered":"2024-05-22T22:59:51.951225+0000","last_unstale":"2024-05-22T22:59:53.636409+0000","last_undegraded":"2024-05-22T22:59:53.636409+0000","last_fullsized":"2024-05-22T22:59:53.636409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:53:32.616083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:54.133512+0000","last_change":"2024-05-22T22:59:51.956925+0000","last_active":"2024-05-22T22:59:54.133512+0000","last_peered":"2024-05-22T22:59:54.133512+0000","last_clean":"2024-05-22T22:59:54.133512+0000","last_became_active":"2024-05-22T22:59:51.956557+0000","last_became_peered":"2024-05-22T22:59:51.956557+0000","last_unstale":"2024-05-22T22:59:54.133512+0000","last_undegraded":"2024-05-22T22:59:54.133512+0000","last_fullsized":"2024-05-22T22:59:54.133512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:18:14.609682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:54.133460+0000","last_change":"2024-05-22T22:59:51.951981+0000","last_active":"2024-05-22T22:59:54.133460+0000","last_peered":"2024-05-22T22:59:54.133460+0000","last_clean":"2024-05-22T22:59:54.133460+0000","last_became_active":"2024-05-22T22:59:51.951522+0000","last_became_peered":"2024-05-22T22:59:51.951522+0000","last_unstale":"2024-05-22T22:59:54.133460+0000","last_undegraded":"2024-05-22T22:59:54.133460+0000","last_fullsized":"2024-05-22T22:59:54.133460+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:56:23.357290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.945227+0000","last_change":"2024-05-22T22:59:51.966141+0000","last_active":"2024-05-22T22:59:52.945227+0000","last_peered":"2024-05-22T22:59:52.945227+0000","last_clean":"2024-05-22T22:59:52.945227+0000","last_became_active":"2024-05-22T22:59:51.965798+0000","last_became_peered":"2024-05-22T22:59:51.965798+0000","last_unstale":"2024-05-22T22:59:52.945227+0000","last_undegraded":"2024-05-22T22:59:52.945227+0000","last_fullsized":"2024-05-22T22:59:52.945227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:23:58.282078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.951236+0000","last_change":"2024-05-22T22:59:51.950425+0000","last_active":"2024-05-22T22:59:51.951236+0000","last_peered":"2024-05-22T22:59:51.951236+0000","last_clean":"2024-05-22T22:59:51.951236+0000","last_became_active":"2024-05-22T22:59:51.950071+0000","last_became_peered":"2024-05-22T22:59:51.950071+0000","last_unstale":"2024-05-22T22:59:51.951236+0000","last_undegraded":"2024-05-22T22:59:51.951236+0000","last_fullsized":"2024-05-22T22:59:51.951236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:40:13.357763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.637229+0000","last_change":"2024-05-22T22:59:51.965131+0000","last_active":"2024-05-22T22:59:53.637229+0000","last_peered":"2024-05-22T22:59:53.637229+0000","last_clean":"2024-05-22T22:59:53.637229+0000","last_became_active":"2024-05-22T22:59:51.964541+0000","last_became_peered":"2024-05-22T22:59:51.964541+0000","last_unstale":"2024-05-22T22:59:53.637229+0000","last_undegraded":"2024-05-22T22:59:53.637229+0000","last_fullsized":"2024-05-22T22:59:53.637229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:42:59.295475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.960043+0000","last_change":"2024-05-22T22:59:51.959583+0000","last_active":"2024-05-22T22:59:51.960043+0000","last_peered":"2024-05-22T22:59:51.960043+0000","last_clean":"2024-05-22T22:59:51.960043+0000","last_became_active":"2024-05-22T22:59:51.959260+0000","last_became_peered":"2024-05-22T22:59:51.959260+0000","last_unstale":"2024-05-22T22:59:51.960043+0000","last_undegraded":"2024-05-22T22:59:51.960043+0000","last_fullsized":"2024-05-22T22:59:51.960043+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:58:59.597179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.946371+0000","last_change":"2024-05-22T22:59:51.968418+0000","last_active":"2024-05-22T22:59:52.946371+0000","last_peered":"2024-05-22T22:59:52.946371+0000","last_clean":"2024-05-22T22:59:52.946371+0000","last_became_active":"2024-05-22T22:59:51.968061+0000","last_became_peered":"2024-05-22T22:59:51.968061+0000","last_unstale":"2024-05-22T22:59:52.946371+0000","last_undegraded":"2024-05-22T22:59:52.946371+0000","last_fullsized":"2024-05-22T22:59:52.946371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:35:26.825313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.958270+0000","last_change":"2024-05-22T22:59:51.956954+0000","last_active":"2024-05-22T22:59:51.958270+0000","last_peered":"2024-05-22T22:59:51.958270+0000","last_clean":"2024-05-22T22:59:51.958270+0000","last_became_active":"2024-05-22T22:59:51.956570+0000","last_became_peered":"2024-05-22T22:59:51.956570+0000","last_unstale":"2024-05-22T22:59:51.958270+0000","last_undegraded":"2024-05-22T22:59:51.958270+0000","last_fullsized":"2024-05-22T22:59:51.958270+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:52:27.512572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.637433+0000","last_change":"2024-05-22T22:59:51.962371+0000","last_active":"2024-05-22T22:59:53.637433+0000","last_peered":"2024-05-22T22:59:53.637433+0000","last_clean":"2024-05-22T22:59:53.637433+0000","last_became_active":"2024-05-22T22:59:51.962036+0000","last_became_peered":"2024-05-22T22:59:51.962036+0000","last_unstale":"2024-05-22T22:59:53.637433+0000","last_undegraded":"2024-05-22T22:59:53.637433+0000","last_fullsized":"2024-05-22T22:59:53.637433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:27:21.745563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.954166+0000","last_change":"2024-05-22T22:59:51.952446+0000","last_active":"2024-05-22T22:59:51.954166+0000","last_peered":"2024-05-22T22:59:51.954166+0000","last_clean":"2024-05-22T22:59:51.954166+0000","last_became_active":"2024-05-22T22:59:51.952115+0000","last_became_peered":"2024-05-22T22:59:51.952115+0000","last_unstale":"2024-05-22T22:59:51.954166+0000","last_undegraded":"2024-05-22T22:59:51.954166+0000","last_fullsized":"2024-05-22T22:59:51.954166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:44:25.285051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.945031+0000","last_change":"2024-05-22T22:59:51.968457+0000","last_active":"2024-05-22T22:59:52.945031+0000","last_peered":"2024-05-22T22:59:52.945031+0000","last_clean":"2024-05-22T22:59:52.945031+0000","last_became_active":"2024-05-22T22:59:51.968070+0000","last_became_peered":"2024-05-22T22:59:51.968070+0000","last_unstale":"2024-05-22T22:59:52.945031+0000","last_undegraded":"2024-05-22T22:59:52.945031+0000","last_fullsized":"2024-05-22T22:59:52.945031+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:07:31.595119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.635747+0000","last_change":"2024-05-22T22:59:51.957357+0000","last_active":"2024-05-22T22:59:53.635747+0000","last_peered":"2024-05-22T22:59:53.635747+0000","last_clean":"2024-05-22T22:59:53.635747+0000","last_became_active":"2024-05-22T22:59:51.957013+0000","last_became_peered":"2024-05-22T22:59:51.957013+0000","last_unstale":"2024-05-22T22:59:53.635747+0000","last_undegraded":"2024-05-22T22:59:53.635747+0000","last_fullsized":"2024-05-22T22:59:53.635747+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:38:48.118178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:54.133210+0000","last_change":"2024-05-22T22:59:51.957084+0000","last_active":"2024-05-22T22:59:54.133210+0000","last_peered":"2024-05-22T22:59:54.133210+0000","last_clean":"2024-05-22T22:59:54.133210+0000","last_became_active":"2024-05-22T22:59:51.956787+0000","last_became_peered":"2024-05-22T22:59:51.956787+0000","last_unstale":"2024-05-22T22:59:54.133210+0000","last_undegraded":"2024-05-22T22:59:54.133210+0000","last_fullsized":"2024-05-22T22:59:54.133210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:30:40.578286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:54.133170+0000","last_change":"2024-05-22T22:59:51.956375+0000","last_active":"2024-05-22T22:59:54.133170+0000","last_peered":"2024-05-22T22:59:54.133170+0000","last_clean":"2024-05-22T22:59:54.133170+0000","last_became_active":"2024-05-22T22:59:51.956012+0000","last_became_peered":"2024-05-22T22:59:51.956012+0000","last_unstale":"2024-05-22T22:59:54.133170+0000","last_undegraded":"2024-05-22T22:59:54.133170+0000","last_fullsized":"2024-05-22T22:59:54.133170+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:33:49.920327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.952173+0000","last_change":"2024-05-22T22:59:51.950287+0000","last_active":"2024-05-22T22:59:51.952173+0000","last_peered":"2024-05-22T22:59:51.952173+0000","last_clean":"2024-05-22T22:59:51.952173+0000","last_became_active":"2024-05-22T22:59:51.949962+0000","last_became_peered":"2024-05-22T22:59:51.949962+0000","last_unstale":"2024-05-22T22:59:51.952173+0000","last_undegraded":"2024-05-22T22:59:51.952173+0000","last_fullsized":"2024-05-22T22:59:51.952173+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:30:43.175576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.636596+0000","last_change":"2024-05-22T22:59:51.952523+0000","last_active":"2024-05-22T22:59:53.636596+0000","last_peered":"2024-05-22T22:59:53.636596+0000","last_clean":"2024-05-22T22:59:53.636596+0000","last_became_active":"2024-05-22T22:59:51.952152+0000","last_became_peered":"2024-05-22T22:59:51.952152+0000","last_unstale":"2024-05-22T22:59:53.636596+0000","last_undegraded":"2024-05-22T22:59:53.636596+0000","last_fullsized":"2024-05-22T22:59:53.636596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:33:01.332667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.591951+0000","last_change":"2024-05-22T22:59:51.960919+0000","last_active":"2024-05-22T22:59:53.591951+0000","last_peered":"2024-05-22T22:59:53.591951+0000","last_clean":"2024-05-22T22:59:53.591951+0000","last_became_active":"2024-05-22T22:59:51.960603+0000","last_became_peered":"2024-05-22T22:59:51.960603+0000","last_unstale":"2024-05-22T22:59:53.591951+0000","last_undegraded":"2024-05-22T22:59:53.591951+0000","last_fullsized":"2024-05-22T22:59:53.591951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:28:05.977612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.636689+0000","last_change":"2024-05-22T22:59:51.949186+0000","last_active":"2024-05-22T22:59:53.636689+0000","last_peered":"2024-05-22T22:59:53.636689+0000","last_clean":"2024-05-22T22:59:53.636689+0000","last_became_active":"2024-05-22T22:59:51.948819+0000","last_became_peered":"2024-05-22T22:59:51.948819+0000","last_unstale":"2024-05-22T22:59:53.636689+0000","last_undegraded":"2024-05-22T22:59:53.636689+0000","last_fullsized":"2024-05-22T22:59:53.636689+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:32:16.153838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:54.608966+0000","last_change":"2024-05-22T22:59:51.592520+0000","last_active":"2024-05-22T22:59:54.608966+0000","last_peered":"2024-05-22T22:59:54.608966+0000","last_clean":"2024-05-22T22:59:54.608966+0000","last_became_active":"2024-05-22T22:59:51.592201+0000","last_became_peered":"2024-05-22T22:59:51.592201+0000","last_unstale":"2024-05-22T22:59:54.608966+0000","last_undegraded":"2024-05-22T22:59:54.608966+0000","last_fullsized":"2024-05-22T22:59:54.608966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:45:06.500085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.946551+0000","last_change":"2024-05-22T22:59:51.963169+0000","last_active":"2024-05-22T22:59:52.946551+0000","last_peered":"2024-05-22T22:59:52.946551+0000","last_clean":"2024-05-22T22:59:52.946551+0000","last_became_active":"2024-05-22T22:59:51.962875+0000","last_became_peered":"2024-05-22T22:59:51.962875+0000","last_unstale":"2024-05-22T22:59:52.946551+0000","last_undegraded":"2024-05-22T22:59:52.946551+0000","last_fullsized":"2024-05-22T22:59:52.946551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:03:24.092690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.946833+0000","last_change":"2024-05-22T22:59:51.595383+0000","last_active":"2024-05-22T22:59:52.946833+0000","last_peered":"2024-05-22T22:59:52.946833+0000","last_clean":"2024-05-22T22:59:52.946833+0000","last_became_active":"2024-05-22T22:59:51.595076+0000","last_became_peered":"2024-05-22T22:59:51.595076+0000","last_unstale":"2024-05-22T22:59:52.946833+0000","last_undegraded":"2024-05-22T22:59:52.946833+0000","last_fullsized":"2024-05-22T22:59:52.946833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:29:57.205196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:54.133623+0000","last_change":"2024-05-22T22:59:51.952447+0000","last_active":"2024-05-22T22:59:54.133623+0000","last_peered":"2024-05-22T22:59:54.133623+0000","last_clean":"2024-05-22T22:59:54.133623+0000","last_became_active":"2024-05-22T22:59:51.951927+0000","last_became_peered":"2024-05-22T22:59:51.951927+0000","last_unstale":"2024-05-22T22:59:54.133623+0000","last_undegraded":"2024-05-22T22:59:54.133623+0000","last_fullsized":"2024-05-22T22:59:54.133623+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:14:59.161404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.596672+0000","last_change":"2024-05-22T22:59:51.597705+0000","last_active":"2024-05-22T22:59:52.596672+0000","last_peered":"2024-05-22T22:59:52.596672+0000","last_clean":"2024-05-22T22:59:52.596672+0000","last_became_active":"2024-05-22T22:59:51.597319+0000","last_became_peered":"2024-05-22T22:59:51.597319+0000","last_unstale":"2024-05-22T22:59:52.596672+0000","last_undegraded":"2024-05-22T22:59:52.596672+0000","last_fullsized":"2024-05-22T22:59:52.596672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:59:59.307499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.957797+0000","last_change":"2024-05-22T22:59:51.955477+0000","last_active":"2024-05-22T22:59:51.957797+0000","last_peered":"2024-05-22T22:59:51.957797+0000","last_clean":"2024-05-22T22:59:51.957797+0000","last_became_active":"2024-05-22T22:59:51.953608+0000","last_became_peered":"2024-05-22T22:59:51.953608+0000","last_unstale":"2024-05-22T22:59:51.957797+0000","last_undegraded":"2024-05-22T22:59:51.957797+0000","last_fullsized":"2024-05-22T22:59:51.957797+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:37:35.939922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.598912+0000","last_change":"2024-05-22T22:59:51.595417+0000","last_active":"2024-05-22T22:59:52.598912+0000","last_peered":"2024-05-22T22:59:52.598912+0000","last_clean":"2024-05-22T22:59:52.598912+0000","last_became_active":"2024-05-22T22:59:51.595051+0000","last_became_peered":"2024-05-22T22:59:51.595051+0000","last_unstale":"2024-05-22T22:59:52.598912+0000","last_undegraded":"2024-05-22T22:59:52.598912+0000","last_fullsized":"2024-05-22T22:59:52.598912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:13:47.936348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.947722+0000","last_change":"2024-05-22T22:59:51.958471+0000","last_active":"2024-05-22T22:59:52.947722+0000","last_peered":"2024-05-22T22:59:52.947722+0000","last_clean":"2024-05-22T22:59:52.947722+0000","last_became_active":"2024-05-22T22:59:51.958017+0000","last_became_peered":"2024-05-22T22:59:51.958017+0000","last_unstale":"2024-05-22T22:59:52.947722+0000","last_undegraded":"2024-05-22T22:59:52.947722+0000","last_fullsized":"2024-05-22T22:59:52.947722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:42:52.196892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.947388+0000","last_change":"2024-05-22T22:59:51.591909+0000","last_active":"2024-05-22T22:59:52.947388+0000","last_peered":"2024-05-22T22:59:52.947388+0000","last_clean":"2024-05-22T22:59:52.947388+0000","last_became_active":"2024-05-22T22:59:51.591531+0000","last_became_peered":"2024-05-22T22:59:51.591531+0000","last_unstale":"2024-05-22T22:59:52.947388+0000","last_undegraded":"2024-05-22T22:59:52.947388+0000","last_fullsized":"2024-05-22T22:59:52.947388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:07:22.997156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.962383+0000","last_change":"2024-05-22T22:59:51.961444+0000","last_active":"2024-05-22T22:59:51.962383+0000","last_peered":"2024-05-22T22:59:51.962383+0000","last_clean":"2024-05-22T22:59:51.962383+0000","last_became_active":"2024-05-22T22:59:51.961088+0000","last_became_peered":"2024-05-22T22:59:51.961088+0000","last_unstale":"2024-05-22T22:59:51.962383+0000","last_undegraded":"2024-05-22T22:59:51.962383+0000","last_fullsized":"2024-05-22T22:59:51.962383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:49:29.238180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.636949+0000","last_change":"2024-05-22T22:59:51.590124+0000","last_active":"2024-05-22T22:59:53.636949+0000","last_peered":"2024-05-22T22:59:53.636949+0000","last_clean":"2024-05-22T22:59:53.636949+0000","last_became_active":"2024-05-22T22:59:51.589661+0000","last_became_peered":"2024-05-22T22:59:51.589661+0000","last_unstale":"2024-05-22T22:59:53.636949+0000","last_undegraded":"2024-05-22T22:59:53.636949+0000","last_fullsized":"2024-05-22T22:59:53.636949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:33:14.293487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.947664+0000","last_change":"2024-05-22T22:59:51.958574+0000","last_active":"2024-05-22T22:59:52.947664+0000","last_peered":"2024-05-22T22:59:52.947664+0000","last_clean":"2024-05-22T22:59:52.947664+0000","last_became_active":"2024-05-22T22:59:51.958252+0000","last_became_peered":"2024-05-22T22:59:51.958252+0000","last_unstale":"2024-05-22T22:59:52.947664+0000","last_undegraded":"2024-05-22T22:59:52.947664+0000","last_fullsized":"2024-05-22T22:59:52.947664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:18:43.192032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.635528+0000","last_change":"2024-05-22T22:59:51.595510+0000","last_active":"2024-05-22T22:59:53.635528+0000","last_peered":"2024-05-22T22:59:53.635528+0000","last_clean":"2024-05-22T22:59:53.635528+0000","last_became_active":"2024-05-22T22:59:51.595208+0000","last_became_peered":"2024-05-22T22:59:51.595208+0000","last_unstale":"2024-05-22T22:59:53.635528+0000","last_undegraded":"2024-05-22T22:59:53.635528+0000","last_fullsized":"2024-05-22T22:59:53.635528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:47:46.957965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.946501+0000","last_change":"2024-05-22T22:59:51.950684+0000","last_active":"2024-05-22T22:59:52.946501+0000","last_peered":"2024-05-22T22:59:52.946501+0000","last_clean":"2024-05-22T22:59:52.946501+0000","last_became_active":"2024-05-22T22:59:51.950380+0000","last_became_peered":"2024-05-22T22:59:51.950380+0000","last_unstale":"2024-05-22T22:59:52.946501+0000","last_undegraded":"2024-05-22T22:59:52.946501+0000","last_fullsized":"2024-05-22T22:59:52.946501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:58:02.035245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.947092+0000","last_change":"2024-05-22T22:59:51.592958+0000","last_active":"2024-05-22T22:59:52.947092+0000","last_peered":"2024-05-22T22:59:52.947092+0000","last_clean":"2024-05-22T22:59:52.947092+0000","last_became_active":"2024-05-22T22:59:51.592586+0000","last_became_peered":"2024-05-22T22:59:51.592586+0000","last_unstale":"2024-05-22T22:59:52.947092+0000","last_undegraded":"2024-05-22T22:59:52.947092+0000","last_fullsized":"2024-05-22T22:59:52.947092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:44:43.859724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.591688+0000","last_change":"2024-05-22T22:59:51.960863+0000","last_active":"2024-05-22T22:59:53.591688+0000","last_peered":"2024-05-22T22:59:53.591688+0000","last_clean":"2024-05-22T22:59:53.591688+0000","last_became_active":"2024-05-22T22:59:51.960537+0000","last_became_peered":"2024-05-22T22:59:51.960537+0000","last_unstale":"2024-05-22T22:59:53.591688+0000","last_undegraded":"2024-05-22T22:59:53.591688+0000","last_fullsized":"2024-05-22T22:59:53.591688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:08:19.735397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.637090+0000","last_change":"2024-05-22T22:59:51.596150+0000","last_active":"2024-05-22T22:59:53.637090+0000","last_peered":"2024-05-22T22:59:53.637090+0000","last_clean":"2024-05-22T22:59:53.637090+0000","last_became_active":"2024-05-22T22:59:51.595791+0000","last_became_peered":"2024-05-22T22:59:51.595791+0000","last_unstale":"2024-05-22T22:59:53.637090+0000","last_undegraded":"2024-05-22T22:59:53.637090+0000","last_fullsized":"2024-05-22T22:59:53.637090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:47:46.144045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.952749+0000","last_change":"2024-05-22T22:59:51.951907+0000","last_active":"2024-05-22T22:59:51.952749+0000","last_peered":"2024-05-22T22:59:51.952749+0000","last_clean":"2024-05-22T22:59:51.952749+0000","last_became_active":"2024-05-22T22:59:51.951264+0000","last_became_peered":"2024-05-22T22:59:51.951264+0000","last_unstale":"2024-05-22T22:59:51.952749+0000","last_undegraded":"2024-05-22T22:59:51.952749+0000","last_fullsized":"2024-05-22T22:59:51.952749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:55:41.743473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.946008+0000","last_change":"2024-05-22T22:59:51.594883+0000","last_active":"2024-05-22T22:59:51.946008+0000","last_peered":"2024-05-22T22:59:51.946008+0000","last_clean":"2024-05-22T22:59:51.946008+0000","last_became_active":"2024-05-22T22:59:51.594458+0000","last_became_peered":"2024-05-22T22:59:51.594458+0000","last_unstale":"2024-05-22T22:59:51.946008+0000","last_undegraded":"2024-05-22T22:59:51.946008+0000","last_fullsized":"2024-05-22T22:59:51.946008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:39:27.366961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:54.133556+0000","last_change":"2024-05-22T22:59:51.958180+0000","last_active":"2024-05-22T22:59:54.133556+0000","last_peered":"2024-05-22T22:59:54.133556+0000","last_clean":"2024-05-22T22:59:54.133556+0000","last_became_active":"2024-05-22T22:59:51.957877+0000","last_became_peered":"2024-05-22T22:59:51.957877+0000","last_unstale":"2024-05-22T22:59:54.133556+0000","last_undegraded":"2024-05-22T22:59:54.133556+0000","last_fullsized":"2024-05-22T22:59:54.133556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:33:42.478340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.947638+0000","last_change":"2024-05-22T22:59:51.592953+0000","last_active":"2024-05-22T22:59:52.947638+0000","last_peered":"2024-05-22T22:59:52.947638+0000","last_clean":"2024-05-22T22:59:52.947638+0000","last_became_active":"2024-05-22T22:59:51.592525+0000","last_became_peered":"2024-05-22T22:59:51.592525+0000","last_unstale":"2024-05-22T22:59:52.947638+0000","last_undegraded":"2024-05-22T22:59:52.947638+0000","last_fullsized":"2024-05-22T22:59:52.947638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:18:03.512123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.963140+0000","last_change":"2024-05-22T22:59:51.959510+0000","last_active":"2024-05-22T22:59:51.963140+0000","last_peered":"2024-05-22T22:59:51.963140+0000","last_clean":"2024-05-22T22:59:51.963140+0000","last_became_active":"2024-05-22T22:59:51.959197+0000","last_became_peered":"2024-05-22T22:59:51.959197+0000","last_unstale":"2024-05-22T22:59:51.963140+0000","last_undegraded":"2024-05-22T22:59:51.963140+0000","last_fullsized":"2024-05-22T22:59:51.963140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:49:07.447380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'17","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:56.993935+0000","last_change":"2024-05-22T22:59:51.593430+0000","last_active":"2024-05-22T22:59:56.993935+0000","last_peered":"2024-05-22T22:59:56.993935+0000","last_clean":"2024-05-22T22:59:56.993935+0000","last_became_active":"2024-05-22T22:59:51.592541+0000","last_became_peered":"2024-05-22T22:59:51.592541+0000","last_unstale":"2024-05-22T22:59:56.993935+0000","last_undegraded":"2024-05-22T22:59:56.993935+0000","last_fullsized":"2024-05-22T22:59:56.993935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:24:26.709211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38192,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":24,"num_write":17,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.953141+0000","last_change":"2024-05-22T22:59:51.952486+0000","last_active":"2024-05-22T22:59:51.953141+0000","last_peered":"2024-05-22T22:59:51.953141+0000","last_clean":"2024-05-22T22:59:51.953141+0000","last_became_active":"2024-05-22T22:59:51.952096+0000","last_became_peered":"2024-05-22T22:59:51.952096+0000","last_unstale":"2024-05-22T22:59:51.953141+0000","last_undegraded":"2024-05-22T22:59:51.953141+0000","last_fullsized":"2024-05-22T22:59:51.953141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:07:37.260966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.636290+0000","last_change":"2024-05-22T22:59:51.595446+0000","last_active":"2024-05-22T22:59:53.636290+0000","last_peered":"2024-05-22T22:59:53.636290+0000","last_clean":"2024-05-22T22:59:53.636290+0000","last_became_active":"2024-05-22T22:59:51.595097+0000","last_became_peered":"2024-05-22T22:59:51.595097+0000","last_unstale":"2024-05-22T22:59:53.636290+0000","last_undegraded":"2024-05-22T22:59:53.636290+0000","last_fullsized":"2024-05-22T22:59:53.636290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:03:29.722773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.635599+0000","last_change":"2024-05-22T22:59:51.953304+0000","last_active":"2024-05-22T22:59:53.635599+0000","last_peered":"2024-05-22T22:59:53.635599+0000","last_clean":"2024-05-22T22:59:53.635599+0000","last_became_active":"2024-05-22T22:59:51.952974+0000","last_became_peered":"2024-05-22T22:59:51.952974+0000","last_unstale":"2024-05-22T22:59:53.635599+0000","last_undegraded":"2024-05-22T22:59:53.635599+0000","last_fullsized":"2024-05-22T22:59:53.635599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:21:06.116285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.947276+0000","last_change":"2024-05-22T22:59:51.593684+0000","last_active":"2024-05-22T22:59:52.947276+0000","last_peered":"2024-05-22T22:59:52.947276+0000","last_clean":"2024-05-22T22:59:52.947276+0000","last_became_active":"2024-05-22T22:59:51.593335+0000","last_became_peered":"2024-05-22T22:59:51.593335+0000","last_unstale":"2024-05-22T22:59:52.947276+0000","last_undegraded":"2024-05-22T22:59:52.947276+0000","last_fullsized":"2024-05-22T22:59:52.947276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:36:24.405609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.961299+0000","last_change":"2024-05-22T22:59:51.960930+0000","last_active":"2024-05-22T22:59:51.961299+0000","last_peered":"2024-05-22T22:59:51.961299+0000","last_clean":"2024-05-22T22:59:51.961299+0000","last_became_active":"2024-05-22T22:59:51.960571+0000","last_became_peered":"2024-05-22T22:59:51.960571+0000","last_unstale":"2024-05-22T22:59:51.961299+0000","last_undegraded":"2024-05-22T22:59:51.961299+0000","last_fullsized":"2024-05-22T22:59:51.961299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:15:36.869955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.944637+0000","last_change":"2024-05-22T22:59:51.592876+0000","last_active":"2024-05-22T22:59:52.944637+0000","last_peered":"2024-05-22T22:59:52.944637+0000","last_clean":"2024-05-22T22:59:52.944637+0000","last_became_active":"2024-05-22T22:59:51.592488+0000","last_became_peered":"2024-05-22T22:59:51.592488+0000","last_unstale":"2024-05-22T22:59:52.944637+0000","last_undegraded":"2024-05-22T22:59:52.944637+0000","last_fullsized":"2024-05-22T22:59:52.944637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:37:16.517302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.962771+0000","last_change":"2024-05-22T22:59:51.962336+0000","last_active":"2024-05-22T22:59:51.962771+0000","last_peered":"2024-05-22T22:59:51.962771+0000","last_clean":"2024-05-22T22:59:51.962771+0000","last_became_active":"2024-05-22T22:59:51.962002+0000","last_became_peered":"2024-05-22T22:59:51.962002+0000","last_unstale":"2024-05-22T22:59:51.962771+0000","last_undegraded":"2024-05-22T22:59:51.962771+0000","last_fullsized":"2024-05-22T22:59:51.962771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:30:17.276737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.599970+0000","last_change":"2024-05-22T22:59:51.596218+0000","last_active":"2024-05-22T22:59:52.599970+0000","last_peered":"2024-05-22T22:59:52.599970+0000","last_clean":"2024-05-22T22:59:52.599970+0000","last_became_active":"2024-05-22T22:59:51.595138+0000","last_became_peered":"2024-05-22T22:59:51.595138+0000","last_unstale":"2024-05-22T22:59:52.599970+0000","last_undegraded":"2024-05-22T22:59:52.599970+0000","last_fullsized":"2024-05-22T22:59:52.599970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:41:49.748215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.947166+0000","last_change":"2024-05-22T22:59:51.961600+0000","last_active":"2024-05-22T22:59:52.947166+0000","last_peered":"2024-05-22T22:59:52.947166+0000","last_clean":"2024-05-22T22:59:52.947166+0000","last_became_active":"2024-05-22T22:59:51.961286+0000","last_became_peered":"2024-05-22T22:59:51.961286+0000","last_unstale":"2024-05-22T22:59:52.947166+0000","last_undegraded":"2024-05-22T22:59:52.947166+0000","last_fullsized":"2024-05-22T22:59:52.947166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:04:26.992865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.598724+0000","last_change":"2024-05-22T22:59:51.594461+0000","last_active":"2024-05-22T22:59:52.598724+0000","last_peered":"2024-05-22T22:59:52.598724+0000","last_clean":"2024-05-22T22:59:52.598724+0000","last_became_active":"2024-05-22T22:59:51.594128+0000","last_became_peered":"2024-05-22T22:59:51.594128+0000","last_unstale":"2024-05-22T22:59:52.598724+0000","last_undegraded":"2024-05-22T22:59:52.598724+0000","last_fullsized":"2024-05-22T22:59:52.598724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:52:05.095814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.636280+0000","last_change":"2024-05-22T22:59:51.958851+0000","last_active":"2024-05-22T22:59:53.636280+0000","last_peered":"2024-05-22T22:59:53.636280+0000","last_clean":"2024-05-22T22:59:53.636280+0000","last_became_active":"2024-05-22T22:59:51.957412+0000","last_became_peered":"2024-05-22T22:59:51.957412+0000","last_unstale":"2024-05-22T22:59:53.636280+0000","last_undegraded":"2024-05-22T22:59:53.636280+0000","last_fullsized":"2024-05-22T22:59:53.636280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:13:02.761205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.592001+0000","last_change":"2024-05-22T22:59:51.588750+0000","last_active":"2024-05-22T22:59:53.592001+0000","last_peered":"2024-05-22T22:59:53.592001+0000","last_clean":"2024-05-22T22:59:53.592001+0000","last_became_active":"2024-05-22T22:59:51.588364+0000","last_became_peered":"2024-05-22T22:59:51.588364+0000","last_unstale":"2024-05-22T22:59:53.592001+0000","last_undegraded":"2024-05-22T22:59:53.592001+0000","last_fullsized":"2024-05-22T22:59:53.592001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:28:41.605406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.637324+0000","last_change":"2024-05-22T22:59:51.964676+0000","last_active":"2024-05-22T22:59:53.637324+0000","last_peered":"2024-05-22T22:59:53.637324+0000","last_clean":"2024-05-22T22:59:53.637324+0000","last_became_active":"2024-05-22T22:59:51.960472+0000","last_became_peered":"2024-05-22T22:59:51.960472+0000","last_unstale":"2024-05-22T22:59:53.637324+0000","last_undegraded":"2024-05-22T22:59:53.637324+0000","last_fullsized":"2024-05-22T22:59:53.637324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:24:55.042637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.946252+0000","last_change":"2024-05-22T22:59:51.600873+0000","last_active":"2024-05-22T22:59:51.946252+0000","last_peered":"2024-05-22T22:59:51.946252+0000","last_clean":"2024-05-22T22:59:51.946252+0000","last_became_active":"2024-05-22T22:59:51.599785+0000","last_became_peered":"2024-05-22T22:59:51.599785+0000","last_unstale":"2024-05-22T22:59:51.946252+0000","last_undegraded":"2024-05-22T22:59:51.946252+0000","last_fullsized":"2024-05-22T22:59:51.946252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:20:09.590765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.637238+0000","last_change":"2024-05-22T22:59:51.964942+0000","last_active":"2024-05-22T22:59:53.637238+0000","last_peered":"2024-05-22T22:59:53.637238+0000","last_clean":"2024-05-22T22:59:53.637238+0000","last_became_active":"2024-05-22T22:59:51.964333+0000","last_became_peered":"2024-05-22T22:59:51.964333+0000","last_unstale":"2024-05-22T22:59:53.637238+0000","last_undegraded":"2024-05-22T22:59:53.637238+0000","last_fullsized":"2024-05-22T22:59:53.637238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:01:07.215075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.947286+0000","last_change":"2024-05-22T22:59:51.957291+0000","last_active":"2024-05-22T22:59:52.947286+0000","last_peered":"2024-05-22T22:59:52.947286+0000","last_clean":"2024-05-22T22:59:52.947286+0000","last_became_active":"2024-05-22T22:59:51.956987+0000","last_became_peered":"2024-05-22T22:59:51.956987+0000","last_unstale":"2024-05-22T22:59:52.947286+0000","last_undegraded":"2024-05-22T22:59:52.947286+0000","last_fullsized":"2024-05-22T22:59:52.947286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:14:10.420756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.635409+0000","last_change":"2024-05-22T22:59:51.590255+0000","last_active":"2024-05-22T22:59:53.635409+0000","last_peered":"2024-05-22T22:59:53.635409+0000","last_clean":"2024-05-22T22:59:53.635409+0000","last_became_active":"2024-05-22T22:59:51.589261+0000","last_became_peered":"2024-05-22T22:59:51.589261+0000","last_unstale":"2024-05-22T22:59:53.635409+0000","last_undegraded":"2024-05-22T22:59:53.635409+0000","last_fullsized":"2024-05-22T22:59:53.635409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:15:14.171311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.946653+0000","last_change":"2024-05-22T22:59:51.961453+0000","last_active":"2024-05-22T22:59:52.946653+0000","last_peered":"2024-05-22T22:59:52.946653+0000","last_clean":"2024-05-22T22:59:52.946653+0000","last_became_active":"2024-05-22T22:59:51.961082+0000","last_became_peered":"2024-05-22T22:59:51.961082+0000","last_unstale":"2024-05-22T22:59:52.946653+0000","last_undegraded":"2024-05-22T22:59:52.946653+0000","last_fullsized":"2024-05-22T22:59:52.946653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:05:50.310688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.944645+0000","last_change":"2024-05-22T22:59:51.595913+0000","last_active":"2024-05-22T22:59:52.944645+0000","last_peered":"2024-05-22T22:59:52.944645+0000","last_clean":"2024-05-22T22:59:52.944645+0000","last_became_active":"2024-05-22T22:59:51.595561+0000","last_became_peered":"2024-05-22T22:59:51.595561+0000","last_unstale":"2024-05-22T22:59:52.944645+0000","last_undegraded":"2024-05-22T22:59:52.944645+0000","last_fullsized":"2024-05-22T22:59:52.944645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:37:04.701575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.945950+0000","last_change":"2024-05-22T22:58:34.927654+0000","last_active":"2024-05-22T22:59:51.945950+0000","last_peered":"2024-05-22T22:59:51.945950+0000","last_clean":"2024-05-22T22:59:51.945950+0000","last_became_active":"2024-05-22T22:58:34.927090+0000","last_became_peered":"2024-05-22T22:58:34.927090+0000","last_unstale":"2024-05-22T22:59:51.945950+0000","last_undegraded":"2024-05-22T22:59:51.945950+0000","last_fullsized":"2024-05-22T22:59:51.945950+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:55:07.719852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:55:07.719852+0000","last_clean_scrub_stamp":"2024-05-22T22:55:07.719852+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:40:53.763146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.635449+0000","last_change":"2024-05-22T22:59:51.956422+0000","last_active":"2024-05-22T22:59:53.635449+0000","last_peered":"2024-05-22T22:59:53.635449+0000","last_clean":"2024-05-22T22:59:53.635449+0000","last_became_active":"2024-05-22T22:59:51.956127+0000","last_became_peered":"2024-05-22T22:59:51.956127+0000","last_unstale":"2024-05-22T22:59:53.635449+0000","last_undegraded":"2024-05-22T22:59:53.635449+0000","last_fullsized":"2024-05-22T22:59:53.635449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:04:36.955644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.946907+0000","last_change":"2024-05-22T22:59:51.599362+0000","last_active":"2024-05-22T22:59:51.946907+0000","last_peered":"2024-05-22T22:59:51.946907+0000","last_clean":"2024-05-22T22:59:51.946907+0000","last_became_active":"2024-05-22T22:59:51.599027+0000","last_became_peered":"2024-05-22T22:59:51.599027+0000","last_unstale":"2024-05-22T22:59:51.946907+0000","last_undegraded":"2024-05-22T22:59:51.946907+0000","last_fullsized":"2024-05-22T22:59:51.946907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:58:07.923155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.941382+0000","last_change":"2024-05-22T22:59:51.598772+0000","last_active":"2024-05-22T22:59:51.941382+0000","last_peered":"2024-05-22T22:59:51.941382+0000","last_clean":"2024-05-22T22:59:51.941382+0000","last_became_active":"2024-05-22T22:59:51.598192+0000","last_became_peered":"2024-05-22T22:59:51.598192+0000","last_unstale":"2024-05-22T22:59:51.941382+0000","last_undegraded":"2024-05-22T22:59:51.941382+0000","last_fullsized":"2024-05-22T22:59:51.941382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:42:47.069554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.635748+0000","last_change":"2024-05-22T22:59:51.942337+0000","last_active":"2024-05-22T22:59:53.635748+0000","last_peered":"2024-05-22T22:59:53.635748+0000","last_clean":"2024-05-22T22:59:53.635748+0000","last_became_active":"2024-05-22T22:59:51.941939+0000","last_became_peered":"2024-05-22T22:59:51.941939+0000","last_unstale":"2024-05-22T22:59:53.635748+0000","last_undegraded":"2024-05-22T22:59:53.635748+0000","last_fullsized":"2024-05-22T22:59:53.635748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:35:42.360128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.593310+0000","last_change":"2024-05-22T22:59:51.589640+0000","last_active":"2024-05-22T22:59:53.593310+0000","last_peered":"2024-05-22T22:59:53.593310+0000","last_clean":"2024-05-22T22:59:53.593310+0000","last_became_active":"2024-05-22T22:59:51.589310+0000","last_became_peered":"2024-05-22T22:59:51.589310+0000","last_unstale":"2024-05-22T22:59:53.593310+0000","last_undegraded":"2024-05-22T22:59:53.593310+0000","last_fullsized":"2024-05-22T22:59:53.593310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:54:39.182002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.946982+0000","last_change":"2024-05-22T22:59:51.952423+0000","last_active":"2024-05-22T22:59:52.946982+0000","last_peered":"2024-05-22T22:59:52.946982+0000","last_clean":"2024-05-22T22:59:52.946982+0000","last_became_active":"2024-05-22T22:59:51.952015+0000","last_became_peered":"2024-05-22T22:59:51.952015+0000","last_unstale":"2024-05-22T22:59:52.946982+0000","last_undegraded":"2024-05-22T22:59:52.946982+0000","last_fullsized":"2024-05-22T22:59:52.946982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:06:23.705248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.945308+0000","last_change":"2024-05-22T22:59:51.594882+0000","last_active":"2024-05-22T22:59:52.945308+0000","last_peered":"2024-05-22T22:59:52.945308+0000","last_clean":"2024-05-22T22:59:52.945308+0000","last_became_active":"2024-05-22T22:59:51.594552+0000","last_became_peered":"2024-05-22T22:59:51.594552+0000","last_unstale":"2024-05-22T22:59:52.945308+0000","last_undegraded":"2024-05-22T22:59:52.945308+0000","last_fullsized":"2024-05-22T22:59:52.945308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:06:40.771969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:54.133518+0000","last_change":"2024-05-22T22:59:51.956534+0000","last_active":"2024-05-22T22:59:54.133518+0000","last_peered":"2024-05-22T22:59:54.133518+0000","last_clean":"2024-05-22T22:59:54.133518+0000","last_became_active":"2024-05-22T22:59:51.956037+0000","last_became_peered":"2024-05-22T22:59:51.956037+0000","last_unstale":"2024-05-22T22:59:54.133518+0000","last_undegraded":"2024-05-22T22:59:54.133518+0000","last_fullsized":"2024-05-22T22:59:54.133518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:24:15.289570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.937588+0000","last_change":"2024-05-22T22:59:51.592916+0000","last_active":"2024-05-22T22:59:51.937588+0000","last_peered":"2024-05-22T22:59:51.937588+0000","last_clean":"2024-05-22T22:59:51.937588+0000","last_became_active":"2024-05-22T22:59:51.592530+0000","last_became_peered":"2024-05-22T22:59:51.592530+0000","last_unstale":"2024-05-22T22:59:51.937588+0000","last_undegraded":"2024-05-22T22:59:51.937588+0000","last_fullsized":"2024-05-22T22:59:51.937588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:56:18.660120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.958967+0000","last_change":"2024-05-22T22:59:51.956188+0000","last_active":"2024-05-22T22:59:51.958967+0000","last_peered":"2024-05-22T22:59:51.958967+0000","last_clean":"2024-05-22T22:59:51.958967+0000","last_became_active":"2024-05-22T22:59:51.955844+0000","last_became_peered":"2024-05-22T22:59:51.955844+0000","last_unstale":"2024-05-22T22:59:51.958967+0000","last_undegraded":"2024-05-22T22:59:51.958967+0000","last_fullsized":"2024-05-22T22:59:51.958967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:56:54.378586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.946406+0000","last_change":"2024-05-22T22:59:51.590641+0000","last_active":"2024-05-22T22:59:52.946406+0000","last_peered":"2024-05-22T22:59:52.946406+0000","last_clean":"2024-05-22T22:59:52.946406+0000","last_became_active":"2024-05-22T22:59:51.590321+0000","last_became_peered":"2024-05-22T22:59:51.590321+0000","last_unstale":"2024-05-22T22:59:52.946406+0000","last_undegraded":"2024-05-22T22:59:52.946406+0000","last_fullsized":"2024-05-22T22:59:52.946406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:46:15.157160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.635764+0000","last_change":"2024-05-22T22:59:51.950256+0000","last_active":"2024-05-22T22:59:53.635764+0000","last_peered":"2024-05-22T22:59:53.635764+0000","last_clean":"2024-05-22T22:59:53.635764+0000","last_became_active":"2024-05-22T22:59:51.949746+0000","last_became_peered":"2024-05-22T22:59:51.949746+0000","last_unstale":"2024-05-22T22:59:53.635764+0000","last_undegraded":"2024-05-22T22:59:53.635764+0000","last_fullsized":"2024-05-22T22:59:53.635764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:33:12.146425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.636107+0000","last_change":"2024-05-22T22:59:51.595761+0000","last_active":"2024-05-22T22:59:53.636107+0000","last_peered":"2024-05-22T22:59:53.636107+0000","last_clean":"2024-05-22T22:59:53.636107+0000","last_became_active":"2024-05-22T22:59:51.595343+0000","last_became_peered":"2024-05-22T22:59:51.595343+0000","last_unstale":"2024-05-22T22:59:53.636107+0000","last_undegraded":"2024-05-22T22:59:53.636107+0000","last_fullsized":"2024-05-22T22:59:53.636107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:58:21.461675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.946653+0000","last_change":"2024-05-22T22:59:51.968432+0000","last_active":"2024-05-22T22:59:52.946653+0000","last_peered":"2024-05-22T22:59:52.946653+0000","last_clean":"2024-05-22T22:59:52.946653+0000","last_became_active":"2024-05-22T22:59:51.968077+0000","last_became_peered":"2024-05-22T22:59:51.968077+0000","last_unstale":"2024-05-22T22:59:52.946653+0000","last_undegraded":"2024-05-22T22:59:52.946653+0000","last_fullsized":"2024-05-22T22:59:52.946653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:08:16.281671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.596737+0000","last_change":"2024-05-22T22:59:51.601616+0000","last_active":"2024-05-22T22:59:52.596737+0000","last_peered":"2024-05-22T22:59:52.596737+0000","last_clean":"2024-05-22T22:59:52.596737+0000","last_became_active":"2024-05-22T22:59:51.601249+0000","last_became_peered":"2024-05-22T22:59:51.601249+0000","last_unstale":"2024-05-22T22:59:52.596737+0000","last_undegraded":"2024-05-22T22:59:52.596737+0000","last_fullsized":"2024-05-22T22:59:52.596737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:07:07.030142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.954100+0000","last_change":"2024-05-22T22:59:51.951578+0000","last_active":"2024-05-22T22:59:51.954100+0000","last_peered":"2024-05-22T22:59:51.954100+0000","last_clean":"2024-05-22T22:59:51.954100+0000","last_became_active":"2024-05-22T22:59:51.951233+0000","last_became_peered":"2024-05-22T22:59:51.951233+0000","last_unstale":"2024-05-22T22:59:51.954100+0000","last_undegraded":"2024-05-22T22:59:51.954100+0000","last_fullsized":"2024-05-22T22:59:51.954100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:53:07.452864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.636388+0000","last_change":"2024-05-22T22:59:51.597819+0000","last_active":"2024-05-22T22:59:53.636388+0000","last_peered":"2024-05-22T22:59:53.636388+0000","last_clean":"2024-05-22T22:59:53.636388+0000","last_became_active":"2024-05-22T22:59:51.597262+0000","last_became_peered":"2024-05-22T22:59:51.597262+0000","last_unstale":"2024-05-22T22:59:53.636388+0000","last_undegraded":"2024-05-22T22:59:53.636388+0000","last_fullsized":"2024-05-22T22:59:53.636388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:05:02.849623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.954143+0000","last_change":"2024-05-22T22:59:51.951668+0000","last_active":"2024-05-22T22:59:51.954143+0000","last_peered":"2024-05-22T22:59:51.954143+0000","last_clean":"2024-05-22T22:59:51.954143+0000","last_became_active":"2024-05-22T22:59:51.951176+0000","last_became_peered":"2024-05-22T22:59:51.951176+0000","last_unstale":"2024-05-22T22:59:51.954143+0000","last_undegraded":"2024-05-22T22:59:51.954143+0000","last_fullsized":"2024-05-22T22:59:51.954143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:22:19.179260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.947547+0000","last_change":"2024-05-22T22:59:51.593144+0000","last_active":"2024-05-22T22:59:52.947547+0000","last_peered":"2024-05-22T22:59:52.947547+0000","last_clean":"2024-05-22T22:59:52.947547+0000","last_became_active":"2024-05-22T22:59:51.592557+0000","last_became_peered":"2024-05-22T22:59:51.592557+0000","last_unstale":"2024-05-22T22:59:52.947547+0000","last_undegraded":"2024-05-22T22:59:52.947547+0000","last_fullsized":"2024-05-22T22:59:52.947547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:28:19.403419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.960072+0000","last_change":"2024-05-22T22:59:51.958866+0000","last_active":"2024-05-22T22:59:51.960072+0000","last_peered":"2024-05-22T22:59:51.960072+0000","last_clean":"2024-05-22T22:59:51.960072+0000","last_became_active":"2024-05-22T22:59:51.958505+0000","last_became_peered":"2024-05-22T22:59:51.958505+0000","last_unstale":"2024-05-22T22:59:51.960072+0000","last_undegraded":"2024-05-22T22:59:51.960072+0000","last_fullsized":"2024-05-22T22:59:51.960072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:07:18.590496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.635743+0000","last_change":"2024-05-22T22:59:51.587391+0000","last_active":"2024-05-22T22:59:53.635743+0000","last_peered":"2024-05-22T22:59:53.635743+0000","last_clean":"2024-05-22T22:59:53.635743+0000","last_became_active":"2024-05-22T22:59:51.587056+0000","last_became_peered":"2024-05-22T22:59:51.587056+0000","last_unstale":"2024-05-22T22:59:53.635743+0000","last_undegraded":"2024-05-22T22:59:53.635743+0000","last_fullsized":"2024-05-22T22:59:53.635743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:04:40.614853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.954109+0000","last_change":"2024-05-22T22:59:51.953466+0000","last_active":"2024-05-22T22:59:51.954109+0000","last_peered":"2024-05-22T22:59:51.954109+0000","last_clean":"2024-05-22T22:59:51.954109+0000","last_became_active":"2024-05-22T22:59:51.952830+0000","last_became_peered":"2024-05-22T22:59:51.952830+0000","last_unstale":"2024-05-22T22:59:51.954109+0000","last_undegraded":"2024-05-22T22:59:51.954109+0000","last_fullsized":"2024-05-22T22:59:51.954109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:57:27.140822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.635787+0000","last_change":"2024-05-22T22:59:51.962993+0000","last_active":"2024-05-22T22:59:53.635787+0000","last_peered":"2024-05-22T22:59:53.635787+0000","last_clean":"2024-05-22T22:59:53.635787+0000","last_became_active":"2024-05-22T22:59:51.962591+0000","last_became_peered":"2024-05-22T22:59:51.962591+0000","last_unstale":"2024-05-22T22:59:53.635787+0000","last_undegraded":"2024-05-22T22:59:53.635787+0000","last_fullsized":"2024-05-22T22:59:53.635787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:55:50.902537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.636269+0000","last_change":"2024-05-22T22:59:51.956304+0000","last_active":"2024-05-22T22:59:53.636269+0000","last_peered":"2024-05-22T22:59:53.636269+0000","last_clean":"2024-05-22T22:59:53.636269+0000","last_became_active":"2024-05-22T22:59:51.955933+0000","last_became_peered":"2024-05-22T22:59:51.955933+0000","last_unstale":"2024-05-22T22:59:53.636269+0000","last_undegraded":"2024-05-22T22:59:53.636269+0000","last_fullsized":"2024-05-22T22:59:53.636269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:28:17.225851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.636039+0000","last_change":"2024-05-22T22:59:51.946806+0000","last_active":"2024-05-22T22:59:53.636039+0000","last_peered":"2024-05-22T22:59:53.636039+0000","last_clean":"2024-05-22T22:59:53.636039+0000","last_became_active":"2024-05-22T22:59:51.946433+0000","last_became_peered":"2024-05-22T22:59:51.946433+0000","last_unstale":"2024-05-22T22:59:53.636039+0000","last_undegraded":"2024-05-22T22:59:53.636039+0000","last_fullsized":"2024-05-22T22:59:53.636039+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:24:06.013688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:54.133298+0000","last_change":"2024-05-22T22:59:51.956466+0000","last_active":"2024-05-22T22:59:54.133298+0000","last_peered":"2024-05-22T22:59:54.133298+0000","last_clean":"2024-05-22T22:59:54.133298+0000","last_became_active":"2024-05-22T22:59:51.956144+0000","last_became_peered":"2024-05-22T22:59:51.956144+0000","last_unstale":"2024-05-22T22:59:54.133298+0000","last_undegraded":"2024-05-22T22:59:54.133298+0000","last_fullsized":"2024-05-22T22:59:54.133298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:08:28.674824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.635674+0000","last_change":"2024-05-22T22:59:51.955690+0000","last_active":"2024-05-22T22:59:53.635674+0000","last_peered":"2024-05-22T22:59:53.635674+0000","last_clean":"2024-05-22T22:59:53.635674+0000","last_became_active":"2024-05-22T22:59:51.955366+0000","last_became_peered":"2024-05-22T22:59:51.955366+0000","last_unstale":"2024-05-22T22:59:53.635674+0000","last_undegraded":"2024-05-22T22:59:53.635674+0000","last_fullsized":"2024-05-22T22:59:53.635674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:08:15.713039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.636005+0000","last_change":"2024-05-22T22:59:51.961971+0000","last_active":"2024-05-22T22:59:53.636005+0000","last_peered":"2024-05-22T22:59:53.636005+0000","last_clean":"2024-05-22T22:59:53.636005+0000","last_became_active":"2024-05-22T22:59:51.961604+0000","last_became_peered":"2024-05-22T22:59:51.961604+0000","last_unstale":"2024-05-22T22:59:53.636005+0000","last_undegraded":"2024-05-22T22:59:53.636005+0000","last_fullsized":"2024-05-22T22:59:53.636005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:32:14.669631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.636708+0000","last_change":"2024-05-22T22:59:51.948173+0000","last_active":"2024-05-22T22:59:53.636708+0000","last_peered":"2024-05-22T22:59:53.636708+0000","last_clean":"2024-05-22T22:59:53.636708+0000","last_became_active":"2024-05-22T22:59:51.947839+0000","last_became_peered":"2024-05-22T22:59:51.947839+0000","last_unstale":"2024-05-22T22:59:53.636708+0000","last_undegraded":"2024-05-22T22:59:53.636708+0000","last_fullsized":"2024-05-22T22:59:53.636708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:55:07.748445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.946793+0000","last_change":"2024-05-22T22:59:51.955627+0000","last_active":"2024-05-22T22:59:52.946793+0000","last_peered":"2024-05-22T22:59:52.946793+0000","last_clean":"2024-05-22T22:59:52.946793+0000","last_became_active":"2024-05-22T22:59:51.955284+0000","last_became_peered":"2024-05-22T22:59:51.955284+0000","last_unstale":"2024-05-22T22:59:52.946793+0000","last_undegraded":"2024-05-22T22:59:52.946793+0000","last_fullsized":"2024-05-22T22:59:52.946793+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:57:13.029317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.947272+0000","last_change":"2024-05-22T22:59:51.955889+0000","last_active":"2024-05-22T22:59:52.947272+0000","last_peered":"2024-05-22T22:59:52.947272+0000","last_clean":"2024-05-22T22:59:52.947272+0000","last_became_active":"2024-05-22T22:59:51.954930+0000","last_became_peered":"2024-05-22T22:59:51.954930+0000","last_unstale":"2024-05-22T22:59:52.947272+0000","last_undegraded":"2024-05-22T22:59:52.947272+0000","last_fullsized":"2024-05-22T22:59:52.947272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:47.536072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.946395+0000","last_change":"2024-05-22T22:59:51.954148+0000","last_active":"2024-05-22T22:59:52.946395+0000","last_peered":"2024-05-22T22:59:52.946395+0000","last_clean":"2024-05-22T22:59:52.946395+0000","last_became_active":"2024-05-22T22:59:51.952336+0000","last_became_peered":"2024-05-22T22:59:51.952336+0000","last_unstale":"2024-05-22T22:59:52.946395+0000","last_undegraded":"2024-05-22T22:59:52.946395+0000","last_fullsized":"2024-05-22T22:59:52.946395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:07:27.928910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.957844+0000","last_change":"2024-05-22T22:59:51.954300+0000","last_active":"2024-05-22T22:59:51.957844+0000","last_peered":"2024-05-22T22:59:51.957844+0000","last_clean":"2024-05-22T22:59:51.957844+0000","last_became_active":"2024-05-22T22:59:51.953779+0000","last_became_peered":"2024-05-22T22:59:51.953779+0000","last_unstale":"2024-05-22T22:59:51.957844+0000","last_undegraded":"2024-05-22T22:59:51.957844+0000","last_fullsized":"2024-05-22T22:59:51.957844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:32:31.799522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.635780+0000","last_change":"2024-05-22T22:59:51.947860+0000","last_active":"2024-05-22T22:59:53.635780+0000","last_peered":"2024-05-22T22:59:53.635780+0000","last_clean":"2024-05-22T22:59:53.635780+0000","last_became_active":"2024-05-22T22:59:51.947420+0000","last_became_peered":"2024-05-22T22:59:51.947420+0000","last_unstale":"2024-05-22T22:59:53.635780+0000","last_undegraded":"2024-05-22T22:59:53.635780+0000","last_fullsized":"2024-05-22T22:59:53.635780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:20:40.629285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.635956+0000","last_change":"2024-05-22T22:59:51.957867+0000","last_active":"2024-05-22T22:59:53.635956+0000","last_peered":"2024-05-22T22:59:53.635956+0000","last_clean":"2024-05-22T22:59:53.635956+0000","last_became_active":"2024-05-22T22:59:51.957522+0000","last_became_peered":"2024-05-22T22:59:51.957522+0000","last_unstale":"2024-05-22T22:59:53.635956+0000","last_undegraded":"2024-05-22T22:59:53.635956+0000","last_fullsized":"2024-05-22T22:59:53.635956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:41:18.631970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280555,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":38990,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":27,"num_write":56,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":61480,"data_compressed":1857,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":435309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612628,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":1055452,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776153,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":444461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037789,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":921600,"data_stored":435913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":1025677,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":443879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":426232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":917504,"data_stored":435399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":452124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":444100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920819,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":929792,"data_stored":443901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":987136,"data_stored":1034429,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":20928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":38386,"data_compressed":1857,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-22T22:59:58.303 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T22:59:58.448 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.444+0000 7fa69961c640 1 -- 172.21.15.84:0/3486085285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa694101a90 msgr2=0x7fa6941083b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:58.448 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.444+0000 7fa69961c640 1 --2- 172.21.15.84:0/3486085285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa694101a90 0x7fa6941083b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa68c009ca0 tx=0x7fa68c02a8c0 comp rx=0 tx=0).stop 2024-05-22T22:59:58.448 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.444+0000 7fa69961c640 1 -- 172.21.15.84:0/3486085285 shutdown_connections 2024-05-22T22:59:58.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.444+0000 7fa69961c640 1 --2- 172.21.15.84:0/3486085285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa694101a90 0x7fa6941083b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:58.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.444+0000 7fa69961c640 1 --2- 172.21.15.84:0/3486085285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6941010d0 0x7fa694101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:58.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.444+0000 7fa69961c640 1 --2- 172.21.15.84:0/3486085285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6940fff00 0x7fa694100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:58.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.444+0000 7fa69961c640 1 -- 172.21.15.84:0/3486085285 >> 172.21.15.84:0/3486085285 conn(0x7fa6940fb4e0 msgr2=0x7fa6940fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:58.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.444+0000 7fa69961c640 1 -- 172.21.15.84:0/3486085285 shutdown_connections 2024-05-22T22:59:58.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.444+0000 7fa69961c640 1 -- 172.21.15.84:0/3486085285 wait complete. 2024-05-22T22:59:58.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.444+0000 7fa69961c640 1 Processor -- start 2024-05-22T22:59:58.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.444+0000 7fa69961c640 1 -- start start 2024-05-22T22:59:58.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.445+0000 7fa69961c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6940fff00 0x7fa694109690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:58.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.445+0000 7fa69961c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6941010d0 0x7fa69419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:58.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.445+0000 7fa69961c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa694101a90 0x7fa6941a1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:58.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.445+0000 7fa69961c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa69410c440 con 0x7fa694101a90 2024-05-22T22:59:58.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.445+0000 7fa69961c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa69410c2c0 con 0x7fa6940fff00 2024-05-22T22:59:58.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.445+0000 7fa69961c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa69410c5c0 con 0x7fa6941010d0 2024-05-22T22:59:58.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.445+0000 7fa6937fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa694101a90 0x7fa6941a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:58.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.445+0000 7fa6937fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa694101a90 0x7fa6941a1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41964/0 (socket says 172.21.15.84:41964) 2024-05-22T22:59:58.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.445+0000 7fa6937fe640 1 -- 172.21.15.84:0/740865173 learned_addr learned my addr 172.21.15.84:0/740865173 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T22:59:58.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.445+0000 7fa68bfff640 1 --2- 172.21.15.84:0/740865173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6941010d0 0x7fa69419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:58.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.445+0000 7fa692ffd640 1 --2- 172.21.15.84:0/740865173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6940fff00 0x7fa694109690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:58.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.445+0000 7fa6937fe640 1 -- 172.21.15.84:0/740865173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6941010d0 msgr2=0x7fa69419ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:58.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.445+0000 7fa6937fe640 1 --2- 172.21.15.84:0/740865173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6941010d0 0x7fa69419ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:58.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.445+0000 7fa6937fe640 1 -- 172.21.15.84:0/740865173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6940fff00 msgr2=0x7fa694109690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:58.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.445+0000 7fa6937fe640 1 --2- 172.21.15.84:0/740865173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6940fff00 0x7fa694109690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:58.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.445+0000 7fa6937fe640 1 -- 172.21.15.84:0/740865173 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6941a19f0 con 0x7fa694101a90 2024-05-22T22:59:58.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.445+0000 7fa692ffd640 1 --2- 172.21.15.84:0/740865173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6940fff00 0x7fa694109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T22:59:58.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.445+0000 7fa6937fe640 1 --2- 172.21.15.84:0/740865173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa694101a90 0x7fa6941a1200 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa68c031ae0 tx=0x7fa68c031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:58.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.445+0000 7fa690ff9640 1 -- 172.21.15.84:0/740865173 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c003c30 con 0x7fa694101a90 2024-05-22T22:59:58.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.445+0000 7fa690ff9640 1 -- 172.21.15.84:0/740865173 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa68c003dd0 con 0x7fa694101a90 2024-05-22T22:59:58.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.445+0000 7fa69961c640 1 -- 172.21.15.84:0/740865173 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6941a1c20 con 0x7fa694101a90 2024-05-22T22:59:58.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.445+0000 7fa690ff9640 1 -- 172.21.15.84:0/740865173 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c02fcb0 con 0x7fa694101a90 2024-05-22T22:59:58.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.445+0000 7fa69961c640 1 -- 172.21.15.84:0/740865173 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6941a2170 con 0x7fa694101a90 2024-05-22T22:59:58.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.447+0000 7fa69961c640 1 -- 172.21.15.84:0/740865173 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa694068ae0 con 0x7fa694101a90 2024-05-22T22:59:58.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.447+0000 7fa690ff9640 1 -- 172.21.15.84:0/740865173 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa68c004050 con 0x7fa694101a90 2024-05-22T22:59:58.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.448+0000 7fa690ff9640 1 --2- 172.21.15.84:0/740865173 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa6800a2c50 0x7fa6800a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T22:59:58.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.448+0000 7fa692ffd640 1 --2- 172.21.15.84:0/740865173 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa6800a2c50 0x7fa6800a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T22:59:58.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.448+0000 7fa690ff9640 1 -- 172.21.15.84:0/740865173 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa68c0f0280 con 0x7fa694101a90 2024-05-22T22:59:58.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.448+0000 7fa692ffd640 1 --2- 172.21.15.84:0/740865173 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa6800a2c50 0x7fa6800a5110 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa6940ffc20 tx=0x7fa684005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T22:59:58.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.452+0000 7fa690ff9640 1 -- 172.21.15.84:0/740865173 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa68c0bd340 con 0x7fa694101a90 2024-05-22T22:59:58.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.633+0000 7fa69961c640 1 -- 172.21.15.84:0/740865173 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa694061cb0 con 0x7fa6800a2c50 2024-05-22T22:59:58.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.641+0000 7fa690ff9640 1 -- 172.21.15.84:0/740865173 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258892 (secure 0 0 0) 0x7fa694061cb0 con 0x7fa6800a2c50 2024-05-22T22:59:58.644 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T22:59:58.645 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2024-05-22T22:59:58.647 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.645+0000 7fa69961c640 1 -- 172.21.15.84:0/740865173 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa6800a2c50 msgr2=0x7fa6800a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:58.647 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.645+0000 7fa69961c640 1 --2- 172.21.15.84:0/740865173 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa6800a2c50 0x7fa6800a5110 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa6940ffc20 tx=0x7fa684005d30 comp rx=0 tx=0).stop 2024-05-22T22:59:58.647 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.645+0000 7fa69961c640 1 -- 172.21.15.84:0/740865173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa694101a90 msgr2=0x7fa6941a1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T22:59:58.647 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.645+0000 7fa69961c640 1 --2- 172.21.15.84:0/740865173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa694101a90 0x7fa6941a1200 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa68c031ae0 tx=0x7fa68c031ee0 comp rx=0 tx=0).stop 2024-05-22T22:59:58.647 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.645+0000 7fa69961c640 1 -- 172.21.15.84:0/740865173 shutdown_connections 2024-05-22T22:59:58.647 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.645+0000 7fa69961c640 1 --2- 172.21.15.84:0/740865173 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa6800a2c50 0x7fa6800a5110 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:58.647 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.645+0000 7fa69961c640 1 --2- 172.21.15.84:0/740865173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa694101a90 0x7fa6941a1200 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:58.647 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.645+0000 7fa69961c640 1 --2- 172.21.15.84:0/740865173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6941010d0 0x7fa69419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:58.647 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.645+0000 7fa69961c640 1 --2- 172.21.15.84:0/740865173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6940fff00 0x7fa694109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T22:59:58.648 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.645+0000 7fa69961c640 1 -- 172.21.15.84:0/740865173 >> 172.21.15.84:0/740865173 conn(0x7fa6940fb4e0 msgr2=0x7fa694104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T22:59:58.648 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.645+0000 7fa69961c640 1 -- 172.21.15.84:0/740865173 shutdown_connections 2024-05-22T22:59:58.648 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T22:59:58.645+0000 7fa69961c640 1 -- 172.21.15.84:0/740865173 wait complete. 2024-05-22T22:59:58.660 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-05-22T22:59:57.494429+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":629475,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":148,"num_read_kb":146,"num_write":238,"num_write_kb":2991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":188,"ondisk_log_size":188,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":161,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376892,"kb_used_data":11028,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564932,"statfs":{"total":1151940427776,"available":1151554490368,"internally_reserved":0,"allocated":11292672,"data_stored":7076876,"data_compressed":107052,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":39107,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":12,"num_read_kb":27,"num_write":57,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001356"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.636409+0000","last_change":"2024-05-22T22:59:51.951625+0000","last_active":"2024-05-22T22:59:53.636409+0000","last_peered":"2024-05-22T22:59:53.636409+0000","last_clean":"2024-05-22T22:59:53.636409+0000","last_became_active":"2024-05-22T22:59:51.951225+0000","last_became_peered":"2024-05-22T22:59:51.951225+0000","last_unstale":"2024-05-22T22:59:53.636409+0000","last_undegraded":"2024-05-22T22:59:53.636409+0000","last_fullsized":"2024-05-22T22:59:53.636409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:53:32.616083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:54.133512+0000","last_change":"2024-05-22T22:59:51.956925+0000","last_active":"2024-05-22T22:59:54.133512+0000","last_peered":"2024-05-22T22:59:54.133512+0000","last_clean":"2024-05-22T22:59:54.133512+0000","last_became_active":"2024-05-22T22:59:51.956557+0000","last_became_peered":"2024-05-22T22:59:51.956557+0000","last_unstale":"2024-05-22T22:59:54.133512+0000","last_undegraded":"2024-05-22T22:59:54.133512+0000","last_fullsized":"2024-05-22T22:59:54.133512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:18:14.609682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:54.133460+0000","last_change":"2024-05-22T22:59:51.951981+0000","last_active":"2024-05-22T22:59:54.133460+0000","last_peered":"2024-05-22T22:59:54.133460+0000","last_clean":"2024-05-22T22:59:54.133460+0000","last_became_active":"2024-05-22T22:59:51.951522+0000","last_became_peered":"2024-05-22T22:59:51.951522+0000","last_unstale":"2024-05-22T22:59:54.133460+0000","last_undegraded":"2024-05-22T22:59:54.133460+0000","last_fullsized":"2024-05-22T22:59:54.133460+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:56:23.357290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.945227+0000","last_change":"2024-05-22T22:59:51.966141+0000","last_active":"2024-05-22T22:59:52.945227+0000","last_peered":"2024-05-22T22:59:52.945227+0000","last_clean":"2024-05-22T22:59:52.945227+0000","last_became_active":"2024-05-22T22:59:51.965798+0000","last_became_peered":"2024-05-22T22:59:51.965798+0000","last_unstale":"2024-05-22T22:59:52.945227+0000","last_undegraded":"2024-05-22T22:59:52.945227+0000","last_fullsized":"2024-05-22T22:59:52.945227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:23:58.282078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.951236+0000","last_change":"2024-05-22T22:59:51.950425+0000","last_active":"2024-05-22T22:59:51.951236+0000","last_peered":"2024-05-22T22:59:51.951236+0000","last_clean":"2024-05-22T22:59:51.951236+0000","last_became_active":"2024-05-22T22:59:51.950071+0000","last_became_peered":"2024-05-22T22:59:51.950071+0000","last_unstale":"2024-05-22T22:59:51.951236+0000","last_undegraded":"2024-05-22T22:59:51.951236+0000","last_fullsized":"2024-05-22T22:59:51.951236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:40:13.357763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.637229+0000","last_change":"2024-05-22T22:59:51.965131+0000","last_active":"2024-05-22T22:59:53.637229+0000","last_peered":"2024-05-22T22:59:53.637229+0000","last_clean":"2024-05-22T22:59:53.637229+0000","last_became_active":"2024-05-22T22:59:51.964541+0000","last_became_peered":"2024-05-22T22:59:51.964541+0000","last_unstale":"2024-05-22T22:59:53.637229+0000","last_undegraded":"2024-05-22T22:59:53.637229+0000","last_fullsized":"2024-05-22T22:59:53.637229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:42:59.295475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.960043+0000","last_change":"2024-05-22T22:59:51.959583+0000","last_active":"2024-05-22T22:59:51.960043+0000","last_peered":"2024-05-22T22:59:51.960043+0000","last_clean":"2024-05-22T22:59:51.960043+0000","last_became_active":"2024-05-22T22:59:51.959260+0000","last_became_peered":"2024-05-22T22:59:51.959260+0000","last_unstale":"2024-05-22T22:59:51.960043+0000","last_undegraded":"2024-05-22T22:59:51.960043+0000","last_fullsized":"2024-05-22T22:59:51.960043+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:58:59.597179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.946371+0000","last_change":"2024-05-22T22:59:51.968418+0000","last_active":"2024-05-22T22:59:52.946371+0000","last_peered":"2024-05-22T22:59:52.946371+0000","last_clean":"2024-05-22T22:59:52.946371+0000","last_became_active":"2024-05-22T22:59:51.968061+0000","last_became_peered":"2024-05-22T22:59:51.968061+0000","last_unstale":"2024-05-22T22:59:52.946371+0000","last_undegraded":"2024-05-22T22:59:52.946371+0000","last_fullsized":"2024-05-22T22:59:52.946371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:35:26.825313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.958270+0000","last_change":"2024-05-22T22:59:51.956954+0000","last_active":"2024-05-22T22:59:51.958270+0000","last_peered":"2024-05-22T22:59:51.958270+0000","last_clean":"2024-05-22T22:59:51.958270+0000","last_became_active":"2024-05-22T22:59:51.956570+0000","last_became_peered":"2024-05-22T22:59:51.956570+0000","last_unstale":"2024-05-22T22:59:51.958270+0000","last_undegraded":"2024-05-22T22:59:51.958270+0000","last_fullsized":"2024-05-22T22:59:51.958270+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:52:27.512572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.637433+0000","last_change":"2024-05-22T22:59:51.962371+0000","last_active":"2024-05-22T22:59:53.637433+0000","last_peered":"2024-05-22T22:59:53.637433+0000","last_clean":"2024-05-22T22:59:53.637433+0000","last_became_active":"2024-05-22T22:59:51.962036+0000","last_became_peered":"2024-05-22T22:59:51.962036+0000","last_unstale":"2024-05-22T22:59:53.637433+0000","last_undegraded":"2024-05-22T22:59:53.637433+0000","last_fullsized":"2024-05-22T22:59:53.637433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:27:21.745563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.954166+0000","last_change":"2024-05-22T22:59:51.952446+0000","last_active":"2024-05-22T22:59:51.954166+0000","last_peered":"2024-05-22T22:59:51.954166+0000","last_clean":"2024-05-22T22:59:51.954166+0000","last_became_active":"2024-05-22T22:59:51.952115+0000","last_became_peered":"2024-05-22T22:59:51.952115+0000","last_unstale":"2024-05-22T22:59:51.954166+0000","last_undegraded":"2024-05-22T22:59:51.954166+0000","last_fullsized":"2024-05-22T22:59:51.954166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:44:25.285051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.945031+0000","last_change":"2024-05-22T22:59:51.968457+0000","last_active":"2024-05-22T22:59:52.945031+0000","last_peered":"2024-05-22T22:59:52.945031+0000","last_clean":"2024-05-22T22:59:52.945031+0000","last_became_active":"2024-05-22T22:59:51.968070+0000","last_became_peered":"2024-05-22T22:59:51.968070+0000","last_unstale":"2024-05-22T22:59:52.945031+0000","last_undegraded":"2024-05-22T22:59:52.945031+0000","last_fullsized":"2024-05-22T22:59:52.945031+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:07:31.595119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.635747+0000","last_change":"2024-05-22T22:59:51.957357+0000","last_active":"2024-05-22T22:59:53.635747+0000","last_peered":"2024-05-22T22:59:53.635747+0000","last_clean":"2024-05-22T22:59:53.635747+0000","last_became_active":"2024-05-22T22:59:51.957013+0000","last_became_peered":"2024-05-22T22:59:51.957013+0000","last_unstale":"2024-05-22T22:59:53.635747+0000","last_undegraded":"2024-05-22T22:59:53.635747+0000","last_fullsized":"2024-05-22T22:59:53.635747+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:38:48.118178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:54.133210+0000","last_change":"2024-05-22T22:59:51.957084+0000","last_active":"2024-05-22T22:59:54.133210+0000","last_peered":"2024-05-22T22:59:54.133210+0000","last_clean":"2024-05-22T22:59:54.133210+0000","last_became_active":"2024-05-22T22:59:51.956787+0000","last_became_peered":"2024-05-22T22:59:51.956787+0000","last_unstale":"2024-05-22T22:59:54.133210+0000","last_undegraded":"2024-05-22T22:59:54.133210+0000","last_fullsized":"2024-05-22T22:59:54.133210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:30:40.578286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:54.133170+0000","last_change":"2024-05-22T22:59:51.956375+0000","last_active":"2024-05-22T22:59:54.133170+0000","last_peered":"2024-05-22T22:59:54.133170+0000","last_clean":"2024-05-22T22:59:54.133170+0000","last_became_active":"2024-05-22T22:59:51.956012+0000","last_became_peered":"2024-05-22T22:59:51.956012+0000","last_unstale":"2024-05-22T22:59:54.133170+0000","last_undegraded":"2024-05-22T22:59:54.133170+0000","last_fullsized":"2024-05-22T22:59:54.133170+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:33:49.920327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.952173+0000","last_change":"2024-05-22T22:59:51.950287+0000","last_active":"2024-05-22T22:59:51.952173+0000","last_peered":"2024-05-22T22:59:51.952173+0000","last_clean":"2024-05-22T22:59:51.952173+0000","last_became_active":"2024-05-22T22:59:51.949962+0000","last_became_peered":"2024-05-22T22:59:51.949962+0000","last_unstale":"2024-05-22T22:59:51.952173+0000","last_undegraded":"2024-05-22T22:59:51.952173+0000","last_fullsized":"2024-05-22T22:59:51.952173+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:30:43.175576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.636596+0000","last_change":"2024-05-22T22:59:51.952523+0000","last_active":"2024-05-22T22:59:53.636596+0000","last_peered":"2024-05-22T22:59:53.636596+0000","last_clean":"2024-05-22T22:59:53.636596+0000","last_became_active":"2024-05-22T22:59:51.952152+0000","last_became_peered":"2024-05-22T22:59:51.952152+0000","last_unstale":"2024-05-22T22:59:53.636596+0000","last_undegraded":"2024-05-22T22:59:53.636596+0000","last_fullsized":"2024-05-22T22:59:53.636596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:33:01.332667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.591951+0000","last_change":"2024-05-22T22:59:51.960919+0000","last_active":"2024-05-22T22:59:53.591951+0000","last_peered":"2024-05-22T22:59:53.591951+0000","last_clean":"2024-05-22T22:59:53.591951+0000","last_became_active":"2024-05-22T22:59:51.960603+0000","last_became_peered":"2024-05-22T22:59:51.960603+0000","last_unstale":"2024-05-22T22:59:53.591951+0000","last_undegraded":"2024-05-22T22:59:53.591951+0000","last_fullsized":"2024-05-22T22:59:53.591951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:28:05.977612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.636689+0000","last_change":"2024-05-22T22:59:51.949186+0000","last_active":"2024-05-22T22:59:53.636689+0000","last_peered":"2024-05-22T22:59:53.636689+0000","last_clean":"2024-05-22T22:59:53.636689+0000","last_became_active":"2024-05-22T22:59:51.948819+0000","last_became_peered":"2024-05-22T22:59:51.948819+0000","last_unstale":"2024-05-22T22:59:53.636689+0000","last_undegraded":"2024-05-22T22:59:53.636689+0000","last_fullsized":"2024-05-22T22:59:53.636689+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:32:16.153838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:54.608966+0000","last_change":"2024-05-22T22:59:51.592520+0000","last_active":"2024-05-22T22:59:54.608966+0000","last_peered":"2024-05-22T22:59:54.608966+0000","last_clean":"2024-05-22T22:59:54.608966+0000","last_became_active":"2024-05-22T22:59:51.592201+0000","last_became_peered":"2024-05-22T22:59:51.592201+0000","last_unstale":"2024-05-22T22:59:54.608966+0000","last_undegraded":"2024-05-22T22:59:54.608966+0000","last_fullsized":"2024-05-22T22:59:54.608966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:45:06.500085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.946551+0000","last_change":"2024-05-22T22:59:51.963169+0000","last_active":"2024-05-22T22:59:52.946551+0000","last_peered":"2024-05-22T22:59:52.946551+0000","last_clean":"2024-05-22T22:59:52.946551+0000","last_became_active":"2024-05-22T22:59:51.962875+0000","last_became_peered":"2024-05-22T22:59:51.962875+0000","last_unstale":"2024-05-22T22:59:52.946551+0000","last_undegraded":"2024-05-22T22:59:52.946551+0000","last_fullsized":"2024-05-22T22:59:52.946551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:03:24.092690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.946833+0000","last_change":"2024-05-22T22:59:51.595383+0000","last_active":"2024-05-22T22:59:52.946833+0000","last_peered":"2024-05-22T22:59:52.946833+0000","last_clean":"2024-05-22T22:59:52.946833+0000","last_became_active":"2024-05-22T22:59:51.595076+0000","last_became_peered":"2024-05-22T22:59:51.595076+0000","last_unstale":"2024-05-22T22:59:52.946833+0000","last_undegraded":"2024-05-22T22:59:52.946833+0000","last_fullsized":"2024-05-22T22:59:52.946833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:29:57.205196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:54.133623+0000","last_change":"2024-05-22T22:59:51.952447+0000","last_active":"2024-05-22T22:59:54.133623+0000","last_peered":"2024-05-22T22:59:54.133623+0000","last_clean":"2024-05-22T22:59:54.133623+0000","last_became_active":"2024-05-22T22:59:51.951927+0000","last_became_peered":"2024-05-22T22:59:51.951927+0000","last_unstale":"2024-05-22T22:59:54.133623+0000","last_undegraded":"2024-05-22T22:59:54.133623+0000","last_fullsized":"2024-05-22T22:59:54.133623+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:14:59.161404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.596672+0000","last_change":"2024-05-22T22:59:51.597705+0000","last_active":"2024-05-22T22:59:52.596672+0000","last_peered":"2024-05-22T22:59:52.596672+0000","last_clean":"2024-05-22T22:59:52.596672+0000","last_became_active":"2024-05-22T22:59:51.597319+0000","last_became_peered":"2024-05-22T22:59:51.597319+0000","last_unstale":"2024-05-22T22:59:52.596672+0000","last_undegraded":"2024-05-22T22:59:52.596672+0000","last_fullsized":"2024-05-22T22:59:52.596672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:59:59.307499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.957797+0000","last_change":"2024-05-22T22:59:51.955477+0000","last_active":"2024-05-22T22:59:51.957797+0000","last_peered":"2024-05-22T22:59:51.957797+0000","last_clean":"2024-05-22T22:59:51.957797+0000","last_became_active":"2024-05-22T22:59:51.953608+0000","last_became_peered":"2024-05-22T22:59:51.953608+0000","last_unstale":"2024-05-22T22:59:51.957797+0000","last_undegraded":"2024-05-22T22:59:51.957797+0000","last_fullsized":"2024-05-22T22:59:51.957797+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:37:35.939922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.598912+0000","last_change":"2024-05-22T22:59:51.595417+0000","last_active":"2024-05-22T22:59:52.598912+0000","last_peered":"2024-05-22T22:59:52.598912+0000","last_clean":"2024-05-22T22:59:52.598912+0000","last_became_active":"2024-05-22T22:59:51.595051+0000","last_became_peered":"2024-05-22T22:59:51.595051+0000","last_unstale":"2024-05-22T22:59:52.598912+0000","last_undegraded":"2024-05-22T22:59:52.598912+0000","last_fullsized":"2024-05-22T22:59:52.598912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:13:47.936348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.947722+0000","last_change":"2024-05-22T22:59:51.958471+0000","last_active":"2024-05-22T22:59:52.947722+0000","last_peered":"2024-05-22T22:59:52.947722+0000","last_clean":"2024-05-22T22:59:52.947722+0000","last_became_active":"2024-05-22T22:59:51.958017+0000","last_became_peered":"2024-05-22T22:59:51.958017+0000","last_unstale":"2024-05-22T22:59:52.947722+0000","last_undegraded":"2024-05-22T22:59:52.947722+0000","last_fullsized":"2024-05-22T22:59:52.947722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:42:52.196892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.947388+0000","last_change":"2024-05-22T22:59:51.591909+0000","last_active":"2024-05-22T22:59:52.947388+0000","last_peered":"2024-05-22T22:59:52.947388+0000","last_clean":"2024-05-22T22:59:52.947388+0000","last_became_active":"2024-05-22T22:59:51.591531+0000","last_became_peered":"2024-05-22T22:59:51.591531+0000","last_unstale":"2024-05-22T22:59:52.947388+0000","last_undegraded":"2024-05-22T22:59:52.947388+0000","last_fullsized":"2024-05-22T22:59:52.947388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:07:22.997156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.962383+0000","last_change":"2024-05-22T22:59:51.961444+0000","last_active":"2024-05-22T22:59:51.962383+0000","last_peered":"2024-05-22T22:59:51.962383+0000","last_clean":"2024-05-22T22:59:51.962383+0000","last_became_active":"2024-05-22T22:59:51.961088+0000","last_became_peered":"2024-05-22T22:59:51.961088+0000","last_unstale":"2024-05-22T22:59:51.962383+0000","last_undegraded":"2024-05-22T22:59:51.962383+0000","last_fullsized":"2024-05-22T22:59:51.962383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:49:29.238180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.636949+0000","last_change":"2024-05-22T22:59:51.590124+0000","last_active":"2024-05-22T22:59:53.636949+0000","last_peered":"2024-05-22T22:59:53.636949+0000","last_clean":"2024-05-22T22:59:53.636949+0000","last_became_active":"2024-05-22T22:59:51.589661+0000","last_became_peered":"2024-05-22T22:59:51.589661+0000","last_unstale":"2024-05-22T22:59:53.636949+0000","last_undegraded":"2024-05-22T22:59:53.636949+0000","last_fullsized":"2024-05-22T22:59:53.636949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:33:14.293487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.947664+0000","last_change":"2024-05-22T22:59:51.958574+0000","last_active":"2024-05-22T22:59:52.947664+0000","last_peered":"2024-05-22T22:59:52.947664+0000","last_clean":"2024-05-22T22:59:52.947664+0000","last_became_active":"2024-05-22T22:59:51.958252+0000","last_became_peered":"2024-05-22T22:59:51.958252+0000","last_unstale":"2024-05-22T22:59:52.947664+0000","last_undegraded":"2024-05-22T22:59:52.947664+0000","last_fullsized":"2024-05-22T22:59:52.947664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:18:43.192032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.635528+0000","last_change":"2024-05-22T22:59:51.595510+0000","last_active":"2024-05-22T22:59:53.635528+0000","last_peered":"2024-05-22T22:59:53.635528+0000","last_clean":"2024-05-22T22:59:53.635528+0000","last_became_active":"2024-05-22T22:59:51.595208+0000","last_became_peered":"2024-05-22T22:59:51.595208+0000","last_unstale":"2024-05-22T22:59:53.635528+0000","last_undegraded":"2024-05-22T22:59:53.635528+0000","last_fullsized":"2024-05-22T22:59:53.635528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:47:46.957965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.946501+0000","last_change":"2024-05-22T22:59:51.950684+0000","last_active":"2024-05-22T22:59:52.946501+0000","last_peered":"2024-05-22T22:59:52.946501+0000","last_clean":"2024-05-22T22:59:52.946501+0000","last_became_active":"2024-05-22T22:59:51.950380+0000","last_became_peered":"2024-05-22T22:59:51.950380+0000","last_unstale":"2024-05-22T22:59:52.946501+0000","last_undegraded":"2024-05-22T22:59:52.946501+0000","last_fullsized":"2024-05-22T22:59:52.946501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:58:02.035245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.947092+0000","last_change":"2024-05-22T22:59:51.592958+0000","last_active":"2024-05-22T22:59:52.947092+0000","last_peered":"2024-05-22T22:59:52.947092+0000","last_clean":"2024-05-22T22:59:52.947092+0000","last_became_active":"2024-05-22T22:59:51.592586+0000","last_became_peered":"2024-05-22T22:59:51.592586+0000","last_unstale":"2024-05-22T22:59:52.947092+0000","last_undegraded":"2024-05-22T22:59:52.947092+0000","last_fullsized":"2024-05-22T22:59:52.947092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:44:43.859724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.591688+0000","last_change":"2024-05-22T22:59:51.960863+0000","last_active":"2024-05-22T22:59:53.591688+0000","last_peered":"2024-05-22T22:59:53.591688+0000","last_clean":"2024-05-22T22:59:53.591688+0000","last_became_active":"2024-05-22T22:59:51.960537+0000","last_became_peered":"2024-05-22T22:59:51.960537+0000","last_unstale":"2024-05-22T22:59:53.591688+0000","last_undegraded":"2024-05-22T22:59:53.591688+0000","last_fullsized":"2024-05-22T22:59:53.591688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:08:19.735397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.637090+0000","last_change":"2024-05-22T22:59:51.596150+0000","last_active":"2024-05-22T22:59:53.637090+0000","last_peered":"2024-05-22T22:59:53.637090+0000","last_clean":"2024-05-22T22:59:53.637090+0000","last_became_active":"2024-05-22T22:59:51.595791+0000","last_became_peered":"2024-05-22T22:59:51.595791+0000","last_unstale":"2024-05-22T22:59:53.637090+0000","last_undegraded":"2024-05-22T22:59:53.637090+0000","last_fullsized":"2024-05-22T22:59:53.637090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:47:46.144045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.952749+0000","last_change":"2024-05-22T22:59:51.951907+0000","last_active":"2024-05-22T22:59:51.952749+0000","last_peered":"2024-05-22T22:59:51.952749+0000","last_clean":"2024-05-22T22:59:51.952749+0000","last_became_active":"2024-05-22T22:59:51.951264+0000","last_became_peered":"2024-05-22T22:59:51.951264+0000","last_unstale":"2024-05-22T22:59:51.952749+0000","last_undegraded":"2024-05-22T22:59:51.952749+0000","last_fullsized":"2024-05-22T22:59:51.952749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:55:41.743473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.946008+0000","last_change":"2024-05-22T22:59:51.594883+0000","last_active":"2024-05-22T22:59:51.946008+0000","last_peered":"2024-05-22T22:59:51.946008+0000","last_clean":"2024-05-22T22:59:51.946008+0000","last_became_active":"2024-05-22T22:59:51.594458+0000","last_became_peered":"2024-05-22T22:59:51.594458+0000","last_unstale":"2024-05-22T22:59:51.946008+0000","last_undegraded":"2024-05-22T22:59:51.946008+0000","last_fullsized":"2024-05-22T22:59:51.946008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:39:27.366961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:54.133556+0000","last_change":"2024-05-22T22:59:51.958180+0000","last_active":"2024-05-22T22:59:54.133556+0000","last_peered":"2024-05-22T22:59:54.133556+0000","last_clean":"2024-05-22T22:59:54.133556+0000","last_became_active":"2024-05-22T22:59:51.957877+0000","last_became_peered":"2024-05-22T22:59:51.957877+0000","last_unstale":"2024-05-22T22:59:54.133556+0000","last_undegraded":"2024-05-22T22:59:54.133556+0000","last_fullsized":"2024-05-22T22:59:54.133556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:33:42.478340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.947638+0000","last_change":"2024-05-22T22:59:51.592953+0000","last_active":"2024-05-22T22:59:52.947638+0000","last_peered":"2024-05-22T22:59:52.947638+0000","last_clean":"2024-05-22T22:59:52.947638+0000","last_became_active":"2024-05-22T22:59:51.592525+0000","last_became_peered":"2024-05-22T22:59:51.592525+0000","last_unstale":"2024-05-22T22:59:52.947638+0000","last_undegraded":"2024-05-22T22:59:52.947638+0000","last_fullsized":"2024-05-22T22:59:52.947638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:18:03.512123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.963140+0000","last_change":"2024-05-22T22:59:51.959510+0000","last_active":"2024-05-22T22:59:51.963140+0000","last_peered":"2024-05-22T22:59:51.963140+0000","last_clean":"2024-05-22T22:59:51.963140+0000","last_became_active":"2024-05-22T22:59:51.959197+0000","last_became_peered":"2024-05-22T22:59:51.959197+0000","last_unstale":"2024-05-22T22:59:51.963140+0000","last_undegraded":"2024-05-22T22:59:51.963140+0000","last_fullsized":"2024-05-22T22:59:51.963140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:49:07.447380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'17","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:56.993935+0000","last_change":"2024-05-22T22:59:51.593430+0000","last_active":"2024-05-22T22:59:56.993935+0000","last_peered":"2024-05-22T22:59:56.993935+0000","last_clean":"2024-05-22T22:59:56.993935+0000","last_became_active":"2024-05-22T22:59:51.592541+0000","last_became_peered":"2024-05-22T22:59:51.592541+0000","last_unstale":"2024-05-22T22:59:56.993935+0000","last_undegraded":"2024-05-22T22:59:56.993935+0000","last_fullsized":"2024-05-22T22:59:56.993935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:24:26.709211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38192,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":24,"num_write":17,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.953141+0000","last_change":"2024-05-22T22:59:51.952486+0000","last_active":"2024-05-22T22:59:51.953141+0000","last_peered":"2024-05-22T22:59:51.953141+0000","last_clean":"2024-05-22T22:59:51.953141+0000","last_became_active":"2024-05-22T22:59:51.952096+0000","last_became_peered":"2024-05-22T22:59:51.952096+0000","last_unstale":"2024-05-22T22:59:51.953141+0000","last_undegraded":"2024-05-22T22:59:51.953141+0000","last_fullsized":"2024-05-22T22:59:51.953141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:07:37.260966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.636290+0000","last_change":"2024-05-22T22:59:51.595446+0000","last_active":"2024-05-22T22:59:53.636290+0000","last_peered":"2024-05-22T22:59:53.636290+0000","last_clean":"2024-05-22T22:59:53.636290+0000","last_became_active":"2024-05-22T22:59:51.595097+0000","last_became_peered":"2024-05-22T22:59:51.595097+0000","last_unstale":"2024-05-22T22:59:53.636290+0000","last_undegraded":"2024-05-22T22:59:53.636290+0000","last_fullsized":"2024-05-22T22:59:53.636290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:03:29.722773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.635599+0000","last_change":"2024-05-22T22:59:51.953304+0000","last_active":"2024-05-22T22:59:53.635599+0000","last_peered":"2024-05-22T22:59:53.635599+0000","last_clean":"2024-05-22T22:59:53.635599+0000","last_became_active":"2024-05-22T22:59:51.952974+0000","last_became_peered":"2024-05-22T22:59:51.952974+0000","last_unstale":"2024-05-22T22:59:53.635599+0000","last_undegraded":"2024-05-22T22:59:53.635599+0000","last_fullsized":"2024-05-22T22:59:53.635599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:21:06.116285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.947276+0000","last_change":"2024-05-22T22:59:51.593684+0000","last_active":"2024-05-22T22:59:52.947276+0000","last_peered":"2024-05-22T22:59:52.947276+0000","last_clean":"2024-05-22T22:59:52.947276+0000","last_became_active":"2024-05-22T22:59:51.593335+0000","last_became_peered":"2024-05-22T22:59:51.593335+0000","last_unstale":"2024-05-22T22:59:52.947276+0000","last_undegraded":"2024-05-22T22:59:52.947276+0000","last_fullsized":"2024-05-22T22:59:52.947276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:36:24.405609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.961299+0000","last_change":"2024-05-22T22:59:51.960930+0000","last_active":"2024-05-22T22:59:51.961299+0000","last_peered":"2024-05-22T22:59:51.961299+0000","last_clean":"2024-05-22T22:59:51.961299+0000","last_became_active":"2024-05-22T22:59:51.960571+0000","last_became_peered":"2024-05-22T22:59:51.960571+0000","last_unstale":"2024-05-22T22:59:51.961299+0000","last_undegraded":"2024-05-22T22:59:51.961299+0000","last_fullsized":"2024-05-22T22:59:51.961299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:15:36.869955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.944637+0000","last_change":"2024-05-22T22:59:51.592876+0000","last_active":"2024-05-22T22:59:52.944637+0000","last_peered":"2024-05-22T22:59:52.944637+0000","last_clean":"2024-05-22T22:59:52.944637+0000","last_became_active":"2024-05-22T22:59:51.592488+0000","last_became_peered":"2024-05-22T22:59:51.592488+0000","last_unstale":"2024-05-22T22:59:52.944637+0000","last_undegraded":"2024-05-22T22:59:52.944637+0000","last_fullsized":"2024-05-22T22:59:52.944637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:37:16.517302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.962771+0000","last_change":"2024-05-22T22:59:51.962336+0000","last_active":"2024-05-22T22:59:51.962771+0000","last_peered":"2024-05-22T22:59:51.962771+0000","last_clean":"2024-05-22T22:59:51.962771+0000","last_became_active":"2024-05-22T22:59:51.962002+0000","last_became_peered":"2024-05-22T22:59:51.962002+0000","last_unstale":"2024-05-22T22:59:51.962771+0000","last_undegraded":"2024-05-22T22:59:51.962771+0000","last_fullsized":"2024-05-22T22:59:51.962771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:30:17.276737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.599970+0000","last_change":"2024-05-22T22:59:51.596218+0000","last_active":"2024-05-22T22:59:52.599970+0000","last_peered":"2024-05-22T22:59:52.599970+0000","last_clean":"2024-05-22T22:59:52.599970+0000","last_became_active":"2024-05-22T22:59:51.595138+0000","last_became_peered":"2024-05-22T22:59:51.595138+0000","last_unstale":"2024-05-22T22:59:52.599970+0000","last_undegraded":"2024-05-22T22:59:52.599970+0000","last_fullsized":"2024-05-22T22:59:52.599970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:41:49.748215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.947166+0000","last_change":"2024-05-22T22:59:51.961600+0000","last_active":"2024-05-22T22:59:52.947166+0000","last_peered":"2024-05-22T22:59:52.947166+0000","last_clean":"2024-05-22T22:59:52.947166+0000","last_became_active":"2024-05-22T22:59:51.961286+0000","last_became_peered":"2024-05-22T22:59:51.961286+0000","last_unstale":"2024-05-22T22:59:52.947166+0000","last_undegraded":"2024-05-22T22:59:52.947166+0000","last_fullsized":"2024-05-22T22:59:52.947166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:04:26.992865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.598724+0000","last_change":"2024-05-22T22:59:51.594461+0000","last_active":"2024-05-22T22:59:52.598724+0000","last_peered":"2024-05-22T22:59:52.598724+0000","last_clean":"2024-05-22T22:59:52.598724+0000","last_became_active":"2024-05-22T22:59:51.594128+0000","last_became_peered":"2024-05-22T22:59:51.594128+0000","last_unstale":"2024-05-22T22:59:52.598724+0000","last_undegraded":"2024-05-22T22:59:52.598724+0000","last_fullsized":"2024-05-22T22:59:52.598724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:52:05.095814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.636280+0000","last_change":"2024-05-22T22:59:51.958851+0000","last_active":"2024-05-22T22:59:53.636280+0000","last_peered":"2024-05-22T22:59:53.636280+0000","last_clean":"2024-05-22T22:59:53.636280+0000","last_became_active":"2024-05-22T22:59:51.957412+0000","last_became_peered":"2024-05-22T22:59:51.957412+0000","last_unstale":"2024-05-22T22:59:53.636280+0000","last_undegraded":"2024-05-22T22:59:53.636280+0000","last_fullsized":"2024-05-22T22:59:53.636280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:13:02.761205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.592001+0000","last_change":"2024-05-22T22:59:51.588750+0000","last_active":"2024-05-22T22:59:53.592001+0000","last_peered":"2024-05-22T22:59:53.592001+0000","last_clean":"2024-05-22T22:59:53.592001+0000","last_became_active":"2024-05-22T22:59:51.588364+0000","last_became_peered":"2024-05-22T22:59:51.588364+0000","last_unstale":"2024-05-22T22:59:53.592001+0000","last_undegraded":"2024-05-22T22:59:53.592001+0000","last_fullsized":"2024-05-22T22:59:53.592001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:28:41.605406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.637324+0000","last_change":"2024-05-22T22:59:51.964676+0000","last_active":"2024-05-22T22:59:53.637324+0000","last_peered":"2024-05-22T22:59:53.637324+0000","last_clean":"2024-05-22T22:59:53.637324+0000","last_became_active":"2024-05-22T22:59:51.960472+0000","last_became_peered":"2024-05-22T22:59:51.960472+0000","last_unstale":"2024-05-22T22:59:53.637324+0000","last_undegraded":"2024-05-22T22:59:53.637324+0000","last_fullsized":"2024-05-22T22:59:53.637324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:24:55.042637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.946252+0000","last_change":"2024-05-22T22:59:51.600873+0000","last_active":"2024-05-22T22:59:51.946252+0000","last_peered":"2024-05-22T22:59:51.946252+0000","last_clean":"2024-05-22T22:59:51.946252+0000","last_became_active":"2024-05-22T22:59:51.599785+0000","last_became_peered":"2024-05-22T22:59:51.599785+0000","last_unstale":"2024-05-22T22:59:51.946252+0000","last_undegraded":"2024-05-22T22:59:51.946252+0000","last_fullsized":"2024-05-22T22:59:51.946252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:20:09.590765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.637238+0000","last_change":"2024-05-22T22:59:51.964942+0000","last_active":"2024-05-22T22:59:53.637238+0000","last_peered":"2024-05-22T22:59:53.637238+0000","last_clean":"2024-05-22T22:59:53.637238+0000","last_became_active":"2024-05-22T22:59:51.964333+0000","last_became_peered":"2024-05-22T22:59:51.964333+0000","last_unstale":"2024-05-22T22:59:53.637238+0000","last_undegraded":"2024-05-22T22:59:53.637238+0000","last_fullsized":"2024-05-22T22:59:53.637238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:01:07.215075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.947286+0000","last_change":"2024-05-22T22:59:51.957291+0000","last_active":"2024-05-22T22:59:52.947286+0000","last_peered":"2024-05-22T22:59:52.947286+0000","last_clean":"2024-05-22T22:59:52.947286+0000","last_became_active":"2024-05-22T22:59:51.956987+0000","last_became_peered":"2024-05-22T22:59:51.956987+0000","last_unstale":"2024-05-22T22:59:52.947286+0000","last_undegraded":"2024-05-22T22:59:52.947286+0000","last_fullsized":"2024-05-22T22:59:52.947286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:14:10.420756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.635409+0000","last_change":"2024-05-22T22:59:51.590255+0000","last_active":"2024-05-22T22:59:53.635409+0000","last_peered":"2024-05-22T22:59:53.635409+0000","last_clean":"2024-05-22T22:59:53.635409+0000","last_became_active":"2024-05-22T22:59:51.589261+0000","last_became_peered":"2024-05-22T22:59:51.589261+0000","last_unstale":"2024-05-22T22:59:53.635409+0000","last_undegraded":"2024-05-22T22:59:53.635409+0000","last_fullsized":"2024-05-22T22:59:53.635409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:15:14.171311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.946653+0000","last_change":"2024-05-22T22:59:51.961453+0000","last_active":"2024-05-22T22:59:52.946653+0000","last_peered":"2024-05-22T22:59:52.946653+0000","last_clean":"2024-05-22T22:59:52.946653+0000","last_became_active":"2024-05-22T22:59:51.961082+0000","last_became_peered":"2024-05-22T22:59:51.961082+0000","last_unstale":"2024-05-22T22:59:52.946653+0000","last_undegraded":"2024-05-22T22:59:52.946653+0000","last_fullsized":"2024-05-22T22:59:52.946653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:05:50.310688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.944645+0000","last_change":"2024-05-22T22:59:51.595913+0000","last_active":"2024-05-22T22:59:52.944645+0000","last_peered":"2024-05-22T22:59:52.944645+0000","last_clean":"2024-05-22T22:59:52.944645+0000","last_became_active":"2024-05-22T22:59:51.595561+0000","last_became_peered":"2024-05-22T22:59:51.595561+0000","last_unstale":"2024-05-22T22:59:52.944645+0000","last_undegraded":"2024-05-22T22:59:52.944645+0000","last_fullsized":"2024-05-22T22:59:52.944645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:37:04.701575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.945950+0000","last_change":"2024-05-22T22:58:34.927654+0000","last_active":"2024-05-22T22:59:51.945950+0000","last_peered":"2024-05-22T22:59:51.945950+0000","last_clean":"2024-05-22T22:59:51.945950+0000","last_became_active":"2024-05-22T22:58:34.927090+0000","last_became_peered":"2024-05-22T22:58:34.927090+0000","last_unstale":"2024-05-22T22:59:51.945950+0000","last_undegraded":"2024-05-22T22:59:51.945950+0000","last_fullsized":"2024-05-22T22:59:51.945950+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:55:07.719852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:55:07.719852+0000","last_clean_scrub_stamp":"2024-05-22T22:55:07.719852+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:40:53.763146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.635449+0000","last_change":"2024-05-22T22:59:51.956422+0000","last_active":"2024-05-22T22:59:53.635449+0000","last_peered":"2024-05-22T22:59:53.635449+0000","last_clean":"2024-05-22T22:59:53.635449+0000","last_became_active":"2024-05-22T22:59:51.956127+0000","last_became_peered":"2024-05-22T22:59:51.956127+0000","last_unstale":"2024-05-22T22:59:53.635449+0000","last_undegraded":"2024-05-22T22:59:53.635449+0000","last_fullsized":"2024-05-22T22:59:53.635449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:04:36.955644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.946907+0000","last_change":"2024-05-22T22:59:51.599362+0000","last_active":"2024-05-22T22:59:51.946907+0000","last_peered":"2024-05-22T22:59:51.946907+0000","last_clean":"2024-05-22T22:59:51.946907+0000","last_became_active":"2024-05-22T22:59:51.599027+0000","last_became_peered":"2024-05-22T22:59:51.599027+0000","last_unstale":"2024-05-22T22:59:51.946907+0000","last_undegraded":"2024-05-22T22:59:51.946907+0000","last_fullsized":"2024-05-22T22:59:51.946907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:58:07.923155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.941382+0000","last_change":"2024-05-22T22:59:51.598772+0000","last_active":"2024-05-22T22:59:51.941382+0000","last_peered":"2024-05-22T22:59:51.941382+0000","last_clean":"2024-05-22T22:59:51.941382+0000","last_became_active":"2024-05-22T22:59:51.598192+0000","last_became_peered":"2024-05-22T22:59:51.598192+0000","last_unstale":"2024-05-22T22:59:51.941382+0000","last_undegraded":"2024-05-22T22:59:51.941382+0000","last_fullsized":"2024-05-22T22:59:51.941382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:42:47.069554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.635748+0000","last_change":"2024-05-22T22:59:51.942337+0000","last_active":"2024-05-22T22:59:53.635748+0000","last_peered":"2024-05-22T22:59:53.635748+0000","last_clean":"2024-05-22T22:59:53.635748+0000","last_became_active":"2024-05-22T22:59:51.941939+0000","last_became_peered":"2024-05-22T22:59:51.941939+0000","last_unstale":"2024-05-22T22:59:53.635748+0000","last_undegraded":"2024-05-22T22:59:53.635748+0000","last_fullsized":"2024-05-22T22:59:53.635748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:35:42.360128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.593310+0000","last_change":"2024-05-22T22:59:51.589640+0000","last_active":"2024-05-22T22:59:53.593310+0000","last_peered":"2024-05-22T22:59:53.593310+0000","last_clean":"2024-05-22T22:59:53.593310+0000","last_became_active":"2024-05-22T22:59:51.589310+0000","last_became_peered":"2024-05-22T22:59:51.589310+0000","last_unstale":"2024-05-22T22:59:53.593310+0000","last_undegraded":"2024-05-22T22:59:53.593310+0000","last_fullsized":"2024-05-22T22:59:53.593310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:54:39.182002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.946982+0000","last_change":"2024-05-22T22:59:51.952423+0000","last_active":"2024-05-22T22:59:52.946982+0000","last_peered":"2024-05-22T22:59:52.946982+0000","last_clean":"2024-05-22T22:59:52.946982+0000","last_became_active":"2024-05-22T22:59:51.952015+0000","last_became_peered":"2024-05-22T22:59:51.952015+0000","last_unstale":"2024-05-22T22:59:52.946982+0000","last_undegraded":"2024-05-22T22:59:52.946982+0000","last_fullsized":"2024-05-22T22:59:52.946982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:06:23.705248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.945308+0000","last_change":"2024-05-22T22:59:51.594882+0000","last_active":"2024-05-22T22:59:52.945308+0000","last_peered":"2024-05-22T22:59:52.945308+0000","last_clean":"2024-05-22T22:59:52.945308+0000","last_became_active":"2024-05-22T22:59:51.594552+0000","last_became_peered":"2024-05-22T22:59:51.594552+0000","last_unstale":"2024-05-22T22:59:52.945308+0000","last_undegraded":"2024-05-22T22:59:52.945308+0000","last_fullsized":"2024-05-22T22:59:52.945308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:06:40.771969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:54.133518+0000","last_change":"2024-05-22T22:59:51.956534+0000","last_active":"2024-05-22T22:59:54.133518+0000","last_peered":"2024-05-22T22:59:54.133518+0000","last_clean":"2024-05-22T22:59:54.133518+0000","last_became_active":"2024-05-22T22:59:51.956037+0000","last_became_peered":"2024-05-22T22:59:51.956037+0000","last_unstale":"2024-05-22T22:59:54.133518+0000","last_undegraded":"2024-05-22T22:59:54.133518+0000","last_fullsized":"2024-05-22T22:59:54.133518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:24:15.289570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.937588+0000","last_change":"2024-05-22T22:59:51.592916+0000","last_active":"2024-05-22T22:59:51.937588+0000","last_peered":"2024-05-22T22:59:51.937588+0000","last_clean":"2024-05-22T22:59:51.937588+0000","last_became_active":"2024-05-22T22:59:51.592530+0000","last_became_peered":"2024-05-22T22:59:51.592530+0000","last_unstale":"2024-05-22T22:59:51.937588+0000","last_undegraded":"2024-05-22T22:59:51.937588+0000","last_fullsized":"2024-05-22T22:59:51.937588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:56:18.660120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.958967+0000","last_change":"2024-05-22T22:59:51.956188+0000","last_active":"2024-05-22T22:59:51.958967+0000","last_peered":"2024-05-22T22:59:51.958967+0000","last_clean":"2024-05-22T22:59:51.958967+0000","last_became_active":"2024-05-22T22:59:51.955844+0000","last_became_peered":"2024-05-22T22:59:51.955844+0000","last_unstale":"2024-05-22T22:59:51.958967+0000","last_undegraded":"2024-05-22T22:59:51.958967+0000","last_fullsized":"2024-05-22T22:59:51.958967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:56:54.378586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.946406+0000","last_change":"2024-05-22T22:59:51.590641+0000","last_active":"2024-05-22T22:59:52.946406+0000","last_peered":"2024-05-22T22:59:52.946406+0000","last_clean":"2024-05-22T22:59:52.946406+0000","last_became_active":"2024-05-22T22:59:51.590321+0000","last_became_peered":"2024-05-22T22:59:51.590321+0000","last_unstale":"2024-05-22T22:59:52.946406+0000","last_undegraded":"2024-05-22T22:59:52.946406+0000","last_fullsized":"2024-05-22T22:59:52.946406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:46:15.157160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.635764+0000","last_change":"2024-05-22T22:59:51.950256+0000","last_active":"2024-05-22T22:59:53.635764+0000","last_peered":"2024-05-22T22:59:53.635764+0000","last_clean":"2024-05-22T22:59:53.635764+0000","last_became_active":"2024-05-22T22:59:51.949746+0000","last_became_peered":"2024-05-22T22:59:51.949746+0000","last_unstale":"2024-05-22T22:59:53.635764+0000","last_undegraded":"2024-05-22T22:59:53.635764+0000","last_fullsized":"2024-05-22T22:59:53.635764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:33:12.146425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.636107+0000","last_change":"2024-05-22T22:59:51.595761+0000","last_active":"2024-05-22T22:59:53.636107+0000","last_peered":"2024-05-22T22:59:53.636107+0000","last_clean":"2024-05-22T22:59:53.636107+0000","last_became_active":"2024-05-22T22:59:51.595343+0000","last_became_peered":"2024-05-22T22:59:51.595343+0000","last_unstale":"2024-05-22T22:59:53.636107+0000","last_undegraded":"2024-05-22T22:59:53.636107+0000","last_fullsized":"2024-05-22T22:59:53.636107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:58:21.461675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.946653+0000","last_change":"2024-05-22T22:59:51.968432+0000","last_active":"2024-05-22T22:59:52.946653+0000","last_peered":"2024-05-22T22:59:52.946653+0000","last_clean":"2024-05-22T22:59:52.946653+0000","last_became_active":"2024-05-22T22:59:51.968077+0000","last_became_peered":"2024-05-22T22:59:51.968077+0000","last_unstale":"2024-05-22T22:59:52.946653+0000","last_undegraded":"2024-05-22T22:59:52.946653+0000","last_fullsized":"2024-05-22T22:59:52.946653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:08:16.281671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.596737+0000","last_change":"2024-05-22T22:59:51.601616+0000","last_active":"2024-05-22T22:59:52.596737+0000","last_peered":"2024-05-22T22:59:52.596737+0000","last_clean":"2024-05-22T22:59:52.596737+0000","last_became_active":"2024-05-22T22:59:51.601249+0000","last_became_peered":"2024-05-22T22:59:51.601249+0000","last_unstale":"2024-05-22T22:59:52.596737+0000","last_undegraded":"2024-05-22T22:59:52.596737+0000","last_fullsized":"2024-05-22T22:59:52.596737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:07:07.030142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.954100+0000","last_change":"2024-05-22T22:59:51.951578+0000","last_active":"2024-05-22T22:59:51.954100+0000","last_peered":"2024-05-22T22:59:51.954100+0000","last_clean":"2024-05-22T22:59:51.954100+0000","last_became_active":"2024-05-22T22:59:51.951233+0000","last_became_peered":"2024-05-22T22:59:51.951233+0000","last_unstale":"2024-05-22T22:59:51.954100+0000","last_undegraded":"2024-05-22T22:59:51.954100+0000","last_fullsized":"2024-05-22T22:59:51.954100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:53:07.452864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.636388+0000","last_change":"2024-05-22T22:59:51.597819+0000","last_active":"2024-05-22T22:59:53.636388+0000","last_peered":"2024-05-22T22:59:53.636388+0000","last_clean":"2024-05-22T22:59:53.636388+0000","last_became_active":"2024-05-22T22:59:51.597262+0000","last_became_peered":"2024-05-22T22:59:51.597262+0000","last_unstale":"2024-05-22T22:59:53.636388+0000","last_undegraded":"2024-05-22T22:59:53.636388+0000","last_fullsized":"2024-05-22T22:59:53.636388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:05:02.849623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.954143+0000","last_change":"2024-05-22T22:59:51.951668+0000","last_active":"2024-05-22T22:59:51.954143+0000","last_peered":"2024-05-22T22:59:51.954143+0000","last_clean":"2024-05-22T22:59:51.954143+0000","last_became_active":"2024-05-22T22:59:51.951176+0000","last_became_peered":"2024-05-22T22:59:51.951176+0000","last_unstale":"2024-05-22T22:59:51.954143+0000","last_undegraded":"2024-05-22T22:59:51.954143+0000","last_fullsized":"2024-05-22T22:59:51.954143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:22:19.179260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.947547+0000","last_change":"2024-05-22T22:59:51.593144+0000","last_active":"2024-05-22T22:59:52.947547+0000","last_peered":"2024-05-22T22:59:52.947547+0000","last_clean":"2024-05-22T22:59:52.947547+0000","last_became_active":"2024-05-22T22:59:51.592557+0000","last_became_peered":"2024-05-22T22:59:51.592557+0000","last_unstale":"2024-05-22T22:59:52.947547+0000","last_undegraded":"2024-05-22T22:59:52.947547+0000","last_fullsized":"2024-05-22T22:59:52.947547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:28:19.403419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.960072+0000","last_change":"2024-05-22T22:59:51.958866+0000","last_active":"2024-05-22T22:59:51.960072+0000","last_peered":"2024-05-22T22:59:51.960072+0000","last_clean":"2024-05-22T22:59:51.960072+0000","last_became_active":"2024-05-22T22:59:51.958505+0000","last_became_peered":"2024-05-22T22:59:51.958505+0000","last_unstale":"2024-05-22T22:59:51.960072+0000","last_undegraded":"2024-05-22T22:59:51.960072+0000","last_fullsized":"2024-05-22T22:59:51.960072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:07:18.590496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.635743+0000","last_change":"2024-05-22T22:59:51.587391+0000","last_active":"2024-05-22T22:59:53.635743+0000","last_peered":"2024-05-22T22:59:53.635743+0000","last_clean":"2024-05-22T22:59:53.635743+0000","last_became_active":"2024-05-22T22:59:51.587056+0000","last_became_peered":"2024-05-22T22:59:51.587056+0000","last_unstale":"2024-05-22T22:59:53.635743+0000","last_undegraded":"2024-05-22T22:59:53.635743+0000","last_fullsized":"2024-05-22T22:59:53.635743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:04:40.614853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.954109+0000","last_change":"2024-05-22T22:59:51.953466+0000","last_active":"2024-05-22T22:59:51.954109+0000","last_peered":"2024-05-22T22:59:51.954109+0000","last_clean":"2024-05-22T22:59:51.954109+0000","last_became_active":"2024-05-22T22:59:51.952830+0000","last_became_peered":"2024-05-22T22:59:51.952830+0000","last_unstale":"2024-05-22T22:59:51.954109+0000","last_undegraded":"2024-05-22T22:59:51.954109+0000","last_fullsized":"2024-05-22T22:59:51.954109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:57:27.140822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.635787+0000","last_change":"2024-05-22T22:59:51.962993+0000","last_active":"2024-05-22T22:59:53.635787+0000","last_peered":"2024-05-22T22:59:53.635787+0000","last_clean":"2024-05-22T22:59:53.635787+0000","last_became_active":"2024-05-22T22:59:51.962591+0000","last_became_peered":"2024-05-22T22:59:51.962591+0000","last_unstale":"2024-05-22T22:59:53.635787+0000","last_undegraded":"2024-05-22T22:59:53.635787+0000","last_fullsized":"2024-05-22T22:59:53.635787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:55:50.902537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.636269+0000","last_change":"2024-05-22T22:59:51.956304+0000","last_active":"2024-05-22T22:59:53.636269+0000","last_peered":"2024-05-22T22:59:53.636269+0000","last_clean":"2024-05-22T22:59:53.636269+0000","last_became_active":"2024-05-22T22:59:51.955933+0000","last_became_peered":"2024-05-22T22:59:51.955933+0000","last_unstale":"2024-05-22T22:59:53.636269+0000","last_undegraded":"2024-05-22T22:59:53.636269+0000","last_fullsized":"2024-05-22T22:59:53.636269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:28:17.225851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.636039+0000","last_change":"2024-05-22T22:59:51.946806+0000","last_active":"2024-05-22T22:59:53.636039+0000","last_peered":"2024-05-22T22:59:53.636039+0000","last_clean":"2024-05-22T22:59:53.636039+0000","last_became_active":"2024-05-22T22:59:51.946433+0000","last_became_peered":"2024-05-22T22:59:51.946433+0000","last_unstale":"2024-05-22T22:59:53.636039+0000","last_undegraded":"2024-05-22T22:59:53.636039+0000","last_fullsized":"2024-05-22T22:59:53.636039+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:24:06.013688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:54.133298+0000","last_change":"2024-05-22T22:59:51.956466+0000","last_active":"2024-05-22T22:59:54.133298+0000","last_peered":"2024-05-22T22:59:54.133298+0000","last_clean":"2024-05-22T22:59:54.133298+0000","last_became_active":"2024-05-22T22:59:51.956144+0000","last_became_peered":"2024-05-22T22:59:51.956144+0000","last_unstale":"2024-05-22T22:59:54.133298+0000","last_undegraded":"2024-05-22T22:59:54.133298+0000","last_fullsized":"2024-05-22T22:59:54.133298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:08:28.674824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.635674+0000","last_change":"2024-05-22T22:59:51.955690+0000","last_active":"2024-05-22T22:59:53.635674+0000","last_peered":"2024-05-22T22:59:53.635674+0000","last_clean":"2024-05-22T22:59:53.635674+0000","last_became_active":"2024-05-22T22:59:51.955366+0000","last_became_peered":"2024-05-22T22:59:51.955366+0000","last_unstale":"2024-05-22T22:59:53.635674+0000","last_undegraded":"2024-05-22T22:59:53.635674+0000","last_fullsized":"2024-05-22T22:59:53.635674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:08:15.713039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.636005+0000","last_change":"2024-05-22T22:59:51.961971+0000","last_active":"2024-05-22T22:59:53.636005+0000","last_peered":"2024-05-22T22:59:53.636005+0000","last_clean":"2024-05-22T22:59:53.636005+0000","last_became_active":"2024-05-22T22:59:51.961604+0000","last_became_peered":"2024-05-22T22:59:51.961604+0000","last_unstale":"2024-05-22T22:59:53.636005+0000","last_undegraded":"2024-05-22T22:59:53.636005+0000","last_fullsized":"2024-05-22T22:59:53.636005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:32:14.669631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.636708+0000","last_change":"2024-05-22T22:59:51.948173+0000","last_active":"2024-05-22T22:59:53.636708+0000","last_peered":"2024-05-22T22:59:53.636708+0000","last_clean":"2024-05-22T22:59:53.636708+0000","last_became_active":"2024-05-22T22:59:51.947839+0000","last_became_peered":"2024-05-22T22:59:51.947839+0000","last_unstale":"2024-05-22T22:59:53.636708+0000","last_undegraded":"2024-05-22T22:59:53.636708+0000","last_fullsized":"2024-05-22T22:59:53.636708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:55:07.748445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.946793+0000","last_change":"2024-05-22T22:59:51.955627+0000","last_active":"2024-05-22T22:59:52.946793+0000","last_peered":"2024-05-22T22:59:52.946793+0000","last_clean":"2024-05-22T22:59:52.946793+0000","last_became_active":"2024-05-22T22:59:51.955284+0000","last_became_peered":"2024-05-22T22:59:51.955284+0000","last_unstale":"2024-05-22T22:59:52.946793+0000","last_undegraded":"2024-05-22T22:59:52.946793+0000","last_fullsized":"2024-05-22T22:59:52.946793+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:57:13.029317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.947272+0000","last_change":"2024-05-22T22:59:51.955889+0000","last_active":"2024-05-22T22:59:52.947272+0000","last_peered":"2024-05-22T22:59:52.947272+0000","last_clean":"2024-05-22T22:59:52.947272+0000","last_became_active":"2024-05-22T22:59:51.954930+0000","last_became_peered":"2024-05-22T22:59:51.954930+0000","last_unstale":"2024-05-22T22:59:52.947272+0000","last_undegraded":"2024-05-22T22:59:52.947272+0000","last_fullsized":"2024-05-22T22:59:52.947272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:47.536072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:52.946395+0000","last_change":"2024-05-22T22:59:51.954148+0000","last_active":"2024-05-22T22:59:52.946395+0000","last_peered":"2024-05-22T22:59:52.946395+0000","last_clean":"2024-05-22T22:59:52.946395+0000","last_became_active":"2024-05-22T22:59:51.952336+0000","last_became_peered":"2024-05-22T22:59:51.952336+0000","last_unstale":"2024-05-22T22:59:52.946395+0000","last_undegraded":"2024-05-22T22:59:52.946395+0000","last_fullsized":"2024-05-22T22:59:52.946395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:07:27.928910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-22T22:59:51.957844+0000","last_change":"2024-05-22T22:59:51.954300+0000","last_active":"2024-05-22T22:59:51.957844+0000","last_peered":"2024-05-22T22:59:51.957844+0000","last_clean":"2024-05-22T22:59:51.957844+0000","last_became_active":"2024-05-22T22:59:51.953779+0000","last_became_peered":"2024-05-22T22:59:51.953779+0000","last_unstale":"2024-05-22T22:59:51.957844+0000","last_undegraded":"2024-05-22T22:59:51.957844+0000","last_fullsized":"2024-05-22T22:59:51.957844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:32:31.799522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.635780+0000","last_change":"2024-05-22T22:59:51.947860+0000","last_active":"2024-05-22T22:59:53.635780+0000","last_peered":"2024-05-22T22:59:53.635780+0000","last_clean":"2024-05-22T22:59:53.635780+0000","last_became_active":"2024-05-22T22:59:51.947420+0000","last_became_peered":"2024-05-22T22:59:51.947420+0000","last_unstale":"2024-05-22T22:59:53.635780+0000","last_undegraded":"2024-05-22T22:59:53.635780+0000","last_fullsized":"2024-05-22T22:59:53.635780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:20:40.629285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-22T22:59:53.635956+0000","last_change":"2024-05-22T22:59:51.957867+0000","last_active":"2024-05-22T22:59:53.635956+0000","last_peered":"2024-05-22T22:59:53.635956+0000","last_clean":"2024-05-22T22:59:53.635956+0000","last_became_active":"2024-05-22T22:59:51.957522+0000","last_became_peered":"2024-05-22T22:59:51.957522+0000","last_unstale":"2024-05-22T22:59:53.635956+0000","last_undegraded":"2024-05-22T22:59:53.635956+0000","last_fullsized":"2024-05-22T22:59:53.635956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:41:18.631970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280555,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":38990,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":27,"num_write":56,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":61480,"data_compressed":1857,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":435309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612628,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":1055452,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776153,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":444461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037789,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":921600,"data_stored":435913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":1025677,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":443879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":426232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":917504,"data_stored":435399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":452124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":444100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920819,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":929792,"data_stored":443901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":987136,"data_stored":1034429,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":20928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":38386,"data_compressed":1857,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-22T22:59:58.663 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-22T22:59:58.664 INFO:tasks.cephadm:Setting up client nodes... 2024-05-22T22:59:58.664 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-22T22:59:58.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:58 smithi084 ceph-mon[32186]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:59:58.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:58 smithi084 ceph-mon[32186]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:59:58.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:58 smithi084 ceph-mon[32186]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:59:58.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:58 smithi084 ceph-mon[32186]: pgmap v292: 97 pgs: 97 active+clean; 615 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8.8 KiB/s wr, 11 op/s 2024-05-22T22:59:58.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:58 smithi084 ceph-mon[32186]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-22T22:59:58.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:58 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2002528979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:58.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:58 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1056644724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T22:59:58.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:58 smithi084 ceph-mon[32186]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-22T22:59:58.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:58 smithi204 ceph-mon[36211]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:59:58.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:58 smithi204 ceph-mon[36211]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:59:58.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:58 smithi204 ceph-mon[36211]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:59:58.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:58 smithi204 ceph-mon[36211]: pgmap v292: 97 pgs: 97 active+clean; 615 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8.8 KiB/s wr, 11 op/s 2024-05-22T22:59:58.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:58 smithi204 ceph-mon[36211]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-22T22:59:58.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:58 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2002528979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:58.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:58 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1056644724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T22:59:58.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:58 smithi204 ceph-mon[36211]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-22T22:59:58.951 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T22:59:59.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:58 smithi096 ceph-mon[36392]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:59:59.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:58 smithi096 ceph-mon[36392]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:59:59.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:58 smithi096 ceph-mon[36392]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T22:59:59.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:58 smithi096 ceph-mon[36392]: pgmap v292: 97 pgs: 97 active+clean; 615 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8.8 KiB/s wr, 11 op/s 2024-05-22T22:59:59.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:58 smithi096 ceph-mon[36392]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-22T22:59:59.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:58 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2002528979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T22:59:59.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:58 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1056644724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T22:59:59.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:58 smithi096 ceph-mon[36392]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-22T22:59:59.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:59 smithi084 ceph-mon[32186]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-22T22:59:59.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 22:59:59 smithi084 ceph-mon[32186]: pgmap v293: 97 pgs: 97 active+clean; 615 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6.8 KiB/s wr, 9 op/s 2024-05-22T22:59:59.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:59 smithi204 ceph-mon[36211]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-22T22:59:59.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 22:59:59 smithi204 ceph-mon[36211]: pgmap v293: 97 pgs: 97 active+clean; 615 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6.8 KiB/s wr, 9 op/s 2024-05-22T23:00:00.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:59 smithi096 ceph-mon[36392]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-22T23:00:00.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 22:59:59 smithi096 ceph-mon[36392]: pgmap v293: 97 pgs: 97 active+clean; 615 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6.8 KiB/s wr, 9 op/s 2024-05-22T23:00:00.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.610+0000 7f931357d700 1 -- 172.21.15.84:0/3778029050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f930c102e80 msgr2=0x7f930c109730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:00.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.610+0000 7f931357d700 1 --2- 172.21.15.84:0/3778029050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f930c102e80 0x7f930c109730 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f930400b0d0 tx=0x7f930400b3e0 comp rx=0 tx=0).stop 2024-05-22T23:00:00.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.610+0000 7f931357d700 1 -- 172.21.15.84:0/3778029050 shutdown_connections 2024-05-22T23:00:00.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.610+0000 7f931357d700 1 --2- 172.21.15.84:0/3778029050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f930c102e80 0x7f930c109730 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:00.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.610+0000 7f931357d700 1 --2- 172.21.15.84:0/3778029050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f930c1012e0 0x7f930c101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:00.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.610+0000 7f931357d700 1 --2- 172.21.15.84:0/3778029050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f930c1024e0 0x7f930c102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:00.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.610+0000 7f931357d700 1 -- 172.21.15.84:0/3778029050 >> 172.21.15.84:0/3778029050 conn(0x7f930c0fc880 msgr2=0x7f930c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:00.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.610+0000 7f931357d700 1 -- 172.21.15.84:0/3778029050 shutdown_connections 2024-05-22T23:00:00.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.610+0000 7f931357d700 1 -- 172.21.15.84:0/3778029050 wait complete. 2024-05-22T23:00:00.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.611+0000 7f931357d700 1 Processor -- start 2024-05-22T23:00:00.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.611+0000 7f931357d700 1 -- start start 2024-05-22T23:00:00.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.611+0000 7f931357d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f930c1012e0 0x7f930c06b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:00.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.611+0000 7f931357d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f930c1024e0 0x7f930c19d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:00.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.611+0000 7f931357d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f930c102e80 0x7f930c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:00.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.611+0000 7f931357d700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f930c10ceb0 con 0x7f930c1024e0 2024-05-22T23:00:00.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.611+0000 7f931357d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f930c10cd30 con 0x7f930c1012e0 2024-05-22T23:00:00.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.611+0000 7f931357d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f930c10d030 con 0x7f930c102e80 2024-05-22T23:00:00.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.611+0000 7f9310b18700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f930c1024e0 0x7f930c19d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:00.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.611+0000 7f9310b18700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f930c1024e0 0x7f930c19d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41982/0 (socket says 172.21.15.84:41982) 2024-05-22T23:00:00.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.611+0000 7f9310b18700 1 -- 172.21.15.84:0/592247546 learned_addr learned my addr 172.21.15.84:0/592247546 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:00.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.611+0000 7f9311319700 1 --2- 172.21.15.84:0/592247546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f930c1012e0 0x7f930c06b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:00.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.611+0000 7f9311b1a700 1 --2- 172.21.15.84:0/592247546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f930c102e80 0x7f930c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:00.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.611+0000 7f9310b18700 1 -- 172.21.15.84:0/592247546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f930c102e80 msgr2=0x7f930c19f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:00.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.611+0000 7f9310b18700 1 --2- 172.21.15.84:0/592247546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f930c102e80 0x7f930c19f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:00.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.611+0000 7f9310b18700 1 -- 172.21.15.84:0/592247546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f930c1012e0 msgr2=0x7f930c06b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:00.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.611+0000 7f9310b18700 1 --2- 172.21.15.84:0/592247546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f930c1012e0 0x7f930c06b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:00.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.611+0000 7f9310b18700 1 -- 172.21.15.84:0/592247546 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930c1a00e0 con 0x7f930c1024e0 2024-05-22T23:00:00.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.612+0000 7f9311b1a700 1 --2- 172.21.15.84:0/592247546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f930c102e80 0x7f930c19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:00:00.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.612+0000 7f9310b18700 1 --2- 172.21.15.84:0/592247546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f930c1024e0 0x7f930c19d1d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f930800d950 tx=0x7f930800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:00.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.612+0000 7f92fe7fc700 1 -- 172.21.15.84:0/592247546 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308016070 con 0x7f930c1024e0 2024-05-22T23:00:00.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.612+0000 7f931357d700 1 -- 172.21.15.84:0/592247546 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f930c1a03d0 con 0x7f930c1024e0 2024-05-22T23:00:00.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.612+0000 7f92fe7fc700 1 -- 172.21.15.84:0/592247546 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9308009e20 con 0x7f930c1024e0 2024-05-22T23:00:00.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.612+0000 7f92fe7fc700 1 -- 172.21.15.84:0/592247546 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930800f620 con 0x7f930c1024e0 2024-05-22T23:00:00.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.612+0000 7f931357d700 1 -- 172.21.15.84:0/592247546 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f930c1a0960 con 0x7f930c1024e0 2024-05-22T23:00:00.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.613+0000 7f931357d700 1 -- 172.21.15.84:0/592247546 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f930c0508d0 con 0x7f930c1024e0 2024-05-22T23:00:00.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.617+0000 7f92fe7fc700 1 -- 172.21.15.84:0/592247546 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9308019070 con 0x7f930c1024e0 2024-05-22T23:00:00.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.618+0000 7f92fe7fc700 1 --2- 172.21.15.84:0/592247546 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f92f80a2de0 0x7f92f80a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:00.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.618+0000 7f9311319700 1 --2- 172.21.15.84:0/592247546 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f92f80a2de0 0x7f92f80a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:00.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.618+0000 7f92fe7fc700 1 -- 172.21.15.84:0/592247546 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f93080ceb80 con 0x7f930c1024e0 2024-05-22T23:00:00.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.618+0000 7f9311319700 1 --2- 172.21.15.84:0/592247546 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f92f80a2de0 0x7f92f80a52a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f930c102340 tx=0x7f930000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:00.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.621+0000 7f92fe7fc700 1 -- 172.21.15.84:0/592247546 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f930809bb30 con 0x7f930c1024e0 2024-05-22T23:00:00.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:00 smithi084 ceph-mon[32186]: overall HEALTH_OK 2024-05-22T23:00:00.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.899+0000 7f931357d700 1 -- 172.21.15.84:0/592247546 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f930c062f70 con 0x7f930c1024e0 2024-05-22T23:00:00.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.902+0000 7f92fe7fc700 1 -- 172.21.15.84:0/592247546 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f930809b430 con 0x7f930c1024e0 2024-05-22T23:00:00.901 INFO:teuthology.orchestra.run.smithi084.stdout:[client.0] 2024-05-22T23:00:00.901 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDweE5mnf2qNRAA7/mb46KoVWTVV2olTLZMAA== 2024-05-22T23:00:00.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.904+0000 7f931357d700 1 -- 172.21.15.84:0/592247546 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f92f80a2de0 msgr2=0x7f92f80a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:00.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.904+0000 7f931357d700 1 --2- 172.21.15.84:0/592247546 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f92f80a2de0 0x7f92f80a52a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f930c102340 tx=0x7f930000a380 comp rx=0 tx=0).stop 2024-05-22T23:00:00.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.904+0000 7f931357d700 1 -- 172.21.15.84:0/592247546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f930c1024e0 msgr2=0x7f930c19d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:00.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.904+0000 7f931357d700 1 --2- 172.21.15.84:0/592247546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f930c1024e0 0x7f930c19d1d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f930800d950 tx=0x7f930800dc60 comp rx=0 tx=0).stop 2024-05-22T23:00:00.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.904+0000 7f931357d700 1 -- 172.21.15.84:0/592247546 shutdown_connections 2024-05-22T23:00:00.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.904+0000 7f931357d700 1 --2- 172.21.15.84:0/592247546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f930c102e80 0x7f930c19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:00.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.904+0000 7f931357d700 1 --2- 172.21.15.84:0/592247546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f930c1012e0 0x7f930c06b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:00.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.904+0000 7f931357d700 1 --2- 172.21.15.84:0/592247546 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f92f80a2de0 0x7f92f80a52a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:00.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.904+0000 7f931357d700 1 --2- 172.21.15.84:0/592247546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f930c1024e0 0x7f930c19d1d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:00.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.904+0000 7f931357d700 1 -- 172.21.15.84:0/592247546 >> 172.21.15.84:0/592247546 conn(0x7f930c0fc880 msgr2=0x7f930c105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:00.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.904+0000 7f931357d700 1 -- 172.21.15.84:0/592247546 shutdown_connections 2024-05-22T23:00:00.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:00.904+0000 7f931357d700 1 -- 172.21.15.84:0/592247546 wait complete. 2024-05-22T23:00:00.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:00 smithi204 ceph-mon[36211]: overall HEALTH_OK 2024-05-22T23:00:01.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:00 smithi096 ceph-mon[36392]: overall HEALTH_OK 2024-05-22T23:00:01.576 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-22T23:00:01.577 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-22T23:00:01.577 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-22T23:00:01.617 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-22T23:00:01.617 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-22T23:00:01.617 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph mgr dump --format=json 2024-05-22T23:00:01.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:01 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/592247546' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-22T23:00:01.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:01 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/592247546' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-22T23:00:01.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:01 smithi084 ceph-mon[32186]: pgmap v294: 97 pgs: 97 active+clean; 615 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5.9 KiB/s wr, 8 op/s 2024-05-22T23:00:01.897 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:01.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:01 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/592247546' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-22T23:00:01.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:01 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/592247546' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-22T23:00:01.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:01 smithi204 ceph-mon[36211]: pgmap v294: 97 pgs: 97 active+clean; 615 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5.9 KiB/s wr, 8 op/s 2024-05-22T23:00:02.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:01 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/592247546' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-22T23:00:02.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:01 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/592247546' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-22T23:00:02.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:01 smithi096 ceph-mon[36392]: pgmap v294: 97 pgs: 97 active+clean; 615 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5.9 KiB/s wr, 8 op/s 2024-05-22T23:00:03.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.500+0000 7f645a1b9700 1 -- 172.21.15.84:0/2964742150 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6454102610 msgr2=0x7f64541029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:03.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.500+0000 7f645a1b9700 1 --2- 172.21.15.84:0/2964742150 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6454102610 0x7f64541029f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6444009990 tx=0x7f6444009ca0 comp rx=0 tx=0).stop 2024-05-22T23:00:03.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.500+0000 7f645a1b9700 1 -- 172.21.15.84:0/2964742150 shutdown_connections 2024-05-22T23:00:03.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.500+0000 7f645a1b9700 1 --2- 172.21.15.84:0/2964742150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64541076a0 0x7f645410f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:03.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.500+0000 7f645a1b9700 1 --2- 172.21.15.84:0/2964742150 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6454102610 0x7f64541029f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:03.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.500+0000 7f645a1b9700 1 --2- 172.21.15.84:0/2964742150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6454102fc0 0x7f6454106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:03.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.500+0000 7f645a1b9700 1 -- 172.21.15.84:0/2964742150 >> 172.21.15.84:0/2964742150 conn(0x7f64540794c0 msgr2=0x7f64540798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:03.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.500+0000 7f645a1b9700 1 -- 172.21.15.84:0/2964742150 shutdown_connections 2024-05-22T23:00:03.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.500+0000 7f645a1b9700 1 -- 172.21.15.84:0/2964742150 wait complete. 2024-05-22T23:00:03.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.501+0000 7f645a1b9700 1 Processor -- start 2024-05-22T23:00:03.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.501+0000 7f645a1b9700 1 -- start start 2024-05-22T23:00:03.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.501+0000 7f645a1b9700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6454102610 0x7f645419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:03.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.501+0000 7f645a1b9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6454102fc0 0x7f645419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:03.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.501+0000 7f645a1b9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64541076a0 0x7f6454198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:03.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.502+0000 7f645a1b9700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6454112660 con 0x7f6454102610 2024-05-22T23:00:03.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.502+0000 7f645a1b9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f64541124e0 con 0x7f64541076a0 2024-05-22T23:00:03.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.502+0000 7f645a1b9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f64541127e0 con 0x7f6454102fc0 2024-05-22T23:00:03.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.502+0000 7f64537fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6454102610 0x7f645419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:03.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.502+0000 7f64537fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6454102610 0x7f645419e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36742/0 (socket says 172.21.15.84:36742) 2024-05-22T23:00:03.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.502+0000 7f64537fe700 1 -- 172.21.15.84:0/1975453690 learned_addr learned my addr 172.21.15.84:0/1975453690 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:03.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.502+0000 7f6452ffd700 1 --2- 172.21.15.84:0/1975453690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6454102fc0 0x7f645419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:03.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.502+0000 7f6453fff700 1 --2- 172.21.15.84:0/1975453690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64541076a0 0x7f6454198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:03.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.502+0000 7f64537fe700 1 -- 172.21.15.84:0/1975453690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6454102fc0 msgr2=0x7f645419ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:03.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.502+0000 7f64537fe700 1 --2- 172.21.15.84:0/1975453690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6454102fc0 0x7f645419ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:03.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.502+0000 7f64537fe700 1 -- 172.21.15.84:0/1975453690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64541076a0 msgr2=0x7f6454198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:03.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.502+0000 7f64537fe700 1 --2- 172.21.15.84:0/1975453690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64541076a0 0x7f6454198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:03.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.502+0000 7f64537fe700 1 -- 172.21.15.84:0/1975453690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6454199060 con 0x7f6454102610 2024-05-22T23:00:03.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.502+0000 7f64537fe700 1 --2- 172.21.15.84:0/1975453690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6454102610 0x7f645419e6f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f644400bff0 tx=0x7f6444005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:03.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.502+0000 7f6450ff9700 1 -- 172.21.15.84:0/1975453690 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6444027070 con 0x7f6454102610 2024-05-22T23:00:03.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.502+0000 7f645a1b9700 1 -- 172.21.15.84:0/1975453690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64541992f0 con 0x7f6454102610 2024-05-22T23:00:03.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.502+0000 7f6450ff9700 1 -- 172.21.15.84:0/1975453690 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6444004150 con 0x7f6454102610 2024-05-22T23:00:03.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.502+0000 7f6450ff9700 1 -- 172.21.15.84:0/1975453690 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644400f460 con 0x7f6454102610 2024-05-22T23:00:03.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.502+0000 7f645a1b9700 1 -- 172.21.15.84:0/1975453690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64541a55c0 con 0x7f6454102610 2024-05-22T23:00:03.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.503+0000 7f645a1b9700 1 -- 172.21.15.84:0/1975453690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6454069050 con 0x7f6454102610 2024-05-22T23:00:03.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.504+0000 7f6450ff9700 1 -- 172.21.15.84:0/1975453690 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f644402a070 con 0x7f6454102610 2024-05-22T23:00:03.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.505+0000 7f6450ff9700 1 --2- 172.21.15.84:0/1975453690 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f643c0a2cb0 0x7f643c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:03.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.505+0000 7f6452ffd700 1 --2- 172.21.15.84:0/1975453690 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f643c0a2cb0 0x7f643c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:03.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.505+0000 7f6450ff9700 1 -- 172.21.15.84:0/1975453690 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f64440cffa0 con 0x7f6454102610 2024-05-22T23:00:03.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.505+0000 7f6452ffd700 1 --2- 172.21.15.84:0/1975453690 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f643c0a2cb0 0x7f643c0a5170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6448005ea0 tx=0x7f6448005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:03.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.509+0000 7f6450ff9700 1 -- 172.21.15.84:0/1975453690 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f644405e850 con 0x7f6454102610 2024-05-22T23:00:03.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.762+0000 7f645a1b9700 1 -- 172.21.15.84:0/1975453690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f645402dbc0 con 0x7f6454102610 2024-05-22T23:00:03.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.767+0000 7f6450ff9700 1 -- 172.21.15.84:0/1975453690 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263201 (secure 0 0 0) 0x7f64440a1e20 con 0x7f6454102610 2024-05-22T23:00:03.770 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:00:03.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.770+0000 7f645a1b9700 1 -- 172.21.15.84:0/1975453690 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f643c0a2cb0 msgr2=0x7f643c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:03.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.770+0000 7f645a1b9700 1 --2- 172.21.15.84:0/1975453690 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f643c0a2cb0 0x7f643c0a5170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6448005ea0 tx=0x7f6448005e30 comp rx=0 tx=0).stop 2024-05-22T23:00:03.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.770+0000 7f645a1b9700 1 -- 172.21.15.84:0/1975453690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6454102610 msgr2=0x7f645419e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:03.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.770+0000 7f645a1b9700 1 --2- 172.21.15.84:0/1975453690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6454102610 0x7f645419e6f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f644400bff0 tx=0x7f6444005640 comp rx=0 tx=0).stop 2024-05-22T23:00:03.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.770+0000 7f645a1b9700 1 -- 172.21.15.84:0/1975453690 shutdown_connections 2024-05-22T23:00:03.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.770+0000 7f645a1b9700 1 --2- 172.21.15.84:0/1975453690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64541076a0 0x7f6454198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:03.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.770+0000 7f645a1b9700 1 --2- 172.21.15.84:0/1975453690 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f643c0a2cb0 0x7f643c0a5170 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:03.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.770+0000 7f645a1b9700 1 --2- 172.21.15.84:0/1975453690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6454102610 0x7f645419e6f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:03.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.770+0000 7f645a1b9700 1 --2- 172.21.15.84:0/1975453690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6454102fc0 0x7f645419ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:03.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.770+0000 7f645a1b9700 1 -- 172.21.15.84:0/1975453690 >> 172.21.15.84:0/1975453690 conn(0x7f64540794c0 msgr2=0x7f6454100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:03.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.770+0000 7f645a1b9700 1 -- 172.21.15.84:0/1975453690 shutdown_connections 2024-05-22T23:00:03.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:03.770+0000 7f645a1b9700 1 -- 172.21.15.84:0/1975453690 wait complete. 2024-05-22T23:00:04.542 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":4223589140},{"type":"v1","addr":"172.21.15.84:6801","nonce":4223589140}]},"active_addr":"172.21.15.84:6801/4223589140","active_change":"2024-05-22T22:52:45.395934+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.84:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":4077200298}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":4170743349}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":3490117794}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":2317224739}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":2234330738}]}]} 2024-05-22T23:00:04.549 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-22T23:00:04.549 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-22T23:00:04.550 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-22T23:00:04.793 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:04.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:04 smithi084 ceph-mon[32186]: pgmap v295: 97 pgs: 97 active+clean; 621 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.1 KiB/s wr, 8 op/s 2024-05-22T23:00:04.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:04 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1975453690' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-22T23:00:04.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:04 smithi204 ceph-mon[36211]: pgmap v295: 97 pgs: 97 active+clean; 621 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.1 KiB/s wr, 8 op/s 2024-05-22T23:00:04.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:04 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1975453690' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-22T23:00:05.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:04 smithi096 ceph-mon[36392]: pgmap v295: 97 pgs: 97 active+clean; 621 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.1 KiB/s wr, 8 op/s 2024-05-22T23:00:05.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:04 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1975453690' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-22T23:00:06.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.631+0000 7f3b2df49700 1 -- 172.21.15.84:0/3266592800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b2806aab0 msgr2=0x7f3b2806af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:06.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.631+0000 7f3b2df49700 1 --2- 172.21.15.84:0/3266592800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b2806aab0 0x7f3b2806af10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c009990 tx=0x7f3b1c009ca0 comp rx=0 tx=0).stop 2024-05-22T23:00:06.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.631+0000 7f3b2df49700 1 -- 172.21.15.84:0/3266592800 shutdown_connections 2024-05-22T23:00:06.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.631+0000 7f3b2df49700 1 --2- 172.21.15.84:0/3266592800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b2806b450 0x7f3b2810f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:06.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.631+0000 7f3b2df49700 1 --2- 172.21.15.84:0/3266592800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b281083e0 0x7f3b281087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:06.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.631+0000 7f3b2df49700 1 --2- 172.21.15.84:0/3266592800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b2806aab0 0x7f3b2806af10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:06.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.631+0000 7f3b2df49700 1 -- 172.21.15.84:0/3266592800 >> 172.21.15.84:0/3266592800 conn(0x7f3b28078230 msgr2=0x7f3b28078640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:06.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.631+0000 7f3b2df49700 1 -- 172.21.15.84:0/3266592800 shutdown_connections 2024-05-22T23:00:06.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.631+0000 7f3b2df49700 1 -- 172.21.15.84:0/3266592800 wait complete. 2024-05-22T23:00:06.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.632+0000 7f3b2df49700 1 Processor -- start 2024-05-22T23:00:06.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.632+0000 7f3b2df49700 1 -- start start 2024-05-22T23:00:06.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.632+0000 7f3b2df49700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b2806aab0 0x7f3b2819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:06.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.632+0000 7f3b2df49700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b2806b450 0x7f3b2819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:06.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.632+0000 7f3b2df49700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b281083e0 0x7f3b281989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:06.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.632+0000 7f3b2df49700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3b28112640 con 0x7f3b2806aab0 2024-05-22T23:00:06.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.632+0000 7f3b2df49700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b281124c0 con 0x7f3b2806b450 2024-05-22T23:00:06.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.632+0000 7f3b2df49700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3b281127c0 con 0x7f3b281083e0 2024-05-22T23:00:06.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.632+0000 7f3b277fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b2806aab0 0x7f3b2819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:06.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.633+0000 7f3b277fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b2806aab0 0x7f3b2819e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36758/0 (socket says 172.21.15.84:36758) 2024-05-22T23:00:06.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.633+0000 7f3b277fe700 1 -- 172.21.15.84:0/1267753673 learned_addr learned my addr 172.21.15.84:0/1267753673 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:06.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.633+0000 7f3b26ffd700 1 --2- 172.21.15.84:0/1267753673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b2806b450 0x7f3b2819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:06.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.633+0000 7f3b27fff700 1 --2- 172.21.15.84:0/1267753673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b281083e0 0x7f3b281989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:06.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.633+0000 7f3b277fe700 1 -- 172.21.15.84:0/1267753673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b281083e0 msgr2=0x7f3b281989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:06.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.633+0000 7f3b277fe700 1 --2- 172.21.15.84:0/1267753673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b281083e0 0x7f3b281989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:06.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.633+0000 7f3b277fe700 1 -- 172.21.15.84:0/1267753673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b2806b450 msgr2=0x7f3b2819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:06.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.633+0000 7f3b277fe700 1 --2- 172.21.15.84:0/1267753673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b2806b450 0x7f3b2819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:06.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.633+0000 7f3b277fe700 1 -- 172.21.15.84:0/1267753673 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b28199100 con 0x7f3b2806aab0 2024-05-22T23:00:06.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.633+0000 7f3b277fe700 1 --2- 172.21.15.84:0/1267753673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b2806aab0 0x7f3b2819e790 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3b1800d950 tx=0x7f3b1800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:06.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.633+0000 7f3b24ff9700 1 -- 172.21.15.84:0/1267753673 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18016070 con 0x7f3b2806aab0 2024-05-22T23:00:06.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.633+0000 7f3b24ff9700 1 -- 172.21.15.84:0/1267753673 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b1800fd70 con 0x7f3b2806aab0 2024-05-22T23:00:06.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.633+0000 7f3b2df49700 1 -- 172.21.15.84:0/1267753673 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b281993f0 con 0x7f3b2806aab0 2024-05-22T23:00:06.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.633+0000 7f3b24ff9700 1 -- 172.21.15.84:0/1267753673 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1800e610 con 0x7f3b2806aab0 2024-05-22T23:00:06.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.633+0000 7f3b2df49700 1 -- 172.21.15.84:0/1267753673 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b281a5660 con 0x7f3b2806aab0 2024-05-22T23:00:06.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.634+0000 7f3b2df49700 1 -- 172.21.15.84:0/1267753673 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b28069050 con 0x7f3b2806aab0 2024-05-22T23:00:06.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.635+0000 7f3b24ff9700 1 -- 172.21.15.84:0/1267753673 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3b18019070 con 0x7f3b2806aab0 2024-05-22T23:00:06.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.636+0000 7f3b24ff9700 1 --2- 172.21.15.84:0/1267753673 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f3b100a2cb0 0x7f3b100a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:06.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.636+0000 7f3b26ffd700 1 --2- 172.21.15.84:0/1267753673 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f3b100a2cb0 0x7f3b100a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:06.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.636+0000 7f3b24ff9700 1 -- 172.21.15.84:0/1267753673 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f3b180cea20 con 0x7f3b2806aab0 2024-05-22T23:00:06.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.636+0000 7f3b26ffd700 1 --2- 172.21.15.84:0/1267753673 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f3b100a2cb0 0x7f3b100a5170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00b5c0 tx=0x7f3b1c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:06.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.639+0000 7f3b24ff9700 1 -- 172.21.15.84:0/1267753673 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3b1809b9d0 con 0x7f3b2806aab0 2024-05-22T23:00:06.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.835+0000 7f3b2df49700 1 -- 172.21.15.84:0/1267753673 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b2802dbc0 con 0x7f3b2806aab0 2024-05-22T23:00:06.838 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:06 smithi084 ceph-mon[32186]: pgmap v296: 97 pgs: 97 active+clean; 621 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5.4 KiB/s wr, 8 op/s 2024-05-22T23:00:06.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.837+0000 7f3b24ff9700 1 -- 172.21.15.84:0/1267753673 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22194 (secure 0 0 0) 0x7f3b1809b2d0 con 0x7f3b2806aab0 2024-05-22T23:00:06.839 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:00:06.839 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":86,"fsid":"c1135a12-188d-11ef-bc9a-c7b262605968","created":"2024-05-22T22:51:49.878218+0000","modified":"2024-05-22T22:59:52.940885+0000","last_up_change":"2024-05-22T22:58:58.550278+0000","last_in_change":"2024-05-22T22:58:37.322139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T22:55:07.719394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T22:59:50.150511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T22:59:51.111182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762df065-fb4f-4c71-aab8-a62d2282c253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6803","nonce":1559945826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6805","nonce":1559945826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6809","nonce":1559945826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6807","nonce":1559945826}]},"public_addr":"172.21.15.84:6803/1559945826","cluster_addr":"172.21.15.84:6805/1559945826","heartbeat_back_addr":"172.21.15.84:6809/1559945826","heartbeat_front_addr":"172.21.15.84:6807/1559945826","state":["exists","up"]},{"osd":1,"uuid":"3fb2385d-6a34-49e3-88e1-116237beb346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6801","nonce":2996505353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6803","nonce":2996505353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6807","nonce":2996505353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6805","nonce":2996505353}]},"public_addr":"172.21.15.96:6801/2996505353","cluster_addr":"172.21.15.96:6803/2996505353","heartbeat_back_addr":"172.21.15.96:6807/2996505353","heartbeat_front_addr":"172.21.15.96:6805/2996505353","state":["exists","up"]},{"osd":2,"uuid":"e306e88e-a308-4585-ae11-c8af10b84038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6801","nonce":1499577897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6803","nonce":1499577897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6807","nonce":1499577897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6805","nonce":1499577897}]},"public_addr":"172.21.15.204:6801/1499577897","cluster_addr":"172.21.15.204:6803/1499577897","heartbeat_back_addr":"172.21.15.204:6807/1499577897","heartbeat_front_addr":"172.21.15.204:6805/1499577897","state":["exists","up"]},{"osd":3,"uuid":"d2a5fb97-6f60-4d45-9f18-447cae480974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6811","nonce":2869340648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6813","nonce":2869340648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6817","nonce":2869340648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6815","nonce":2869340648}]},"public_addr":"172.21.15.84:6811/2869340648","cluster_addr":"172.21.15.84:6813/2869340648","heartbeat_back_addr":"172.21.15.84:6817/2869340648","heartbeat_front_addr":"172.21.15.84:6815/2869340648","state":["exists","up"]},{"osd":4,"uuid":"4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6809","nonce":2554265648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6811","nonce":2554265648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6815","nonce":2554265648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6813","nonce":2554265648}]},"public_addr":"172.21.15.96:6809/2554265648","cluster_addr":"172.21.15.96:6811/2554265648","heartbeat_back_addr":"172.21.15.96:6815/2554265648","heartbeat_front_addr":"172.21.15.96:6813/2554265648","state":["exists","up"]},{"osd":5,"uuid":"7996bf5e-93a5-41b0-a9b1-99e7ef4514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6809","nonce":4075476986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6811","nonce":4075476986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6815","nonce":4075476986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6813","nonce":4075476986}]},"public_addr":"172.21.15.204:6809/4075476986","cluster_addr":"172.21.15.204:6811/4075476986","heartbeat_back_addr":"172.21.15.204:6815/4075476986","heartbeat_front_addr":"172.21.15.204:6813/4075476986","state":["exists","up"]},{"osd":6,"uuid":"f4b7c5de-1836-4d16-8603-3c517524d9fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6819","nonce":1806202865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6821","nonce":1806202865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6825","nonce":1806202865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6823","nonce":1806202865}]},"public_addr":"172.21.15.84:6819/1806202865","cluster_addr":"172.21.15.84:6821/1806202865","heartbeat_back_addr":"172.21.15.84:6825/1806202865","heartbeat_front_addr":"172.21.15.84:6823/1806202865","state":["exists","up"]},{"osd":7,"uuid":"69a18c43-5447-480d-9466-190830070c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6817","nonce":3195792774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6819","nonce":3195792774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6823","nonce":3195792774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6821","nonce":3195792774}]},"public_addr":"172.21.15.96:6817/3195792774","cluster_addr":"172.21.15.96:6819/3195792774","heartbeat_back_addr":"172.21.15.96:6823/3195792774","heartbeat_front_addr":"172.21.15.96:6821/3195792774","state":["exists","up"]},{"osd":8,"uuid":"2c5cadca-689a-441e-847c-ba370d400ef1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6817","nonce":606997927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6819","nonce":606997927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6823","nonce":606997927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6821","nonce":606997927}]},"public_addr":"172.21.15.204:6817/606997927","cluster_addr":"172.21.15.204:6819/606997927","heartbeat_back_addr":"172.21.15.204:6823/606997927","heartbeat_front_addr":"172.21.15.204:6821/606997927","state":["exists","up"]},{"osd":9,"uuid":"e3984fb2-7c00-4275-a10e-bca7c859f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6827","nonce":2786189012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6829","nonce":2786189012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6833","nonce":2786189012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6831","nonce":2786189012}]},"public_addr":"172.21.15.84:6827/2786189012","cluster_addr":"172.21.15.84:6829/2786189012","heartbeat_back_addr":"172.21.15.84:6833/2786189012","heartbeat_front_addr":"172.21.15.84:6831/2786189012","state":["exists","up"]},{"osd":10,"uuid":"b4938160-a808-4ded-91c2-d245c99fff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6825","nonce":1985398526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6827","nonce":1985398526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6831","nonce":1985398526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6829","nonce":1985398526}]},"public_addr":"172.21.15.96:6825/1985398526","cluster_addr":"172.21.15.96:6827/1985398526","heartbeat_back_addr":"172.21.15.96:6831/1985398526","heartbeat_front_addr":"172.21.15.96:6829/1985398526","state":["exists","up"]},{"osd":11,"uuid":"238922ef-d046-40cc-833d-25b7e7a523cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6825","nonce":3106116519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6827","nonce":3106116519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6831","nonce":3106116519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6829","nonce":3106116519}]},"public_addr":"172.21.15.204:6825/3106116519","cluster_addr":"172.21.15.204:6827/3106116519","heartbeat_back_addr":"172.21.15.204:6831/3106116519","heartbeat_front_addr":"172.21.15.204:6829/3106116519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:16.443480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:39.898133+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:03.664544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:28.838687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:53.829733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:19.269307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:45.166910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:10.771614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:36.678516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:03.150592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:29.327769+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:56.315938+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6801/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:6800/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/2766288608":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/950455999":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/4000289608":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1650467363":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1052746114":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6800/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6801/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/14610813":"2024-05-23T22:52:45.395791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T23:00:06.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.840+0000 7f3b2df49700 1 -- 172.21.15.84:0/1267753673 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f3b100a2cb0 msgr2=0x7f3b100a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:06.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.840+0000 7f3b2df49700 1 --2- 172.21.15.84:0/1267753673 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f3b100a2cb0 0x7f3b100a5170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00b5c0 tx=0x7f3b1c005960 comp rx=0 tx=0).stop 2024-05-22T23:00:06.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.840+0000 7f3b2df49700 1 -- 172.21.15.84:0/1267753673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b2806aab0 msgr2=0x7f3b2819e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:06.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.840+0000 7f3b2df49700 1 --2- 172.21.15.84:0/1267753673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b2806aab0 0x7f3b2819e790 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3b1800d950 tx=0x7f3b1800dc60 comp rx=0 tx=0).stop 2024-05-22T23:00:06.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.840+0000 7f3b2df49700 1 -- 172.21.15.84:0/1267753673 shutdown_connections 2024-05-22T23:00:06.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.840+0000 7f3b2df49700 1 --2- 172.21.15.84:0/1267753673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b281083e0 0x7f3b281989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:06.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.840+0000 7f3b2df49700 1 --2- 172.21.15.84:0/1267753673 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f3b100a2cb0 0x7f3b100a5170 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:06.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.840+0000 7f3b2df49700 1 --2- 172.21.15.84:0/1267753673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b2806aab0 0x7f3b2819e790 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:06.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.840+0000 7f3b2df49700 1 --2- 172.21.15.84:0/1267753673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b2806b450 0x7f3b2819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:06.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.840+0000 7f3b2df49700 1 -- 172.21.15.84:0/1267753673 >> 172.21.15.84:0/1267753673 conn(0x7f3b28078230 msgr2=0x7f3b280fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:06.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.840+0000 7f3b2df49700 1 -- 172.21.15.84:0/1267753673 shutdown_connections 2024-05-22T23:00:06.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:06.840+0000 7f3b2df49700 1 -- 172.21.15.84:0/1267753673 wait complete. 2024-05-22T23:00:06.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:06 smithi204 ceph-mon[36211]: pgmap v296: 97 pgs: 97 active+clean; 621 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5.4 KiB/s wr, 8 op/s 2024-05-22T23:00:07.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:06 smithi096 ceph-mon[36392]: pgmap v296: 97 pgs: 97 active+clean; 621 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5.4 KiB/s wr, 8 op/s 2024-05-22T23:00:07.576 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-22T23:00:07.576 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-22T23:00:07.817 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:07.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:07 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1267753673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:00:07.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:07 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1267753673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:00:08.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:07 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1267753673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:00:08.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:08 smithi084 ceph-mon[32186]: pgmap v297: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.6 KiB/s wr, 7 op/s 2024-05-22T23:00:08.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:08 smithi204 ceph-mon[36211]: pgmap v297: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.6 KiB/s wr, 7 op/s 2024-05-22T23:00:09.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:08 smithi096 ceph-mon[36392]: pgmap v297: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.6 KiB/s wr, 7 op/s 2024-05-22T23:00:09.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.641+0000 7fb011e8d700 1 -- 172.21.15.84:0/1702200169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb00c100950 msgr2=0x7fb00c100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:09.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.641+0000 7fb011e8d700 1 --2- 172.21.15.84:0/1702200169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb00c100950 0x7fb00c100db0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb000009990 tx=0x7fb000009ca0 comp rx=0 tx=0).stop 2024-05-22T23:00:09.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.641+0000 7fb011e8d700 1 -- 172.21.15.84:0/1702200169 shutdown_connections 2024-05-22T23:00:09.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.641+0000 7fb011e8d700 1 --2- 172.21.15.84:0/1702200169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb00c1012f0 0x7fb00c10f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:09.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.641+0000 7fb011e8d700 1 --2- 172.21.15.84:0/1702200169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb00c1083f0 0x7fb00c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:09.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.641+0000 7fb011e8d700 1 --2- 172.21.15.84:0/1702200169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb00c100950 0x7fb00c100db0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:09.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.641+0000 7fb011e8d700 1 -- 172.21.15.84:0/1702200169 >> 172.21.15.84:0/1702200169 conn(0x7fb00c0778d0 msgr2=0x7fb00c0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:09.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.641+0000 7fb011e8d700 1 -- 172.21.15.84:0/1702200169 shutdown_connections 2024-05-22T23:00:09.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.641+0000 7fb011e8d700 1 -- 172.21.15.84:0/1702200169 wait complete. 2024-05-22T23:00:09.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.642+0000 7fb011e8d700 1 Processor -- start 2024-05-22T23:00:09.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.642+0000 7fb011e8d700 1 -- start start 2024-05-22T23:00:09.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.642+0000 7fb011e8d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb00c100950 0x7fb00c101b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:09.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.642+0000 7fb011e8d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb00c1012f0 0x7fb00c102070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:09.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.642+0000 7fb011e8d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb00c1083f0 0x7fb00c1025b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:09.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.642+0000 7fb011e8d700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c1126f0 con 0x7fb00c1083f0 2024-05-22T23:00:09.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.642+0000 7fb011e8d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c112570 con 0x7fb00c1012f0 2024-05-22T23:00:09.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.642+0000 7fb011e8d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c112870 con 0x7fb00c100950 2024-05-22T23:00:09.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.643+0000 7fb00bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb00c1083f0 0x7fb00c1025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:09.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.643+0000 7fb00bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb00c1083f0 0x7fb00c1025b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36770/0 (socket says 172.21.15.84:36770) 2024-05-22T23:00:09.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.643+0000 7fb00bfff700 1 -- 172.21.15.84:0/4017660684 learned_addr learned my addr 172.21.15.84:0/4017660684 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:09.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.643+0000 7fb00b7fe700 1 --2- 172.21.15.84:0/4017660684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb00c100950 0x7fb00c101b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:09.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.643+0000 7fb00affd700 1 --2- 172.21.15.84:0/4017660684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb00c1012f0 0x7fb00c102070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:09.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.643+0000 7fb00bfff700 1 -- 172.21.15.84:0/4017660684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb00c100950 msgr2=0x7fb00c101b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:09.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.643+0000 7fb00bfff700 1 --2- 172.21.15.84:0/4017660684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb00c100950 0x7fb00c101b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:09.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.643+0000 7fb00bfff700 1 -- 172.21.15.84:0/4017660684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb00c1012f0 msgr2=0x7fb00c102070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:09.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.643+0000 7fb00bfff700 1 --2- 172.21.15.84:0/4017660684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb00c1012f0 0x7fb00c102070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:09.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.643+0000 7fb00bfff700 1 -- 172.21.15.84:0/4017660684 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00c1a7090 con 0x7fb00c1083f0 2024-05-22T23:00:09.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.643+0000 7fb00bfff700 1 --2- 172.21.15.84:0/4017660684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb00c1083f0 0x7fb00c1025b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb00400b970 tx=0x7fb00400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:09.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.643+0000 7fb008ff9700 1 -- 172.21.15.84:0/4017660684 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00400d860 con 0x7fb00c1083f0 2024-05-22T23:00:09.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.643+0000 7fb008ff9700 1 -- 172.21.15.84:0/4017660684 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb00400e430 con 0x7fb00c1083f0 2024-05-22T23:00:09.646 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.643+0000 7fb011e8d700 1 -- 172.21.15.84:0/4017660684 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb00c1a7380 con 0x7fb00c1083f0 2024-05-22T23:00:09.646 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.643+0000 7fb008ff9700 1 -- 172.21.15.84:0/4017660684 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb004017660 con 0x7fb00c1083f0 2024-05-22T23:00:09.646 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.644+0000 7fb011e8d700 1 -- 172.21.15.84:0/4017660684 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb00c1a7890 con 0x7fb00c1083f0 2024-05-22T23:00:09.646 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.644+0000 7fb011e8d700 1 -- 172.21.15.84:0/4017660684 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb00c103c70 con 0x7fb00c1083f0 2024-05-22T23:00:09.647 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.646+0000 7fb008ff9700 1 -- 172.21.15.84:0/4017660684 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb00400e5e0 con 0x7fb00c1083f0 2024-05-22T23:00:09.647 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.646+0000 7fb008ff9700 1 --2- 172.21.15.84:0/4017660684 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7faff40a2cb0 0x7faff40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:09.647 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.646+0000 7fb00b7fe700 1 --2- 172.21.15.84:0/4017660684 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7faff40a2cb0 0x7faff40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:09.647 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.646+0000 7fb008ff9700 1 -- 172.21.15.84:0/4017660684 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb0040ceba0 con 0x7fb00c1083f0 2024-05-22T23:00:09.647 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.647+0000 7fb00b7fe700 1 --2- 172.21.15.84:0/4017660684 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7faff40a2cb0 0x7faff40a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faffc005950 tx=0x7faffc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:09.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.651+0000 7fb008ff9700 1 -- 172.21.15.84:0/4017660684 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb004013030 con 0x7fb00c1083f0 2024-05-22T23:00:09.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.849+0000 7fb011e8d700 1 -- 172.21.15.84:0/4017660684 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb00c02dbc0 con 0x7fb00c1083f0 2024-05-22T23:00:09.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.850+0000 7fb008ff9700 1 -- 172.21.15.84:0/4017660684 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22194 (secure 0 0 0) 0x7fb00409bb50 con 0x7fb00c1083f0 2024-05-22T23:00:09.850 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:00:09.850 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":86,"fsid":"c1135a12-188d-11ef-bc9a-c7b262605968","created":"2024-05-22T22:51:49.878218+0000","modified":"2024-05-22T22:59:52.940885+0000","last_up_change":"2024-05-22T22:58:58.550278+0000","last_in_change":"2024-05-22T22:58:37.322139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T22:55:07.719394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T22:59:50.150511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T22:59:51.111182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762df065-fb4f-4c71-aab8-a62d2282c253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6803","nonce":1559945826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6805","nonce":1559945826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6809","nonce":1559945826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6807","nonce":1559945826}]},"public_addr":"172.21.15.84:6803/1559945826","cluster_addr":"172.21.15.84:6805/1559945826","heartbeat_back_addr":"172.21.15.84:6809/1559945826","heartbeat_front_addr":"172.21.15.84:6807/1559945826","state":["exists","up"]},{"osd":1,"uuid":"3fb2385d-6a34-49e3-88e1-116237beb346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6801","nonce":2996505353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6803","nonce":2996505353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6807","nonce":2996505353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6805","nonce":2996505353}]},"public_addr":"172.21.15.96:6801/2996505353","cluster_addr":"172.21.15.96:6803/2996505353","heartbeat_back_addr":"172.21.15.96:6807/2996505353","heartbeat_front_addr":"172.21.15.96:6805/2996505353","state":["exists","up"]},{"osd":2,"uuid":"e306e88e-a308-4585-ae11-c8af10b84038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6801","nonce":1499577897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6803","nonce":1499577897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6807","nonce":1499577897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6805","nonce":1499577897}]},"public_addr":"172.21.15.204:6801/1499577897","cluster_addr":"172.21.15.204:6803/1499577897","heartbeat_back_addr":"172.21.15.204:6807/1499577897","heartbeat_front_addr":"172.21.15.204:6805/1499577897","state":["exists","up"]},{"osd":3,"uuid":"d2a5fb97-6f60-4d45-9f18-447cae480974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6811","nonce":2869340648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6813","nonce":2869340648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6817","nonce":2869340648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6815","nonce":2869340648}]},"public_addr":"172.21.15.84:6811/2869340648","cluster_addr":"172.21.15.84:6813/2869340648","heartbeat_back_addr":"172.21.15.84:6817/2869340648","heartbeat_front_addr":"172.21.15.84:6815/2869340648","state":["exists","up"]},{"osd":4,"uuid":"4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6809","nonce":2554265648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6811","nonce":2554265648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6815","nonce":2554265648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6813","nonce":2554265648}]},"public_addr":"172.21.15.96:6809/2554265648","cluster_addr":"172.21.15.96:6811/2554265648","heartbeat_back_addr":"172.21.15.96:6815/2554265648","heartbeat_front_addr":"172.21.15.96:6813/2554265648","state":["exists","up"]},{"osd":5,"uuid":"7996bf5e-93a5-41b0-a9b1-99e7ef4514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6809","nonce":4075476986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6811","nonce":4075476986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6815","nonce":4075476986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6813","nonce":4075476986}]},"public_addr":"172.21.15.204:6809/4075476986","cluster_addr":"172.21.15.204:6811/4075476986","heartbeat_back_addr":"172.21.15.204:6815/4075476986","heartbeat_front_addr":"172.21.15.204:6813/4075476986","state":["exists","up"]},{"osd":6,"uuid":"f4b7c5de-1836-4d16-8603-3c517524d9fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6819","nonce":1806202865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6821","nonce":1806202865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6825","nonce":1806202865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6823","nonce":1806202865}]},"public_addr":"172.21.15.84:6819/1806202865","cluster_addr":"172.21.15.84:6821/1806202865","heartbeat_back_addr":"172.21.15.84:6825/1806202865","heartbeat_front_addr":"172.21.15.84:6823/1806202865","state":["exists","up"]},{"osd":7,"uuid":"69a18c43-5447-480d-9466-190830070c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6817","nonce":3195792774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6819","nonce":3195792774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6823","nonce":3195792774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6821","nonce":3195792774}]},"public_addr":"172.21.15.96:6817/3195792774","cluster_addr":"172.21.15.96:6819/3195792774","heartbeat_back_addr":"172.21.15.96:6823/3195792774","heartbeat_front_addr":"172.21.15.96:6821/3195792774","state":["exists","up"]},{"osd":8,"uuid":"2c5cadca-689a-441e-847c-ba370d400ef1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6817","nonce":606997927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6819","nonce":606997927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6823","nonce":606997927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6821","nonce":606997927}]},"public_addr":"172.21.15.204:6817/606997927","cluster_addr":"172.21.15.204:6819/606997927","heartbeat_back_addr":"172.21.15.204:6823/606997927","heartbeat_front_addr":"172.21.15.204:6821/606997927","state":["exists","up"]},{"osd":9,"uuid":"e3984fb2-7c00-4275-a10e-bca7c859f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6827","nonce":2786189012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6829","nonce":2786189012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6833","nonce":2786189012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6831","nonce":2786189012}]},"public_addr":"172.21.15.84:6827/2786189012","cluster_addr":"172.21.15.84:6829/2786189012","heartbeat_back_addr":"172.21.15.84:6833/2786189012","heartbeat_front_addr":"172.21.15.84:6831/2786189012","state":["exists","up"]},{"osd":10,"uuid":"b4938160-a808-4ded-91c2-d245c99fff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6825","nonce":1985398526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6827","nonce":1985398526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6831","nonce":1985398526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6829","nonce":1985398526}]},"public_addr":"172.21.15.96:6825/1985398526","cluster_addr":"172.21.15.96:6827/1985398526","heartbeat_back_addr":"172.21.15.96:6831/1985398526","heartbeat_front_addr":"172.21.15.96:6829/1985398526","state":["exists","up"]},{"osd":11,"uuid":"238922ef-d046-40cc-833d-25b7e7a523cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6825","nonce":3106116519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6827","nonce":3106116519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6831","nonce":3106116519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6829","nonce":3106116519}]},"public_addr":"172.21.15.204:6825/3106116519","cluster_addr":"172.21.15.204:6827/3106116519","heartbeat_back_addr":"172.21.15.204:6831/3106116519","heartbeat_front_addr":"172.21.15.204:6829/3106116519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:16.443480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:39.898133+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:03.664544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:28.838687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:53.829733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:19.269307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:45.166910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:10.771614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:36.678516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:03.150592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:29.327769+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:56.315938+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6801/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:6800/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/2766288608":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/950455999":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/4000289608":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1650467363":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1052746114":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6800/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6801/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/14610813":"2024-05-23T22:52:45.395791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T23:00:09.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.853+0000 7fb011e8d700 1 -- 172.21.15.84:0/4017660684 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7faff40a2cb0 msgr2=0x7faff40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:09.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.853+0000 7fb011e8d700 1 --2- 172.21.15.84:0/4017660684 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7faff40a2cb0 0x7faff40a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faffc005950 tx=0x7faffc00a300 comp rx=0 tx=0).stop 2024-05-22T23:00:09.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.853+0000 7fb011e8d700 1 -- 172.21.15.84:0/4017660684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb00c1083f0 msgr2=0x7fb00c1025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:09.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.853+0000 7fb011e8d700 1 --2- 172.21.15.84:0/4017660684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb00c1083f0 0x7fb00c1025b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb00400b970 tx=0x7fb00400bc80 comp rx=0 tx=0).stop 2024-05-22T23:00:09.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.853+0000 7fb011e8d700 1 -- 172.21.15.84:0/4017660684 shutdown_connections 2024-05-22T23:00:09.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.853+0000 7fb011e8d700 1 --2- 172.21.15.84:0/4017660684 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7faff40a2cb0 0x7faff40a5170 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:09.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.853+0000 7fb011e8d700 1 --2- 172.21.15.84:0/4017660684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb00c1083f0 0x7fb00c1025b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:09.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.853+0000 7fb011e8d700 1 --2- 172.21.15.84:0/4017660684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb00c100950 0x7fb00c101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:09.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.853+0000 7fb011e8d700 1 --2- 172.21.15.84:0/4017660684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb00c1012f0 0x7fb00c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:09.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.853+0000 7fb011e8d700 1 -- 172.21.15.84:0/4017660684 >> 172.21.15.84:0/4017660684 conn(0x7fb00c0778d0 msgr2=0x7fb00c0fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:09.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.853+0000 7fb011e8d700 1 -- 172.21.15.84:0/4017660684 shutdown_connections 2024-05-22T23:00:09.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:09.853+0000 7fb011e8d700 1 -- 172.21.15.84:0/4017660684 wait complete. 2024-05-22T23:00:10.518 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-22T23:00:10.519 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-22T23:00:10.519 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-22T23:00:10.519 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-22T23:00:10.520 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-22T23:00:10.520 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-22T23:00:10.521 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-22T23:00:10.521 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-22T23:00:10.521 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-22T23:00:10.522 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-22T23:00:10.522 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-22T23:00:10.522 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-22T23:00:10.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:10 smithi084 ceph-mon[32186]: pgmap v298: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-05-22T23:00:10.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:10 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/4017660684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:00:10.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:10 smithi204 ceph-mon[36211]: pgmap v298: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-05-22T23:00:10.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:10 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/4017660684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:00:11.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:10 smithi096 ceph-mon[36392]: pgmap v298: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-05-22T23:00:11.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:10 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/4017660684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:00:11.211 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:11.502 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:11.505 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:11.506 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:11.507 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:11.508 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:11.938 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:11.938 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:11.939 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:12.383 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:12.384 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:12.384 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:12.624 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:12 smithi084 ceph-mon[32186]: pgmap v299: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-05-22T23:00:12.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:12 smithi204 ceph-mon[36211]: pgmap v299: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-05-22T23:00:13.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:12 smithi096 ceph-mon[36392]: pgmap v299: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-05-22T23:00:14.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:14 smithi084 ceph-mon[32186]: pgmap v300: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.1 KiB/s wr, 3 op/s 2024-05-22T23:00:14.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:14 smithi204 ceph-mon[36211]: pgmap v300: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.1 KiB/s wr, 3 op/s 2024-05-22T23:00:15.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:14 smithi096 ceph-mon[36392]: pgmap v300: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.1 KiB/s wr, 3 op/s 2024-05-22T23:00:16.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:16 smithi084 ceph-mon[32186]: pgmap v301: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 2 op/s 2024-05-22T23:00:16.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:16 smithi204 ceph-mon[36211]: pgmap v301: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 2 op/s 2024-05-22T23:00:17.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:16 smithi096 ceph-mon[36392]: pgmap v301: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 2 op/s 2024-05-22T23:00:17.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.302+0000 7f72f0b09700 1 -- 172.21.15.84:0/2576143165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72ec100950 msgr2=0x7f72ec100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:17.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.302+0000 7f72f0b09700 1 --2- 172.21.15.84:0/2576143165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72ec100950 0x7f72ec100db0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009990 tx=0x7f72e0009ca0 comp rx=0 tx=0).stop 2024-05-22T23:00:17.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.303+0000 7f72f0b09700 1 -- 172.21.15.84:0/2576143165 shutdown_connections 2024-05-22T23:00:17.305 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.303+0000 7f72f0b09700 1 --2- 172.21.15.84:0/2576143165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72ec1012f0 0x7f72ec10f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:17.305 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.303+0000 7f72f0b09700 1 --2- 172.21.15.84:0/2576143165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72ec1083f0 0x7f72ec1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:17.305 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.303+0000 7f72f0b09700 1 --2- 172.21.15.84:0/2576143165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72ec100950 0x7f72ec100db0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:17.305 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.303+0000 7f72f0b09700 1 -- 172.21.15.84:0/2576143165 >> 172.21.15.84:0/2576143165 conn(0x7f72ec0778d0 msgr2=0x7f72ec0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:17.305 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.303+0000 7f72f0b09700 1 -- 172.21.15.84:0/2576143165 shutdown_connections 2024-05-22T23:00:17.305 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.303+0000 7f72f0b09700 1 -- 172.21.15.84:0/2576143165 wait complete. 2024-05-22T23:00:17.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.303+0000 7f72f0b09700 1 Processor -- start 2024-05-22T23:00:17.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.303+0000 7f72f0b09700 1 -- start start 2024-05-22T23:00:17.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.304+0000 7f72f0b09700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72ec100950 0x7f72ec19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:17.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.304+0000 7f72f0b09700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72ec1012f0 0x7f72ec19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:17.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.304+0000 7f72f0b09700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72ec1083f0 0x7f72ec198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:17.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.304+0000 7f72f0b09700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec112730 con 0x7f72ec1083f0 2024-05-22T23:00:17.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.304+0000 7f72f0b09700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec1125b0 con 0x7f72ec1012f0 2024-05-22T23:00:17.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.304+0000 7f72f0b09700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec1128b0 con 0x7f72ec100950 2024-05-22T23:00:17.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.304+0000 7f72ead9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72ec1083f0 0x7f72ec198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:17.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.304+0000 7f72ead9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72ec1083f0 0x7f72ec198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56672/0 (socket says 172.21.15.84:56672) 2024-05-22T23:00:17.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.304+0000 7f72ead9d700 1 -- 172.21.15.84:0/3148042166 learned_addr learned my addr 172.21.15.84:0/3148042166 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:17.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.304+0000 7f72ea59c700 1 --2- 172.21.15.84:0/3148042166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72ec100950 0x7f72ec19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:17.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.304+0000 7f72e9d9b700 1 --2- 172.21.15.84:0/3148042166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72ec1012f0 0x7f72ec19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:17.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.304+0000 7f72ead9d700 1 -- 172.21.15.84:0/3148042166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72ec100950 msgr2=0x7f72ec19e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:17.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.304+0000 7f72ead9d700 1 --2- 172.21.15.84:0/3148042166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72ec100950 0x7f72ec19e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:17.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.304+0000 7f72ead9d700 1 -- 172.21.15.84:0/3148042166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72ec1012f0 msgr2=0x7f72ec19edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:17.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.304+0000 7f72ead9d700 1 --2- 172.21.15.84:0/3148042166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72ec1012f0 0x7f72ec19edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:17.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.304+0000 7f72ead9d700 1 -- 172.21.15.84:0/3148042166 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ec1991d0 con 0x7f72ec1083f0 2024-05-22T23:00:17.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.304+0000 7f72ea59c700 1 --2- 172.21.15.84:0/3148042166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72ec100950 0x7f72ec19e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:00:17.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.304+0000 7f72ead9d700 1 --2- 172.21.15.84:0/3148042166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72ec1083f0 0x7f72ec198a70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f72e400b970 tx=0x7f72e400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:17.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.304+0000 7f72db7fe700 1 -- 172.21.15.84:0/3148042166 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e4019070 con 0x7f72ec1083f0 2024-05-22T23:00:17.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.304+0000 7f72db7fe700 1 -- 172.21.15.84:0/3148042166 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72e4014440 con 0x7f72ec1083f0 2024-05-22T23:00:17.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.304+0000 7f72f0b09700 1 -- 172.21.15.84:0/3148042166 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72ec1994c0 con 0x7f72ec1083f0 2024-05-22T23:00:17.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.304+0000 7f72db7fe700 1 -- 172.21.15.84:0/3148042166 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e40075d0 con 0x7f72ec1083f0 2024-05-22T23:00:17.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.304+0000 7f72f0b09700 1 -- 172.21.15.84:0/3148042166 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72ec1a5780 con 0x7f72ec1083f0 2024-05-22T23:00:17.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.305+0000 7f72f0b09700 1 -- 172.21.15.84:0/3148042166 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f72ec062af0 con 0x7f72ec1083f0 2024-05-22T23:00:17.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.307+0000 7f72db7fe700 1 -- 172.21.15.84:0/3148042166 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f72e401c070 con 0x7f72ec1083f0 2024-05-22T23:00:17.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.307+0000 7f72db7fe700 1 --2- 172.21.15.84:0/3148042166 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f72d40a2d30 0x7f72d40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:17.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.307+0000 7f72ea59c700 1 --2- 172.21.15.84:0/3148042166 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f72d40a2d30 0x7f72d40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:17.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.307+0000 7f72db7fe700 1 -- 172.21.15.84:0/3148042166 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f72e40cead0 con 0x7f72ec1083f0 2024-05-22T23:00:17.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.307+0000 7f72db7fe700 1 --2- 172.21.15.84:0/3148042166 >> [v2:172.21.15.204:6808/4075476986,v1:172.21.15.204:6809/4075476986] conn(0x7f72d40ac510 0x7f72d40ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:17.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.307+0000 7f72db7fe700 1 -- 172.21.15.84:0/3148042166 --> [v2:172.21.15.204:6808/4075476986,v1:172.21.15.204:6809/4075476986] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f72e400de80 con 0x7f72d40ac510 2024-05-22T23:00:17.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.307+0000 7f72db7fe700 1 -- 172.21.15.84:0/3148042166 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f72e40cefb0 con 0x7f72ec1083f0 2024-05-22T23:00:17.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.308+0000 7f72ea59c700 1 --2- 172.21.15.84:0/3148042166 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f72d40a2d30 0x7f72d40a51f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f72ec199bb0 tx=0x7f72dc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:17.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.308+0000 7f72e9d9b700 1 --2- 172.21.15.84:0/3148042166 >> [v2:172.21.15.204:6808/4075476986,v1:172.21.15.204:6809/4075476986] conn(0x7f72d40ac510 0x7f72d40ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:17.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.308+0000 7f72e9d9b700 1 --2- 172.21.15.84:0/3148042166 >> [v2:172.21.15.204:6808/4075476986,v1:172.21.15.204:6809/4075476986] conn(0x7f72d40ac510 0x7f72d40ae930 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:17.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.309+0000 7f72db7fe700 1 -- 172.21.15.84:0/3148042166 <== osd.5 v2:172.21.15.204:6808/4075476986 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f72e400de80 con 0x7f72d40ac510 2024-05-22T23:00:17.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.325+0000 7f72f0b09700 1 -- 172.21.15.84:0/3148042166 --> [v2:172.21.15.204:6808/4075476986,v1:172.21.15.204:6809/4075476986] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f72ec062ef0 con 0x7f72d40ac510 2024-05-22T23:00:17.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.326+0000 7f72db7fe700 1 -- 172.21.15.84:0/3148042166 <== osd.5 v2:172.21.15.204:6808/4075476986 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f72ec062ef0 con 0x7f72d40ac510 2024-05-22T23:00:17.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.326+0000 7f72f0b09700 1 -- 172.21.15.84:0/3148042166 >> [v2:172.21.15.204:6808/4075476986,v1:172.21.15.204:6809/4075476986] conn(0x7f72d40ac510 msgr2=0x7f72d40ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:17.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.326+0000 7f72f0b09700 1 --2- 172.21.15.84:0/3148042166 >> [v2:172.21.15.204:6808/4075476986,v1:172.21.15.204:6809/4075476986] conn(0x7f72d40ac510 0x7f72d40ae930 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:17.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.326+0000 7f72f0b09700 1 -- 172.21.15.84:0/3148042166 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f72d40a2d30 msgr2=0x7f72d40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:17.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.326+0000 7f72f0b09700 1 --2- 172.21.15.84:0/3148042166 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f72d40a2d30 0x7f72d40a51f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f72ec199bb0 tx=0x7f72dc008040 comp rx=0 tx=0).stop 2024-05-22T23:00:17.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.326+0000 7f72f0b09700 1 -- 172.21.15.84:0/3148042166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72ec1083f0 msgr2=0x7f72ec198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:17.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.326+0000 7f72f0b09700 1 --2- 172.21.15.84:0/3148042166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72ec1083f0 0x7f72ec198a70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f72e400b970 tx=0x7f72e400bd30 comp rx=0 tx=0).stop 2024-05-22T23:00:17.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.326+0000 7f72ead9d700 1 -- 172.21.15.84:0/3148042166 reap_dead start 2024-05-22T23:00:17.328 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.326+0000 7f72f0b09700 1 -- 172.21.15.84:0/3148042166 shutdown_connections 2024-05-22T23:00:17.328 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.326+0000 7f72f0b09700 1 -- 172.21.15.84:0/3148042166 >> 172.21.15.84:0/3148042166 conn(0x7f72ec0778d0 msgr2=0x7f72ec0ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:17.328 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.326+0000 7f72f0b09700 1 -- 172.21.15.84:0/3148042166 shutdown_connections 2024-05-22T23:00:17.328 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:17.326+0000 7f72f0b09700 1 -- 172.21.15.84:0/3148042166 wait complete. 2024-05-22T23:00:18.624 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:18 smithi084 ceph-mon[32186]: pgmap v302: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-05-22T23:00:18.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:18 smithi204 ceph-mon[36211]: pgmap v302: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-05-22T23:00:19.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:18 smithi096 ceph-mon[36392]: pgmap v302: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-05-22T23:00:20.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:20 smithi084 ceph-mon[32186]: pgmap v303: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:00:20.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:20 smithi204 ceph-mon[36211]: pgmap v303: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:00:21.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:20 smithi096 ceph-mon[36392]: pgmap v303: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:00:21.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.656+0000 7f9e34376700 1 -- 172.21.15.84:0/2922646743 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e2c06aa10 msgr2=0x7f9e2c10f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:21.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.656+0000 7f9e34376700 1 --2- 172.21.15.84:0/2922646743 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e2c06aa10 0x7f9e2c10f220 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400b0a0 tx=0x7f9e2400b3b0 comp rx=0 tx=0).stop 2024-05-22T23:00:21.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.656+0000 7f9e34376700 1 -- 172.21.15.84:0/2922646743 shutdown_connections 2024-05-22T23:00:21.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.656+0000 7f9e34376700 1 --2- 172.21.15.84:0/2922646743 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e2c06aa10 0x7f9e2c10f220 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:21.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.656+0000 7f9e34376700 1 --2- 172.21.15.84:0/2922646743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e2c1083e0 0x7f9e2c1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:21.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.656+0000 7f9e34376700 1 --2- 172.21.15.84:0/2922646743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e2c06a070 0x7f9e2c06a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:21.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.656+0000 7f9e34376700 1 -- 172.21.15.84:0/2922646743 >> 172.21.15.84:0/2922646743 conn(0x7f9e2c078210 msgr2=0x7f9e2c078620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:21.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.656+0000 7f9e34376700 1 -- 172.21.15.84:0/2922646743 shutdown_connections 2024-05-22T23:00:21.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.656+0000 7f9e34376700 1 -- 172.21.15.84:0/2922646743 wait complete. 2024-05-22T23:00:21.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.656+0000 7f9e34376700 1 Processor -- start 2024-05-22T23:00:21.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.657+0000 7f9e34376700 1 -- start start 2024-05-22T23:00:21.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.657+0000 7f9e34376700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e2c06a070 0x7f9e2c19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:21.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.657+0000 7f9e34376700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e2c06aa10 0x7f9e2c19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:21.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.657+0000 7f9e34376700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e2c1083e0 0x7f9e2c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:21.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.657+0000 7f9e34376700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c112630 con 0x7f9e2c06a070 2024-05-22T23:00:21.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.657+0000 7f9e34376700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c1124b0 con 0x7f9e2c06aa10 2024-05-22T23:00:21.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.657+0000 7f9e34376700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c1127b0 con 0x7f9e2c1083e0 2024-05-22T23:00:21.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.657+0000 7f9e32112700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e2c06a070 0x7f9e2c19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:21.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.657+0000 7f9e32112700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e2c06a070 0x7f9e2c19e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41580/0 (socket says 172.21.15.84:41580) 2024-05-22T23:00:21.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.657+0000 7f9e32112700 1 -- 172.21.15.84:0/2852513190 learned_addr learned my addr 172.21.15.84:0/2852513190 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:21.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.657+0000 7f9e31911700 1 --2- 172.21.15.84:0/2852513190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e2c06aa10 0x7f9e2c19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:21.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.657+0000 7f9e32112700 1 -- 172.21.15.84:0/2852513190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e2c1083e0 msgr2=0x7f9e2c198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:21.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.657+0000 7f9e32112700 1 --2- 172.21.15.84:0/2852513190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e2c1083e0 0x7f9e2c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:21.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.657+0000 7f9e32112700 1 -- 172.21.15.84:0/2852513190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e2c06aa10 msgr2=0x7f9e2c19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:21.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.657+0000 7f9e32112700 1 --2- 172.21.15.84:0/2852513190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e2c06aa10 0x7f9e2c19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:21.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.657+0000 7f9e32112700 1 -- 172.21.15.84:0/2852513190 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e2c199050 con 0x7f9e2c06a070 2024-05-22T23:00:21.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.657+0000 7f9e31911700 1 --2- 172.21.15.84:0/2852513190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e2c06aa10 0x7f9e2c19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:00:21.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.657+0000 7f9e32112700 1 --2- 172.21.15.84:0/2852513190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e2c06a070 0x7f9e2c19e6a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00c9b0 tx=0x7f9e1c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:21.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.657+0000 7f9e237fe700 1 -- 172.21.15.84:0/2852513190 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c007b00 con 0x7f9e2c06a070 2024-05-22T23:00:21.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.658+0000 7f9e34376700 1 -- 172.21.15.84:0/2852513190 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e2c199340 con 0x7f9e2c06a070 2024-05-22T23:00:21.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.658+0000 7f9e237fe700 1 -- 172.21.15.84:0/2852513190 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e1c007ca0 con 0x7f9e2c06a070 2024-05-22T23:00:21.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.658+0000 7f9e237fe700 1 -- 172.21.15.84:0/2852513190 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c018780 con 0x7f9e2c06a070 2024-05-22T23:00:21.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.658+0000 7f9e34376700 1 -- 172.21.15.84:0/2852513190 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e2c1a5500 con 0x7f9e2c06a070 2024-05-22T23:00:21.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.658+0000 7f9e34376700 1 -- 172.21.15.84:0/2852513190 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9e2c062ab0 con 0x7f9e2c06a070 2024-05-22T23:00:21.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.660+0000 7f9e237fe700 1 -- 172.21.15.84:0/2852513190 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9e1c00f450 con 0x7f9e2c06a070 2024-05-22T23:00:21.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.660+0000 7f9e237fe700 1 --2- 172.21.15.84:0/2852513190 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9e180a2cb0 0x7f9e180a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:21.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.660+0000 7f9e31911700 1 --2- 172.21.15.84:0/2852513190 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9e180a2cb0 0x7f9e180a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:21.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.660+0000 7f9e237fe700 1 -- 172.21.15.84:0/2852513190 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f9e1c0cf470 con 0x7f9e2c06a070 2024-05-22T23:00:21.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.661+0000 7f9e237fe700 1 --2- 172.21.15.84:0/2852513190 >> [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826] conn(0x7f9e180ac490 0x7f9e180ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:21.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.661+0000 7f9e237fe700 1 -- 172.21.15.84:0/2852513190 --> [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9e1c015e70 con 0x7f9e180ac490 2024-05-22T23:00:21.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.661+0000 7f9e237fe700 1 -- 172.21.15.84:0/2852513190 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f9e1c0cf950 con 0x7f9e2c06a070 2024-05-22T23:00:21.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.661+0000 7f9e31911700 1 --2- 172.21.15.84:0/2852513190 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9e180a2cb0 0x7f9e180a5170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9e28005fd0 tx=0x7f9e28005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:21.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.661+0000 7f9e32913700 1 --2- 172.21.15.84:0/2852513190 >> [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826] conn(0x7f9e180ac490 0x7f9e180ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:21.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.661+0000 7f9e32913700 1 --2- 172.21.15.84:0/2852513190 >> [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826] conn(0x7f9e180ac490 0x7f9e180ae8b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:21.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.662+0000 7f9e237fe700 1 -- 172.21.15.84:0/2852513190 <== osd.0 v2:172.21.15.84:6802/1559945826 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9e1c015e70 con 0x7f9e180ac490 2024-05-22T23:00:21.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.678+0000 7f9e34376700 1 -- 172.21.15.84:0/2852513190 --> [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9e2c062ee0 con 0x7f9e180ac490 2024-05-22T23:00:21.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.678+0000 7f9e237fe700 1 -- 172.21.15.84:0/2852513190 <== osd.0 v2:172.21.15.84:6802/1559945826 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9e2c062ee0 con 0x7f9e180ac490 2024-05-22T23:00:21.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.679+0000 7f9e34376700 1 -- 172.21.15.84:0/2852513190 >> [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826] conn(0x7f9e180ac490 msgr2=0x7f9e180ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:21.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.679+0000 7f9e34376700 1 --2- 172.21.15.84:0/2852513190 >> [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826] conn(0x7f9e180ac490 0x7f9e180ae8b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:21.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.679+0000 7f9e34376700 1 -- 172.21.15.84:0/2852513190 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9e180a2cb0 msgr2=0x7f9e180a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:21.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.679+0000 7f9e34376700 1 --2- 172.21.15.84:0/2852513190 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9e180a2cb0 0x7f9e180a5170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9e28005fd0 tx=0x7f9e28005dc0 comp rx=0 tx=0).stop 2024-05-22T23:00:21.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.679+0000 7f9e34376700 1 -- 172.21.15.84:0/2852513190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e2c06a070 msgr2=0x7f9e2c19e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:21.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.679+0000 7f9e34376700 1 --2- 172.21.15.84:0/2852513190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e2c06a070 0x7f9e2c19e6a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00c9b0 tx=0x7f9e1c00cd70 comp rx=0 tx=0).stop 2024-05-22T23:00:21.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.679+0000 7f9e32913700 1 -- 172.21.15.84:0/2852513190 reap_dead start 2024-05-22T23:00:21.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.679+0000 7f9e34376700 1 -- 172.21.15.84:0/2852513190 shutdown_connections 2024-05-22T23:00:21.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.679+0000 7f9e34376700 1 -- 172.21.15.84:0/2852513190 >> 172.21.15.84:0/2852513190 conn(0x7f9e2c078210 msgr2=0x7f9e2c0ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:21.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.679+0000 7f9e34376700 1 -- 172.21.15.84:0/2852513190 shutdown_connections 2024-05-22T23:00:21.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:21.679+0000 7f9e34376700 1 -- 172.21.15.84:0/2852513190 wait complete. 2024-05-22T23:00:22.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:22 smithi084 ceph-mon[32186]: pgmap v304: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:00:22.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:22 smithi204 ceph-mon[36211]: pgmap v304: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:00:23.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:22 smithi096 ceph-mon[36392]: pgmap v304: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:00:24.658 INFO:teuthology.orchestra.run.smithi084.stdout:163208757299 2024-05-22T23:00:24.658 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-22T23:00:24.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.715+0000 7f0c79a7d700 1 -- 172.21.15.84:0/2035777930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c74100950 msgr2=0x7f0c74100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:24.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.715+0000 7f0c79a7d700 1 --2- 172.21.15.84:0/2035777930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c74100950 0x7f0c74100db0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0c68009990 tx=0x7f0c68009ca0 comp rx=0 tx=0).stop 2024-05-22T23:00:24.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.715+0000 7f0c79a7d700 1 -- 172.21.15.84:0/2035777930 shutdown_connections 2024-05-22T23:00:24.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.715+0000 7f0c79a7d700 1 --2- 172.21.15.84:0/2035777930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c741012f0 0x7f0c7410f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:24.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.715+0000 7f0c79a7d700 1 --2- 172.21.15.84:0/2035777930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c741083f0 0x7f0c741087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:24.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.715+0000 7f0c79a7d700 1 --2- 172.21.15.84:0/2035777930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c74100950 0x7f0c74100db0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:24.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.715+0000 7f0c79a7d700 1 -- 172.21.15.84:0/2035777930 >> 172.21.15.84:0/2035777930 conn(0x7f0c740778d0 msgr2=0x7f0c740fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:24.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.715+0000 7f0c79a7d700 1 -- 172.21.15.84:0/2035777930 shutdown_connections 2024-05-22T23:00:24.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.715+0000 7f0c79a7d700 1 -- 172.21.15.84:0/2035777930 wait complete. 2024-05-22T23:00:24.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.716+0000 7f0c79a7d700 1 Processor -- start 2024-05-22T23:00:24.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.716+0000 7f0c79a7d700 1 -- start start 2024-05-22T23:00:24.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.717+0000 7f0c79a7d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c74100950 0x7f0c74101b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:24.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.717+0000 7f0c79a7d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c741012f0 0x7f0c74102070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:24.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.717+0000 7f0c79a7d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c741083f0 0x7f0c741025b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:24.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.717+0000 7f0c79a7d700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0c74112730 con 0x7f0c741012f0 2024-05-22T23:00:24.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.717+0000 7f0c79a7d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0c741125b0 con 0x7f0c74100950 2024-05-22T23:00:24.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.717+0000 7f0c79a7d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c741128b0 con 0x7f0c741083f0 2024-05-22T23:00:24.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.717+0000 7f0c72ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c741012f0 0x7f0c74102070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:24.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.717+0000 7f0c72ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c741012f0 0x7f0c74102070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41594/0 (socket says 172.21.15.84:41594) 2024-05-22T23:00:24.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.717+0000 7f0c72ffd700 1 -- 172.21.15.84:0/293589561 learned_addr learned my addr 172.21.15.84:0/293589561 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:24.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.717+0000 7f0c737fe700 1 --2- 172.21.15.84:0/293589561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c74100950 0x7f0c74101b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:24.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.717+0000 7f0c72ffd700 1 -- 172.21.15.84:0/293589561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c741083f0 msgr2=0x7f0c741025b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:24.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.717+0000 7f0c72ffd700 1 --2- 172.21.15.84:0/293589561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c741083f0 0x7f0c741025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:24.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.717+0000 7f0c72ffd700 1 -- 172.21.15.84:0/293589561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c74100950 msgr2=0x7f0c74101b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:24.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.717+0000 7f0c72ffd700 1 --2- 172.21.15.84:0/293589561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c74100950 0x7f0c74101b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:24.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.717+0000 7f0c72ffd700 1 -- 172.21.15.84:0/293589561 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c741a7190 con 0x7f0c741012f0 2024-05-22T23:00:24.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.717+0000 7f0c72ffd700 1 --2- 172.21.15.84:0/293589561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c741012f0 0x7f0c74102070 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0c68000c00 tx=0x7f0c68003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:24.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.717+0000 7f0c70ff9700 1 -- 172.21.15.84:0/293589561 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c68027070 con 0x7f0c741012f0 2024-05-22T23:00:24.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.717+0000 7f0c70ff9700 1 -- 172.21.15.84:0/293589561 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c68004190 con 0x7f0c741012f0 2024-05-22T23:00:24.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.717+0000 7f0c79a7d700 1 -- 172.21.15.84:0/293589561 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c741a7420 con 0x7f0c741012f0 2024-05-22T23:00:24.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.717+0000 7f0c70ff9700 1 -- 172.21.15.84:0/293589561 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6800f460 con 0x7f0c741012f0 2024-05-22T23:00:24.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.717+0000 7f0c79a7d700 1 -- 172.21.15.84:0/293589561 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c741a7870 con 0x7f0c741012f0 2024-05-22T23:00:24.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.718+0000 7f0c79a7d700 1 -- 172.21.15.84:0/293589561 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0c74062a70 con 0x7f0c741012f0 2024-05-22T23:00:24.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.720+0000 7f0c70ff9700 1 -- 172.21.15.84:0/293589561 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0c6802a070 con 0x7f0c741012f0 2024-05-22T23:00:24.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.720+0000 7f0c70ff9700 1 --2- 172.21.15.84:0/293589561 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0c5c0a2cb0 0x7f0c5c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:24.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.720+0000 7f0c737fe700 1 --2- 172.21.15.84:0/293589561 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0c5c0a2cb0 0x7f0c5c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:24.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.720+0000 7f0c70ff9700 1 -- 172.21.15.84:0/293589561 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f0c680d0720 con 0x7f0c741012f0 2024-05-22T23:00:24.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.720+0000 7f0c70ff9700 1 --2- 172.21.15.84:0/293589561 >> [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648] conn(0x7f0c5c0ac490 0x7f0c5c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:24.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.720+0000 7f0c70ff9700 1 -- 172.21.15.84:0/293589561 --> [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0c68001d30 con 0x7f0c5c0ac490 2024-05-22T23:00:24.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.720+0000 7f0c70ff9700 1 -- 172.21.15.84:0/293589561 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f0c680d0c00 con 0x7f0c741012f0 2024-05-22T23:00:24.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.721+0000 7f0c73fff700 1 --2- 172.21.15.84:0/293589561 >> [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648] conn(0x7f0c5c0ac490 0x7f0c5c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:24.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.721+0000 7f0c737fe700 1 --2- 172.21.15.84:0/293589561 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0c5c0a2cb0 0x7f0c5c0a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0c64006fd0 tx=0x7f0c64008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:24.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.721+0000 7f0c73fff700 1 --2- 172.21.15.84:0/293589561 >> [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648] conn(0x7f0c5c0ac490 0x7f0c5c0ae8b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:24.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.722+0000 7f0c70ff9700 1 -- 172.21.15.84:0/293589561 <== osd.3 v2:172.21.15.84:6810/2869340648 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0c68001d30 con 0x7f0c5c0ac490 2024-05-22T23:00:24.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.738+0000 7f0c79a7d700 1 -- 172.21.15.84:0/293589561 --> [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0c74062e70 con 0x7f0c5c0ac490 2024-05-22T23:00:24.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.738+0000 7f0c70ff9700 1 -- 172.21.15.84:0/293589561 <== osd.3 v2:172.21.15.84:6810/2869340648 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0c74062e70 con 0x7f0c5c0ac490 2024-05-22T23:00:24.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.738+0000 7f0c79a7d700 1 -- 172.21.15.84:0/293589561 >> [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648] conn(0x7f0c5c0ac490 msgr2=0x7f0c5c0ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:24.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.738+0000 7f0c79a7d700 1 --2- 172.21.15.84:0/293589561 >> [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648] conn(0x7f0c5c0ac490 0x7f0c5c0ae8b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:24.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.738+0000 7f0c79a7d700 1 -- 172.21.15.84:0/293589561 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0c5c0a2cb0 msgr2=0x7f0c5c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:24.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.738+0000 7f0c79a7d700 1 --2- 172.21.15.84:0/293589561 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0c5c0a2cb0 0x7f0c5c0a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0c64006fd0 tx=0x7f0c64008040 comp rx=0 tx=0).stop 2024-05-22T23:00:24.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.738+0000 7f0c79a7d700 1 -- 172.21.15.84:0/293589561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c741012f0 msgr2=0x7f0c74102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:24.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.738+0000 7f0c79a7d700 1 --2- 172.21.15.84:0/293589561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c741012f0 0x7f0c74102070 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0c68000c00 tx=0x7f0c68003730 comp rx=0 tx=0).stop 2024-05-22T23:00:24.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.738+0000 7f0c73fff700 1 -- 172.21.15.84:0/293589561 reap_dead start 2024-05-22T23:00:24.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.738+0000 7f0c79a7d700 1 -- 172.21.15.84:0/293589561 shutdown_connections 2024-05-22T23:00:24.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.738+0000 7f0c79a7d700 1 -- 172.21.15.84:0/293589561 >> 172.21.15.84:0/293589561 conn(0x7f0c740778d0 msgr2=0x7f0c740ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:24.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.738+0000 7f0c79a7d700 1 -- 172.21.15.84:0/293589561 shutdown_connections 2024-05-22T23:00:24.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:23.738+0000 7f0c79a7d700 1 -- 172.21.15.84:0/293589561 wait complete. 2024-05-22T23:00:24.850 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:24 smithi096 ceph-mon[36392]: pgmap v305: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:00:24.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:24 smithi084 ceph-mon[32186]: pgmap v305: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:00:24.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:24 smithi204 ceph-mon[36211]: pgmap v305: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:00:25.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.757+0000 7fd447646700 1 -- 172.21.15.84:0/2893221663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4401076a0 msgr2=0x7fd44010f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:25.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.757+0000 7fd447646700 1 --2- 172.21.15.84:0/2893221663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4401076a0 0x7fd44010f270 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd43800b0a0 tx=0x7fd43800b3b0 comp rx=0 tx=0).stop 2024-05-22T23:00:25.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.757+0000 7fd447646700 1 -- 172.21.15.84:0/2893221663 shutdown_connections 2024-05-22T23:00:25.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.757+0000 7fd447646700 1 --2- 172.21.15.84:0/2893221663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4401076a0 0x7fd44010f270 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:25.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.757+0000 7fd447646700 1 --2- 172.21.15.84:0/2893221663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd440102610 0x7fd4401029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:25.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.757+0000 7fd447646700 1 --2- 172.21.15.84:0/2893221663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd440102fc0 0x7fd440106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:25.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.757+0000 7fd447646700 1 -- 172.21.15.84:0/2893221663 >> 172.21.15.84:0/2893221663 conn(0x7fd4400794c0 msgr2=0x7fd4400798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:25.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.757+0000 7fd447646700 1 -- 172.21.15.84:0/2893221663 shutdown_connections 2024-05-22T23:00:25.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.757+0000 7fd447646700 1 -- 172.21.15.84:0/2893221663 wait complete. 2024-05-22T23:00:25.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.757+0000 7fd447646700 1 Processor -- start 2024-05-22T23:00:25.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.758+0000 7fd447646700 1 -- start start 2024-05-22T23:00:25.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.758+0000 7fd447646700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd440102610 0x7fd44019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:25.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.758+0000 7fd447646700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd440102fc0 0x7fd44019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:25.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.758+0000 7fd447646700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4401076a0 0x7fd4401988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:25.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.758+0000 7fd447646700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd440112680 con 0x7fd4401076a0 2024-05-22T23:00:25.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.758+0000 7fd447646700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd440112500 con 0x7fd440102fc0 2024-05-22T23:00:25.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.758+0000 7fd447646700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd440112800 con 0x7fd440102610 2024-05-22T23:00:25.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.758+0000 7fd445be3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4401076a0 0x7fd4401988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:25.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.758+0000 7fd445be3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4401076a0 0x7fd4401988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41618/0 (socket says 172.21.15.84:41618) 2024-05-22T23:00:25.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.758+0000 7fd445be3700 1 -- 172.21.15.84:0/1162016941 learned_addr learned my addr 172.21.15.84:0/1162016941 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:25.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.758+0000 7fd4453e2700 1 --2- 172.21.15.84:0/1162016941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd440102610 0x7fd44019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:25.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.758+0000 7fd445be3700 1 -- 172.21.15.84:0/1162016941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd440102610 msgr2=0x7fd44019e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:25.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.758+0000 7fd445be3700 1 --2- 172.21.15.84:0/1162016941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd440102610 0x7fd44019e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:25.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.758+0000 7fd445be3700 1 -- 172.21.15.84:0/1162016941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd440102fc0 msgr2=0x7fd44019ec10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-22T23:00:25.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.759+0000 7fd444be1700 1 --2- 172.21.15.84:0/1162016941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd440102fc0 0x7fd44019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:25.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.759+0000 7fd445be3700 1 --2- 172.21.15.84:0/1162016941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd440102fc0 0x7fd44019ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:25.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.759+0000 7fd445be3700 1 -- 172.21.15.84:0/1162016941 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd440199080 con 0x7fd4401076a0 2024-05-22T23:00:25.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.759+0000 7fd4453e2700 1 --2- 172.21.15.84:0/1162016941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd440102610 0x7fd44019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:00:25.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.759+0000 7fd445be3700 1 --2- 172.21.15.84:0/1162016941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4401076a0 0x7fd4401988a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd43800b780 tx=0x7fd438004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:25.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.759+0000 7fd4367fc700 1 -- 172.21.15.84:0/1162016941 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438021070 con 0x7fd4401076a0 2024-05-22T23:00:25.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.759+0000 7fd4367fc700 1 -- 172.21.15.84:0/1162016941 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd438007c80 con 0x7fd4401076a0 2024-05-22T23:00:25.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.759+0000 7fd4367fc700 1 -- 172.21.15.84:0/1162016941 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438005400 con 0x7fd4401076a0 2024-05-22T23:00:25.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.759+0000 7fd447646700 1 -- 172.21.15.84:0/1162016941 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd440199310 con 0x7fd4401076a0 2024-05-22T23:00:25.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.759+0000 7fd447646700 1 -- 172.21.15.84:0/1162016941 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4401a5530 con 0x7fd4401076a0 2024-05-22T23:00:25.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.760+0000 7fd447646700 1 -- 172.21.15.84:0/1162016941 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd440062af0 con 0x7fd4401076a0 2024-05-22T23:00:25.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.761+0000 7fd4367fc700 1 -- 172.21.15.84:0/1162016941 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd438024070 con 0x7fd4401076a0 2024-05-22T23:00:25.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.762+0000 7fd4367fc700 1 --2- 172.21.15.84:0/1162016941 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fd42c0a2cb0 0x7fd42c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:25.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.762+0000 7fd4453e2700 1 --2- 172.21.15.84:0/1162016941 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fd42c0a2cb0 0x7fd42c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:25.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.762+0000 7fd4367fc700 1 -- 172.21.15.84:0/1162016941 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd43800f020 con 0x7fd4401076a0 2024-05-22T23:00:25.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.762+0000 7fd4367fc700 1 --2- 172.21.15.84:0/1162016941 >> [v2:172.21.15.96:6816/3195792774,v1:172.21.15.96:6817/3195792774] conn(0x7fd42c0ac490 0x7fd42c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:25.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.762+0000 7fd4367fc700 1 -- 172.21.15.84:0/1162016941 --> [v2:172.21.15.96:6816/3195792774,v1:172.21.15.96:6817/3195792774] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd43802be50 con 0x7fd42c0ac490 2024-05-22T23:00:25.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.762+0000 7fd4367fc700 1 -- 172.21.15.84:0/1162016941 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fd4380d0f60 con 0x7fd4401076a0 2024-05-22T23:00:25.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.763+0000 7fd444be1700 1 --2- 172.21.15.84:0/1162016941 >> [v2:172.21.15.96:6816/3195792774,v1:172.21.15.96:6817/3195792774] conn(0x7fd42c0ac490 0x7fd42c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:25.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.763+0000 7fd4453e2700 1 --2- 172.21.15.84:0/1162016941 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fd42c0a2cb0 0x7fd42c0a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd430009cb0 tx=0x7fd430009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:25.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.763+0000 7fd444be1700 1 --2- 172.21.15.84:0/1162016941 >> [v2:172.21.15.96:6816/3195792774,v1:172.21.15.96:6817/3195792774] conn(0x7fd42c0ac490 0x7fd42c0ae8b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:25.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.764+0000 7fd4367fc700 1 -- 172.21.15.84:0/1162016941 <== osd.7 v2:172.21.15.96:6816/3195792774 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd43802be50 con 0x7fd42c0ac490 2024-05-22T23:00:25.780 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.780+0000 7fd447646700 1 -- 172.21.15.84:0/1162016941 --> [v2:172.21.15.96:6816/3195792774,v1:172.21.15.96:6817/3195792774] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd440062f70 con 0x7fd42c0ac490 2024-05-22T23:00:25.781 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.780+0000 7fd4367fc700 1 -- 172.21.15.84:0/1162016941 <== osd.7 v2:172.21.15.96:6816/3195792774 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd440062f70 con 0x7fd42c0ac490 2024-05-22T23:00:25.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.780+0000 7fd447646700 1 -- 172.21.15.84:0/1162016941 >> [v2:172.21.15.96:6816/3195792774,v1:172.21.15.96:6817/3195792774] conn(0x7fd42c0ac490 msgr2=0x7fd42c0ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:25.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.780+0000 7fd447646700 1 --2- 172.21.15.84:0/1162016941 >> [v2:172.21.15.96:6816/3195792774,v1:172.21.15.96:6817/3195792774] conn(0x7fd42c0ac490 0x7fd42c0ae8b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:25.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.781+0000 7fd447646700 1 -- 172.21.15.84:0/1162016941 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fd42c0a2cb0 msgr2=0x7fd42c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:25.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.781+0000 7fd447646700 1 --2- 172.21.15.84:0/1162016941 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fd42c0a2cb0 0x7fd42c0a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd430009cb0 tx=0x7fd430009260 comp rx=0 tx=0).stop 2024-05-22T23:00:25.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.781+0000 7fd447646700 1 -- 172.21.15.84:0/1162016941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4401076a0 msgr2=0x7fd4401988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:25.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.781+0000 7fd447646700 1 --2- 172.21.15.84:0/1162016941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4401076a0 0x7fd4401988a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd43800b780 tx=0x7fd438004db0 comp rx=0 tx=0).stop 2024-05-22T23:00:25.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.781+0000 7fd445be3700 1 -- 172.21.15.84:0/1162016941 reap_dead start 2024-05-22T23:00:25.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.781+0000 7fd447646700 1 -- 172.21.15.84:0/1162016941 shutdown_connections 2024-05-22T23:00:25.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.781+0000 7fd447646700 1 -- 172.21.15.84:0/1162016941 >> 172.21.15.84:0/1162016941 conn(0x7fd4400794c0 msgr2=0x7fd440100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:25.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.781+0000 7fd447646700 1 -- 172.21.15.84:0/1162016941 shutdown_connections 2024-05-22T23:00:25.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.781+0000 7fd447646700 1 -- 172.21.15.84:0/1162016941 wait complete. 2024-05-22T23:00:25.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.948+0000 7f29465b7700 1 -- 172.21.15.84:0/1387493917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f294006b6e0 msgr2=0x7f2940106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:25.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.948+0000 7f29465b7700 1 --2- 172.21.15.84:0/1387493917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f294006b6e0 0x7f2940106fe0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f29340099c0 tx=0x7f2934009cd0 comp rx=0 tx=0).stop 2024-05-22T23:00:25.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.948+0000 7f29465b7700 1 -- 172.21.15.84:0/1387493917 shutdown_connections 2024-05-22T23:00:25.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.948+0000 7f29465b7700 1 --2- 172.21.15.84:0/1387493917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2940107710 0x7f294010f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:25.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.948+0000 7f29465b7700 1 --2- 172.21.15.84:0/1387493917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f294006ad30 0x7f294006b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:25.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.948+0000 7f29465b7700 1 --2- 172.21.15.84:0/1387493917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f294006b6e0 0x7f2940106fe0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:25.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.948+0000 7f29465b7700 1 -- 172.21.15.84:0/1387493917 >> 172.21.15.84:0/1387493917 conn(0x7f29400772d0 msgr2=0x7f29400776e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:25.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.948+0000 7f29465b7700 1 -- 172.21.15.84:0/1387493917 shutdown_connections 2024-05-22T23:00:25.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.948+0000 7f29465b7700 1 -- 172.21.15.84:0/1387493917 wait complete. 2024-05-22T23:00:25.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.948+0000 7f29465b7700 1 Processor -- start 2024-05-22T23:00:25.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.948+0000 7f29465b7700 1 -- start start 2024-05-22T23:00:25.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.949+0000 7f29465b7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f294006ad30 0x7f2940101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:25.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.949+0000 7f29465b7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f294006b6e0 0x7f29401018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:25.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.949+0000 7f29465b7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2940107710 0x7f2940101de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:25.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.949+0000 7f29465b7700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2940112680 con 0x7f2940107710 2024-05-22T23:00:25.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.949+0000 7f29465b7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2940112500 con 0x7f294006ad30 2024-05-22T23:00:25.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.949+0000 7f29465b7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2940112800 con 0x7f294006b6e0 2024-05-22T23:00:25.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.949+0000 7f2944b54700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2940107710 0x7f2940101de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:25.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.949+0000 7f2944b54700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2940107710 0x7f2940101de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41644/0 (socket says 172.21.15.84:41644) 2024-05-22T23:00:25.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.949+0000 7f2944b54700 1 -- 172.21.15.84:0/1910565081 learned_addr learned my addr 172.21.15.84:0/1910565081 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:25.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.949+0000 7f293ffff700 1 --2- 172.21.15.84:0/1910565081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f294006ad30 0x7f2940101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:25.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.949+0000 7f293f7fe700 1 --2- 172.21.15.84:0/1910565081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f294006b6e0 0x7f29401018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:25.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.949+0000 7f2944b54700 1 -- 172.21.15.84:0/1910565081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f294006b6e0 msgr2=0x7f29401018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:25.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.949+0000 7f2944b54700 1 --2- 172.21.15.84:0/1910565081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f294006b6e0 0x7f29401018a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:25.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.949+0000 7f2944b54700 1 -- 172.21.15.84:0/1910565081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f294006ad30 msgr2=0x7f2940101360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:25.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.949+0000 7f2944b54700 1 --2- 172.21.15.84:0/1910565081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f294006ad30 0x7f2940101360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:25.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.949+0000 7f2944b54700 1 -- 172.21.15.84:0/1910565081 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29401a70a0 con 0x7f2940107710 2024-05-22T23:00:25.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.949+0000 7f2944b54700 1 --2- 172.21.15.84:0/1910565081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2940107710 0x7f2940101de0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f293800ef30 tx=0x7f293800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:25.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.950+0000 7f293d7fa700 1 -- 172.21.15.84:0/1910565081 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29380092e0 con 0x7f2940107710 2024-05-22T23:00:25.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.950+0000 7f293d7fa700 1 -- 172.21.15.84:0/1910565081 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2938004bb0 con 0x7f2940107710 2024-05-22T23:00:25.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.950+0000 7f293d7fa700 1 -- 172.21.15.84:0/1910565081 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938004280 con 0x7f2940107710 2024-05-22T23:00:25.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.950+0000 7f29465b7700 1 -- 172.21.15.84:0/1910565081 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29401a7330 con 0x7f2940107710 2024-05-22T23:00:25.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.950+0000 7f29465b7700 1 -- 172.21.15.84:0/1910565081 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29401a7820 con 0x7f2940107710 2024-05-22T23:00:25.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.951+0000 7f29465b7700 1 -- 172.21.15.84:0/1910565081 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2940062a70 con 0x7f2940107710 2024-05-22T23:00:25.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.952+0000 7f293d7fa700 1 -- 172.21.15.84:0/1910565081 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f293801b030 con 0x7f2940107710 2024-05-22T23:00:25.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.952+0000 7f293d7fa700 1 --2- 172.21.15.84:0/1910565081 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f29280a2cb0 0x7f29280a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:25.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.953+0000 7f293ffff700 1 --2- 172.21.15.84:0/1910565081 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f29280a2cb0 0x7f29280a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:25.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.953+0000 7f293d7fa700 1 -- 172.21.15.84:0/1910565081 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f29380cf470 con 0x7f2940107710 2024-05-22T23:00:25.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.953+0000 7f293d7fa700 1 --2- 172.21.15.84:0/1910565081 >> [v2:172.21.15.84:6826/2786189012,v1:172.21.15.84:6827/2786189012] conn(0x7f29280ac490 0x7f29280ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:25.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.953+0000 7f293d7fa700 1 -- 172.21.15.84:0/1910565081 --> [v2:172.21.15.84:6826/2786189012,v1:172.21.15.84:6827/2786189012] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f293801fe50 con 0x7f29280ac490 2024-05-22T23:00:25.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.953+0000 7f293d7fa700 1 -- 172.21.15.84:0/1910565081 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f29380cf950 con 0x7f2940107710 2024-05-22T23:00:25.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.953+0000 7f293f7fe700 1 --2- 172.21.15.84:0/1910565081 >> [v2:172.21.15.84:6826/2786189012,v1:172.21.15.84:6827/2786189012] conn(0x7f29280ac490 0x7f29280ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:25.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.953+0000 7f293ffff700 1 --2- 172.21.15.84:0/1910565081 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f29280a2cb0 0x7f29280a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2930005950 tx=0x7f293000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:25.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.953+0000 7f293f7fe700 1 --2- 172.21.15.84:0/1910565081 >> [v2:172.21.15.84:6826/2786189012,v1:172.21.15.84:6827/2786189012] conn(0x7f29280ac490 0x7f29280ae8b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:25.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.954+0000 7f293d7fa700 1 -- 172.21.15.84:0/1910565081 <== osd.9 v2:172.21.15.84:6826/2786189012 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f293801fe50 con 0x7f29280ac490 2024-05-22T23:00:25.976 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.975+0000 7f29465b7700 1 -- 172.21.15.84:0/1910565081 --> [v2:172.21.15.84:6826/2786189012,v1:172.21.15.84:6827/2786189012] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2940062ef0 con 0x7f29280ac490 2024-05-22T23:00:25.976 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.976+0000 7f293d7fa700 1 -- 172.21.15.84:0/1910565081 <== osd.9 v2:172.21.15.84:6826/2786189012 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2940062ef0 con 0x7f29280ac490 2024-05-22T23:00:25.977 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.976+0000 7f29465b7700 1 -- 172.21.15.84:0/1910565081 >> [v2:172.21.15.84:6826/2786189012,v1:172.21.15.84:6827/2786189012] conn(0x7f29280ac490 msgr2=0x7f29280ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:25.977 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.976+0000 7f29465b7700 1 --2- 172.21.15.84:0/1910565081 >> [v2:172.21.15.84:6826/2786189012,v1:172.21.15.84:6827/2786189012] conn(0x7f29280ac490 0x7f29280ae8b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:25.977 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.976+0000 7f29465b7700 1 -- 172.21.15.84:0/1910565081 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f29280a2cb0 msgr2=0x7f29280a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:25.977 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.976+0000 7f29465b7700 1 --2- 172.21.15.84:0/1910565081 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f29280a2cb0 0x7f29280a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2930005950 tx=0x7f293000a300 comp rx=0 tx=0).stop 2024-05-22T23:00:25.978 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.976+0000 7f29465b7700 1 -- 172.21.15.84:0/1910565081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2940107710 msgr2=0x7f2940101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:25.978 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.976+0000 7f29465b7700 1 --2- 172.21.15.84:0/1910565081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2940107710 0x7f2940101de0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f293800ef30 tx=0x7f293800c3a0 comp rx=0 tx=0).stop 2024-05-22T23:00:25.978 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.976+0000 7f2944b54700 1 -- 172.21.15.84:0/1910565081 reap_dead start 2024-05-22T23:00:25.978 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.976+0000 7f29465b7700 1 -- 172.21.15.84:0/1910565081 shutdown_connections 2024-05-22T23:00:25.978 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.976+0000 7f29465b7700 1 -- 172.21.15.84:0/1910565081 >> 172.21.15.84:0/1910565081 conn(0x7f29400772d0 msgr2=0x7f29400ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:25.978 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.976+0000 7f29465b7700 1 -- 172.21.15.84:0/1910565081 shutdown_connections 2024-05-22T23:00:25.978 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:25.976+0000 7f29465b7700 1 -- 172.21.15.84:0/1910565081 wait complete. 2024-05-22T23:00:26.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:26 smithi084 ceph-mon[32186]: pgmap v306: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:00:26.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:26 smithi204 ceph-mon[36211]: pgmap v306: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:00:27.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:26 smithi096 ceph-mon[36392]: pgmap v306: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:00:27.952 INFO:teuthology.orchestra.run.smithi084.stdout:34359738444 2024-05-22T23:00:27.953 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-22T23:00:28.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:28 smithi084 ceph-mon[32186]: pgmap v307: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:00:28.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:28 smithi204 ceph-mon[36211]: pgmap v307: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:00:29.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:28 smithi096 ceph-mon[36392]: pgmap v307: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:00:29.074 INFO:teuthology.orchestra.run.smithi084.stdout:223338299434 2024-05-22T23:00:29.075 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-22T23:00:29.698 INFO:teuthology.orchestra.run.smithi084.stdout:292057776160 2024-05-22T23:00:29.699 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-22T23:00:29.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.928+0000 7f00278bb700 1 -- 172.21.15.84:0/2905023664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020108490 msgr2=0x7f0020108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:29.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.928+0000 7f00278bb700 1 --2- 172.21.15.84:0/2905023664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020108490 0x7f0020108870 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f001c009990 tx=0x7f001c009ca0 comp rx=0 tx=0).stop 2024-05-22T23:00:29.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.928+0000 7f00278bb700 1 -- 172.21.15.84:0/2905023664 shutdown_connections 2024-05-22T23:00:29.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.928+0000 7f00278bb700 1 --2- 172.21.15.84:0/2905023664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0020102db0 0x7f002010f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:29.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.928+0000 7f00278bb700 1 --2- 172.21.15.84:0/2905023664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020108490 0x7f0020108870 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:29.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.928+0000 7f00278bb700 1 --2- 172.21.15.84:0/2905023664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00201023f0 0x7f0020102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:29.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.928+0000 7f00278bb700 1 -- 172.21.15.84:0/2905023664 >> 172.21.15.84:0/2905023664 conn(0x7f0020079550 msgr2=0x7f0020079960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:29.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.929+0000 7f00278bb700 1 -- 172.21.15.84:0/2905023664 shutdown_connections 2024-05-22T23:00:29.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.929+0000 7f00278bb700 1 -- 172.21.15.84:0/2905023664 wait complete. 2024-05-22T23:00:29.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.929+0000 7f00278bb700 1 Processor -- start 2024-05-22T23:00:29.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.929+0000 7f00278bb700 1 -- start start 2024-05-22T23:00:29.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.929+0000 7f00278bb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00201023f0 0x7f0020074660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:29.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.929+0000 7f00278bb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020102db0 0x7f0020074ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:29.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.929+0000 7f00278bb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0020108490 0x7f00200750e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:29.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.929+0000 7f00278bb700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f00201127b0 con 0x7f0020102db0 2024-05-22T23:00:29.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.929+0000 7f00278bb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0020112630 con 0x7f0020108490 2024-05-22T23:00:29.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.929+0000 7f00278bb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0020112930 con 0x7f00201023f0 2024-05-22T23:00:29.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.930+0000 7f0024e56700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020102db0 0x7f0020074ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:29.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.930+0000 7f0024e56700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020102db0 0x7f0020074ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41650/0 (socket says 172.21.15.84:41650) 2024-05-22T23:00:29.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.930+0000 7f0024e56700 1 -- 172.21.15.84:0/3432741148 learned_addr learned my addr 172.21.15.84:0/3432741148 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:29.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.930+0000 7f0025657700 1 --2- 172.21.15.84:0/3432741148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00201023f0 0x7f0020074660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:29.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.930+0000 7f0025e58700 1 --2- 172.21.15.84:0/3432741148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0020108490 0x7f00200750e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:29.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.930+0000 7f0024e56700 1 -- 172.21.15.84:0/3432741148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00201023f0 msgr2=0x7f0020074660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:29.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.930+0000 7f0024e56700 1 --2- 172.21.15.84:0/3432741148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00201023f0 0x7f0020074660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:29.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.930+0000 7f0024e56700 1 -- 172.21.15.84:0/3432741148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0020108490 msgr2=0x7f00200750e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:29.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.930+0000 7f0024e56700 1 --2- 172.21.15.84:0/3432741148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0020108490 0x7f00200750e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:29.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.930+0000 7f0024e56700 1 -- 172.21.15.84:0/3432741148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00201a71e0 con 0x7f0020102db0 2024-05-22T23:00:29.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.930+0000 7f0025657700 1 --2- 172.21.15.84:0/3432741148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00201023f0 0x7f0020074660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:00:29.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.930+0000 7f0024e56700 1 --2- 172.21.15.84:0/3432741148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020102db0 0x7f0020074ba0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f001000b530 tx=0x7f001000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:29.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.930+0000 7f00167fc700 1 -- 172.21.15.84:0/3432741148 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010015020 con 0x7f0020102db0 2024-05-22T23:00:29.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.930+0000 7f00167fc700 1 -- 172.21.15.84:0/3432741148 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0010011d10 con 0x7f0020102db0 2024-05-22T23:00:29.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.930+0000 7f00278bb700 1 -- 172.21.15.84:0/3432741148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00201a74d0 con 0x7f0020102db0 2024-05-22T23:00:29.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.930+0000 7f00167fc700 1 -- 172.21.15.84:0/3432741148 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010010e50 con 0x7f0020102db0 2024-05-22T23:00:29.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.930+0000 7f00278bb700 1 -- 172.21.15.84:0/3432741148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00201a7a10 con 0x7f0020102db0 2024-05-22T23:00:29.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.931+0000 7f00278bb700 1 -- 172.21.15.84:0/3432741148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0020062a70 con 0x7f0020102db0 2024-05-22T23:00:29.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.933+0000 7f00167fc700 1 -- 172.21.15.84:0/3432741148 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0010018020 con 0x7f0020102db0 2024-05-22T23:00:29.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.933+0000 7f00167fc700 1 --2- 172.21.15.84:0/3432741148 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f000c0a2e50 0x7f000c0a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:29.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.933+0000 7f0025657700 1 --2- 172.21.15.84:0/3432741148 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f000c0a2e50 0x7f000c0a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:29.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.933+0000 7f00167fc700 1 -- 172.21.15.84:0/3432741148 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f00100cecb0 con 0x7f0020102db0 2024-05-22T23:00:29.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.934+0000 7f00167fc700 1 --2- 172.21.15.84:0/3432741148 >> [v2:172.21.15.96:6808/2554265648,v1:172.21.15.96:6809/2554265648] conn(0x7f000c0ac630 0x7f000c0aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:29.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.934+0000 7f00167fc700 1 -- 172.21.15.84:0/3432741148 --> [v2:172.21.15.96:6808/2554265648,v1:172.21.15.96:6809/2554265648] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0010011b60 con 0x7f000c0ac630 2024-05-22T23:00:29.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.934+0000 7f00167fc700 1 -- 172.21.15.84:0/3432741148 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f00100cf190 con 0x7f0020102db0 2024-05-22T23:00:29.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.934+0000 7f0025657700 1 --2- 172.21.15.84:0/3432741148 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f000c0a2e50 0x7f000c0a5310 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f001c006010 tx=0x7f001c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:29.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.934+0000 7f0025e58700 1 --2- 172.21.15.84:0/3432741148 >> [v2:172.21.15.96:6808/2554265648,v1:172.21.15.96:6809/2554265648] conn(0x7f000c0ac630 0x7f000c0aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:29.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.935+0000 7f0025e58700 1 --2- 172.21.15.84:0/3432741148 >> [v2:172.21.15.96:6808/2554265648,v1:172.21.15.96:6809/2554265648] conn(0x7f000c0ac630 0x7f000c0aea50 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:29.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.935+0000 7f00167fc700 1 -- 172.21.15.84:0/3432741148 <== osd.4 v2:172.21.15.96:6808/2554265648 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0010011b60 con 0x7f000c0ac630 2024-05-22T23:00:29.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.953+0000 7f00278bb700 1 -- 172.21.15.84:0/3432741148 --> [v2:172.21.15.96:6808/2554265648,v1:172.21.15.96:6809/2554265648] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0020062ef0 con 0x7f000c0ac630 2024-05-22T23:00:29.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.954+0000 7f00167fc700 1 -- 172.21.15.84:0/3432741148 <== osd.4 v2:172.21.15.96:6808/2554265648 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0020062ef0 con 0x7f000c0ac630 2024-05-22T23:00:29.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.954+0000 7f00278bb700 1 -- 172.21.15.84:0/3432741148 >> [v2:172.21.15.96:6808/2554265648,v1:172.21.15.96:6809/2554265648] conn(0x7f000c0ac630 msgr2=0x7f000c0aea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:29.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.954+0000 7f00278bb700 1 --2- 172.21.15.84:0/3432741148 >> [v2:172.21.15.96:6808/2554265648,v1:172.21.15.96:6809/2554265648] conn(0x7f000c0ac630 0x7f000c0aea50 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:29.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.954+0000 7f00278bb700 1 -- 172.21.15.84:0/3432741148 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f000c0a2e50 msgr2=0x7f000c0a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:29.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.954+0000 7f00278bb700 1 --2- 172.21.15.84:0/3432741148 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f000c0a2e50 0x7f000c0a5310 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f001c006010 tx=0x7f001c00b540 comp rx=0 tx=0).stop 2024-05-22T23:00:29.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.954+0000 7f00278bb700 1 -- 172.21.15.84:0/3432741148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020102db0 msgr2=0x7f0020074ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:29.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.954+0000 7f00278bb700 1 --2- 172.21.15.84:0/3432741148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020102db0 0x7f0020074ba0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f001000b530 tx=0x7f001000b8f0 comp rx=0 tx=0).stop 2024-05-22T23:00:29.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.954+0000 7f0025e58700 1 -- 172.21.15.84:0/3432741148 reap_dead start 2024-05-22T23:00:29.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.955+0000 7f00278bb700 1 -- 172.21.15.84:0/3432741148 shutdown_connections 2024-05-22T23:00:29.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.955+0000 7f00278bb700 1 -- 172.21.15.84:0/3432741148 >> 172.21.15.84:0/3432741148 conn(0x7f0020079550 msgr2=0x7f00201011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:29.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.955+0000 7f00278bb700 1 -- 172.21.15.84:0/3432741148 shutdown_connections 2024-05-22T23:00:29.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:29.955+0000 7f00278bb700 1 -- 172.21.15.84:0/3432741148 wait complete. 2024-05-22T23:00:30.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:30 smithi084 ceph-mon[32186]: pgmap v308: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:00:30.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:30 smithi204 ceph-mon[36211]: pgmap v308: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:00:31.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:30 smithi096 ceph-mon[36392]: pgmap v308: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:00:31.258 INFO:teuthology.orchestra.run.smithi084.stdout:107374182461 2024-05-22T23:00:31.259 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-22T23:00:31.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.736+0000 7f413b5c4700 1 -- 172.21.15.84:0/843372746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4134102610 msgr2=0x7f41341029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:31.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.736+0000 7f413b5c4700 1 --2- 172.21.15.84:0/843372746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4134102610 0x7f41341029f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f412400b0a0 tx=0x7f412400b3b0 comp rx=0 tx=0).stop 2024-05-22T23:00:31.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.736+0000 7f413b5c4700 1 -- 172.21.15.84:0/843372746 shutdown_connections 2024-05-22T23:00:31.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.736+0000 7f413b5c4700 1 --2- 172.21.15.84:0/843372746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41341076a0 0x7f413410f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:31.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.736+0000 7f413b5c4700 1 --2- 172.21.15.84:0/843372746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4134102610 0x7f41341029f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:31.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.736+0000 7f413b5c4700 1 --2- 172.21.15.84:0/843372746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4134102fc0 0x7f4134106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:31.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.736+0000 7f413b5c4700 1 -- 172.21.15.84:0/843372746 >> 172.21.15.84:0/843372746 conn(0x7f41340794c0 msgr2=0x7f41340798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:31.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.737+0000 7f413b5c4700 1 -- 172.21.15.84:0/843372746 shutdown_connections 2024-05-22T23:00:31.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.737+0000 7f413b5c4700 1 -- 172.21.15.84:0/843372746 wait complete. 2024-05-22T23:00:31.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.737+0000 7f413b5c4700 1 Processor -- start 2024-05-22T23:00:31.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.737+0000 7f413b5c4700 1 -- start start 2024-05-22T23:00:31.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.737+0000 7f413b5c4700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4134102610 0x7f413419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:31.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.737+0000 7f413b5c4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4134102fc0 0x7f413419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:31.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.737+0000 7f413b5c4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41341076a0 0x7f4134198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:31.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.737+0000 7f413b5c4700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4134112690 con 0x7f4134102610 2024-05-22T23:00:31.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.737+0000 7f413b5c4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4134112510 con 0x7f41341076a0 2024-05-22T23:00:31.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.737+0000 7f413b5c4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4134112810 con 0x7f4134102fc0 2024-05-22T23:00:31.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.738+0000 7f4139360700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4134102610 0x7f413419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:31.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.738+0000 7f4139360700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4134102610 0x7f413419e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59552/0 (socket says 172.21.15.84:59552) 2024-05-22T23:00:31.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.738+0000 7f4139360700 1 -- 172.21.15.84:0/4114120614 learned_addr learned my addr 172.21.15.84:0/4114120614 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:31.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.738+0000 7f4139b61700 1 --2- 172.21.15.84:0/4114120614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41341076a0 0x7f4134198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:31.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.738+0000 7f4139360700 1 -- 172.21.15.84:0/4114120614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4134102fc0 msgr2=0x7f413419eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:31.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.738+0000 7f4139360700 1 --2- 172.21.15.84:0/4114120614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4134102fc0 0x7f413419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:31.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.738+0000 7f4139360700 1 -- 172.21.15.84:0/4114120614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41341076a0 msgr2=0x7f4134198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:31.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.738+0000 7f4139360700 1 --2- 172.21.15.84:0/4114120614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41341076a0 0x7f4134198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:31.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.738+0000 7f4139360700 1 -- 172.21.15.84:0/4114120614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41341990d0 con 0x7f4134102610 2024-05-22T23:00:31.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.738+0000 7f4139360700 1 --2- 172.21.15.84:0/4114120614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4134102610 0x7f413419e760 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f412400b070 tx=0x7f4124004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:31.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.738+0000 7f41327fc700 1 -- 172.21.15.84:0/4114120614 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4124021070 con 0x7f4134102610 2024-05-22T23:00:31.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.738+0000 7f413b5c4700 1 -- 172.21.15.84:0/4114120614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4134199360 con 0x7f4134102610 2024-05-22T23:00:31.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.738+0000 7f41327fc700 1 -- 172.21.15.84:0/4114120614 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4124007c80 con 0x7f4134102610 2024-05-22T23:00:31.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.738+0000 7f41327fc700 1 -- 172.21.15.84:0/4114120614 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4124005390 con 0x7f4134102610 2024-05-22T23:00:31.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.738+0000 7f413b5c4700 1 -- 172.21.15.84:0/4114120614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41341a5630 con 0x7f4134102610 2024-05-22T23:00:31.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.739+0000 7f413b5c4700 1 -- 172.21.15.84:0/4114120614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4134062ab0 con 0x7f4134102610 2024-05-22T23:00:31.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.741+0000 7f41327fc700 1 -- 172.21.15.84:0/4114120614 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4124024070 con 0x7f4134102610 2024-05-22T23:00:31.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.741+0000 7f41327fc700 1 --2- 172.21.15.84:0/4114120614 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f411c0a2d30 0x7f411c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:31.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.741+0000 7f4138b5f700 1 --2- 172.21.15.84:0/4114120614 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f411c0a2d30 0x7f411c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:31.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.742+0000 7f41327fc700 1 -- 172.21.15.84:0/4114120614 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f412400f020 con 0x7f4134102610 2024-05-22T23:00:31.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.742+0000 7f41327fc700 1 --2- 172.21.15.84:0/4114120614 >> [v2:172.21.15.204:6800/1499577897,v1:172.21.15.204:6801/1499577897] conn(0x7f411c0ac510 0x7f411c0ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:31.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.742+0000 7f41327fc700 1 -- 172.21.15.84:0/4114120614 --> [v2:172.21.15.204:6800/1499577897,v1:172.21.15.204:6801/1499577897] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4124026e50 con 0x7f411c0ac510 2024-05-22T23:00:31.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.742+0000 7f41327fc700 1 -- 172.21.15.84:0/4114120614 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f41240d0a50 con 0x7f4134102610 2024-05-22T23:00:31.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.742+0000 7f4138b5f700 1 --2- 172.21.15.84:0/4114120614 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f411c0a2d30 0x7f411c0a51f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4128005950 tx=0x7f41280058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:31.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.742+0000 7f4139b61700 1 --2- 172.21.15.84:0/4114120614 >> [v2:172.21.15.204:6800/1499577897,v1:172.21.15.204:6801/1499577897] conn(0x7f411c0ac510 0x7f411c0ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:31.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.743+0000 7f4139b61700 1 --2- 172.21.15.84:0/4114120614 >> [v2:172.21.15.204:6800/1499577897,v1:172.21.15.204:6801/1499577897] conn(0x7f411c0ac510 0x7f411c0ae930 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:31.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.743+0000 7f41327fc700 1 -- 172.21.15.84:0/4114120614 <== osd.2 v2:172.21.15.204:6800/1499577897 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4124026e50 con 0x7f411c0ac510 2024-05-22T23:00:31.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.759+0000 7f413b5c4700 1 -- 172.21.15.84:0/4114120614 --> [v2:172.21.15.204:6800/1499577897,v1:172.21.15.204:6801/1499577897] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4134062eb0 con 0x7f411c0ac510 2024-05-22T23:00:31.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.760+0000 7f41327fc700 1 -- 172.21.15.84:0/4114120614 <== osd.2 v2:172.21.15.204:6800/1499577897 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4134062eb0 con 0x7f411c0ac510 2024-05-22T23:00:31.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.760+0000 7f413b5c4700 1 -- 172.21.15.84:0/4114120614 >> [v2:172.21.15.204:6800/1499577897,v1:172.21.15.204:6801/1499577897] conn(0x7f411c0ac510 msgr2=0x7f411c0ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:31.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.760+0000 7f413b5c4700 1 --2- 172.21.15.84:0/4114120614 >> [v2:172.21.15.204:6800/1499577897,v1:172.21.15.204:6801/1499577897] conn(0x7f411c0ac510 0x7f411c0ae930 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:31.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.760+0000 7f413b5c4700 1 -- 172.21.15.84:0/4114120614 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f411c0a2d30 msgr2=0x7f411c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:31.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.760+0000 7f413b5c4700 1 --2- 172.21.15.84:0/4114120614 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f411c0a2d30 0x7f411c0a51f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4128005950 tx=0x7f41280058e0 comp rx=0 tx=0).stop 2024-05-22T23:00:31.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.760+0000 7f413b5c4700 1 -- 172.21.15.84:0/4114120614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4134102610 msgr2=0x7f413419e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:31.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.760+0000 7f413b5c4700 1 --2- 172.21.15.84:0/4114120614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4134102610 0x7f413419e760 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f412400b070 tx=0x7f4124004e40 comp rx=0 tx=0).stop 2024-05-22T23:00:31.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.760+0000 7f4139b61700 1 -- 172.21.15.84:0/4114120614 reap_dead start 2024-05-22T23:00:31.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.760+0000 7f413b5c4700 1 -- 172.21.15.84:0/4114120614 shutdown_connections 2024-05-22T23:00:31.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.760+0000 7f413b5c4700 1 -- 172.21.15.84:0/4114120614 >> 172.21.15.84:0/4114120614 conn(0x7f41340794c0 msgr2=0x7f4134101b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:31.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.761+0000 7f413b5c4700 1 -- 172.21.15.84:0/4114120614 shutdown_connections 2024-05-22T23:00:31.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.761+0000 7f413b5c4700 1 -- 172.21.15.84:0/4114120614 wait complete. 2024-05-22T23:00:32.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.361+0000 7f7948e34700 1 -- 172.21.15.84:0/2090555358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7944107670 msgr2=0x7f794410f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:32.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.361+0000 7f7948e34700 1 --2- 172.21.15.84:0/2090555358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7944107670 0x7f794410f240 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f793c00b0d0 tx=0x7f793c00b3e0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.361+0000 7f7948e34700 1 -- 172.21.15.84:0/2090555358 shutdown_connections 2024-05-22T23:00:32.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.361+0000 7f7948e34700 1 --2- 172.21.15.84:0/2090555358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7944107670 0x7f794410f240 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.361+0000 7f7948e34700 1 --2- 172.21.15.84:0/2090555358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7944102680 0x7f7944102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.361+0000 7f7948e34700 1 --2- 172.21.15.84:0/2090555358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7944103030 0x7f7944106f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.361+0000 7f7948e34700 1 -- 172.21.15.84:0/2090555358 >> 172.21.15.84:0/2090555358 conn(0x7f7944079520 msgr2=0x7f7944079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:32.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.361+0000 7f7948e34700 1 -- 172.21.15.84:0/2090555358 shutdown_connections 2024-05-22T23:00:32.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.361+0000 7f7948e34700 1 -- 172.21.15.84:0/2090555358 wait complete. 2024-05-22T23:00:32.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.362+0000 7f7948e34700 1 Processor -- start 2024-05-22T23:00:32.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.362+0000 7f7948e34700 1 -- start start 2024-05-22T23:00:32.379 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.362+0000 7f7948e34700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7944102680 0x7f794419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:32.379 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.362+0000 7f7948e34700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7944103030 0x7f794419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:32.379 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.362+0000 7f7948e34700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7944107670 0x7f79441989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:32.379 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.362+0000 7f7948e34700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f79441126f0 con 0x7f7944103030 2024-05-22T23:00:32.379 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.362+0000 7f7948e34700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7944112570 con 0x7f7944102680 2024-05-22T23:00:32.379 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.362+0000 7f7948e34700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7944112870 con 0x7f7944107670 2024-05-22T23:00:32.379 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.362+0000 7f7941d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7944103030 0x7f794419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:32.379 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.362+0000 7f7941d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7944103030 0x7f794419ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59582/0 (socket says 172.21.15.84:59582) 2024-05-22T23:00:32.379 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.362+0000 7f7941d9b700 1 -- 172.21.15.84:0/1248390373 learned_addr learned my addr 172.21.15.84:0/1248390373 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:32.379 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.362+0000 7f794259c700 1 --2- 172.21.15.84:0/1248390373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7944102680 0x7f794419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:32.379 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.363+0000 7f7942d9d700 1 --2- 172.21.15.84:0/1248390373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7944107670 0x7f79441989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:32.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.363+0000 7f7941d9b700 1 -- 172.21.15.84:0/1248390373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7944107670 msgr2=0x7f79441989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:32.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.363+0000 7f7941d9b700 1 --2- 172.21.15.84:0/1248390373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7944107670 0x7f79441989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.363+0000 7f7941d9b700 1 -- 172.21.15.84:0/1248390373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7944102680 msgr2=0x7f794419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:32.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.363+0000 7f7941d9b700 1 --2- 172.21.15.84:0/1248390373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7944102680 0x7f794419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.363+0000 7f7941d9b700 1 -- 172.21.15.84:0/1248390373 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7944199110 con 0x7f7944103030 2024-05-22T23:00:32.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.363+0000 7f7941d9b700 1 --2- 172.21.15.84:0/1248390373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7944103030 0x7f794419ed30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f793400b500 tx=0x7f793400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:32.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.363+0000 7f793b7fe700 1 -- 172.21.15.84:0/1248390373 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7934015020 con 0x7f7944103030 2024-05-22T23:00:32.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.363+0000 7f7948e34700 1 -- 172.21.15.84:0/1248390373 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7944199400 con 0x7f7944103030 2024-05-22T23:00:32.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.363+0000 7f793b7fe700 1 -- 172.21.15.84:0/1248390373 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7934011d10 con 0x7f7944103030 2024-05-22T23:00:32.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.363+0000 7f793b7fe700 1 -- 172.21.15.84:0/1248390373 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793400f2e0 con 0x7f7944103030 2024-05-22T23:00:32.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.363+0000 7f7948e34700 1 -- 172.21.15.84:0/1248390373 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79441a56c0 con 0x7f7944103030 2024-05-22T23:00:32.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.364+0000 7f7948e34700 1 -- 172.21.15.84:0/1248390373 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7944062af0 con 0x7f7944103030 2024-05-22T23:00:32.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.365+0000 7f793b7fe700 1 -- 172.21.15.84:0/1248390373 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7934018020 con 0x7f7944103030 2024-05-22T23:00:32.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.366+0000 7f793b7fe700 1 --2- 172.21.15.84:0/1248390373 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f79300a2cb0 0x7f79300a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:32.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.366+0000 7f794259c700 1 --2- 172.21.15.84:0/1248390373 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f79300a2cb0 0x7f79300a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:32.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.366+0000 7f793b7fe700 1 -- 172.21.15.84:0/1248390373 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f79340cec80 con 0x7f7944103030 2024-05-22T23:00:32.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.366+0000 7f793b7fe700 1 --2- 172.21.15.84:0/1248390373 >> [v2:172.21.15.204:6824/3106116519,v1:172.21.15.204:6825/3106116519] conn(0x7f79300ac490 0x7f79300ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:32.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.366+0000 7f793b7fe700 1 -- 172.21.15.84:0/1248390373 --> [v2:172.21.15.204:6824/3106116519,v1:172.21.15.204:6825/3106116519] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f79300aef60 con 0x7f79300ac490 2024-05-22T23:00:32.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.366+0000 7f793b7fe700 1 -- 172.21.15.84:0/1248390373 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f79340cf160 con 0x7f7944103030 2024-05-22T23:00:32.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.366+0000 7f794259c700 1 --2- 172.21.15.84:0/1248390373 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f79300a2cb0 0x7f79300a5170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f792c005950 tx=0x7f792c005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:32.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.366+0000 7f7942d9d700 1 --2- 172.21.15.84:0/1248390373 >> [v2:172.21.15.204:6824/3106116519,v1:172.21.15.204:6825/3106116519] conn(0x7f79300ac490 0x7f79300ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:32.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.367+0000 7f7942d9d700 1 --2- 172.21.15.84:0/1248390373 >> [v2:172.21.15.204:6824/3106116519,v1:172.21.15.204:6825/3106116519] conn(0x7f79300ac490 0x7f79300ae8b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:32.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.368+0000 7f793b7fe700 1 -- 172.21.15.84:0/1248390373 <== osd.11 v2:172.21.15.204:6824/3106116519 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f79300aef60 con 0x7f79300ac490 2024-05-22T23:00:32.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.383+0000 7f7948e34700 1 -- 172.21.15.84:0/1248390373 --> [v2:172.21.15.204:6824/3106116519,v1:172.21.15.204:6825/3106116519] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7944062f70 con 0x7f79300ac490 2024-05-22T23:00:32.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.384+0000 7f793b7fe700 1 -- 172.21.15.84:0/1248390373 <== osd.11 v2:172.21.15.204:6824/3106116519 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7944062f70 con 0x7f79300ac490 2024-05-22T23:00:32.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.384+0000 7f7948e34700 1 -- 172.21.15.84:0/1248390373 >> [v2:172.21.15.204:6824/3106116519,v1:172.21.15.204:6825/3106116519] conn(0x7f79300ac490 msgr2=0x7f79300ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:32.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.384+0000 7f7948e34700 1 --2- 172.21.15.84:0/1248390373 >> [v2:172.21.15.204:6824/3106116519,v1:172.21.15.204:6825/3106116519] conn(0x7f79300ac490 0x7f79300ae8b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.384+0000 7f7948e34700 1 -- 172.21.15.84:0/1248390373 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f79300a2cb0 msgr2=0x7f79300a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:32.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.384+0000 7f7948e34700 1 --2- 172.21.15.84:0/1248390373 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f79300a2cb0 0x7f79300a5170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f792c005950 tx=0x7f792c005eb0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.384+0000 7f7948e34700 1 -- 172.21.15.84:0/1248390373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7944103030 msgr2=0x7f794419ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:32.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.384+0000 7f7948e34700 1 --2- 172.21.15.84:0/1248390373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7944103030 0x7f794419ed30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f793400b500 tx=0x7f793400b8c0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.384+0000 7f7942d9d700 1 -- 172.21.15.84:0/1248390373 reap_dead start 2024-05-22T23:00:32.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.385+0000 7f7948e34700 1 -- 172.21.15.84:0/1248390373 shutdown_connections 2024-05-22T23:00:32.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.385+0000 7f7948e34700 1 -- 172.21.15.84:0/1248390373 >> 172.21.15.84:0/1248390373 conn(0x7f7944079520 msgr2=0x7f7944100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:32.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.385+0000 7f7948e34700 1 -- 172.21.15.84:0/1248390373 shutdown_connections 2024-05-22T23:00:32.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.385+0000 7f7948e34700 1 -- 172.21.15.84:0/1248390373 wait complete. 2024-05-22T23:00:32.478 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.652+0000 7f0d286ee700 1 -- 172.21.15.84:0/409406398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d201023e0 msgr2=0x7f0d20102860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:32.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.652+0000 7f0d286ee700 1 --2- 172.21.15.84:0/409406398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d201023e0 0x7f0d20102860 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0d10009990 tx=0x7f0d10009ca0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.652+0000 7f0d286ee700 1 -- 172.21.15.84:0/409406398 shutdown_connections 2024-05-22T23:00:32.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.652+0000 7f0d286ee700 1 --2- 172.21.15.84:0/409406398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d20102da0 0x7f0d2010f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.652+0000 7f0d286ee700 1 --2- 172.21.15.84:0/409406398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d20108460 0x7f0d20108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.652+0000 7f0d286ee700 1 --2- 172.21.15.84:0/409406398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d201023e0 0x7f0d20102860 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.652+0000 7f0d286ee700 1 -- 172.21.15.84:0/409406398 >> 172.21.15.84:0/409406398 conn(0x7f0d20079510 msgr2=0x7f0d20079920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:32.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.652+0000 7f0d286ee700 1 -- 172.21.15.84:0/409406398 shutdown_connections 2024-05-22T23:00:32.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.652+0000 7f0d286ee700 1 -- 172.21.15.84:0/409406398 wait complete. 2024-05-22T23:00:32.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.652+0000 7f0d286ee700 1 Processor -- start 2024-05-22T23:00:32.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.652+0000 7f0d286ee700 1 -- start start 2024-05-22T23:00:32.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.653+0000 7f0d286ee700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d201023e0 0x7f0d2019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:32.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.653+0000 7f0d286ee700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d20102da0 0x7f0d2019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:32.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.653+0000 7f0d286ee700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d20108460 0x7f0d20198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:32.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.653+0000 7f0d286ee700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0d201126f0 con 0x7f0d20108460 2024-05-22T23:00:32.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.653+0000 7f0d286ee700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20112570 con 0x7f0d201023e0 2024-05-22T23:00:32.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.653+0000 7f0d286ee700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20112870 con 0x7f0d20102da0 2024-05-22T23:00:32.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.653+0000 7f0d26c8b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d20108460 0x7f0d20198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:32.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.653+0000 7f0d26c8b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d20108460 0x7f0d20198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59532/0 (socket says 172.21.15.84:59532) 2024-05-22T23:00:32.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.653+0000 7f0d26c8b700 1 -- 172.21.15.84:0/1532111023 learned_addr learned my addr 172.21.15.84:0/1532111023 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:32.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.653+0000 7f0d2648a700 1 --2- 172.21.15.84:0/1532111023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d201023e0 0x7f0d2019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:32.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.653+0000 7f0d25c89700 1 --2- 172.21.15.84:0/1532111023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d20102da0 0x7f0d2019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:32.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.653+0000 7f0d26c8b700 1 -- 172.21.15.84:0/1532111023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d20102da0 msgr2=0x7f0d2019ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:32.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.653+0000 7f0d26c8b700 1 --2- 172.21.15.84:0/1532111023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d20102da0 0x7f0d2019ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.653+0000 7f0d26c8b700 1 -- 172.21.15.84:0/1532111023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d201023e0 msgr2=0x7f0d2019e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:32.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.653+0000 7f0d26c8b700 1 --2- 172.21.15.84:0/1532111023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d201023e0 0x7f0d2019e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.653+0000 7f0d26c8b700 1 -- 172.21.15.84:0/1532111023 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d201990d0 con 0x7f0d20108460 2024-05-22T23:00:32.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.653+0000 7f0d25c89700 1 --2- 172.21.15.84:0/1532111023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d20102da0 0x7f0d2019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:00:32.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.653+0000 7f0d26c8b700 1 --2- 172.21.15.84:0/1532111023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d20108460 0x7f0d20198970 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0d18007ee0 tx=0x7f0d1800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:32.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.653+0000 7f0d177fe700 1 -- 172.21.15.84:0/1532111023 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1800dbd0 con 0x7f0d20108460 2024-05-22T23:00:32.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.653+0000 7f0d286ee700 1 -- 172.21.15.84:0/1532111023 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d201993c0 con 0x7f0d20108460 2024-05-22T23:00:32.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.654+0000 7f0d177fe700 1 -- 172.21.15.84:0/1532111023 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d1800dd70 con 0x7f0d20108460 2024-05-22T23:00:32.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.654+0000 7f0d177fe700 1 -- 172.21.15.84:0/1532111023 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18005a60 con 0x7f0d20108460 2024-05-22T23:00:32.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.654+0000 7f0d286ee700 1 -- 172.21.15.84:0/1532111023 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d201a5680 con 0x7f0d20108460 2024-05-22T23:00:32.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.654+0000 7f0d286ee700 1 -- 172.21.15.84:0/1532111023 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0d20062a30 con 0x7f0d20108460 2024-05-22T23:00:32.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.656+0000 7f0d177fe700 1 -- 172.21.15.84:0/1532111023 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0d1801a020 con 0x7f0d20108460 2024-05-22T23:00:32.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.657+0000 7f0d177fe700 1 --2- 172.21.15.84:0/1532111023 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0d0c0a2cb0 0x7f0d0c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:32.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.657+0000 7f0d2648a700 1 --2- 172.21.15.84:0/1532111023 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0d0c0a2cb0 0x7f0d0c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:32.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.657+0000 7f0d177fe700 1 -- 172.21.15.84:0/1532111023 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f0d180cee40 con 0x7f0d20108460 2024-05-22T23:00:32.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.657+0000 7f0d177fe700 1 --2- 172.21.15.84:0/1532111023 >> [v2:172.21.15.96:6800/2996505353,v1:172.21.15.96:6801/2996505353] conn(0x7f0d0c0ac490 0x7f0d0c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:32.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.657+0000 7f0d177fe700 1 -- 172.21.15.84:0/1532111023 --> [v2:172.21.15.96:6800/2996505353,v1:172.21.15.96:6801/2996505353] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0d1801de50 con 0x7f0d0c0ac490 2024-05-22T23:00:32.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.657+0000 7f0d177fe700 1 -- 172.21.15.84:0/1532111023 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f0d180cf320 con 0x7f0d20108460 2024-05-22T23:00:32.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.657+0000 7f0d2648a700 1 --2- 172.21.15.84:0/1532111023 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0d0c0a2cb0 0x7f0d0c0a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0d20199ab0 tx=0x7f0d1c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:32.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.658+0000 7f0d25c89700 1 --2- 172.21.15.84:0/1532111023 >> [v2:172.21.15.96:6800/2996505353,v1:172.21.15.96:6801/2996505353] conn(0x7f0d0c0ac490 0x7f0d0c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.658+0000 7f0d25c89700 1 --2- 172.21.15.84:0/1532111023 >> [v2:172.21.15.96:6800/2996505353,v1:172.21.15.96:6801/2996505353] conn(0x7f0d0c0ac490 0x7f0d0c0ae8b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.659+0000 7f0d177fe700 1 -- 172.21.15.84:0/1532111023 <== osd.1 v2:172.21.15.96:6800/2996505353 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0d1801de50 con 0x7f0d0c0ac490 2024-05-22T23:00:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.675+0000 7f0d286ee700 1 -- 172.21.15.84:0/1532111023 --> [v2:172.21.15.96:6800/2996505353,v1:172.21.15.96:6801/2996505353] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0d20062e30 con 0x7f0d0c0ac490 2024-05-22T23:00:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.675+0000 7f0d177fe700 1 -- 172.21.15.84:0/1532111023 <== osd.1 v2:172.21.15.96:6800/2996505353 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0d20062e30 con 0x7f0d0c0ac490 2024-05-22T23:00:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.675+0000 7f0d286ee700 1 -- 172.21.15.84:0/1532111023 >> [v2:172.21.15.96:6800/2996505353,v1:172.21.15.96:6801/2996505353] conn(0x7f0d0c0ac490 msgr2=0x7f0d0c0ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.675+0000 7f0d286ee700 1 --2- 172.21.15.84:0/1532111023 >> [v2:172.21.15.96:6800/2996505353,v1:172.21.15.96:6801/2996505353] conn(0x7f0d0c0ac490 0x7f0d0c0ae8b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.676+0000 7f0d286ee700 1 -- 172.21.15.84:0/1532111023 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0d0c0a2cb0 msgr2=0x7f0d0c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.676+0000 7f0d286ee700 1 --2- 172.21.15.84:0/1532111023 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0d0c0a2cb0 0x7f0d0c0a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0d20199ab0 tx=0x7f0d1c00b410 comp rx=0 tx=0).stop 2024-05-22T23:00:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.676+0000 7f0d286ee700 1 -- 172.21.15.84:0/1532111023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d20108460 msgr2=0x7f0d20198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.676+0000 7f0d286ee700 1 --2- 172.21.15.84:0/1532111023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d20108460 0x7f0d20198970 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0d18007ee0 tx=0x7f0d1800d3b0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.676+0000 7f0d26c8b700 1 -- 172.21.15.84:0/1532111023 reap_dead start 2024-05-22T23:00:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.676+0000 7f0d286ee700 1 -- 172.21.15.84:0/1532111023 shutdown_connections 2024-05-22T23:00:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.676+0000 7f0d286ee700 1 -- 172.21.15.84:0/1532111023 >> 172.21.15.84:0/1532111023 conn(0x7f0d20079510 msgr2=0x7f0d201003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.676+0000 7f0d286ee700 1 -- 172.21.15.84:0/1532111023 shutdown_connections 2024-05-22T23:00:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:31.676+0000 7f0d286ee700 1 -- 172.21.15.84:0/1532111023 wait complete. 2024-05-22T23:00:32.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.690+0000 7f927390f700 1 -- 172.21.15.84:0/1031776982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f926c101250 msgr2=0x7f926c1016d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:32.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.690+0000 7f927390f700 1 --2- 172.21.15.84:0/1031776982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f926c101250 0x7f926c1016d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f925c009990 tx=0x7f925c009ca0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.690+0000 7f927390f700 1 -- 172.21.15.84:0/1031776982 shutdown_connections 2024-05-22T23:00:32.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.690+0000 7f927390f700 1 --2- 172.21.15.84:0/1031776982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f926c101c10 0x7f926c10e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.690+0000 7f927390f700 1 --2- 172.21.15.84:0/1031776982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f926c1072c0 0x7f926c1076a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.690+0000 7f927390f700 1 --2- 172.21.15.84:0/1031776982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f926c101250 0x7f926c1016d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.690+0000 7f927390f700 1 -- 172.21.15.84:0/1031776982 >> 172.21.15.84:0/1031776982 conn(0x7f926c0781d0 msgr2=0x7f926c0785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:32.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.690+0000 7f927390f700 1 -- 172.21.15.84:0/1031776982 shutdown_connections 2024-05-22T23:00:32.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.690+0000 7f927390f700 1 -- 172.21.15.84:0/1031776982 wait complete. 2024-05-22T23:00:32.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.691+0000 7f927390f700 1 Processor -- start 2024-05-22T23:00:32.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.691+0000 7f927390f700 1 -- start start 2024-05-22T23:00:32.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.691+0000 7f927390f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f926c101250 0x7f926c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:32.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.691+0000 7f927390f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f926c101c10 0x7f926c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:32.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.691+0000 7f927390f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f926c1072c0 0x7f926c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:32.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.691+0000 7f927390f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f926c111630 con 0x7f926c1072c0 2024-05-22T23:00:32.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.691+0000 7f927390f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f926c1114b0 con 0x7f926c101250 2024-05-22T23:00:32.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.691+0000 7f927390f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f926c1117b0 con 0x7f926c101c10 2024-05-22T23:00:32.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.691+0000 7f9271eac700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f926c1072c0 0x7f926c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:32.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.691+0000 7f9271eac700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f926c1072c0 0x7f926c198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59586/0 (socket says 172.21.15.84:59586) 2024-05-22T23:00:32.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.691+0000 7f9271eac700 1 -- 172.21.15.84:0/2495212104 learned_addr learned my addr 172.21.15.84:0/2495212104 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:32.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.691+0000 7f9270eaa700 1 --2- 172.21.15.84:0/2495212104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f926c101c10 0x7f926c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:32.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.691+0000 7f92716ab700 1 --2- 172.21.15.84:0/2495212104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f926c101250 0x7f926c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:32.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.691+0000 7f9271eac700 1 -- 172.21.15.84:0/2495212104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f926c101c10 msgr2=0x7f926c19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:32.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.691+0000 7f9271eac700 1 --2- 172.21.15.84:0/2495212104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f926c101c10 0x7f926c19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.691+0000 7f9271eac700 1 -- 172.21.15.84:0/2495212104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f926c101250 msgr2=0x7f926c19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:32.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.691+0000 7f9271eac700 1 --2- 172.21.15.84:0/2495212104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f926c101250 0x7f926c19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.691+0000 7f9271eac700 1 -- 172.21.15.84:0/2495212104 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926c1991d0 con 0x7f926c1072c0 2024-05-22T23:00:32.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.691+0000 7f92716ab700 1 --2- 172.21.15.84:0/2495212104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f926c101250 0x7f926c19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:00:32.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.692+0000 7f9271eac700 1 --2- 172.21.15.84:0/2495212104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f926c1072c0 0x7f926c198a70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f926400b550 tx=0x7f926400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:32.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.692+0000 7f92627fc700 1 -- 172.21.15.84:0/2495212104 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264015020 con 0x7f926c1072c0 2024-05-22T23:00:32.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.692+0000 7f92627fc700 1 -- 172.21.15.84:0/2495212104 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9264011d10 con 0x7f926c1072c0 2024-05-22T23:00:32.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.692+0000 7f92627fc700 1 -- 172.21.15.84:0/2495212104 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926400f2e0 con 0x7f926c1072c0 2024-05-22T23:00:32.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.692+0000 7f927390f700 1 -- 172.21.15.84:0/2495212104 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f926c1994c0 con 0x7f926c1072c0 2024-05-22T23:00:32.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.692+0000 7f927390f700 1 -- 172.21.15.84:0/2495212104 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f926c1a5730 con 0x7f926c1072c0 2024-05-22T23:00:32.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.693+0000 7f927390f700 1 -- 172.21.15.84:0/2495212104 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f926c06ac20 con 0x7f926c1072c0 2024-05-22T23:00:32.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.694+0000 7f92627fc700 1 -- 172.21.15.84:0/2495212104 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9264018020 con 0x7f926c1072c0 2024-05-22T23:00:32.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.694+0000 7f92627fc700 1 --2- 172.21.15.84:0/2495212104 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f92580a2cb0 0x7f92580a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:32.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.695+0000 7f92716ab700 1 --2- 172.21.15.84:0/2495212104 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f92580a2cb0 0x7f92580a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:32.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.695+0000 7f92627fc700 1 -- 172.21.15.84:0/2495212104 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f92640ced30 con 0x7f926c1072c0 2024-05-22T23:00:32.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.695+0000 7f92627fc700 1 --2- 172.21.15.84:0/2495212104 >> [v2:172.21.15.96:6824/1985398526,v1:172.21.15.96:6825/1985398526] conn(0x7f92580ac490 0x7f92580ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:32.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.695+0000 7f92627fc700 1 -- 172.21.15.84:0/2495212104 --> [v2:172.21.15.96:6824/1985398526,v1:172.21.15.96:6825/1985398526] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f92580aef60 con 0x7f92580ac490 2024-05-22T23:00:32.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.695+0000 7f92627fc700 1 -- 172.21.15.84:0/2495212104 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f92640cf210 con 0x7f926c1072c0 2024-05-22T23:00:32.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.695+0000 7f92716ab700 1 --2- 172.21.15.84:0/2495212104 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f92580a2cb0 0x7f92580a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f926c0ffa10 tx=0x7f92680073e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:32.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.695+0000 7f9270eaa700 1 --2- 172.21.15.84:0/2495212104 >> [v2:172.21.15.96:6824/1985398526,v1:172.21.15.96:6825/1985398526] conn(0x7f92580ac490 0x7f92580ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:32.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.696+0000 7f9270eaa700 1 --2- 172.21.15.84:0/2495212104 >> [v2:172.21.15.96:6824/1985398526,v1:172.21.15.96:6825/1985398526] conn(0x7f92580ac490 0x7f92580ae8b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:32.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.696+0000 7f92627fc700 1 -- 172.21.15.84:0/2495212104 <== osd.10 v2:172.21.15.96:6824/1985398526 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f92580aef60 con 0x7f92580ac490 2024-05-22T23:00:32.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.712+0000 7f927390f700 1 -- 172.21.15.84:0/2495212104 --> [v2:172.21.15.96:6824/1985398526,v1:172.21.15.96:6825/1985398526] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f926c102970 con 0x7f92580ac490 2024-05-22T23:00:32.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.713+0000 7f92627fc700 1 -- 172.21.15.84:0/2495212104 <== osd.10 v2:172.21.15.96:6824/1985398526 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f926c102970 con 0x7f92580ac490 2024-05-22T23:00:32.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.713+0000 7f927390f700 1 -- 172.21.15.84:0/2495212104 >> [v2:172.21.15.96:6824/1985398526,v1:172.21.15.96:6825/1985398526] conn(0x7f92580ac490 msgr2=0x7f92580ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:32.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.713+0000 7f927390f700 1 --2- 172.21.15.84:0/2495212104 >> [v2:172.21.15.96:6824/1985398526,v1:172.21.15.96:6825/1985398526] conn(0x7f92580ac490 0x7f92580ae8b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.713+0000 7f927390f700 1 -- 172.21.15.84:0/2495212104 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f92580a2cb0 msgr2=0x7f92580a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:32.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.713+0000 7f927390f700 1 --2- 172.21.15.84:0/2495212104 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f92580a2cb0 0x7f92580a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f926c0ffa10 tx=0x7f92680073e0 comp rx=0 tx=0).stop 2024-05-22T23:00:32.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.713+0000 7f927390f700 1 -- 172.21.15.84:0/2495212104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f926c1072c0 msgr2=0x7f926c198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:32.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.713+0000 7f927390f700 1 --2- 172.21.15.84:0/2495212104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f926c1072c0 0x7f926c198a70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f926400b550 tx=0x7f926400b860 comp rx=0 tx=0).stop 2024-05-22T23:00:32.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.713+0000 7f9271eac700 1 -- 172.21.15.84:0/2495212104 reap_dead start 2024-05-22T23:00:32.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.713+0000 7f927390f700 1 -- 172.21.15.84:0/2495212104 shutdown_connections 2024-05-22T23:00:32.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.713+0000 7f927390f700 1 -- 172.21.15.84:0/2495212104 >> 172.21.15.84:0/2495212104 conn(0x7f926c0781d0 msgr2=0x7f926c105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:32.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.713+0000 7f927390f700 1 -- 172.21.15.84:0/2495212104 shutdown_connections 2024-05-22T23:00:32.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:32.713+0000 7f927390f700 1 -- 172.21.15.84:0/2495212104 wait complete. 2024-05-22T23:00:32.764 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:32 smithi084 ceph-mon[32186]: pgmap v309: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:00:32.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:32 smithi204 ceph-mon[36211]: pgmap v309: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:00:33.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:32 smithi096 ceph-mon[36392]: pgmap v309: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:00:33.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.489+0000 7fe45f8b5700 1 -- 172.21.15.84:0/4044573812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4581013c0 msgr2=0x7fe45810d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:33.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.489+0000 7fe45f8b5700 1 --2- 172.21.15.84:0/4044573812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4581013c0 0x7fe45810d8d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe45000b0a0 tx=0x7fe45000b3b0 comp rx=0 tx=0).stop 2024-05-22T23:00:33.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.489+0000 7fe45f8b5700 1 -- 172.21.15.84:0/4044573812 shutdown_connections 2024-05-22T23:00:33.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.489+0000 7fe45f8b5700 1 --2- 172.21.15.84:0/4044573812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4581013c0 0x7fe45810d8d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:33.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.489+0000 7fe45f8b5700 1 --2- 172.21.15.84:0/4044573812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe458106a00 0x7fe458106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:33.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.489+0000 7fe45f8b5700 1 --2- 172.21.15.84:0/4044573812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe458100a00 0x7fe458100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:33.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.489+0000 7fe45f8b5700 1 -- 172.21.15.84:0/4044573812 >> 172.21.15.84:0/4044573812 conn(0x7fe458077860 msgr2=0x7fe4580febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:33.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.489+0000 7fe45f8b5700 1 -- 172.21.15.84:0/4044573812 shutdown_connections 2024-05-22T23:00:33.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.489+0000 7fe45f8b5700 1 -- 172.21.15.84:0/4044573812 wait complete. 2024-05-22T23:00:33.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.489+0000 7fe45f8b5700 1 Processor -- start 2024-05-22T23:00:33.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.490+0000 7fe45f8b5700 1 -- start start 2024-05-22T23:00:33.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.490+0000 7fe45f8b5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe458100a00 0x7fe45819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:33.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.490+0000 7fe45f8b5700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4581013c0 0x7fe45819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:33.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.490+0000 7fe45f8b5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe458106a00 0x7fe4581988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:33.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.490+0000 7fe45f8b5700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe458110510 con 0x7fe4581013c0 2024-05-22T23:00:33.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.490+0000 7fe45f8b5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe458110390 con 0x7fe458106a00 2024-05-22T23:00:33.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.490+0000 7fe45f8b5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe458110690 con 0x7fe458100a00 2024-05-22T23:00:33.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.490+0000 7fe45ce50700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4581013c0 0x7fe45819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:33.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.490+0000 7fe45ce50700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4581013c0 0x7fe45819ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59600/0 (socket says 172.21.15.84:59600) 2024-05-22T23:00:33.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.490+0000 7fe45ce50700 1 -- 172.21.15.84:0/2867819373 learned_addr learned my addr 172.21.15.84:0/2867819373 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:33.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.490+0000 7fe45d651700 1 --2- 172.21.15.84:0/2867819373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe458100a00 0x7fe45819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:33.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.490+0000 7fe45de52700 1 --2- 172.21.15.84:0/2867819373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe458106a00 0x7fe4581988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:33.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.490+0000 7fe45ce50700 1 -- 172.21.15.84:0/2867819373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe458100a00 msgr2=0x7fe45819e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:33.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.490+0000 7fe45ce50700 1 --2- 172.21.15.84:0/2867819373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe458100a00 0x7fe45819e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:33.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.490+0000 7fe45ce50700 1 -- 172.21.15.84:0/2867819373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe458106a00 msgr2=0x7fe4581988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:33.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.490+0000 7fe45ce50700 1 --2- 172.21.15.84:0/2867819373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe458106a00 0x7fe4581988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:33.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.490+0000 7fe45ce50700 1 -- 172.21.15.84:0/2867819373 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe458199040 con 0x7fe4581013c0 2024-05-22T23:00:33.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.490+0000 7fe45ce50700 1 --2- 172.21.15.84:0/2867819373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4581013c0 0x7fe45819ec10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe44800dbe0 tx=0x7fe44800def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:33.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.490+0000 7fe44e7fc700 1 -- 172.21.15.84:0/2867819373 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe448007c60 con 0x7fe4581013c0 2024-05-22T23:00:33.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.491+0000 7fe45f8b5700 1 -- 172.21.15.84:0/2867819373 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe458199330 con 0x7fe4581013c0 2024-05-22T23:00:33.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.491+0000 7fe44e7fc700 1 -- 172.21.15.84:0/2867819373 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe448004500 con 0x7fe4581013c0 2024-05-22T23:00:33.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.491+0000 7fe44e7fc700 1 -- 172.21.15.84:0/2867819373 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe448005280 con 0x7fe4581013c0 2024-05-22T23:00:33.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.491+0000 7fe45f8b5700 1 -- 172.21.15.84:0/2867819373 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4581a55a0 con 0x7fe4581013c0 2024-05-22T23:00:33.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.492+0000 7fe45f8b5700 1 -- 172.21.15.84:0/2867819373 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe458104050 con 0x7fe4581013c0 2024-05-22T23:00:33.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.493+0000 7fe44e7fc700 1 -- 172.21.15.84:0/2867819373 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe448004750 con 0x7fe4581013c0 2024-05-22T23:00:33.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.493+0000 7fe44e7fc700 1 --2- 172.21.15.84:0/2867819373 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe4440a2b90 0x7fe4440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:33.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.493+0000 7fe45d651700 1 --2- 172.21.15.84:0/2867819373 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe4440a2b90 0x7fe4440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:33.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.493+0000 7fe44e7fc700 1 -- 172.21.15.84:0/2867819373 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe4480ceb50 con 0x7fe4581013c0 2024-05-22T23:00:33.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.494+0000 7fe44e7fc700 1 --2- 172.21.15.84:0/2867819373 >> [v2:172.21.15.204:6816/606997927,v1:172.21.15.204:6817/606997927] conn(0x7fe4440ac370 0x7fe4440ae790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:33.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.494+0000 7fe44e7fc700 1 -- 172.21.15.84:0/2867819373 --> [v2:172.21.15.204:6816/606997927,v1:172.21.15.204:6817/606997927] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe448010e50 con 0x7fe4440ac370 2024-05-22T23:00:33.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.494+0000 7fe44e7fc700 1 -- 172.21.15.84:0/2867819373 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fe4480d32c0 con 0x7fe4581013c0 2024-05-22T23:00:33.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.494+0000 7fe45d651700 1 --2- 172.21.15.84:0/2867819373 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe4440a2b90 0x7fe4440a5050 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe4580fff00 tx=0x7fe45400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:33.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.494+0000 7fe45de52700 1 --2- 172.21.15.84:0/2867819373 >> [v2:172.21.15.204:6816/606997927,v1:172.21.15.204:6817/606997927] conn(0x7fe4440ac370 0x7fe4440ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:33.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.494+0000 7fe45de52700 1 --2- 172.21.15.84:0/2867819373 >> [v2:172.21.15.204:6816/606997927,v1:172.21.15.204:6817/606997927] conn(0x7fe4440ac370 0x7fe4440ae790 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:33.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.495+0000 7fe44e7fc700 1 -- 172.21.15.84:0/2867819373 <== osd.8 v2:172.21.15.204:6816/606997927 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe448010e50 con 0x7fe4440ac370 2024-05-22T23:00:33.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.512+0000 7fe45f8b5700 1 -- 172.21.15.84:0/2867819373 --> [v2:172.21.15.204:6816/606997927,v1:172.21.15.204:6817/606997927] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe458104c50 con 0x7fe4440ac370 2024-05-22T23:00:33.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.513+0000 7fe44e7fc700 1 -- 172.21.15.84:0/2867819373 <== osd.8 v2:172.21.15.204:6816/606997927 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe450009450 con 0x7fe4440ac370 2024-05-22T23:00:33.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.513+0000 7fe45f8b5700 1 -- 172.21.15.84:0/2867819373 >> [v2:172.21.15.204:6816/606997927,v1:172.21.15.204:6817/606997927] conn(0x7fe4440ac370 msgr2=0x7fe4440ae790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:33.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.513+0000 7fe45f8b5700 1 --2- 172.21.15.84:0/2867819373 >> [v2:172.21.15.204:6816/606997927,v1:172.21.15.204:6817/606997927] conn(0x7fe4440ac370 0x7fe4440ae790 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:33.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.513+0000 7fe45f8b5700 1 -- 172.21.15.84:0/2867819373 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe4440a2b90 msgr2=0x7fe4440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:33.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.513+0000 7fe45f8b5700 1 --2- 172.21.15.84:0/2867819373 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe4440a2b90 0x7fe4440a5050 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe4580fff00 tx=0x7fe45400b410 comp rx=0 tx=0).stop 2024-05-22T23:00:33.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.513+0000 7fe45f8b5700 1 -- 172.21.15.84:0/2867819373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4581013c0 msgr2=0x7fe45819ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:33.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.513+0000 7fe45f8b5700 1 --2- 172.21.15.84:0/2867819373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4581013c0 0x7fe45819ec10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe44800dbe0 tx=0x7fe44800def0 comp rx=0 tx=0).stop 2024-05-22T23:00:33.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.513+0000 7fe45de52700 1 -- 172.21.15.84:0/2867819373 reap_dead start 2024-05-22T23:00:33.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.513+0000 7fe45f8b5700 1 -- 172.21.15.84:0/2867819373 shutdown_connections 2024-05-22T23:00:33.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.513+0000 7fe45f8b5700 1 -- 172.21.15.84:0/2867819373 >> 172.21.15.84:0/2867819373 conn(0x7fe458077860 msgr2=0x7fe4581035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:33.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.513+0000 7fe45f8b5700 1 -- 172.21.15.84:0/2867819373 shutdown_connections 2024-05-22T23:00:33.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:33.513+0000 7fe45f8b5700 1 -- 172.21.15.84:0/2867819373 wait complete. 2024-05-22T23:00:34.659 INFO:teuthology.orchestra.run.smithi084.stdout:77309411397 2024-05-22T23:00:34.659 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-22T23:00:34.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:34 smithi084 ceph-mon[32186]: pgmap v310: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-22T23:00:34.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:34 smithi204 ceph-mon[36211]: pgmap v310: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-22T23:00:35.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:34 smithi096 ceph-mon[36392]: pgmap v310: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-22T23:00:35.117 INFO:teuthology.orchestra.run.smithi084.stdout:257698037797 2024-05-22T23:00:35.117 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-22T23:00:35.538 INFO:teuthology.orchestra.run.smithi084.stdout:347892350998 2024-05-22T23:00:35.538 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-22T23:00:35.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:35 smithi084 ceph-mon[32186]: pgmap v311: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-05-22T23:00:35.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:35 smithi204 ceph-mon[36211]: pgmap v311: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-05-22T23:00:35.977 INFO:teuthology.orchestra.run.smithi084.stdout:313532612636 2024-05-22T23:00:35.977 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-22T23:00:36.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:35 smithi096 ceph-mon[36392]: pgmap v311: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-05-22T23:00:36.639 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:36.677 INFO:teuthology.orchestra.run.smithi084.stdout:55834574921 2024-05-22T23:00:36.678 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-22T23:00:37.074 INFO:teuthology.orchestra.run.smithi084.stdout:141733920826 2024-05-22T23:00:37.074 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-22T23:00:37.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.355+0000 7f0827af5700 1 -- 172.21.15.84:0/3825273464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08201072c0 msgr2=0x7f08201076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:37.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.355+0000 7f0827af5700 1 --2- 172.21.15.84:0/3825273464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08201072c0 0x7f08201076a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f081c009990 tx=0x7f081c009ca0 comp rx=0 tx=0).stop 2024-05-22T23:00:37.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.355+0000 7f0827af5700 1 -- 172.21.15.84:0/3825273464 shutdown_connections 2024-05-22T23:00:37.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.355+0000 7f0827af5700 1 --2- 172.21.15.84:0/3825273464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0820101c10 0x7f082010e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:37.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.355+0000 7f0827af5700 1 --2- 172.21.15.84:0/3825273464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08201072c0 0x7f08201076a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:37.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.355+0000 7f0827af5700 1 --2- 172.21.15.84:0/3825273464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0820101250 0x7f08201016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:37.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.355+0000 7f0827af5700 1 -- 172.21.15.84:0/3825273464 >> 172.21.15.84:0/3825273464 conn(0x7f08200781d0 msgr2=0x7f08200785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:37.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.355+0000 7f0827af5700 1 -- 172.21.15.84:0/3825273464 shutdown_connections 2024-05-22T23:00:37.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.355+0000 7f0827af5700 1 -- 172.21.15.84:0/3825273464 wait complete. 2024-05-22T23:00:37.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.356+0000 7f0827af5700 1 Processor -- start 2024-05-22T23:00:37.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.356+0000 7f0827af5700 1 -- start start 2024-05-22T23:00:37.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.356+0000 7f0827af5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0820101250 0x7f082019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:37.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.356+0000 7f0827af5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0820101c10 0x7f082019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:37.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.356+0000 7f0827af5700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08201072c0 0x7f08201989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:37.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.356+0000 7f0827af5700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f08201115b0 con 0x7f08201072c0 2024-05-22T23:00:37.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.356+0000 7f0827af5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0820111430 con 0x7f0820101250 2024-05-22T23:00:37.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.356+0000 7f0827af5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0820111730 con 0x7f0820101c10 2024-05-22T23:00:37.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.356+0000 7f0825891700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0820101250 0x7f082019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:37.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.356+0000 7f0825891700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0820101250 0x7f082019e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:47454/0 (socket says 172.21.15.84:47454) 2024-05-22T23:00:37.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.356+0000 7f0825891700 1 -- 172.21.15.84:0/488358291 learned_addr learned my addr 172.21.15.84:0/488358291 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:37.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.356+0000 7f0826092700 1 --2- 172.21.15.84:0/488358291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08201072c0 0x7f08201989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:37.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.356+0000 7f0825090700 1 --2- 172.21.15.84:0/488358291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0820101c10 0x7f082019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:37.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.357+0000 7f0825891700 1 -- 172.21.15.84:0/488358291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0820101c10 msgr2=0x7f082019ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:37.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.357+0000 7f0825891700 1 --2- 172.21.15.84:0/488358291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0820101c10 0x7f082019ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:37.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.357+0000 7f0825891700 1 -- 172.21.15.84:0/488358291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08201072c0 msgr2=0x7f08201989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:37.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.357+0000 7f0825891700 1 --2- 172.21.15.84:0/488358291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08201072c0 0x7f08201989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:37.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.357+0000 7f0825891700 1 -- 172.21.15.84:0/488358291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0820199100 con 0x7f0820101250 2024-05-22T23:00:37.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.357+0000 7f0825891700 1 --2- 172.21.15.84:0/488358291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0820101250 0x7f082019e7e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f081c0038b0 tx=0x7f081c009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:37.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.357+0000 7f0816ffd700 1 -- 172.21.15.84:0/488358291 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c027070 con 0x7f0820101250 2024-05-22T23:00:37.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.357+0000 7f0827af5700 1 -- 172.21.15.84:0/488358291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0820199390 con 0x7f0820101250 2024-05-22T23:00:37.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.357+0000 7f0816ffd700 1 -- 172.21.15.84:0/488358291 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f081c004050 con 0x7f0820101250 2024-05-22T23:00:37.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.357+0000 7f0816ffd700 1 -- 172.21.15.84:0/488358291 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c022c00 con 0x7f0820101250 2024-05-22T23:00:37.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.357+0000 7f0827af5700 1 -- 172.21.15.84:0/488358291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08201031d0 con 0x7f0820101250 2024-05-22T23:00:37.361 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.358+0000 7f0827af5700 1 -- 172.21.15.84:0/488358291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f082006ac20 con 0x7f0820101250 2024-05-22T23:00:37.361 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.360+0000 7f0816ffd700 1 -- 172.21.15.84:0/488358291 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f081c02a070 con 0x7f0820101250 2024-05-22T23:00:37.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.360+0000 7f0816ffd700 1 --2- 172.21.15.84:0/488358291 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f080c0a2d30 0x7f080c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:37.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.361+0000 7f0825090700 1 --2- 172.21.15.84:0/488358291 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f080c0a2d30 0x7f080c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:37.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.361+0000 7f0816ffd700 1 -- 172.21.15.84:0/488358291 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f081c0d09d0 con 0x7f0820101250 2024-05-22T23:00:37.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.361+0000 7f0816ffd700 1 --2- 172.21.15.84:0/488358291 >> [v2:172.21.15.84:6818/1806202865,v1:172.21.15.84:6819/1806202865] conn(0x7f080c0ac510 0x7f080c0ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:37.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.361+0000 7f0816ffd700 1 -- 172.21.15.84:0/488358291 --> [v2:172.21.15.84:6818/1806202865,v1:172.21.15.84:6819/1806202865] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f080c0aefe0 con 0x7f080c0ac510 2024-05-22T23:00:37.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.361+0000 7f0816ffd700 1 -- 172.21.15.84:0/488358291 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f081c0d0e50 con 0x7f0820101250 2024-05-22T23:00:37.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.361+0000 7f0826092700 1 --2- 172.21.15.84:0/488358291 >> [v2:172.21.15.84:6818/1806202865,v1:172.21.15.84:6819/1806202865] conn(0x7f080c0ac510 0x7f080c0ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:37.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.361+0000 7f0825090700 1 --2- 172.21.15.84:0/488358291 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f080c0a2d30 0x7f080c0a51f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f082006b9b0 tx=0x7f0810009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:37.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.361+0000 7f0826092700 1 --2- 172.21.15.84:0/488358291 >> [v2:172.21.15.84:6818/1806202865,v1:172.21.15.84:6819/1806202865] conn(0x7f080c0ac510 0x7f080c0ae930 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:37.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.362+0000 7f0816ffd700 1 -- 172.21.15.84:0/488358291 <== osd.6 v2:172.21.15.84:6818/1806202865 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f080c0aefe0 con 0x7f080c0ac510 2024-05-22T23:00:37.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.382+0000 7f0827af5700 1 -- 172.21.15.84:0/488358291 --> [v2:172.21.15.84:6818/1806202865,v1:172.21.15.84:6819/1806202865] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0820102970 con 0x7f080c0ac510 2024-05-22T23:00:37.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.382+0000 7f0816ffd700 1 -- 172.21.15.84:0/488358291 <== osd.6 v2:172.21.15.84:6818/1806202865 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0820102970 con 0x7f080c0ac510 2024-05-22T23:00:37.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.383+0000 7f0827af5700 1 -- 172.21.15.84:0/488358291 >> [v2:172.21.15.84:6818/1806202865,v1:172.21.15.84:6819/1806202865] conn(0x7f080c0ac510 msgr2=0x7f080c0ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:37.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.383+0000 7f0827af5700 1 --2- 172.21.15.84:0/488358291 >> [v2:172.21.15.84:6818/1806202865,v1:172.21.15.84:6819/1806202865] conn(0x7f080c0ac510 0x7f080c0ae930 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:37.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.383+0000 7f0827af5700 1 -- 172.21.15.84:0/488358291 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f080c0a2d30 msgr2=0x7f080c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:37.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.383+0000 7f0827af5700 1 --2- 172.21.15.84:0/488358291 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f080c0a2d30 0x7f080c0a51f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f082006b9b0 tx=0x7f0810009190 comp rx=0 tx=0).stop 2024-05-22T23:00:37.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.383+0000 7f0827af5700 1 -- 172.21.15.84:0/488358291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0820101250 msgr2=0x7f082019e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:37.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.383+0000 7f0827af5700 1 --2- 172.21.15.84:0/488358291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0820101250 0x7f082019e7e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f081c0038b0 tx=0x7f081c009f50 comp rx=0 tx=0).stop 2024-05-22T23:00:37.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.383+0000 7f0826092700 1 -- 172.21.15.84:0/488358291 reap_dead start 2024-05-22T23:00:37.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.383+0000 7f0827af5700 1 -- 172.21.15.84:0/488358291 shutdown_connections 2024-05-22T23:00:37.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.383+0000 7f0827af5700 1 -- 172.21.15.84:0/488358291 >> 172.21.15.84:0/488358291 conn(0x7f08200781d0 msgr2=0x7f08201054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:37.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.383+0000 7f0827af5700 1 -- 172.21.15.84:0/488358291 shutdown_connections 2024-05-22T23:00:37.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:37.383+0000 7f0827af5700 1 -- 172.21.15.84:0/488358291 wait complete. 2024-05-22T23:00:37.651 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:37.672 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:37.678 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:37.681 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:37.683 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:38.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:38 smithi084 ceph-mon[32186]: pgmap v312: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:00:38.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:38 smithi204 ceph-mon[36211]: pgmap v312: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:00:38.984 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:39.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:38 smithi096 ceph-mon[36392]: pgmap v312: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:00:40.873 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:40 smithi084 ceph-mon[32186]: pgmap v313: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:00:40.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:40 smithi204 ceph-mon[36211]: pgmap v313: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:00:41.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:40 smithi096 ceph-mon[36392]: pgmap v313: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:00:42.771 INFO:teuthology.orchestra.run.smithi084.stdout:197568495665 2024-05-22T23:00:42.771 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-22T23:00:42.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:42 smithi084 ceph-mon[32186]: pgmap v314: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:00:42.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:42 smithi204 ceph-mon[36211]: pgmap v314: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:00:43.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:42 smithi096 ceph-mon[36392]: pgmap v314: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:00:44.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.418+0000 7f87651f1700 1 -- 172.21.15.84:0/3823004671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8760101260 msgr2=0x7f87601016e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:44.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.418+0000 7f87651f1700 1 --2- 172.21.15.84:0/3823004671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8760101260 0x7f87601016e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8754009990 tx=0x7f8754009ca0 comp rx=0 tx=0).stop 2024-05-22T23:00:44.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.418+0000 7f87651f1700 1 -- 172.21.15.84:0/3823004671 shutdown_connections 2024-05-22T23:00:44.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.418+0000 7f87651f1700 1 --2- 172.21.15.84:0/3823004671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8760101c20 0x7f876010e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:44.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.418+0000 7f87651f1700 1 --2- 172.21.15.84:0/3823004671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87601072d0 0x7f87601076b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:44.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.418+0000 7f87651f1700 1 --2- 172.21.15.84:0/3823004671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8760101260 0x7f87601016e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:44.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.418+0000 7f87651f1700 1 -- 172.21.15.84:0/3823004671 >> 172.21.15.84:0/3823004671 conn(0x7f8760078210 msgr2=0x7f8760078620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:44.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.418+0000 7f87651f1700 1 -- 172.21.15.84:0/3823004671 shutdown_connections 2024-05-22T23:00:44.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.418+0000 7f87651f1700 1 -- 172.21.15.84:0/3823004671 wait complete. 2024-05-22T23:00:44.422 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.419+0000 7f87651f1700 1 Processor -- start 2024-05-22T23:00:44.422 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.419+0000 7f87651f1700 1 -- start start 2024-05-22T23:00:44.422 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.419+0000 7f87651f1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8760101260 0x7f876019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:44.422 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.419+0000 7f87651f1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8760101c20 0x7f876019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:44.422 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.419+0000 7f87651f1700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87601072d0 0x7f8760198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:44.422 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.419+0000 7f87651f1700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8760111610 con 0x7f87601072d0 2024-05-22T23:00:44.422 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.419+0000 7f87651f1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8760111490 con 0x7f8760101260 2024-05-22T23:00:44.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.419+0000 7f87651f1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8760111790 con 0x7f8760101c20 2024-05-22T23:00:44.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.419+0000 7f875f59e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87601072d0 0x7f8760198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:44.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.419+0000 7f875f59e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87601072d0 0x7f8760198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50198/0 (socket says 172.21.15.84:50198) 2024-05-22T23:00:44.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.419+0000 7f875f59e700 1 -- 172.21.15.84:0/1348660063 learned_addr learned my addr 172.21.15.84:0/1348660063 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:44.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.419+0000 7f875ed9d700 1 --2- 172.21.15.84:0/1348660063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8760101260 0x7f876019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:44.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.420+0000 7f875e59c700 1 --2- 172.21.15.84:0/1348660063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8760101c20 0x7f876019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:44.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.420+0000 7f875f59e700 1 -- 172.21.15.84:0/1348660063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8760101c20 msgr2=0x7f876019ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:44.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.420+0000 7f875f59e700 1 --2- 172.21.15.84:0/1348660063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8760101c20 0x7f876019ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:44.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.420+0000 7f875f59e700 1 -- 172.21.15.84:0/1348660063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8760101260 msgr2=0x7f876019e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:44.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.420+0000 7f875f59e700 1 --2- 172.21.15.84:0/1348660063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8760101260 0x7f876019e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:44.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.420+0000 7f875f59e700 1 -- 172.21.15.84:0/1348660063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87601990f0 con 0x7f87601072d0 2024-05-22T23:00:44.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.420+0000 7f875f59e700 1 --2- 172.21.15.84:0/1348660063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87601072d0 0x7f8760198990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f875800b970 tx=0x7f875800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:44.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.420+0000 7f8747fff700 1 -- 172.21.15.84:0/1348660063 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758019070 con 0x7f87601072d0 2024-05-22T23:00:44.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.420+0000 7f87651f1700 1 -- 172.21.15.84:0/1348660063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87601993e0 con 0x7f87601072d0 2024-05-22T23:00:44.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.420+0000 7f8747fff700 1 -- 172.21.15.84:0/1348660063 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f875800ce10 con 0x7f87601072d0 2024-05-22T23:00:44.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.420+0000 7f8747fff700 1 -- 172.21.15.84:0/1348660063 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758007370 con 0x7f87601072d0 2024-05-22T23:00:44.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.420+0000 7f87651f1700 1 -- 172.21.15.84:0/1348660063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87601a56a0 con 0x7f87601072d0 2024-05-22T23:00:44.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.421+0000 7f87651f1700 1 -- 172.21.15.84:0/1348660063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8760069050 con 0x7f87601072d0 2024-05-22T23:00:44.425 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.422+0000 7f8747fff700 1 -- 172.21.15.84:0/1348660063 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f875801c070 con 0x7f87601072d0 2024-05-22T23:00:44.425 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.423+0000 7f8747fff700 1 --2- 172.21.15.84:0/1348660063 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f87480a2cb0 0x7f87480a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:44.425 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.423+0000 7f875ed9d700 1 --2- 172.21.15.84:0/1348660063 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f87480a2cb0 0x7f87480a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:44.425 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.423+0000 7f8747fff700 1 -- 172.21.15.84:0/1348660063 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f87580ce280 con 0x7f87601072d0 2024-05-22T23:00:44.425 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.423+0000 7f875ed9d700 1 --2- 172.21.15.84:0/1348660063 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f87480a2cb0 0x7f87480a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8760199ad0 tx=0x7f875000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:44.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.427+0000 7f8747fff700 1 -- 172.21.15.84:0/1348660063 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f875809b340 con 0x7f87601072d0 2024-05-22T23:00:44.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.629+0000 7f87651f1700 1 -- 172.21.15.84:0/1348660063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f87601a5930 con 0x7f87601072d0 2024-05-22T23:00:44.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.629+0000 7f8747fff700 1 -- 172.21.15.84:0/1348660063 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f875809ac40 con 0x7f87601072d0 2024-05-22T23:00:44.631 INFO:teuthology.orchestra.run.smithi084.stdout:223338299437 2024-05-22T23:00:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.631+0000 7f87651f1700 1 -- 172.21.15.84:0/1348660063 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f87480a2cb0 msgr2=0x7f87480a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.631+0000 7f87651f1700 1 --2- 172.21.15.84:0/1348660063 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f87480a2cb0 0x7f87480a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8760199ad0 tx=0x7f875000a380 comp rx=0 tx=0).stop 2024-05-22T23:00:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.631+0000 7f87651f1700 1 -- 172.21.15.84:0/1348660063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87601072d0 msgr2=0x7f8760198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.631+0000 7f87651f1700 1 --2- 172.21.15.84:0/1348660063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87601072d0 0x7f8760198990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f875800b970 tx=0x7f875800bc80 comp rx=0 tx=0).stop 2024-05-22T23:00:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.632+0000 7f87651f1700 1 -- 172.21.15.84:0/1348660063 shutdown_connections 2024-05-22T23:00:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.632+0000 7f87651f1700 1 --2- 172.21.15.84:0/1348660063 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f87480a2cb0 0x7f87480a5170 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.632+0000 7f87651f1700 1 --2- 172.21.15.84:0/1348660063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87601072d0 0x7f8760198990 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.632+0000 7f87651f1700 1 --2- 172.21.15.84:0/1348660063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8760101260 0x7f876019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.632+0000 7f87651f1700 1 --2- 172.21.15.84:0/1348660063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8760101c20 0x7f876019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.632+0000 7f87651f1700 1 -- 172.21.15.84:0/1348660063 >> 172.21.15.84:0/1348660063 conn(0x7f8760078210 msgr2=0x7f8760100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.632+0000 7f87651f1700 1 -- 172.21.15.84:0/1348660063 shutdown_connections 2024-05-22T23:00:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:44.632+0000 7f87651f1700 1 -- 172.21.15.84:0/1348660063 wait complete. 2024-05-22T23:00:44.804 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:44 smithi084 ceph-mon[32186]: pgmap v315: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:00:44.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:44 smithi204 ceph-mon[36211]: pgmap v315: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:00:45.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:44 smithi096 ceph-mon[36392]: pgmap v315: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:00:45.637 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:45 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1348660063' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-22T23:00:45.637 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-05-22T23:00:45.638 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-22T23:00:45.638 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-05-22T23:00:45.638 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-05-22T23:00:45.638 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-05-22T23:00:45.638 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 7]}]: dispatch 2024-05-22T23:00:45.638 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-05-22T23:00:45.638 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-05-22T23:00:45.638 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-05-22T23:00:45.638 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:45 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-05-22T23:00:45.663 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299434 got 223338299437 for osd.7 2024-05-22T23:00:45.664 DEBUG:teuthology.parallel:result is None 2024-05-22T23:00:45.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:45 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1348660063' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-22T23:00:45.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-05-22T23:00:45.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-22T23:00:45.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-05-22T23:00:45.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-05-22T23:00:45.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-05-22T23:00:45.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 7]}]: dispatch 2024-05-22T23:00:45.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-05-22T23:00:45.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-05-22T23:00:45.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-05-22T23:00:45.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:45 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-05-22T23:00:46.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:45 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1348660063' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-22T23:00:46.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-05-22T23:00:46.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-22T23:00:46.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-05-22T23:00:46.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-05-22T23:00:46.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-05-22T23:00:46.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 7]}]: dispatch 2024-05-22T23:00:46.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-05-22T23:00:46.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-05-22T23:00:46.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-05-22T23:00:46.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:45 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-05-22T23:00:46.479 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:46.480 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:46.480 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:46.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.688+0000 7fdde12d2700 1 -- 172.21.15.84:0/3536863352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdddc107790 msgr2=0x7fdddc10f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:46.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.688+0000 7fdde12d2700 1 --2- 172.21.15.84:0/3536863352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdddc107790 0x7fdddc10f340 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fddd400b0a0 tx=0x7fddd400b3b0 comp rx=0 tx=0).stop 2024-05-22T23:00:46.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.688+0000 7fdde12d2700 1 -- 172.21.15.84:0/3536863352 shutdown_connections 2024-05-22T23:00:46.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.688+0000 7fdde12d2700 1 --2- 172.21.15.84:0/3536863352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdddc107790 0x7fdddc10f340 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:46.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.688+0000 7fdde12d2700 1 --2- 172.21.15.84:0/3536863352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdddc1028e0 0x7fdddc102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:46.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.688+0000 7fdde12d2700 1 --2- 172.21.15.84:0/3536863352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdddc103290 0x7fdddc1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:46.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.688+0000 7fdde12d2700 1 -- 172.21.15.84:0/3536863352 >> 172.21.15.84:0/3536863352 conn(0x7fdddc07a2f0 msgr2=0x7fdddc07a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:46.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.689+0000 7fdde12d2700 1 -- 172.21.15.84:0/3536863352 shutdown_connections 2024-05-22T23:00:46.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.689+0000 7fdde12d2700 1 -- 172.21.15.84:0/3536863352 wait complete. 2024-05-22T23:00:46.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.689+0000 7fdde12d2700 1 Processor -- start 2024-05-22T23:00:46.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.690+0000 7fdde12d2700 1 -- start start 2024-05-22T23:00:46.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.690+0000 7fdde12d2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdddc1028e0 0x7fdddc19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:46.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.690+0000 7fdde12d2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdddc103290 0x7fdddc19ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:46.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.690+0000 7fdde12d2700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdddc107790 0x7fdddc198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:46.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.690+0000 7fdde12d2700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc112740 con 0x7fdddc107790 2024-05-22T23:00:46.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.690+0000 7fdde12d2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc1125c0 con 0x7fdddc1028e0 2024-05-22T23:00:46.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.690+0000 7fdde12d2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc1128c0 con 0x7fdddc103290 2024-05-22T23:00:46.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.690+0000 7fdddaffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdddc1028e0 0x7fdddc19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:46.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.690+0000 7fdddb7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdddc107790 0x7fdddc198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:46.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.690+0000 7fdddb7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdddc107790 0x7fdddc198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50216/0 (socket says 172.21.15.84:50216) 2024-05-22T23:00:46.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.690+0000 7fdddb7fe700 1 -- 172.21.15.84:0/769932938 learned_addr learned my addr 172.21.15.84:0/769932938 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:46.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.691+0000 7fdddb7fe700 1 -- 172.21.15.84:0/769932938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdddc103290 msgr2=0x7fdddc19ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:46.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.691+0000 7fdddb7fe700 1 --2- 172.21.15.84:0/769932938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdddc103290 0x7fdddc19ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:46.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.691+0000 7fdddb7fe700 1 -- 172.21.15.84:0/769932938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdddc1028e0 msgr2=0x7fdddc19e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:46.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.691+0000 7fdddb7fe700 1 --2- 172.21.15.84:0/769932938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdddc1028e0 0x7fdddc19e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:46.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.691+0000 7fdddb7fe700 1 -- 172.21.15.84:0/769932938 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdddc1990c0 con 0x7fdddc107790 2024-05-22T23:00:46.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.691+0000 7fdddb7fe700 1 --2- 172.21.15.84:0/769932938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdddc107790 0x7fdddc198960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fddd40062a0 tx=0x7fddd4004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:46.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.691+0000 7fddc3fff700 1 -- 172.21.15.84:0/769932938 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4022070 con 0x7fdddc107790 2024-05-22T23:00:46.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.691+0000 7fddc3fff700 1 -- 172.21.15.84:0/769932938 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddd4003c00 con 0x7fdddc107790 2024-05-22T23:00:46.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.691+0000 7fdde12d2700 1 -- 172.21.15.84:0/769932938 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdddc199350 con 0x7fdddc107790 2024-05-22T23:00:46.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.691+0000 7fddc3fff700 1 -- 172.21.15.84:0/769932938 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4004020 con 0x7fdddc107790 2024-05-22T23:00:46.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.692+0000 7fdde12d2700 1 -- 172.21.15.84:0/769932938 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdddc074590 con 0x7fdddc107790 2024-05-22T23:00:46.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.694+0000 7fdde12d2700 1 -- 172.21.15.84:0/769932938 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdddc069050 con 0x7fdddc107790 2024-05-22T23:00:46.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.695+0000 7fddc3fff700 1 -- 172.21.15.84:0/769932938 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fddd401c6c0 con 0x7fdddc107790 2024-05-22T23:00:46.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.695+0000 7fddc3fff700 1 --2- 172.21.15.84:0/769932938 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fddc40a2d30 0x7fddc40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:46.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.696+0000 7fddc3fff700 1 -- 172.21.15.84:0/769932938 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fddd400f030 con 0x7fdddc107790 2024-05-22T23:00:46.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.696+0000 7fdddaffd700 1 --2- 172.21.15.84:0/769932938 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fddc40a2d30 0x7fddc40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:46.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.696+0000 7fdddaffd700 1 --2- 172.21.15.84:0/769932938 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fddc40a2d30 0x7fddc40a51f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdddc1075c0 tx=0x7fddcc00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:46.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.701+0000 7fddc3fff700 1 -- 172.21.15.84:0/769932938 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fddd40a0e30 con 0x7fdddc107790 2024-05-22T23:00:46.719 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:46 smithi084 ceph-mon[32186]: pgmap v316: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-05-22T23:00:46.719 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:46 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-05-22T23:00:46.719 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:46 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-22T23:00:46.719 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:46 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-05-22T23:00:46.719 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:46 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-05-22T23:00:46.720 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:46 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-05-22T23:00:46.720 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:46 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 7]}]': finished 2024-05-22T23:00:46.720 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:46 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-05-22T23:00:46.720 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:46 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-05-22T23:00:46.720 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:46 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-05-22T23:00:46.720 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:46 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-05-22T23:00:46.720 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:46 smithi084 ceph-mon[32186]: osdmap e87: 12 total, 12 up, 12 in 2024-05-22T23:00:46.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:46 smithi204 ceph-mon[36211]: pgmap v316: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-05-22T23:00:46.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:46 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-05-22T23:00:46.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:46 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-22T23:00:46.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:46 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-05-22T23:00:46.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:46 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-05-22T23:00:46.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:46 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-05-22T23:00:46.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:46 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 7]}]': finished 2024-05-22T23:00:46.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:46 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-05-22T23:00:46.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:46 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-05-22T23:00:46.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:46 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-05-22T23:00:46.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:46 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-05-22T23:00:46.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:46 smithi204 ceph-mon[36211]: osdmap e87: 12 total, 12 up, 12 in 2024-05-22T23:00:46.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.910+0000 7fdde12d2700 1 -- 172.21.15.84:0/769932938 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fdddc02dbc0 con 0x7fdddc107790 2024-05-22T23:00:46.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.910+0000 7fddc3fff700 1 -- 172.21.15.84:0/769932938 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fddd4025e40 con 0x7fdddc107790 2024-05-22T23:00:46.912 INFO:teuthology.orchestra.run.smithi084.stdout:77309411399 2024-05-22T23:00:46.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.913+0000 7fdde12d2700 1 -- 172.21.15.84:0/769932938 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fddc40a2d30 msgr2=0x7fddc40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:46.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.913+0000 7fdde12d2700 1 --2- 172.21.15.84:0/769932938 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fddc40a2d30 0x7fddc40a51f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdddc1075c0 tx=0x7fddcc00a3b0 comp rx=0 tx=0).stop 2024-05-22T23:00:46.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.913+0000 7fdde12d2700 1 -- 172.21.15.84:0/769932938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdddc107790 msgr2=0x7fdddc198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:46.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.913+0000 7fdde12d2700 1 --2- 172.21.15.84:0/769932938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdddc107790 0x7fdddc198960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fddd40062a0 tx=0x7fddd4004ce0 comp rx=0 tx=0).stop 2024-05-22T23:00:46.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.913+0000 7fdde12d2700 1 -- 172.21.15.84:0/769932938 shutdown_connections 2024-05-22T23:00:46.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.913+0000 7fdde12d2700 1 --2- 172.21.15.84:0/769932938 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fddc40a2d30 0x7fddc40a51f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:46.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.913+0000 7fdde12d2700 1 --2- 172.21.15.84:0/769932938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdddc107790 0x7fdddc198960 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:46.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.913+0000 7fdde12d2700 1 --2- 172.21.15.84:0/769932938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdddc1028e0 0x7fdddc19e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:46.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.913+0000 7fdde12d2700 1 --2- 172.21.15.84:0/769932938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdddc103290 0x7fdddc19ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:46.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.913+0000 7fdde12d2700 1 -- 172.21.15.84:0/769932938 >> 172.21.15.84:0/769932938 conn(0x7fdddc07a2f0 msgr2=0x7fdddc1006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:46.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.913+0000 7fdde12d2700 1 -- 172.21.15.84:0/769932938 shutdown_connections 2024-05-22T23:00:46.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.913+0000 7fdde12d2700 1 -- 172.21.15.84:0/769932938 wait complete. 2024-05-22T23:00:46.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.922+0000 7fd03d4b0700 1 -- 172.21.15.84:0/585078288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd03806a900 msgr2=0x7fd03806ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:46.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.922+0000 7fd03d4b0700 1 --2- 172.21.15.84:0/585078288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd03806a900 0x7fd03806ad60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd02c009990 tx=0x7fd02c009ca0 comp rx=0 tx=0).stop 2024-05-22T23:00:46.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.922+0000 7fd03d4b0700 1 -- 172.21.15.84:0/585078288 shutdown_connections 2024-05-22T23:00:46.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.922+0000 7fd03d4b0700 1 --2- 172.21.15.84:0/585078288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd03806b2a0 0x7fd03810f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:46.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.922+0000 7fd03d4b0700 1 --2- 172.21.15.84:0/585078288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0381083d0 0x7fd0381087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:46.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.922+0000 7fd03d4b0700 1 --2- 172.21.15.84:0/585078288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd03806a900 0x7fd03806ad60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:46.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.922+0000 7fd03d4b0700 1 -- 172.21.15.84:0/585078288 >> 172.21.15.84:0/585078288 conn(0x7fd0380781d0 msgr2=0x7fd0380785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:46.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.922+0000 7fd03d4b0700 1 -- 172.21.15.84:0/585078288 shutdown_connections 2024-05-22T23:00:46.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.923+0000 7fd03d4b0700 1 -- 172.21.15.84:0/585078288 wait complete. 2024-05-22T23:00:46.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.923+0000 7fd03d4b0700 1 Processor -- start 2024-05-22T23:00:46.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.923+0000 7fd03d4b0700 1 -- start start 2024-05-22T23:00:46.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.924+0000 7fd03d4b0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd03806a900 0x7fd0381a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:46.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.924+0000 7fd03d4b0700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd03806b2a0 0x7fd0381a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:46.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.924+0000 7fd03d4b0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0381083d0 0x7fd03819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:46.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.924+0000 7fd03d4b0700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd038112690 con 0x7fd03806b2a0 2024-05-22T23:00:46.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.924+0000 7fd03d4b0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd038112510 con 0x7fd0381083d0 2024-05-22T23:00:46.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.924+0000 7fd03d4b0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd038112810 con 0x7fd03806a900 2024-05-22T23:00:46.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.924+0000 7fd0367fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd03806b2a0 0x7fd0381a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:46.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.924+0000 7fd0367fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd03806b2a0 0x7fd0381a0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50240/0 (socket says 172.21.15.84:50240) 2024-05-22T23:00:46.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.924+0000 7fd0367fc700 1 -- 172.21.15.84:0/2151563651 learned_addr learned my addr 172.21.15.84:0/2151563651 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:46.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.924+0000 7fd036ffd700 1 --2- 172.21.15.84:0/2151563651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd03806a900 0x7fd0381a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:46.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.924+0000 7fd0377fe700 1 --2- 172.21.15.84:0/2151563651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0381083d0 0x7fd03819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:46.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.924+0000 7fd0367fc700 1 -- 172.21.15.84:0/2151563651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd03806a900 msgr2=0x7fd0381a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:46.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.924+0000 7fd0367fc700 1 --2- 172.21.15.84:0/2151563651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd03806a900 0x7fd0381a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:46.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.924+0000 7fd0367fc700 1 -- 172.21.15.84:0/2151563651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0381083d0 msgr2=0x7fd03819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:46.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.924+0000 7fd0367fc700 1 --2- 172.21.15.84:0/2151563651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0381083d0 0x7fd03819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:46.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.924+0000 7fd0367fc700 1 -- 172.21.15.84:0/2151563651 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03819b260 con 0x7fd03806b2a0 2024-05-22T23:00:46.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.924+0000 7fd0367fc700 1 --2- 172.21.15.84:0/2151563651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd03806b2a0 0x7fd0381a0e80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd02c000c00 tx=0x7fd02c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:46.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.925+0000 7fd01ffff700 1 -- 172.21.15.84:0/2151563651 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c027070 con 0x7fd03806b2a0 2024-05-22T23:00:46.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.925+0000 7fd01ffff700 1 -- 172.21.15.84:0/2151563651 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd02c004190 con 0x7fd03806b2a0 2024-05-22T23:00:46.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.925+0000 7fd01ffff700 1 -- 172.21.15.84:0/2151563651 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c00f460 con 0x7fd03806b2a0 2024-05-22T23:00:46.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.925+0000 7fd03d4b0700 1 -- 172.21.15.84:0/2151563651 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03819b4f0 con 0x7fd03806b2a0 2024-05-22T23:00:46.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.925+0000 7fd03d4b0700 1 -- 172.21.15.84:0/2151563651 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0381a7810 con 0x7fd03806b2a0 2024-05-22T23:00:46.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.926+0000 7fd03d4b0700 1 -- 172.21.15.84:0/2151563651 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd038103a80 con 0x7fd03806b2a0 2024-05-22T23:00:46.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.927+0000 7fd01ffff700 1 -- 172.21.15.84:0/2151563651 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd02c02a070 con 0x7fd03806b2a0 2024-05-22T23:00:46.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.928+0000 7fd01ffff700 1 --2- 172.21.15.84:0/2151563651 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fd0200a2d30 0x7fd0200a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:46.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.928+0000 7fd036ffd700 1 --2- 172.21.15.84:0/2151563651 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fd0200a2d30 0x7fd0200a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:46.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.928+0000 7fd01ffff700 1 -- 172.21.15.84:0/2151563651 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fd02c0d0490 con 0x7fd03806b2a0 2024-05-22T23:00:46.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.929+0000 7fd036ffd700 1 --2- 172.21.15.84:0/2151563651 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fd0200a2d30 0x7fd0200a51f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd028006fd0 tx=0x7fd028008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:46.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:46.932+0000 7fd01ffff700 1 -- 172.21.15.84:0/2151563651 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd02c013070 con 0x7fd03806b2a0 2024-05-22T23:00:47.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:46 smithi096 ceph-mon[36392]: pgmap v316: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-05-22T23:00:47.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:46 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-05-22T23:00:47.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:46 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-22T23:00:47.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:46 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-05-22T23:00:47.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:46 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-05-22T23:00:47.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:46 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-05-22T23:00:47.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:46 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 7]}]': finished 2024-05-22T23:00:47.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:46 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-05-22T23:00:47.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:46 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-05-22T23:00:47.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:46 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-05-22T23:00:47.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:46 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-05-22T23:00:47.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:46 smithi096 ceph-mon[36392]: osdmap e87: 12 total, 12 up, 12 in 2024-05-22T23:00:47.145 INFO:teuthology.orchestra.run.smithi084.stdout:347892351001 2024-05-22T23:00:47.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.143+0000 7fd03d4b0700 1 -- 172.21.15.84:0/2151563651 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fd038069050 con 0x7fd03806b2a0 2024-05-22T23:00:47.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.144+0000 7fd01ffff700 1 -- 172.21.15.84:0/2151563651 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd038069050 con 0x7fd03806b2a0 2024-05-22T23:00:47.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.146+0000 7fd03d4b0700 1 -- 172.21.15.84:0/2151563651 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fd0200a2d30 msgr2=0x7fd0200a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:47.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.146+0000 7fd03d4b0700 1 --2- 172.21.15.84:0/2151563651 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fd0200a2d30 0x7fd0200a51f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd028006fd0 tx=0x7fd028008040 comp rx=0 tx=0).stop 2024-05-22T23:00:47.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.146+0000 7fd03d4b0700 1 -- 172.21.15.84:0/2151563651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd03806b2a0 msgr2=0x7fd0381a0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:47.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.146+0000 7fd03d4b0700 1 --2- 172.21.15.84:0/2151563651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd03806b2a0 0x7fd0381a0e80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd02c000c00 tx=0x7fd02c003730 comp rx=0 tx=0).stop 2024-05-22T23:00:47.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.147+0000 7fd03d4b0700 1 -- 172.21.15.84:0/2151563651 shutdown_connections 2024-05-22T23:00:47.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.147+0000 7fd03d4b0700 1 --2- 172.21.15.84:0/2151563651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0381083d0 0x7fd03819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:47.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.147+0000 7fd03d4b0700 1 --2- 172.21.15.84:0/2151563651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd03806a900 0x7fd0381a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:47.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.147+0000 7fd03d4b0700 1 --2- 172.21.15.84:0/2151563651 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fd0200a2d30 0x7fd0200a51f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:47.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.147+0000 7fd03d4b0700 1 --2- 172.21.15.84:0/2151563651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd03806b2a0 0x7fd0381a0e80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:47.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.147+0000 7fd03d4b0700 1 -- 172.21.15.84:0/2151563651 >> 172.21.15.84:0/2151563651 conn(0x7fd0380781d0 msgr2=0x7fd03810d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:47.149 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.147+0000 7fd03d4b0700 1 -- 172.21.15.84:0/2151563651 shutdown_connections 2024-05-22T23:00:47.149 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.147+0000 7fd03d4b0700 1 -- 172.21.15.84:0/2151563651 wait complete. 2024-05-22T23:00:47.624 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:47 smithi084 ceph-mon[32186]: osdmap e88: 12 total, 12 up, 12 in 2024-05-22T23:00:47.624 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:47 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/769932938' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-22T23:00:47.624 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:47 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2151563651' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-22T23:00:47.624 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:47 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/4236686851' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-22T23:00:47.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:47 smithi204 ceph-mon[36211]: osdmap e88: 12 total, 12 up, 12 in 2024-05-22T23:00:47.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:47 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/769932938' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-22T23:00:47.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:47 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2151563651' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-22T23:00:47.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:47 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/4236686851' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-22T23:00:48.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:47 smithi096 ceph-mon[36392]: osdmap e88: 12 total, 12 up, 12 in 2024-05-22T23:00:48.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:47 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/769932938' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-22T23:00:48.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:47 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2151563651' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-22T23:00:48.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:47 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/4236686851' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-22T23:00:48.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.442+0000 7f4693fb4700 1 -- 172.21.15.84:0/3140409362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f468c1023e0 msgr2=0x7f468c102860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:48.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.442+0000 7f4693fb4700 1 --2- 172.21.15.84:0/3140409362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f468c1023e0 0x7f468c102860 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4680009990 tx=0x7f4680009ca0 comp rx=0 tx=0).stop 2024-05-22T23:00:48.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.442+0000 7f4693fb4700 1 -- 172.21.15.84:0/3140409362 shutdown_connections 2024-05-22T23:00:48.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.442+0000 7f4693fb4700 1 --2- 172.21.15.84:0/3140409362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f468c102da0 0x7f468c10f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:48.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.442+0000 7f4693fb4700 1 --2- 172.21.15.84:0/3140409362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f468c108460 0x7f468c108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:48.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.442+0000 7f4693fb4700 1 --2- 172.21.15.84:0/3140409362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f468c1023e0 0x7f468c102860 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:48.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.442+0000 7f4693fb4700 1 -- 172.21.15.84:0/3140409362 >> 172.21.15.84:0/3140409362 conn(0x7f468c079510 msgr2=0x7f468c079920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:48.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.442+0000 7f4693fb4700 1 -- 172.21.15.84:0/3140409362 shutdown_connections 2024-05-22T23:00:48.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.442+0000 7f4693fb4700 1 -- 172.21.15.84:0/3140409362 wait complete. 2024-05-22T23:00:48.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.443+0000 7f4693fb4700 1 Processor -- start 2024-05-22T23:00:48.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.443+0000 7f4693fb4700 1 -- start start 2024-05-22T23:00:48.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.443+0000 7f4693fb4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f468c1023e0 0x7f468c19e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:48.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.443+0000 7f4693fb4700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f468c102da0 0x7f468c19ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:48.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.443+0000 7f4693fb4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f468c108460 0x7f468c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:48.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.443+0000 7f4693fb4700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f468c1126d0 con 0x7f468c102da0 2024-05-22T23:00:48.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.443+0000 7f4693fb4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f468c112550 con 0x7f468c1023e0 2024-05-22T23:00:48.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.443+0000 7f4693fb4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f468c112850 con 0x7f468c108460 2024-05-22T23:00:48.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.443+0000 7f469154f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f468c102da0 0x7f468c19ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:48.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.443+0000 7f469154f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f468c102da0 0x7f468c19ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50276/0 (socket says 172.21.15.84:50276) 2024-05-22T23:00:48.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.443+0000 7f469154f700 1 -- 172.21.15.84:0/1093222695 learned_addr learned my addr 172.21.15.84:0/1093222695 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:48.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.443+0000 7f4691d50700 1 --2- 172.21.15.84:0/1093222695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f468c1023e0 0x7f468c19e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:48.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.443+0000 7f4692551700 1 --2- 172.21.15.84:0/1093222695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f468c108460 0x7f468c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:48.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.443+0000 7f469154f700 1 -- 172.21.15.84:0/1093222695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f468c108460 msgr2=0x7f468c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:48.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.443+0000 7f469154f700 1 --2- 172.21.15.84:0/1093222695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f468c108460 0x7f468c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:48.465 INFO:teuthology.orchestra.run.smithi084.stdout:107374182466 2024-05-22T23:00:48.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.443+0000 7f469154f700 1 -- 172.21.15.84:0/1093222695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f468c1023e0 msgr2=0x7f468c19e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:48.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.443+0000 7f469154f700 1 --2- 172.21.15.84:0/1093222695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f468c1023e0 0x7f468c19e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:48.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.444+0000 7f469154f700 1 -- 172.21.15.84:0/1093222695 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468c199090 con 0x7f468c102da0 2024-05-22T23:00:48.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.444+0000 7f469154f700 1 --2- 172.21.15.84:0/1093222695 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f468c102da0 0x7f468c19ecb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4680000c00 tx=0x7f46800047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:48.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.444+0000 7f468affd700 1 -- 172.21.15.84:0/1093222695 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4680003b60 con 0x7f468c102da0 2024-05-22T23:00:48.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.444+0000 7f468affd700 1 -- 172.21.15.84:0/1093222695 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4680003780 con 0x7f468c102da0 2024-05-22T23:00:48.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.444+0000 7f468affd700 1 -- 172.21.15.84:0/1093222695 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468000f460 con 0x7f468c102da0 2024-05-22T23:00:48.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.444+0000 7f4693fb4700 1 -- 172.21.15.84:0/1093222695 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f468c199320 con 0x7f468c102da0 2024-05-22T23:00:48.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.444+0000 7f4693fb4700 1 -- 172.21.15.84:0/1093222695 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f468c074590 con 0x7f468c102da0 2024-05-22T23:00:48.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.445+0000 7f4693fb4700 1 -- 172.21.15.84:0/1093222695 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f468c103b10 con 0x7f468c102da0 2024-05-22T23:00:48.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.447+0000 7f468affd700 1 -- 172.21.15.84:0/1093222695 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f468001f640 con 0x7f468c102da0 2024-05-22T23:00:48.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.448+0000 7f468affd700 1 --2- 172.21.15.84:0/1093222695 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f46740a2d30 0x7f46740a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:48.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.448+0000 7f4691d50700 1 --2- 172.21.15.84:0/1093222695 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f46740a2d30 0x7f46740a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:48.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.448+0000 7f468affd700 1 -- 172.21.15.84:0/1093222695 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f46800d4530 con 0x7f468c102da0 2024-05-22T23:00:48.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.448+0000 7f4691d50700 1 --2- 172.21.15.84:0/1093222695 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f46740a2d30 0x7f46740a51f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f467c009550 tx=0x7f467c0091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:48.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.455+0000 7f468affd700 1 -- 172.21.15.84:0/1093222695 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4680013070 con 0x7f468c102da0 2024-05-22T23:00:48.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.667+0000 7f4693fb4700 1 -- 172.21.15.84:0/1093222695 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f468c199e60 con 0x7f468c102da0 2024-05-22T23:00:48.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.668+0000 7f468affd700 1 -- 172.21.15.84:0/1093222695 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f46800a14b0 con 0x7f468c102da0 2024-05-22T23:00:48.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.670+0000 7f4693fb4700 1 -- 172.21.15.84:0/1093222695 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f46740a2d30 msgr2=0x7f46740a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:48.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.670+0000 7f4693fb4700 1 --2- 172.21.15.84:0/1093222695 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f46740a2d30 0x7f46740a51f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f467c009550 tx=0x7f467c0091b0 comp rx=0 tx=0).stop 2024-05-22T23:00:48.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.670+0000 7f4693fb4700 1 -- 172.21.15.84:0/1093222695 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f468c102da0 msgr2=0x7f468c19ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:48.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.670+0000 7f4693fb4700 1 --2- 172.21.15.84:0/1093222695 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f468c102da0 0x7f468c19ecb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4680000c00 tx=0x7f46800047b0 comp rx=0 tx=0).stop 2024-05-22T23:00:48.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.670+0000 7f4693fb4700 1 -- 172.21.15.84:0/1093222695 shutdown_connections 2024-05-22T23:00:48.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.670+0000 7f4693fb4700 1 --2- 172.21.15.84:0/1093222695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f468c108460 0x7f468c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:48.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.670+0000 7f4693fb4700 1 --2- 172.21.15.84:0/1093222695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f468c1023e0 0x7f468c19e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:48.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.670+0000 7f4693fb4700 1 --2- 172.21.15.84:0/1093222695 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f46740a2d30 0x7f46740a51f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:48.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.670+0000 7f4693fb4700 1 --2- 172.21.15.84:0/1093222695 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f468c102da0 0x7f468c19ecb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:48.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.670+0000 7f4693fb4700 1 -- 172.21.15.84:0/1093222695 >> 172.21.15.84:0/1093222695 conn(0x7f468c079510 msgr2=0x7f468c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:48.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.670+0000 7f4693fb4700 1 -- 172.21.15.84:0/1093222695 shutdown_connections 2024-05-22T23:00:48.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.670+0000 7f4693fb4700 1 -- 172.21.15.84:0/1093222695 wait complete. 2024-05-22T23:00:48.784 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:48 smithi084 ceph-mon[32186]: pgmap v319: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s rd, 0 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-05-22T23:00:48.784 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:48 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1093222695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-22T23:00:48.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:48 smithi204 ceph-mon[36211]: pgmap v319: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s rd, 0 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-05-22T23:00:48.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:48 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1093222695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-22T23:00:49.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:48 smithi096 ceph-mon[36392]: pgmap v319: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s rd, 0 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-05-22T23:00:49.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:48 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1093222695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-22T23:00:49.095 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182461 got 107374182466 for osd.3 2024-05-22T23:00:49.095 DEBUG:teuthology.parallel:result is None 2024-05-22T23:00:49.186 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.348+0000 7fee3759e700 1 -- 172.21.15.84:0/133915143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee381064d0 msgr2=0x7fee3810e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:49.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.348+0000 7fee3759e700 1 --2- 172.21.15.84:0/133915143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee381064d0 0x7fee3810e0a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fee3000b0d0 tx=0x7fee3000b3e0 comp rx=0 tx=0).stop 2024-05-22T23:00:49.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.348+0000 7fee3759e700 1 -- 172.21.15.84:0/133915143 shutdown_connections 2024-05-22T23:00:49.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.348+0000 7fee3759e700 1 --2- 172.21.15.84:0/133915143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee381064d0 0x7fee3810e0a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:49.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.348+0000 7fee3759e700 1 --2- 172.21.15.84:0/133915143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee381013f0 0x7fee381017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:49.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.348+0000 7fee3759e700 1 --2- 172.21.15.84:0/133915143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee38101da0 0x7fee38105df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:49.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.348+0000 7fee3759e700 1 -- 172.21.15.84:0/133915143 >> 172.21.15.84:0/133915143 conn(0x7fee380fcc80 msgr2=0x7fee380ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:49.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.348+0000 7fee3759e700 1 -- 172.21.15.84:0/133915143 shutdown_connections 2024-05-22T23:00:49.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.348+0000 7fee3759e700 1 -- 172.21.15.84:0/133915143 wait complete. 2024-05-22T23:00:49.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.349+0000 7fee3759e700 1 Processor -- start 2024-05-22T23:00:49.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.349+0000 7fee3759e700 1 -- start start 2024-05-22T23:00:49.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.349+0000 7fee3759e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee381013f0 0x7fee3819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:49.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.349+0000 7fee3759e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee38101da0 0x7fee3819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:49.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.349+0000 7fee3759e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee381064d0 0x7fee38198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:49.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.349+0000 7fee3759e700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fee381114c0 con 0x7fee381013f0 2024-05-22T23:00:49.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.349+0000 7fee3759e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fee38111340 con 0x7fee381064d0 2024-05-22T23:00:49.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.349+0000 7fee3759e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fee38111640 con 0x7fee38101da0 2024-05-22T23:00:49.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.350+0000 7fee3659c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee381013f0 0x7fee3819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:49.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.350+0000 7fee3659c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee381013f0 0x7fee3819e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50258/0 (socket says 172.21.15.84:50258) 2024-05-22T23:00:49.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.350+0000 7fee3659c700 1 -- 172.21.15.84:0/4236686851 learned_addr learned my addr 172.21.15.84:0/4236686851 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:49.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.350+0000 7fee36d9d700 1 --2- 172.21.15.84:0/4236686851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee381064d0 0x7fee38198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:49.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.350+0000 7fee35d9b700 1 --2- 172.21.15.84:0/4236686851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee38101da0 0x7fee3819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:49.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.350+0000 7fee3659c700 1 -- 172.21.15.84:0/4236686851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee38101da0 msgr2=0x7fee3819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:49.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.350+0000 7fee3659c700 1 --2- 172.21.15.84:0/4236686851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee38101da0 0x7fee3819eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:49.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.350+0000 7fee3659c700 1 -- 172.21.15.84:0/4236686851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee381064d0 msgr2=0x7fee38198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:49.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.350+0000 7fee3659c700 1 --2- 172.21.15.84:0/4236686851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee381064d0 0x7fee38198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:49.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.350+0000 7fee3659c700 1 -- 172.21.15.84:0/4236686851 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee38198ff0 con 0x7fee381013f0 2024-05-22T23:00:49.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.350+0000 7fee3659c700 1 --2- 172.21.15.84:0/4236686851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee381013f0 0x7fee3819e640 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fee2800d980 tx=0x7fee2800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:49.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.350+0000 7fee277fe700 1 -- 172.21.15.84:0/4236686851 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28016070 con 0x7fee381013f0 2024-05-22T23:00:49.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.350+0000 7fee3759e700 1 -- 172.21.15.84:0/4236686851 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee381992e0 con 0x7fee381013f0 2024-05-22T23:00:49.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.350+0000 7fee277fe700 1 -- 172.21.15.84:0/4236686851 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee28004d10 con 0x7fee381013f0 2024-05-22T23:00:49.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.350+0000 7fee277fe700 1 -- 172.21.15.84:0/4236686851 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28005670 con 0x7fee381013f0 2024-05-22T23:00:49.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.350+0000 7fee3759e700 1 -- 172.21.15.84:0/4236686851 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee381a55c0 con 0x7fee381013f0 2024-05-22T23:00:49.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.351+0000 7fee3759e700 1 -- 172.21.15.84:0/4236686851 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee381028d0 con 0x7fee381013f0 2024-05-22T23:00:49.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.353+0000 7fee277fe700 1 -- 172.21.15.84:0/4236686851 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fee28004750 con 0x7fee381013f0 2024-05-22T23:00:49.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.354+0000 7fee277fe700 1 --2- 172.21.15.84:0/4236686851 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fee200a2cb0 0x7fee200a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:49.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.354+0000 7fee277fe700 1 -- 172.21.15.84:0/4236686851 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fee280cf4c0 con 0x7fee381013f0 2024-05-22T23:00:49.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.354+0000 7fee35d9b700 1 --2- 172.21.15.84:0/4236686851 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fee200a2cb0 0x7fee200a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:49.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.355+0000 7fee35d9b700 1 --2- 172.21.15.84:0/4236686851 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fee200a2cb0 0x7fee200a5170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fee38199ed0 tx=0x7fee2c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:49.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.358+0000 7fee277fe700 1 -- 172.21.15.84:0/4236686851 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fee2801f020 con 0x7fee381013f0 2024-05-22T23:00:49.190 INFO:teuthology.orchestra.run.smithi084.stdout:257698037800 2024-05-22T23:00:49.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.571+0000 7fee3759e700 1 -- 172.21.15.84:0/4236686851 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fee38069050 con 0x7fee381013f0 2024-05-22T23:00:49.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.571+0000 7fee277fe700 1 -- 172.21.15.84:0/4236686851 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fee38069050 con 0x7fee381013f0 2024-05-22T23:00:49.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.573+0000 7fee3759e700 1 -- 172.21.15.84:0/4236686851 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fee200a2cb0 msgr2=0x7fee200a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:49.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.573+0000 7fee3759e700 1 --2- 172.21.15.84:0/4236686851 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fee200a2cb0 0x7fee200a5170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fee38199ed0 tx=0x7fee2c00a380 comp rx=0 tx=0).stop 2024-05-22T23:00:49.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.573+0000 7fee3759e700 1 -- 172.21.15.84:0/4236686851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee381013f0 msgr2=0x7fee3819e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:49.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.573+0000 7fee3759e700 1 --2- 172.21.15.84:0/4236686851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee381013f0 0x7fee3819e640 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fee2800d980 tx=0x7fee2800dc90 comp rx=0 tx=0).stop 2024-05-22T23:00:49.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.573+0000 7fee3759e700 1 -- 172.21.15.84:0/4236686851 shutdown_connections 2024-05-22T23:00:49.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.573+0000 7fee3759e700 1 --2- 172.21.15.84:0/4236686851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee381064d0 0x7fee38198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:49.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.573+0000 7fee3759e700 1 --2- 172.21.15.84:0/4236686851 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fee200a2cb0 0x7fee200a5170 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:49.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.573+0000 7fee3759e700 1 --2- 172.21.15.84:0/4236686851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee381013f0 0x7fee3819e640 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:49.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.573+0000 7fee3759e700 1 --2- 172.21.15.84:0/4236686851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee38101da0 0x7fee3819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:49.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.573+0000 7fee3759e700 1 -- 172.21.15.84:0/4236686851 >> 172.21.15.84:0/4236686851 conn(0x7fee380fcc80 msgr2=0x7fee38104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:49.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.573+0000 7fee3759e700 1 -- 172.21.15.84:0/4236686851 shutdown_connections 2024-05-22T23:00:49.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:47.573+0000 7fee3759e700 1 -- 172.21.15.84:0/4236686851 wait complete. 2024-05-22T23:00:50.084 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411397 got 77309411399 for osd.2 2024-05-22T23:00:50.084 DEBUG:teuthology.parallel:result is None 2024-05-22T23:00:50.827 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:50 smithi084 ceph-mon[32186]: pgmap v320: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s rd, 0 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-05-22T23:00:50.827 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:50 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:00:50.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:50 smithi204 ceph-mon[36211]: pgmap v320: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s rd, 0 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-05-22T23:00:50.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:50 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:00:51.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:50 smithi096 ceph-mon[36392]: pgmap v320: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s rd, 0 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-05-22T23:00:51.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:50 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:00:52.036 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.035+0000 7f533e032700 1 -- 172.21.15.84:0/3009811424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53381065a0 msgr2=0x7f533810e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:52.036 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.035+0000 7f533e032700 1 --2- 172.21.15.84:0/3009811424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53381065a0 0x7f533810e170 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f533000b0d0 tx=0x7f533000b3e0 comp rx=0 tx=0).stop 2024-05-22T23:00:52.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.035+0000 7f533e032700 1 -- 172.21.15.84:0/3009811424 shutdown_connections 2024-05-22T23:00:52.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.035+0000 7f533e032700 1 --2- 172.21.15.84:0/3009811424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53381065a0 0x7f533810e170 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:52.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.035+0000 7f533e032700 1 --2- 172.21.15.84:0/3009811424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5338101550 0x7f5338101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:52.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.035+0000 7f533e032700 1 --2- 172.21.15.84:0/3009811424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5338101f00 0x7f5338105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:52.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.035+0000 7f533e032700 1 -- 172.21.15.84:0/3009811424 >> 172.21.15.84:0/3009811424 conn(0x7f53380772b0 msgr2=0x7f53380776c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:52.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.035+0000 7f533e032700 1 -- 172.21.15.84:0/3009811424 shutdown_connections 2024-05-22T23:00:52.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.035+0000 7f533e032700 1 -- 172.21.15.84:0/3009811424 wait complete. 2024-05-22T23:00:52.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.035+0000 7f533e032700 1 Processor -- start 2024-05-22T23:00:52.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.036+0000 7f533e032700 1 -- start start 2024-05-22T23:00:52.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.036+0000 7f533e032700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5338101550 0x7f5338102f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:52.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.036+0000 7f533e032700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5338101f00 0x7f53381034a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:52.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.036+0000 7f533e032700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53381065a0 0x7f53381039e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:52.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.036+0000 7f533e032700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5338111590 con 0x7f5338101f00 2024-05-22T23:00:52.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.036+0000 7f533e032700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5338111410 con 0x7f5338101550 2024-05-22T23:00:52.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.036+0000 7f533e032700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5338111710 con 0x7f53381065a0 2024-05-22T23:00:52.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.036+0000 7f532ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5338101f00 0x7f53381034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:52.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.036+0000 7f532ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5338101f00 0x7f53381034a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39664/0 (socket says 172.21.15.84:39664) 2024-05-22T23:00:52.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.036+0000 7f53377fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5338101550 0x7f5338102f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:52.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.036+0000 7f532ffff700 1 -- 172.21.15.84:0/428073500 learned_addr learned my addr 172.21.15.84:0/428073500 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:52.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.036+0000 7f5337fff700 1 --2- 172.21.15.84:0/428073500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53381065a0 0x7f53381039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:52.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.036+0000 7f532ffff700 1 -- 172.21.15.84:0/428073500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53381065a0 msgr2=0x7f53381039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:52.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.036+0000 7f532ffff700 1 --2- 172.21.15.84:0/428073500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53381065a0 0x7f53381039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:52.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.036+0000 7f532ffff700 1 -- 172.21.15.84:0/428073500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5338101550 msgr2=0x7f5338102f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:52.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.036+0000 7f532ffff700 1 --2- 172.21.15.84:0/428073500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5338101550 0x7f5338102f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:52.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.036+0000 7f532ffff700 1 -- 172.21.15.84:0/428073500 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53381a6fd0 con 0x7f5338101f00 2024-05-22T23:00:52.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.036+0000 7f53377fe700 1 --2- 172.21.15.84:0/428073500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5338101550 0x7f5338102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:00:52.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.036+0000 7f532ffff700 1 --2- 172.21.15.84:0/428073500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5338101f00 0x7f53381034a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f532000d980 tx=0x7f532000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:52.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.036+0000 7f53357fa700 1 -- 172.21.15.84:0/428073500 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320017070 con 0x7f5338101f00 2024-05-22T23:00:52.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.037+0000 7f53357fa700 1 -- 172.21.15.84:0/428073500 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f532000fd70 con 0x7f5338101f00 2024-05-22T23:00:52.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.037+0000 7f533e032700 1 -- 172.21.15.84:0/428073500 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53381a72c0 con 0x7f5338101f00 2024-05-22T23:00:52.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.037+0000 7f53357fa700 1 -- 172.21.15.84:0/428073500 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532000e3b0 con 0x7f5338101f00 2024-05-22T23:00:52.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.037+0000 7f533e032700 1 -- 172.21.15.84:0/428073500 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53381a7800 con 0x7f5338101f00 2024-05-22T23:00:52.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.038+0000 7f533e032700 1 -- 172.21.15.84:0/428073500 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5338069050 con 0x7f5338101f00 2024-05-22T23:00:52.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.039+0000 7f53357fa700 1 -- 172.21.15.84:0/428073500 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f532000f890 con 0x7f5338101f00 2024-05-22T23:00:52.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.039+0000 7f53357fa700 1 --2- 172.21.15.84:0/428073500 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f53240a2cb0 0x7f53240a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:52.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.039+0000 7f53377fe700 1 --2- 172.21.15.84:0/428073500 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f53240a2cb0 0x7f53240a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:52.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.039+0000 7f53357fa700 1 -- 172.21.15.84:0/428073500 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f53200cec50 con 0x7f5338101f00 2024-05-22T23:00:52.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.040+0000 7f53377fe700 1 --2- 172.21.15.84:0/428073500 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f53240a2cb0 0x7f53240a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5328011490 tx=0x7f5328009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:52.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.046+0000 7f53357fa700 1 -- 172.21.15.84:0/428073500 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f532009ba30 con 0x7f5338101f00 2024-05-22T23:00:52.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.259+0000 7f533e032700 1 -- 172.21.15.84:0/428073500 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f533802dbc0 con 0x7f5338101f00 2024-05-22T23:00:52.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.259+0000 7f53357fa700 1 -- 172.21.15.84:0/428073500 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f532009b330 con 0x7f5338101f00 2024-05-22T23:00:52.261 INFO:teuthology.orchestra.run.smithi084.stdout:292057776165 2024-05-22T23:00:52.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.262+0000 7f533e032700 1 -- 172.21.15.84:0/428073500 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f53240a2cb0 msgr2=0x7f53240a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:52.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.262+0000 7f533e032700 1 --2- 172.21.15.84:0/428073500 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f53240a2cb0 0x7f53240a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5328011490 tx=0x7f5328009260 comp rx=0 tx=0).stop 2024-05-22T23:00:52.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.262+0000 7f533e032700 1 -- 172.21.15.84:0/428073500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5338101f00 msgr2=0x7f53381034a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:52.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.262+0000 7f533e032700 1 --2- 172.21.15.84:0/428073500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5338101f00 0x7f53381034a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f532000d980 tx=0x7f532000dc90 comp rx=0 tx=0).stop 2024-05-22T23:00:52.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.262+0000 7f533e032700 1 -- 172.21.15.84:0/428073500 shutdown_connections 2024-05-22T23:00:52.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.262+0000 7f533e032700 1 --2- 172.21.15.84:0/428073500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53381065a0 0x7f53381039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:52.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.262+0000 7f533e032700 1 --2- 172.21.15.84:0/428073500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5338101550 0x7f5338102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:52.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.262+0000 7f533e032700 1 --2- 172.21.15.84:0/428073500 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f53240a2cb0 0x7f53240a5170 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:52.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.262+0000 7f533e032700 1 --2- 172.21.15.84:0/428073500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5338101f00 0x7f53381034a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:52.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.262+0000 7f533e032700 1 -- 172.21.15.84:0/428073500 >> 172.21.15.84:0/428073500 conn(0x7f53380772b0 msgr2=0x7f533806acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:52.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.263+0000 7f533e032700 1 -- 172.21.15.84:0/428073500 shutdown_connections 2024-05-22T23:00:52.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:52.263+0000 7f533e032700 1 -- 172.21.15.84:0/428073500 wait complete. 2024-05-22T23:00:52.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:52 smithi084 ceph-mon[32186]: pgmap v321: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s, 0 objects/s recovering 2024-05-22T23:00:52.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:52 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T23:00:52.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:52 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T23:00:52.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:52 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T23:00:52.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:52 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T23:00:52.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:52 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/428073500' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-22T23:00:52.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:52 smithi204 ceph-mon[36211]: pgmap v321: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s, 0 objects/s recovering 2024-05-22T23:00:52.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:52 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T23:00:52.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:52 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T23:00:52.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:52 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T23:00:52.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:52 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T23:00:52.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:52 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/428073500' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-22T23:00:53.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:52 smithi096 ceph-mon[36392]: pgmap v321: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s, 0 objects/s recovering 2024-05-22T23:00:53.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:52 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T23:00:53.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:52 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T23:00:53.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:52 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T23:00:53.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:52 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T23:00:53.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:52 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/428073500' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-22T23:00:53.066 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:53.092 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037797 got 257698037800 for osd.8 2024-05-22T23:00:53.093 DEBUG:teuthology.parallel:result is None 2024-05-22T23:00:54.143 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350998 got 347892351001 for osd.11 2024-05-22T23:00:54.143 DEBUG:teuthology.parallel:result is None 2024-05-22T23:00:54.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.422+0000 7f286fd08700 1 -- 172.21.15.84:0/2539757797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2868101c20 msgr2=0x7f286810e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:54.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.422+0000 7f286fd08700 1 --2- 172.21.15.84:0/2539757797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2868101c20 0x7f286810e110 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2860009990 tx=0x7f2860009ca0 comp rx=0 tx=0).stop 2024-05-22T23:00:54.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.422+0000 7f286fd08700 1 -- 172.21.15.84:0/2539757797 shutdown_connections 2024-05-22T23:00:54.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.422+0000 7f286fd08700 1 --2- 172.21.15.84:0/2539757797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2868101c20 0x7f286810e110 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:54.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.422+0000 7f286fd08700 1 --2- 172.21.15.84:0/2539757797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28681072d0 0x7f28681076b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:54.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.422+0000 7f286fd08700 1 --2- 172.21.15.84:0/2539757797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2868101260 0x7f28681016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:54.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.422+0000 7f286fd08700 1 -- 172.21.15.84:0/2539757797 >> 172.21.15.84:0/2539757797 conn(0x7f2868078210 msgr2=0x7f2868078620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:54.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.422+0000 7f286fd08700 1 -- 172.21.15.84:0/2539757797 shutdown_connections 2024-05-22T23:00:54.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.422+0000 7f286fd08700 1 -- 172.21.15.84:0/2539757797 wait complete. 2024-05-22T23:00:54.425 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.423+0000 7f286fd08700 1 Processor -- start 2024-05-22T23:00:54.425 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.423+0000 7f286fd08700 1 -- start start 2024-05-22T23:00:54.426 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.423+0000 7f286fd08700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2868101260 0x7f28681999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:54.426 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.423+0000 7f286fd08700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2868101c20 0x7f2868199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:54.426 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.423+0000 7f286fd08700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28681072d0 0x7f28681a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:54.426 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.423+0000 7f286fd08700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2868111560 con 0x7f2868101c20 2024-05-22T23:00:54.426 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.423+0000 7f286fd08700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f28681113e0 con 0x7f28681072d0 2024-05-22T23:00:54.426 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.423+0000 7f286fd08700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f28681116e0 con 0x7f2868101260 2024-05-22T23:00:54.426 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.423+0000 7f286d2a3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2868101c20 0x7f2868199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:54.426 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.423+0000 7f286d2a3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2868101c20 0x7f2868199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39678/0 (socket says 172.21.15.84:39678) 2024-05-22T23:00:54.426 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.423+0000 7f286d2a3700 1 -- 172.21.15.84:0/2550940528 learned_addr learned my addr 172.21.15.84:0/2550940528 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:54.426 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.423+0000 7f286d2a3700 1 -- 172.21.15.84:0/2550940528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2868101260 msgr2=0x7f28681999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:54.426 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.423+0000 7f286d2a3700 1 --2- 172.21.15.84:0/2550940528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2868101260 0x7f28681999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:54.426 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.423+0000 7f286d2a3700 1 -- 172.21.15.84:0/2550940528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28681072d0 msgr2=0x7f28681a4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:54.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.423+0000 7f286d2a3700 1 --2- 172.21.15.84:0/2550940528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28681072d0 0x7f28681a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:54.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.423+0000 7f286d2a3700 1 -- 172.21.15.84:0/2550940528 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28681a4f60 con 0x7f2868101c20 2024-05-22T23:00:54.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.423+0000 7f286d2a3700 1 --2- 172.21.15.84:0/2550940528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2868101c20 0x7f2868199ee0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f285c00c920 tx=0x7f285c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:54.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.423+0000 7f2866ffd700 1 -- 172.21.15.84:0/2550940528 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c007b00 con 0x7f2868101c20 2024-05-22T23:00:54.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.424+0000 7f286fd08700 1 -- 172.21.15.84:0/2550940528 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28681a5130 con 0x7f2868101c20 2024-05-22T23:00:54.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.424+0000 7f2866ffd700 1 -- 172.21.15.84:0/2550940528 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f285c004d10 con 0x7f2868101c20 2024-05-22T23:00:54.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.424+0000 7f2866ffd700 1 -- 172.21.15.84:0/2550940528 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c0056e0 con 0x7f2868101c20 2024-05-22T23:00:54.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.424+0000 7f286fd08700 1 -- 172.21.15.84:0/2550940528 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28681a56c0 con 0x7f2868101c20 2024-05-22T23:00:54.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.424+0000 7f286fd08700 1 -- 172.21.15.84:0/2550940528 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2868102980 con 0x7f2868101c20 2024-05-22T23:00:54.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.426+0000 7f2866ffd700 1 -- 172.21.15.84:0/2550940528 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f285c007ca0 con 0x7f2868101c20 2024-05-22T23:00:54.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.426+0000 7f2866ffd700 1 --2- 172.21.15.84:0/2550940528 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f28500a2cb0 0x7f28500a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:54.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.426+0000 7f286daa4700 1 --2- 172.21.15.84:0/2550940528 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f28500a2cb0 0x7f28500a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:54.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.426+0000 7f2866ffd700 1 -- 172.21.15.84:0/2550940528 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f285c0cee60 con 0x7f2868101c20 2024-05-22T23:00:54.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.427+0000 7f286daa4700 1 --2- 172.21.15.84:0/2550940528 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f28500a2cb0 0x7f28500a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2858007f80 tx=0x7f28580074d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:54.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.431+0000 7f2866ffd700 1 -- 172.21.15.84:0/2550940528 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f285c014030 con 0x7f2868101c20 2024-05-22T23:00:54.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.635+0000 7f286fd08700 1 -- 172.21.15.84:0/2550940528 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f286806b090 con 0x7f2868101c20 2024-05-22T23:00:54.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.635+0000 7f2866ffd700 1 -- 172.21.15.84:0/2550940528 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f285c09bc40 con 0x7f2868101c20 2024-05-22T23:00:54.636 INFO:teuthology.orchestra.run.smithi084.stdout:163208757306 2024-05-22T23:00:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.637+0000 7f286fd08700 1 -- 172.21.15.84:0/2550940528 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f28500a2cb0 msgr2=0x7f28500a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.637+0000 7f286fd08700 1 --2- 172.21.15.84:0/2550940528 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f28500a2cb0 0x7f28500a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2858007f80 tx=0x7f28580074d0 comp rx=0 tx=0).stop 2024-05-22T23:00:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.637+0000 7f286fd08700 1 -- 172.21.15.84:0/2550940528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2868101c20 msgr2=0x7f2868199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.637+0000 7f286fd08700 1 --2- 172.21.15.84:0/2550940528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2868101c20 0x7f2868199ee0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f285c00c920 tx=0x7f285c00cce0 comp rx=0 tx=0).stop 2024-05-22T23:00:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.637+0000 7f286fd08700 1 -- 172.21.15.84:0/2550940528 shutdown_connections 2024-05-22T23:00:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.637+0000 7f286fd08700 1 --2- 172.21.15.84:0/2550940528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28681072d0 0x7f28681a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.637+0000 7f286fd08700 1 --2- 172.21.15.84:0/2550940528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2868101260 0x7f28681999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.637+0000 7f286fd08700 1 --2- 172.21.15.84:0/2550940528 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f28500a2cb0 0x7f28500a5170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.637+0000 7f286fd08700 1 --2- 172.21.15.84:0/2550940528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2868101c20 0x7f2868199ee0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.637+0000 7f286fd08700 1 -- 172.21.15.84:0/2550940528 >> 172.21.15.84:0/2550940528 conn(0x7f2868078210 msgr2=0x7f28680ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.637+0000 7f286fd08700 1 -- 172.21.15.84:0/2550940528 shutdown_connections 2024-05-22T23:00:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:54.637+0000 7f286fd08700 1 -- 172.21.15.84:0/2550940528 wait complete. 2024-05-22T23:00:54.830 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:54 smithi084 ceph-mon[32186]: pgmap v322: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s rd, 0 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-05-22T23:00:54.851 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:54 smithi096 ceph-mon[36392]: pgmap v322: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s rd, 0 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-05-22T23:00:54.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:54 smithi204 ceph-mon[36211]: pgmap v322: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s rd, 0 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-05-22T23:00:55.847 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:55 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2550940528' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-22T23:00:55.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:55 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2550940528' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-22T23:00:56.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:55 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2550940528' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-22T23:00:56.285 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776160 got 292057776165 for osd.9 2024-05-22T23:00:56.286 DEBUG:teuthology.parallel:result is None 2024-05-22T23:00:56.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.612+0000 7fa44361f700 1 -- 172.21.15.84:0/3967846899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa43c1072d0 msgr2=0x7fa43c1076b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:56.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.612+0000 7fa44361f700 1 --2- 172.21.15.84:0/3967846899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa43c1072d0 0x7fa43c1076b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa43800b0a0 tx=0x7fa43800b3b0 comp rx=0 tx=0).stop 2024-05-22T23:00:56.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.612+0000 7fa44361f700 1 -- 172.21.15.84:0/3967846899 shutdown_connections 2024-05-22T23:00:56.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.612+0000 7fa44361f700 1 --2- 172.21.15.84:0/3967846899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa43c101c20 0x7fa43c10e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:56.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.612+0000 7fa44361f700 1 --2- 172.21.15.84:0/3967846899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa43c1072d0 0x7fa43c1076b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:56.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.612+0000 7fa44361f700 1 --2- 172.21.15.84:0/3967846899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa43c101260 0x7fa43c1016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:56.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.612+0000 7fa44361f700 1 -- 172.21.15.84:0/3967846899 >> 172.21.15.84:0/3967846899 conn(0x7fa43c078210 msgr2=0x7fa43c078620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:56.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.612+0000 7fa44361f700 1 -- 172.21.15.84:0/3967846899 shutdown_connections 2024-05-22T23:00:56.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.612+0000 7fa44361f700 1 -- 172.21.15.84:0/3967846899 wait complete. 2024-05-22T23:00:56.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.612+0000 7fa44361f700 1 Processor -- start 2024-05-22T23:00:56.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.612+0000 7fa44361f700 1 -- start start 2024-05-22T23:00:56.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.613+0000 7fa44361f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa43c101260 0x7fa43c199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:56.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.613+0000 7fa44361f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa43c101c20 0x7fa43c199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:56.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.613+0000 7fa44361f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa43c1072d0 0x7fa43c1a4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:56.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.613+0000 7fa44361f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1115f0 con 0x7fa43c101c20 2024-05-22T23:00:56.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.613+0000 7fa44361f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c111470 con 0x7fa43c101260 2024-05-22T23:00:56.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.613+0000 7fa44361f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c111770 con 0x7fa43c1072d0 2024-05-22T23:00:56.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.613+0000 7fa440bba700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa43c101c20 0x7fa43c199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:56.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.613+0000 7fa440bba700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa43c101c20 0x7fa43c199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39694/0 (socket says 172.21.15.84:39694) 2024-05-22T23:00:56.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.613+0000 7fa440bba700 1 -- 172.21.15.84:0/3209663178 learned_addr learned my addr 172.21.15.84:0/3209663178 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:56.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.613+0000 7fa441bbc700 1 --2- 172.21.15.84:0/3209663178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa43c1072d0 0x7fa43c1a4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:56.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.613+0000 7fa4413bb700 1 --2- 172.21.15.84:0/3209663178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa43c101260 0x7fa43c199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:56.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.613+0000 7fa440bba700 1 -- 172.21.15.84:0/3209663178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa43c1072d0 msgr2=0x7fa43c1a4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:56.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.613+0000 7fa440bba700 1 --2- 172.21.15.84:0/3209663178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa43c1072d0 0x7fa43c1a4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:56.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.613+0000 7fa440bba700 1 -- 172.21.15.84:0/3209663178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa43c101260 msgr2=0x7fa43c199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:56.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.613+0000 7fa440bba700 1 --2- 172.21.15.84:0/3209663178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa43c101260 0x7fa43c199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:56.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.613+0000 7fa440bba700 1 -- 172.21.15.84:0/3209663178 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43c1a4f50 con 0x7fa43c101c20 2024-05-22T23:00:56.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.613+0000 7fa440bba700 1 --2- 172.21.15.84:0/3209663178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa43c101c20 0x7fa43c199ed0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00b530 tx=0x7fa42c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:56.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.613+0000 7fa4327fc700 1 -- 172.21.15.84:0/3209663178 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c015020 con 0x7fa43c101c20 2024-05-22T23:00:56.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.613+0000 7fa44361f700 1 -- 172.21.15.84:0/3209663178 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa43c1a5180 con 0x7fa43c101c20 2024-05-22T23:00:56.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.613+0000 7fa4327fc700 1 -- 172.21.15.84:0/3209663178 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa42c011d10 con 0x7fa43c101c20 2024-05-22T23:00:56.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.613+0000 7fa4327fc700 1 -- 172.21.15.84:0/3209663178 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c00f2e0 con 0x7fa43c101c20 2024-05-22T23:00:56.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.613+0000 7fa44361f700 1 -- 172.21.15.84:0/3209663178 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa43c1a56e0 con 0x7fa43c101c20 2024-05-22T23:00:56.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.614+0000 7fa44361f700 1 -- 172.21.15.84:0/3209663178 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa43c069050 con 0x7fa43c101c20 2024-05-22T23:00:56.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.615+0000 7fa4327fc700 1 -- 172.21.15.84:0/3209663178 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa42c018020 con 0x7fa43c101c20 2024-05-22T23:00:56.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.616+0000 7fa4327fc700 1 --2- 172.21.15.84:0/3209663178 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa4280a2cb0 0x7fa4280a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:56.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.616+0000 7fa4413bb700 1 --2- 172.21.15.84:0/3209663178 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa4280a2cb0 0x7fa4280a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:56.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.616+0000 7fa4327fc700 1 -- 172.21.15.84:0/3209663178 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fa42c0ce790 con 0x7fa43c101c20 2024-05-22T23:00:56.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.616+0000 7fa4413bb700 1 --2- 172.21.15.84:0/3209663178 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa4280a2cb0 0x7fa4280a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa438009e60 tx=0x7fa43800b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:56.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.620+0000 7fa4327fc700 1 -- 172.21.15.84:0/3209663178 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa42c01f020 con 0x7fa43c101c20 2024-05-22T23:00:56.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.822+0000 7fa44361f700 1 -- 172.21.15.84:0/3209663178 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa43c02dbc0 con 0x7fa43c101c20 2024-05-22T23:00:56.824 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:56 smithi084 ceph-mon[32186]: pgmap v323: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 309 B/s rd, 0 op/s; 5.8 KiB/s, 0 objects/s recovering 2024-05-22T23:00:56.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.822+0000 7fa4327fc700 1 -- 172.21.15.84:0/3209663178 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa42c09b680 con 0x7fa43c101c20 2024-05-22T23:00:56.824 INFO:teuthology.orchestra.run.smithi084.stdout:34359738451 2024-05-22T23:00:56.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.825+0000 7fa44361f700 1 -- 172.21.15.84:0/3209663178 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa4280a2cb0 msgr2=0x7fa4280a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:56.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.825+0000 7fa44361f700 1 --2- 172.21.15.84:0/3209663178 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa4280a2cb0 0x7fa4280a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa438009e60 tx=0x7fa43800b700 comp rx=0 tx=0).stop 2024-05-22T23:00:56.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.825+0000 7fa44361f700 1 -- 172.21.15.84:0/3209663178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa43c101c20 msgr2=0x7fa43c199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:56.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.825+0000 7fa44361f700 1 --2- 172.21.15.84:0/3209663178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa43c101c20 0x7fa43c199ed0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00b530 tx=0x7fa42c00b840 comp rx=0 tx=0).stop 2024-05-22T23:00:56.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.825+0000 7fa44361f700 1 -- 172.21.15.84:0/3209663178 shutdown_connections 2024-05-22T23:00:56.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.825+0000 7fa44361f700 1 --2- 172.21.15.84:0/3209663178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa43c1072d0 0x7fa43c1a4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:56.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.825+0000 7fa44361f700 1 --2- 172.21.15.84:0/3209663178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa43c101260 0x7fa43c199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:56.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.825+0000 7fa44361f700 1 --2- 172.21.15.84:0/3209663178 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa4280a2cb0 0x7fa4280a5170 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:56.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.825+0000 7fa44361f700 1 --2- 172.21.15.84:0/3209663178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa43c101c20 0x7fa43c199ed0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:56.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.825+0000 7fa44361f700 1 -- 172.21.15.84:0/3209663178 >> 172.21.15.84:0/3209663178 conn(0x7fa43c078210 msgr2=0x7fa43c0ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:56.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.825+0000 7fa44361f700 1 -- 172.21.15.84:0/3209663178 shutdown_connections 2024-05-22T23:00:56.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:56.825+0000 7fa44361f700 1 -- 172.21.15.84:0/3209663178 wait complete. 2024-05-22T23:00:56.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:56 smithi204 ceph-mon[36211]: pgmap v323: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 309 B/s rd, 0 op/s; 5.8 KiB/s, 0 objects/s recovering 2024-05-22T23:00:57.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:56 smithi096 ceph-mon[36392]: pgmap v323: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 309 B/s rd, 0 op/s; 5.8 KiB/s, 0 objects/s recovering 2024-05-22T23:00:57.687 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:57 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3209663178' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-22T23:00:57.712 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738444 got 34359738451 for osd.0 2024-05-22T23:00:57.712 DEBUG:teuthology.parallel:result is None 2024-05-22T23:00:57.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:57 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3209663178' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-22T23:00:58.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:57 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3209663178' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-22T23:00:58.395 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757299 got 163208757306 for osd.5 2024-05-22T23:00:58.396 DEBUG:teuthology.parallel:result is None 2024-05-22T23:00:58.624 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:58 smithi084 ceph-mon[32186]: pgmap v324: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 749 B/s rd, 1 op/s; 5.3 KiB/s, 0 objects/s recovering 2024-05-22T23:00:58.658 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:00:58.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.665+0000 7f9c65b93700 1 -- 172.21.15.84:0/1408725739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c601065e0 msgr2=0x7f9c6010e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:58.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.665+0000 7f9c65b93700 1 --2- 172.21.15.84:0/1408725739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c601065e0 0x7f9c6010e1f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9c5800b0d0 tx=0x7f9c5800b3e0 comp rx=0 tx=0).stop 2024-05-22T23:00:58.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.665+0000 7f9c65b93700 1 -- 172.21.15.84:0/1408725739 shutdown_connections 2024-05-22T23:00:58.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.665+0000 7f9c65b93700 1 --2- 172.21.15.84:0/1408725739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c601065e0 0x7f9c6010e1f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:58.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.665+0000 7f9c65b93700 1 --2- 172.21.15.84:0/1408725739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c60101cc0 0x7f9c601020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:58.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.665+0000 7f9c65b93700 1 --2- 172.21.15.84:0/1408725739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c60102670 0x7f9c60105f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:58.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.665+0000 7f9c65b93700 1 -- 172.21.15.84:0/1408725739 >> 172.21.15.84:0/1408725739 conn(0x7f9c600778d0 msgr2=0x7f9c600fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:58.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.666+0000 7f9c65b93700 1 -- 172.21.15.84:0/1408725739 shutdown_connections 2024-05-22T23:00:58.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.666+0000 7f9c65b93700 1 -- 172.21.15.84:0/1408725739 wait complete. 2024-05-22T23:00:58.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.666+0000 7f9c65b93700 1 Processor -- start 2024-05-22T23:00:58.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.667+0000 7f9c65b93700 1 -- start start 2024-05-22T23:00:58.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.667+0000 7f9c65b93700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c60101cc0 0x7f9c60199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:58.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.667+0000 7f9c65b93700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c60102670 0x7f9c60199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:58.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.667+0000 7f9c65b93700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c601065e0 0x7f9c601a4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:58.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.667+0000 7f9c5f7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c60101cc0 0x7f9c60199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:58.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.667+0000 7f9c5f7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c60101cc0 0x7f9c60199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39722/0 (socket says 172.21.15.84:39722) 2024-05-22T23:00:58.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.667+0000 7f9c5f7fe700 1 -- 172.21.15.84:0/3240715759 learned_addr learned my addr 172.21.15.84:0/3240715759 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:58.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.667+0000 7f9c65b93700 1 -- 172.21.15.84:0/3240715759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9c60111620 con 0x7f9c60101cc0 2024-05-22T23:00:58.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.667+0000 7f9c5ffff700 1 --2- 172.21.15.84:0/3240715759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c601065e0 0x7f9c601a4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:58.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.667+0000 7f9c5effd700 1 --2- 172.21.15.84:0/3240715759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c60102670 0x7f9c60199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:58.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.667+0000 7f9c65b93700 1 -- 172.21.15.84:0/3240715759 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9c601114a0 con 0x7f9c601065e0 2024-05-22T23:00:58.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.667+0000 7f9c65b93700 1 -- 172.21.15.84:0/3240715759 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9c601117a0 con 0x7f9c60102670 2024-05-22T23:00:58.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.668+0000 7f9c5f7fe700 1 -- 172.21.15.84:0/3240715759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c60102670 msgr2=0x7f9c60199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:58.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.668+0000 7f9c5f7fe700 1 --2- 172.21.15.84:0/3240715759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c60102670 0x7f9c60199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:58.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.668+0000 7f9c5f7fe700 1 -- 172.21.15.84:0/3240715759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c601065e0 msgr2=0x7f9c601a4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:58.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.668+0000 7f9c5f7fe700 1 --2- 172.21.15.84:0/3240715759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c601065e0 0x7f9c601a4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:58.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.668+0000 7f9c5f7fe700 1 -- 172.21.15.84:0/3240715759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c601a4f40 con 0x7f9c60101cc0 2024-05-22T23:00:58.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.668+0000 7f9c5effd700 1 --2- 172.21.15.84:0/3240715759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c60102670 0x7f9c60199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:00:58.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.668+0000 7f9c5f7fe700 1 --2- 172.21.15.84:0/3240715759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c60101cc0 0x7f9c60199980 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9c5000d980 tx=0x7f9c5000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:58.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.668+0000 7f9c5cff9700 1 -- 172.21.15.84:0/3240715759 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c50016070 con 0x7f9c60101cc0 2024-05-22T23:00:58.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.668+0000 7f9c65b93700 1 -- 172.21.15.84:0/3240715759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c601a5170 con 0x7f9c60101cc0 2024-05-22T23:00:58.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.668+0000 7f9c5cff9700 1 -- 172.21.15.84:0/3240715759 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c50004d10 con 0x7f9c60101cc0 2024-05-22T23:00:58.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.668+0000 7f9c5cff9700 1 -- 172.21.15.84:0/3240715759 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c500056a0 con 0x7f9c60101cc0 2024-05-22T23:00:58.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.668+0000 7f9c65b93700 1 -- 172.21.15.84:0/3240715759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c601a56d0 con 0x7f9c60101cc0 2024-05-22T23:00:58.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.669+0000 7f9c65b93700 1 -- 172.21.15.84:0/3240715759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c60102a70 con 0x7f9c60101cc0 2024-05-22T23:00:58.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.672+0000 7f9c5cff9700 1 -- 172.21.15.84:0/3240715759 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9c50004750 con 0x7f9c60101cc0 2024-05-22T23:00:58.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.673+0000 7f9c5cff9700 1 --2- 172.21.15.84:0/3240715759 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9c480a2db0 0x7f9c480a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:58.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.674+0000 7f9c5effd700 1 --2- 172.21.15.84:0/3240715759 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9c480a2db0 0x7f9c480a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:58.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.674+0000 7f9c5cff9700 1 -- 172.21.15.84:0/3240715759 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f9c500cf470 con 0x7f9c60101cc0 2024-05-22T23:00:58.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.674+0000 7f9c5effd700 1 --2- 172.21.15.84:0/3240715759 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9c480a2db0 0x7f9c480a5270 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9c60103da0 tx=0x7f9c54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:58.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.676+0000 7f9c5cff9700 1 -- 172.21.15.84:0/3240715759 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9c5001f020 con 0x7f9c60101cc0 2024-05-22T23:00:58.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.890+0000 7f9c65b93700 1 -- 172.21.15.84:0/3240715759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9c6002dbc0 con 0x7f9c60101cc0 2024-05-22T23:00:58.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.891+0000 7f9c5cff9700 1 -- 172.21.15.84:0/3240715759 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9c50004220 con 0x7f9c60101cc0 2024-05-22T23:00:58.891 INFO:teuthology.orchestra.run.smithi084.stdout:55834574926 2024-05-22T23:00:58.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.893+0000 7f9c65b93700 1 -- 172.21.15.84:0/3240715759 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9c480a2db0 msgr2=0x7f9c480a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:58.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.893+0000 7f9c65b93700 1 --2- 172.21.15.84:0/3240715759 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9c480a2db0 0x7f9c480a5270 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9c60103da0 tx=0x7f9c54009260 comp rx=0 tx=0).stop 2024-05-22T23:00:58.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.893+0000 7f9c65b93700 1 -- 172.21.15.84:0/3240715759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c60101cc0 msgr2=0x7f9c60199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:58.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.893+0000 7f9c65b93700 1 --2- 172.21.15.84:0/3240715759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c60101cc0 0x7f9c60199980 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9c5000d980 tx=0x7f9c5000dc90 comp rx=0 tx=0).stop 2024-05-22T23:00:58.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.893+0000 7f9c65b93700 1 -- 172.21.15.84:0/3240715759 shutdown_connections 2024-05-22T23:00:58.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.893+0000 7f9c65b93700 1 --2- 172.21.15.84:0/3240715759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c601065e0 0x7f9c601a4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:58.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.893+0000 7f9c65b93700 1 --2- 172.21.15.84:0/3240715759 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9c480a2db0 0x7f9c480a5270 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:58.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.893+0000 7f9c65b93700 1 --2- 172.21.15.84:0/3240715759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c60101cc0 0x7f9c60199980 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:58.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.893+0000 7f9c65b93700 1 --2- 172.21.15.84:0/3240715759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c60102670 0x7f9c60199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:58.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.893+0000 7f9c65b93700 1 -- 172.21.15.84:0/3240715759 >> 172.21.15.84:0/3240715759 conn(0x7f9c600778d0 msgr2=0x7f9c60104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:58.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.893+0000 7f9c65b93700 1 -- 172.21.15.84:0/3240715759 shutdown_connections 2024-05-22T23:00:58.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:58.893+0000 7f9c65b93700 1 -- 172.21.15.84:0/3240715759 wait complete. 2024-05-22T23:00:58.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:58 smithi204 ceph-mon[36211]: pgmap v324: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 749 B/s rd, 1 op/s; 5.3 KiB/s, 0 objects/s recovering 2024-05-22T23:00:59.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:58 smithi096 ceph-mon[36392]: pgmap v324: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 749 B/s rd, 1 op/s; 5.3 KiB/s, 0 objects/s recovering 2024-05-22T23:00:59.756 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:00:59 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3240715759' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-22T23:00:59.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.763+0000 7f8dc8338700 1 -- 172.21.15.84:0/505074042 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dc01083d0 msgr2=0x7f8dc01087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:59.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.763+0000 7f8dc8338700 1 --2- 172.21.15.84:0/505074042 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dc01083d0 0x7f8dc01087b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8db0009990 tx=0x7f8db0009ca0 comp rx=0 tx=0).stop 2024-05-22T23:00:59.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.763+0000 7f8dc8338700 1 -- 172.21.15.84:0/505074042 shutdown_connections 2024-05-22T23:00:59.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.763+0000 7f8dc8338700 1 --2- 172.21.15.84:0/505074042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dc006b2a0 0x7f8dc010f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:59.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.763+0000 7f8dc8338700 1 --2- 172.21.15.84:0/505074042 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dc01083d0 0x7f8dc01087b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:59.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.763+0000 7f8dc8338700 1 --2- 172.21.15.84:0/505074042 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8dc006a900 0x7f8dc006ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:59.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.763+0000 7f8dc8338700 1 -- 172.21.15.84:0/505074042 >> 172.21.15.84:0/505074042 conn(0x7f8dc00781d0 msgr2=0x7f8dc00785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:59.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.763+0000 7f8dc8338700 1 -- 172.21.15.84:0/505074042 shutdown_connections 2024-05-22T23:00:59.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.763+0000 7f8dc8338700 1 -- 172.21.15.84:0/505074042 wait complete. 2024-05-22T23:00:59.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.764+0000 7f8dc8338700 1 Processor -- start 2024-05-22T23:00:59.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.764+0000 7f8dc8338700 1 -- start start 2024-05-22T23:00:59.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.764+0000 7f8dc8338700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dc006a900 0x7f8dc019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:59.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.764+0000 7f8dc8338700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8dc006b2a0 0x7f8dc019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:59.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.764+0000 7f8dc8338700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dc01083d0 0x7f8dc0198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:59.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.764+0000 7f8dc8338700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc0112630 con 0x7f8dc006a900 2024-05-22T23:00:59.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.764+0000 7f8dc8338700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc01124b0 con 0x7f8dc01083d0 2024-05-22T23:00:59.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.764+0000 7f8dc8338700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc01127b0 con 0x7f8dc006b2a0 2024-05-22T23:00:59.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.764+0000 7f8dc60d4700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dc006a900 0x7f8dc019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:59.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.764+0000 7f8dc60d4700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dc006a900 0x7f8dc019e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39764/0 (socket says 172.21.15.84:39764) 2024-05-22T23:00:59.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.764+0000 7f8dc60d4700 1 -- 172.21.15.84:0/3086599395 learned_addr learned my addr 172.21.15.84:0/3086599395 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:59.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.764+0000 7f8dc58d3700 1 --2- 172.21.15.84:0/3086599395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8dc006b2a0 0x7f8dc019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:59.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.764+0000 7f8dc68d5700 1 --2- 172.21.15.84:0/3086599395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dc01083d0 0x7f8dc0198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:59.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.764+0000 7f8dc60d4700 1 -- 172.21.15.84:0/3086599395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8dc006b2a0 msgr2=0x7f8dc019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:59.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.764+0000 7f8dc60d4700 1 --2- 172.21.15.84:0/3086599395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8dc006b2a0 0x7f8dc019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:59.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.764+0000 7f8dc60d4700 1 -- 172.21.15.84:0/3086599395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dc01083d0 msgr2=0x7f8dc0198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:59.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.764+0000 7f8dc60d4700 1 --2- 172.21.15.84:0/3086599395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dc01083d0 0x7f8dc0198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:59.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.764+0000 7f8dc60d4700 1 -- 172.21.15.84:0/3086599395 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc01990b0 con 0x7f8dc006a900 2024-05-22T23:00:59.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.764+0000 7f8dc68d5700 1 --2- 172.21.15.84:0/3086599395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dc01083d0 0x7f8dc0198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:00:59.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.764+0000 7f8dc60d4700 1 --2- 172.21.15.84:0/3086599395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dc006a900 0x7f8dc019e790 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8db0000c00 tx=0x7f8db000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:59.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.765+0000 7f8db77fe700 1 -- 172.21.15.84:0/3086599395 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db0027070 con 0x7f8dc006a900 2024-05-22T23:00:59.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.765+0000 7f8dc8338700 1 -- 172.21.15.84:0/3086599395 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dc0199340 con 0x7f8dc006a900 2024-05-22T23:00:59.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.765+0000 7f8db77fe700 1 -- 172.21.15.84:0/3086599395 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8db0003d70 con 0x7f8dc006a900 2024-05-22T23:00:59.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.765+0000 7f8db77fe700 1 -- 172.21.15.84:0/3086599395 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db000f6c0 con 0x7f8dc006a900 2024-05-22T23:00:59.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.765+0000 7f8dc8338700 1 -- 172.21.15.84:0/3086599395 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dc01a5660 con 0x7f8dc006a900 2024-05-22T23:00:59.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.766+0000 7f8dc8338700 1 -- 172.21.15.84:0/3086599395 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dc0069050 con 0x7f8dc006a900 2024-05-22T23:00:59.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.767+0000 7f8db77fe700 1 -- 172.21.15.84:0/3086599395 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8db002a070 con 0x7f8dc006a900 2024-05-22T23:00:59.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.767+0000 7f8db77fe700 1 --2- 172.21.15.84:0/3086599395 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8dac0a2cb0 0x7f8dac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:59.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.768+0000 7f8dc58d3700 1 --2- 172.21.15.84:0/3086599395 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8dac0a2cb0 0x7f8dac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:59.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.768+0000 7f8db77fe700 1 -- 172.21.15.84:0/3086599395 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f8db00d06e0 con 0x7f8dc006a900 2024-05-22T23:00:59.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.768+0000 7f8dc58d3700 1 --2- 172.21.15.84:0/3086599395 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8dac0a2cb0 0x7f8dac0a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8dc00788b0 tx=0x7f8dbc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:59.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.771+0000 7f8db77fe700 1 -- 172.21.15.84:0/3086599395 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8db0031070 con 0x7f8dc006a900 2024-05-22T23:00:59.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.672+0000 7f5ce09b7700 1 -- 172.21.15.84:0/443682276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5cdc1013e0 msgr2=0x7f5cdc10d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:59.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.672+0000 7f5ce09b7700 1 --2- 172.21.15.84:0/443682276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5cdc1013e0 0x7f5cdc10d8a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5cd400b0a0 tx=0x7f5cd400b3b0 comp rx=0 tx=0).stop 2024-05-22T23:00:59.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.672+0000 7f5ce09b7700 1 -- 172.21.15.84:0/443682276 shutdown_connections 2024-05-22T23:00:59.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.672+0000 7f5ce09b7700 1 --2- 172.21.15.84:0/443682276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5cdc1013e0 0x7f5cdc10d8a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:59.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.672+0000 7f5ce09b7700 1 --2- 172.21.15.84:0/443682276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cdc1069d0 0x7f5cdc106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:59.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.672+0000 7f5ce09b7700 1 --2- 172.21.15.84:0/443682276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5cdc100a20 0x7f5cdc100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:59.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.672+0000 7f5ce09b7700 1 -- 172.21.15.84:0/443682276 >> 172.21.15.84:0/443682276 conn(0x7f5cdc0778d0 msgr2=0x7f5cdc0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:59.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.672+0000 7f5ce09b7700 1 -- 172.21.15.84:0/443682276 shutdown_connections 2024-05-22T23:00:59.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.672+0000 7f5ce09b7700 1 -- 172.21.15.84:0/443682276 wait complete. 2024-05-22T23:00:59.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.673+0000 7f5ce09b7700 1 Processor -- start 2024-05-22T23:00:59.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.673+0000 7f5ce09b7700 1 -- start start 2024-05-22T23:00:59.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.674+0000 7f5ce09b7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cdc100a20 0x7f5cdc19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:59.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.674+0000 7f5ce09b7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5cdc1013e0 0x7f5cdc19db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:59.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.674+0000 7f5ce09b7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5cdc1069d0 0x7f5cdc197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:59.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.674+0000 7f5ce09b7700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc110d60 con 0x7f5cdc1013e0 2024-05-22T23:00:59.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.674+0000 7f5ce09b7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc110be0 con 0x7f5cdc1069d0 2024-05-22T23:00:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.674+0000 7f5ce09b7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc110ee0 con 0x7f5cdc100a20 2024-05-22T23:00:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.674+0000 7f5cd9d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5cdc1013e0 0x7f5cdc19db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.674+0000 7f5cd9d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5cdc1013e0 0x7f5cdc19db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39746/0 (socket says 172.21.15.84:39746) 2024-05-22T23:00:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.674+0000 7f5cd9d9b700 1 -- 172.21.15.84:0/759350948 learned_addr learned my addr 172.21.15.84:0/759350948 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:00:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.674+0000 7f5cda59c700 1 --2- 172.21.15.84:0/759350948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cdc100a20 0x7f5cdc19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.674+0000 7f5cdad9d700 1 --2- 172.21.15.84:0/759350948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5cdc1069d0 0x7f5cdc197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.674+0000 7f5cd9d9b700 1 -- 172.21.15.84:0/759350948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cdc100a20 msgr2=0x7f5cdc19d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.674+0000 7f5cd9d9b700 1 --2- 172.21.15.84:0/759350948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cdc100a20 0x7f5cdc19d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.674+0000 7f5cd9d9b700 1 -- 172.21.15.84:0/759350948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5cdc1069d0 msgr2=0x7f5cdc197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.674+0000 7f5cd9d9b700 1 --2- 172.21.15.84:0/759350948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5cdc1069d0 0x7f5cdc197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.674+0000 7f5cd9d9b700 1 -- 172.21.15.84:0/759350948 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cdc198000 con 0x7f5cdc1013e0 2024-05-22T23:00:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.674+0000 7f5cdad9d700 1 --2- 172.21.15.84:0/759350948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5cdc1069d0 0x7f5cdc197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:00:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.674+0000 7f5cd9d9b700 1 --2- 172.21.15.84:0/759350948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5cdc1013e0 0x7f5cdc19db90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000d950 tx=0x7f5cd000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.674+0000 7f5ccb7fe700 1 -- 172.21.15.84:0/759350948 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0016070 con 0x7f5cdc1013e0 2024-05-22T23:00:59.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.675+0000 7f5ce09b7700 1 -- 172.21.15.84:0/759350948 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cdc1982f0 con 0x7f5cdc1013e0 2024-05-22T23:00:59.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.675+0000 7f5ce09b7700 1 -- 172.21.15.84:0/759350948 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cdc0fcc70 con 0x7f5cdc1013e0 2024-05-22T23:00:59.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.675+0000 7f5ccb7fe700 1 -- 172.21.15.84:0/759350948 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd0009b40 con 0x7f5cdc1013e0 2024-05-22T23:00:59.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.675+0000 7f5ccb7fe700 1 -- 172.21.15.84:0/759350948 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0020060 con 0x7f5cdc1013e0 2024-05-22T23:00:59.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.676+0000 7f5ce09b7700 1 -- 172.21.15.84:0/759350948 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cbc005330 con 0x7f5cdc1013e0 2024-05-22T23:00:59.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.677+0000 7f5ccb7fe700 1 -- 172.21.15.84:0/759350948 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5cd0019070 con 0x7f5cdc1013e0 2024-05-22T23:00:59.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.677+0000 7f5ccb7fe700 1 --2- 172.21.15.84:0/759350948 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f5cc40a2cb0 0x7f5cc40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:00:59.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.678+0000 7f5cda59c700 1 --2- 172.21.15.84:0/759350948 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f5cc40a2cb0 0x7f5cc40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:00:59.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.678+0000 7f5ccb7fe700 1 -- 172.21.15.84:0/759350948 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f5cd00cf2b0 con 0x7f5cdc1013e0 2024-05-22T23:00:59.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.678+0000 7f5cda59c700 1 --2- 172.21.15.84:0/759350948 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f5cc40a2cb0 0x7f5cc40a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00bbb0 tx=0x7f5ccc00b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:00:59.902 INFO:teuthology.orchestra.run.smithi084.stdout:141733920832 2024-05-22T23:00:59.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.682+0000 7f5ccb7fe700 1 -- 172.21.15.84:0/759350948 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5cd009c090 con 0x7f5cdc1013e0 2024-05-22T23:00:59.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.891+0000 7f5ce09b7700 1 -- 172.21.15.84:0/759350948 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f5cbc0058f0 con 0x7f5cdc1013e0 2024-05-22T23:00:59.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.892+0000 7f5ccb7fe700 1 -- 172.21.15.84:0/759350948 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5cd009b990 con 0x7f5cdc1013e0 2024-05-22T23:00:59.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.894+0000 7f5ce09b7700 1 -- 172.21.15.84:0/759350948 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f5cc40a2cb0 msgr2=0x7f5cc40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:59.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.894+0000 7f5ce09b7700 1 --2- 172.21.15.84:0/759350948 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f5cc40a2cb0 0x7f5cc40a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00bbb0 tx=0x7f5ccc00b590 comp rx=0 tx=0).stop 2024-05-22T23:00:59.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.894+0000 7f5ce09b7700 1 -- 172.21.15.84:0/759350948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5cdc1013e0 msgr2=0x7f5cdc19db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:59.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.894+0000 7f5ce09b7700 1 --2- 172.21.15.84:0/759350948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5cdc1013e0 0x7f5cdc19db90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000d950 tx=0x7f5cd000dd10 comp rx=0 tx=0).stop 2024-05-22T23:00:59.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.894+0000 7f5ce09b7700 1 -- 172.21.15.84:0/759350948 shutdown_connections 2024-05-22T23:00:59.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.894+0000 7f5ce09b7700 1 --2- 172.21.15.84:0/759350948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5cdc1069d0 0x7f5cdc197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:59.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.894+0000 7f5ce09b7700 1 --2- 172.21.15.84:0/759350948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cdc100a20 0x7f5cdc19d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:59.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.894+0000 7f5ce09b7700 1 --2- 172.21.15.84:0/759350948 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f5cc40a2cb0 0x7f5cc40a5170 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:59.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.894+0000 7f5ce09b7700 1 --2- 172.21.15.84:0/759350948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5cdc1013e0 0x7f5cdc19db90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:59.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.894+0000 7f5ce09b7700 1 -- 172.21.15.84:0/759350948 >> 172.21.15.84:0/759350948 conn(0x7f5cdc0778d0 msgr2=0x7f5cdc0698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:59.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.894+0000 7f5ce09b7700 1 -- 172.21.15.84:0/759350948 shutdown_connections 2024-05-22T23:00:59.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.894+0000 7f5ce09b7700 1 -- 172.21.15.84:0/759350948 wait complete. 2024-05-22T23:00:59.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:00:59 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3240715759' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-22T23:00:59.976 INFO:teuthology.orchestra.run.smithi084.stdout:313532612641 2024-05-22T23:00:59.977 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.976+0000 7f8dc8338700 1 -- 172.21.15.84:0/3086599395 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f8dc002dbc0 con 0x7f8dc006a900 2024-05-22T23:00:59.977 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.976+0000 7f8db77fe700 1 -- 172.21.15.84:0/3086599395 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f8db0013070 con 0x7f8dc006a900 2024-05-22T23:00:59.978 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.978+0000 7f8dc8338700 1 -- 172.21.15.84:0/3086599395 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8dac0a2cb0 msgr2=0x7f8dac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:59.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.978+0000 7f8dc8338700 1 --2- 172.21.15.84:0/3086599395 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8dac0a2cb0 0x7f8dac0a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8dc00788b0 tx=0x7f8dbc009260 comp rx=0 tx=0).stop 2024-05-22T23:00:59.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.978+0000 7f8dc8338700 1 -- 172.21.15.84:0/3086599395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dc006a900 msgr2=0x7f8dc019e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:00:59.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.978+0000 7f8dc8338700 1 --2- 172.21.15.84:0/3086599395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dc006a900 0x7f8dc019e790 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8db0000c00 tx=0x7f8db000bab0 comp rx=0 tx=0).stop 2024-05-22T23:00:59.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.978+0000 7f8dc8338700 1 -- 172.21.15.84:0/3086599395 shutdown_connections 2024-05-22T23:00:59.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.978+0000 7f8dc8338700 1 --2- 172.21.15.84:0/3086599395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dc01083d0 0x7f8dc0198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:59.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.978+0000 7f8dc8338700 1 --2- 172.21.15.84:0/3086599395 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8dac0a2cb0 0x7f8dac0a5170 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:59.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.978+0000 7f8dc8338700 1 --2- 172.21.15.84:0/3086599395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dc006a900 0x7f8dc019e790 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:59.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.978+0000 7f8dc8338700 1 --2- 172.21.15.84:0/3086599395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8dc006b2a0 0x7f8dc019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:00:59.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.978+0000 7f8dc8338700 1 -- 172.21.15.84:0/3086599395 >> 172.21.15.84:0/3086599395 conn(0x7f8dc00781d0 msgr2=0x7f8dc0100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:00:59.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.978+0000 7f8dc8338700 1 -- 172.21.15.84:0/3086599395 shutdown_connections 2024-05-22T23:00:59.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:00:59.978+0000 7f8dc8338700 1 -- 172.21.15.84:0/3086599395 wait complete. 2024-05-22T23:01:00.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:00:59 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3240715759' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-22T23:01:00.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:00 smithi084 ceph-mon[32186]: pgmap v325: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-05-22T23:01:00.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:00 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/759350948' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-22T23:01:00.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:00 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3086599395' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-22T23:01:00.908 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574921 got 55834574926 for osd.1 2024-05-22T23:01:00.908 DEBUG:teuthology.parallel:result is None 2024-05-22T23:01:00.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:00 smithi204 ceph-mon[36211]: pgmap v325: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-05-22T23:01:00.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:00 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/759350948' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-22T23:01:00.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:00 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3086599395' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-22T23:01:01.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:00 smithi096 ceph-mon[36392]: pgmap v325: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-05-22T23:01:01.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:00 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/759350948' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-22T23:01:01.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:00 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3086599395' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-22T23:01:01.311 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612636 got 313532612641 for osd.10 2024-05-22T23:01:01.312 DEBUG:teuthology.parallel:result is None 2024-05-22T23:01:01.855 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:01 smithi084 ceph-mon[32186]: pgmap v326: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:01:01.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:01 smithi204 ceph-mon[36211]: pgmap v326: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:01:02.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:01 smithi096 ceph-mon[36392]: pgmap v326: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:01:02.580 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920826 got 141733920832 for osd.4 2024-05-22T23:01:02.580 DEBUG:teuthology.parallel:result is None 2024-05-22T23:01:04.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.209+0000 7fb4c22e6700 1 -- 172.21.15.84:0/2743766518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4bc102fc0 msgr2=0x7fb4bc106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:04.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.209+0000 7fb4c22e6700 1 --2- 172.21.15.84:0/2743766518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4bc102fc0 0x7fb4bc106fc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4009990 tx=0x7fb4a4009ca0 comp rx=0 tx=0).stop 2024-05-22T23:01:04.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.209+0000 7fb4c22e6700 1 -- 172.21.15.84:0/2743766518 shutdown_connections 2024-05-22T23:01:04.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.209+0000 7fb4c22e6700 1 --2- 172.21.15.84:0/2743766518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4bc1076a0 0x7fb4bc10f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:04.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.209+0000 7fb4c22e6700 1 --2- 172.21.15.84:0/2743766518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4bc102610 0x7fb4bc1029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:04.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.209+0000 7fb4c22e6700 1 --2- 172.21.15.84:0/2743766518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4bc102fc0 0x7fb4bc106fc0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:04.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.209+0000 7fb4c22e6700 1 -- 172.21.15.84:0/2743766518 >> 172.21.15.84:0/2743766518 conn(0x7fb4bc0794c0 msgr2=0x7fb4bc0798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:04.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.209+0000 7fb4c22e6700 1 -- 172.21.15.84:0/2743766518 shutdown_connections 2024-05-22T23:01:04.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.209+0000 7fb4c22e6700 1 -- 172.21.15.84:0/2743766518 wait complete. 2024-05-22T23:01:04.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.209+0000 7fb4c22e6700 1 Processor -- start 2024-05-22T23:01:04.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.209+0000 7fb4c22e6700 1 -- start start 2024-05-22T23:01:04.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.209+0000 7fb4c22e6700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4bc102610 0x7fb4bc19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:04.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.209+0000 7fb4c22e6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4bc102fc0 0x7fb4bc19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:04.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.210+0000 7fb4c22e6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4bc1076a0 0x7fb4bc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:04.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.210+0000 7fb4c22e6700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc1126c0 con 0x7fb4bc102610 2024-05-22T23:01:04.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.210+0000 7fb4c22e6700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc112540 con 0x7fb4bc102fc0 2024-05-22T23:01:04.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.210+0000 7fb4c22e6700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc112840 con 0x7fb4bc1076a0 2024-05-22T23:01:04.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.210+0000 7fb4bbfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4bc102610 0x7fb4bc19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:04.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.210+0000 7fb4bbfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4bc102610 0x7fb4bc19e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59490/0 (socket says 172.21.15.84:59490) 2024-05-22T23:01:04.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.210+0000 7fb4bbfff700 1 -- 172.21.15.84:0/3327903644 learned_addr learned my addr 172.21.15.84:0/3327903644 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:04.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.210+0000 7fb4b3fff700 1 --2- 172.21.15.84:0/3327903644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4bc102fc0 0x7fb4bc19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:04.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.210+0000 7fb4c0883700 1 --2- 172.21.15.84:0/3327903644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4bc1076a0 0x7fb4bc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:04.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.210+0000 7fb4bbfff700 1 -- 172.21.15.84:0/3327903644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4bc1076a0 msgr2=0x7fb4bc198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:04.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.210+0000 7fb4bbfff700 1 --2- 172.21.15.84:0/3327903644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4bc1076a0 0x7fb4bc198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:04.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.210+0000 7fb4bbfff700 1 -- 172.21.15.84:0/3327903644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4bc102fc0 msgr2=0x7fb4bc19ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:04.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.210+0000 7fb4bbfff700 1 --2- 172.21.15.84:0/3327903644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4bc102fc0 0x7fb4bc19ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:04.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.210+0000 7fb4bbfff700 1 -- 172.21.15.84:0/3327903644 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4bc1990a0 con 0x7fb4bc102610 2024-05-22T23:01:04.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.210+0000 7fb4b3fff700 1 --2- 172.21.15.84:0/3327903644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4bc102fc0 0x7fb4bc19ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:01:04.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.210+0000 7fb4bbfff700 1 --2- 172.21.15.84:0/3327903644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4bc102610 0x7fb4bc19e730 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00c9b0 tx=0x7fb4ac00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:04.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.210+0000 7fb4b9ffb700 1 -- 172.21.15.84:0/3327903644 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac007b00 con 0x7fb4bc102610 2024-05-22T23:01:04.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.210+0000 7fb4b9ffb700 1 -- 172.21.15.84:0/3327903644 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4ac007ca0 con 0x7fb4bc102610 2024-05-22T23:01:04.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.210+0000 7fb4c22e6700 1 -- 172.21.15.84:0/3327903644 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4bc199390 con 0x7fb4bc102610 2024-05-22T23:01:04.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.210+0000 7fb4b9ffb700 1 -- 172.21.15.84:0/3327903644 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac018730 con 0x7fb4bc102610 2024-05-22T23:01:04.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.210+0000 7fb4c22e6700 1 -- 172.21.15.84:0/3327903644 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4bc1a5600 con 0x7fb4bc102610 2024-05-22T23:01:04.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.211+0000 7fb4c22e6700 1 -- 172.21.15.84:0/3327903644 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4bc069050 con 0x7fb4bc102610 2024-05-22T23:01:04.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.213+0000 7fb4b9ffb700 1 -- 172.21.15.84:0/3327903644 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb4ac0188d0 con 0x7fb4bc102610 2024-05-22T23:01:04.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.214+0000 7fb4b9ffb700 1 --2- 172.21.15.84:0/3327903644 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb4a80a2cb0 0x7fb4a80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:04.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.214+0000 7fb4b9ffb700 1 -- 172.21.15.84:0/3327903644 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fb4ac0cf490 con 0x7fb4bc102610 2024-05-22T23:01:04.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.214+0000 7fb4b3fff700 1 --2- 172.21.15.84:0/3327903644 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb4a80a2cb0 0x7fb4a80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:04.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.214+0000 7fb4b3fff700 1 --2- 172.21.15.84:0/3327903644 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb4a80a2cb0 0x7fb4a80a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4006010 tx=0x7fb4a400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:04.218 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.217+0000 7fb4b9ffb700 1 -- 172.21.15.84:0/3327903644 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb4ac014030 con 0x7fb4bc102610 2024-05-22T23:01:04.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.439+0000 7fb4c22e6700 1 -- 172.21.15.84:0/3327903644 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fb4bc199e20 con 0x7fb4bc102610 2024-05-22T23:01:04.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.439+0000 7fb4b9ffb700 1 -- 172.21.15.84:0/3327903644 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb4ac09c300 con 0x7fb4bc102610 2024-05-22T23:01:04.440 INFO:teuthology.orchestra.run.smithi084.stdout:197568495670 2024-05-22T23:01:04.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.442+0000 7fb4c22e6700 1 -- 172.21.15.84:0/3327903644 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb4a80a2cb0 msgr2=0x7fb4a80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:04.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.442+0000 7fb4c22e6700 1 --2- 172.21.15.84:0/3327903644 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb4a80a2cb0 0x7fb4a80a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4006010 tx=0x7fb4a400b540 comp rx=0 tx=0).stop 2024-05-22T23:01:04.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.442+0000 7fb4c22e6700 1 -- 172.21.15.84:0/3327903644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4bc102610 msgr2=0x7fb4bc19e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:04.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.442+0000 7fb4c22e6700 1 --2- 172.21.15.84:0/3327903644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4bc102610 0x7fb4bc19e730 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00c9b0 tx=0x7fb4ac00ccc0 comp rx=0 tx=0).stop 2024-05-22T23:01:04.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.442+0000 7fb4c22e6700 1 -- 172.21.15.84:0/3327903644 shutdown_connections 2024-05-22T23:01:04.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.442+0000 7fb4c22e6700 1 --2- 172.21.15.84:0/3327903644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4bc1076a0 0x7fb4bc198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:04.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.442+0000 7fb4c22e6700 1 --2- 172.21.15.84:0/3327903644 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb4a80a2cb0 0x7fb4a80a5170 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:04.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.442+0000 7fb4c22e6700 1 --2- 172.21.15.84:0/3327903644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4bc102610 0x7fb4bc19e730 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:04.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.442+0000 7fb4c22e6700 1 --2- 172.21.15.84:0/3327903644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4bc102fc0 0x7fb4bc19ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:04.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.442+0000 7fb4c22e6700 1 -- 172.21.15.84:0/3327903644 >> 172.21.15.84:0/3327903644 conn(0x7fb4bc0794c0 msgr2=0x7fb4bc100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:04.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.442+0000 7fb4c22e6700 1 -- 172.21.15.84:0/3327903644 shutdown_connections 2024-05-22T23:01:04.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:04.442+0000 7fb4c22e6700 1 -- 172.21.15.84:0/3327903644 wait complete. 2024-05-22T23:01:04.863 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:04 smithi084 ceph-mon[32186]: pgmap v327: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:01:04.863 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:04 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3327903644' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-22T23:01:04.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:04 smithi204 ceph-mon[36211]: pgmap v327: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:01:04.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:04 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3327903644' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-22T23:01:05.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:04 smithi096 ceph-mon[36392]: pgmap v327: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:01:05.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:04 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3327903644' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-22T23:01:05.114 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495665 got 197568495670 for osd.6 2024-05-22T23:01:05.114 DEBUG:teuthology.parallel:result is None 2024-05-22T23:01:05.115 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-22T23:01:05.115 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-22T23:01:05.363 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:01:06.535 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T23:01:06.536 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T23:01:06.536 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:06 smithi084 ceph-mon[32186]: pgmap v328: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:01:06.536 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:01:06.536 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:01:06.536 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:06 smithi084 ceph-mon[32186]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T23:01:06.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T23:01:06.550 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T23:01:06.550 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:06 smithi096 ceph-mon[36392]: pgmap v328: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:01:06.550 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:01:06.550 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:01:06.550 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:06 smithi096 ceph-mon[36392]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T23:01:06.661 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T23:01:06.661 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T23:01:06.661 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:06 smithi204 ceph-mon[36211]: pgmap v328: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:01:06.661 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:01:06.661 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:01:06.661 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:06 smithi204 ceph-mon[36211]: from='mgr.14152 172.21.15.84:0/1670706513' entity='mgr.x' 2024-05-22T23:01:07.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.049+0000 7fe0a9019700 1 -- 172.21.15.84:0/970491545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a4101260 msgr2=0x7fe0a41016e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:07.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.049+0000 7fe0a9019700 1 --2- 172.21.15.84:0/970491545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a4101260 0x7fe0a41016e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe08c009990 tx=0x7fe08c009ca0 comp rx=0 tx=0).stop 2024-05-22T23:01:07.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.049+0000 7fe0a9019700 1 -- 172.21.15.84:0/970491545 shutdown_connections 2024-05-22T23:01:07.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.049+0000 7fe0a9019700 1 --2- 172.21.15.84:0/970491545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0a4101c20 0x7fe0a410e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:07.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.049+0000 7fe0a9019700 1 --2- 172.21.15.84:0/970491545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0a41072d0 0x7fe0a41076b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:07.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.049+0000 7fe0a9019700 1 --2- 172.21.15.84:0/970491545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a4101260 0x7fe0a41016e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:07.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.049+0000 7fe0a9019700 1 -- 172.21.15.84:0/970491545 >> 172.21.15.84:0/970491545 conn(0x7fe0a4078210 msgr2=0x7fe0a4078620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:07.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.049+0000 7fe0a9019700 1 -- 172.21.15.84:0/970491545 shutdown_connections 2024-05-22T23:01:07.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.049+0000 7fe0a9019700 1 -- 172.21.15.84:0/970491545 wait complete. 2024-05-22T23:01:07.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.050+0000 7fe0a9019700 1 Processor -- start 2024-05-22T23:01:07.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.050+0000 7fe0a9019700 1 -- start start 2024-05-22T23:01:07.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.050+0000 7fe0a9019700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a4101260 0x7fe0a419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:07.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.050+0000 7fe0a9019700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0a4101c20 0x7fe0a419b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:07.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.050+0000 7fe0a9019700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0a41072d0 0x7fe0a419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:07.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.050+0000 7fe0a9019700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a41115c0 con 0x7fe0a4101260 2024-05-22T23:01:07.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.050+0000 7fe0a9019700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a4111440 con 0x7fe0a41072d0 2024-05-22T23:01:07.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.050+0000 7fe0a9019700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a4111740 con 0x7fe0a4101c20 2024-05-22T23:01:07.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.050+0000 7fe0a2d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a4101260 0x7fe0a419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:07.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.050+0000 7fe0a2d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a4101260 0x7fe0a419ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59514/0 (socket says 172.21.15.84:59514) 2024-05-22T23:01:07.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.051+0000 7fe0a2d9d700 1 -- 172.21.15.84:0/554143839 learned_addr learned my addr 172.21.15.84:0/554143839 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:07.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.051+0000 7fe0a259c700 1 --2- 172.21.15.84:0/554143839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0a4101c20 0x7fe0a419b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:07.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.051+0000 7fe0a359e700 1 --2- 172.21.15.84:0/554143839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0a41072d0 0x7fe0a419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:07.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.051+0000 7fe0a2d9d700 1 -- 172.21.15.84:0/554143839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0a4101c20 msgr2=0x7fe0a419b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:07.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.051+0000 7fe0a2d9d700 1 --2- 172.21.15.84:0/554143839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0a4101c20 0x7fe0a419b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:07.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.051+0000 7fe0a2d9d700 1 -- 172.21.15.84:0/554143839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0a41072d0 msgr2=0x7fe0a419f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:07.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.051+0000 7fe0a2d9d700 1 --2- 172.21.15.84:0/554143839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0a41072d0 0x7fe0a419f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:07.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.051+0000 7fe0a2d9d700 1 -- 172.21.15.84:0/554143839 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a419fd20 con 0x7fe0a4101260 2024-05-22T23:01:07.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.051+0000 7fe0a2d9d700 1 --2- 172.21.15.84:0/554143839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a4101260 0x7fe0a419ad10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe09400ba50 tx=0x7fe09400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:07.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.051+0000 7fe09bfff700 1 -- 172.21.15.84:0/554143839 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe094019070 con 0x7fe0a4101260 2024-05-22T23:01:07.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.051+0000 7fe0a9019700 1 -- 172.21.15.84:0/554143839 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0a41a0010 con 0x7fe0a4101260 2024-05-22T23:01:07.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.051+0000 7fe09bfff700 1 -- 172.21.15.84:0/554143839 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe094014440 con 0x7fe0a4101260 2024-05-22T23:01:07.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.051+0000 7fe09bfff700 1 -- 172.21.15.84:0/554143839 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0940075d0 con 0x7fe0a4101260 2024-05-22T23:01:07.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.051+0000 7fe0a9019700 1 -- 172.21.15.84:0/554143839 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0a41a7890 con 0x7fe0a4101260 2024-05-22T23:01:07.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.052+0000 7fe0a9019700 1 -- 172.21.15.84:0/554143839 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0a4069050 con 0x7fe0a4101260 2024-05-22T23:01:07.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.053+0000 7fe09bfff700 1 -- 172.21.15.84:0/554143839 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe09401c070 con 0x7fe0a4101260 2024-05-22T23:01:07.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.054+0000 7fe09bfff700 1 --2- 172.21.15.84:0/554143839 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe0900a2cb0 0x7fe0900a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:07.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.054+0000 7fe0a259c700 1 --2- 172.21.15.84:0/554143839 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe0900a2cb0 0x7fe0900a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:07.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.054+0000 7fe09bfff700 1 -- 172.21.15.84:0/554143839 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fe0940ce540 con 0x7fe0a4101260 2024-05-22T23:01:07.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.055+0000 7fe0a259c700 1 --2- 172.21.15.84:0/554143839 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe0900a2cb0 0x7fe0900a5170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00b5c0 tx=0x7fe08c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:07.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.059+0000 7fe09bfff700 1 -- 172.21.15.84:0/554143839 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe09409b430 con 0x7fe0a4101260 2024-05-22T23:01:07.251 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.251+0000 7fe0a9019700 1 -- 172.21.15.84:0/554143839 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe0a406a750 con 0x7fe0900a2cb0 2024-05-22T23:01:07.258 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.258+0000 7fe09bfff700 1 -- 172.21.15.84:0/554143839 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258946 (secure 0 0 0) 0x7fe0a406a750 con 0x7fe0900a2cb0 2024-05-22T23:01:07.262 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:07.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.261+0000 7fe0a9019700 1 -- 172.21.15.84:0/554143839 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe0900a2cb0 msgr2=0x7fe0900a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:07.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.261+0000 7fe0a9019700 1 --2- 172.21.15.84:0/554143839 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe0900a2cb0 0x7fe0900a5170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00b5c0 tx=0x7fe08c005960 comp rx=0 tx=0).stop 2024-05-22T23:01:07.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.261+0000 7fe0a9019700 1 -- 172.21.15.84:0/554143839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a4101260 msgr2=0x7fe0a419ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:07.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.261+0000 7fe0a9019700 1 --2- 172.21.15.84:0/554143839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a4101260 0x7fe0a419ad10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe09400ba50 tx=0x7fe09400bd60 comp rx=0 tx=0).stop 2024-05-22T23:01:07.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.261+0000 7fe0a9019700 1 -- 172.21.15.84:0/554143839 shutdown_connections 2024-05-22T23:01:07.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.261+0000 7fe0a9019700 1 --2- 172.21.15.84:0/554143839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0a41072d0 0x7fe0a419f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:07.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.261+0000 7fe0a9019700 1 --2- 172.21.15.84:0/554143839 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe0900a2cb0 0x7fe0900a5170 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:07.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.261+0000 7fe0a9019700 1 --2- 172.21.15.84:0/554143839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a4101260 0x7fe0a419ad10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:07.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.261+0000 7fe0a9019700 1 --2- 172.21.15.84:0/554143839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0a4101c20 0x7fe0a419b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:07.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.261+0000 7fe0a9019700 1 -- 172.21.15.84:0/554143839 >> 172.21.15.84:0/554143839 conn(0x7fe0a4078210 msgr2=0x7fe0a406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:07.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.261+0000 7fe0a9019700 1 -- 172.21.15.84:0/554143839 shutdown_connections 2024-05-22T23:01:07.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:07.261+0000 7fe0a9019700 1 -- 172.21.15.84:0/554143839 wait complete. 2024-05-22T23:01:07.265 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2024-05-22T23:01:07.923 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-05-22T23:01:05.511735+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650311,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":270,"num_read_kb":246,"num_write":255,"num_write_kb":3019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1239647,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377380,"kb_used_data":11516,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564444,"statfs":{"total":1151940427776,"available":1151553990656,"internally_reserved":0,"allocated":11792384,"data_stored":7484889,"data_compressed":189361,"data_compressed_allocated":253952,"data_compressed_original":2039808,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002753"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.577659+0000","last_change":"2024-05-22T22:59:51.951625+0000","last_active":"2024-05-22T23:00:46.577659+0000","last_peered":"2024-05-22T23:00:46.577659+0000","last_clean":"2024-05-22T23:00:46.577659+0000","last_became_active":"2024-05-22T22:59:51.951225+0000","last_became_peered":"2024-05-22T22:59:51.951225+0000","last_unstale":"2024-05-22T23:00:46.577659+0000","last_undegraded":"2024-05-22T23:00:46.577659+0000","last_fullsized":"2024-05-22T23:00:46.577659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:53:32.616083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580762+0000","last_change":"2024-05-22T22:59:51.956925+0000","last_active":"2024-05-22T23:00:46.580762+0000","last_peered":"2024-05-22T23:00:46.580762+0000","last_clean":"2024-05-22T23:00:46.580762+0000","last_became_active":"2024-05-22T22:59:51.956557+0000","last_became_peered":"2024-05-22T22:59:51.956557+0000","last_unstale":"2024-05-22T23:00:46.580762+0000","last_undegraded":"2024-05-22T23:00:46.580762+0000","last_fullsized":"2024-05-22T23:00:46.580762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:18:14.609682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580836+0000","last_change":"2024-05-22T22:59:51.951981+0000","last_active":"2024-05-22T23:00:46.580836+0000","last_peered":"2024-05-22T23:00:46.580836+0000","last_clean":"2024-05-22T23:00:46.580836+0000","last_became_active":"2024-05-22T22:59:51.951522+0000","last_became_peered":"2024-05-22T22:59:51.951522+0000","last_unstale":"2024-05-22T23:00:46.580836+0000","last_undegraded":"2024-05-22T23:00:46.580836+0000","last_fullsized":"2024-05-22T23:00:46.580836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:56:23.357290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.873333+0000","last_change":"2024-05-22T22:59:51.966141+0000","last_active":"2024-05-22T23:00:46.873333+0000","last_peered":"2024-05-22T23:00:46.873333+0000","last_clean":"2024-05-22T23:00:46.873333+0000","last_became_active":"2024-05-22T22:59:51.965798+0000","last_became_peered":"2024-05-22T22:59:51.965798+0000","last_unstale":"2024-05-22T23:00:46.873333+0000","last_undegraded":"2024-05-22T23:00:46.873333+0000","last_fullsized":"2024-05-22T23:00:46.873333+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:23:58.282078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578081+0000","last_change":"2024-05-22T22:59:51.950425+0000","last_active":"2024-05-22T23:00:46.578081+0000","last_peered":"2024-05-22T23:00:46.578081+0000","last_clean":"2024-05-22T23:00:46.578081+0000","last_became_active":"2024-05-22T22:59:51.950071+0000","last_became_peered":"2024-05-22T22:59:51.950071+0000","last_unstale":"2024-05-22T23:00:46.578081+0000","last_undegraded":"2024-05-22T23:00:46.578081+0000","last_fullsized":"2024-05-22T23:00:46.578081+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:42:43.913816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.582890+0000","last_change":"2024-05-22T23:00:46.582890+0000","last_active":"2024-05-22T23:00:46.582890+0000","last_peered":"2024-05-22T23:00:46.582890+0000","last_clean":"2024-05-22T23:00:46.582890+0000","last_became_active":"2024-05-22T23:00:46.582497+0000","last_became_peered":"2024-05-22T23:00:46.582497+0000","last_unstale":"2024-05-22T23:00:46.582890+0000","last_undegraded":"2024-05-22T23:00:46.582890+0000","last_fullsized":"2024-05-22T23:00:46.582890+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:37:39.420650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580150+0000","last_change":"2024-05-22T22:59:51.959583+0000","last_active":"2024-05-22T23:00:46.580150+0000","last_peered":"2024-05-22T23:00:46.580150+0000","last_clean":"2024-05-22T23:00:46.580150+0000","last_became_active":"2024-05-22T22:59:51.959260+0000","last_became_peered":"2024-05-22T22:59:51.959260+0000","last_unstale":"2024-05-22T23:00:46.580150+0000","last_undegraded":"2024-05-22T23:00:46.580150+0000","last_fullsized":"2024-05-22T23:00:46.580150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:55:48.960861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.585335+0000","last_change":"2024-05-22T23:00:46.584888+0000","last_active":"2024-05-22T23:00:46.585335+0000","last_peered":"2024-05-22T23:00:46.585335+0000","last_clean":"2024-05-22T23:00:46.585335+0000","last_became_active":"2024-05-22T23:00:46.584534+0000","last_became_peered":"2024-05-22T23:00:46.584534+0000","last_unstale":"2024-05-22T23:00:46.585335+0000","last_undegraded":"2024-05-22T23:00:46.585335+0000","last_fullsized":"2024-05-22T23:00:46.585335+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:01:14.551458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.579732+0000","last_change":"2024-05-22T22:59:51.956954+0000","last_active":"2024-05-22T23:00:46.579732+0000","last_peered":"2024-05-22T23:00:46.579732+0000","last_clean":"2024-05-22T23:00:46.579732+0000","last_became_active":"2024-05-22T22:59:51.956570+0000","last_became_peered":"2024-05-22T22:59:51.956570+0000","last_unstale":"2024-05-22T23:00:46.579732+0000","last_undegraded":"2024-05-22T23:00:46.579732+0000","last_fullsized":"2024-05-22T23:00:46.579732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:55:18.828023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.577561+0000","last_change":"2024-05-22T22:59:51.962371+0000","last_active":"2024-05-22T23:00:46.577561+0000","last_peered":"2024-05-22T23:00:46.577561+0000","last_clean":"2024-05-22T23:00:46.577561+0000","last_became_active":"2024-05-22T22:59:51.962036+0000","last_became_peered":"2024-05-22T22:59:51.962036+0000","last_unstale":"2024-05-22T23:00:46.577561+0000","last_undegraded":"2024-05-22T23:00:46.577561+0000","last_fullsized":"2024-05-22T23:00:46.577561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:27:21.745563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580634+0000","last_change":"2024-05-22T22:59:51.952446+0000","last_active":"2024-05-22T23:00:46.580634+0000","last_peered":"2024-05-22T23:00:46.580634+0000","last_clean":"2024-05-22T23:00:46.580634+0000","last_became_active":"2024-05-22T22:59:51.952115+0000","last_became_peered":"2024-05-22T22:59:51.952115+0000","last_unstale":"2024-05-22T23:00:46.580634+0000","last_undegraded":"2024-05-22T23:00:46.580634+0000","last_fullsized":"2024-05-22T23:00:46.580634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:33:56.960951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.873308+0000","last_change":"2024-05-22T22:59:51.968457+0000","last_active":"2024-05-22T23:00:46.873308+0000","last_peered":"2024-05-22T23:00:46.873308+0000","last_clean":"2024-05-22T23:00:46.873308+0000","last_became_active":"2024-05-22T22:59:51.968070+0000","last_became_peered":"2024-05-22T22:59:51.968070+0000","last_unstale":"2024-05-22T23:00:46.873308+0000","last_undegraded":"2024-05-22T23:00:46.873308+0000","last_fullsized":"2024-05-22T23:00:46.873308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:07:31.595119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578528+0000","last_change":"2024-05-22T22:59:51.957357+0000","last_active":"2024-05-22T23:00:46.578528+0000","last_peered":"2024-05-22T23:00:46.578528+0000","last_clean":"2024-05-22T23:00:46.578528+0000","last_became_active":"2024-05-22T22:59:51.957013+0000","last_became_peered":"2024-05-22T22:59:51.957013+0000","last_unstale":"2024-05-22T23:00:46.578528+0000","last_undegraded":"2024-05-22T23:00:46.578528+0000","last_fullsized":"2024-05-22T23:00:46.578528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:38:48.118178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.581006+0000","last_change":"2024-05-22T22:59:51.957084+0000","last_active":"2024-05-22T23:00:46.581006+0000","last_peered":"2024-05-22T23:00:46.581006+0000","last_clean":"2024-05-22T23:00:46.581006+0000","last_became_active":"2024-05-22T22:59:51.956787+0000","last_became_peered":"2024-05-22T22:59:51.956787+0000","last_unstale":"2024-05-22T23:00:46.581006+0000","last_undegraded":"2024-05-22T23:00:46.581006+0000","last_fullsized":"2024-05-22T23:00:46.581006+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:30:40.578286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.579773+0000","last_change":"2024-05-22T22:59:51.956375+0000","last_active":"2024-05-22T23:00:46.579773+0000","last_peered":"2024-05-22T23:00:46.579773+0000","last_clean":"2024-05-22T23:00:46.579773+0000","last_became_active":"2024-05-22T22:59:51.956012+0000","last_became_peered":"2024-05-22T22:59:51.956012+0000","last_unstale":"2024-05-22T23:00:46.579773+0000","last_undegraded":"2024-05-22T23:00:46.579773+0000","last_fullsized":"2024-05-22T23:00:46.579773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:33:49.920327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.579651+0000","last_change":"2024-05-22T22:59:51.950287+0000","last_active":"2024-05-22T23:00:46.579651+0000","last_peered":"2024-05-22T23:00:46.579651+0000","last_clean":"2024-05-22T23:00:46.579651+0000","last_became_active":"2024-05-22T22:59:51.949962+0000","last_became_peered":"2024-05-22T22:59:51.949962+0000","last_unstale":"2024-05-22T23:00:46.579651+0000","last_undegraded":"2024-05-22T23:00:46.579651+0000","last_fullsized":"2024-05-22T23:00:46.579651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:33:20.674713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.577803+0000","last_change":"2024-05-22T22:59:51.952523+0000","last_active":"2024-05-22T23:00:46.577803+0000","last_peered":"2024-05-22T23:00:46.577803+0000","last_clean":"2024-05-22T23:00:46.577803+0000","last_became_active":"2024-05-22T22:59:51.952152+0000","last_became_peered":"2024-05-22T22:59:51.952152+0000","last_unstale":"2024-05-22T23:00:46.577803+0000","last_undegraded":"2024-05-22T23:00:46.577803+0000","last_fullsized":"2024-05-22T23:00:46.577803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:33:01.332667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.576185+0000","last_change":"2024-05-22T22:59:51.960919+0000","last_active":"2024-05-22T23:00:46.576185+0000","last_peered":"2024-05-22T23:00:46.576185+0000","last_clean":"2024-05-22T23:00:46.576185+0000","last_became_active":"2024-05-22T22:59:51.960603+0000","last_became_peered":"2024-05-22T22:59:51.960603+0000","last_unstale":"2024-05-22T23:00:46.576185+0000","last_undegraded":"2024-05-22T23:00:46.576185+0000","last_fullsized":"2024-05-22T23:00:46.576185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:28:05.977612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.581038+0000","last_change":"2024-05-22T22:59:51.949186+0000","last_active":"2024-05-22T23:00:46.581038+0000","last_peered":"2024-05-22T23:00:46.581038+0000","last_clean":"2024-05-22T23:00:46.581038+0000","last_became_active":"2024-05-22T22:59:51.948819+0000","last_became_peered":"2024-05-22T22:59:51.948819+0000","last_unstale":"2024-05-22T23:00:46.581038+0000","last_undegraded":"2024-05-22T23:00:46.581038+0000","last_fullsized":"2024-05-22T23:00:46.581038+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:32:16.153838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.576061+0000","last_change":"2024-05-22T22:59:51.592520+0000","last_active":"2024-05-22T23:00:46.576061+0000","last_peered":"2024-05-22T23:00:46.576061+0000","last_clean":"2024-05-22T23:00:46.576061+0000","last_became_active":"2024-05-22T22:59:51.592201+0000","last_became_peered":"2024-05-22T22:59:51.592201+0000","last_unstale":"2024-05-22T23:00:46.576061+0000","last_undegraded":"2024-05-22T23:00:46.576061+0000","last_fullsized":"2024-05-22T23:00:46.576061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:45:06.500085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.585804+0000","last_change":"2024-05-22T23:00:46.585147+0000","last_active":"2024-05-22T23:00:46.585804+0000","last_peered":"2024-05-22T23:00:46.585804+0000","last_clean":"2024-05-22T23:00:46.585804+0000","last_became_active":"2024-05-22T23:00:46.584828+0000","last_became_peered":"2024-05-22T23:00:46.584828+0000","last_unstale":"2024-05-22T23:00:46.585804+0000","last_undegraded":"2024-05-22T23:00:46.585804+0000","last_fullsized":"2024-05-22T23:00:46.585804+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:28:29.641493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.872962+0000","last_change":"2024-05-22T22:59:51.595383+0000","last_active":"2024-05-22T23:00:46.872962+0000","last_peered":"2024-05-22T23:00:46.872962+0000","last_clean":"2024-05-22T23:00:46.872962+0000","last_became_active":"2024-05-22T22:59:51.595076+0000","last_became_peered":"2024-05-22T22:59:51.595076+0000","last_unstale":"2024-05-22T23:00:46.872962+0000","last_undegraded":"2024-05-22T23:00:46.872962+0000","last_fullsized":"2024-05-22T23:00:46.872962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:29:57.205196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580990+0000","last_change":"2024-05-22T22:59:51.952447+0000","last_active":"2024-05-22T23:00:46.580990+0000","last_peered":"2024-05-22T23:00:46.580990+0000","last_clean":"2024-05-22T23:00:46.580990+0000","last_became_active":"2024-05-22T22:59:51.951927+0000","last_became_peered":"2024-05-22T22:59:51.951927+0000","last_unstale":"2024-05-22T23:00:46.580990+0000","last_undegraded":"2024-05-22T23:00:46.580990+0000","last_fullsized":"2024-05-22T23:00:46.580990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:14:59.161404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.579689+0000","last_change":"2024-05-22T22:59:51.597705+0000","last_active":"2024-05-22T23:00:46.579689+0000","last_peered":"2024-05-22T23:00:46.579689+0000","last_clean":"2024-05-22T23:00:46.579689+0000","last_became_active":"2024-05-22T22:59:51.597319+0000","last_became_peered":"2024-05-22T22:59:51.597319+0000","last_unstale":"2024-05-22T23:00:46.579689+0000","last_undegraded":"2024-05-22T23:00:46.579689+0000","last_fullsized":"2024-05-22T23:00:46.579689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:59:59.307499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.579842+0000","last_change":"2024-05-22T22:59:51.955477+0000","last_active":"2024-05-22T23:00:46.579842+0000","last_peered":"2024-05-22T23:00:46.579842+0000","last_clean":"2024-05-22T23:00:46.579842+0000","last_became_active":"2024-05-22T22:59:51.953608+0000","last_became_peered":"2024-05-22T22:59:51.953608+0000","last_unstale":"2024-05-22T23:00:46.579842+0000","last_undegraded":"2024-05-22T23:00:46.579842+0000","last_fullsized":"2024-05-22T23:00:46.579842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:27:35.192556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578795+0000","last_change":"2024-05-22T22:59:51.595417+0000","last_active":"2024-05-22T23:00:46.578795+0000","last_peered":"2024-05-22T23:00:46.578795+0000","last_clean":"2024-05-22T23:00:46.578795+0000","last_became_active":"2024-05-22T22:59:51.595051+0000","last_became_peered":"2024-05-22T22:59:51.595051+0000","last_unstale":"2024-05-22T23:00:46.578795+0000","last_undegraded":"2024-05-22T23:00:46.578795+0000","last_fullsized":"2024-05-22T23:00:46.578795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:13:47.936348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.582621+0000","last_change":"2024-05-22T22:59:51.958471+0000","last_active":"2024-05-22T23:00:46.582621+0000","last_peered":"2024-05-22T23:00:46.582621+0000","last_clean":"2024-05-22T23:00:46.582621+0000","last_became_active":"2024-05-22T22:59:51.958017+0000","last_became_peered":"2024-05-22T22:59:51.958017+0000","last_unstale":"2024-05-22T23:00:46.582621+0000","last_undegraded":"2024-05-22T23:00:46.582621+0000","last_fullsized":"2024-05-22T23:00:46.582621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:42:52.196892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.582468+0000","last_change":"2024-05-22T22:59:51.591909+0000","last_active":"2024-05-22T23:00:46.582468+0000","last_peered":"2024-05-22T23:00:46.582468+0000","last_clean":"2024-05-22T23:00:46.582468+0000","last_became_active":"2024-05-22T22:59:51.591531+0000","last_became_peered":"2024-05-22T22:59:51.591531+0000","last_unstale":"2024-05-22T23:00:46.582468+0000","last_undegraded":"2024-05-22T23:00:46.582468+0000","last_fullsized":"2024-05-22T23:00:46.582468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:07:22.997156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.581083+0000","last_change":"2024-05-22T23:00:46.581083+0000","last_active":"2024-05-22T23:00:46.581083+0000","last_peered":"2024-05-22T23:00:46.581083+0000","last_clean":"2024-05-22T23:00:46.581083+0000","last_became_active":"2024-05-22T23:00:46.580458+0000","last_became_peered":"2024-05-22T23:00:46.580458+0000","last_unstale":"2024-05-22T23:00:46.581083+0000","last_undegraded":"2024-05-22T23:00:46.581083+0000","last_fullsized":"2024-05-22T23:00:46.581083+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:38:53.136828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.576874+0000","last_change":"2024-05-22T22:59:51.590124+0000","last_active":"2024-05-22T23:00:46.576874+0000","last_peered":"2024-05-22T23:00:46.576874+0000","last_clean":"2024-05-22T23:00:46.576874+0000","last_became_active":"2024-05-22T22:59:51.589661+0000","last_became_peered":"2024-05-22T22:59:51.589661+0000","last_unstale":"2024-05-22T23:00:46.576874+0000","last_undegraded":"2024-05-22T23:00:46.576874+0000","last_fullsized":"2024-05-22T23:00:46.576874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:33:14.293487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.873993+0000","last_change":"2024-05-22T23:00:46.873993+0000","last_active":"2024-05-22T23:00:46.873993+0000","last_peered":"2024-05-22T23:00:46.873993+0000","last_clean":"2024-05-22T23:00:46.873993+0000","last_became_active":"2024-05-22T23:00:46.873523+0000","last_became_peered":"2024-05-22T23:00:46.873523+0000","last_unstale":"2024-05-22T23:00:46.873993+0000","last_undegraded":"2024-05-22T23:00:46.873993+0000","last_fullsized":"2024-05-22T23:00:46.873993+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:39:20.586352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578330+0000","last_change":"2024-05-22T22:59:51.595510+0000","last_active":"2024-05-22T23:00:46.578330+0000","last_peered":"2024-05-22T23:00:46.578330+0000","last_clean":"2024-05-22T23:00:46.578330+0000","last_became_active":"2024-05-22T22:59:51.595208+0000","last_became_peered":"2024-05-22T22:59:51.595208+0000","last_unstale":"2024-05-22T23:00:46.578330+0000","last_undegraded":"2024-05-22T23:00:46.578330+0000","last_fullsized":"2024-05-22T23:00:46.578330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:47:46.957965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.872063+0000","last_change":"2024-05-22T22:59:51.950684+0000","last_active":"2024-05-22T23:00:46.872063+0000","last_peered":"2024-05-22T23:00:46.872063+0000","last_clean":"2024-05-22T23:00:46.872063+0000","last_became_active":"2024-05-22T22:59:51.950380+0000","last_became_peered":"2024-05-22T22:59:51.950380+0000","last_unstale":"2024-05-22T23:00:46.872063+0000","last_undegraded":"2024-05-22T23:00:46.872063+0000","last_fullsized":"2024-05-22T23:00:46.872063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:58:02.035245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.582592+0000","last_change":"2024-05-22T22:59:51.592958+0000","last_active":"2024-05-22T23:00:46.582592+0000","last_peered":"2024-05-22T23:00:46.582592+0000","last_clean":"2024-05-22T23:00:46.582592+0000","last_became_active":"2024-05-22T22:59:51.592586+0000","last_became_peered":"2024-05-22T22:59:51.592586+0000","last_unstale":"2024-05-22T23:00:46.582592+0000","last_undegraded":"2024-05-22T23:00:46.582592+0000","last_fullsized":"2024-05-22T23:00:46.582592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:44:43.859724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.575722+0000","last_change":"2024-05-22T22:59:51.960863+0000","last_active":"2024-05-22T23:00:46.575722+0000","last_peered":"2024-05-22T23:00:46.575722+0000","last_clean":"2024-05-22T23:00:46.575722+0000","last_became_active":"2024-05-22T22:59:51.960537+0000","last_became_peered":"2024-05-22T22:59:51.960537+0000","last_unstale":"2024-05-22T23:00:46.575722+0000","last_undegraded":"2024-05-22T23:00:46.575722+0000","last_fullsized":"2024-05-22T23:00:46.575722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:08:19.735397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.577542+0000","last_change":"2024-05-22T22:59:51.596150+0000","last_active":"2024-05-22T23:00:46.577542+0000","last_peered":"2024-05-22T23:00:46.577542+0000","last_clean":"2024-05-22T23:00:46.577542+0000","last_became_active":"2024-05-22T22:59:51.595791+0000","last_became_peered":"2024-05-22T22:59:51.595791+0000","last_unstale":"2024-05-22T23:00:46.577542+0000","last_undegraded":"2024-05-22T23:00:46.577542+0000","last_fullsized":"2024-05-22T23:00:46.577542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:47:46.144045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.579638+0000","last_change":"2024-05-22T22:59:51.951907+0000","last_active":"2024-05-22T23:00:46.579638+0000","last_peered":"2024-05-22T23:00:46.579638+0000","last_clean":"2024-05-22T23:00:46.579638+0000","last_became_active":"2024-05-22T22:59:51.951264+0000","last_became_peered":"2024-05-22T22:59:51.951264+0000","last_unstale":"2024-05-22T23:00:46.579638+0000","last_undegraded":"2024-05-22T23:00:46.579638+0000","last_fullsized":"2024-05-22T23:00:46.579638+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:26:44.024848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580516+0000","last_change":"2024-05-22T22:59:51.594883+0000","last_active":"2024-05-22T23:00:46.580516+0000","last_peered":"2024-05-22T23:00:46.580516+0000","last_clean":"2024-05-22T23:00:46.580516+0000","last_became_active":"2024-05-22T22:59:51.594458+0000","last_became_peered":"2024-05-22T22:59:51.594458+0000","last_unstale":"2024-05-22T23:00:46.580516+0000","last_undegraded":"2024-05-22T23:00:46.580516+0000","last_fullsized":"2024-05-22T23:00:46.580516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:39:27.366961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580943+0000","last_change":"2024-05-22T22:59:51.958180+0000","last_active":"2024-05-22T23:00:46.580943+0000","last_peered":"2024-05-22T23:00:46.580943+0000","last_clean":"2024-05-22T23:00:46.580943+0000","last_became_active":"2024-05-22T22:59:51.957877+0000","last_became_peered":"2024-05-22T22:59:51.957877+0000","last_unstale":"2024-05-22T23:00:46.580943+0000","last_undegraded":"2024-05-22T23:00:46.580943+0000","last_fullsized":"2024-05-22T23:00:46.580943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:33:42.478340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.583107+0000","last_change":"2024-05-22T22:59:51.592953+0000","last_active":"2024-05-22T23:00:46.583107+0000","last_peered":"2024-05-22T23:00:46.583107+0000","last_clean":"2024-05-22T23:00:46.583107+0000","last_became_active":"2024-05-22T22:59:51.592525+0000","last_became_peered":"2024-05-22T22:59:51.592525+0000","last_unstale":"2024-05-22T23:00:46.583107+0000","last_undegraded":"2024-05-22T23:00:46.583107+0000","last_fullsized":"2024-05-22T23:00:46.583107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:18:03.512123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.579957+0000","last_change":"2024-05-22T22:59:51.959510+0000","last_active":"2024-05-22T23:00:46.579957+0000","last_peered":"2024-05-22T23:00:46.579957+0000","last_clean":"2024-05-22T23:00:46.579957+0000","last_became_active":"2024-05-22T22:59:51.959197+0000","last_became_peered":"2024-05-22T22:59:51.959197+0000","last_unstale":"2024-05-22T23:00:46.579957+0000","last_undegraded":"2024-05-22T23:00:46.579957+0000","last_fullsized":"2024-05-22T23:00:46.579957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:33:22.616668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'23","reported_seq":198,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:01:01.897235+0000","last_change":"2024-05-22T23:00:46.882566+0000","last_active":"2024-05-22T23:01:01.897235+0000","last_peered":"2024-05-22T23:01:01.897235+0000","last_clean":"2024-05-22T23:01:01.897235+0000","last_became_active":"2024-05-22T23:00:46.874658+0000","last_became_peered":"2024-05-22T23:00:46.874658+0000","last_unstale":"2024-05-22T23:01:01.897235+0000","last_undegraded":"2024-05-22T23:01:01.897235+0000","last_fullsized":"2024-05-22T23:01:01.897235+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:20:09.011734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58911,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":128,"num_read_kb":121,"num_write":23,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":58911,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.579306+0000","last_change":"2024-05-22T22:59:51.952486+0000","last_active":"2024-05-22T23:00:46.579306+0000","last_peered":"2024-05-22T23:00:46.579306+0000","last_clean":"2024-05-22T23:00:46.579306+0000","last_became_active":"2024-05-22T22:59:51.952096+0000","last_became_peered":"2024-05-22T22:59:51.952096+0000","last_unstale":"2024-05-22T23:00:46.579306+0000","last_undegraded":"2024-05-22T23:00:46.579306+0000","last_fullsized":"2024-05-22T23:00:46.579306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:52:33.877655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580801+0000","last_change":"2024-05-22T22:59:51.595446+0000","last_active":"2024-05-22T23:00:46.580801+0000","last_peered":"2024-05-22T23:00:46.580801+0000","last_clean":"2024-05-22T23:00:46.580801+0000","last_became_active":"2024-05-22T22:59:51.595097+0000","last_became_peered":"2024-05-22T22:59:51.595097+0000","last_unstale":"2024-05-22T23:00:46.580801+0000","last_undegraded":"2024-05-22T23:00:46.580801+0000","last_fullsized":"2024-05-22T23:00:46.580801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:03:29.722773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578434+0000","last_change":"2024-05-22T22:59:51.953304+0000","last_active":"2024-05-22T23:00:46.578434+0000","last_peered":"2024-05-22T23:00:46.578434+0000","last_clean":"2024-05-22T23:00:46.578434+0000","last_became_active":"2024-05-22T22:59:51.952974+0000","last_became_peered":"2024-05-22T22:59:51.952974+0000","last_unstale":"2024-05-22T23:00:46.578434+0000","last_undegraded":"2024-05-22T23:00:46.578434+0000","last_fullsized":"2024-05-22T23:00:46.578434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:21:06.116285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.582543+0000","last_change":"2024-05-22T22:59:51.593684+0000","last_active":"2024-05-22T23:00:46.582543+0000","last_peered":"2024-05-22T23:00:46.582543+0000","last_clean":"2024-05-22T23:00:46.582543+0000","last_became_active":"2024-05-22T22:59:51.593335+0000","last_became_peered":"2024-05-22T22:59:51.593335+0000","last_unstale":"2024-05-22T23:00:46.582543+0000","last_undegraded":"2024-05-22T23:00:46.582543+0000","last_fullsized":"2024-05-22T23:00:46.582543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:36:24.405609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580654+0000","last_change":"2024-05-22T22:59:51.960930+0000","last_active":"2024-05-22T23:00:46.580654+0000","last_peered":"2024-05-22T23:00:46.580654+0000","last_clean":"2024-05-22T23:00:46.580654+0000","last_became_active":"2024-05-22T22:59:51.960571+0000","last_became_peered":"2024-05-22T22:59:51.960571+0000","last_unstale":"2024-05-22T23:00:46.580654+0000","last_undegraded":"2024-05-22T23:00:46.580654+0000","last_fullsized":"2024-05-22T23:00:46.580654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:50:31.472449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.872580+0000","last_change":"2024-05-22T22:59:51.592876+0000","last_active":"2024-05-22T23:00:46.872580+0000","last_peered":"2024-05-22T23:00:46.872580+0000","last_clean":"2024-05-22T23:00:46.872580+0000","last_became_active":"2024-05-22T22:59:51.592488+0000","last_became_peered":"2024-05-22T22:59:51.592488+0000","last_unstale":"2024-05-22T23:00:46.872580+0000","last_undegraded":"2024-05-22T23:00:46.872580+0000","last_fullsized":"2024-05-22T23:00:46.872580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:37:16.517302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.579973+0000","last_change":"2024-05-22T22:59:51.962336+0000","last_active":"2024-05-22T23:00:46.579973+0000","last_peered":"2024-05-22T23:00:46.579973+0000","last_clean":"2024-05-22T23:00:46.579973+0000","last_became_active":"2024-05-22T22:59:51.962002+0000","last_became_peered":"2024-05-22T22:59:51.962002+0000","last_unstale":"2024-05-22T23:00:46.579973+0000","last_undegraded":"2024-05-22T23:00:46.579973+0000","last_fullsized":"2024-05-22T23:00:46.579973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:23:19.409418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.592248+0000","last_change":"2024-05-22T23:00:46.591734+0000","last_active":"2024-05-22T23:00:46.592248+0000","last_peered":"2024-05-22T23:00:46.592248+0000","last_clean":"2024-05-22T23:00:46.592248+0000","last_became_active":"2024-05-22T23:00:46.584075+0000","last_became_peered":"2024-05-22T23:00:46.584075+0000","last_unstale":"2024-05-22T23:00:46.592248+0000","last_undegraded":"2024-05-22T23:00:46.592248+0000","last_fullsized":"2024-05-22T23:00:46.592248+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:32:13.548197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.582162+0000","last_change":"2024-05-22T22:59:51.961600+0000","last_active":"2024-05-22T23:00:46.582162+0000","last_peered":"2024-05-22T23:00:46.582162+0000","last_clean":"2024-05-22T23:00:46.582162+0000","last_became_active":"2024-05-22T22:59:51.961286+0000","last_became_peered":"2024-05-22T22:59:51.961286+0000","last_unstale":"2024-05-22T23:00:46.582162+0000","last_undegraded":"2024-05-22T23:00:46.582162+0000","last_fullsized":"2024-05-22T23:00:46.582162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:04:26.992865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578974+0000","last_change":"2024-05-22T22:59:51.594461+0000","last_active":"2024-05-22T23:00:46.578974+0000","last_peered":"2024-05-22T23:00:46.578974+0000","last_clean":"2024-05-22T23:00:46.578974+0000","last_became_active":"2024-05-22T22:59:51.594128+0000","last_became_peered":"2024-05-22T22:59:51.594128+0000","last_unstale":"2024-05-22T23:00:46.578974+0000","last_undegraded":"2024-05-22T23:00:46.578974+0000","last_fullsized":"2024-05-22T23:00:46.578974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:52:05.095814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.577429+0000","last_change":"2024-05-22T22:59:51.958851+0000","last_active":"2024-05-22T23:00:46.577429+0000","last_peered":"2024-05-22T23:00:46.577429+0000","last_clean":"2024-05-22T23:00:46.577429+0000","last_became_active":"2024-05-22T22:59:51.957412+0000","last_became_peered":"2024-05-22T22:59:51.957412+0000","last_unstale":"2024-05-22T23:00:46.577429+0000","last_undegraded":"2024-05-22T23:00:46.577429+0000","last_fullsized":"2024-05-22T23:00:46.577429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:13:02.761205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.575778+0000","last_change":"2024-05-22T22:59:51.588750+0000","last_active":"2024-05-22T23:00:46.575778+0000","last_peered":"2024-05-22T23:00:46.575778+0000","last_clean":"2024-05-22T23:00:46.575778+0000","last_became_active":"2024-05-22T22:59:51.588364+0000","last_became_peered":"2024-05-22T22:59:51.588364+0000","last_unstale":"2024-05-22T23:00:46.575778+0000","last_undegraded":"2024-05-22T23:00:46.575778+0000","last_fullsized":"2024-05-22T23:00:46.575778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:28:41.605406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.576921+0000","last_change":"2024-05-22T22:59:51.964676+0000","last_active":"2024-05-22T23:00:46.576921+0000","last_peered":"2024-05-22T23:00:46.576921+0000","last_clean":"2024-05-22T23:00:46.576921+0000","last_became_active":"2024-05-22T22:59:51.960472+0000","last_became_peered":"2024-05-22T22:59:51.960472+0000","last_unstale":"2024-05-22T23:00:46.576921+0000","last_undegraded":"2024-05-22T23:00:46.576921+0000","last_fullsized":"2024-05-22T23:00:46.576921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:24:55.042637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580200+0000","last_change":"2024-05-22T22:59:51.600873+0000","last_active":"2024-05-22T23:00:46.580200+0000","last_peered":"2024-05-22T23:00:46.580200+0000","last_clean":"2024-05-22T23:00:46.580200+0000","last_became_active":"2024-05-22T22:59:51.599785+0000","last_became_peered":"2024-05-22T22:59:51.599785+0000","last_unstale":"2024-05-22T23:00:46.580200+0000","last_undegraded":"2024-05-22T23:00:46.580200+0000","last_fullsized":"2024-05-22T23:00:46.580200+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:20:09.590765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.577178+0000","last_change":"2024-05-22T22:59:51.964942+0000","last_active":"2024-05-22T23:00:46.577178+0000","last_peered":"2024-05-22T23:00:46.577178+0000","last_clean":"2024-05-22T23:00:46.577178+0000","last_became_active":"2024-05-22T22:59:51.964333+0000","last_became_peered":"2024-05-22T22:59:51.964333+0000","last_unstale":"2024-05-22T23:00:46.577178+0000","last_undegraded":"2024-05-22T23:00:46.577178+0000","last_fullsized":"2024-05-22T23:00:46.577178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:01:07.215075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.582908+0000","last_change":"2024-05-22T22:59:51.957291+0000","last_active":"2024-05-22T23:00:46.582908+0000","last_peered":"2024-05-22T23:00:46.582908+0000","last_clean":"2024-05-22T23:00:46.582908+0000","last_became_active":"2024-05-22T22:59:51.956987+0000","last_became_peered":"2024-05-22T22:59:51.956987+0000","last_unstale":"2024-05-22T23:00:46.582908+0000","last_undegraded":"2024-05-22T23:00:46.582908+0000","last_fullsized":"2024-05-22T23:00:46.582908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:14:10.420756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578254+0000","last_change":"2024-05-22T22:59:51.590255+0000","last_active":"2024-05-22T23:00:46.578254+0000","last_peered":"2024-05-22T23:00:46.578254+0000","last_clean":"2024-05-22T23:00:46.578254+0000","last_became_active":"2024-05-22T22:59:51.589261+0000","last_became_peered":"2024-05-22T22:59:51.589261+0000","last_unstale":"2024-05-22T23:00:46.578254+0000","last_undegraded":"2024-05-22T23:00:46.578254+0000","last_fullsized":"2024-05-22T23:00:46.578254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:15:14.171311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.873140+0000","last_change":"2024-05-22T22:59:51.961453+0000","last_active":"2024-05-22T23:00:46.873140+0000","last_peered":"2024-05-22T23:00:46.873140+0000","last_clean":"2024-05-22T23:00:46.873140+0000","last_became_active":"2024-05-22T22:59:51.961082+0000","last_became_peered":"2024-05-22T22:59:51.961082+0000","last_unstale":"2024-05-22T23:00:46.873140+0000","last_undegraded":"2024-05-22T23:00:46.873140+0000","last_fullsized":"2024-05-22T23:00:46.873140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:05:50.310688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.872913+0000","last_change":"2024-05-22T22:59:51.595913+0000","last_active":"2024-05-22T23:00:46.872913+0000","last_peered":"2024-05-22T23:00:46.872913+0000","last_clean":"2024-05-22T23:00:46.872913+0000","last_became_active":"2024-05-22T22:59:51.595561+0000","last_became_peered":"2024-05-22T22:59:51.595561+0000","last_unstale":"2024-05-22T23:00:46.872913+0000","last_undegraded":"2024-05-22T23:00:46.872913+0000","last_fullsized":"2024-05-22T23:00:46.872913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:37:04.701575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":354,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580190+0000","last_change":"2024-05-22T22:58:34.927654+0000","last_active":"2024-05-22T23:00:46.580190+0000","last_peered":"2024-05-22T23:00:46.580190+0000","last_clean":"2024-05-22T23:00:46.580190+0000","last_became_active":"2024-05-22T22:58:34.927090+0000","last_became_peered":"2024-05-22T22:58:34.927090+0000","last_unstale":"2024-05-22T23:00:46.580190+0000","last_undegraded":"2024-05-22T23:00:46.580190+0000","last_fullsized":"2024-05-22T23:00:46.580190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:55:07.719852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:55:07.719852+0000","last_clean_scrub_stamp":"2024-05-22T22:55:07.719852+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:40:53.763146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578233+0000","last_change":"2024-05-22T22:59:51.956422+0000","last_active":"2024-05-22T23:00:46.578233+0000","last_peered":"2024-05-22T23:00:46.578233+0000","last_clean":"2024-05-22T23:00:46.578233+0000","last_became_active":"2024-05-22T22:59:51.956127+0000","last_became_peered":"2024-05-22T22:59:51.956127+0000","last_unstale":"2024-05-22T23:00:46.578233+0000","last_undegraded":"2024-05-22T23:00:46.578233+0000","last_fullsized":"2024-05-22T23:00:46.578233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:04:36.955644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.874231+0000","last_change":"2024-05-22T23:00:46.874231+0000","last_active":"2024-05-22T23:00:46.874231+0000","last_peered":"2024-05-22T23:00:46.874231+0000","last_clean":"2024-05-22T23:00:46.874231+0000","last_became_active":"2024-05-22T23:00:46.873728+0000","last_became_peered":"2024-05-22T23:00:46.873728+0000","last_unstale":"2024-05-22T23:00:46.874231+0000","last_undegraded":"2024-05-22T23:00:46.874231+0000","last_fullsized":"2024-05-22T23:00:46.874231+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:49:12.372179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.579460+0000","last_change":"2024-05-22T22:59:51.598772+0000","last_active":"2024-05-22T23:00:46.579460+0000","last_peered":"2024-05-22T23:00:46.579460+0000","last_clean":"2024-05-22T23:00:46.579460+0000","last_became_active":"2024-05-22T22:59:51.598192+0000","last_became_peered":"2024-05-22T22:59:51.598192+0000","last_unstale":"2024-05-22T23:00:46.579460+0000","last_undegraded":"2024-05-22T23:00:46.579460+0000","last_fullsized":"2024-05-22T23:00:46.579460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:42:47.069554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578768+0000","last_change":"2024-05-22T22:59:51.942337+0000","last_active":"2024-05-22T23:00:46.578768+0000","last_peered":"2024-05-22T23:00:46.578768+0000","last_clean":"2024-05-22T23:00:46.578768+0000","last_became_active":"2024-05-22T22:59:51.941939+0000","last_became_peered":"2024-05-22T22:59:51.941939+0000","last_unstale":"2024-05-22T23:00:46.578768+0000","last_undegraded":"2024-05-22T23:00:46.578768+0000","last_fullsized":"2024-05-22T23:00:46.578768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:35:42.360128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.576300+0000","last_change":"2024-05-22T22:59:51.589640+0000","last_active":"2024-05-22T23:00:46.576300+0000","last_peered":"2024-05-22T23:00:46.576300+0000","last_clean":"2024-05-22T23:00:46.576300+0000","last_became_active":"2024-05-22T22:59:51.589310+0000","last_became_peered":"2024-05-22T22:59:51.589310+0000","last_unstale":"2024-05-22T23:00:46.576300+0000","last_undegraded":"2024-05-22T23:00:46.576300+0000","last_fullsized":"2024-05-22T23:00:46.576300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:54:39.182002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.872152+0000","last_change":"2024-05-22T22:59:51.952423+0000","last_active":"2024-05-22T23:00:46.872152+0000","last_peered":"2024-05-22T23:00:46.872152+0000","last_clean":"2024-05-22T23:00:46.872152+0000","last_became_active":"2024-05-22T22:59:51.952015+0000","last_became_peered":"2024-05-22T22:59:51.952015+0000","last_unstale":"2024-05-22T23:00:46.872152+0000","last_undegraded":"2024-05-22T23:00:46.872152+0000","last_fullsized":"2024-05-22T23:00:46.872152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:06:23.705248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.872910+0000","last_change":"2024-05-22T22:59:51.594882+0000","last_active":"2024-05-22T23:00:46.872910+0000","last_peered":"2024-05-22T23:00:46.872910+0000","last_clean":"2024-05-22T23:00:46.872910+0000","last_became_active":"2024-05-22T22:59:51.594552+0000","last_became_peered":"2024-05-22T22:59:51.594552+0000","last_unstale":"2024-05-22T23:00:46.872910+0000","last_undegraded":"2024-05-22T23:00:46.872910+0000","last_fullsized":"2024-05-22T23:00:46.872910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:06:40.771969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580806+0000","last_change":"2024-05-22T22:59:51.956534+0000","last_active":"2024-05-22T23:00:46.580806+0000","last_peered":"2024-05-22T23:00:46.580806+0000","last_clean":"2024-05-22T23:00:46.580806+0000","last_became_active":"2024-05-22T22:59:51.956037+0000","last_became_peered":"2024-05-22T22:59:51.956037+0000","last_unstale":"2024-05-22T23:00:46.580806+0000","last_undegraded":"2024-05-22T23:00:46.580806+0000","last_fullsized":"2024-05-22T23:00:46.580806+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:24:15.289570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.579471+0000","last_change":"2024-05-22T22:59:51.592916+0000","last_active":"2024-05-22T23:00:46.579471+0000","last_peered":"2024-05-22T23:00:46.579471+0000","last_clean":"2024-05-22T23:00:46.579471+0000","last_became_active":"2024-05-22T22:59:51.592530+0000","last_became_peered":"2024-05-22T22:59:51.592530+0000","last_unstale":"2024-05-22T23:00:46.579471+0000","last_undegraded":"2024-05-22T23:00:46.579471+0000","last_fullsized":"2024-05-22T23:00:46.579471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:56:18.660120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578007+0000","last_change":"2024-05-22T22:59:51.956188+0000","last_active":"2024-05-22T23:00:46.578007+0000","last_peered":"2024-05-22T23:00:46.578007+0000","last_clean":"2024-05-22T23:00:46.578007+0000","last_became_active":"2024-05-22T22:59:51.955844+0000","last_became_peered":"2024-05-22T22:59:51.955844+0000","last_unstale":"2024-05-22T23:00:46.578007+0000","last_undegraded":"2024-05-22T23:00:46.578007+0000","last_fullsized":"2024-05-22T23:00:46.578007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:09:12.848822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.872231+0000","last_change":"2024-05-22T22:59:51.590641+0000","last_active":"2024-05-22T23:00:46.872231+0000","last_peered":"2024-05-22T23:00:46.872231+0000","last_clean":"2024-05-22T23:00:46.872231+0000","last_became_active":"2024-05-22T22:59:51.590321+0000","last_became_peered":"2024-05-22T22:59:51.590321+0000","last_unstale":"2024-05-22T23:00:46.872231+0000","last_undegraded":"2024-05-22T23:00:46.872231+0000","last_fullsized":"2024-05-22T23:00:46.872231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:46:15.157160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.576955+0000","last_change":"2024-05-22T22:59:51.950256+0000","last_active":"2024-05-22T23:00:46.576955+0000","last_peered":"2024-05-22T23:00:46.576955+0000","last_clean":"2024-05-22T23:00:46.576955+0000","last_became_active":"2024-05-22T22:59:51.949746+0000","last_became_peered":"2024-05-22T22:59:51.949746+0000","last_unstale":"2024-05-22T23:00:46.576955+0000","last_undegraded":"2024-05-22T23:00:46.576955+0000","last_fullsized":"2024-05-22T23:00:46.576955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:33:12.146425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.874879+0000","last_change":"2024-05-22T23:00:46.874312+0000","last_active":"2024-05-22T23:00:46.874879+0000","last_peered":"2024-05-22T23:00:46.874879+0000","last_clean":"2024-05-22T23:00:46.874879+0000","last_became_active":"2024-05-22T23:00:46.873803+0000","last_became_peered":"2024-05-22T23:00:46.873803+0000","last_unstale":"2024-05-22T23:00:46.874879+0000","last_undegraded":"2024-05-22T23:00:46.874879+0000","last_fullsized":"2024-05-22T23:00:46.874879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:44:48.915692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.872161+0000","last_change":"2024-05-22T22:59:51.968432+0000","last_active":"2024-05-22T23:00:46.872161+0000","last_peered":"2024-05-22T23:00:46.872161+0000","last_clean":"2024-05-22T23:00:46.872161+0000","last_became_active":"2024-05-22T22:59:51.968077+0000","last_became_peered":"2024-05-22T22:59:51.968077+0000","last_unstale":"2024-05-22T23:00:46.872161+0000","last_undegraded":"2024-05-22T23:00:46.872161+0000","last_fullsized":"2024-05-22T23:00:46.872161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:08:16.281671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578210+0000","last_change":"2024-05-22T22:59:51.601616+0000","last_active":"2024-05-22T23:00:46.578210+0000","last_peered":"2024-05-22T23:00:46.578210+0000","last_clean":"2024-05-22T23:00:46.578210+0000","last_became_active":"2024-05-22T22:59:51.601249+0000","last_became_peered":"2024-05-22T22:59:51.601249+0000","last_unstale":"2024-05-22T23:00:46.578210+0000","last_undegraded":"2024-05-22T23:00:46.578210+0000","last_fullsized":"2024-05-22T23:00:46.578210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:07:07.030142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580327+0000","last_change":"2024-05-22T22:59:51.951578+0000","last_active":"2024-05-22T23:00:46.580327+0000","last_peered":"2024-05-22T23:00:46.580327+0000","last_clean":"2024-05-22T23:00:46.580327+0000","last_became_active":"2024-05-22T22:59:51.951233+0000","last_became_peered":"2024-05-22T22:59:51.951233+0000","last_unstale":"2024-05-22T23:00:46.580327+0000","last_undegraded":"2024-05-22T23:00:46.580327+0000","last_fullsized":"2024-05-22T23:00:46.580327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:25.882943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580890+0000","last_change":"2024-05-22T22:59:51.597819+0000","last_active":"2024-05-22T23:00:46.580890+0000","last_peered":"2024-05-22T23:00:46.580890+0000","last_clean":"2024-05-22T23:00:46.580890+0000","last_became_active":"2024-05-22T22:59:51.597262+0000","last_became_peered":"2024-05-22T22:59:51.597262+0000","last_unstale":"2024-05-22T23:00:46.580890+0000","last_undegraded":"2024-05-22T23:00:46.580890+0000","last_fullsized":"2024-05-22T23:00:46.580890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:05:02.849623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580598+0000","last_change":"2024-05-22T22:59:51.951668+0000","last_active":"2024-05-22T23:00:46.580598+0000","last_peered":"2024-05-22T23:00:46.580598+0000","last_clean":"2024-05-22T23:00:46.580598+0000","last_became_active":"2024-05-22T22:59:51.951176+0000","last_became_peered":"2024-05-22T22:59:51.951176+0000","last_unstale":"2024-05-22T23:00:46.580598+0000","last_undegraded":"2024-05-22T23:00:46.580598+0000","last_fullsized":"2024-05-22T23:00:46.580598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:01:46.010858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.582480+0000","last_change":"2024-05-22T22:59:51.593144+0000","last_active":"2024-05-22T23:00:46.582480+0000","last_peered":"2024-05-22T23:00:46.582480+0000","last_clean":"2024-05-22T23:00:46.582480+0000","last_became_active":"2024-05-22T22:59:51.592557+0000","last_became_peered":"2024-05-22T22:59:51.592557+0000","last_unstale":"2024-05-22T23:00:46.582480+0000","last_undegraded":"2024-05-22T23:00:46.582480+0000","last_fullsized":"2024-05-22T23:00:46.582480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:28:19.403419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580452+0000","last_change":"2024-05-22T22:59:51.958866+0000","last_active":"2024-05-22T23:00:46.580452+0000","last_peered":"2024-05-22T23:00:46.580452+0000","last_clean":"2024-05-22T23:00:46.580452+0000","last_became_active":"2024-05-22T22:59:51.958505+0000","last_became_peered":"2024-05-22T22:59:51.958505+0000","last_unstale":"2024-05-22T23:00:46.580452+0000","last_undegraded":"2024-05-22T23:00:46.580452+0000","last_fullsized":"2024-05-22T23:00:46.580452+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:54:24.601966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578823+0000","last_change":"2024-05-22T22:59:51.587391+0000","last_active":"2024-05-22T23:00:46.578823+0000","last_peered":"2024-05-22T23:00:46.578823+0000","last_clean":"2024-05-22T23:00:46.578823+0000","last_became_active":"2024-05-22T22:59:51.587056+0000","last_became_peered":"2024-05-22T22:59:51.587056+0000","last_unstale":"2024-05-22T23:00:46.578823+0000","last_undegraded":"2024-05-22T23:00:46.578823+0000","last_fullsized":"2024-05-22T23:00:46.578823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:04:40.614853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580699+0000","last_change":"2024-05-22T22:59:51.953466+0000","last_active":"2024-05-22T23:00:46.580699+0000","last_peered":"2024-05-22T23:00:46.580699+0000","last_clean":"2024-05-22T23:00:46.580699+0000","last_became_active":"2024-05-22T22:59:51.952830+0000","last_became_peered":"2024-05-22T22:59:51.952830+0000","last_unstale":"2024-05-22T23:00:46.580699+0000","last_undegraded":"2024-05-22T23:00:46.580699+0000","last_fullsized":"2024-05-22T23:00:46.580699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:36:06.314680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.577075+0000","last_change":"2024-05-22T22:59:51.962993+0000","last_active":"2024-05-22T23:00:46.577075+0000","last_peered":"2024-05-22T23:00:46.577075+0000","last_clean":"2024-05-22T23:00:46.577075+0000","last_became_active":"2024-05-22T22:59:51.962591+0000","last_became_peered":"2024-05-22T22:59:51.962591+0000","last_unstale":"2024-05-22T23:00:46.577075+0000","last_undegraded":"2024-05-22T23:00:46.577075+0000","last_fullsized":"2024-05-22T23:00:46.577075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:55:50.902537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580625+0000","last_change":"2024-05-22T22:59:51.956304+0000","last_active":"2024-05-22T23:00:46.580625+0000","last_peered":"2024-05-22T23:00:46.580625+0000","last_clean":"2024-05-22T23:00:46.580625+0000","last_became_active":"2024-05-22T22:59:51.955933+0000","last_became_peered":"2024-05-22T22:59:51.955933+0000","last_unstale":"2024-05-22T23:00:46.580625+0000","last_undegraded":"2024-05-22T23:00:46.580625+0000","last_fullsized":"2024-05-22T23:00:46.580625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:28:17.225851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580619+0000","last_change":"2024-05-22T22:59:51.946806+0000","last_active":"2024-05-22T23:00:46.580619+0000","last_peered":"2024-05-22T23:00:46.580619+0000","last_clean":"2024-05-22T23:00:46.580619+0000","last_became_active":"2024-05-22T22:59:51.946433+0000","last_became_peered":"2024-05-22T22:59:51.946433+0000","last_unstale":"2024-05-22T23:00:46.580619+0000","last_undegraded":"2024-05-22T23:00:46.580619+0000","last_fullsized":"2024-05-22T23:00:46.580619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:24:06.013688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580610+0000","last_change":"2024-05-22T22:59:51.956466+0000","last_active":"2024-05-22T23:00:46.580610+0000","last_peered":"2024-05-22T23:00:46.580610+0000","last_clean":"2024-05-22T23:00:46.580610+0000","last_became_active":"2024-05-22T22:59:51.956144+0000","last_became_peered":"2024-05-22T22:59:51.956144+0000","last_unstale":"2024-05-22T23:00:46.580610+0000","last_undegraded":"2024-05-22T23:00:46.580610+0000","last_fullsized":"2024-05-22T23:00:46.580610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:08:28.674824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.582147+0000","last_change":"2024-05-22T23:00:46.582147+0000","last_active":"2024-05-22T23:00:46.582147+0000","last_peered":"2024-05-22T23:00:46.582147+0000","last_clean":"2024-05-22T23:00:46.582147+0000","last_became_active":"2024-05-22T23:00:46.581804+0000","last_became_peered":"2024-05-22T23:00:46.581804+0000","last_unstale":"2024-05-22T23:00:46.582147+0000","last_undegraded":"2024-05-22T23:00:46.582147+0000","last_fullsized":"2024-05-22T23:00:46.582147+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:02:15.933895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.577417+0000","last_change":"2024-05-22T22:59:51.961971+0000","last_active":"2024-05-22T23:00:46.577417+0000","last_peered":"2024-05-22T23:00:46.577417+0000","last_clean":"2024-05-22T23:00:46.577417+0000","last_became_active":"2024-05-22T22:59:51.961604+0000","last_became_peered":"2024-05-22T22:59:51.961604+0000","last_unstale":"2024-05-22T23:00:46.577417+0000","last_undegraded":"2024-05-22T23:00:46.577417+0000","last_fullsized":"2024-05-22T23:00:46.577417+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:32:14.669631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.581792+0000","last_change":"2024-05-22T22:59:51.948173+0000","last_active":"2024-05-22T23:00:46.581792+0000","last_peered":"2024-05-22T23:00:46.581792+0000","last_clean":"2024-05-22T23:00:46.581792+0000","last_became_active":"2024-05-22T22:59:51.947839+0000","last_became_peered":"2024-05-22T22:59:51.947839+0000","last_unstale":"2024-05-22T23:00:46.581792+0000","last_undegraded":"2024-05-22T23:00:46.581792+0000","last_fullsized":"2024-05-22T23:00:46.581792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:55:07.748445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.872035+0000","last_change":"2024-05-22T22:59:51.955627+0000","last_active":"2024-05-22T23:00:46.872035+0000","last_peered":"2024-05-22T23:00:46.872035+0000","last_clean":"2024-05-22T23:00:46.872035+0000","last_became_active":"2024-05-22T22:59:51.955284+0000","last_became_peered":"2024-05-22T22:59:51.955284+0000","last_unstale":"2024-05-22T23:00:46.872035+0000","last_undegraded":"2024-05-22T23:00:46.872035+0000","last_fullsized":"2024-05-22T23:00:46.872035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:57:13.029317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.582786+0000","last_change":"2024-05-22T22:59:51.955889+0000","last_active":"2024-05-22T23:00:46.582786+0000","last_peered":"2024-05-22T23:00:46.582786+0000","last_clean":"2024-05-22T23:00:46.582786+0000","last_became_active":"2024-05-22T22:59:51.954930+0000","last_became_peered":"2024-05-22T22:59:51.954930+0000","last_unstale":"2024-05-22T23:00:46.582786+0000","last_undegraded":"2024-05-22T23:00:46.582786+0000","last_fullsized":"2024-05-22T23:00:46.582786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:47.536072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.872113+0000","last_change":"2024-05-22T22:59:51.954148+0000","last_active":"2024-05-22T23:00:46.872113+0000","last_peered":"2024-05-22T23:00:46.872113+0000","last_clean":"2024-05-22T23:00:46.872113+0000","last_became_active":"2024-05-22T22:59:51.952336+0000","last_became_peered":"2024-05-22T22:59:51.952336+0000","last_unstale":"2024-05-22T23:00:46.872113+0000","last_undegraded":"2024-05-22T23:00:46.872113+0000","last_fullsized":"2024-05-22T23:00:46.872113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:07:27.928910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.579454+0000","last_change":"2024-05-22T22:59:51.954300+0000","last_active":"2024-05-22T23:00:46.579454+0000","last_peered":"2024-05-22T23:00:46.579454+0000","last_clean":"2024-05-22T23:00:46.579454+0000","last_became_active":"2024-05-22T22:59:51.953779+0000","last_became_peered":"2024-05-22T22:59:51.953779+0000","last_unstale":"2024-05-22T23:00:46.579454+0000","last_undegraded":"2024-05-22T23:00:46.579454+0000","last_fullsized":"2024-05-22T23:00:46.579454+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:24:59.363634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578618+0000","last_change":"2024-05-22T22:59:51.947860+0000","last_active":"2024-05-22T23:00:46.578618+0000","last_peered":"2024-05-22T23:00:46.578618+0000","last_clean":"2024-05-22T23:00:46.578618+0000","last_became_active":"2024-05-22T22:59:51.947420+0000","last_became_peered":"2024-05-22T22:59:51.947420+0000","last_unstale":"2024-05-22T23:00:46.578618+0000","last_undegraded":"2024-05-22T23:00:46.578618+0000","last_fullsized":"2024-05-22T23:00:46.578618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:20:40.629285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.576982+0000","last_change":"2024-05-22T22:59:51.957867+0000","last_active":"2024-05-22T23:00:46.576982+0000","last_peered":"2024-05-22T23:00:46.576982+0000","last_clean":"2024-05-22T23:00:46.576982+0000","last_became_active":"2024-05-22T22:59:51.957522+0000","last_became_peered":"2024-05-22T22:59:51.957522+0000","last_unstale":"2024-05-22T23:00:46.576982+0000","last_undegraded":"2024-05-22T23:00:46.576982+0000","last_fullsized":"2024-05-22T23:00:46.576982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:41:18.631970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280555,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59709,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":134,"num_read_kb":127,"num_write":72,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":58911,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":217088,"data_stored":179127,"data_compressed":13249,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":0,"internal_metadata":0},"log_size":61,"ondisk_log_size":61,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351005,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":461163,"data_compressed":5755,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612642,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":1110753,"data_compressed":43296,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776168,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":461745,"data_compressed":5755,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037804,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":461767,"data_compressed":5755,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299442,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":1052230,"data_compressed":41439,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495670,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":461280,"data_compressed":5755,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1015808,"data_stored":520074,"data_compressed":7612,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":954368,"data_stored":461253,"data_compressed":5755,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":461197,"data_compressed":5755,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":966656,"data_stored":461501,"data_compressed":5755,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":520096,"data_compressed":15290,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1015808,"data_stored":1051830,"data_compressed":41439,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":58911,"data_compressed":1857,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":58933,"data_compressed":9535,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":59105,"data_compressed":1857,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-22T23:01:07.928 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-22T23:01:08.167 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:01:08.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:08 smithi084 ceph-mon[32186]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-22T23:01:08.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:08 smithi084 ceph-mon[32186]: pgmap v329: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:01:08.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:08 smithi204 ceph-mon[36211]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-22T23:01:08.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:08 smithi204 ceph-mon[36211]: pgmap v329: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:01:09.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:08 smithi096 ceph-mon[36392]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-22T23:01:09.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:08 smithi096 ceph-mon[36392]: pgmap v329: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:01:09.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.848+0000 7ff898cb3700 1 -- 172.21.15.84:0/4039649772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff894102610 msgr2=0x7ff8941029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:09.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.848+0000 7ff898cb3700 1 --2- 172.21.15.84:0/4039649772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff894102610 0x7ff8941029f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00b0a0 tx=0x7ff88c00b3b0 comp rx=0 tx=0).stop 2024-05-22T23:01:09.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.848+0000 7ff898cb3700 1 -- 172.21.15.84:0/4039649772 shutdown_connections 2024-05-22T23:01:09.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.848+0000 7ff898cb3700 1 --2- 172.21.15.84:0/4039649772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8941076a0 0x7ff89410f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:09.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.848+0000 7ff898cb3700 1 --2- 172.21.15.84:0/4039649772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff894102610 0x7ff8941029f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:09.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.848+0000 7ff898cb3700 1 --2- 172.21.15.84:0/4039649772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff894102fc0 0x7ff894106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:09.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.848+0000 7ff898cb3700 1 -- 172.21.15.84:0/4039649772 >> 172.21.15.84:0/4039649772 conn(0x7ff8940794c0 msgr2=0x7ff8940798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:09.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.848+0000 7ff898cb3700 1 -- 172.21.15.84:0/4039649772 shutdown_connections 2024-05-22T23:01:09.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.848+0000 7ff898cb3700 1 -- 172.21.15.84:0/4039649772 wait complete. 2024-05-22T23:01:09.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.849+0000 7ff898cb3700 1 Processor -- start 2024-05-22T23:01:09.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.849+0000 7ff898cb3700 1 -- start start 2024-05-22T23:01:09.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.849+0000 7ff898cb3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff894102610 0x7ff894074660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:09.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.849+0000 7ff898cb3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff894102fc0 0x7ff894074ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:09.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.849+0000 7ff898cb3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8941076a0 0x7ff8940750e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:09.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.849+0000 7ff898cb3700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff894112760 con 0x7ff8941076a0 2024-05-22T23:01:09.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.849+0000 7ff898cb3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff8941125e0 con 0x7ff894102fc0 2024-05-22T23:01:09.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.849+0000 7ff898cb3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff8941128e0 con 0x7ff894102610 2024-05-22T23:01:09.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.849+0000 7ff892d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8941076a0 0x7ff8940750e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:09.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.849+0000 7ff892d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8941076a0 0x7ff8940750e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59538/0 (socket says 172.21.15.84:59538) 2024-05-22T23:01:09.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.849+0000 7ff89259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff894102610 0x7ff894074660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:09.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.849+0000 7ff892d9d700 1 -- 172.21.15.84:0/4184754164 learned_addr learned my addr 172.21.15.84:0/4184754164 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:09.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.849+0000 7ff891d9b700 1 --2- 172.21.15.84:0/4184754164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff894102fc0 0x7ff894074ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:09.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.850+0000 7ff892d9d700 1 -- 172.21.15.84:0/4184754164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff894102610 msgr2=0x7ff894074660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:09.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.850+0000 7ff892d9d700 1 --2- 172.21.15.84:0/4184754164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff894102610 0x7ff894074660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:09.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.850+0000 7ff892d9d700 1 -- 172.21.15.84:0/4184754164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff894102fc0 msgr2=0x7ff894074ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:09.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.850+0000 7ff892d9d700 1 --2- 172.21.15.84:0/4184754164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff894102fc0 0x7ff894074ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:09.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.850+0000 7ff892d9d700 1 -- 172.21.15.84:0/4184754164 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8941a70f0 con 0x7ff8941076a0 2024-05-22T23:01:09.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.850+0000 7ff892d9d700 1 --2- 172.21.15.84:0/4184754164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8941076a0 0x7ff8940750e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff88400b550 tx=0x7ff88400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:09.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.850+0000 7ff88b7fe700 1 -- 172.21.15.84:0/4184754164 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff884015020 con 0x7ff8941076a0 2024-05-22T23:01:09.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.850+0000 7ff898cb3700 1 -- 172.21.15.84:0/4184754164 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8941a73e0 con 0x7ff8941076a0 2024-05-22T23:01:09.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.850+0000 7ff88b7fe700 1 -- 172.21.15.84:0/4184754164 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff884011d10 con 0x7ff8941076a0 2024-05-22T23:01:09.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.850+0000 7ff88b7fe700 1 -- 172.21.15.84:0/4184754164 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff884010e00 con 0x7ff8941076a0 2024-05-22T23:01:09.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.850+0000 7ff898cb3700 1 -- 172.21.15.84:0/4184754164 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8941a7970 con 0x7ff8941076a0 2024-05-22T23:01:09.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.851+0000 7ff898cb3700 1 -- 172.21.15.84:0/4184754164 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff89410a8c0 con 0x7ff8941076a0 2024-05-22T23:01:09.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.852+0000 7ff88b7fe700 1 -- 172.21.15.84:0/4184754164 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff884018020 con 0x7ff8941076a0 2024-05-22T23:01:09.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.853+0000 7ff88b7fe700 1 --2- 172.21.15.84:0/4184754164 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7ff8800a2cb0 0x7ff8800a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:09.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.853+0000 7ff89259c700 1 --2- 172.21.15.84:0/4184754164 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7ff8800a2cb0 0x7ff8800a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:09.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.853+0000 7ff88b7fe700 1 -- 172.21.15.84:0/4184754164 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7ff8840ce720 con 0x7ff8941076a0 2024-05-22T23:01:09.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.853+0000 7ff89259c700 1 --2- 172.21.15.84:0/4184754164 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7ff8800a2cb0 0x7ff8800a5170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00bf50 tx=0x7ff88c00b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:09.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:09.857+0000 7ff88b7fe700 1 -- 172.21.15.84:0/4184754164 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff88401f020 con 0x7ff8941076a0 2024-05-22T23:01:10.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:10.045+0000 7ff898cb3700 1 -- 172.21.15.84:0/4184754164 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff8940627a0 con 0x7ff8800a2cb0 2024-05-22T23:01:10.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:10.052+0000 7ff88b7fe700 1 -- 172.21.15.84:0/4184754164 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258946 (secure 0 0 0) 0x7ff8940627a0 con 0x7ff8800a2cb0 2024-05-22T23:01:10.056 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:10.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:10.055+0000 7ff898cb3700 1 -- 172.21.15.84:0/4184754164 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7ff8800a2cb0 msgr2=0x7ff8800a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:10.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:10.055+0000 7ff898cb3700 1 --2- 172.21.15.84:0/4184754164 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7ff8800a2cb0 0x7ff8800a5170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00bf50 tx=0x7ff88c00b7b0 comp rx=0 tx=0).stop 2024-05-22T23:01:10.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:10.055+0000 7ff898cb3700 1 -- 172.21.15.84:0/4184754164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8941076a0 msgr2=0x7ff8940750e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:10.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:10.055+0000 7ff898cb3700 1 --2- 172.21.15.84:0/4184754164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8941076a0 0x7ff8940750e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff88400b550 tx=0x7ff88400b860 comp rx=0 tx=0).stop 2024-05-22T23:01:10.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:10.055+0000 7ff898cb3700 1 -- 172.21.15.84:0/4184754164 shutdown_connections 2024-05-22T23:01:10.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:10.055+0000 7ff898cb3700 1 --2- 172.21.15.84:0/4184754164 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7ff8800a2cb0 0x7ff8800a5170 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:10.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:10.055+0000 7ff898cb3700 1 --2- 172.21.15.84:0/4184754164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8941076a0 0x7ff8940750e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:10.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:10.055+0000 7ff898cb3700 1 --2- 172.21.15.84:0/4184754164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff894102610 0x7ff894074660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:10.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:10.055+0000 7ff898cb3700 1 --2- 172.21.15.84:0/4184754164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff894102fc0 0x7ff894074ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:10.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:10.055+0000 7ff898cb3700 1 -- 172.21.15.84:0/4184754164 >> 172.21.15.84:0/4184754164 conn(0x7ff8940794c0 msgr2=0x7ff894101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:10.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:10.055+0000 7ff898cb3700 1 -- 172.21.15.84:0/4184754164 shutdown_connections 2024-05-22T23:01:10.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:10.055+0000 7ff898cb3700 1 -- 172.21.15.84:0/4184754164 wait complete. 2024-05-22T23:01:10.059 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2024-05-22T23:01:10.763 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:10 smithi084 ceph-mon[32186]: pgmap v330: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:01:10.791 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-05-22T23:01:09.512696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650311,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":280,"num_read_kb":251,"num_write":255,"num_write_kb":3019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1239647,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377380,"kb_used_data":11516,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564444,"statfs":{"total":1151940427776,"available":1151553990656,"internally_reserved":0,"allocated":11792384,"data_stored":7484889,"data_compressed":189361,"data_compressed_allocated":253952,"data_compressed_original":2039808,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002765"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.577659+0000","last_change":"2024-05-22T22:59:51.951625+0000","last_active":"2024-05-22T23:00:46.577659+0000","last_peered":"2024-05-22T23:00:46.577659+0000","last_clean":"2024-05-22T23:00:46.577659+0000","last_became_active":"2024-05-22T22:59:51.951225+0000","last_became_peered":"2024-05-22T22:59:51.951225+0000","last_unstale":"2024-05-22T23:00:46.577659+0000","last_undegraded":"2024-05-22T23:00:46.577659+0000","last_fullsized":"2024-05-22T23:00:46.577659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:53:32.616083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580762+0000","last_change":"2024-05-22T22:59:51.956925+0000","last_active":"2024-05-22T23:00:46.580762+0000","last_peered":"2024-05-22T23:00:46.580762+0000","last_clean":"2024-05-22T23:00:46.580762+0000","last_became_active":"2024-05-22T22:59:51.956557+0000","last_became_peered":"2024-05-22T22:59:51.956557+0000","last_unstale":"2024-05-22T23:00:46.580762+0000","last_undegraded":"2024-05-22T23:00:46.580762+0000","last_fullsized":"2024-05-22T23:00:46.580762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:18:14.609682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580836+0000","last_change":"2024-05-22T22:59:51.951981+0000","last_active":"2024-05-22T23:00:46.580836+0000","last_peered":"2024-05-22T23:00:46.580836+0000","last_clean":"2024-05-22T23:00:46.580836+0000","last_became_active":"2024-05-22T22:59:51.951522+0000","last_became_peered":"2024-05-22T22:59:51.951522+0000","last_unstale":"2024-05-22T23:00:46.580836+0000","last_undegraded":"2024-05-22T23:00:46.580836+0000","last_fullsized":"2024-05-22T23:00:46.580836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:56:23.357290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.873333+0000","last_change":"2024-05-22T22:59:51.966141+0000","last_active":"2024-05-22T23:00:46.873333+0000","last_peered":"2024-05-22T23:00:46.873333+0000","last_clean":"2024-05-22T23:00:46.873333+0000","last_became_active":"2024-05-22T22:59:51.965798+0000","last_became_peered":"2024-05-22T22:59:51.965798+0000","last_unstale":"2024-05-22T23:00:46.873333+0000","last_undegraded":"2024-05-22T23:00:46.873333+0000","last_fullsized":"2024-05-22T23:00:46.873333+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:23:58.282078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578081+0000","last_change":"2024-05-22T22:59:51.950425+0000","last_active":"2024-05-22T23:00:46.578081+0000","last_peered":"2024-05-22T23:00:46.578081+0000","last_clean":"2024-05-22T23:00:46.578081+0000","last_became_active":"2024-05-22T22:59:51.950071+0000","last_became_peered":"2024-05-22T22:59:51.950071+0000","last_unstale":"2024-05-22T23:00:46.578081+0000","last_undegraded":"2024-05-22T23:00:46.578081+0000","last_fullsized":"2024-05-22T23:00:46.578081+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:42:43.913816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.582890+0000","last_change":"2024-05-22T23:00:46.582890+0000","last_active":"2024-05-22T23:00:46.582890+0000","last_peered":"2024-05-22T23:00:46.582890+0000","last_clean":"2024-05-22T23:00:46.582890+0000","last_became_active":"2024-05-22T23:00:46.582497+0000","last_became_peered":"2024-05-22T23:00:46.582497+0000","last_unstale":"2024-05-22T23:00:46.582890+0000","last_undegraded":"2024-05-22T23:00:46.582890+0000","last_fullsized":"2024-05-22T23:00:46.582890+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:37:39.420650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580150+0000","last_change":"2024-05-22T22:59:51.959583+0000","last_active":"2024-05-22T23:00:46.580150+0000","last_peered":"2024-05-22T23:00:46.580150+0000","last_clean":"2024-05-22T23:00:46.580150+0000","last_became_active":"2024-05-22T22:59:51.959260+0000","last_became_peered":"2024-05-22T22:59:51.959260+0000","last_unstale":"2024-05-22T23:00:46.580150+0000","last_undegraded":"2024-05-22T23:00:46.580150+0000","last_fullsized":"2024-05-22T23:00:46.580150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:55:48.960861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.585335+0000","last_change":"2024-05-22T23:00:46.584888+0000","last_active":"2024-05-22T23:00:46.585335+0000","last_peered":"2024-05-22T23:00:46.585335+0000","last_clean":"2024-05-22T23:00:46.585335+0000","last_became_active":"2024-05-22T23:00:46.584534+0000","last_became_peered":"2024-05-22T23:00:46.584534+0000","last_unstale":"2024-05-22T23:00:46.585335+0000","last_undegraded":"2024-05-22T23:00:46.585335+0000","last_fullsized":"2024-05-22T23:00:46.585335+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:01:14.551458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.579732+0000","last_change":"2024-05-22T22:59:51.956954+0000","last_active":"2024-05-22T23:00:46.579732+0000","last_peered":"2024-05-22T23:00:46.579732+0000","last_clean":"2024-05-22T23:00:46.579732+0000","last_became_active":"2024-05-22T22:59:51.956570+0000","last_became_peered":"2024-05-22T22:59:51.956570+0000","last_unstale":"2024-05-22T23:00:46.579732+0000","last_undegraded":"2024-05-22T23:00:46.579732+0000","last_fullsized":"2024-05-22T23:00:46.579732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:55:18.828023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.577561+0000","last_change":"2024-05-22T22:59:51.962371+0000","last_active":"2024-05-22T23:00:46.577561+0000","last_peered":"2024-05-22T23:00:46.577561+0000","last_clean":"2024-05-22T23:00:46.577561+0000","last_became_active":"2024-05-22T22:59:51.962036+0000","last_became_peered":"2024-05-22T22:59:51.962036+0000","last_unstale":"2024-05-22T23:00:46.577561+0000","last_undegraded":"2024-05-22T23:00:46.577561+0000","last_fullsized":"2024-05-22T23:00:46.577561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:27:21.745563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580634+0000","last_change":"2024-05-22T22:59:51.952446+0000","last_active":"2024-05-22T23:00:46.580634+0000","last_peered":"2024-05-22T23:00:46.580634+0000","last_clean":"2024-05-22T23:00:46.580634+0000","last_became_active":"2024-05-22T22:59:51.952115+0000","last_became_peered":"2024-05-22T22:59:51.952115+0000","last_unstale":"2024-05-22T23:00:46.580634+0000","last_undegraded":"2024-05-22T23:00:46.580634+0000","last_fullsized":"2024-05-22T23:00:46.580634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:33:56.960951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.873308+0000","last_change":"2024-05-22T22:59:51.968457+0000","last_active":"2024-05-22T23:00:46.873308+0000","last_peered":"2024-05-22T23:00:46.873308+0000","last_clean":"2024-05-22T23:00:46.873308+0000","last_became_active":"2024-05-22T22:59:51.968070+0000","last_became_peered":"2024-05-22T22:59:51.968070+0000","last_unstale":"2024-05-22T23:00:46.873308+0000","last_undegraded":"2024-05-22T23:00:46.873308+0000","last_fullsized":"2024-05-22T23:00:46.873308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:07:31.595119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578528+0000","last_change":"2024-05-22T22:59:51.957357+0000","last_active":"2024-05-22T23:00:46.578528+0000","last_peered":"2024-05-22T23:00:46.578528+0000","last_clean":"2024-05-22T23:00:46.578528+0000","last_became_active":"2024-05-22T22:59:51.957013+0000","last_became_peered":"2024-05-22T22:59:51.957013+0000","last_unstale":"2024-05-22T23:00:46.578528+0000","last_undegraded":"2024-05-22T23:00:46.578528+0000","last_fullsized":"2024-05-22T23:00:46.578528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:38:48.118178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.581006+0000","last_change":"2024-05-22T22:59:51.957084+0000","last_active":"2024-05-22T23:00:46.581006+0000","last_peered":"2024-05-22T23:00:46.581006+0000","last_clean":"2024-05-22T23:00:46.581006+0000","last_became_active":"2024-05-22T22:59:51.956787+0000","last_became_peered":"2024-05-22T22:59:51.956787+0000","last_unstale":"2024-05-22T23:00:46.581006+0000","last_undegraded":"2024-05-22T23:00:46.581006+0000","last_fullsized":"2024-05-22T23:00:46.581006+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:30:40.578286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.579773+0000","last_change":"2024-05-22T22:59:51.956375+0000","last_active":"2024-05-22T23:00:46.579773+0000","last_peered":"2024-05-22T23:00:46.579773+0000","last_clean":"2024-05-22T23:00:46.579773+0000","last_became_active":"2024-05-22T22:59:51.956012+0000","last_became_peered":"2024-05-22T22:59:51.956012+0000","last_unstale":"2024-05-22T23:00:46.579773+0000","last_undegraded":"2024-05-22T23:00:46.579773+0000","last_fullsized":"2024-05-22T23:00:46.579773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:33:49.920327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.579651+0000","last_change":"2024-05-22T22:59:51.950287+0000","last_active":"2024-05-22T23:00:46.579651+0000","last_peered":"2024-05-22T23:00:46.579651+0000","last_clean":"2024-05-22T23:00:46.579651+0000","last_became_active":"2024-05-22T22:59:51.949962+0000","last_became_peered":"2024-05-22T22:59:51.949962+0000","last_unstale":"2024-05-22T23:00:46.579651+0000","last_undegraded":"2024-05-22T23:00:46.579651+0000","last_fullsized":"2024-05-22T23:00:46.579651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:33:20.674713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.577803+0000","last_change":"2024-05-22T22:59:51.952523+0000","last_active":"2024-05-22T23:00:46.577803+0000","last_peered":"2024-05-22T23:00:46.577803+0000","last_clean":"2024-05-22T23:00:46.577803+0000","last_became_active":"2024-05-22T22:59:51.952152+0000","last_became_peered":"2024-05-22T22:59:51.952152+0000","last_unstale":"2024-05-22T23:00:46.577803+0000","last_undegraded":"2024-05-22T23:00:46.577803+0000","last_fullsized":"2024-05-22T23:00:46.577803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:33:01.332667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.576185+0000","last_change":"2024-05-22T22:59:51.960919+0000","last_active":"2024-05-22T23:00:46.576185+0000","last_peered":"2024-05-22T23:00:46.576185+0000","last_clean":"2024-05-22T23:00:46.576185+0000","last_became_active":"2024-05-22T22:59:51.960603+0000","last_became_peered":"2024-05-22T22:59:51.960603+0000","last_unstale":"2024-05-22T23:00:46.576185+0000","last_undegraded":"2024-05-22T23:00:46.576185+0000","last_fullsized":"2024-05-22T23:00:46.576185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:28:05.977612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.581038+0000","last_change":"2024-05-22T22:59:51.949186+0000","last_active":"2024-05-22T23:00:46.581038+0000","last_peered":"2024-05-22T23:00:46.581038+0000","last_clean":"2024-05-22T23:00:46.581038+0000","last_became_active":"2024-05-22T22:59:51.948819+0000","last_became_peered":"2024-05-22T22:59:51.948819+0000","last_unstale":"2024-05-22T23:00:46.581038+0000","last_undegraded":"2024-05-22T23:00:46.581038+0000","last_fullsized":"2024-05-22T23:00:46.581038+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:32:16.153838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.576061+0000","last_change":"2024-05-22T22:59:51.592520+0000","last_active":"2024-05-22T23:00:46.576061+0000","last_peered":"2024-05-22T23:00:46.576061+0000","last_clean":"2024-05-22T23:00:46.576061+0000","last_became_active":"2024-05-22T22:59:51.592201+0000","last_became_peered":"2024-05-22T22:59:51.592201+0000","last_unstale":"2024-05-22T23:00:46.576061+0000","last_undegraded":"2024-05-22T23:00:46.576061+0000","last_fullsized":"2024-05-22T23:00:46.576061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:45:06.500085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.585804+0000","last_change":"2024-05-22T23:00:46.585147+0000","last_active":"2024-05-22T23:00:46.585804+0000","last_peered":"2024-05-22T23:00:46.585804+0000","last_clean":"2024-05-22T23:00:46.585804+0000","last_became_active":"2024-05-22T23:00:46.584828+0000","last_became_peered":"2024-05-22T23:00:46.584828+0000","last_unstale":"2024-05-22T23:00:46.585804+0000","last_undegraded":"2024-05-22T23:00:46.585804+0000","last_fullsized":"2024-05-22T23:00:46.585804+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:28:29.641493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.872962+0000","last_change":"2024-05-22T22:59:51.595383+0000","last_active":"2024-05-22T23:00:46.872962+0000","last_peered":"2024-05-22T23:00:46.872962+0000","last_clean":"2024-05-22T23:00:46.872962+0000","last_became_active":"2024-05-22T22:59:51.595076+0000","last_became_peered":"2024-05-22T22:59:51.595076+0000","last_unstale":"2024-05-22T23:00:46.872962+0000","last_undegraded":"2024-05-22T23:00:46.872962+0000","last_fullsized":"2024-05-22T23:00:46.872962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:29:57.205196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580990+0000","last_change":"2024-05-22T22:59:51.952447+0000","last_active":"2024-05-22T23:00:46.580990+0000","last_peered":"2024-05-22T23:00:46.580990+0000","last_clean":"2024-05-22T23:00:46.580990+0000","last_became_active":"2024-05-22T22:59:51.951927+0000","last_became_peered":"2024-05-22T22:59:51.951927+0000","last_unstale":"2024-05-22T23:00:46.580990+0000","last_undegraded":"2024-05-22T23:00:46.580990+0000","last_fullsized":"2024-05-22T23:00:46.580990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:14:59.161404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.579689+0000","last_change":"2024-05-22T22:59:51.597705+0000","last_active":"2024-05-22T23:00:46.579689+0000","last_peered":"2024-05-22T23:00:46.579689+0000","last_clean":"2024-05-22T23:00:46.579689+0000","last_became_active":"2024-05-22T22:59:51.597319+0000","last_became_peered":"2024-05-22T22:59:51.597319+0000","last_unstale":"2024-05-22T23:00:46.579689+0000","last_undegraded":"2024-05-22T23:00:46.579689+0000","last_fullsized":"2024-05-22T23:00:46.579689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:59:59.307499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.579842+0000","last_change":"2024-05-22T22:59:51.955477+0000","last_active":"2024-05-22T23:00:46.579842+0000","last_peered":"2024-05-22T23:00:46.579842+0000","last_clean":"2024-05-22T23:00:46.579842+0000","last_became_active":"2024-05-22T22:59:51.953608+0000","last_became_peered":"2024-05-22T22:59:51.953608+0000","last_unstale":"2024-05-22T23:00:46.579842+0000","last_undegraded":"2024-05-22T23:00:46.579842+0000","last_fullsized":"2024-05-22T23:00:46.579842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:27:35.192556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578795+0000","last_change":"2024-05-22T22:59:51.595417+0000","last_active":"2024-05-22T23:00:46.578795+0000","last_peered":"2024-05-22T23:00:46.578795+0000","last_clean":"2024-05-22T23:00:46.578795+0000","last_became_active":"2024-05-22T22:59:51.595051+0000","last_became_peered":"2024-05-22T22:59:51.595051+0000","last_unstale":"2024-05-22T23:00:46.578795+0000","last_undegraded":"2024-05-22T23:00:46.578795+0000","last_fullsized":"2024-05-22T23:00:46.578795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:13:47.936348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.582621+0000","last_change":"2024-05-22T22:59:51.958471+0000","last_active":"2024-05-22T23:00:46.582621+0000","last_peered":"2024-05-22T23:00:46.582621+0000","last_clean":"2024-05-22T23:00:46.582621+0000","last_became_active":"2024-05-22T22:59:51.958017+0000","last_became_peered":"2024-05-22T22:59:51.958017+0000","last_unstale":"2024-05-22T23:00:46.582621+0000","last_undegraded":"2024-05-22T23:00:46.582621+0000","last_fullsized":"2024-05-22T23:00:46.582621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:42:52.196892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.582468+0000","last_change":"2024-05-22T22:59:51.591909+0000","last_active":"2024-05-22T23:00:46.582468+0000","last_peered":"2024-05-22T23:00:46.582468+0000","last_clean":"2024-05-22T23:00:46.582468+0000","last_became_active":"2024-05-22T22:59:51.591531+0000","last_became_peered":"2024-05-22T22:59:51.591531+0000","last_unstale":"2024-05-22T23:00:46.582468+0000","last_undegraded":"2024-05-22T23:00:46.582468+0000","last_fullsized":"2024-05-22T23:00:46.582468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:07:22.997156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.581083+0000","last_change":"2024-05-22T23:00:46.581083+0000","last_active":"2024-05-22T23:00:46.581083+0000","last_peered":"2024-05-22T23:00:46.581083+0000","last_clean":"2024-05-22T23:00:46.581083+0000","last_became_active":"2024-05-22T23:00:46.580458+0000","last_became_peered":"2024-05-22T23:00:46.580458+0000","last_unstale":"2024-05-22T23:00:46.581083+0000","last_undegraded":"2024-05-22T23:00:46.581083+0000","last_fullsized":"2024-05-22T23:00:46.581083+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:38:53.136828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.576874+0000","last_change":"2024-05-22T22:59:51.590124+0000","last_active":"2024-05-22T23:00:46.576874+0000","last_peered":"2024-05-22T23:00:46.576874+0000","last_clean":"2024-05-22T23:00:46.576874+0000","last_became_active":"2024-05-22T22:59:51.589661+0000","last_became_peered":"2024-05-22T22:59:51.589661+0000","last_unstale":"2024-05-22T23:00:46.576874+0000","last_undegraded":"2024-05-22T23:00:46.576874+0000","last_fullsized":"2024-05-22T23:00:46.576874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:33:14.293487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.873993+0000","last_change":"2024-05-22T23:00:46.873993+0000","last_active":"2024-05-22T23:00:46.873993+0000","last_peered":"2024-05-22T23:00:46.873993+0000","last_clean":"2024-05-22T23:00:46.873993+0000","last_became_active":"2024-05-22T23:00:46.873523+0000","last_became_peered":"2024-05-22T23:00:46.873523+0000","last_unstale":"2024-05-22T23:00:46.873993+0000","last_undegraded":"2024-05-22T23:00:46.873993+0000","last_fullsized":"2024-05-22T23:00:46.873993+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:39:20.586352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578330+0000","last_change":"2024-05-22T22:59:51.595510+0000","last_active":"2024-05-22T23:00:46.578330+0000","last_peered":"2024-05-22T23:00:46.578330+0000","last_clean":"2024-05-22T23:00:46.578330+0000","last_became_active":"2024-05-22T22:59:51.595208+0000","last_became_peered":"2024-05-22T22:59:51.595208+0000","last_unstale":"2024-05-22T23:00:46.578330+0000","last_undegraded":"2024-05-22T23:00:46.578330+0000","last_fullsized":"2024-05-22T23:00:46.578330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:47:46.957965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.872063+0000","last_change":"2024-05-22T22:59:51.950684+0000","last_active":"2024-05-22T23:00:46.872063+0000","last_peered":"2024-05-22T23:00:46.872063+0000","last_clean":"2024-05-22T23:00:46.872063+0000","last_became_active":"2024-05-22T22:59:51.950380+0000","last_became_peered":"2024-05-22T22:59:51.950380+0000","last_unstale":"2024-05-22T23:00:46.872063+0000","last_undegraded":"2024-05-22T23:00:46.872063+0000","last_fullsized":"2024-05-22T23:00:46.872063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:58:02.035245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.582592+0000","last_change":"2024-05-22T22:59:51.592958+0000","last_active":"2024-05-22T23:00:46.582592+0000","last_peered":"2024-05-22T23:00:46.582592+0000","last_clean":"2024-05-22T23:00:46.582592+0000","last_became_active":"2024-05-22T22:59:51.592586+0000","last_became_peered":"2024-05-22T22:59:51.592586+0000","last_unstale":"2024-05-22T23:00:46.582592+0000","last_undegraded":"2024-05-22T23:00:46.582592+0000","last_fullsized":"2024-05-22T23:00:46.582592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:44:43.859724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.575722+0000","last_change":"2024-05-22T22:59:51.960863+0000","last_active":"2024-05-22T23:00:46.575722+0000","last_peered":"2024-05-22T23:00:46.575722+0000","last_clean":"2024-05-22T23:00:46.575722+0000","last_became_active":"2024-05-22T22:59:51.960537+0000","last_became_peered":"2024-05-22T22:59:51.960537+0000","last_unstale":"2024-05-22T23:00:46.575722+0000","last_undegraded":"2024-05-22T23:00:46.575722+0000","last_fullsized":"2024-05-22T23:00:46.575722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:08:19.735397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.577542+0000","last_change":"2024-05-22T22:59:51.596150+0000","last_active":"2024-05-22T23:00:46.577542+0000","last_peered":"2024-05-22T23:00:46.577542+0000","last_clean":"2024-05-22T23:00:46.577542+0000","last_became_active":"2024-05-22T22:59:51.595791+0000","last_became_peered":"2024-05-22T22:59:51.595791+0000","last_unstale":"2024-05-22T23:00:46.577542+0000","last_undegraded":"2024-05-22T23:00:46.577542+0000","last_fullsized":"2024-05-22T23:00:46.577542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:47:46.144045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.579638+0000","last_change":"2024-05-22T22:59:51.951907+0000","last_active":"2024-05-22T23:00:46.579638+0000","last_peered":"2024-05-22T23:00:46.579638+0000","last_clean":"2024-05-22T23:00:46.579638+0000","last_became_active":"2024-05-22T22:59:51.951264+0000","last_became_peered":"2024-05-22T22:59:51.951264+0000","last_unstale":"2024-05-22T23:00:46.579638+0000","last_undegraded":"2024-05-22T23:00:46.579638+0000","last_fullsized":"2024-05-22T23:00:46.579638+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:26:44.024848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580516+0000","last_change":"2024-05-22T22:59:51.594883+0000","last_active":"2024-05-22T23:00:46.580516+0000","last_peered":"2024-05-22T23:00:46.580516+0000","last_clean":"2024-05-22T23:00:46.580516+0000","last_became_active":"2024-05-22T22:59:51.594458+0000","last_became_peered":"2024-05-22T22:59:51.594458+0000","last_unstale":"2024-05-22T23:00:46.580516+0000","last_undegraded":"2024-05-22T23:00:46.580516+0000","last_fullsized":"2024-05-22T23:00:46.580516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:39:27.366961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580943+0000","last_change":"2024-05-22T22:59:51.958180+0000","last_active":"2024-05-22T23:00:46.580943+0000","last_peered":"2024-05-22T23:00:46.580943+0000","last_clean":"2024-05-22T23:00:46.580943+0000","last_became_active":"2024-05-22T22:59:51.957877+0000","last_became_peered":"2024-05-22T22:59:51.957877+0000","last_unstale":"2024-05-22T23:00:46.580943+0000","last_undegraded":"2024-05-22T23:00:46.580943+0000","last_fullsized":"2024-05-22T23:00:46.580943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:33:42.478340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.583107+0000","last_change":"2024-05-22T22:59:51.592953+0000","last_active":"2024-05-22T23:00:46.583107+0000","last_peered":"2024-05-22T23:00:46.583107+0000","last_clean":"2024-05-22T23:00:46.583107+0000","last_became_active":"2024-05-22T22:59:51.592525+0000","last_became_peered":"2024-05-22T22:59:51.592525+0000","last_unstale":"2024-05-22T23:00:46.583107+0000","last_undegraded":"2024-05-22T23:00:46.583107+0000","last_fullsized":"2024-05-22T23:00:46.583107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:18:03.512123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.579957+0000","last_change":"2024-05-22T22:59:51.959510+0000","last_active":"2024-05-22T23:00:46.579957+0000","last_peered":"2024-05-22T23:00:46.579957+0000","last_clean":"2024-05-22T23:00:46.579957+0000","last_became_active":"2024-05-22T22:59:51.959197+0000","last_became_peered":"2024-05-22T22:59:51.959197+0000","last_unstale":"2024-05-22T23:00:46.579957+0000","last_undegraded":"2024-05-22T23:00:46.579957+0000","last_fullsized":"2024-05-22T23:00:46.579957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:33:22.616668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'23","reported_seq":208,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:01:06.902851+0000","last_change":"2024-05-22T23:00:46.882566+0000","last_active":"2024-05-22T23:01:06.902851+0000","last_peered":"2024-05-22T23:01:06.902851+0000","last_clean":"2024-05-22T23:01:06.902851+0000","last_became_active":"2024-05-22T23:00:46.874658+0000","last_became_peered":"2024-05-22T23:00:46.874658+0000","last_unstale":"2024-05-22T23:01:06.902851+0000","last_undegraded":"2024-05-22T23:01:06.902851+0000","last_fullsized":"2024-05-22T23:01:06.902851+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:20:09.011734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58911,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":138,"num_read_kb":126,"num_write":23,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":58911,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.579306+0000","last_change":"2024-05-22T22:59:51.952486+0000","last_active":"2024-05-22T23:00:46.579306+0000","last_peered":"2024-05-22T23:00:46.579306+0000","last_clean":"2024-05-22T23:00:46.579306+0000","last_became_active":"2024-05-22T22:59:51.952096+0000","last_became_peered":"2024-05-22T22:59:51.952096+0000","last_unstale":"2024-05-22T23:00:46.579306+0000","last_undegraded":"2024-05-22T23:00:46.579306+0000","last_fullsized":"2024-05-22T23:00:46.579306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:52:33.877655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580801+0000","last_change":"2024-05-22T22:59:51.595446+0000","last_active":"2024-05-22T23:00:46.580801+0000","last_peered":"2024-05-22T23:00:46.580801+0000","last_clean":"2024-05-22T23:00:46.580801+0000","last_became_active":"2024-05-22T22:59:51.595097+0000","last_became_peered":"2024-05-22T22:59:51.595097+0000","last_unstale":"2024-05-22T23:00:46.580801+0000","last_undegraded":"2024-05-22T23:00:46.580801+0000","last_fullsized":"2024-05-22T23:00:46.580801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:03:29.722773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578434+0000","last_change":"2024-05-22T22:59:51.953304+0000","last_active":"2024-05-22T23:00:46.578434+0000","last_peered":"2024-05-22T23:00:46.578434+0000","last_clean":"2024-05-22T23:00:46.578434+0000","last_became_active":"2024-05-22T22:59:51.952974+0000","last_became_peered":"2024-05-22T22:59:51.952974+0000","last_unstale":"2024-05-22T23:00:46.578434+0000","last_undegraded":"2024-05-22T23:00:46.578434+0000","last_fullsized":"2024-05-22T23:00:46.578434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:21:06.116285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.582543+0000","last_change":"2024-05-22T22:59:51.593684+0000","last_active":"2024-05-22T23:00:46.582543+0000","last_peered":"2024-05-22T23:00:46.582543+0000","last_clean":"2024-05-22T23:00:46.582543+0000","last_became_active":"2024-05-22T22:59:51.593335+0000","last_became_peered":"2024-05-22T22:59:51.593335+0000","last_unstale":"2024-05-22T23:00:46.582543+0000","last_undegraded":"2024-05-22T23:00:46.582543+0000","last_fullsized":"2024-05-22T23:00:46.582543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:36:24.405609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580654+0000","last_change":"2024-05-22T22:59:51.960930+0000","last_active":"2024-05-22T23:00:46.580654+0000","last_peered":"2024-05-22T23:00:46.580654+0000","last_clean":"2024-05-22T23:00:46.580654+0000","last_became_active":"2024-05-22T22:59:51.960571+0000","last_became_peered":"2024-05-22T22:59:51.960571+0000","last_unstale":"2024-05-22T23:00:46.580654+0000","last_undegraded":"2024-05-22T23:00:46.580654+0000","last_fullsized":"2024-05-22T23:00:46.580654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:50:31.472449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.872580+0000","last_change":"2024-05-22T22:59:51.592876+0000","last_active":"2024-05-22T23:00:46.872580+0000","last_peered":"2024-05-22T23:00:46.872580+0000","last_clean":"2024-05-22T23:00:46.872580+0000","last_became_active":"2024-05-22T22:59:51.592488+0000","last_became_peered":"2024-05-22T22:59:51.592488+0000","last_unstale":"2024-05-22T23:00:46.872580+0000","last_undegraded":"2024-05-22T23:00:46.872580+0000","last_fullsized":"2024-05-22T23:00:46.872580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:37:16.517302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.579973+0000","last_change":"2024-05-22T22:59:51.962336+0000","last_active":"2024-05-22T23:00:46.579973+0000","last_peered":"2024-05-22T23:00:46.579973+0000","last_clean":"2024-05-22T23:00:46.579973+0000","last_became_active":"2024-05-22T22:59:51.962002+0000","last_became_peered":"2024-05-22T22:59:51.962002+0000","last_unstale":"2024-05-22T23:00:46.579973+0000","last_undegraded":"2024-05-22T23:00:46.579973+0000","last_fullsized":"2024-05-22T23:00:46.579973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:23:19.409418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.592248+0000","last_change":"2024-05-22T23:00:46.591734+0000","last_active":"2024-05-22T23:00:46.592248+0000","last_peered":"2024-05-22T23:00:46.592248+0000","last_clean":"2024-05-22T23:00:46.592248+0000","last_became_active":"2024-05-22T23:00:46.584075+0000","last_became_peered":"2024-05-22T23:00:46.584075+0000","last_unstale":"2024-05-22T23:00:46.592248+0000","last_undegraded":"2024-05-22T23:00:46.592248+0000","last_fullsized":"2024-05-22T23:00:46.592248+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:32:13.548197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.582162+0000","last_change":"2024-05-22T22:59:51.961600+0000","last_active":"2024-05-22T23:00:46.582162+0000","last_peered":"2024-05-22T23:00:46.582162+0000","last_clean":"2024-05-22T23:00:46.582162+0000","last_became_active":"2024-05-22T22:59:51.961286+0000","last_became_peered":"2024-05-22T22:59:51.961286+0000","last_unstale":"2024-05-22T23:00:46.582162+0000","last_undegraded":"2024-05-22T23:00:46.582162+0000","last_fullsized":"2024-05-22T23:00:46.582162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:04:26.992865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578974+0000","last_change":"2024-05-22T22:59:51.594461+0000","last_active":"2024-05-22T23:00:46.578974+0000","last_peered":"2024-05-22T23:00:46.578974+0000","last_clean":"2024-05-22T23:00:46.578974+0000","last_became_active":"2024-05-22T22:59:51.594128+0000","last_became_peered":"2024-05-22T22:59:51.594128+0000","last_unstale":"2024-05-22T23:00:46.578974+0000","last_undegraded":"2024-05-22T23:00:46.578974+0000","last_fullsized":"2024-05-22T23:00:46.578974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:52:05.095814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.577429+0000","last_change":"2024-05-22T22:59:51.958851+0000","last_active":"2024-05-22T23:00:46.577429+0000","last_peered":"2024-05-22T23:00:46.577429+0000","last_clean":"2024-05-22T23:00:46.577429+0000","last_became_active":"2024-05-22T22:59:51.957412+0000","last_became_peered":"2024-05-22T22:59:51.957412+0000","last_unstale":"2024-05-22T23:00:46.577429+0000","last_undegraded":"2024-05-22T23:00:46.577429+0000","last_fullsized":"2024-05-22T23:00:46.577429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:13:02.761205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.575778+0000","last_change":"2024-05-22T22:59:51.588750+0000","last_active":"2024-05-22T23:00:46.575778+0000","last_peered":"2024-05-22T23:00:46.575778+0000","last_clean":"2024-05-22T23:00:46.575778+0000","last_became_active":"2024-05-22T22:59:51.588364+0000","last_became_peered":"2024-05-22T22:59:51.588364+0000","last_unstale":"2024-05-22T23:00:46.575778+0000","last_undegraded":"2024-05-22T23:00:46.575778+0000","last_fullsized":"2024-05-22T23:00:46.575778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:28:41.605406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.576921+0000","last_change":"2024-05-22T22:59:51.964676+0000","last_active":"2024-05-22T23:00:46.576921+0000","last_peered":"2024-05-22T23:00:46.576921+0000","last_clean":"2024-05-22T23:00:46.576921+0000","last_became_active":"2024-05-22T22:59:51.960472+0000","last_became_peered":"2024-05-22T22:59:51.960472+0000","last_unstale":"2024-05-22T23:00:46.576921+0000","last_undegraded":"2024-05-22T23:00:46.576921+0000","last_fullsized":"2024-05-22T23:00:46.576921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:24:55.042637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580200+0000","last_change":"2024-05-22T22:59:51.600873+0000","last_active":"2024-05-22T23:00:46.580200+0000","last_peered":"2024-05-22T23:00:46.580200+0000","last_clean":"2024-05-22T23:00:46.580200+0000","last_became_active":"2024-05-22T22:59:51.599785+0000","last_became_peered":"2024-05-22T22:59:51.599785+0000","last_unstale":"2024-05-22T23:00:46.580200+0000","last_undegraded":"2024-05-22T23:00:46.580200+0000","last_fullsized":"2024-05-22T23:00:46.580200+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:20:09.590765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.577178+0000","last_change":"2024-05-22T22:59:51.964942+0000","last_active":"2024-05-22T23:00:46.577178+0000","last_peered":"2024-05-22T23:00:46.577178+0000","last_clean":"2024-05-22T23:00:46.577178+0000","last_became_active":"2024-05-22T22:59:51.964333+0000","last_became_peered":"2024-05-22T22:59:51.964333+0000","last_unstale":"2024-05-22T23:00:46.577178+0000","last_undegraded":"2024-05-22T23:00:46.577178+0000","last_fullsized":"2024-05-22T23:00:46.577178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:01:07.215075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.582908+0000","last_change":"2024-05-22T22:59:51.957291+0000","last_active":"2024-05-22T23:00:46.582908+0000","last_peered":"2024-05-22T23:00:46.582908+0000","last_clean":"2024-05-22T23:00:46.582908+0000","last_became_active":"2024-05-22T22:59:51.956987+0000","last_became_peered":"2024-05-22T22:59:51.956987+0000","last_unstale":"2024-05-22T23:00:46.582908+0000","last_undegraded":"2024-05-22T23:00:46.582908+0000","last_fullsized":"2024-05-22T23:00:46.582908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:14:10.420756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578254+0000","last_change":"2024-05-22T22:59:51.590255+0000","last_active":"2024-05-22T23:00:46.578254+0000","last_peered":"2024-05-22T23:00:46.578254+0000","last_clean":"2024-05-22T23:00:46.578254+0000","last_became_active":"2024-05-22T22:59:51.589261+0000","last_became_peered":"2024-05-22T22:59:51.589261+0000","last_unstale":"2024-05-22T23:00:46.578254+0000","last_undegraded":"2024-05-22T23:00:46.578254+0000","last_fullsized":"2024-05-22T23:00:46.578254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:15:14.171311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.873140+0000","last_change":"2024-05-22T22:59:51.961453+0000","last_active":"2024-05-22T23:00:46.873140+0000","last_peered":"2024-05-22T23:00:46.873140+0000","last_clean":"2024-05-22T23:00:46.873140+0000","last_became_active":"2024-05-22T22:59:51.961082+0000","last_became_peered":"2024-05-22T22:59:51.961082+0000","last_unstale":"2024-05-22T23:00:46.873140+0000","last_undegraded":"2024-05-22T23:00:46.873140+0000","last_fullsized":"2024-05-22T23:00:46.873140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:05:50.310688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.872913+0000","last_change":"2024-05-22T22:59:51.595913+0000","last_active":"2024-05-22T23:00:46.872913+0000","last_peered":"2024-05-22T23:00:46.872913+0000","last_clean":"2024-05-22T23:00:46.872913+0000","last_became_active":"2024-05-22T22:59:51.595561+0000","last_became_peered":"2024-05-22T22:59:51.595561+0000","last_unstale":"2024-05-22T23:00:46.872913+0000","last_undegraded":"2024-05-22T23:00:46.872913+0000","last_fullsized":"2024-05-22T23:00:46.872913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:37:04.701575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":354,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580190+0000","last_change":"2024-05-22T22:58:34.927654+0000","last_active":"2024-05-22T23:00:46.580190+0000","last_peered":"2024-05-22T23:00:46.580190+0000","last_clean":"2024-05-22T23:00:46.580190+0000","last_became_active":"2024-05-22T22:58:34.927090+0000","last_became_peered":"2024-05-22T22:58:34.927090+0000","last_unstale":"2024-05-22T23:00:46.580190+0000","last_undegraded":"2024-05-22T23:00:46.580190+0000","last_fullsized":"2024-05-22T23:00:46.580190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:55:07.719852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:55:07.719852+0000","last_clean_scrub_stamp":"2024-05-22T22:55:07.719852+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:40:53.763146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578233+0000","last_change":"2024-05-22T22:59:51.956422+0000","last_active":"2024-05-22T23:00:46.578233+0000","last_peered":"2024-05-22T23:00:46.578233+0000","last_clean":"2024-05-22T23:00:46.578233+0000","last_became_active":"2024-05-22T22:59:51.956127+0000","last_became_peered":"2024-05-22T22:59:51.956127+0000","last_unstale":"2024-05-22T23:00:46.578233+0000","last_undegraded":"2024-05-22T23:00:46.578233+0000","last_fullsized":"2024-05-22T23:00:46.578233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:04:36.955644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.874231+0000","last_change":"2024-05-22T23:00:46.874231+0000","last_active":"2024-05-22T23:00:46.874231+0000","last_peered":"2024-05-22T23:00:46.874231+0000","last_clean":"2024-05-22T23:00:46.874231+0000","last_became_active":"2024-05-22T23:00:46.873728+0000","last_became_peered":"2024-05-22T23:00:46.873728+0000","last_unstale":"2024-05-22T23:00:46.874231+0000","last_undegraded":"2024-05-22T23:00:46.874231+0000","last_fullsized":"2024-05-22T23:00:46.874231+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:49:12.372179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.579460+0000","last_change":"2024-05-22T22:59:51.598772+0000","last_active":"2024-05-22T23:00:46.579460+0000","last_peered":"2024-05-22T23:00:46.579460+0000","last_clean":"2024-05-22T23:00:46.579460+0000","last_became_active":"2024-05-22T22:59:51.598192+0000","last_became_peered":"2024-05-22T22:59:51.598192+0000","last_unstale":"2024-05-22T23:00:46.579460+0000","last_undegraded":"2024-05-22T23:00:46.579460+0000","last_fullsized":"2024-05-22T23:00:46.579460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:42:47.069554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578768+0000","last_change":"2024-05-22T22:59:51.942337+0000","last_active":"2024-05-22T23:00:46.578768+0000","last_peered":"2024-05-22T23:00:46.578768+0000","last_clean":"2024-05-22T23:00:46.578768+0000","last_became_active":"2024-05-22T22:59:51.941939+0000","last_became_peered":"2024-05-22T22:59:51.941939+0000","last_unstale":"2024-05-22T23:00:46.578768+0000","last_undegraded":"2024-05-22T23:00:46.578768+0000","last_fullsized":"2024-05-22T23:00:46.578768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:35:42.360128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.576300+0000","last_change":"2024-05-22T22:59:51.589640+0000","last_active":"2024-05-22T23:00:46.576300+0000","last_peered":"2024-05-22T23:00:46.576300+0000","last_clean":"2024-05-22T23:00:46.576300+0000","last_became_active":"2024-05-22T22:59:51.589310+0000","last_became_peered":"2024-05-22T22:59:51.589310+0000","last_unstale":"2024-05-22T23:00:46.576300+0000","last_undegraded":"2024-05-22T23:00:46.576300+0000","last_fullsized":"2024-05-22T23:00:46.576300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:54:39.182002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.872152+0000","last_change":"2024-05-22T22:59:51.952423+0000","last_active":"2024-05-22T23:00:46.872152+0000","last_peered":"2024-05-22T23:00:46.872152+0000","last_clean":"2024-05-22T23:00:46.872152+0000","last_became_active":"2024-05-22T22:59:51.952015+0000","last_became_peered":"2024-05-22T22:59:51.952015+0000","last_unstale":"2024-05-22T23:00:46.872152+0000","last_undegraded":"2024-05-22T23:00:46.872152+0000","last_fullsized":"2024-05-22T23:00:46.872152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:06:23.705248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.872910+0000","last_change":"2024-05-22T22:59:51.594882+0000","last_active":"2024-05-22T23:00:46.872910+0000","last_peered":"2024-05-22T23:00:46.872910+0000","last_clean":"2024-05-22T23:00:46.872910+0000","last_became_active":"2024-05-22T22:59:51.594552+0000","last_became_peered":"2024-05-22T22:59:51.594552+0000","last_unstale":"2024-05-22T23:00:46.872910+0000","last_undegraded":"2024-05-22T23:00:46.872910+0000","last_fullsized":"2024-05-22T23:00:46.872910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:06:40.771969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580806+0000","last_change":"2024-05-22T22:59:51.956534+0000","last_active":"2024-05-22T23:00:46.580806+0000","last_peered":"2024-05-22T23:00:46.580806+0000","last_clean":"2024-05-22T23:00:46.580806+0000","last_became_active":"2024-05-22T22:59:51.956037+0000","last_became_peered":"2024-05-22T22:59:51.956037+0000","last_unstale":"2024-05-22T23:00:46.580806+0000","last_undegraded":"2024-05-22T23:00:46.580806+0000","last_fullsized":"2024-05-22T23:00:46.580806+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:24:15.289570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.579471+0000","last_change":"2024-05-22T22:59:51.592916+0000","last_active":"2024-05-22T23:00:46.579471+0000","last_peered":"2024-05-22T23:00:46.579471+0000","last_clean":"2024-05-22T23:00:46.579471+0000","last_became_active":"2024-05-22T22:59:51.592530+0000","last_became_peered":"2024-05-22T22:59:51.592530+0000","last_unstale":"2024-05-22T23:00:46.579471+0000","last_undegraded":"2024-05-22T23:00:46.579471+0000","last_fullsized":"2024-05-22T23:00:46.579471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:56:18.660120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578007+0000","last_change":"2024-05-22T22:59:51.956188+0000","last_active":"2024-05-22T23:00:46.578007+0000","last_peered":"2024-05-22T23:00:46.578007+0000","last_clean":"2024-05-22T23:00:46.578007+0000","last_became_active":"2024-05-22T22:59:51.955844+0000","last_became_peered":"2024-05-22T22:59:51.955844+0000","last_unstale":"2024-05-22T23:00:46.578007+0000","last_undegraded":"2024-05-22T23:00:46.578007+0000","last_fullsized":"2024-05-22T23:00:46.578007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:09:12.848822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.872231+0000","last_change":"2024-05-22T22:59:51.590641+0000","last_active":"2024-05-22T23:00:46.872231+0000","last_peered":"2024-05-22T23:00:46.872231+0000","last_clean":"2024-05-22T23:00:46.872231+0000","last_became_active":"2024-05-22T22:59:51.590321+0000","last_became_peered":"2024-05-22T22:59:51.590321+0000","last_unstale":"2024-05-22T23:00:46.872231+0000","last_undegraded":"2024-05-22T23:00:46.872231+0000","last_fullsized":"2024-05-22T23:00:46.872231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:46:15.157160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.576955+0000","last_change":"2024-05-22T22:59:51.950256+0000","last_active":"2024-05-22T23:00:46.576955+0000","last_peered":"2024-05-22T23:00:46.576955+0000","last_clean":"2024-05-22T23:00:46.576955+0000","last_became_active":"2024-05-22T22:59:51.949746+0000","last_became_peered":"2024-05-22T22:59:51.949746+0000","last_unstale":"2024-05-22T23:00:46.576955+0000","last_undegraded":"2024-05-22T23:00:46.576955+0000","last_fullsized":"2024-05-22T23:00:46.576955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:33:12.146425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.874879+0000","last_change":"2024-05-22T23:00:46.874312+0000","last_active":"2024-05-22T23:00:46.874879+0000","last_peered":"2024-05-22T23:00:46.874879+0000","last_clean":"2024-05-22T23:00:46.874879+0000","last_became_active":"2024-05-22T23:00:46.873803+0000","last_became_peered":"2024-05-22T23:00:46.873803+0000","last_unstale":"2024-05-22T23:00:46.874879+0000","last_undegraded":"2024-05-22T23:00:46.874879+0000","last_fullsized":"2024-05-22T23:00:46.874879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:44:48.915692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.872161+0000","last_change":"2024-05-22T22:59:51.968432+0000","last_active":"2024-05-22T23:00:46.872161+0000","last_peered":"2024-05-22T23:00:46.872161+0000","last_clean":"2024-05-22T23:00:46.872161+0000","last_became_active":"2024-05-22T22:59:51.968077+0000","last_became_peered":"2024-05-22T22:59:51.968077+0000","last_unstale":"2024-05-22T23:00:46.872161+0000","last_undegraded":"2024-05-22T23:00:46.872161+0000","last_fullsized":"2024-05-22T23:00:46.872161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:08:16.281671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578210+0000","last_change":"2024-05-22T22:59:51.601616+0000","last_active":"2024-05-22T23:00:46.578210+0000","last_peered":"2024-05-22T23:00:46.578210+0000","last_clean":"2024-05-22T23:00:46.578210+0000","last_became_active":"2024-05-22T22:59:51.601249+0000","last_became_peered":"2024-05-22T22:59:51.601249+0000","last_unstale":"2024-05-22T23:00:46.578210+0000","last_undegraded":"2024-05-22T23:00:46.578210+0000","last_fullsized":"2024-05-22T23:00:46.578210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:07:07.030142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580327+0000","last_change":"2024-05-22T22:59:51.951578+0000","last_active":"2024-05-22T23:00:46.580327+0000","last_peered":"2024-05-22T23:00:46.580327+0000","last_clean":"2024-05-22T23:00:46.580327+0000","last_became_active":"2024-05-22T22:59:51.951233+0000","last_became_peered":"2024-05-22T22:59:51.951233+0000","last_unstale":"2024-05-22T23:00:46.580327+0000","last_undegraded":"2024-05-22T23:00:46.580327+0000","last_fullsized":"2024-05-22T23:00:46.580327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:25.882943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580890+0000","last_change":"2024-05-22T22:59:51.597819+0000","last_active":"2024-05-22T23:00:46.580890+0000","last_peered":"2024-05-22T23:00:46.580890+0000","last_clean":"2024-05-22T23:00:46.580890+0000","last_became_active":"2024-05-22T22:59:51.597262+0000","last_became_peered":"2024-05-22T22:59:51.597262+0000","last_unstale":"2024-05-22T23:00:46.580890+0000","last_undegraded":"2024-05-22T23:00:46.580890+0000","last_fullsized":"2024-05-22T23:00:46.580890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:05:02.849623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580598+0000","last_change":"2024-05-22T22:59:51.951668+0000","last_active":"2024-05-22T23:00:46.580598+0000","last_peered":"2024-05-22T23:00:46.580598+0000","last_clean":"2024-05-22T23:00:46.580598+0000","last_became_active":"2024-05-22T22:59:51.951176+0000","last_became_peered":"2024-05-22T22:59:51.951176+0000","last_unstale":"2024-05-22T23:00:46.580598+0000","last_undegraded":"2024-05-22T23:00:46.580598+0000","last_fullsized":"2024-05-22T23:00:46.580598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:01:46.010858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.582480+0000","last_change":"2024-05-22T22:59:51.593144+0000","last_active":"2024-05-22T23:00:46.582480+0000","last_peered":"2024-05-22T23:00:46.582480+0000","last_clean":"2024-05-22T23:00:46.582480+0000","last_became_active":"2024-05-22T22:59:51.592557+0000","last_became_peered":"2024-05-22T22:59:51.592557+0000","last_unstale":"2024-05-22T23:00:46.582480+0000","last_undegraded":"2024-05-22T23:00:46.582480+0000","last_fullsized":"2024-05-22T23:00:46.582480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:28:19.403419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580452+0000","last_change":"2024-05-22T22:59:51.958866+0000","last_active":"2024-05-22T23:00:46.580452+0000","last_peered":"2024-05-22T23:00:46.580452+0000","last_clean":"2024-05-22T23:00:46.580452+0000","last_became_active":"2024-05-22T22:59:51.958505+0000","last_became_peered":"2024-05-22T22:59:51.958505+0000","last_unstale":"2024-05-22T23:00:46.580452+0000","last_undegraded":"2024-05-22T23:00:46.580452+0000","last_fullsized":"2024-05-22T23:00:46.580452+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:54:24.601966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578823+0000","last_change":"2024-05-22T22:59:51.587391+0000","last_active":"2024-05-22T23:00:46.578823+0000","last_peered":"2024-05-22T23:00:46.578823+0000","last_clean":"2024-05-22T23:00:46.578823+0000","last_became_active":"2024-05-22T22:59:51.587056+0000","last_became_peered":"2024-05-22T22:59:51.587056+0000","last_unstale":"2024-05-22T23:00:46.578823+0000","last_undegraded":"2024-05-22T23:00:46.578823+0000","last_fullsized":"2024-05-22T23:00:46.578823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:04:40.614853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580699+0000","last_change":"2024-05-22T22:59:51.953466+0000","last_active":"2024-05-22T23:00:46.580699+0000","last_peered":"2024-05-22T23:00:46.580699+0000","last_clean":"2024-05-22T23:00:46.580699+0000","last_became_active":"2024-05-22T22:59:51.952830+0000","last_became_peered":"2024-05-22T22:59:51.952830+0000","last_unstale":"2024-05-22T23:00:46.580699+0000","last_undegraded":"2024-05-22T23:00:46.580699+0000","last_fullsized":"2024-05-22T23:00:46.580699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:36:06.314680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.577075+0000","last_change":"2024-05-22T22:59:51.962993+0000","last_active":"2024-05-22T23:00:46.577075+0000","last_peered":"2024-05-22T23:00:46.577075+0000","last_clean":"2024-05-22T23:00:46.577075+0000","last_became_active":"2024-05-22T22:59:51.962591+0000","last_became_peered":"2024-05-22T22:59:51.962591+0000","last_unstale":"2024-05-22T23:00:46.577075+0000","last_undegraded":"2024-05-22T23:00:46.577075+0000","last_fullsized":"2024-05-22T23:00:46.577075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:55:50.902537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580625+0000","last_change":"2024-05-22T22:59:51.956304+0000","last_active":"2024-05-22T23:00:46.580625+0000","last_peered":"2024-05-22T23:00:46.580625+0000","last_clean":"2024-05-22T23:00:46.580625+0000","last_became_active":"2024-05-22T22:59:51.955933+0000","last_became_peered":"2024-05-22T22:59:51.955933+0000","last_unstale":"2024-05-22T23:00:46.580625+0000","last_undegraded":"2024-05-22T23:00:46.580625+0000","last_fullsized":"2024-05-22T23:00:46.580625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:28:17.225851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580619+0000","last_change":"2024-05-22T22:59:51.946806+0000","last_active":"2024-05-22T23:00:46.580619+0000","last_peered":"2024-05-22T23:00:46.580619+0000","last_clean":"2024-05-22T23:00:46.580619+0000","last_became_active":"2024-05-22T22:59:51.946433+0000","last_became_peered":"2024-05-22T22:59:51.946433+0000","last_unstale":"2024-05-22T23:00:46.580619+0000","last_undegraded":"2024-05-22T23:00:46.580619+0000","last_fullsized":"2024-05-22T23:00:46.580619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:24:06.013688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.580610+0000","last_change":"2024-05-22T22:59:51.956466+0000","last_active":"2024-05-22T23:00:46.580610+0000","last_peered":"2024-05-22T23:00:46.580610+0000","last_clean":"2024-05-22T23:00:46.580610+0000","last_became_active":"2024-05-22T22:59:51.956144+0000","last_became_peered":"2024-05-22T22:59:51.956144+0000","last_unstale":"2024-05-22T23:00:46.580610+0000","last_undegraded":"2024-05-22T23:00:46.580610+0000","last_fullsized":"2024-05-22T23:00:46.580610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:08:28.674824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.582147+0000","last_change":"2024-05-22T23:00:46.582147+0000","last_active":"2024-05-22T23:00:46.582147+0000","last_peered":"2024-05-22T23:00:46.582147+0000","last_clean":"2024-05-22T23:00:46.582147+0000","last_became_active":"2024-05-22T23:00:46.581804+0000","last_became_peered":"2024-05-22T23:00:46.581804+0000","last_unstale":"2024-05-22T23:00:46.582147+0000","last_undegraded":"2024-05-22T23:00:46.582147+0000","last_fullsized":"2024-05-22T23:00:46.582147+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:02:15.933895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.577417+0000","last_change":"2024-05-22T22:59:51.961971+0000","last_active":"2024-05-22T23:00:46.577417+0000","last_peered":"2024-05-22T23:00:46.577417+0000","last_clean":"2024-05-22T23:00:46.577417+0000","last_became_active":"2024-05-22T22:59:51.961604+0000","last_became_peered":"2024-05-22T22:59:51.961604+0000","last_unstale":"2024-05-22T23:00:46.577417+0000","last_undegraded":"2024-05-22T23:00:46.577417+0000","last_fullsized":"2024-05-22T23:00:46.577417+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:32:14.669631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.581792+0000","last_change":"2024-05-22T22:59:51.948173+0000","last_active":"2024-05-22T23:00:46.581792+0000","last_peered":"2024-05-22T23:00:46.581792+0000","last_clean":"2024-05-22T23:00:46.581792+0000","last_became_active":"2024-05-22T22:59:51.947839+0000","last_became_peered":"2024-05-22T22:59:51.947839+0000","last_unstale":"2024-05-22T23:00:46.581792+0000","last_undegraded":"2024-05-22T23:00:46.581792+0000","last_fullsized":"2024-05-22T23:00:46.581792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:55:07.748445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.872035+0000","last_change":"2024-05-22T22:59:51.955627+0000","last_active":"2024-05-22T23:00:46.872035+0000","last_peered":"2024-05-22T23:00:46.872035+0000","last_clean":"2024-05-22T23:00:46.872035+0000","last_became_active":"2024-05-22T22:59:51.955284+0000","last_became_peered":"2024-05-22T22:59:51.955284+0000","last_unstale":"2024-05-22T23:00:46.872035+0000","last_undegraded":"2024-05-22T23:00:46.872035+0000","last_fullsized":"2024-05-22T23:00:46.872035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:57:13.029317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.582786+0000","last_change":"2024-05-22T22:59:51.955889+0000","last_active":"2024-05-22T23:00:46.582786+0000","last_peered":"2024-05-22T23:00:46.582786+0000","last_clean":"2024-05-22T23:00:46.582786+0000","last_became_active":"2024-05-22T22:59:51.954930+0000","last_became_peered":"2024-05-22T22:59:51.954930+0000","last_unstale":"2024-05-22T23:00:46.582786+0000","last_undegraded":"2024-05-22T23:00:46.582786+0000","last_fullsized":"2024-05-22T23:00:46.582786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:47.536072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.872113+0000","last_change":"2024-05-22T22:59:51.954148+0000","last_active":"2024-05-22T23:00:46.872113+0000","last_peered":"2024-05-22T23:00:46.872113+0000","last_clean":"2024-05-22T23:00:46.872113+0000","last_became_active":"2024-05-22T22:59:51.952336+0000","last_became_peered":"2024-05-22T22:59:51.952336+0000","last_unstale":"2024-05-22T23:00:46.872113+0000","last_undegraded":"2024-05-22T23:00:46.872113+0000","last_fullsized":"2024-05-22T23:00:46.872113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:07:27.928910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.579454+0000","last_change":"2024-05-22T22:59:51.954300+0000","last_active":"2024-05-22T23:00:46.579454+0000","last_peered":"2024-05-22T23:00:46.579454+0000","last_clean":"2024-05-22T23:00:46.579454+0000","last_became_active":"2024-05-22T22:59:51.953779+0000","last_became_peered":"2024-05-22T22:59:51.953779+0000","last_unstale":"2024-05-22T23:00:46.579454+0000","last_undegraded":"2024-05-22T23:00:46.579454+0000","last_fullsized":"2024-05-22T23:00:46.579454+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:24:59.363634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.578618+0000","last_change":"2024-05-22T22:59:51.947860+0000","last_active":"2024-05-22T23:00:46.578618+0000","last_peered":"2024-05-22T23:00:46.578618+0000","last_clean":"2024-05-22T23:00:46.578618+0000","last_became_active":"2024-05-22T22:59:51.947420+0000","last_became_peered":"2024-05-22T22:59:51.947420+0000","last_unstale":"2024-05-22T23:00:46.578618+0000","last_undegraded":"2024-05-22T23:00:46.578618+0000","last_fullsized":"2024-05-22T23:00:46.578618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:20:40.629285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-22T23:00:46.576982+0000","last_change":"2024-05-22T22:59:51.957867+0000","last_active":"2024-05-22T23:00:46.576982+0000","last_peered":"2024-05-22T23:00:46.576982+0000","last_clean":"2024-05-22T23:00:46.576982+0000","last_became_active":"2024-05-22T22:59:51.957522+0000","last_became_peered":"2024-05-22T22:59:51.957522+0000","last_unstale":"2024-05-22T23:00:46.576982+0000","last_undegraded":"2024-05-22T23:00:46.576982+0000","last_fullsized":"2024-05-22T23:00:46.576982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:41:18.631970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280555,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59709,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":144,"num_read_kb":132,"num_write":72,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":58911,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":217088,"data_stored":179127,"data_compressed":13249,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":0,"internal_metadata":0},"log_size":61,"ondisk_log_size":61,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351006,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":461163,"data_compressed":5755,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612643,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":1110753,"data_compressed":43296,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776168,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":461745,"data_compressed":5755,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037805,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":461767,"data_compressed":5755,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299443,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":1052230,"data_compressed":41439,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":461280,"data_compressed":5755,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1015808,"data_stored":520074,"data_compressed":7612,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":954368,"data_stored":461253,"data_compressed":5755,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":461197,"data_compressed":5755,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182470,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":966656,"data_stored":461501,"data_compressed":5755,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":520096,"data_compressed":15290,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757309,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1015808,"data_stored":1051830,"data_compressed":41439,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34045,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":58911,"data_compressed":1857,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":58933,"data_compressed":9535,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":59105,"data_compressed":1857,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-22T23:01:10.796 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-22T23:01:10.796 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-22T23:01:10.796 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-22T23:01:10.796 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph health --format=json 2024-05-22T23:01:10.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:10 smithi204 ceph-mon[36211]: pgmap v330: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:01:11.037 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:01:11.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:10 smithi096 ceph-mon[36392]: pgmap v330: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:01:11.829 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:11 smithi084 ceph-mon[32186]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-22T23:01:11.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:11 smithi204 ceph-mon[36211]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-22T23:01:12.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:11 smithi096 ceph-mon[36392]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-22T23:01:12.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.625+0000 7f3908b2a700 1 -- 172.21.15.84:0/1835602666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39041072d0 msgr2=0x7f39041076b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:12.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.625+0000 7f3908b2a700 1 --2- 172.21.15.84:0/1835602666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39041072d0 0x7f39041076b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009990 tx=0x7f38f4009ca0 comp rx=0 tx=0).stop 2024-05-22T23:01:12.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.625+0000 7f3908b2a700 1 -- 172.21.15.84:0/1835602666 shutdown_connections 2024-05-22T23:01:12.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.625+0000 7f3908b2a700 1 --2- 172.21.15.84:0/1835602666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3904101c20 0x7f390410e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:12.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.625+0000 7f3908b2a700 1 --2- 172.21.15.84:0/1835602666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39041072d0 0x7f39041076b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:12.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.625+0000 7f3908b2a700 1 --2- 172.21.15.84:0/1835602666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904101260 0x7f39041016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:12.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.625+0000 7f3908b2a700 1 -- 172.21.15.84:0/1835602666 >> 172.21.15.84:0/1835602666 conn(0x7f3904078210 msgr2=0x7f3904078620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:12.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.625+0000 7f3908b2a700 1 -- 172.21.15.84:0/1835602666 shutdown_connections 2024-05-22T23:01:12.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.625+0000 7f3908b2a700 1 -- 172.21.15.84:0/1835602666 wait complete. 2024-05-22T23:01:12.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.626+0000 7f3908b2a700 1 Processor -- start 2024-05-22T23:01:12.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.626+0000 7f3908b2a700 1 -- start start 2024-05-22T23:01:12.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.626+0000 7f3908b2a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904101260 0x7f390419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:12.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.626+0000 7f3908b2a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3904101c20 0x7f390419db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:12.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.626+0000 7f3908b2a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39041072d0 0x7f39041977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:12.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.626+0000 7f3908b2a700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3904111540 con 0x7f39041072d0 2024-05-22T23:01:12.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.626+0000 7f3908b2a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f39041113c0 con 0x7f3904101260 2024-05-22T23:01:12.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.626+0000 7f3908b2a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f39041116c0 con 0x7f3904101c20 2024-05-22T23:01:12.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.626+0000 7f3902d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39041072d0 0x7f39041977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:12.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.626+0000 7f3902d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39041072d0 0x7f39041977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52476/0 (socket says 172.21.15.84:52476) 2024-05-22T23:01:12.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.626+0000 7f3902d9d700 1 -- 172.21.15.84:0/663440903 learned_addr learned my addr 172.21.15.84:0/663440903 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:12.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.626+0000 7f390259c700 1 --2- 172.21.15.84:0/663440903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904101260 0x7f390419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:12.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.626+0000 7f38fbfff700 1 --2- 172.21.15.84:0/663440903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3904101c20 0x7f390419db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:12.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.627+0000 7f3902d9d700 1 -- 172.21.15.84:0/663440903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3904101c20 msgr2=0x7f390419db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:12.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.627+0000 7f3902d9d700 1 --2- 172.21.15.84:0/663440903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3904101c20 0x7f390419db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:12.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.627+0000 7f3902d9d700 1 -- 172.21.15.84:0/663440903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904101260 msgr2=0x7f390419d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:12.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.627+0000 7f3902d9d700 1 --2- 172.21.15.84:0/663440903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904101260 0x7f390419d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:12.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.627+0000 7f3902d9d700 1 -- 172.21.15.84:0/663440903 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3904197fc0 con 0x7f39041072d0 2024-05-22T23:01:12.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.627+0000 7f390259c700 1 --2- 172.21.15.84:0/663440903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904101260 0x7f390419d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:01:12.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.627+0000 7f3902d9d700 1 --2- 172.21.15.84:0/663440903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39041072d0 0x7f39041977e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f38fc007d70 tx=0x7f38fc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:12.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.627+0000 7f38fb7fe700 1 -- 172.21.15.84:0/663440903 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc016020 con 0x7f39041072d0 2024-05-22T23:01:12.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.627+0000 7f3908b2a700 1 -- 172.21.15.84:0/663440903 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39041982b0 con 0x7f39041072d0 2024-05-22T23:01:12.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.627+0000 7f38fb7fe700 1 -- 172.21.15.84:0/663440903 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38fc00ad40 con 0x7f39041072d0 2024-05-22T23:01:12.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.627+0000 7f38fb7fe700 1 -- 172.21.15.84:0/663440903 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc0116f0 con 0x7f39041072d0 2024-05-22T23:01:12.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.627+0000 7f3908b2a700 1 -- 172.21.15.84:0/663440903 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39041a4470 con 0x7f39041072d0 2024-05-22T23:01:12.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.628+0000 7f3908b2a700 1 -- 172.21.15.84:0/663440903 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3904102980 con 0x7f39041072d0 2024-05-22T23:01:12.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.629+0000 7f38fb7fe700 1 -- 172.21.15.84:0/663440903 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f38fc019020 con 0x7f39041072d0 2024-05-22T23:01:12.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.630+0000 7f38fb7fe700 1 --2- 172.21.15.84:0/663440903 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f38f00a2cb0 0x7f38f00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:12.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.630+0000 7f390259c700 1 --2- 172.21.15.84:0/663440903 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f38f00a2cb0 0x7f38f00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:12.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.630+0000 7f38fb7fe700 1 -- 172.21.15.84:0/663440903 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f38fc0cf4c0 con 0x7f39041072d0 2024-05-22T23:01:12.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.630+0000 7f390259c700 1 --2- 172.21.15.84:0/663440903 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f38f00a2cb0 0x7f38f00a5170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f38f4006010 tx=0x7f38f400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:12.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.633+0000 7f38fb7fe700 1 -- 172.21.15.84:0/663440903 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f38fc020020 con 0x7f39041072d0 2024-05-22T23:01:12.874 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:12 smithi084 ceph-mon[32186]: pgmap v331: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:01:12.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.875+0000 7f3908b2a700 1 -- 172.21.15.84:0/663440903 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f390402dbc0 con 0x7f39041072d0 2024-05-22T23:01:12.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.875+0000 7f38fb7fe700 1 -- 172.21.15.84:0/663440903 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f38fc022300 con 0x7f39041072d0 2024-05-22T23:01:12.877 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:12.877 INFO:teuthology.orchestra.run.smithi084.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-22T23:01:12.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.877+0000 7f3908b2a700 1 -- 172.21.15.84:0/663440903 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f38f00a2cb0 msgr2=0x7f38f00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:12.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.877+0000 7f3908b2a700 1 --2- 172.21.15.84:0/663440903 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f38f00a2cb0 0x7f38f00a5170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f38f4006010 tx=0x7f38f400b540 comp rx=0 tx=0).stop 2024-05-22T23:01:12.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.877+0000 7f3908b2a700 1 -- 172.21.15.84:0/663440903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39041072d0 msgr2=0x7f39041977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:12.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.877+0000 7f3908b2a700 1 --2- 172.21.15.84:0/663440903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39041072d0 0x7f39041977e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f38fc007d70 tx=0x7f38fc00a340 comp rx=0 tx=0).stop 2024-05-22T23:01:12.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.877+0000 7f3908b2a700 1 -- 172.21.15.84:0/663440903 shutdown_connections 2024-05-22T23:01:12.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.877+0000 7f3908b2a700 1 --2- 172.21.15.84:0/663440903 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f38f00a2cb0 0x7f38f00a5170 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:12.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.877+0000 7f3908b2a700 1 --2- 172.21.15.84:0/663440903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39041072d0 0x7f39041977e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:12.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.877+0000 7f3908b2a700 1 --2- 172.21.15.84:0/663440903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904101260 0x7f390419d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:12.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.877+0000 7f3908b2a700 1 --2- 172.21.15.84:0/663440903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3904101c20 0x7f390419db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:12.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.877+0000 7f3908b2a700 1 -- 172.21.15.84:0/663440903 >> 172.21.15.84:0/663440903 conn(0x7f3904078210 msgr2=0x7f39040ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:12.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.877+0000 7f3908b2a700 1 -- 172.21.15.84:0/663440903 shutdown_connections 2024-05-22T23:01:12.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:12.877+0000 7f3908b2a700 1 -- 172.21.15.84:0/663440903 wait complete. 2024-05-22T23:01:12.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:12 smithi204 ceph-mon[36211]: pgmap v331: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:01:13.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:12 smithi096 ceph-mon[36392]: pgmap v331: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:01:13.491 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-22T23:01:13.492 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T23:01:13.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.595+0000 7f411c3f9640 1 -- 172.21.15.84:0/783236745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41141010d0 msgr2=0x7f4114101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:13.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.595+0000 7f411c3f9640 1 --2- 172.21.15.84:0/783236745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41141010d0 0x7f4114101550 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4108009900 tx=0x7f410802a410 comp rx=0 tx=0).stop 2024-05-22T23:01:13.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.595+0000 7f411c3f9640 1 -- 172.21.15.84:0/783236745 shutdown_connections 2024-05-22T23:01:13.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.595+0000 7f411c3f9640 1 --2- 172.21.15.84:0/783236745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4114101a90 0x7f41141083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:13.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.595+0000 7f411c3f9640 1 --2- 172.21.15.84:0/783236745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41141010d0 0x7f4114101550 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:13.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.595+0000 7f411c3f9640 1 --2- 172.21.15.84:0/783236745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41140fff00 0x7f4114100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:13.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.595+0000 7f411c3f9640 1 -- 172.21.15.84:0/783236745 >> 172.21.15.84:0/783236745 conn(0x7f41140fb4e0 msgr2=0x7f41140fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:13.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.595+0000 7f411c3f9640 1 -- 172.21.15.84:0/783236745 shutdown_connections 2024-05-22T23:01:13.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.595+0000 7f411c3f9640 1 -- 172.21.15.84:0/783236745 wait complete. 2024-05-22T23:01:13.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.595+0000 7f411c3f9640 1 Processor -- start 2024-05-22T23:01:13.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.596+0000 7f411c3f9640 1 -- start start 2024-05-22T23:01:13.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.596+0000 7f411c3f9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41140fff00 0x7f4114196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:13.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.596+0000 7f411c3f9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41141010d0 0x7f4114197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:13.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.596+0000 7f411c3f9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4114101a90 0x7f411419e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:13.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.596+0000 7f411c3f9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f411410bc20 con 0x7f41140fff00 2024-05-22T23:01:13.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.596+0000 7f411c3f9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f411410baa0 con 0x7f4114101a90 2024-05-22T23:01:13.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.596+0000 7f411c3f9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f411410bda0 con 0x7f41141010d0 2024-05-22T23:01:13.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.596+0000 7f411a195640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41140fff00 0x7f4114196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:13.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.596+0000 7f411a195640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41140fff00 0x7f4114196ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52486/0 (socket says 172.21.15.84:52486) 2024-05-22T23:01:13.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.596+0000 7f411a195640 1 -- 172.21.15.84:0/2658251672 learned_addr learned my addr 172.21.15.84:0/2658251672 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:13.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.596+0000 7f411a195640 1 -- 172.21.15.84:0/2658251672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41141010d0 msgr2=0x7f4114197010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:13.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.596+0000 7f411a195640 1 --2- 172.21.15.84:0/2658251672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41141010d0 0x7f4114197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:13.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.596+0000 7f411a195640 1 -- 172.21.15.84:0/2658251672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4114101a90 msgr2=0x7f411419e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:13.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.596+0000 7f411a195640 1 --2- 172.21.15.84:0/2658251672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4114101a90 0x7f411419e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:13.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.596+0000 7f411a195640 1 -- 172.21.15.84:0/2658251672 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411419e6f0 con 0x7f41140fff00 2024-05-22T23:01:13.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.596+0000 7f411a195640 1 --2- 172.21.15.84:0/2658251672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41140fff00 0x7f4114196ad0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f410c00c970 tx=0x7f410c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:13.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.596+0000 7f41137fe640 1 -- 172.21.15.84:0/2658251672 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c007c10 con 0x7f41140fff00 2024-05-22T23:01:13.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.596+0000 7f411c3f9640 1 -- 172.21.15.84:0/2658251672 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f411419e9e0 con 0x7f41140fff00 2024-05-22T23:01:13.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.596+0000 7f41137fe640 1 -- 172.21.15.84:0/2658251672 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f410c007db0 con 0x7f41140fff00 2024-05-22T23:01:13.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.596+0000 7f41137fe640 1 -- 172.21.15.84:0/2658251672 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c00f8a0 con 0x7f41140fff00 2024-05-22T23:01:13.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.596+0000 7f411c3f9640 1 -- 172.21.15.84:0/2658251672 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f411419efb0 con 0x7f41140fff00 2024-05-22T23:01:13.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.597+0000 7f411c3f9640 1 -- 172.21.15.84:0/2658251672 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f411419f2a0 con 0x7f41140fff00 2024-05-22T23:01:13.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.598+0000 7f41137fe640 1 -- 172.21.15.84:0/2658251672 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f410c00fa60 con 0x7f41140fff00 2024-05-22T23:01:13.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.599+0000 7f41137fe640 1 --2- 172.21.15.84:0/2658251672 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f40fc0a2c20 0x7f40fc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:13.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.599+0000 7f4119994640 1 --2- 172.21.15.84:0/2658251672 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f40fc0a2c20 0x7f40fc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:13.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.599+0000 7f41137fe640 1 -- 172.21.15.84:0/2658251672 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f410c0cf5f0 con 0x7f41140fff00 2024-05-22T23:01:13.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.599+0000 7f4119994640 1 --2- 172.21.15.84:0/2658251672 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f40fc0a2c20 0x7f40fc0a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f41080096c0 tx=0x7f4108031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:13.605 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.604+0000 7f41137fe640 1 -- 172.21.15.84:0/2658251672 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f410c09c460 con 0x7f41140fff00 2024-05-22T23:01:13.780 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.779+0000 7f411c3f9640 1 -- 172.21.15.84:0/2658251672 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4114107620 con 0x7f41140fff00 2024-05-22T23:01:13.781 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.780+0000 7f41137fe640 1 -- 172.21.15.84:0/2658251672 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22722 (secure 0 0 0) 0x7f410c09bd60 con 0x7f41140fff00 2024-05-22T23:01:13.782 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:13.782 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":88,"fsid":"c1135a12-188d-11ef-bc9a-c7b262605968","created":"2024-05-22T22:51:49.878218+0000","modified":"2024-05-22T23:00:46.572591+0000","last_up_change":"2024-05-22T22:58:58.550278+0000","last_in_change":"2024-05-22T22:58:37.322139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T22:55:07.719394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T22:59:50.150511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T22:59:51.111182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762df065-fb4f-4c71-aab8-a62d2282c253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6803","nonce":1559945826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6805","nonce":1559945826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6809","nonce":1559945826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6807","nonce":1559945826}]},"public_addr":"172.21.15.84:6803/1559945826","cluster_addr":"172.21.15.84:6805/1559945826","heartbeat_back_addr":"172.21.15.84:6809/1559945826","heartbeat_front_addr":"172.21.15.84:6807/1559945826","state":["exists","up"]},{"osd":1,"uuid":"3fb2385d-6a34-49e3-88e1-116237beb346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6801","nonce":2996505353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6803","nonce":2996505353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6807","nonce":2996505353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6805","nonce":2996505353}]},"public_addr":"172.21.15.96:6801/2996505353","cluster_addr":"172.21.15.96:6803/2996505353","heartbeat_back_addr":"172.21.15.96:6807/2996505353","heartbeat_front_addr":"172.21.15.96:6805/2996505353","state":["exists","up"]},{"osd":2,"uuid":"e306e88e-a308-4585-ae11-c8af10b84038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6801","nonce":1499577897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6803","nonce":1499577897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6807","nonce":1499577897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6805","nonce":1499577897}]},"public_addr":"172.21.15.204:6801/1499577897","cluster_addr":"172.21.15.204:6803/1499577897","heartbeat_back_addr":"172.21.15.204:6807/1499577897","heartbeat_front_addr":"172.21.15.204:6805/1499577897","state":["exists","up"]},{"osd":3,"uuid":"d2a5fb97-6f60-4d45-9f18-447cae480974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6811","nonce":2869340648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6813","nonce":2869340648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6817","nonce":2869340648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6815","nonce":2869340648}]},"public_addr":"172.21.15.84:6811/2869340648","cluster_addr":"172.21.15.84:6813/2869340648","heartbeat_back_addr":"172.21.15.84:6817/2869340648","heartbeat_front_addr":"172.21.15.84:6815/2869340648","state":["exists","up"]},{"osd":4,"uuid":"4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6809","nonce":2554265648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6811","nonce":2554265648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6815","nonce":2554265648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6813","nonce":2554265648}]},"public_addr":"172.21.15.96:6809/2554265648","cluster_addr":"172.21.15.96:6811/2554265648","heartbeat_back_addr":"172.21.15.96:6815/2554265648","heartbeat_front_addr":"172.21.15.96:6813/2554265648","state":["exists","up"]},{"osd":5,"uuid":"7996bf5e-93a5-41b0-a9b1-99e7ef4514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6809","nonce":4075476986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6811","nonce":4075476986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6815","nonce":4075476986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6813","nonce":4075476986}]},"public_addr":"172.21.15.204:6809/4075476986","cluster_addr":"172.21.15.204:6811/4075476986","heartbeat_back_addr":"172.21.15.204:6815/4075476986","heartbeat_front_addr":"172.21.15.204:6813/4075476986","state":["exists","up"]},{"osd":6,"uuid":"f4b7c5de-1836-4d16-8603-3c517524d9fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6819","nonce":1806202865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6821","nonce":1806202865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6825","nonce":1806202865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6823","nonce":1806202865}]},"public_addr":"172.21.15.84:6819/1806202865","cluster_addr":"172.21.15.84:6821/1806202865","heartbeat_back_addr":"172.21.15.84:6825/1806202865","heartbeat_front_addr":"172.21.15.84:6823/1806202865","state":["exists","up"]},{"osd":7,"uuid":"69a18c43-5447-480d-9466-190830070c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6817","nonce":3195792774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6819","nonce":3195792774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6823","nonce":3195792774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6821","nonce":3195792774}]},"public_addr":"172.21.15.96:6817/3195792774","cluster_addr":"172.21.15.96:6819/3195792774","heartbeat_back_addr":"172.21.15.96:6823/3195792774","heartbeat_front_addr":"172.21.15.96:6821/3195792774","state":["exists","up"]},{"osd":8,"uuid":"2c5cadca-689a-441e-847c-ba370d400ef1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6817","nonce":606997927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6819","nonce":606997927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6823","nonce":606997927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6821","nonce":606997927}]},"public_addr":"172.21.15.204:6817/606997927","cluster_addr":"172.21.15.204:6819/606997927","heartbeat_back_addr":"172.21.15.204:6823/606997927","heartbeat_front_addr":"172.21.15.204:6821/606997927","state":["exists","up"]},{"osd":9,"uuid":"e3984fb2-7c00-4275-a10e-bca7c859f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6827","nonce":2786189012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6829","nonce":2786189012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6833","nonce":2786189012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6831","nonce":2786189012}]},"public_addr":"172.21.15.84:6827/2786189012","cluster_addr":"172.21.15.84:6829/2786189012","heartbeat_back_addr":"172.21.15.84:6833/2786189012","heartbeat_front_addr":"172.21.15.84:6831/2786189012","state":["exists","up"]},{"osd":10,"uuid":"b4938160-a808-4ded-91c2-d245c99fff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6825","nonce":1985398526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6827","nonce":1985398526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6831","nonce":1985398526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6829","nonce":1985398526}]},"public_addr":"172.21.15.96:6825/1985398526","cluster_addr":"172.21.15.96:6827/1985398526","heartbeat_back_addr":"172.21.15.96:6831/1985398526","heartbeat_front_addr":"172.21.15.96:6829/1985398526","state":["exists","up"]},{"osd":11,"uuid":"238922ef-d046-40cc-833d-25b7e7a523cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6825","nonce":3106116519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6827","nonce":3106116519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6831","nonce":3106116519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6829","nonce":3106116519}]},"public_addr":"172.21.15.204:6825/3106116519","cluster_addr":"172.21.15.204:6827/3106116519","heartbeat_back_addr":"172.21.15.204:6831/3106116519","heartbeat_front_addr":"172.21.15.204:6829/3106116519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:16.443480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:39.898133+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:03.664544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:28.838687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:53.829733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:19.269307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:45.166910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:10.771614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:36.678516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:03.150592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:29.327769+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:56.315938+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6801/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:6800/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/2766288608":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/950455999":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/4000289608":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1650467363":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1052746114":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6800/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6801/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/14610813":"2024-05-23T22:52:45.395791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T23:01:13.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.782+0000 7f411c3f9640 1 -- 172.21.15.84:0/2658251672 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f40fc0a2c20 msgr2=0x7f40fc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:13.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.782+0000 7f411c3f9640 1 --2- 172.21.15.84:0/2658251672 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f40fc0a2c20 0x7f40fc0a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f41080096c0 tx=0x7f4108031400 comp rx=0 tx=0).stop 2024-05-22T23:01:13.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.782+0000 7f411c3f9640 1 -- 172.21.15.84:0/2658251672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41140fff00 msgr2=0x7f4114196ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:13.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.782+0000 7f411c3f9640 1 --2- 172.21.15.84:0/2658251672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41140fff00 0x7f4114196ad0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f410c00c970 tx=0x7f410c00ce30 comp rx=0 tx=0).stop 2024-05-22T23:01:13.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.782+0000 7f411c3f9640 1 -- 172.21.15.84:0/2658251672 shutdown_connections 2024-05-22T23:01:13.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.782+0000 7f411c3f9640 1 --2- 172.21.15.84:0/2658251672 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f40fc0a2c20 0x7f40fc0a50e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:13.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.782+0000 7f411c3f9640 1 --2- 172.21.15.84:0/2658251672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4114101a90 0x7f411419e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:13.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.782+0000 7f411c3f9640 1 --2- 172.21.15.84:0/2658251672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41141010d0 0x7f4114197010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:13.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.782+0000 7f411c3f9640 1 --2- 172.21.15.84:0/2658251672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41140fff00 0x7f4114196ad0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:13.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.782+0000 7f411c3f9640 1 -- 172.21.15.84:0/2658251672 >> 172.21.15.84:0/2658251672 conn(0x7f41140fb4e0 msgr2=0x7f4114104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:13.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.782+0000 7f411c3f9640 1 -- 172.21.15.84:0/2658251672 shutdown_connections 2024-05-22T23:01:13.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.782+0000 7f411c3f9640 1 -- 172.21.15.84:0/2658251672 wait complete. 2024-05-22T23:01:13.794 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:13 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/663440903' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-22T23:01:13.795 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T22:55:07.719394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T22:59:50.150511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T22:59:51.111182+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T23:01:13.796 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T23:01:13.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.898+0000 7f8e57f63640 1 -- 172.21.15.84:0/2584147145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e501019c0 msgr2=0x7f8e50101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:13.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.898+0000 7f8e57f63640 1 --2- 172.21.15.84:0/2584147145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e501019c0 0x7f8e50101e40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8e40009900 tx=0x7f8e4002a410 comp rx=0 tx=0).stop 2024-05-22T23:01:13.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.898+0000 7f8e57f63640 1 -- 172.21.15.84:0/2584147145 shutdown_connections 2024-05-22T23:01:13.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.898+0000 7f8e57f63640 1 --2- 172.21.15.84:0/2584147145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e50102380 0x7f8e50108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:13.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.898+0000 7f8e57f63640 1 --2- 172.21.15.84:0/2584147145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e501019c0 0x7f8e50101e40 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:13.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.898+0000 7f8e57f63640 1 --2- 172.21.15.84:0/2584147145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e501007d0 0x7f8e50100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:13.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.898+0000 7f8e57f63640 1 -- 172.21.15.84:0/2584147145 >> 172.21.15.84:0/2584147145 conn(0x7f8e500fbd40 msgr2=0x7f8e500fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:13.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.898+0000 7f8e57f63640 1 -- 172.21.15.84:0/2584147145 shutdown_connections 2024-05-22T23:01:13.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.898+0000 7f8e57f63640 1 -- 172.21.15.84:0/2584147145 wait complete. 2024-05-22T23:01:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.898+0000 7f8e57f63640 1 Processor -- start 2024-05-22T23:01:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.899+0000 7f8e57f63640 1 -- start start 2024-05-22T23:01:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.899+0000 7f8e57f63640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e501007d0 0x7f8e50197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.899+0000 7f8e57f63640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e501019c0 0x7f8e50198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.899+0000 7f8e57f63640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e50102380 0x7f8e5019f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.899+0000 7f8e57f63640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5010d570 con 0x7f8e50102380 2024-05-22T23:01:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.899+0000 7f8e57f63640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5010d3f0 con 0x7f8e501019c0 2024-05-22T23:01:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.899+0000 7f8e57f63640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5010d6f0 con 0x7f8e501007d0 2024-05-22T23:01:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.899+0000 7f8e56500640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e50102380 0x7f8e5019f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.899+0000 7f8e56500640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e50102380 0x7f8e5019f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52508/0 (socket says 172.21.15.84:52508) 2024-05-22T23:01:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.899+0000 7f8e56500640 1 -- 172.21.15.84:0/3309373292 learned_addr learned my addr 172.21.15.84:0/3309373292 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.899+0000 7f8e55cff640 1 --2- 172.21.15.84:0/3309373292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e501007d0 0x7f8e50197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:13.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.899+0000 7f8e56500640 1 -- 172.21.15.84:0/3309373292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e501007d0 msgr2=0x7f8e50197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:13.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.899+0000 7f8e56500640 1 --2- 172.21.15.84:0/3309373292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e501007d0 0x7f8e50197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:13.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.899+0000 7f8e56500640 1 -- 172.21.15.84:0/3309373292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e501019c0 msgr2=0x7f8e50198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:13.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.899+0000 7f8e554fe640 1 --2- 172.21.15.84:0/3309373292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e501019c0 0x7f8e50198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:13.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.899+0000 7f8e56500640 1 --2- 172.21.15.84:0/3309373292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e501019c0 0x7f8e50198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:13.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.899+0000 7f8e56500640 1 -- 172.21.15.84:0/3309373292 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5019f850 con 0x7f8e50102380 2024-05-22T23:01:13.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.899+0000 7f8e55cff640 1 --2- 172.21.15.84:0/3309373292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e501007d0 0x7f8e50197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:01:13.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.899+0000 7f8e56500640 1 --2- 172.21.15.84:0/3309373292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e50102380 0x7f8e5019f1b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800d750 tx=0x7f8e4800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:13.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.899+0000 7f8e46ffd640 1 -- 172.21.15.84:0/3309373292 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48017070 con 0x7f8e50102380 2024-05-22T23:01:13.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.899+0000 7f8e57f63640 1 -- 172.21.15.84:0/3309373292 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e5019fb40 con 0x7f8e50102380 2024-05-22T23:01:13.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.899+0000 7f8e46ffd640 1 -- 172.21.15.84:0/3309373292 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e48005500 con 0x7f8e50102380 2024-05-22T23:01:13.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.899+0000 7f8e46ffd640 1 -- 172.21.15.84:0/3309373292 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48004d90 con 0x7f8e50102380 2024-05-22T23:01:13.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.899+0000 7f8e57f63640 1 -- 172.21.15.84:0/3309373292 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e501a0040 con 0x7f8e50102380 2024-05-22T23:01:13.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.900+0000 7f8e57f63640 1 -- 172.21.15.84:0/3309373292 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e50107b40 con 0x7f8e50102380 2024-05-22T23:01:13.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.901+0000 7f8e46ffd640 1 -- 172.21.15.84:0/3309373292 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8e48004f30 con 0x7f8e50102380 2024-05-22T23:01:13.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.902+0000 7f8e46ffd640 1 --2- 172.21.15.84:0/3309373292 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8e3c0a2c20 0x7f8e3c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:13.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.902+0000 7f8e55cff640 1 --2- 172.21.15.84:0/3309373292 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8e3c0a2c20 0x7f8e3c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:13.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.902+0000 7f8e46ffd640 1 -- 172.21.15.84:0/3309373292 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f8e480ced70 con 0x7f8e50102380 2024-05-22T23:01:13.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.903+0000 7f8e55cff640 1 --2- 172.21.15.84:0/3309373292 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8e3c0a2c20 0x7f8e3c0a50e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8e50199110 tx=0x7f8e4c009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:13.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:13.906+0000 7f8e46ffd640 1 -- 172.21.15.84:0/3309373292 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8e4809bb50 con 0x7f8e50102380 2024-05-22T23:01:13.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:13 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/663440903' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-22T23:01:14.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:13 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/663440903' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-22T23:01:14.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.084+0000 7f8e57f63640 1 -- 172.21.15.84:0/3309373292 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8e5002d400 con 0x7f8e50102380 2024-05-22T23:01:14.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.084+0000 7f8e46ffd640 1 -- 172.21.15.84:0/3309373292 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f8e4809b450 con 0x7f8e50102380 2024-05-22T23:01:14.086 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2024-05-22T23:01:14.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.086+0000 7f8e57f63640 1 -- 172.21.15.84:0/3309373292 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8e3c0a2c20 msgr2=0x7f8e3c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:14.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.086+0000 7f8e57f63640 1 --2- 172.21.15.84:0/3309373292 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8e3c0a2c20 0x7f8e3c0a50e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8e50199110 tx=0x7f8e4c009520 comp rx=0 tx=0).stop 2024-05-22T23:01:14.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.086+0000 7f8e57f63640 1 -- 172.21.15.84:0/3309373292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e50102380 msgr2=0x7f8e5019f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:14.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.086+0000 7f8e57f63640 1 --2- 172.21.15.84:0/3309373292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e50102380 0x7f8e5019f1b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800d750 tx=0x7f8e4800dc10 comp rx=0 tx=0).stop 2024-05-22T23:01:14.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.086+0000 7f8e57f63640 1 -- 172.21.15.84:0/3309373292 shutdown_connections 2024-05-22T23:01:14.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.086+0000 7f8e57f63640 1 --2- 172.21.15.84:0/3309373292 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8e3c0a2c20 0x7f8e3c0a50e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.086+0000 7f8e57f63640 1 --2- 172.21.15.84:0/3309373292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e50102380 0x7f8e5019f1b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.086+0000 7f8e57f63640 1 --2- 172.21.15.84:0/3309373292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e501019c0 0x7f8e50198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.086+0000 7f8e57f63640 1 --2- 172.21.15.84:0/3309373292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e501007d0 0x7f8e50197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.086+0000 7f8e57f63640 1 -- 172.21.15.84:0/3309373292 >> 172.21.15.84:0/3309373292 conn(0x7f8e500fbd40 msgr2=0x7f8e50104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:14.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.087+0000 7f8e57f63640 1 -- 172.21.15.84:0/3309373292 shutdown_connections 2024-05-22T23:01:14.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.087+0000 7f8e57f63640 1 -- 172.21.15.84:0/3309373292 wait complete. 2024-05-22T23:01:14.099 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T23:01:14.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.201+0000 7ff44ae58640 1 -- 172.21.15.84:0/1690507523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff444101a50 msgr2=0x7ff444108370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:14.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.201+0000 7ff44ae58640 1 --2- 172.21.15.84:0/1690507523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff444101a50 0x7ff444108370 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff43c00adb0 tx=0x7ff43c02b7a0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.201+0000 7ff44ae58640 1 -- 172.21.15.84:0/1690507523 shutdown_connections 2024-05-22T23:01:14.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.201+0000 7ff44ae58640 1 --2- 172.21.15.84:0/1690507523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff444101a50 0x7ff444108370 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.201+0000 7ff44ae58640 1 --2- 172.21.15.84:0/1690507523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff444101090 0x7ff444101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.201+0000 7ff44ae58640 1 --2- 172.21.15.84:0/1690507523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff444100080 0x7ff4441004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.201+0000 7ff44ae58640 1 -- 172.21.15.84:0/1690507523 >> 172.21.15.84:0/1690507523 conn(0x7ff4440fb070 msgr2=0x7ff4440fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:14.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.201+0000 7ff44ae58640 1 -- 172.21.15.84:0/1690507523 shutdown_connections 2024-05-22T23:01:14.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.201+0000 7ff44ae58640 1 -- 172.21.15.84:0/1690507523 wait complete. 2024-05-22T23:01:14.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.202+0000 7ff44ae58640 1 Processor -- start 2024-05-22T23:01:14.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.202+0000 7ff44ae58640 1 -- start start 2024-05-22T23:01:14.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.202+0000 7ff44ae58640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff444100080 0x7ff44410b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:14.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.202+0000 7ff44ae58640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff444101090 0x7ff444109b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:14.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.202+0000 7ff44ae58640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff444101a50 0x7ff44410a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:14.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.202+0000 7ff44ae58640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff44410d520 con 0x7ff444101090 2024-05-22T23:01:14.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.202+0000 7ff44ae58640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff44410d3a0 con 0x7ff444101a50 2024-05-22T23:01:14.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.202+0000 7ff44ae58640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff44410d6a0 con 0x7ff444100080 2024-05-22T23:01:14.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.202+0000 7ff443fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff444101090 0x7ff444109b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:14.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.202+0000 7ff443fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff444101090 0x7ff444109b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52520/0 (socket says 172.21.15.84:52520) 2024-05-22T23:01:14.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.202+0000 7ff443fff640 1 -- 172.21.15.84:0/2171252070 learned_addr learned my addr 172.21.15.84:0/2171252070 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:14.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.202+0000 7ff448bf4640 1 --2- 172.21.15.84:0/2171252070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff444100080 0x7ff44410b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:14.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.202+0000 7ff4493f5640 1 --2- 172.21.15.84:0/2171252070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff444101a50 0x7ff44410a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:14.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.203+0000 7ff443fff640 1 -- 172.21.15.84:0/2171252070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff444100080 msgr2=0x7ff44410b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:14.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.203+0000 7ff443fff640 1 --2- 172.21.15.84:0/2171252070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff444100080 0x7ff44410b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.203+0000 7ff443fff640 1 -- 172.21.15.84:0/2171252070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff444101a50 msgr2=0x7ff44410a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:14.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.203+0000 7ff443fff640 1 --2- 172.21.15.84:0/2171252070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff444101a50 0x7ff44410a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.203+0000 7ff443fff640 1 -- 172.21.15.84:0/2171252070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff44410a960 con 0x7ff444101090 2024-05-22T23:01:14.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.203+0000 7ff4493f5640 1 --2- 172.21.15.84:0/2171252070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff444101a50 0x7ff44410a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:01:14.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.203+0000 7ff448bf4640 1 --2- 172.21.15.84:0/2171252070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff444100080 0x7ff44410b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:01:14.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.203+0000 7ff443fff640 1 --2- 172.21.15.84:0/2171252070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff444101090 0x7ff444109b90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff43800c890 tx=0x7ff43800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:14.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.203+0000 7ff441ffb640 1 -- 172.21.15.84:0/2171252070 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff438004220 con 0x7ff444101090 2024-05-22T23:01:14.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.203+0000 7ff441ffb640 1 -- 172.21.15.84:0/2171252070 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4380043c0 con 0x7ff444101090 2024-05-22T23:01:14.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.203+0000 7ff44ae58640 1 -- 172.21.15.84:0/2171252070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4441a1c80 con 0x7ff444101090 2024-05-22T23:01:14.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.203+0000 7ff441ffb640 1 -- 172.21.15.84:0/2171252070 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4380049e0 con 0x7ff444101090 2024-05-22T23:01:14.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.203+0000 7ff44ae58640 1 -- 172.21.15.84:0/2171252070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4441a21d0 con 0x7ff444101090 2024-05-22T23:01:14.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.204+0000 7ff44ae58640 1 -- 172.21.15.84:0/2171252070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff444107250 con 0x7ff444101090 2024-05-22T23:01:14.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.205+0000 7ff441ffb640 1 -- 172.21.15.84:0/2171252070 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff4380079b0 con 0x7ff444101090 2024-05-22T23:01:14.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.206+0000 7ff441ffb640 1 --2- 172.21.15.84:0/2171252070 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7ff42c0a2c20 0x7ff42c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:14.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.206+0000 7ff448bf4640 1 --2- 172.21.15.84:0/2171252070 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7ff42c0a2c20 0x7ff42c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:14.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.206+0000 7ff441ffb640 1 -- 172.21.15.84:0/2171252070 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7ff4380cf390 con 0x7ff444101090 2024-05-22T23:01:14.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.207+0000 7ff448bf4640 1 --2- 172.21.15.84:0/2171252070 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7ff42c0a2c20 0x7ff42c0a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff4340058c0 tx=0x7ff434005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:14.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.211+0000 7ff441ffb640 1 -- 172.21.15.84:0/2171252070 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff438014030 con 0x7ff444101090 2024-05-22T23:01:14.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.387+0000 7ff44ae58640 1 -- 172.21.15.84:0/2171252070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff44402d440 con 0x7ff444101090 2024-05-22T23:01:14.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.387+0000 7ff441ffb640 1 -- 172.21.15.84:0/2171252070 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7ff43809c170 con 0x7ff444101090 2024-05-22T23:01:14.389 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2024-05-22T23:01:14.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.389+0000 7ff44ae58640 1 -- 172.21.15.84:0/2171252070 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7ff42c0a2c20 msgr2=0x7ff42c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:14.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.389+0000 7ff44ae58640 1 --2- 172.21.15.84:0/2171252070 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7ff42c0a2c20 0x7ff42c0a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff4340058c0 tx=0x7ff434005850 comp rx=0 tx=0).stop 2024-05-22T23:01:14.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.389+0000 7ff44ae58640 1 -- 172.21.15.84:0/2171252070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff444101090 msgr2=0x7ff444109b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:14.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.389+0000 7ff44ae58640 1 --2- 172.21.15.84:0/2171252070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff444101090 0x7ff444109b90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff43800c890 tx=0x7ff43800cd50 comp rx=0 tx=0).stop 2024-05-22T23:01:14.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.389+0000 7ff44ae58640 1 -- 172.21.15.84:0/2171252070 shutdown_connections 2024-05-22T23:01:14.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.389+0000 7ff44ae58640 1 --2- 172.21.15.84:0/2171252070 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7ff42c0a2c20 0x7ff42c0a50e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.389+0000 7ff44ae58640 1 --2- 172.21.15.84:0/2171252070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff444101a50 0x7ff44410a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.389+0000 7ff44ae58640 1 --2- 172.21.15.84:0/2171252070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff444101090 0x7ff444109b90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.389+0000 7ff44ae58640 1 --2- 172.21.15.84:0/2171252070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff444100080 0x7ff44410b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.389+0000 7ff44ae58640 1 -- 172.21.15.84:0/2171252070 >> 172.21.15.84:0/2171252070 conn(0x7ff4440fb070 msgr2=0x7ff444104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:14.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.389+0000 7ff44ae58640 1 -- 172.21.15.84:0/2171252070 shutdown_connections 2024-05-22T23:01:14.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.389+0000 7ff44ae58640 1 -- 172.21.15.84:0/2171252070 wait complete. 2024-05-22T23:01:14.402 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T23:01:14.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.505+0000 7f774e62a640 1 -- 172.21.15.84:0/3206754761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77481007b0 msgr2=0x7f7748100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:14.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.505+0000 7f774e62a640 1 --2- 172.21.15.84:0/3206754761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77481007b0 0x7f7748100bd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7738009900 tx=0x7f773802a410 comp rx=0 tx=0).stop 2024-05-22T23:01:14.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.505+0000 7f774e62a640 1 -- 172.21.15.84:0/3206754761 shutdown_connections 2024-05-22T23:01:14.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.505+0000 7f774e62a640 1 --2- 172.21.15.84:0/3206754761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7748102360 0x7f7748108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.505+0000 7f774e62a640 1 --2- 172.21.15.84:0/3206754761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77481019a0 0x7f7748101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.505+0000 7f774e62a640 1 --2- 172.21.15.84:0/3206754761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77481007b0 0x7f7748100bd0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.505+0000 7f774e62a640 1 -- 172.21.15.84:0/3206754761 >> 172.21.15.84:0/3206754761 conn(0x7f77480fbd20 msgr2=0x7f77480fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.505+0000 7f774e62a640 1 -- 172.21.15.84:0/3206754761 shutdown_connections 2024-05-22T23:01:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.505+0000 7f774e62a640 1 -- 172.21.15.84:0/3206754761 wait complete. 2024-05-22T23:01:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.505+0000 7f774e62a640 1 Processor -- start 2024-05-22T23:01:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.505+0000 7f774e62a640 1 -- start start 2024-05-22T23:01:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.505+0000 7f774e62a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77481007b0 0x7f7748104740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.505+0000 7f774e62a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77481019a0 0x7f7748102d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.505+0000 7f774e62a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7748102360 0x7f77481032c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.505+0000 7f774e62a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f774810c430 con 0x7f77481019a0 2024-05-22T23:01:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.505+0000 7f774e62a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f774810c2b0 con 0x7f77481007b0 2024-05-22T23:01:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.505+0000 7f774e62a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f774810c5b0 con 0x7f7748102360 2024-05-22T23:01:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.506+0000 7f773ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77481019a0 0x7f7748102d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.506+0000 7f773ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77481019a0 0x7f7748102d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52530/0 (socket says 172.21.15.84:52530) 2024-05-22T23:01:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.506+0000 7f773ffff640 1 -- 172.21.15.84:0/3881334599 learned_addr learned my addr 172.21.15.84:0/3881334599 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.506+0000 7f774cbc7640 1 --2- 172.21.15.84:0/3881334599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7748102360 0x7f77481032c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.506+0000 7f773ffff640 1 -- 172.21.15.84:0/3881334599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7748102360 msgr2=0x7f77481032c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.506+0000 7f773ffff640 1 --2- 172.21.15.84:0/3881334599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7748102360 0x7f77481032c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.506+0000 7f7747fff640 1 --2- 172.21.15.84:0/3881334599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77481007b0 0x7f7748104740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.506+0000 7f773ffff640 1 -- 172.21.15.84:0/3881334599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77481007b0 msgr2=0x7f7748104740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.506+0000 7f773ffff640 1 --2- 172.21.15.84:0/3881334599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77481007b0 0x7f7748104740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.506+0000 7f773ffff640 1 -- 172.21.15.84:0/3881334599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7748103b50 con 0x7f77481019a0 2024-05-22T23:01:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.506+0000 7f773ffff640 1 --2- 172.21.15.84:0/3881334599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77481019a0 0x7f7748102d80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f773000e9f0 tx=0x7f773000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.506+0000 7f7745ffb640 1 -- 172.21.15.84:0/3881334599 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773000cd80 con 0x7f77481019a0 2024-05-22T23:01:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.506+0000 7f7745ffb640 1 -- 172.21.15.84:0/3881334599 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7730010910 con 0x7f77481019a0 2024-05-22T23:01:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.506+0000 7f774e62a640 1 -- 172.21.15.84:0/3881334599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77481a1d50 con 0x7f77481019a0 2024-05-22T23:01:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.506+0000 7f7745ffb640 1 -- 172.21.15.84:0/3881334599 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7730018990 con 0x7f77481019a0 2024-05-22T23:01:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.506+0000 7f774e62a640 1 -- 172.21.15.84:0/3881334599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77481a21e0 con 0x7f77481019a0 2024-05-22T23:01:14.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.507+0000 7f774e62a640 1 -- 172.21.15.84:0/3881334599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77481a2470 con 0x7f77481019a0 2024-05-22T23:01:14.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.508+0000 7f7745ffb640 1 -- 172.21.15.84:0/3881334599 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7730010ac0 con 0x7f77481019a0 2024-05-22T23:01:14.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.509+0000 7f7745ffb640 1 --2- 172.21.15.84:0/3881334599 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f77340a2c20 0x7f77340a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:14.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.509+0000 7f7747fff640 1 --2- 172.21.15.84:0/3881334599 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f77340a2c20 0x7f77340a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:14.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.509+0000 7f7745ffb640 1 -- 172.21.15.84:0/3881334599 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f77300cf650 con 0x7f77481019a0 2024-05-22T23:01:14.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.509+0000 7f7747fff640 1 --2- 172.21.15.84:0/3881334599 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f77340a2c20 0x7f77340a50e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7738031610 tx=0x7f7738000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:14.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.513+0000 7f7745ffb640 1 -- 172.21.15.84:0/3881334599 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f773009c430 con 0x7f77481019a0 2024-05-22T23:01:14.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.689+0000 7f774e62a640 1 -- 172.21.15.84:0/3881334599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f77481a2760 con 0x7f77481019a0 2024-05-22T23:01:14.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.689+0000 7f7745ffb640 1 -- 172.21.15.84:0/3881334599 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f773009bd30 con 0x7f77481019a0 2024-05-22T23:01:14.691 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2024-05-22T23:01:14.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.691+0000 7f774e62a640 1 -- 172.21.15.84:0/3881334599 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f77340a2c20 msgr2=0x7f77340a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:14.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.691+0000 7f774e62a640 1 --2- 172.21.15.84:0/3881334599 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f77340a2c20 0x7f77340a50e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7738031610 tx=0x7f7738000bc0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.691+0000 7f774e62a640 1 -- 172.21.15.84:0/3881334599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77481019a0 msgr2=0x7f7748102d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:14.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.691+0000 7f774e62a640 1 --2- 172.21.15.84:0/3881334599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77481019a0 0x7f7748102d80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f773000e9f0 tx=0x7f773000eeb0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.691+0000 7f774e62a640 1 -- 172.21.15.84:0/3881334599 shutdown_connections 2024-05-22T23:01:14.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.691+0000 7f774e62a640 1 --2- 172.21.15.84:0/3881334599 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f77340a2c20 0x7f77340a50e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.691+0000 7f774e62a640 1 --2- 172.21.15.84:0/3881334599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7748102360 0x7f77481032c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.691+0000 7f774e62a640 1 --2- 172.21.15.84:0/3881334599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77481019a0 0x7f7748102d80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.691+0000 7f774e62a640 1 --2- 172.21.15.84:0/3881334599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77481007b0 0x7f7748104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.691+0000 7f774e62a640 1 -- 172.21.15.84:0/3881334599 >> 172.21.15.84:0/3881334599 conn(0x7f77480fbd20 msgr2=0x7f77480fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:14.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.692+0000 7f774e62a640 1 -- 172.21.15.84:0/3881334599 shutdown_connections 2024-05-22T23:01:14.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.692+0000 7f774e62a640 1 -- 172.21.15.84:0/3881334599 wait complete. 2024-05-22T23:01:14.703 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:14 smithi084 ceph-mon[32186]: pgmap v332: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:01:14.703 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:14 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2658251672' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:14.703 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:14 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3309373292' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:01:14.704 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:14 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2171252070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:01:14.704 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:01:14.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.804+0000 7f60ac163640 1 -- 172.21.15.84:0/1042211876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60a4102340 msgr2=0x7f60a4104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:14.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.804+0000 7f60ac163640 1 --2- 172.21.15.84:0/1042211876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60a4102340 0x7f60a4104730 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6094009900 tx=0x7f609402a410 comp rx=0 tx=0).stop 2024-05-22T23:01:14.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.804+0000 7f60ac163640 1 -- 172.21.15.84:0/1042211876 shutdown_connections 2024-05-22T23:01:14.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.804+0000 7f60ac163640 1 --2- 172.21.15.84:0/1042211876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60a41075a0 0x7f60a41099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.804+0000 7f60ac163640 1 --2- 172.21.15.84:0/1042211876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60a4104c70 0x7f60a4107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.804+0000 7f60ac163640 1 --2- 172.21.15.84:0/1042211876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60a4102340 0x7f60a4104730 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.804+0000 7f60ac163640 1 -- 172.21.15.84:0/1042211876 >> 172.21.15.84:0/1042211876 conn(0x7f60a40fbd70 msgr2=0x7f60a40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:14.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.804+0000 7f60ac163640 1 -- 172.21.15.84:0/1042211876 shutdown_connections 2024-05-22T23:01:14.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.804+0000 7f60ac163640 1 -- 172.21.15.84:0/1042211876 wait complete. 2024-05-22T23:01:14.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.804+0000 7f60ac163640 1 Processor -- start 2024-05-22T23:01:14.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.804+0000 7f60ac163640 1 -- start start 2024-05-22T23:01:14.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.804+0000 7f60ac163640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60a4102340 0x7f60a4197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:14.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.804+0000 7f60ac163640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60a4104c70 0x7f60a41981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:14.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.804+0000 7f60ac163640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60a41075a0 0x7f60a419f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:14.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.804+0000 7f60ac163640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f60a410d5c0 con 0x7f60a4104c70 2024-05-22T23:01:14.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.804+0000 7f60ac163640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f60a410d440 con 0x7f60a4102340 2024-05-22T23:01:14.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.804+0000 7f60ac163640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f60a410d740 con 0x7f60a41075a0 2024-05-22T23:01:14.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.805+0000 7f60a96fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60a4104c70 0x7f60a41981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:14.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.805+0000 7f60a96fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60a4104c70 0x7f60a41981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52544/0 (socket says 172.21.15.84:52544) 2024-05-22T23:01:14.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.805+0000 7f60a96fe640 1 -- 172.21.15.84:0/2348604196 learned_addr learned my addr 172.21.15.84:0/2348604196 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:14.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.805+0000 7f60aa700640 1 --2- 172.21.15.84:0/2348604196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60a41075a0 0x7f60a419f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:14.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.805+0000 7f60a96fe640 1 -- 172.21.15.84:0/2348604196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60a41075a0 msgr2=0x7f60a419f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:14.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.805+0000 7f60a9eff640 1 --2- 172.21.15.84:0/2348604196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60a4102340 0x7f60a4197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:14.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.805+0000 7f60a96fe640 1 --2- 172.21.15.84:0/2348604196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60a41075a0 0x7f60a419f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.805+0000 7f60a96fe640 1 -- 172.21.15.84:0/2348604196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60a4102340 msgr2=0x7f60a4197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:14.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.805+0000 7f60a96fe640 1 --2- 172.21.15.84:0/2348604196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60a4102340 0x7f60a4197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:14.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.805+0000 7f60a96fe640 1 -- 172.21.15.84:0/2348604196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a419f870 con 0x7f60a4104c70 2024-05-22T23:01:14.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.805+0000 7f60a96fe640 1 --2- 172.21.15.84:0/2348604196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60a4104c70 0x7f60a41981a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f60a000b9c0 tx=0x7f60a000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:14.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.805+0000 7f609affd640 1 -- 172.21.15.84:0/2348604196 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a0019070 con 0x7f60a4104c70 2024-05-22T23:01:14.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.805+0000 7f609affd640 1 -- 172.21.15.84:0/2348604196 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60a000cba0 con 0x7f60a4104c70 2024-05-22T23:01:14.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.805+0000 7f60ac163640 1 -- 172.21.15.84:0/2348604196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a419fb60 con 0x7f60a4104c70 2024-05-22T23:01:14.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.805+0000 7f609affd640 1 -- 172.21.15.84:0/2348604196 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a00126b0 con 0x7f60a4104c70 2024-05-22T23:01:14.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.805+0000 7f60ac163640 1 -- 172.21.15.84:0/2348604196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a41a0060 con 0x7f60a4104c70 2024-05-22T23:01:14.810 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.806+0000 7f60ac163640 1 -- 172.21.15.84:0/2348604196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a40fd860 con 0x7f60a4104c70 2024-05-22T23:01:14.810 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.807+0000 7f609affd640 1 -- 172.21.15.84:0/2348604196 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f60a001c070 con 0x7f60a4104c70 2024-05-22T23:01:14.810 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.808+0000 7f609affd640 1 --2- 172.21.15.84:0/2348604196 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f60900a2c20 0x7f60900a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:14.810 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.808+0000 7f60a9eff640 1 --2- 172.21.15.84:0/2348604196 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f60900a2c20 0x7f60900a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:14.811 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.808+0000 7f609affd640 1 -- 172.21.15.84:0/2348604196 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f60a00ce800 con 0x7f60a4104c70 2024-05-22T23:01:14.811 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.808+0000 7f60a9eff640 1 --2- 172.21.15.84:0/2348604196 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f60900a2c20 0x7f60900a50e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6094031610 tx=0x7f6094000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:14.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:14.812+0000 7f609affd640 1 -- 172.21.15.84:0/2348604196 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f60a009b6f0 con 0x7f60a4104c70 2024-05-22T23:01:14.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:14 smithi204 ceph-mon[36211]: pgmap v332: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:01:14.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:14 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2658251672' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:14.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:14 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3309373292' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:01:14.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:14 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2171252070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:01:15.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.026+0000 7f60ac163640 1 -- 172.21.15.84:0/2348604196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60a402d200 con 0x7f60a4104c70 2024-05-22T23:01:15.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.026+0000 7f609affd640 1 -- 172.21.15.84:0/2348604196 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7f60a009aff0 con 0x7f60a4104c70 2024-05-22T23:01:15.028 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:15.028 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:01:15.028 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:01:15.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.028+0000 7f60ac163640 1 -- 172.21.15.84:0/2348604196 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f60900a2c20 msgr2=0x7f60900a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:15.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.028+0000 7f60ac163640 1 --2- 172.21.15.84:0/2348604196 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f60900a2c20 0x7f60900a50e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6094031610 tx=0x7f6094000bc0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.028+0000 7f60ac163640 1 -- 172.21.15.84:0/2348604196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60a4104c70 msgr2=0x7f60a41981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:15.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.028+0000 7f60ac163640 1 --2- 172.21.15.84:0/2348604196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60a4104c70 0x7f60a41981a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f60a000b9c0 tx=0x7f60a000be80 comp rx=0 tx=0).stop 2024-05-22T23:01:15.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.028+0000 7f60ac163640 1 -- 172.21.15.84:0/2348604196 shutdown_connections 2024-05-22T23:01:15.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.028+0000 7f60ac163640 1 --2- 172.21.15.84:0/2348604196 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f60900a2c20 0x7f60900a50e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.028+0000 7f60ac163640 1 --2- 172.21.15.84:0/2348604196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60a41075a0 0x7f60a419f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.028+0000 7f60ac163640 1 --2- 172.21.15.84:0/2348604196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60a4104c70 0x7f60a41981a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.028+0000 7f60ac163640 1 --2- 172.21.15.84:0/2348604196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60a4102340 0x7f60a4197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.028+0000 7f60ac163640 1 -- 172.21.15.84:0/2348604196 >> 172.21.15.84:0/2348604196 conn(0x7f60a40fbd70 msgr2=0x7f60a4102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:15.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.029+0000 7f60ac163640 1 -- 172.21.15.84:0/2348604196 shutdown_connections 2024-05-22T23:01:15.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.029+0000 7f60ac163640 1 -- 172.21.15.84:0/2348604196 wait complete. 2024-05-22T23:01:15.042 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T23:01:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:14 smithi096 ceph-mon[36392]: pgmap v332: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:01:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:14 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2658251672' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:14 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3309373292' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:01:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:14 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2171252070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:01:15.143 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.140+0000 7fed74c2a640 1 -- 172.21.15.84:0/1202165374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed701019b0 msgr2=0x7fed70101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:15.143 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.140+0000 7fed74c2a640 1 --2- 172.21.15.84:0/1202165374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed701019b0 0x7fed70101e30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fed58009900 tx=0x7fed5802a410 comp rx=0 tx=0).stop 2024-05-22T23:01:15.143 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.140+0000 7fed74c2a640 1 -- 172.21.15.84:0/1202165374 shutdown_connections 2024-05-22T23:01:15.143 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.140+0000 7fed74c2a640 1 --2- 172.21.15.84:0/1202165374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed70102370 0x7fed70108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.143 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.140+0000 7fed74c2a640 1 --2- 172.21.15.84:0/1202165374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed701019b0 0x7fed70101e30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.140+0000 7fed74c2a640 1 --2- 172.21.15.84:0/1202165374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed701007c0 0x7fed70100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.140+0000 7fed74c2a640 1 -- 172.21.15.84:0/1202165374 >> 172.21.15.84:0/1202165374 conn(0x7fed700fbd50 msgr2=0x7fed700fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:15.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.140+0000 7fed74c2a640 1 -- 172.21.15.84:0/1202165374 shutdown_connections 2024-05-22T23:01:15.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.140+0000 7fed74c2a640 1 -- 172.21.15.84:0/1202165374 wait complete. 2024-05-22T23:01:15.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.140+0000 7fed74c2a640 1 Processor -- start 2024-05-22T23:01:15.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.141+0000 7fed74c2a640 1 -- start start 2024-05-22T23:01:15.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.141+0000 7fed74c2a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed701007c0 0x7fed70196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:15.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.141+0000 7fed74c2a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed701019b0 0x7fed70197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:15.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.141+0000 7fed74c2a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed70102370 0x7fed7019e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:15.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.141+0000 7fed74c2a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fed7010c440 con 0x7fed70102370 2024-05-22T23:01:15.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.141+0000 7fed74c2a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fed7010c2c0 con 0x7fed701007c0 2024-05-22T23:01:15.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.141+0000 7fed74c2a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fed7010c5c0 con 0x7fed701019b0 2024-05-22T23:01:15.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.141+0000 7fed6ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed70102370 0x7fed7019e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:15.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.141+0000 7fed6ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed70102370 0x7fed7019e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52566/0 (socket says 172.21.15.84:52566) 2024-05-22T23:01:15.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.141+0000 7fed6ed9d640 1 -- 172.21.15.84:0/328592334 learned_addr learned my addr 172.21.15.84:0/328592334 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:15.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.141+0000 7fed6e59c640 1 --2- 172.21.15.84:0/328592334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed701007c0 0x7fed70196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:15.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.141+0000 7fed67fff640 1 --2- 172.21.15.84:0/328592334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed701019b0 0x7fed70197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:15.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.141+0000 7fed6ed9d640 1 -- 172.21.15.84:0/328592334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed701019b0 msgr2=0x7fed70197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:15.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.141+0000 7fed6ed9d640 1 --2- 172.21.15.84:0/328592334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed701019b0 0x7fed70197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.141+0000 7fed6ed9d640 1 -- 172.21.15.84:0/328592334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed701007c0 msgr2=0x7fed70196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:15.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.141+0000 7fed6ed9d640 1 --2- 172.21.15.84:0/328592334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed701007c0 0x7fed70196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.141+0000 7fed6ed9d640 1 -- 172.21.15.84:0/328592334 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7019e720 con 0x7fed70102370 2024-05-22T23:01:15.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.141+0000 7fed67fff640 1 --2- 172.21.15.84:0/328592334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed701019b0 0x7fed70197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:01:15.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.141+0000 7fed6ed9d640 1 --2- 172.21.15.84:0/328592334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed70102370 0x7fed7019e0b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fed68007c30 tx=0x7fed6800e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:15.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.141+0000 7fed677fe640 1 -- 172.21.15.84:0/328592334 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed68019020 con 0x7fed70102370 2024-05-22T23:01:15.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.141+0000 7fed677fe640 1 -- 172.21.15.84:0/328592334 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed68007e40 con 0x7fed70102370 2024-05-22T23:01:15.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.141+0000 7fed677fe640 1 -- 172.21.15.84:0/328592334 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed68005590 con 0x7fed70102370 2024-05-22T23:01:15.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.141+0000 7fed74c2a640 1 -- 172.21.15.84:0/328592334 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed70069f30 con 0x7fed70102370 2024-05-22T23:01:15.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.141+0000 7fed74c2a640 1 -- 172.21.15.84:0/328592334 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed7006a400 con 0x7fed70102370 2024-05-22T23:01:15.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.142+0000 7fed74c2a640 1 -- 172.21.15.84:0/328592334 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed70068ae0 con 0x7fed70102370 2024-05-22T23:01:15.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.144+0000 7fed677fe640 1 -- 172.21.15.84:0/328592334 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fed68005750 con 0x7fed70102370 2024-05-22T23:01:15.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.145+0000 7fed677fe640 1 --2- 172.21.15.84:0/328592334 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fed5c0a2c20 0x7fed5c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:15.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.145+0000 7fed6e59c640 1 --2- 172.21.15.84:0/328592334 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fed5c0a2c20 0x7fed5c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:15.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.145+0000 7fed677fe640 1 -- 172.21.15.84:0/328592334 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fed680d1200 con 0x7fed70102370 2024-05-22T23:01:15.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.146+0000 7fed6e59c640 1 --2- 172.21.15.84:0/328592334 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fed5c0a2c20 0x7fed5c0a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fed60005fd0 tx=0x7fed60005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:15.150 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.149+0000 7fed677fe640 1 -- 172.21.15.84:0/328592334 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fed68023020 con 0x7fed70102370 2024-05-22T23:01:15.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.324+0000 7fed74c2a640 1 -- 172.21.15.84:0/328592334 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed70107550 con 0x7fed70102370 2024-05-22T23:01:15.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.326+0000 7fed677fe640 1 -- 172.21.15.84:0/328592334 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22722 (secure 0 0 0) 0x7fed6809e070 con 0x7fed70102370 2024-05-22T23:01:15.327 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:15.327 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":88,"fsid":"c1135a12-188d-11ef-bc9a-c7b262605968","created":"2024-05-22T22:51:49.878218+0000","modified":"2024-05-22T23:00:46.572591+0000","last_up_change":"2024-05-22T22:58:58.550278+0000","last_in_change":"2024-05-22T22:58:37.322139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T22:55:07.719394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T22:59:50.150511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T22:59:51.111182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762df065-fb4f-4c71-aab8-a62d2282c253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6803","nonce":1559945826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6805","nonce":1559945826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6809","nonce":1559945826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6807","nonce":1559945826}]},"public_addr":"172.21.15.84:6803/1559945826","cluster_addr":"172.21.15.84:6805/1559945826","heartbeat_back_addr":"172.21.15.84:6809/1559945826","heartbeat_front_addr":"172.21.15.84:6807/1559945826","state":["exists","up"]},{"osd":1,"uuid":"3fb2385d-6a34-49e3-88e1-116237beb346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6801","nonce":2996505353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6803","nonce":2996505353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6807","nonce":2996505353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6805","nonce":2996505353}]},"public_addr":"172.21.15.96:6801/2996505353","cluster_addr":"172.21.15.96:6803/2996505353","heartbeat_back_addr":"172.21.15.96:6807/2996505353","heartbeat_front_addr":"172.21.15.96:6805/2996505353","state":["exists","up"]},{"osd":2,"uuid":"e306e88e-a308-4585-ae11-c8af10b84038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6801","nonce":1499577897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6803","nonce":1499577897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6807","nonce":1499577897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6805","nonce":1499577897}]},"public_addr":"172.21.15.204:6801/1499577897","cluster_addr":"172.21.15.204:6803/1499577897","heartbeat_back_addr":"172.21.15.204:6807/1499577897","heartbeat_front_addr":"172.21.15.204:6805/1499577897","state":["exists","up"]},{"osd":3,"uuid":"d2a5fb97-6f60-4d45-9f18-447cae480974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6811","nonce":2869340648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6813","nonce":2869340648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6817","nonce":2869340648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6815","nonce":2869340648}]},"public_addr":"172.21.15.84:6811/2869340648","cluster_addr":"172.21.15.84:6813/2869340648","heartbeat_back_addr":"172.21.15.84:6817/2869340648","heartbeat_front_addr":"172.21.15.84:6815/2869340648","state":["exists","up"]},{"osd":4,"uuid":"4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6809","nonce":2554265648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6811","nonce":2554265648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6815","nonce":2554265648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6813","nonce":2554265648}]},"public_addr":"172.21.15.96:6809/2554265648","cluster_addr":"172.21.15.96:6811/2554265648","heartbeat_back_addr":"172.21.15.96:6815/2554265648","heartbeat_front_addr":"172.21.15.96:6813/2554265648","state":["exists","up"]},{"osd":5,"uuid":"7996bf5e-93a5-41b0-a9b1-99e7ef4514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6809","nonce":4075476986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6811","nonce":4075476986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6815","nonce":4075476986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6813","nonce":4075476986}]},"public_addr":"172.21.15.204:6809/4075476986","cluster_addr":"172.21.15.204:6811/4075476986","heartbeat_back_addr":"172.21.15.204:6815/4075476986","heartbeat_front_addr":"172.21.15.204:6813/4075476986","state":["exists","up"]},{"osd":6,"uuid":"f4b7c5de-1836-4d16-8603-3c517524d9fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6819","nonce":1806202865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6821","nonce":1806202865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6825","nonce":1806202865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6823","nonce":1806202865}]},"public_addr":"172.21.15.84:6819/1806202865","cluster_addr":"172.21.15.84:6821/1806202865","heartbeat_back_addr":"172.21.15.84:6825/1806202865","heartbeat_front_addr":"172.21.15.84:6823/1806202865","state":["exists","up"]},{"osd":7,"uuid":"69a18c43-5447-480d-9466-190830070c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6817","nonce":3195792774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6819","nonce":3195792774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6823","nonce":3195792774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6821","nonce":3195792774}]},"public_addr":"172.21.15.96:6817/3195792774","cluster_addr":"172.21.15.96:6819/3195792774","heartbeat_back_addr":"172.21.15.96:6823/3195792774","heartbeat_front_addr":"172.21.15.96:6821/3195792774","state":["exists","up"]},{"osd":8,"uuid":"2c5cadca-689a-441e-847c-ba370d400ef1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6817","nonce":606997927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6819","nonce":606997927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6823","nonce":606997927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6821","nonce":606997927}]},"public_addr":"172.21.15.204:6817/606997927","cluster_addr":"172.21.15.204:6819/606997927","heartbeat_back_addr":"172.21.15.204:6823/606997927","heartbeat_front_addr":"172.21.15.204:6821/606997927","state":["exists","up"]},{"osd":9,"uuid":"e3984fb2-7c00-4275-a10e-bca7c859f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6827","nonce":2786189012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6829","nonce":2786189012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6833","nonce":2786189012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6831","nonce":2786189012}]},"public_addr":"172.21.15.84:6827/2786189012","cluster_addr":"172.21.15.84:6829/2786189012","heartbeat_back_addr":"172.21.15.84:6833/2786189012","heartbeat_front_addr":"172.21.15.84:6831/2786189012","state":["exists","up"]},{"osd":10,"uuid":"b4938160-a808-4ded-91c2-d245c99fff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6825","nonce":1985398526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6827","nonce":1985398526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6831","nonce":1985398526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6829","nonce":1985398526}]},"public_addr":"172.21.15.96:6825/1985398526","cluster_addr":"172.21.15.96:6827/1985398526","heartbeat_back_addr":"172.21.15.96:6831/1985398526","heartbeat_front_addr":"172.21.15.96:6829/1985398526","state":["exists","up"]},{"osd":11,"uuid":"238922ef-d046-40cc-833d-25b7e7a523cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6825","nonce":3106116519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6827","nonce":3106116519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6831","nonce":3106116519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6829","nonce":3106116519}]},"public_addr":"172.21.15.204:6825/3106116519","cluster_addr":"172.21.15.204:6827/3106116519","heartbeat_back_addr":"172.21.15.204:6831/3106116519","heartbeat_front_addr":"172.21.15.204:6829/3106116519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:16.443480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:39.898133+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:03.664544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:28.838687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:53.829733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:19.269307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:45.166910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:10.771614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:36.678516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:03.150592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:29.327769+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:56.315938+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6801/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:6800/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/2766288608":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/950455999":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/4000289608":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1650467363":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1052746114":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6800/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6801/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/14610813":"2024-05-23T22:52:45.395791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T23:01:15.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.328+0000 7fed74c2a640 1 -- 172.21.15.84:0/328592334 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fed5c0a2c20 msgr2=0x7fed5c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:15.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.328+0000 7fed74c2a640 1 --2- 172.21.15.84:0/328592334 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fed5c0a2c20 0x7fed5c0a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fed60005fd0 tx=0x7fed60005850 comp rx=0 tx=0).stop 2024-05-22T23:01:15.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.328+0000 7fed74c2a640 1 -- 172.21.15.84:0/328592334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed70102370 msgr2=0x7fed7019e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:15.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.328+0000 7fed74c2a640 1 --2- 172.21.15.84:0/328592334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed70102370 0x7fed7019e0b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fed68007c30 tx=0x7fed6800e420 comp rx=0 tx=0).stop 2024-05-22T23:01:15.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.328+0000 7fed74c2a640 1 -- 172.21.15.84:0/328592334 shutdown_connections 2024-05-22T23:01:15.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.328+0000 7fed74c2a640 1 --2- 172.21.15.84:0/328592334 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fed5c0a2c20 0x7fed5c0a50e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.328+0000 7fed74c2a640 1 --2- 172.21.15.84:0/328592334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed70102370 0x7fed7019e0b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.328+0000 7fed74c2a640 1 --2- 172.21.15.84:0/328592334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed701019b0 0x7fed70197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.328+0000 7fed74c2a640 1 --2- 172.21.15.84:0/328592334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed701007c0 0x7fed70196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.328+0000 7fed74c2a640 1 -- 172.21.15.84:0/328592334 >> 172.21.15.84:0/328592334 conn(0x7fed700fbd50 msgr2=0x7fed70104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:15.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.328+0000 7fed74c2a640 1 -- 172.21.15.84:0/328592334 shutdown_connections 2024-05-22T23:01:15.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.328+0000 7fed74c2a640 1 -- 172.21.15.84:0/328592334 wait complete. 2024-05-22T23:01:15.341 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T22:55:07.719394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T22:59:50.150511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T22:59:51.111182+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T23:01:15.341 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T23:01:15.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.444+0000 7fd62ba4d640 1 -- 172.21.15.84:0/3945308494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd624107590 msgr2=0x7fd6241099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:15.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.444+0000 7fd62ba4d640 1 --2- 172.21.15.84:0/3945308494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd624107590 0x7fd6241099e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009ca0 tx=0x7fd61c02a8c0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.444+0000 7fd62ba4d640 1 -- 172.21.15.84:0/3945308494 shutdown_connections 2024-05-22T23:01:15.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.444+0000 7fd62ba4d640 1 --2- 172.21.15.84:0/3945308494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd624107590 0x7fd6241099e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.444+0000 7fd62ba4d640 1 --2- 172.21.15.84:0/3945308494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd624104c60 0x7fd624107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.444+0000 7fd62ba4d640 1 --2- 172.21.15.84:0/3945308494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd624102330 0x7fd624104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.448 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.444+0000 7fd62ba4d640 1 -- 172.21.15.84:0/3945308494 >> 172.21.15.84:0/3945308494 conn(0x7fd6240fbd60 msgr2=0x7fd6240fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:15.448 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.444+0000 7fd62ba4d640 1 -- 172.21.15.84:0/3945308494 shutdown_connections 2024-05-22T23:01:15.448 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.444+0000 7fd62ba4d640 1 -- 172.21.15.84:0/3945308494 wait complete. 2024-05-22T23:01:15.448 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.444+0000 7fd62ba4d640 1 Processor -- start 2024-05-22T23:01:15.448 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.444+0000 7fd62ba4d640 1 -- start start 2024-05-22T23:01:15.448 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.444+0000 7fd62ba4d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd624102330 0x7fd624197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:15.448 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.444+0000 7fd62ba4d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd624104c60 0x7fd624198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:15.448 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.444+0000 7fd62ba4d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd624107590 0x7fd62419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:15.448 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.444+0000 7fd62ba4d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd62410d550 con 0x7fd624107590 2024-05-22T23:01:15.448 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.444+0000 7fd62ba4d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd62410d3d0 con 0x7fd624104c60 2024-05-22T23:01:15.448 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.444+0000 7fd62ba4d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd62410d6d0 con 0x7fd624102330 2024-05-22T23:01:15.448 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.445+0000 7fd629fea640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd624107590 0x7fd62419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:15.448 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.445+0000 7fd629fea640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd624107590 0x7fd62419f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52598/0 (socket says 172.21.15.84:52598) 2024-05-22T23:01:15.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.445+0000 7fd629fea640 1 -- 172.21.15.84:0/979279985 learned_addr learned my addr 172.21.15.84:0/979279985 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:15.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.445+0000 7fd6297e9640 1 --2- 172.21.15.84:0/979279985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd624102330 0x7fd624197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:15.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.445+0000 7fd629fea640 1 -- 172.21.15.84:0/979279985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd624102330 msgr2=0x7fd624197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:15.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.445+0000 7fd629fea640 1 --2- 172.21.15.84:0/979279985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd624102330 0x7fd624197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.445+0000 7fd629fea640 1 -- 172.21.15.84:0/979279985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd624104c60 msgr2=0x7fd624198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:15.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.445+0000 7fd629fea640 1 --2- 172.21.15.84:0/979279985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd624104c60 0x7fd624198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.445+0000 7fd629fea640 1 -- 172.21.15.84:0/979279985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62419f730 con 0x7fd624107590 2024-05-22T23:01:15.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.445+0000 7fd629fea640 1 --2- 172.21.15.84:0/979279985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd624107590 0x7fd62419f0c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd61c031ae0 tx=0x7fd61c031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:15.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.445+0000 7fd6167fc640 1 -- 172.21.15.84:0/979279985 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c003c30 con 0x7fd624107590 2024-05-22T23:01:15.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.445+0000 7fd6167fc640 1 -- 172.21.15.84:0/979279985 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd61c003dd0 con 0x7fd624107590 2024-05-22T23:01:15.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.445+0000 7fd62ba4d640 1 -- 172.21.15.84:0/979279985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd62419f9c0 con 0x7fd624107590 2024-05-22T23:01:15.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.445+0000 7fd6167fc640 1 -- 172.21.15.84:0/979279985 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c02fcb0 con 0x7fd624107590 2024-05-22T23:01:15.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.445+0000 7fd62ba4d640 1 -- 172.21.15.84:0/979279985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd62419ff10 con 0x7fd624107590 2024-05-22T23:01:15.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.446+0000 7fd62ba4d640 1 -- 172.21.15.84:0/979279985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6240fd850 con 0x7fd624107590 2024-05-22T23:01:15.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.447+0000 7fd6167fc640 1 -- 172.21.15.84:0/979279985 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd61c004050 con 0x7fd624107590 2024-05-22T23:01:15.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.448+0000 7fd6167fc640 1 --2- 172.21.15.84:0/979279985 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fd6100a2c20 0x7fd6100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:15.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.448+0000 7fd6297e9640 1 --2- 172.21.15.84:0/979279985 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fd6100a2c20 0x7fd6100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:15.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.448+0000 7fd6167fc640 1 -- 172.21.15.84:0/979279985 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fd61c0f0470 con 0x7fd624107590 2024-05-22T23:01:15.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.448+0000 7fd6297e9640 1 --2- 172.21.15.84:0/979279985 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fd6100a2c20 0x7fd6100a50e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd618009aa0 tx=0x7fd618005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:15.453 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.452+0000 7fd6167fc640 1 -- 172.21.15.84:0/979279985 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd61c0bd360 con 0x7fd624107590 2024-05-22T23:01:15.603 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.599+0000 7fd6167fc640 1 -- 172.21.15.84:0/979279985 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd61c0b92d0 con 0x7fd624107590 2024-05-22T23:01:15.625 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:15 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3881334599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:01:15.625 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:15 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2348604196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:01:15.625 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:15 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/328592334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:15.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.663+0000 7fd62ba4d640 1 -- 172.21.15.84:0/979279985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd62402d3e0 con 0x7fd624107590 2024-05-22T23:01:15.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.663+0000 7fd6167fc640 1 -- 172.21.15.84:0/979279985 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fd61c0a88a0 con 0x7fd624107590 2024-05-22T23:01:15.665 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2024-05-22T23:01:15.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.665+0000 7fd62ba4d640 1 -- 172.21.15.84:0/979279985 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fd6100a2c20 msgr2=0x7fd6100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:15.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.665+0000 7fd62ba4d640 1 --2- 172.21.15.84:0/979279985 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fd6100a2c20 0x7fd6100a50e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd618009aa0 tx=0x7fd618005f90 comp rx=0 tx=0).stop 2024-05-22T23:01:15.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.665+0000 7fd62ba4d640 1 -- 172.21.15.84:0/979279985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd624107590 msgr2=0x7fd62419f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:15.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.665+0000 7fd62ba4d640 1 --2- 172.21.15.84:0/979279985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd624107590 0x7fd62419f0c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd61c031ae0 tx=0x7fd61c031ee0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.665+0000 7fd62ba4d640 1 -- 172.21.15.84:0/979279985 shutdown_connections 2024-05-22T23:01:15.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.665+0000 7fd62ba4d640 1 --2- 172.21.15.84:0/979279985 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fd6100a2c20 0x7fd6100a50e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.665+0000 7fd62ba4d640 1 --2- 172.21.15.84:0/979279985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd624107590 0x7fd62419f0c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.665+0000 7fd62ba4d640 1 --2- 172.21.15.84:0/979279985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd624104c60 0x7fd624198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.665+0000 7fd62ba4d640 1 --2- 172.21.15.84:0/979279985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd624102330 0x7fd624197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.665+0000 7fd62ba4d640 1 -- 172.21.15.84:0/979279985 >> 172.21.15.84:0/979279985 conn(0x7fd6240fbd60 msgr2=0x7fd624102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:15.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.665+0000 7fd62ba4d640 1 -- 172.21.15.84:0/979279985 shutdown_connections 2024-05-22T23:01:15.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.665+0000 7fd62ba4d640 1 -- 172.21.15.84:0/979279985 wait complete. 2024-05-22T23:01:15.678 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T23:01:15.781 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.779+0000 7fe7e90e1640 1 -- 172.21.15.84:0/2510214557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e4103470 msgr2=0x7fe7e4109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:15.781 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.779+0000 7fe7e90e1640 1 --2- 172.21.15.84:0/2510214557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e4103470 0x7fe7e4109d00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009ca0 tx=0x7fe7dc02a8c0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.781 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.779+0000 7fe7e90e1640 1 -- 172.21.15.84:0/2510214557 shutdown_connections 2024-05-22T23:01:15.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.779+0000 7fe7e90e1640 1 --2- 172.21.15.84:0/2510214557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e4103470 0x7fe7e4109d00 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.779+0000 7fe7e90e1640 1 --2- 172.21.15.84:0/2510214557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7e4102ab0 0x7fe7e4102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.779+0000 7fe7e90e1640 1 --2- 172.21.15.84:0/2510214557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7e41018c0 0x7fe7e4101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.779+0000 7fe7e90e1640 1 -- 172.21.15.84:0/2510214557 >> 172.21.15.84:0/2510214557 conn(0x7fe7e40fce10 msgr2=0x7fe7e40ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:15.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.780+0000 7fe7e90e1640 1 -- 172.21.15.84:0/2510214557 shutdown_connections 2024-05-22T23:01:15.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.780+0000 7fe7e90e1640 1 -- 172.21.15.84:0/2510214557 wait complete. 2024-05-22T23:01:15.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.780+0000 7fe7e90e1640 1 Processor -- start 2024-05-22T23:01:15.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.780+0000 7fe7e90e1640 1 -- start start 2024-05-22T23:01:15.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.780+0000 7fe7e90e1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7e41018c0 0x7fe7e4197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:15.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.780+0000 7fe7e90e1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e4102ab0 0x7fe7e41980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:15.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.781+0000 7fe7e90e1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7e4103470 0x7fe7e419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:15.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.781+0000 7fe7e90e1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e410d540 con 0x7fe7e4102ab0 2024-05-22T23:01:15.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.781+0000 7fe7e90e1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e410d3c0 con 0x7fe7e4103470 2024-05-22T23:01:15.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.781+0000 7fe7e90e1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e410d6c0 con 0x7fe7e41018c0 2024-05-22T23:01:15.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.781+0000 7fe7e259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e4102ab0 0x7fe7e41980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:15.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.781+0000 7fe7e259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e4102ab0 0x7fe7e41980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52622/0 (socket says 172.21.15.84:52622) 2024-05-22T23:01:15.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.781+0000 7fe7e259c640 1 -- 172.21.15.84:0/1391781504 learned_addr learned my addr 172.21.15.84:0/1391781504 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:15.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.781+0000 7fe7e359e640 1 --2- 172.21.15.84:0/1391781504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7e4103470 0x7fe7e419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:15.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.781+0000 7fe7e2d9d640 1 --2- 172.21.15.84:0/1391781504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7e41018c0 0x7fe7e4197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:15.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.781+0000 7fe7e259c640 1 -- 172.21.15.84:0/1391781504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7e41018c0 msgr2=0x7fe7e4197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:15.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.781+0000 7fe7e259c640 1 --2- 172.21.15.84:0/1391781504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7e41018c0 0x7fe7e4197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.781+0000 7fe7e259c640 1 -- 172.21.15.84:0/1391781504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7e4103470 msgr2=0x7fe7e419f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:15.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.781+0000 7fe7e259c640 1 --2- 172.21.15.84:0/1391781504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7e4103470 0x7fe7e419f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.781+0000 7fe7e259c640 1 -- 172.21.15.84:0/1391781504 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e419f7b0 con 0x7fe7e4102ab0 2024-05-22T23:01:15.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.781+0000 7fe7e259c640 1 --2- 172.21.15.84:0/1391781504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e4102ab0 0x7fe7e41980d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe7d800d950 tx=0x7fe7d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:15.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.781+0000 7fe7cbfff640 1 -- 172.21.15.84:0/1391781504 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8017070 con 0x7fe7e4102ab0 2024-05-22T23:01:15.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.781+0000 7fe7e90e1640 1 -- 172.21.15.84:0/1391781504 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e419faa0 con 0x7fe7e4102ab0 2024-05-22T23:01:15.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.781+0000 7fe7cbfff640 1 -- 172.21.15.84:0/1391781504 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7d8009c30 con 0x7fe7e4102ab0 2024-05-22T23:01:15.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.781+0000 7fe7cbfff640 1 -- 172.21.15.84:0/1391781504 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d800f6b0 con 0x7fe7e4102ab0 2024-05-22T23:01:15.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.781+0000 7fe7e90e1640 1 -- 172.21.15.84:0/1391781504 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e419fff0 con 0x7fe7e4102ab0 2024-05-22T23:01:15.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.782+0000 7fe7e90e1640 1 -- 172.21.15.84:0/1391781504 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e4108b80 con 0x7fe7e4102ab0 2024-05-22T23:01:15.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.783+0000 7fe7cbfff640 1 -- 172.21.15.84:0/1391781504 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe7d80109d0 con 0x7fe7e4102ab0 2024-05-22T23:01:15.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.784+0000 7fe7cbfff640 1 --2- 172.21.15.84:0/1391781504 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe7cc0a2b90 0x7fe7cc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:15.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.784+0000 7fe7e2d9d640 1 --2- 172.21.15.84:0/1391781504 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe7cc0a2b90 0x7fe7cc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:15.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.785+0000 7fe7cbfff640 1 -- 172.21.15.84:0/1391781504 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fe7d80ced20 con 0x7fe7e4102ab0 2024-05-22T23:01:15.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.785+0000 7fe7e2d9d640 1 --2- 172.21.15.84:0/1391781504 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe7cc0a2b90 0x7fe7cc0a5050 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4005fd0 tx=0x7fe7d4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:15.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.789+0000 7fe7cbfff640 1 -- 172.21.15.84:0/1391781504 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe7d8009de0 con 0x7fe7e4102ab0 2024-05-22T23:01:15.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:15 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3881334599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:01:15.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:15 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2348604196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:01:15.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:15 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/328592334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:15.964 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.963+0000 7fe7e90e1640 1 -- 172.21.15.84:0/1391781504 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe7e40624a0 con 0x7fe7e4102ab0 2024-05-22T23:01:15.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.964+0000 7fe7cbfff640 1 -- 172.21.15.84:0/1391781504 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fe7d809bc60 con 0x7fe7e4102ab0 2024-05-22T23:01:15.965 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2024-05-22T23:01:15.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.965+0000 7fe7e90e1640 1 -- 172.21.15.84:0/1391781504 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe7cc0a2b90 msgr2=0x7fe7cc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:15.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.965+0000 7fe7e90e1640 1 --2- 172.21.15.84:0/1391781504 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe7cc0a2b90 0x7fe7cc0a5050 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4005fd0 tx=0x7fe7d4005850 comp rx=0 tx=0).stop 2024-05-22T23:01:15.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.966+0000 7fe7e90e1640 1 -- 172.21.15.84:0/1391781504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e4102ab0 msgr2=0x7fe7e41980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:15.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.966+0000 7fe7e90e1640 1 --2- 172.21.15.84:0/1391781504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e4102ab0 0x7fe7e41980d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe7d800d950 tx=0x7fe7d800de10 comp rx=0 tx=0).stop 2024-05-22T23:01:15.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.966+0000 7fe7e90e1640 1 -- 172.21.15.84:0/1391781504 shutdown_connections 2024-05-22T23:01:15.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.966+0000 7fe7e90e1640 1 --2- 172.21.15.84:0/1391781504 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fe7cc0a2b90 0x7fe7cc0a5050 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.966+0000 7fe7e90e1640 1 --2- 172.21.15.84:0/1391781504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7e4103470 0x7fe7e419f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.966+0000 7fe7e90e1640 1 --2- 172.21.15.84:0/1391781504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e4102ab0 0x7fe7e41980d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.966+0000 7fe7e90e1640 1 --2- 172.21.15.84:0/1391781504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7e41018c0 0x7fe7e4197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:15.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.966+0000 7fe7e90e1640 1 -- 172.21.15.84:0/1391781504 >> 172.21.15.84:0/1391781504 conn(0x7fe7e40fce10 msgr2=0x7fe7e4105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:15.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.966+0000 7fe7e90e1640 1 -- 172.21.15.84:0/1391781504 shutdown_connections 2024-05-22T23:01:15.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:15.966+0000 7fe7e90e1640 1 -- 172.21.15.84:0/1391781504 wait complete. 2024-05-22T23:01:15.978 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T23:01:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:15 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3881334599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:01:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:15 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2348604196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:01:16.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:15 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/328592334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:16.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.080+0000 7fdbcf3f9640 1 -- 172.21.15.84:0/1642245858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbc81019a0 msgr2=0x7fdbc8101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:16.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.080+0000 7fdbcf3f9640 1 --2- 172.21.15.84:0/1642245858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbc81019a0 0x7fdbc8101e20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8009900 tx=0x7fdbb802a410 comp rx=0 tx=0).stop 2024-05-22T23:01:16.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.080+0000 7fdbcf3f9640 1 -- 172.21.15.84:0/1642245858 shutdown_connections 2024-05-22T23:01:16.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.080+0000 7fdbcf3f9640 1 --2- 172.21.15.84:0/1642245858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbc8102360 0x7fdbc8108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.080+0000 7fdbcf3f9640 1 --2- 172.21.15.84:0/1642245858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbc81019a0 0x7fdbc8101e20 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.080+0000 7fdbcf3f9640 1 --2- 172.21.15.84:0/1642245858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbc81007b0 0x7fdbc8100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.080+0000 7fdbcf3f9640 1 -- 172.21.15.84:0/1642245858 >> 172.21.15.84:0/1642245858 conn(0x7fdbc80fbd20 msgr2=0x7fdbc80fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:16.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.080+0000 7fdbcf3f9640 1 -- 172.21.15.84:0/1642245858 shutdown_connections 2024-05-22T23:01:16.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.080+0000 7fdbcf3f9640 1 -- 172.21.15.84:0/1642245858 wait complete. 2024-05-22T23:01:16.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.081+0000 7fdbcf3f9640 1 Processor -- start 2024-05-22T23:01:16.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.081+0000 7fdbcf3f9640 1 -- start start 2024-05-22T23:01:16.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.081+0000 7fdbcf3f9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbc81007b0 0x7fdbc8197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:16.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.081+0000 7fdbcf3f9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbc81019a0 0x7fdbc8198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:16.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.081+0000 7fdbcf3f9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbc8102360 0x7fdbc819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:16.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.081+0000 7fdbcf3f9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc810c430 con 0x7fdbc8102360 2024-05-22T23:01:16.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.081+0000 7fdbcf3f9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc810c2b0 con 0x7fdbc81007b0 2024-05-22T23:01:16.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.081+0000 7fdbcf3f9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc810c5b0 con 0x7fdbc81019a0 2024-05-22T23:01:16.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.081+0000 7fdbcd996640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbc8102360 0x7fdbc819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:16.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.081+0000 7fdbcd996640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbc8102360 0x7fdbc819f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52642/0 (socket says 172.21.15.84:52642) 2024-05-22T23:01:16.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.081+0000 7fdbcd996640 1 -- 172.21.15.84:0/2413593065 learned_addr learned my addr 172.21.15.84:0/2413593065 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:16.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.081+0000 7fdbcd195640 1 --2- 172.21.15.84:0/2413593065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbc81007b0 0x7fdbc8197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:16.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.082+0000 7fdbcc994640 1 --2- 172.21.15.84:0/2413593065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbc81019a0 0x7fdbc8198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:16.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.082+0000 7fdbcd996640 1 -- 172.21.15.84:0/2413593065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbc81019a0 msgr2=0x7fdbc8198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:16.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.082+0000 7fdbcd996640 1 --2- 172.21.15.84:0/2413593065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbc81019a0 0x7fdbc8198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.082+0000 7fdbcd996640 1 -- 172.21.15.84:0/2413593065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbc81007b0 msgr2=0x7fdbc8197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:16.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.082+0000 7fdbcd996640 1 --2- 172.21.15.84:0/2413593065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbc81007b0 0x7fdbc8197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.082+0000 7fdbcd996640 1 -- 172.21.15.84:0/2413593065 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc819f830 con 0x7fdbc8102360 2024-05-22T23:01:16.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.082+0000 7fdbcc994640 1 --2- 172.21.15.84:0/2413593065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbc81019a0 0x7fdbc8198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:01:16.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.082+0000 7fdbcd996640 1 --2- 172.21.15.84:0/2413593065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbc8102360 0x7fdbc819f190 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000eaa0 tx=0x7fdbc000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:16.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.082+0000 7fdbbe7fc640 1 -- 172.21.15.84:0/2413593065 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc000cdb0 con 0x7fdbc8102360 2024-05-22T23:01:16.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.082+0000 7fdbbe7fc640 1 -- 172.21.15.84:0/2413593065 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc0005020 con 0x7fdbc8102360 2024-05-22T23:01:16.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.082+0000 7fdbcf3f9640 1 -- 172.21.15.84:0/2413593065 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc819fb20 con 0x7fdbc8102360 2024-05-22T23:01:16.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.082+0000 7fdbbe7fc640 1 -- 172.21.15.84:0/2413593065 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc00116b0 con 0x7fdbc8102360 2024-05-22T23:01:16.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.082+0000 7fdbcf3f9640 1 -- 172.21.15.84:0/2413593065 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc81a0070 con 0x7fdbc8102360 2024-05-22T23:01:16.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.083+0000 7fdbcf3f9640 1 -- 172.21.15.84:0/2413593065 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbc81a02a0 con 0x7fdbc8102360 2024-05-22T23:01:16.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.084+0000 7fdbbe7fc640 1 -- 172.21.15.84:0/2413593065 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdbc0005560 con 0x7fdbc8102360 2024-05-22T23:01:16.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.084+0000 7fdbbe7fc640 1 --2- 172.21.15.84:0/2413593065 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fdbb40a2b90 0x7fdbb40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:16.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.085+0000 7fdbcd195640 1 --2- 172.21.15.84:0/2413593065 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fdbb40a2b90 0x7fdbb40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:16.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.085+0000 7fdbbe7fc640 1 -- 172.21.15.84:0/2413593065 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fdbc00cf360 con 0x7fdbc8102360 2024-05-22T23:01:16.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.085+0000 7fdbcd195640 1 --2- 172.21.15.84:0/2413593065 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fdbb40a2b90 0x7fdbb40a5050 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdbc81990f0 tx=0x7fdbc4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:16.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.089+0000 7fdbbe7fc640 1 -- 172.21.15.84:0/2413593065 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdbc0016030 con 0x7fdbc8102360 2024-05-22T23:01:16.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.264+0000 7fdbcf3f9640 1 -- 172.21.15.84:0/2413593065 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdbc81a0510 con 0x7fdbc8102360 2024-05-22T23:01:16.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.265+0000 7fdbbe7fc640 1 -- 172.21.15.84:0/2413593065 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fdbc009c2a0 con 0x7fdbc8102360 2024-05-22T23:01:16.266 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2024-05-22T23:01:16.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.267+0000 7fdbcf3f9640 1 -- 172.21.15.84:0/2413593065 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fdbb40a2b90 msgr2=0x7fdbb40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:16.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.267+0000 7fdbcf3f9640 1 --2- 172.21.15.84:0/2413593065 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fdbb40a2b90 0x7fdbb40a5050 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdbc81990f0 tx=0x7fdbc4005f90 comp rx=0 tx=0).stop 2024-05-22T23:01:16.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.267+0000 7fdbcf3f9640 1 -- 172.21.15.84:0/2413593065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbc8102360 msgr2=0x7fdbc819f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:16.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.267+0000 7fdbcf3f9640 1 --2- 172.21.15.84:0/2413593065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbc8102360 0x7fdbc819f190 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000eaa0 tx=0x7fdbc000ef60 comp rx=0 tx=0).stop 2024-05-22T23:01:16.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.267+0000 7fdbcf3f9640 1 -- 172.21.15.84:0/2413593065 shutdown_connections 2024-05-22T23:01:16.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.267+0000 7fdbcf3f9640 1 --2- 172.21.15.84:0/2413593065 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fdbb40a2b90 0x7fdbb40a5050 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.267+0000 7fdbcf3f9640 1 --2- 172.21.15.84:0/2413593065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbc8102360 0x7fdbc819f190 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.267+0000 7fdbcf3f9640 1 --2- 172.21.15.84:0/2413593065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbc81019a0 0x7fdbc8198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.267+0000 7fdbcf3f9640 1 --2- 172.21.15.84:0/2413593065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbc81007b0 0x7fdbc8197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.267+0000 7fdbcf3f9640 1 -- 172.21.15.84:0/2413593065 >> 172.21.15.84:0/2413593065 conn(0x7fdbc80fbd20 msgr2=0x7fdbc80fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:16.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.267+0000 7fdbcf3f9640 1 -- 172.21.15.84:0/2413593065 shutdown_connections 2024-05-22T23:01:16.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.267+0000 7fdbcf3f9640 1 -- 172.21.15.84:0/2413593065 wait complete. 2024-05-22T23:01:16.280 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:01:16.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.378+0000 7f4c90348640 1 -- 172.21.15.84:0/3677991529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c88103470 msgr2=0x7f4c88109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:16.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.378+0000 7f4c90348640 1 --2- 172.21.15.84:0/3677991529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c88103470 0x7f4c88109d00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009ca0 tx=0x7f4c8002a8c0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.379+0000 7f4c90348640 1 -- 172.21.15.84:0/3677991529 shutdown_connections 2024-05-22T23:01:16.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.379+0000 7f4c90348640 1 --2- 172.21.15.84:0/3677991529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c88103470 0x7f4c88109d00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.379+0000 7f4c90348640 1 --2- 172.21.15.84:0/3677991529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c88102ab0 0x7f4c88102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.379+0000 7f4c90348640 1 --2- 172.21.15.84:0/3677991529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c881018c0 0x7f4c88101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.379+0000 7f4c90348640 1 -- 172.21.15.84:0/3677991529 >> 172.21.15.84:0/3677991529 conn(0x7f4c880fce50 msgr2=0x7f4c880ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:16.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.379+0000 7f4c90348640 1 -- 172.21.15.84:0/3677991529 shutdown_connections 2024-05-22T23:01:16.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.379+0000 7f4c90348640 1 -- 172.21.15.84:0/3677991529 wait complete. 2024-05-22T23:01:16.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.379+0000 7f4c90348640 1 Processor -- start 2024-05-22T23:01:16.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.379+0000 7f4c90348640 1 -- start start 2024-05-22T23:01:16.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.379+0000 7f4c90348640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c881018c0 0x7f4c88197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:16.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.379+0000 7f4c90348640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c88102ab0 0x7f4c88198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:16.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.379+0000 7f4c90348640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c88103470 0x7f4c8819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:16.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.379+0000 7f4c90348640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8810d630 con 0x7f4c88102ab0 2024-05-22T23:01:16.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.379+0000 7f4c90348640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8810d4b0 con 0x7f4c881018c0 2024-05-22T23:01:16.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.379+0000 7f4c90348640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8810d7b0 con 0x7f4c88103470 2024-05-22T23:01:16.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.379+0000 7f4c8d8e3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c88102ab0 0x7f4c88198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:16.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.379+0000 7f4c8d8e3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c88102ab0 0x7f4c88198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52658/0 (socket says 172.21.15.84:52658) 2024-05-22T23:01:16.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.379+0000 7f4c8d8e3640 1 -- 172.21.15.84:0/453039049 learned_addr learned my addr 172.21.15.84:0/453039049 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:16.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.379+0000 7f4c8e8e5640 1 --2- 172.21.15.84:0/453039049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c88103470 0x7f4c8819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:16.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.380+0000 7f4c8e0e4640 1 --2- 172.21.15.84:0/453039049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c881018c0 0x7f4c88197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:16.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.380+0000 7f4c8d8e3640 1 -- 172.21.15.84:0/453039049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c88103470 msgr2=0x7f4c8819f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:16.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.380+0000 7f4c8d8e3640 1 --2- 172.21.15.84:0/453039049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c88103470 0x7f4c8819f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.380+0000 7f4c8d8e3640 1 -- 172.21.15.84:0/453039049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c881018c0 msgr2=0x7f4c88197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:16.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.380+0000 7f4c8d8e3640 1 --2- 172.21.15.84:0/453039049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c881018c0 0x7f4c88197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.380+0000 7f4c8d8e3640 1 -- 172.21.15.84:0/453039049 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8819f900 con 0x7f4c88102ab0 2024-05-22T23:01:16.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.380+0000 7f4c8e0e4640 1 --2- 172.21.15.84:0/453039049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c881018c0 0x7f4c88197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:01:16.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.380+0000 7f4c8e8e5640 1 --2- 172.21.15.84:0/453039049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c88103470 0x7f4c8819f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:01:16.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.380+0000 7f4c8d8e3640 1 --2- 172.21.15.84:0/453039049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c88102ab0 0x7f4c88198230 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4c7800b4a0 tx=0x7f4c7800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:16.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.380+0000 7f4c7f7fe640 1 -- 172.21.15.84:0/453039049 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c78014070 con 0x7f4c88102ab0 2024-05-22T23:01:16.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.380+0000 7f4c7f7fe640 1 -- 172.21.15.84:0/453039049 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c78010af0 con 0x7f4c88102ab0 2024-05-22T23:01:16.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.380+0000 7f4c90348640 1 -- 172.21.15.84:0/453039049 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8819fbf0 con 0x7f4c88102ab0 2024-05-22T23:01:16.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.380+0000 7f4c7f7fe640 1 -- 172.21.15.84:0/453039049 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c78017d10 con 0x7f4c88102ab0 2024-05-22T23:01:16.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.380+0000 7f4c90348640 1 -- 172.21.15.84:0/453039049 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c881a0140 con 0x7f4c88102ab0 2024-05-22T23:01:16.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.381+0000 7f4c90348640 1 -- 172.21.15.84:0/453039049 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c88108c00 con 0x7f4c88102ab0 2024-05-22T23:01:16.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.382+0000 7f4c7f7fe640 1 -- 172.21.15.84:0/453039049 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4c78017070 con 0x7f4c88102ab0 2024-05-22T23:01:16.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.383+0000 7f4c7f7fe640 1 --2- 172.21.15.84:0/453039049 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f4c740a2b90 0x7f4c740a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:16.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.383+0000 7f4c8e0e4640 1 --2- 172.21.15.84:0/453039049 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f4c740a2b90 0x7f4c740a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:16.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.383+0000 7f4c7f7fe640 1 -- 172.21.15.84:0/453039049 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f4c780ce510 con 0x7f4c88102ab0 2024-05-22T23:01:16.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.384+0000 7f4c8e0e4640 1 --2- 172.21.15.84:0/453039049 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f4c740a2b90 0x7f4c740a5050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4c84006fd0 tx=0x7f4c84008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:16.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.388+0000 7f4c7f7fe640 1 -- 172.21.15.84:0/453039049 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4c7809b450 con 0x7f4c88102ab0 2024-05-22T23:01:16.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.599+0000 7f4c90348640 1 -- 172.21.15.84:0/453039049 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c8802d200 con 0x7f4c88102ab0 2024-05-22T23:01:16.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.600+0000 7f4c7f7fe640 1 -- 172.21.15.84:0/453039049 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7f4c7809ad50 con 0x7f4c88102ab0 2024-05-22T23:01:16.602 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:16.602 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:01:16.602 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:01:16.604 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.602+0000 7f4c90348640 1 -- 172.21.15.84:0/453039049 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f4c740a2b90 msgr2=0x7f4c740a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:16.604 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.602+0000 7f4c90348640 1 --2- 172.21.15.84:0/453039049 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f4c740a2b90 0x7f4c740a5050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4c84006fd0 tx=0x7f4c84008040 comp rx=0 tx=0).stop 2024-05-22T23:01:16.604 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.602+0000 7f4c90348640 1 -- 172.21.15.84:0/453039049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c88102ab0 msgr2=0x7f4c88198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:16.604 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.602+0000 7f4c90348640 1 --2- 172.21.15.84:0/453039049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c88102ab0 0x7f4c88198230 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4c7800b4a0 tx=0x7f4c7800b960 comp rx=0 tx=0).stop 2024-05-22T23:01:16.604 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.602+0000 7f4c90348640 1 -- 172.21.15.84:0/453039049 shutdown_connections 2024-05-22T23:01:16.604 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.602+0000 7f4c90348640 1 --2- 172.21.15.84:0/453039049 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f4c740a2b90 0x7f4c740a5050 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.604 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.602+0000 7f4c90348640 1 --2- 172.21.15.84:0/453039049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c88103470 0x7f4c8819f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.604 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.602+0000 7f4c90348640 1 --2- 172.21.15.84:0/453039049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c88102ab0 0x7f4c88198230 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.604 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.602+0000 7f4c90348640 1 --2- 172.21.15.84:0/453039049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c881018c0 0x7f4c88197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.605 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.602+0000 7f4c90348640 1 -- 172.21.15.84:0/453039049 >> 172.21.15.84:0/453039049 conn(0x7f4c880fce50 msgr2=0x7f4c88105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:16.605 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.602+0000 7f4c90348640 1 -- 172.21.15.84:0/453039049 shutdown_connections 2024-05-22T23:01:16.605 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.602+0000 7f4c90348640 1 -- 172.21.15.84:0/453039049 wait complete. 2024-05-22T23:01:16.615 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T23:01:16.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.713+0000 7fb244742640 1 -- 172.21.15.84:0/374639920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb23c1007c0 msgr2=0x7fb23c100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:16.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.713+0000 7fb244742640 1 --2- 172.21.15.84:0/374639920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb23c1007c0 0x7fb23c100be0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb230009900 tx=0x7fb23002a410 comp rx=0 tx=0).stop 2024-05-22T23:01:16.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.713+0000 7fb244742640 1 -- 172.21.15.84:0/374639920 shutdown_connections 2024-05-22T23:01:16.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.713+0000 7fb244742640 1 --2- 172.21.15.84:0/374639920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb23c102370 0x7fb23c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.713+0000 7fb244742640 1 --2- 172.21.15.84:0/374639920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb23c1019b0 0x7fb23c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.713+0000 7fb244742640 1 --2- 172.21.15.84:0/374639920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb23c1007c0 0x7fb23c100be0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.713+0000 7fb244742640 1 -- 172.21.15.84:0/374639920 >> 172.21.15.84:0/374639920 conn(0x7fb23c0fbd50 msgr2=0x7fb23c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:16.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.713+0000 7fb244742640 1 -- 172.21.15.84:0/374639920 shutdown_connections 2024-05-22T23:01:16.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.713+0000 7fb244742640 1 -- 172.21.15.84:0/374639920 wait complete. 2024-05-22T23:01:16.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.714+0000 7fb244742640 1 Processor -- start 2024-05-22T23:01:16.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.714+0000 7fb244742640 1 -- start start 2024-05-22T23:01:16.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.714+0000 7fb244742640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb23c1007c0 0x7fb23c197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:16.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.714+0000 7fb244742640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb23c1019b0 0x7fb23c198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:16.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.714+0000 7fb244742640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb23c102370 0x7fb23c19f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:16.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.714+0000 7fb244742640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c10d5a0 con 0x7fb23c1007c0 2024-05-22T23:01:16.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.714+0000 7fb244742640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c10d420 con 0x7fb23c1019b0 2024-05-22T23:01:16.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.714+0000 7fb244742640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c10d720 con 0x7fb23c102370 2024-05-22T23:01:16.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.714+0000 7fb2424de640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb23c1007c0 0x7fb23c197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:16.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.714+0000 7fb2424de640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb23c1007c0 0x7fb23c197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52672/0 (socket says 172.21.15.84:52672) 2024-05-22T23:01:16.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.714+0000 7fb2424de640 1 -- 172.21.15.84:0/2559597590 learned_addr learned my addr 172.21.15.84:0/2559597590 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:16.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.714+0000 7fb2424de640 1 -- 172.21.15.84:0/2559597590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb23c102370 msgr2=0x7fb23c19f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:16.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.714+0000 7fb2424de640 1 --2- 172.21.15.84:0/2559597590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb23c102370 0x7fb23c19f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.714+0000 7fb2424de640 1 -- 172.21.15.84:0/2559597590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb23c1019b0 msgr2=0x7fb23c198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:16.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.714+0000 7fb241cdd640 1 --2- 172.21.15.84:0/2559597590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb23c1019b0 0x7fb23c198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:16.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.714+0000 7fb2424de640 1 --2- 172.21.15.84:0/2559597590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb23c1019b0 0x7fb23c198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.714+0000 7fb2424de640 1 -- 172.21.15.84:0/2559597590 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23c19f810 con 0x7fb23c1007c0 2024-05-22T23:01:16.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.714+0000 7fb241cdd640 1 --2- 172.21.15.84:0/2559597590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb23c1019b0 0x7fb23c198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:01:16.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.714+0000 7fb2424de640 1 --2- 172.21.15.84:0/2559597590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb23c1007c0 0x7fb23c197bf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb230031610 tx=0x7fb2300043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:16.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.715+0000 7fb22f7fe640 1 -- 172.21.15.84:0/2559597590 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23003f030 con 0x7fb23c1007c0 2024-05-22T23:01:16.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.715+0000 7fb22f7fe640 1 -- 172.21.15.84:0/2559597590 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb230003ce0 con 0x7fb23c1007c0 2024-05-22T23:01:16.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.715+0000 7fb244742640 1 -- 172.21.15.84:0/2559597590 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb23c19faa0 con 0x7fb23c1007c0 2024-05-22T23:01:16.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.715+0000 7fb22f7fe640 1 -- 172.21.15.84:0/2559597590 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230035070 con 0x7fb23c1007c0 2024-05-22T23:01:16.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.715+0000 7fb244742640 1 -- 172.21.15.84:0/2559597590 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb23c1a0010 con 0x7fb23c1007c0 2024-05-22T23:01:16.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.716+0000 7fb244742640 1 -- 172.21.15.84:0/2559597590 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb23c107ae0 con 0x7fb23c1007c0 2024-05-22T23:01:16.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.717+0000 7fb22f7fe640 1 -- 172.21.15.84:0/2559597590 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb230042030 con 0x7fb23c1007c0 2024-05-22T23:01:16.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.717+0000 7fb22f7fe640 1 --2- 172.21.15.84:0/2559597590 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb2280a2b90 0x7fb2280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:16.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.717+0000 7fb241cdd640 1 --2- 172.21.15.84:0/2559597590 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb2280a2b90 0x7fb2280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:16.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.717+0000 7fb22f7fe640 1 -- 172.21.15.84:0/2559597590 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fb2300ef460 con 0x7fb23c1007c0 2024-05-22T23:01:16.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.718+0000 7fb241cdd640 1 --2- 172.21.15.84:0/2559597590 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb2280a2b90 0x7fb2280a5050 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb238006fd0 tx=0x7fb238008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:16.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.722+0000 7fb22f7fe640 1 -- 172.21.15.84:0/2559597590 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb2300bc3a0 con 0x7fb23c1007c0 2024-05-22T23:01:16.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:16 smithi084 ceph-mon[32186]: pgmap v333: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:01:16.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:16 smithi084 ceph-mon[32186]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-05-22T23:01:16.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:16 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/979279985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:01:16.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:16 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1391781504' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:01:16.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:16 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2413593065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:01:16.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.897+0000 7fb244742640 1 -- 172.21.15.84:0/2559597590 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb23c062470 con 0x7fb23c1007c0 2024-05-22T23:01:16.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.898+0000 7fb22f7fe640 1 -- 172.21.15.84:0/2559597590 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22722 (secure 0 0 0) 0x7fb2300bbca0 con 0x7fb23c1007c0 2024-05-22T23:01:16.900 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:16.900 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":88,"fsid":"c1135a12-188d-11ef-bc9a-c7b262605968","created":"2024-05-22T22:51:49.878218+0000","modified":"2024-05-22T23:00:46.572591+0000","last_up_change":"2024-05-22T22:58:58.550278+0000","last_in_change":"2024-05-22T22:58:37.322139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T22:55:07.719394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T22:59:50.150511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T22:59:51.111182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762df065-fb4f-4c71-aab8-a62d2282c253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6803","nonce":1559945826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6805","nonce":1559945826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6809","nonce":1559945826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6807","nonce":1559945826}]},"public_addr":"172.21.15.84:6803/1559945826","cluster_addr":"172.21.15.84:6805/1559945826","heartbeat_back_addr":"172.21.15.84:6809/1559945826","heartbeat_front_addr":"172.21.15.84:6807/1559945826","state":["exists","up"]},{"osd":1,"uuid":"3fb2385d-6a34-49e3-88e1-116237beb346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6801","nonce":2996505353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6803","nonce":2996505353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6807","nonce":2996505353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6805","nonce":2996505353}]},"public_addr":"172.21.15.96:6801/2996505353","cluster_addr":"172.21.15.96:6803/2996505353","heartbeat_back_addr":"172.21.15.96:6807/2996505353","heartbeat_front_addr":"172.21.15.96:6805/2996505353","state":["exists","up"]},{"osd":2,"uuid":"e306e88e-a308-4585-ae11-c8af10b84038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6801","nonce":1499577897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6803","nonce":1499577897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6807","nonce":1499577897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6805","nonce":1499577897}]},"public_addr":"172.21.15.204:6801/1499577897","cluster_addr":"172.21.15.204:6803/1499577897","heartbeat_back_addr":"172.21.15.204:6807/1499577897","heartbeat_front_addr":"172.21.15.204:6805/1499577897","state":["exists","up"]},{"osd":3,"uuid":"d2a5fb97-6f60-4d45-9f18-447cae480974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6811","nonce":2869340648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6813","nonce":2869340648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6817","nonce":2869340648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6815","nonce":2869340648}]},"public_addr":"172.21.15.84:6811/2869340648","cluster_addr":"172.21.15.84:6813/2869340648","heartbeat_back_addr":"172.21.15.84:6817/2869340648","heartbeat_front_addr":"172.21.15.84:6815/2869340648","state":["exists","up"]},{"osd":4,"uuid":"4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6809","nonce":2554265648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6811","nonce":2554265648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6815","nonce":2554265648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6813","nonce":2554265648}]},"public_addr":"172.21.15.96:6809/2554265648","cluster_addr":"172.21.15.96:6811/2554265648","heartbeat_back_addr":"172.21.15.96:6815/2554265648","heartbeat_front_addr":"172.21.15.96:6813/2554265648","state":["exists","up"]},{"osd":5,"uuid":"7996bf5e-93a5-41b0-a9b1-99e7ef4514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6809","nonce":4075476986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6811","nonce":4075476986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6815","nonce":4075476986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6813","nonce":4075476986}]},"public_addr":"172.21.15.204:6809/4075476986","cluster_addr":"172.21.15.204:6811/4075476986","heartbeat_back_addr":"172.21.15.204:6815/4075476986","heartbeat_front_addr":"172.21.15.204:6813/4075476986","state":["exists","up"]},{"osd":6,"uuid":"f4b7c5de-1836-4d16-8603-3c517524d9fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6819","nonce":1806202865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6821","nonce":1806202865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6825","nonce":1806202865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6823","nonce":1806202865}]},"public_addr":"172.21.15.84:6819/1806202865","cluster_addr":"172.21.15.84:6821/1806202865","heartbeat_back_addr":"172.21.15.84:6825/1806202865","heartbeat_front_addr":"172.21.15.84:6823/1806202865","state":["exists","up"]},{"osd":7,"uuid":"69a18c43-5447-480d-9466-190830070c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6817","nonce":3195792774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6819","nonce":3195792774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6823","nonce":3195792774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6821","nonce":3195792774}]},"public_addr":"172.21.15.96:6817/3195792774","cluster_addr":"172.21.15.96:6819/3195792774","heartbeat_back_addr":"172.21.15.96:6823/3195792774","heartbeat_front_addr":"172.21.15.96:6821/3195792774","state":["exists","up"]},{"osd":8,"uuid":"2c5cadca-689a-441e-847c-ba370d400ef1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6817","nonce":606997927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6819","nonce":606997927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6823","nonce":606997927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6821","nonce":606997927}]},"public_addr":"172.21.15.204:6817/606997927","cluster_addr":"172.21.15.204:6819/606997927","heartbeat_back_addr":"172.21.15.204:6823/606997927","heartbeat_front_addr":"172.21.15.204:6821/606997927","state":["exists","up"]},{"osd":9,"uuid":"e3984fb2-7c00-4275-a10e-bca7c859f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6827","nonce":2786189012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6829","nonce":2786189012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6833","nonce":2786189012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6831","nonce":2786189012}]},"public_addr":"172.21.15.84:6827/2786189012","cluster_addr":"172.21.15.84:6829/2786189012","heartbeat_back_addr":"172.21.15.84:6833/2786189012","heartbeat_front_addr":"172.21.15.84:6831/2786189012","state":["exists","up"]},{"osd":10,"uuid":"b4938160-a808-4ded-91c2-d245c99fff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6825","nonce":1985398526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6827","nonce":1985398526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6831","nonce":1985398526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6829","nonce":1985398526}]},"public_addr":"172.21.15.96:6825/1985398526","cluster_addr":"172.21.15.96:6827/1985398526","heartbeat_back_addr":"172.21.15.96:6831/1985398526","heartbeat_front_addr":"172.21.15.96:6829/1985398526","state":["exists","up"]},{"osd":11,"uuid":"238922ef-d046-40cc-833d-25b7e7a523cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6825","nonce":3106116519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6827","nonce":3106116519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6831","nonce":3106116519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6829","nonce":3106116519}]},"public_addr":"172.21.15.204:6825/3106116519","cluster_addr":"172.21.15.204:6827/3106116519","heartbeat_back_addr":"172.21.15.204:6831/3106116519","heartbeat_front_addr":"172.21.15.204:6829/3106116519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:16.443480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:39.898133+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:03.664544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:28.838687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:53.829733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:19.269307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:45.166910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:10.771614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:36.678516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:03.150592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:29.327769+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:56.315938+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6801/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:6800/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/2766288608":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/950455999":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/4000289608":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1650467363":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1052746114":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6800/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6801/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/14610813":"2024-05-23T22:52:45.395791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T23:01:16.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.900+0000 7fb244742640 1 -- 172.21.15.84:0/2559597590 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb2280a2b90 msgr2=0x7fb2280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:16.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.900+0000 7fb244742640 1 --2- 172.21.15.84:0/2559597590 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb2280a2b90 0x7fb2280a5050 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb238006fd0 tx=0x7fb238008040 comp rx=0 tx=0).stop 2024-05-22T23:01:16.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.900+0000 7fb244742640 1 -- 172.21.15.84:0/2559597590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb23c1007c0 msgr2=0x7fb23c197bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:16.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.900+0000 7fb244742640 1 --2- 172.21.15.84:0/2559597590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb23c1007c0 0x7fb23c197bf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb230031610 tx=0x7fb2300043b0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.900+0000 7fb244742640 1 -- 172.21.15.84:0/2559597590 shutdown_connections 2024-05-22T23:01:16.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.900+0000 7fb244742640 1 --2- 172.21.15.84:0/2559597590 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb2280a2b90 0x7fb2280a5050 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.900+0000 7fb244742640 1 --2- 172.21.15.84:0/2559597590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb23c102370 0x7fb23c19f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.900+0000 7fb244742640 1 --2- 172.21.15.84:0/2559597590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb23c1019b0 0x7fb23c198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.901+0000 7fb244742640 1 --2- 172.21.15.84:0/2559597590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb23c1007c0 0x7fb23c197bf0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:16.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.901+0000 7fb244742640 1 -- 172.21.15.84:0/2559597590 >> 172.21.15.84:0/2559597590 conn(0x7fb23c0fbd50 msgr2=0x7fb23c104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:16.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.901+0000 7fb244742640 1 -- 172.21.15.84:0/2559597590 shutdown_connections 2024-05-22T23:01:16.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:16.901+0000 7fb244742640 1 -- 172.21.15.84:0/2559597590 wait complete. 2024-05-22T23:01:16.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:16 smithi204 ceph-mon[36211]: pgmap v333: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:01:16.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:16 smithi204 ceph-mon[36211]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-05-22T23:01:16.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:16 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/979279985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:01:16.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:16 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1391781504' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:01:16.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:16 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2413593065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:01:16.914 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:01:17.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.013+0000 7f3f730bd640 1 -- 172.21.15.84:0/3295897361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f6c1013f0 msgr2=0x7f3f6c108c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:17.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.013+0000 7f3f730bd640 1 --2- 172.21.15.84:0/3295897361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f6c1013f0 0x7f3f6c108c90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3f64009ca0 tx=0x7f3f6402a8c0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.013+0000 7f3f730bd640 1 -- 172.21.15.84:0/3295897361 shutdown_connections 2024-05-22T23:01:17.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.013+0000 7f3f730bd640 1 --2- 172.21.15.84:0/3295897361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f6c1013f0 0x7f3f6c108c90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.013+0000 7f3f730bd640 1 --2- 172.21.15.84:0/3295897361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f6c100a30 0x7f3f6c100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.013+0000 7f3f730bd640 1 --2- 172.21.15.84:0/3295897361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f6c101de0 0x7f3f6c102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.013+0000 7f3f730bd640 1 -- 172.21.15.84:0/3295897361 >> 172.21.15.84:0/3295897361 conn(0x7f3f6c0fb4e0 msgr2=0x7f3f6c0fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:17.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.013+0000 7f3f730bd640 1 -- 172.21.15.84:0/3295897361 shutdown_connections 2024-05-22T23:01:17.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.013+0000 7f3f730bd640 1 -- 172.21.15.84:0/3295897361 wait complete. 2024-05-22T23:01:17.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.013+0000 7f3f730bd640 1 Processor -- start 2024-05-22T23:01:17.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.013+0000 7f3f730bd640 1 -- start start 2024-05-22T23:01:17.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.014+0000 7f3f730bd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f6c100a30 0x7f3f6c1004b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:17.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.014+0000 7f3f730bd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f6c1013f0 0x7f3f6c19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:17.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.014+0000 7f3f730bd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f6c101de0 0x7f3f6c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:17.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.014+0000 7f3f730bd640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6c10c4e0 con 0x7f3f6c101de0 2024-05-22T23:01:17.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.014+0000 7f3f730bd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6c10c360 con 0x7f3f6c100a30 2024-05-22T23:01:17.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.014+0000 7f3f730bd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6c10c660 con 0x7f3f6c1013f0 2024-05-22T23:01:17.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.014+0000 7f3f7165a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f6c101de0 0x7f3f6c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:17.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.014+0000 7f3f7165a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f6c101de0 0x7f3f6c19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52692/0 (socket says 172.21.15.84:52692) 2024-05-22T23:01:17.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.014+0000 7f3f7165a640 1 -- 172.21.15.84:0/4013548375 learned_addr learned my addr 172.21.15.84:0/4013548375 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:17.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.014+0000 7f3f6bfff640 1 --2- 172.21.15.84:0/4013548375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f6c1013f0 0x7f3f6c19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:17.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.014+0000 7f3f70e59640 1 --2- 172.21.15.84:0/4013548375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f6c100a30 0x7f3f6c1004b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:17.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.014+0000 7f3f7165a640 1 -- 172.21.15.84:0/4013548375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f6c1013f0 msgr2=0x7f3f6c19c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:17.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.014+0000 7f3f7165a640 1 --2- 172.21.15.84:0/4013548375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f6c1013f0 0x7f3f6c19c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.014+0000 7f3f7165a640 1 -- 172.21.15.84:0/4013548375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f6c100a30 msgr2=0x7f3f6c1004b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:17.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.014+0000 7f3f7165a640 1 --2- 172.21.15.84:0/4013548375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f6c100a30 0x7f3f6c1004b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.014+0000 7f3f7165a640 1 -- 172.21.15.84:0/4013548375 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f6c19f7f0 con 0x7f3f6c101de0 2024-05-22T23:01:17.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.014+0000 7f3f7165a640 1 --2- 172.21.15.84:0/4013548375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f6c101de0 0x7f3f6c19ef30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3f64005e30 tx=0x7f3f64031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:17.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.014+0000 7f3f69ffb640 1 -- 172.21.15.84:0/4013548375 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f64003c30 con 0x7f3f6c101de0 2024-05-22T23:01:17.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.014+0000 7f3f69ffb640 1 -- 172.21.15.84:0/4013548375 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f64003dd0 con 0x7f3f6c101de0 2024-05-22T23:01:17.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.014+0000 7f3f730bd640 1 -- 172.21.15.84:0/4013548375 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f6c19fa80 con 0x7f3f6c101de0 2024-05-22T23:01:17.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.014+0000 7f3f69ffb640 1 -- 172.21.15.84:0/4013548375 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6402fda0 con 0x7f3f6c101de0 2024-05-22T23:01:17.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.015+0000 7f3f730bd640 1 -- 172.21.15.84:0/4013548375 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f6c1a00a0 con 0x7f3f6c101de0 2024-05-22T23:01:17.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.015+0000 7f3f730bd640 1 -- 172.21.15.84:0/4013548375 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f6c107b10 con 0x7f3f6c101de0 2024-05-22T23:01:17.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.016+0000 7f3f69ffb640 1 -- 172.21.15.84:0/4013548375 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3f64004050 con 0x7f3f6c101de0 2024-05-22T23:01:17.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.017+0000 7f3f69ffb640 1 --2- 172.21.15.84:0/4013548375 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f3f5c0a2c10 0x7f3f5c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:17.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.017+0000 7f3f70e59640 1 --2- 172.21.15.84:0/4013548375 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f3f5c0a2c10 0x7f3f5c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:17.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.018+0000 7f3f69ffb640 1 -- 172.21.15.84:0/4013548375 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f3f640f0330 con 0x7f3f6c101de0 2024-05-22T23:01:17.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.018+0000 7f3f70e59640 1 --2- 172.21.15.84:0/4013548375 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f3f5c0a2c10 0x7f3f5c0a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3f60005fd0 tx=0x7f3f60005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:17.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.022+0000 7f3f69ffb640 1 -- 172.21.15.84:0/4013548375 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3f640bd270 con 0x7f3f6c101de0 2024-05-22T23:01:17.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:16 smithi096 ceph-mon[36392]: pgmap v333: 97 pgs: 97 active+clean; 635 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:01:17.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:16 smithi096 ceph-mon[36392]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-05-22T23:01:17.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:16 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/979279985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:01:17.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:16 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1391781504' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:01:17.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:16 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2413593065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:01:17.236 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.234+0000 7f3f730bd640 1 -- 172.21.15.84:0/4013548375 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f6c02d200 con 0x7f3f6c101de0 2024-05-22T23:01:17.236 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.235+0000 7f3f69ffb640 1 -- 172.21.15.84:0/4013548375 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7f3f640bcb70 con 0x7f3f6c101de0 2024-05-22T23:01:17.237 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:17.237 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:01:17.237 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:01:17.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.237+0000 7f3f730bd640 1 -- 172.21.15.84:0/4013548375 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f3f5c0a2c10 msgr2=0x7f3f5c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:17.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.237+0000 7f3f730bd640 1 --2- 172.21.15.84:0/4013548375 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f3f5c0a2c10 0x7f3f5c0a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3f60005fd0 tx=0x7f3f60005850 comp rx=0 tx=0).stop 2024-05-22T23:01:17.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.237+0000 7f3f730bd640 1 -- 172.21.15.84:0/4013548375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f6c101de0 msgr2=0x7f3f6c19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:17.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.237+0000 7f3f730bd640 1 --2- 172.21.15.84:0/4013548375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f6c101de0 0x7f3f6c19ef30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3f64005e30 tx=0x7f3f64031ee0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.237+0000 7f3f730bd640 1 -- 172.21.15.84:0/4013548375 shutdown_connections 2024-05-22T23:01:17.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.237+0000 7f3f730bd640 1 --2- 172.21.15.84:0/4013548375 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f3f5c0a2c10 0x7f3f5c0a50d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.237+0000 7f3f730bd640 1 --2- 172.21.15.84:0/4013548375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f6c101de0 0x7f3f6c19ef30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.237+0000 7f3f730bd640 1 --2- 172.21.15.84:0/4013548375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f6c1013f0 0x7f3f6c19c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.237+0000 7f3f730bd640 1 --2- 172.21.15.84:0/4013548375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f6c100a30 0x7f3f6c1004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.237+0000 7f3f730bd640 1 -- 172.21.15.84:0/4013548375 >> 172.21.15.84:0/4013548375 conn(0x7f3f6c0fb4e0 msgr2=0x7f3f6c104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:17.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.237+0000 7f3f730bd640 1 -- 172.21.15.84:0/4013548375 shutdown_connections 2024-05-22T23:01:17.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.237+0000 7f3f730bd640 1 -- 172.21.15.84:0/4013548375 wait complete. 2024-05-22T23:01:17.250 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-22T22:59:51.926923+0000', 'modified': '2024-05-22T22:59:54.945583+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24530}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24494': {'gid': 24494, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6837/1191862158', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6836', 'nonce': 1191862158}, {'type': 'v1', 'addr': '172.21.15.96:6837', 'nonce': 1191862158}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24530': {'gid': 24530, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6839/3272422729', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6838', 'nonce': 3272422729}, {'type': 'v1', 'addr': '172.21.15.96:6839', 'nonce': 3272422729}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24530, 'qdb_cluster': [24530]} 2024-05-22T23:01:17.250 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-22T23:01:17.251 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-22T23:01:17.354 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.350+0000 7eff174ee640 1 -- 172.21.15.84:0/1727732500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff101019c0 msgr2=0x7eff10101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:17.354 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.350+0000 7eff174ee640 1 --2- 172.21.15.84:0/1727732500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff101019c0 0x7eff10101e40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009900 tx=0x7eff0c02a410 comp rx=0 tx=0).stop 2024-05-22T23:01:17.354 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.351+0000 7eff174ee640 1 -- 172.21.15.84:0/1727732500 shutdown_connections 2024-05-22T23:01:17.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.351+0000 7eff174ee640 1 --2- 172.21.15.84:0/1727732500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff10102380 0x7eff10108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.351+0000 7eff174ee640 1 --2- 172.21.15.84:0/1727732500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff101019c0 0x7eff10101e40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.351+0000 7eff174ee640 1 --2- 172.21.15.84:0/1727732500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff101007d0 0x7eff10100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.351+0000 7eff174ee640 1 -- 172.21.15.84:0/1727732500 >> 172.21.15.84:0/1727732500 conn(0x7eff100fbd60 msgr2=0x7eff100fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:17.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.351+0000 7eff174ee640 1 -- 172.21.15.84:0/1727732500 shutdown_connections 2024-05-22T23:01:17.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.351+0000 7eff174ee640 1 -- 172.21.15.84:0/1727732500 wait complete. 2024-05-22T23:01:17.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.351+0000 7eff174ee640 1 Processor -- start 2024-05-22T23:01:17.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.351+0000 7eff174ee640 1 -- start start 2024-05-22T23:01:17.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.351+0000 7eff174ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff101007d0 0x7eff10197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:17.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.351+0000 7eff174ee640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff101019c0 0x7eff101981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:17.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.351+0000 7eff174ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff10102380 0x7eff1019f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:17.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.351+0000 7eff174ee640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7eff1010d5f0 con 0x7eff10102380 2024-05-22T23:01:17.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.351+0000 7eff174ee640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7eff1010d470 con 0x7eff101007d0 2024-05-22T23:01:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.351+0000 7eff174ee640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7eff1010d770 con 0x7eff101019c0 2024-05-22T23:01:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.351+0000 7eff15a8b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff10102380 0x7eff1019f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.351+0000 7eff15a8b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff10102380 0x7eff1019f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52708/0 (socket says 172.21.15.84:52708) 2024-05-22T23:01:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.351+0000 7eff15a8b640 1 -- 172.21.15.84:0/1612450417 learned_addr learned my addr 172.21.15.84:0/1612450417 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.351+0000 7eff14a89640 1 --2- 172.21.15.84:0/1612450417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff101019c0 0x7eff101981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.352+0000 7eff1528a640 1 --2- 172.21.15.84:0/1612450417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff101007d0 0x7eff10197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.352+0000 7eff15a8b640 1 -- 172.21.15.84:0/1612450417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff101019c0 msgr2=0x7eff101981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.352+0000 7eff15a8b640 1 --2- 172.21.15.84:0/1612450417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff101019c0 0x7eff101981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.352+0000 7eff15a8b640 1 -- 172.21.15.84:0/1612450417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff101007d0 msgr2=0x7eff10197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.352+0000 7eff15a8b640 1 --2- 172.21.15.84:0/1612450417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff101007d0 0x7eff10197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.352+0000 7eff15a8b640 1 -- 172.21.15.84:0/1612450417 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1019f8b0 con 0x7eff10102380 2024-05-22T23:01:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.352+0000 7eff1528a640 1 --2- 172.21.15.84:0/1612450417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff101007d0 0x7eff10197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:01:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.352+0000 7eff15a8b640 1 --2- 172.21.15.84:0/1612450417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff10102380 0x7eff1019f210 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7eff0800eaa0 tx=0x7eff0800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.352+0000 7eff027fc640 1 -- 172.21.15.84:0/1612450417 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0800cdb0 con 0x7eff10102380 2024-05-22T23:01:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.352+0000 7eff027fc640 1 -- 172.21.15.84:0/1612450417 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff08005020 con 0x7eff10102380 2024-05-22T23:01:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.352+0000 7eff174ee640 1 -- 172.21.15.84:0/1612450417 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff1019fba0 con 0x7eff10102380 2024-05-22T23:01:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.352+0000 7eff027fc640 1 -- 172.21.15.84:0/1612450417 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff080116b0 con 0x7eff10102380 2024-05-22T23:01:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.352+0000 7eff174ee640 1 -- 172.21.15.84:0/1612450417 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff101a0140 con 0x7eff10102380 2024-05-22T23:01:17.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.354+0000 7eff027fc640 1 -- 172.21.15.84:0/1612450417 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7eff08005560 con 0x7eff10102380 2024-05-22T23:01:17.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.355+0000 7eff1a1f8740 1 Processor -- start 2024-05-22T23:01:17.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.355+0000 7eff1a1f8740 1 -- start start 2024-05-22T23:01:17.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.355+0000 7eff1a1f8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643ddd289b0 0x5643ddd28dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:17.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.355+0000 7eff1a1f8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd293a0 0x5643ddd2dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:17.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.355+0000 7eff1a1f8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ddd2e510 0x5643ddd309a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:17.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.355+0000 7eff1a1f8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5643ddc8b680 con 0x5643ddd293a0 2024-05-22T23:01:17.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.355+0000 7eff1a1f8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5643ddc2a250 con 0x5643ddd2e510 2024-05-22T23:01:17.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.355+0000 7eff1a1f8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5643ddc87640 con 0x5643ddd289b0 2024-05-22T23:01:17.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.355+0000 7eff1528a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd293a0 0x5643ddd2dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:17.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.355+0000 7eff1528a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd293a0 0x5643ddd2dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52712/0 (socket says 172.21.15.84:52712) 2024-05-22T23:01:17.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.355+0000 7eff1528a640 1 -- 172.21.15.84:0/3181365084 learned_addr learned my addr 172.21.15.84:0/3181365084 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:17.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.355+0000 7eff027fc640 1 --2- 172.21.15.84:0/1612450417 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7efefc0a2b90 0x7efefc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:17.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.355+0000 7eff15a8b640 1 --2- 172.21.15.84:0/1612450417 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7efefc0a2b90 0x7efefc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:17.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.355+0000 7eff14a89640 1 --2- 172.21.15.84:0/3181365084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ddd2e510 0x5643ddd309a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:17.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.355+0000 7eff1528a640 1 -- 172.21.15.84:0/3181365084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643ddd289b0 msgr2=0x5643ddd28dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:17.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.355+0000 7eff14a89640 1 --2- 172.21.15.84:0/3181365084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643ddd289b0 0x5643ddd28dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:17.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.355+0000 7eff027fc640 1 -- 172.21.15.84:0/1612450417 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7eff080cf380 con 0x7eff10102380 2024-05-22T23:01:17.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.355+0000 7eff1528a640 1 --2- 172.21.15.84:0/3181365084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643ddd289b0 0x5643ddd28dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.355+0000 7eff1528a640 1 -- 172.21.15.84:0/3181365084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ddd2e510 msgr2=0x5643ddd309a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:17.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.355+0000 7eff1528a640 1 --2- 172.21.15.84:0/3181365084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ddd2e510 0x5643ddd309a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.355+0000 7eff1528a640 1 -- 172.21.15.84:0/3181365084 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643dd99f2d0 con 0x5643ddd293a0 2024-05-22T23:01:17.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.355+0000 7eff14a89640 1 --2- 172.21.15.84:0/3181365084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643ddd289b0 0x5643ddd28dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:01:17.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.356+0000 7eff1528a640 1 --2- 172.21.15.84:0/3181365084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd293a0 0x5643ddd2dea0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7eff0400a9a0 tx=0x7eff0400ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6eda2bf1b30751 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:17.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.356+0000 7eff15a8b640 1 --2- 172.21.15.84:0/1612450417 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7efefc0a2b90 0x7efefc0a5050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7eff08005910 tx=0x7eff0806ec60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:17.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.356+0000 7efefb7fe640 1 -- 172.21.15.84:0/3181365084 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff04015070 con 0x5643ddd293a0 2024-05-22T23:01:17.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.356+0000 7efefb7fe640 1 -- 172.21.15.84:0/3181365084 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff0400cc00 con 0x5643ddd293a0 2024-05-22T23:01:17.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.356+0000 7efefb7fe640 1 -- 172.21.15.84:0/3181365084 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff04010590 con 0x5643ddd293a0 2024-05-22T23:01:17.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.356+0000 7eff1a1f8740 1 -- 172.21.15.84:0/3181365084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd293a0 msgr2=0x5643ddd2dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:17.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.356+0000 7eff1a1f8740 1 --2- 172.21.15.84:0/3181365084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd293a0 0x5643ddd2dea0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7eff0400a9a0 tx=0x7eff0400ae60 comp rx=0 tx=0).stop 2024-05-22T23:01:17.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.356+0000 7eff1a1f8740 1 -- 172.21.15.84:0/3181365084 shutdown_connections 2024-05-22T23:01:17.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.356+0000 7eff1a1f8740 1 --2- 172.21.15.84:0/3181365084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ddd2e510 0x5643ddd309a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.356+0000 7eff1a1f8740 1 --2- 172.21.15.84:0/3181365084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd293a0 0x5643ddd2dea0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.356+0000 7eff1a1f8740 1 --2- 172.21.15.84:0/3181365084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643ddd289b0 0x5643ddd28dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.356+0000 7eff1a1f8740 1 -- 172.21.15.84:0/3181365084 >> 172.21.15.84:0/3181365084 conn(0x5643ddd24020 msgr2=0x5643ddd24420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:17.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.357+0000 7eff1a1f8740 1 -- 172.21.15.84:0/3181365084 shutdown_connections 2024-05-22T23:01:17.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.357+0000 7eff1a1f8740 1 -- 172.21.15.84:0/3181365084 wait complete. 2024-05-22T23:01:17.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.357+0000 7eff1a1f8740 1 -- start start 2024-05-22T23:01:17.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.357+0000 7eff1a1f8740 1 Processor -- start 2024-05-22T23:01:17.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.357+0000 7eff1a1f8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ddd289b0 0x5643ddd32da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:17.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.357+0000 7eff1a1f8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd293a0 0x5643ddd332e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:17.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.357+0000 7eff1a1f8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643ddd2e510 0x5643ddd3a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:17.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.357+0000 7eff1a1f8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5643ddd314c0 con 0x5643ddd293a0 2024-05-22T23:01:17.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.357+0000 7eff1a1f8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5643ddc153f0 con 0x5643ddd289b0 2024-05-22T23:01:17.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.357+0000 7eff1a1f8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5643ddd31640 con 0x5643ddd2e510 2024-05-22T23:01:17.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.357+0000 7eff1a1f8740 10 client.0 _ll_register_callbacks cb 0x5643ddc4afe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:01:17.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.357+0000 7eff1528a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd293a0 0x5643ddd332e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:17.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.357+0000 7eff1528a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd293a0 0x5643ddd332e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52714/0 (socket says 172.21.15.84:52714) 2024-05-22T23:01:17.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.357+0000 7eff1528a640 1 -- 172.21.15.84:0/630131104 learned_addr learned my addr 172.21.15.84:0/630131104 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:17.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.358+0000 7eff14a89640 1 --2- 172.21.15.84:0/630131104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ddd289b0 0x5643ddd32da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:17.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.358+0000 7eff14a89640 1 --2- 172.21.15.84:0/630131104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643ddd2e510 0x5643ddd3a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:17.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.358+0000 7eff1528a640 1 -- 172.21.15.84:0/630131104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643ddd2e510 msgr2=0x5643ddd3a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:17.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.358+0000 7eff1528a640 1 --2- 172.21.15.84:0/630131104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643ddd2e510 0x5643ddd3a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.358+0000 7eff1528a640 1 -- 172.21.15.84:0/630131104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ddd289b0 msgr2=0x5643ddd32da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:17.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.358+0000 7eff1528a640 1 --2- 172.21.15.84:0/630131104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ddd289b0 0x5643ddd32da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.358+0000 7eff1528a640 1 -- 172.21.15.84:0/630131104 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643dda696b0 con 0x5643ddd293a0 2024-05-22T23:01:17.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.358+0000 7eff14a89640 1 --2- 172.21.15.84:0/630131104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ddd289b0 0x5643ddd32da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:01:17.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.358+0000 7eff1528a640 1 --2- 172.21.15.84:0/630131104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd293a0 0x5643ddd332e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7eff04008010 tx=0x7eff04011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5221fb9121c4196 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:17.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.358+0000 7efef8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:01:17.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.358+0000 7efef8ff9640 1 -- 172.21.15.84:0/630131104 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff04020070 con 0x5643ddd293a0 2024-05-22T23:01:17.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.358+0000 7efef8ff9640 1 -- 172.21.15.84:0/630131104 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7eff0401a660 con 0x5643ddd293a0 2024-05-22T23:01:17.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.358+0000 7eff1a1f8740 1 -- 172.21.15.84:0/630131104 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff0400cc00 con 0x5643ddd293a0 2024-05-22T23:01:17.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.358+0000 7efef8ff9640 1 -- 172.21.15.84:0/630131104 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7eff04015040 con 0x5643ddd293a0 2024-05-22T23:01:17.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.358+0000 7eff1a1f8740 10 client.15030 fetch_fsmap learned FSMap version 18 2024-05-22T23:01:17.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.358+0000 7eff1a1f8740 1 -- 172.21.15.84:0/630131104 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5643ddd3d600 con 0x5643ddd293a0 2024-05-22T23:01:17.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.358+0000 7efef8ff9640 1 -- 172.21.15.84:0/630131104 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff0400cc00 con 0x5643ddd293a0 2024-05-22T23:01:17.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.358+0000 7efef8ff9640 1 -- 172.21.15.84:0/630131104 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7eff0402e8c0 con 0x5643ddd293a0 2024-05-22T23:01:17.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.358+0000 7eff1a1f8740 10 client.15030 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:01:17.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.358+0000 7eff1a1f8740 10 client.15030 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-22T23:01:17.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.359+0000 7eff1a1f8740 20 client.15030 populate_metadata read hostname 'smithi084' 2024-05-22T23:01:17.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.359+0000 7eff1a1f8740 1 --2- 172.21.15.84:0/630131104 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5643ddd3f890 0x5643ddd40d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:17.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.359+0000 7eff1a1f8740 4 client.15030 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:01:17.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.359+0000 7eff1a1f8740 1 -- 172.21.15.84:0/630131104 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643ddc98db0 con 0x5643ddd3f890 2024-05-22T23:01:17.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.359+0000 7eff14a89640 1 --2- 172.21.15.84:0/630131104 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5643ddd3f890 0x5643ddd40d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:17.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.359+0000 7eff14a89640 1 --2- 172.21.15.84:0/630131104 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5643ddd3f890 0x5643ddd40d00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e235f2c2fef640f server_cookie=8409fa8a46195cdf in_seq=0 out_seq=0 2024-05-22T23:01:17.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.359+0000 7efef8ff9640 10 client.15030 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:01:17.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.360+0000 7efef8ff9640 1 -- 172.21.15.84:0/630131104 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5643ddc98db0 con 0x5643ddd3f890 2024-05-22T23:01:17.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.360+0000 7efef8ff9640 10 client.15030 handle_command_reply: tid=0 2024-05-22T23:01:17.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.360+0000 7eff1a1f8740 1 client.15030 shutdown 2024-05-22T23:01:17.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.360+0000 7eff1a1f8740 1 -- 172.21.15.84:0/630131104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd293a0 msgr2=0x5643ddd332e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:17.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.360+0000 7eff1a1f8740 1 --2- 172.21.15.84:0/630131104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd293a0 0x5643ddd332e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7eff04008010 tx=0x7eff04011bc0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.360+0000 7eff1a1f8740 1 -- 172.21.15.84:0/630131104 shutdown_connections 2024-05-22T23:01:17.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.360+0000 7eff1a1f8740 1 --2- 172.21.15.84:0/630131104 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5643ddd3f890 0x5643ddd40d00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.360+0000 7eff1a1f8740 1 --2- 172.21.15.84:0/630131104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643ddd2e510 0x5643ddd3a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.360+0000 7efef8ff9640 0 client.15030 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:01:17.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.360+0000 7eff1a1f8740 1 --2- 172.21.15.84:0/630131104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd293a0 0x5643ddd332e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.360+0000 7eff1a1f8740 1 --2- 172.21.15.84:0/630131104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ddd289b0 0x5643ddd32da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.360+0000 7eff1a1f8740 1 -- 172.21.15.84:0/630131104 >> 172.21.15.84:0/630131104 conn(0x5643ddd24020 msgr2=0x5643ddd23180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:17.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.360+0000 7eff1a1f8740 1 -- 172.21.15.84:0/630131104 shutdown_connections 2024-05-22T23:01:17.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.361+0000 7eff1a1f8740 1 -- 172.21.15.84:0/630131104 wait complete. 2024-05-22T23:01:17.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.361+0000 7eff1a1f8740 20 client.15030 trim_cache size 0 max 16384 2024-05-22T23:01:17.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.379+0000 7eff1a1f8740 1 Processor -- start 2024-05-22T23:01:17.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.379+0000 7eff1a1f8740 1 -- start start 2024-05-22T23:01:17.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.379+0000 7eff1a1f8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ddd289b0 0x5643ddd24400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:17.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.379+0000 7eff1a1f8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643ddd293a0 0x5643ddd3a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:17.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.379+0000 7eff1a1f8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd2e510 0x5643ddd26ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:17.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.379+0000 7eff1a1f8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5643ddd3f040 con 0x5643ddd2e510 2024-05-22T23:01:17.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.379+0000 7eff1a1f8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5643ddd3f710 con 0x5643ddd289b0 2024-05-22T23:01:17.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.379+0000 7eff1a1f8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5643ddd3fde0 con 0x5643ddd293a0 2024-05-22T23:01:17.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.379+0000 7eff14a89640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd2e510 0x5643ddd26ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:17.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.379+0000 7eff14a89640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd2e510 0x5643ddd26ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52728/0 (socket says 172.21.15.84:52728) 2024-05-22T23:01:17.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.379+0000 7eff14a89640 1 -- 172.21.15.84:0/2854864891 learned_addr learned my addr 172.21.15.84:0/2854864891 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:17.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.379+0000 7eff14a89640 1 -- 172.21.15.84:0/2854864891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643ddd293a0 msgr2=0x5643ddd3a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:17.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.379+0000 7eff14a89640 1 --2- 172.21.15.84:0/2854864891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643ddd293a0 0x5643ddd3a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.379+0000 7eff14a89640 1 -- 172.21.15.84:0/2854864891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ddd289b0 msgr2=0x5643ddd24400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:17.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.379+0000 7eff14a89640 1 --2- 172.21.15.84:0/2854864891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ddd289b0 0x5643ddd24400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.379+0000 7eff14a89640 1 -- 172.21.15.84:0/2854864891 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643ddd29780 con 0x5643ddd2e510 2024-05-22T23:01:17.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.379+0000 7eff14a89640 1 --2- 172.21.15.84:0/2854864891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd2e510 0x5643ddd26ed0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7eff0c000c00 tx=0x7eff0c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ed2985bccf54f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:17.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.379+0000 7efefaffd640 1 -- 172.21.15.84:0/2854864891 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c048020 con 0x5643ddd2e510 2024-05-22T23:01:17.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.379+0000 7efefaffd640 1 -- 172.21.15.84:0/2854864891 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff0c044d50 con 0x5643ddd2e510 2024-05-22T23:01:17.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.379+0000 7efefaffd640 1 -- 172.21.15.84:0/2854864891 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c035040 con 0x5643ddd2e510 2024-05-22T23:01:17.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.380+0000 7eff1a1f8740 1 -- 172.21.15.84:0/2854864891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd2e510 msgr2=0x5643ddd26ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:17.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.380+0000 7eff1a1f8740 1 --2- 172.21.15.84:0/2854864891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd2e510 0x5643ddd26ed0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7eff0c000c00 tx=0x7eff0c02a410 comp rx=0 tx=0).stop 2024-05-22T23:01:17.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.380+0000 7eff1a1f8740 1 -- 172.21.15.84:0/2854864891 shutdown_connections 2024-05-22T23:01:17.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.380+0000 7eff1a1f8740 1 --2- 172.21.15.84:0/2854864891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd2e510 0x5643ddd26ed0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.380+0000 7eff1a1f8740 1 --2- 172.21.15.84:0/2854864891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643ddd293a0 0x5643ddd3a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.380+0000 7eff1a1f8740 1 --2- 172.21.15.84:0/2854864891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ddd289b0 0x5643ddd24400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.380+0000 7eff1a1f8740 1 -- 172.21.15.84:0/2854864891 >> 172.21.15.84:0/2854864891 conn(0x5643ddd24020 msgr2=0x5643ddd1ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:17.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.380+0000 7eff1a1f8740 1 -- 172.21.15.84:0/2854864891 shutdown_connections 2024-05-22T23:01:17.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.380+0000 7eff1a1f8740 1 -- 172.21.15.84:0/2854864891 wait complete. 2024-05-22T23:01:17.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.380+0000 7eff1a1f8740 1 -- start start 2024-05-22T23:01:17.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.380+0000 7eff1a1f8740 1 Processor -- start 2024-05-22T23:01:17.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.381+0000 7eff1a1f8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd289b0 0x5643ddd36510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:17.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.381+0000 7eff1a1f8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ddd293a0 0x5643ddd36a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:17.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.381+0000 7eff1a1f8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643ddd2e510 0x5643ddd39620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:17.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.381+0000 7eff1a1f8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5643ddd40b80 con 0x5643ddd289b0 2024-05-22T23:01:17.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.381+0000 7eff1a1f8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5643ddd404b0 con 0x5643ddd293a0 2024-05-22T23:01:17.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.381+0000 7eff1a1f8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5643ddd41250 con 0x5643ddd2e510 2024-05-22T23:01:17.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.381+0000 7eff14a89640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd289b0 0x5643ddd36510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:17.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.381+0000 7eff1a1f8740 10 client.0 _ll_register_callbacks cb 0x7eff0400a7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:01:17.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.381+0000 7eff14a89640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd289b0 0x5643ddd36510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52732/0 (socket says 172.21.15.84:52732) 2024-05-22T23:01:17.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.381+0000 7eff14a89640 1 -- 172.21.15.84:0/1069674841 learned_addr learned my addr 172.21.15.84:0/1069674841 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:17.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.381+0000 7eff14a89640 1 --2- 172.21.15.84:0/1069674841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643ddd2e510 0x5643ddd39620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:17.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.381+0000 7eff14a89640 1 -- 172.21.15.84:0/1069674841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643ddd2e510 msgr2=0x5643ddd39620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:17.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.381+0000 7eff14a89640 1 --2- 172.21.15.84:0/1069674841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643ddd2e510 0x5643ddd39620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.381+0000 7eff14a89640 1 -- 172.21.15.84:0/1069674841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ddd293a0 msgr2=0x5643ddd36a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:17.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.381+0000 7eff14a89640 1 --2- 172.21.15.84:0/1069674841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ddd293a0 0x5643ddd36a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.381+0000 7eff14a89640 1 -- 172.21.15.84:0/1069674841 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643ddd39c10 con 0x5643ddd289b0 2024-05-22T23:01:17.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.381+0000 7eff14a89640 1 --2- 172.21.15.84:0/1069674841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd289b0 0x5643ddd36510 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009850 tx=0x7eff0c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe3ef18db3f57c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:17.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.381+0000 7efef8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:01:17.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.381+0000 7efef8ff9640 1 -- 172.21.15.84:0/1069674841 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c055020 con 0x5643ddd289b0 2024-05-22T23:01:17.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.381+0000 7efef8ff9640 1 -- 172.21.15.84:0/1069674841 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7eff0c0035f0 con 0x5643ddd289b0 2024-05-22T23:01:17.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.381+0000 7eff1a1f8740 1 -- 172.21.15.84:0/1069674841 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff0c044d50 con 0x5643ddd289b0 2024-05-22T23:01:17.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.382+0000 7efef8ff9640 1 -- 172.21.15.84:0/1069674841 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7eff0c035070 con 0x5643ddd289b0 2024-05-22T23:01:17.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.382+0000 7eff1a1f8740 10 client.15036 fetch_fsmap learned FSMap version 18 2024-05-22T23:01:17.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.382+0000 7eff1a1f8740 1 -- 172.21.15.84:0/1069674841 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5643ddda5e90 con 0x5643ddd289b0 2024-05-22T23:01:17.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.382+0000 7efef8ff9640 1 -- 172.21.15.84:0/1069674841 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff0c044d50 con 0x5643ddd289b0 2024-05-22T23:01:17.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.382+0000 7efef8ff9640 1 -- 172.21.15.84:0/1069674841 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7eff0c04e950 con 0x5643ddd289b0 2024-05-22T23:01:17.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.382+0000 7eff1a1f8740 10 client.15036 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:01:17.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.382+0000 7eff1a1f8740 10 client.15036 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-22T23:01:17.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.382+0000 7eff1a1f8740 20 client.15036 populate_metadata read hostname 'smithi084' 2024-05-22T23:01:17.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.382+0000 7eff1a1f8740 1 --2- 172.21.15.84:0/1069674841 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5643dddaa630 0x5643ddda9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:17.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.382+0000 7eff1a1f8740 4 client.15036 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-22T23:01:17.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.382+0000 7eff1a1f8740 1 -- 172.21.15.84:0/1069674841 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "status"}) -- 0x5643ddc93100 con 0x5643dddaa630 2024-05-22T23:01:17.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.382+0000 7eff1528a640 1 --2- 172.21.15.84:0/1069674841 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5643dddaa630 0x5643ddda9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:17.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.383+0000 7efef8ff9640 10 client.15036 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:01:17.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.383+0000 7eff1528a640 1 --2- 172.21.15.84:0/1069674841 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5643dddaa630 0x5643ddda9340 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a91dfe713be21a53 server_cookie=efb7e86667196928 in_seq=0 out_seq=0 2024-05-22T23:01:17.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.383+0000 7efef8ff9640 1 -- 172.21.15.84:0/1069674841 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5643ddc93100 con 0x5643dddaa630 2024-05-22T23:01:17.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.383+0000 7efef8ff9640 10 client.15036 handle_command_reply: tid=0 2024-05-22T23:01:17.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.383+0000 7eff1a1f8740 1 client.15036 shutdown 2024-05-22T23:01:17.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.383+0000 7eff1a1f8740 1 -- 172.21.15.84:0/1069674841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd289b0 msgr2=0x5643ddd36510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:17.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.383+0000 7eff1a1f8740 1 --2- 172.21.15.84:0/1069674841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd289b0 0x5643ddd36510 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009850 tx=0x7eff0c031400 comp rx=0 tx=0).stop 2024-05-22T23:01:17.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.383+0000 7eff1a1f8740 1 -- 172.21.15.84:0/1069674841 shutdown_connections 2024-05-22T23:01:17.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.383+0000 7eff1a1f8740 1 --2- 172.21.15.84:0/1069674841 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5643dddaa630 0x5643ddda9340 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.383+0000 7eff1a1f8740 1 --2- 172.21.15.84:0/1069674841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643ddd2e510 0x5643ddd39620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.383+0000 7efef8ff9640 0 client.15036 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:01:17.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.383+0000 7eff1a1f8740 1 --2- 172.21.15.84:0/1069674841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ddd293a0 0x5643ddd36a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.383+0000 7eff1a1f8740 1 --2- 172.21.15.84:0/1069674841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643ddd289b0 0x5643ddd36510 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.383+0000 7eff1a1f8740 1 -- 172.21.15.84:0/1069674841 >> 172.21.15.84:0/1069674841 conn(0x5643ddd24020 msgr2=0x5643ddd24400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:17.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.383+0000 7eff1a1f8740 1 -- 172.21.15.84:0/1069674841 shutdown_connections 2024-05-22T23:01:17.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.383+0000 7eff1a1f8740 1 -- 172.21.15.84:0/1069674841 wait complete. 2024-05-22T23:01:17.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.383+0000 7eff1a1f8740 20 client.15036 trim_cache size 0 max 16384 2024-05-22T23:01:17.392 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:01:17.392 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "c1135a12-188d-11ef-bc9a-c7b262605968", 2024-05-22T23:01:17.392 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2024-05-22T23:01:17.392 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 24530, 2024-05-22T23:01:17.392 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2024-05-22T23:01:17.392 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2024-05-22T23:01:17.393 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2024-05-22T23:01:17.393 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 84.797576368999998, 2024-05-22T23:01:17.393 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 18, 2024-05-22T23:01:17.393 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 88, 2024-05-22T23:01:17.393 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 86, 2024-05-22T23:01:17.393 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 91.418767090000003 2024-05-22T23:01:17.393 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:01:17.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.384+0000 7eff174ee640 1 -- 172.21.15.84:0/1612450417 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7efefc0a2b90 msgr2=0x7efefc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:17.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.384+0000 7eff174ee640 1 --2- 172.21.15.84:0/1612450417 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7efefc0a2b90 0x7efefc0a5050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7eff08005910 tx=0x7eff0806ec60 comp rx=0 tx=0).stop 2024-05-22T23:01:17.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.384+0000 7eff174ee640 1 -- 172.21.15.84:0/1612450417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff10102380 msgr2=0x7eff1019f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:17.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.384+0000 7eff174ee640 1 --2- 172.21.15.84:0/1612450417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff10102380 0x7eff1019f210 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7eff0800eaa0 tx=0x7eff0800ef60 comp rx=0 tx=0).stop 2024-05-22T23:01:17.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.384+0000 7eff174ee640 1 -- 172.21.15.84:0/1612450417 shutdown_connections 2024-05-22T23:01:17.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.384+0000 7eff174ee640 1 --2- 172.21.15.84:0/1612450417 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7efefc0a2b90 0x7efefc0a5050 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.384+0000 7eff174ee640 1 --2- 172.21.15.84:0/1612450417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff10102380 0x7eff1019f210 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.384+0000 7eff174ee640 1 --2- 172.21.15.84:0/1612450417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff101019c0 0x7eff101981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.384+0000 7eff174ee640 1 --2- 172.21.15.84:0/1612450417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff101007d0 0x7eff10197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:17.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.384+0000 7eff174ee640 1 -- 172.21.15.84:0/1612450417 >> 172.21.15.84:0/1612450417 conn(0x7eff100fbd60 msgr2=0x7eff10104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:17.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.384+0000 7eff174ee640 1 -- 172.21.15.84:0/1612450417 shutdown_connections 2024-05-22T23:01:17.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:17.384+0000 7eff174ee640 1 -- 172.21.15.84:0/1612450417 wait complete. 2024-05-22T23:01:17.396 INFO:tasks.cephadm:Setup complete, yielding 2024-05-22T23:01:17.396 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-22T23:01:17.408 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi084.front.sepia.ceph.com 2024-05-22T23:01:17.409 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch status' 2024-05-22T23:01:17.649 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:01:17.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:17 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/453039049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:01:17.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:17 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2559597590' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:17.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:17 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/4013548375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:01:17.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:17 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/453039049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:01:17.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:17 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2559597590' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:17.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:17 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/4013548375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:01:18.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:17 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/453039049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:01:18.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:17 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2559597590' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:18.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:17 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/4013548375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:01:18.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:18 smithi084 ceph-mon[32186]: pgmap v334: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-22T23:01:18.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:18 smithi204 ceph-mon[36211]: pgmap v334: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-22T23:01:19.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:18 smithi096 ceph-mon[36392]: pgmap v334: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-22T23:01:19.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.269+0000 7f04e51d0700 1 -- 172.21.15.84:0/4167214368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e0102e60 msgr2=0x7f04e0109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:19.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.269+0000 7f04e51d0700 1 --2- 172.21.15.84:0/4167214368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e0102e60 0x7f04e0109710 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f04c80099c0 tx=0x7f04c8009cd0 comp rx=0 tx=0).stop 2024-05-22T23:01:19.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.269+0000 7f04e51d0700 1 -- 172.21.15.84:0/4167214368 shutdown_connections 2024-05-22T23:01:19.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.269+0000 7f04e51d0700 1 --2- 172.21.15.84:0/4167214368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e0102e60 0x7f04e0109710 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:19.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.269+0000 7f04e51d0700 1 --2- 172.21.15.84:0/4167214368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04e01012c0 0x7f04e01016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:19.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.269+0000 7f04e51d0700 1 --2- 172.21.15.84:0/4167214368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04e01024c0 0x7f04e0102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:19.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.269+0000 7f04e51d0700 1 -- 172.21.15.84:0/4167214368 >> 172.21.15.84:0/4167214368 conn(0x7f04e00fc860 msgr2=0x7f04e00feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:19.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.269+0000 7f04e51d0700 1 -- 172.21.15.84:0/4167214368 shutdown_connections 2024-05-22T23:01:19.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.269+0000 7f04e51d0700 1 -- 172.21.15.84:0/4167214368 wait complete. 2024-05-22T23:01:19.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.270+0000 7f04e51d0700 1 Processor -- start 2024-05-22T23:01:19.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.270+0000 7f04e51d0700 1 -- start start 2024-05-22T23:01:19.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.271+0000 7f04e51d0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04e01012c0 0x7f04e0197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:19.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.271+0000 7f04e51d0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04e01024c0 0x7f04e0197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:19.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.271+0000 7f04e51d0700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e0102e60 0x7f04e019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:19.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.271+0000 7f04e51d0700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f04e010cfa0 con 0x7f04e0102e60 2024-05-22T23:01:19.275 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.271+0000 7f04e51d0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f04e010ce20 con 0x7f04e01024c0 2024-05-22T23:01:19.275 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.271+0000 7f04e51d0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f04e010d120 con 0x7f04e01012c0 2024-05-22T23:01:19.275 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.271+0000 7f04df59e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e0102e60 0x7f04e019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:19.275 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.271+0000 7f04df59e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e0102e60 0x7f04e019ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52746/0 (socket says 172.21.15.84:52746) 2024-05-22T23:01:19.275 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.271+0000 7f04df59e700 1 -- 172.21.15.84:0/3982147507 learned_addr learned my addr 172.21.15.84:0/3982147507 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:19.275 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.271+0000 7f04de59c700 1 --2- 172.21.15.84:0/3982147507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04e01024c0 0x7f04e0197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:19.275 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.271+0000 7f04ded9d700 1 --2- 172.21.15.84:0/3982147507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04e01012c0 0x7f04e0197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:19.275 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.271+0000 7f04df59e700 1 -- 172.21.15.84:0/3982147507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04e01012c0 msgr2=0x7f04e0197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:19.275 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.271+0000 7f04df59e700 1 --2- 172.21.15.84:0/3982147507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04e01012c0 0x7f04e0197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:19.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.271+0000 7f04df59e700 1 -- 172.21.15.84:0/3982147507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04e01024c0 msgr2=0x7f04e0197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:19.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.271+0000 7f04df59e700 1 --2- 172.21.15.84:0/3982147507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04e01024c0 0x7f04e0197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:19.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.271+0000 7f04df59e700 1 -- 172.21.15.84:0/3982147507 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e019f260 con 0x7f04e0102e60 2024-05-22T23:01:19.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.271+0000 7f04ded9d700 1 --2- 172.21.15.84:0/3982147507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04e01012c0 0x7f04e0197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:01:19.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.271+0000 7f04df59e700 1 --2- 172.21.15.84:0/3982147507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e0102e60 0x7f04e019ebc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009990 tx=0x7f04c8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:19.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.271+0000 7f04d7fff700 1 -- 172.21.15.84:0/3982147507 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8026070 con 0x7f04e0102e60 2024-05-22T23:01:19.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.271+0000 7f04e51d0700 1 -- 172.21.15.84:0/3982147507 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e019f4f0 con 0x7f04e0102e60 2024-05-22T23:01:19.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.272+0000 7f04d7fff700 1 -- 172.21.15.84:0/3982147507 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c8004150 con 0x7f04e0102e60 2024-05-22T23:01:19.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.272+0000 7f04d7fff700 1 -- 172.21.15.84:0/3982147507 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8004b50 con 0x7f04e0102e60 2024-05-22T23:01:19.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.272+0000 7f04e51d0700 1 -- 172.21.15.84:0/3982147507 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e006a040 con 0x7f04e0102e60 2024-05-22T23:01:19.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.273+0000 7f04e51d0700 1 -- 172.21.15.84:0/3982147507 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e0069050 con 0x7f04e0102e60 2024-05-22T23:01:19.277 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.274+0000 7f04d7fff700 1 -- 172.21.15.84:0/3982147507 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f04c8029070 con 0x7f04e0102e60 2024-05-22T23:01:19.278 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.274+0000 7f04d7fff700 1 --2- 172.21.15.84:0/3982147507 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f04cc0a2c20 0x7f04cc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:19.278 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.274+0000 7f04ded9d700 1 --2- 172.21.15.84:0/3982147507 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f04cc0a2c20 0x7f04cc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:19.278 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.274+0000 7f04d7fff700 1 -- 172.21.15.84:0/3982147507 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f04c80cfc40 con 0x7f04e0102e60 2024-05-22T23:01:19.278 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.275+0000 7f04ded9d700 1 --2- 172.21.15.84:0/3982147507 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f04cc0a2c20 0x7f04cc0a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f04e0102320 tx=0x7f04d0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:19.280 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.279+0000 7f04d7fff700 1 -- 172.21.15.84:0/3982147507 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f04c805d850 con 0x7f04e0102e60 2024-05-22T23:01:19.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.492+0000 7f04e51d0700 1 -- 172.21.15.84:0/3982147507 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f04e006a360 con 0x7f04cc0a2c20 2024-05-22T23:01:19.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.493+0000 7f04d7fff700 1 -- 172.21.15.84:0/3982147507 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f04e006a360 con 0x7f04cc0a2c20 2024-05-22T23:01:19.495 INFO:teuthology.orchestra.run.smithi084.stdout:Backend: cephadm 2024-05-22T23:01:19.495 INFO:teuthology.orchestra.run.smithi084.stdout:Available: Yes 2024-05-22T23:01:19.495 INFO:teuthology.orchestra.run.smithi084.stdout:Paused: No 2024-05-22T23:01:19.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.496+0000 7f04e51d0700 1 -- 172.21.15.84:0/3982147507 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f04cc0a2c20 msgr2=0x7f04cc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:19.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.496+0000 7f04e51d0700 1 --2- 172.21.15.84:0/3982147507 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f04cc0a2c20 0x7f04cc0a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f04e0102320 tx=0x7f04d0008040 comp rx=0 tx=0).stop 2024-05-22T23:01:19.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.496+0000 7f04e51d0700 1 -- 172.21.15.84:0/3982147507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e0102e60 msgr2=0x7f04e019ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:19.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.496+0000 7f04e51d0700 1 --2- 172.21.15.84:0/3982147507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e0102e60 0x7f04e019ebc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009990 tx=0x7f04c8005ef0 comp rx=0 tx=0).stop 2024-05-22T23:01:19.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.496+0000 7f04e51d0700 1 -- 172.21.15.84:0/3982147507 shutdown_connections 2024-05-22T23:01:19.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.496+0000 7f04e51d0700 1 --2- 172.21.15.84:0/3982147507 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f04cc0a2c20 0x7f04cc0a50e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:19.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.496+0000 7f04e51d0700 1 --2- 172.21.15.84:0/3982147507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e0102e60 0x7f04e019ebc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:19.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.496+0000 7f04e51d0700 1 --2- 172.21.15.84:0/3982147507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04e01012c0 0x7f04e0197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:19.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.496+0000 7f04e51d0700 1 --2- 172.21.15.84:0/3982147507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04e01024c0 0x7f04e0197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:19.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.496+0000 7f04e51d0700 1 -- 172.21.15.84:0/3982147507 >> 172.21.15.84:0/3982147507 conn(0x7f04e00fc860 msgr2=0x7f04e0105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:19.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.496+0000 7f04e51d0700 1 -- 172.21.15.84:0/3982147507 shutdown_connections 2024-05-22T23:01:19.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:19.496+0000 7f04e51d0700 1 -- 172.21.15.84:0/3982147507 wait complete. 2024-05-22T23:01:20.294 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-22T23:01:20.596 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:01:20.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:20 smithi084 ceph-mon[32186]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:20.866 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:20 smithi084 ceph-mon[32186]: pgmap v335: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-05-22T23:01:20.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:20 smithi204 ceph-mon[36211]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:20.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:20 smithi204 ceph-mon[36211]: pgmap v335: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-05-22T23:01:21.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:20 smithi096 ceph-mon[36392]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:21.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:20 smithi096 ceph-mon[36392]: pgmap v335: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-05-22T23:01:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.397+0000 7fb107177700 1 -- 172.21.15.84:0/3383075369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb100108150 msgr2=0x7fb10010a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.397+0000 7fb107177700 1 --2- 172.21.15.84:0/3383075369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb100108150 0x7fb10010a540 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800b0a0 tx=0x7fb0f800b3b0 comp rx=0 tx=0).stop 2024-05-22T23:01:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.397+0000 7fb107177700 1 -- 172.21.15.84:0/3383075369 shutdown_connections 2024-05-22T23:01:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.397+0000 7fb107177700 1 --2- 172.21.15.84:0/3383075369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb100108150 0x7fb10010a540 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.397+0000 7fb107177700 1 --2- 172.21.15.84:0/3383075369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb100102ef0 0x7fb1001052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.397+0000 7fb107177700 1 --2- 172.21.15.84:0/3383075369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb100105820 0x7fb100107c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.397+0000 7fb107177700 1 -- 172.21.15.84:0/3383075369 >> 172.21.15.84:0/3383075369 conn(0x7fb1000fc860 msgr2=0x7fb1000feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.397+0000 7fb107177700 1 -- 172.21.15.84:0/3383075369 shutdown_connections 2024-05-22T23:01:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.397+0000 7fb107177700 1 -- 172.21.15.84:0/3383075369 wait complete. 2024-05-22T23:01:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.398+0000 7fb107177700 1 Processor -- start 2024-05-22T23:01:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.398+0000 7fb107177700 1 -- start start 2024-05-22T23:01:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.398+0000 7fb107177700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb100102ef0 0x7fb1001985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.398+0000 7fb107177700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb100105820 0x7fb100198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.398+0000 7fb107177700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb100108150 0x7fb10019fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.398+0000 7fb107177700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb10010dff0 con 0x7fb100108150 2024-05-22T23:01:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.398+0000 7fb107177700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb10010de70 con 0x7fb100105820 2024-05-22T23:01:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.398+0000 7fb107177700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb10010e170 con 0x7fb100102ef0 2024-05-22T23:01:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.398+0000 7fb105714700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb100108150 0x7fb10019fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.398+0000 7fb105714700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb100108150 0x7fb10019fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50084/0 (socket says 172.21.15.84:50084) 2024-05-22T23:01:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.398+0000 7fb105714700 1 -- 172.21.15.84:0/2011139835 learned_addr learned my addr 172.21.15.84:0/2011139835 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.398+0000 7fb104f13700 1 --2- 172.21.15.84:0/2011139835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb100102ef0 0x7fb1001985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.398+0000 7fb0f7fff700 1 --2- 172.21.15.84:0/2011139835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb100105820 0x7fb100198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.398+0000 7fb105714700 1 -- 172.21.15.84:0/2011139835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb100102ef0 msgr2=0x7fb1001985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.398+0000 7fb105714700 1 --2- 172.21.15.84:0/2011139835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb100102ef0 0x7fb1001985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.398+0000 7fb105714700 1 -- 172.21.15.84:0/2011139835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb100105820 msgr2=0x7fb100198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.398+0000 7fb105714700 1 --2- 172.21.15.84:0/2011139835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb100105820 0x7fb100198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.398+0000 7fb105714700 1 -- 172.21.15.84:0/2011139835 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1001a0240 con 0x7fb100108150 2024-05-22T23:01:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.398+0000 7fb0f7fff700 1 --2- 172.21.15.84:0/2011139835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb100105820 0x7fb100198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:01:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.398+0000 7fb105714700 1 --2- 172.21.15.84:0/2011139835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb100108150 0x7fb10019fba0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800d010 tx=0x7fb0f8004e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:22.403 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.399+0000 7fb0f5ffb700 1 -- 172.21.15.84:0/2011139835 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8021070 con 0x7fb100108150 2024-05-22T23:01:22.403 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.399+0000 7fb0f5ffb700 1 -- 172.21.15.84:0/2011139835 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0f8007d00 con 0x7fb100108150 2024-05-22T23:01:22.403 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.399+0000 7fb0f5ffb700 1 -- 172.21.15.84:0/2011139835 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f801d6b0 con 0x7fb100108150 2024-05-22T23:01:22.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.399+0000 7fb107177700 1 -- 172.21.15.84:0/2011139835 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1001a04d0 con 0x7fb100108150 2024-05-22T23:01:22.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.399+0000 7fb107177700 1 -- 172.21.15.84:0/2011139835 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1001a09d0 con 0x7fb100108150 2024-05-22T23:01:22.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.400+0000 7fb107177700 1 -- 172.21.15.84:0/2011139835 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb100069050 con 0x7fb100108150 2024-05-22T23:01:22.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.401+0000 7fb0f5ffb700 1 -- 172.21.15.84:0/2011139835 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb0f8004220 con 0x7fb100108150 2024-05-22T23:01:22.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.402+0000 7fb0f5ffb700 1 --2- 172.21.15.84:0/2011139835 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb0e80a2ca0 0x7fb0e80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:22.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.402+0000 7fb104f13700 1 --2- 172.21.15.84:0/2011139835 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb0e80a2ca0 0x7fb0e80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:22.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.402+0000 7fb0f5ffb700 1 -- 172.21.15.84:0/2011139835 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fb0f800f020 con 0x7fb100108150 2024-05-22T23:01:22.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.402+0000 7fb104f13700 1 --2- 172.21.15.84:0/2011139835 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb0e80a2ca0 0x7fb0e80a5160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc005950 tx=0x7fb0fc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:22.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.405+0000 7fb0f5ffb700 1 -- 172.21.15.84:0/2011139835 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb0f802a020 con 0x7fb100108150 2024-05-22T23:01:22.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.616+0000 7fb107177700 1 -- 172.21.15.84:0/2011139835 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fb1000627a0 con 0x7fb0e80a2ca0 2024-05-22T23:01:22.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.629+0000 7fb0f5ffb700 1 -- 172.21.15.84:0/2011139835 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7fb1000627a0 con 0x7fb0e80a2ca0 2024-05-22T23:01:22.631 INFO:teuthology.orchestra.run.smithi084.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-22T23:01:22.631 INFO:teuthology.orchestra.run.smithi084.stdout:mds.a smithi084 running (99s) 17s ago 99s 16.2M - 19.0.0-2765-g2e1f7622 f0ae8441b76c d06a7e93c3c9 2024-05-22T23:01:22.631 INFO:teuthology.orchestra.run.smithi084.stdout:mds.b smithi096 running (97s) 30s ago 96s 28.1M - 19.0.0-2765-g2e1f7622 f0ae8441b76c e4324ef0d993 2024-05-22T23:01:22.631 INFO:teuthology.orchestra.run.smithi084.stdout:mds.c smithi204 running (111s) 30s ago 110s 15.3M - 19.0.0-2765-g2e1f7622 f0ae8441b76c c208e68ab1ed 2024-05-22T23:01:22.631 INFO:teuthology.orchestra.run.smithi084.stdout:mds.d smithi084 running (2m) 17s ago 119s 13.8M - 19.0.0-2765-g2e1f7622 f0ae8441b76c 275f492e41f0 2024-05-22T23:01:22.631 INFO:teuthology.orchestra.run.smithi084.stdout:mds.e smithi096 running (117s) 30s ago 116s 13.0M - 19.0.0-2765-g2e1f7622 f0ae8441b76c 61c5842649bc 2024-05-22T23:01:22.631 INFO:teuthology.orchestra.run.smithi084.stdout:mds.f smithi204 running (114s) 30s ago 113s 15.3M - 19.0.0-2765-g2e1f7622 f0ae8441b76c a1ab7dd8ec58 2024-05-22T23:01:22.631 INFO:teuthology.orchestra.run.smithi084.stdout:mds.g smithi084 running (102s) 17s ago 102s 13.1M - 19.0.0-2765-g2e1f7622 f0ae8441b76c dfa016465ba0 2024-05-22T23:01:22.631 INFO:teuthology.orchestra.run.smithi084.stdout:mds.h smithi096 running (108s) 30s ago 107s 17.8M - 19.0.0-2765-g2e1f7622 f0ae8441b76c 1c2d2eeaf388 2024-05-22T23:01:22.631 INFO:teuthology.orchestra.run.smithi084.stdout:mds.i smithi204 running (105s) 30s ago 104s 13.9M - 19.0.0-2765-g2e1f7622 f0ae8441b76c 81b84a23d555 2024-05-22T23:01:22.631 INFO:teuthology.orchestra.run.smithi084.stdout:mds.j smithi084 running (2m) 17s ago 2m 16.2M - 19.0.0-2765-g2e1f7622 f0ae8441b76c af78cff96490 2024-05-22T23:01:22.631 INFO:teuthology.orchestra.run.smithi084.stdout:mds.k smithi096 running (2m) 30s ago 2m 12.9M - 19.0.0-2765-g2e1f7622 f0ae8441b76c a9999061cbc9 2024-05-22T23:01:22.632 INFO:teuthology.orchestra.run.smithi084.stdout:mds.l smithi204 running (2m) 30s ago 2m 13.9M - 19.0.0-2765-g2e1f7622 f0ae8441b76c ccde4371bca2 2024-05-22T23:01:22.632 INFO:teuthology.orchestra.run.smithi084.stdout:mgr.x smithi084 *:9283,8765 running (9m) 17s ago 9m 515M - 19.0.0-2765-g2e1f7622 f0ae8441b76c 17eaaed076d3 2024-05-22T23:01:22.632 INFO:teuthology.orchestra.run.smithi084.stdout:mgr.y smithi096 *:8443,8765 running (7m) 30s ago 7m 455M - 19.0.0-2765-g2e1f7622 f0ae8441b76c f4badc5353f9 2024-05-22T23:01:22.632 INFO:teuthology.orchestra.run.smithi084.stdout:mgr.z smithi204 *:8443,8765 running (7m) 30s ago 7m 454M - 19.0.0-2765-g2e1f7622 f0ae8441b76c f67fd07efabb 2024-05-22T23:01:22.632 INFO:teuthology.orchestra.run.smithi084.stdout:mon.a smithi084 running (9m) 17s ago 9m 50.0M 2048M 19.0.0-2765-g2e1f7622 f0ae8441b76c 16db2f34d801 2024-05-22T23:01:22.632 INFO:teuthology.orchestra.run.smithi084.stdout:mon.b smithi096 running (7m) 30s ago 7m 39.1M 2048M 19.0.0-2765-g2e1f7622 f0ae8441b76c 5d1e127aea7c 2024-05-22T23:01:22.632 INFO:teuthology.orchestra.run.smithi084.stdout:mon.c smithi204 running (7m) 30s ago 7m 38.1M 2048M 19.0.0-2765-g2e1f7622 f0ae8441b76c 2617a778f6ff 2024-05-22T23:01:22.632 INFO:teuthology.orchestra.run.smithi084.stdout:osd.0 smithi084 running (7m) 17s ago 7m 55.4M 4272M 19.0.0-2765-g2e1f7622 f0ae8441b76c 2b28939cd94e 2024-05-22T23:01:22.632 INFO:teuthology.orchestra.run.smithi084.stdout:osd.1 smithi096 running (6m) 30s ago 6m 55.2M 4272M 19.0.0-2765-g2e1f7622 f0ae8441b76c 17d2d24f431a 2024-05-22T23:01:22.632 INFO:teuthology.orchestra.run.smithi084.stdout:osd.2 smithi204 running (6m) 30s ago 6m 54.5M 4272M 19.0.0-2765-g2e1f7622 f0ae8441b76c 870ddf27a800 2024-05-22T23:01:22.632 INFO:teuthology.orchestra.run.smithi084.stdout:osd.3 smithi084 running (5m) 17s ago 5m 54.3M 4272M 19.0.0-2765-g2e1f7622 f0ae8441b76c fa8878b06a16 2024-05-22T23:01:22.632 INFO:teuthology.orchestra.run.smithi084.stdout:osd.4 smithi096 running (5m) 30s ago 5m 42.8M 4272M 19.0.0-2765-g2e1f7622 f0ae8441b76c 6c60197e2e00 2024-05-22T23:01:22.632 INFO:teuthology.orchestra.run.smithi084.stdout:osd.5 smithi204 running (5m) 30s ago 5m 54.2M 4272M 19.0.0-2765-g2e1f7622 f0ae8441b76c 695acd9c4cf1 2024-05-22T23:01:22.632 INFO:teuthology.orchestra.run.smithi084.stdout:osd.6 smithi084 running (4m) 17s ago 4m 42.2M 4272M 19.0.0-2765-g2e1f7622 f0ae8441b76c 833460ad6832 2024-05-22T23:01:22.632 INFO:teuthology.orchestra.run.smithi084.stdout:osd.7 smithi096 running (4m) 30s ago 4m 54.0M 4272M 19.0.0-2765-g2e1f7622 f0ae8441b76c f935196acd9b 2024-05-22T23:01:22.633 INFO:teuthology.orchestra.run.smithi084.stdout:osd.8 smithi204 running (3m) 30s ago 3m 53.4M 4272M 19.0.0-2765-g2e1f7622 f0ae8441b76c 67af32f59f3c 2024-05-22T23:01:22.633 INFO:teuthology.orchestra.run.smithi084.stdout:osd.9 smithi084 running (3m) 17s ago 3m 43.9M 4272M 19.0.0-2765-g2e1f7622 f0ae8441b76c 7d9f57beecc6 2024-05-22T23:01:22.633 INFO:teuthology.orchestra.run.smithi084.stdout:osd.10 smithi096 running (2m) 30s ago 2m 55.5M 4272M 19.0.0-2765-g2e1f7622 f0ae8441b76c 4d44d69606f2 2024-05-22T23:01:22.633 INFO:teuthology.orchestra.run.smithi084.stdout:osd.11 smithi204 running (2m) 30s ago 2m 54.0M 4272M 19.0.0-2765-g2e1f7622 f0ae8441b76c dab3c44ea774 2024-05-22T23:01:22.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.632+0000 7fb107177700 1 -- 172.21.15.84:0/2011139835 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb0e80a2ca0 msgr2=0x7fb0e80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:22.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.632+0000 7fb107177700 1 --2- 172.21.15.84:0/2011139835 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb0e80a2ca0 0x7fb0e80a5160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc005950 tx=0x7fb0fc0058e0 comp rx=0 tx=0).stop 2024-05-22T23:01:22.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.632+0000 7fb107177700 1 -- 172.21.15.84:0/2011139835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb100108150 msgr2=0x7fb10019fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:22.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.632+0000 7fb107177700 1 --2- 172.21.15.84:0/2011139835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb100108150 0x7fb10019fba0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800d010 tx=0x7fb0f8004e80 comp rx=0 tx=0).stop 2024-05-22T23:01:22.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.632+0000 7fb107177700 1 -- 172.21.15.84:0/2011139835 shutdown_connections 2024-05-22T23:01:22.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.632+0000 7fb107177700 1 --2- 172.21.15.84:0/2011139835 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb0e80a2ca0 0x7fb0e80a5160 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:22.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.632+0000 7fb107177700 1 --2- 172.21.15.84:0/2011139835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb100108150 0x7fb10019fba0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:22.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.632+0000 7fb107177700 1 --2- 172.21.15.84:0/2011139835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb100102ef0 0x7fb1001985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:22.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.632+0000 7fb107177700 1 --2- 172.21.15.84:0/2011139835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb100105820 0x7fb100198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:22.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.632+0000 7fb107177700 1 -- 172.21.15.84:0/2011139835 >> 172.21.15.84:0/2011139835 conn(0x7fb1000fc860 msgr2=0x7fb100103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:22.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.632+0000 7fb107177700 1 -- 172.21.15.84:0/2011139835 shutdown_connections 2024-05-22T23:01:22.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:22.632+0000 7fb107177700 1 -- 172.21.15.84:0/2011139835 wait complete. 2024-05-22T23:01:22.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:22 smithi084 ceph-mon[32186]: pgmap v336: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-22T23:01:22.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:22 smithi204 ceph-mon[36211]: pgmap v336: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-22T23:01:23.051 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:22 smithi096 ceph-mon[36392]: pgmap v336: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-22T23:01:23.366 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-22T23:01:23.610 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:01:24.866 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:24 smithi096 ceph-mon[36392]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:24.866 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:24 smithi096 ceph-mon[36392]: pgmap v337: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-22T23:01:24.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:24 smithi084 ceph-mon[32186]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:24.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:24 smithi084 ceph-mon[32186]: pgmap v337: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-22T23:01:24.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:24 smithi204 ceph-mon[36211]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:24.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:24 smithi204 ceph-mon[36211]: pgmap v337: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-22T23:01:25.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.452+0000 7f17fd7bc700 1 -- 172.21.15.84:0/806873010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17f81024c0 msgr2=0x7f17f8102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:25.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.452+0000 7f17fd7bc700 1 --2- 172.21.15.84:0/806873010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17f81024c0 0x7f17f8102920 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f17e80099c0 tx=0x7f17e8009cd0 comp rx=0 tx=0).stop 2024-05-22T23:01:25.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.452+0000 7f17fd7bc700 1 -- 172.21.15.84:0/806873010 shutdown_connections 2024-05-22T23:01:25.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.452+0000 7f17fd7bc700 1 --2- 172.21.15.84:0/806873010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17f8102e60 0x7f17f8109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:25.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.452+0000 7f17fd7bc700 1 --2- 172.21.15.84:0/806873010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17f81012c0 0x7f17f81016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:25.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.452+0000 7f17fd7bc700 1 --2- 172.21.15.84:0/806873010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17f81024c0 0x7f17f8102920 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:25.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.452+0000 7f17fd7bc700 1 -- 172.21.15.84:0/806873010 >> 172.21.15.84:0/806873010 conn(0x7f17f80fc860 msgr2=0x7f17f80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:25.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.452+0000 7f17fd7bc700 1 -- 172.21.15.84:0/806873010 shutdown_connections 2024-05-22T23:01:25.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.452+0000 7f17fd7bc700 1 -- 172.21.15.84:0/806873010 wait complete. 2024-05-22T23:01:25.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.453+0000 7f17fd7bc700 1 Processor -- start 2024-05-22T23:01:25.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.453+0000 7f17fd7bc700 1 -- start start 2024-05-22T23:01:25.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.453+0000 7f17fd7bc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17f81012c0 0x7f17f81986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:25.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.453+0000 7f17fd7bc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17f81024c0 0x7f17f8198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:25.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.453+0000 7f17fd7bc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17f8102e60 0x7f17f819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:25.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.453+0000 7f17fd7bc700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f17f810cf00 con 0x7f17f81024c0 2024-05-22T23:01:25.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.453+0000 7f17fd7bc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f17f810cd80 con 0x7f17f8102e60 2024-05-22T23:01:25.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.453+0000 7f17fd7bc700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f17f810d080 con 0x7f17f81012c0 2024-05-22T23:01:25.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.453+0000 7f17f67fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17f81024c0 0x7f17f8198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:25.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.453+0000 7f17f67fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17f81024c0 0x7f17f8198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50102/0 (socket says 172.21.15.84:50102) 2024-05-22T23:01:25.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.453+0000 7f17f67fc700 1 -- 172.21.15.84:0/242776799 learned_addr learned my addr 172.21.15.84:0/242776799 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:25.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.453+0000 7f17f77fe700 1 --2- 172.21.15.84:0/242776799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17f8102e60 0x7f17f819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:25.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.454+0000 7f17f6ffd700 1 --2- 172.21.15.84:0/242776799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17f81012c0 0x7f17f81986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:25.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.454+0000 7f17f67fc700 1 -- 172.21.15.84:0/242776799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17f81012c0 msgr2=0x7f17f81986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:25.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.454+0000 7f17f67fc700 1 --2- 172.21.15.84:0/242776799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17f81012c0 0x7f17f81986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:25.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.454+0000 7f17f67fc700 1 -- 172.21.15.84:0/242776799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17f8102e60 msgr2=0x7f17f819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:25.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.454+0000 7f17f67fc700 1 --2- 172.21.15.84:0/242776799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17f8102e60 0x7f17f819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:25.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.454+0000 7f17f67fc700 1 -- 172.21.15.84:0/242776799 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f81a0310 con 0x7f17f81024c0 2024-05-22T23:01:25.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.454+0000 7f17f6ffd700 1 --2- 172.21.15.84:0/242776799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17f81012c0 0x7f17f81986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:01:25.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.454+0000 7f17f67fc700 1 --2- 172.21.15.84:0/242776799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17f81024c0 0x7f17f8198bf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f17e8000c00 tx=0x7f17e8009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:25.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.454+0000 7f17dffff700 1 -- 172.21.15.84:0/242776799 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8026070 con 0x7f17f81024c0 2024-05-22T23:01:25.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.454+0000 7f17fd7bc700 1 -- 172.21.15.84:0/242776799 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f81a05a0 con 0x7f17f81024c0 2024-05-22T23:01:25.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.454+0000 7f17dffff700 1 -- 172.21.15.84:0/242776799 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e8004150 con 0x7f17f81024c0 2024-05-22T23:01:25.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.454+0000 7f17dffff700 1 -- 172.21.15.84:0/242776799 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8004a00 con 0x7f17f81024c0 2024-05-22T23:01:25.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.454+0000 7f17fd7bc700 1 -- 172.21.15.84:0/242776799 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f81a0a80 con 0x7f17f81024c0 2024-05-22T23:01:25.459 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.455+0000 7f17fd7bc700 1 -- 172.21.15.84:0/242776799 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17f8069050 con 0x7f17f81024c0 2024-05-22T23:01:25.459 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.456+0000 7f17dffff700 1 -- 172.21.15.84:0/242776799 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f17e8029070 con 0x7f17f81024c0 2024-05-22T23:01:25.459 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.457+0000 7f17dffff700 1 --2- 172.21.15.84:0/242776799 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f17e00a2c20 0x7f17e00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:25.459 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.457+0000 7f17f6ffd700 1 --2- 172.21.15.84:0/242776799 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f17e00a2c20 0x7f17e00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:25.459 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.457+0000 7f17dffff700 1 -- 172.21.15.84:0/242776799 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f17e80cfa30 con 0x7f17f81024c0 2024-05-22T23:01:25.460 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.457+0000 7f17f6ffd700 1 --2- 172.21.15.84:0/242776799 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f17e00a2c20 0x7f17e00a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f17ec005fd0 tx=0x7f17ec005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:25.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.461+0000 7f17dffff700 1 -- 172.21.15.84:0/242776799 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f17e805d850 con 0x7f17f81024c0 2024-05-22T23:01:25.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.671+0000 7f17fd7bc700 1 -- 172.21.15.84:0/242776799 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f17f80fea80 con 0x7f17e00a2c20 2024-05-22T23:01:25.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.677+0000 7f17dffff700 1 -- 172.21.15.84:0/242776799 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f17f80fea80 con 0x7f17e00a2c20 2024-05-22T23:01:25.679 INFO:teuthology.orchestra.run.smithi084.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-22T23:01:25.679 INFO:teuthology.orchestra.run.smithi084.stdout:mds.all 12/12 33s ago 2m smithi084=a;smithi084=d;smithi084=g;smithi084=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-05-22T23:01:25.679 INFO:teuthology.orchestra.run.smithi084.stdout:mgr 3/3 33s ago 7m smithi084=x;smithi096=y;smithi204=z;count:3 2024-05-22T23:01:25.679 INFO:teuthology.orchestra.run.smithi084.stdout:mon 3/3 33s ago 8m smithi084:172.21.15.84=a;smithi096:172.21.15.96=b;smithi204:172.21.15.204=c;count:3 2024-05-22T23:01:25.679 INFO:teuthology.orchestra.run.smithi084.stdout:osd 12 33s ago - 2024-05-22T23:01:25.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.679+0000 7f17fd7bc700 1 -- 172.21.15.84:0/242776799 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f17e00a2c20 msgr2=0x7f17e00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:25.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.679+0000 7f17fd7bc700 1 --2- 172.21.15.84:0/242776799 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f17e00a2c20 0x7f17e00a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f17ec005fd0 tx=0x7f17ec005f00 comp rx=0 tx=0).stop 2024-05-22T23:01:25.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.679+0000 7f17fd7bc700 1 -- 172.21.15.84:0/242776799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17f81024c0 msgr2=0x7f17f8198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:25.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.679+0000 7f17fd7bc700 1 --2- 172.21.15.84:0/242776799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17f81024c0 0x7f17f8198bf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f17e8000c00 tx=0x7f17e8009f10 comp rx=0 tx=0).stop 2024-05-22T23:01:25.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.679+0000 7f17fd7bc700 1 -- 172.21.15.84:0/242776799 shutdown_connections 2024-05-22T23:01:25.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.679+0000 7f17fd7bc700 1 --2- 172.21.15.84:0/242776799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17f8102e60 0x7f17f819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:25.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.679+0000 7f17fd7bc700 1 --2- 172.21.15.84:0/242776799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17f81012c0 0x7f17f81986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:25.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.679+0000 7f17fd7bc700 1 --2- 172.21.15.84:0/242776799 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f17e00a2c20 0x7f17e00a50e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:25.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.679+0000 7f17fd7bc700 1 --2- 172.21.15.84:0/242776799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17f81024c0 0x7f17f8198bf0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:25.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.679+0000 7f17fd7bc700 1 -- 172.21.15.84:0/242776799 >> 172.21.15.84:0/242776799 conn(0x7f17f80fc860 msgr2=0x7f17f80fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:25.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.680+0000 7f17fd7bc700 1 -- 172.21.15.84:0/242776799 shutdown_connections 2024-05-22T23:01:25.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:25.680+0000 7f17fd7bc700 1 -- 172.21.15.84:0/242776799 wait complete. 2024-05-22T23:01:26.339 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-22T23:01:26.577 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:01:26.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:26 smithi084 ceph-mon[32186]: pgmap v338: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-22T23:01:26.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:26 smithi204 ceph-mon[36211]: pgmap v338: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-22T23:01:27.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:26 smithi096 ceph-mon[36392]: pgmap v338: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-22T23:01:27.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:27 smithi084 ceph-mon[32186]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:27.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:27 smithi204 ceph-mon[36211]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:28.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:27 smithi096 ceph-mon[36392]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:28.246 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.244+0000 7f450dca1700 1 -- 172.21.15.84:0/160375283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4508103f60 msgr2=0x7f450810a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:28.246 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.244+0000 7f450dca1700 1 --2- 172.21.15.84:0/160375283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4508103f60 0x7f450810a810 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f450000b0d0 tx=0x7f450000b3e0 comp rx=0 tx=0).stop 2024-05-22T23:01:28.247 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.244+0000 7f450dca1700 1 -- 172.21.15.84:0/160375283 shutdown_connections 2024-05-22T23:01:28.247 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.244+0000 7f450dca1700 1 --2- 172.21.15.84:0/160375283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4508103f60 0x7f450810a810 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:28.247 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.244+0000 7f450dca1700 1 --2- 172.21.15.84:0/160375283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45081023c0 0x7f45081027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:28.247 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.244+0000 7f450dca1700 1 --2- 172.21.15.84:0/160375283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45081035c0 0x7f4508103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:28.247 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.244+0000 7f450dca1700 1 -- 172.21.15.84:0/160375283 >> 172.21.15.84:0/160375283 conn(0x7f45080fd960 msgr2=0x7f45080ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:28.247 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.245+0000 7f450dca1700 1 -- 172.21.15.84:0/160375283 shutdown_connections 2024-05-22T23:01:28.247 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.245+0000 7f450dca1700 1 -- 172.21.15.84:0/160375283 wait complete. 2024-05-22T23:01:28.247 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.245+0000 7f450dca1700 1 Processor -- start 2024-05-22T23:01:28.249 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.245+0000 7f450dca1700 1 -- start start 2024-05-22T23:01:28.249 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.245+0000 7f450dca1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45081023c0 0x7f4508198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:28.249 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.245+0000 7f450dca1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45081035c0 0x7f4508198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:28.249 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.245+0000 7f450dca1700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4508103f60 0x7f450819fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:28.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.246+0000 7f450dca1700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f450810dfb0 con 0x7f4508103f60 2024-05-22T23:01:28.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.246+0000 7f450dca1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f450810de30 con 0x7f45081023c0 2024-05-22T23:01:28.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.246+0000 7f450dca1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f450810e130 con 0x7f45081035c0 2024-05-22T23:01:28.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.246+0000 7f4507fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4508103f60 0x7f450819fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:28.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.246+0000 7f4507fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4508103f60 0x7f450819fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50122/0 (socket says 172.21.15.84:50122) 2024-05-22T23:01:28.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.246+0000 7f4507fff700 1 -- 172.21.15.84:0/3399223386 learned_addr learned my addr 172.21.15.84:0/3399223386 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:28.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.246+0000 7f45077fe700 1 --2- 172.21.15.84:0/3399223386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45081023c0 0x7f4508198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:28.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.246+0000 7f4506ffd700 1 --2- 172.21.15.84:0/3399223386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45081035c0 0x7f4508198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:28.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.246+0000 7f4507fff700 1 -- 172.21.15.84:0/3399223386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45081035c0 msgr2=0x7f4508198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:28.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.246+0000 7f4507fff700 1 --2- 172.21.15.84:0/3399223386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45081035c0 0x7f4508198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:28.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.246+0000 7f4507fff700 1 -- 172.21.15.84:0/3399223386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45081023c0 msgr2=0x7f4508198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:28.251 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.246+0000 7f4507fff700 1 --2- 172.21.15.84:0/3399223386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45081023c0 0x7f4508198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:28.251 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.246+0000 7f4507fff700 1 -- 172.21.15.84:0/3399223386 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45081a01c0 con 0x7f4508103f60 2024-05-22T23:01:28.251 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.246+0000 7f4507fff700 1 --2- 172.21.15.84:0/3399223386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4508103f60 0x7f450819fb20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4500006070 tx=0x7f4500004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:28.251 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.246+0000 7f4504ff9700 1 -- 172.21.15.84:0/3399223386 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4500021030 con 0x7f4508103f60 2024-05-22T23:01:28.251 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.246+0000 7f450dca1700 1 -- 172.21.15.84:0/3399223386 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45081a0450 con 0x7f4508103f60 2024-05-22T23:01:28.251 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.246+0000 7f4504ff9700 1 -- 172.21.15.84:0/3399223386 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4500007a80 con 0x7f4508103f60 2024-05-22T23:01:28.251 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.246+0000 7f4504ff9700 1 -- 172.21.15.84:0/3399223386 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45000047d0 con 0x7f4508103f60 2024-05-22T23:01:28.251 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.246+0000 7f450dca1700 1 -- 172.21.15.84:0/3399223386 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45081a0900 con 0x7f4508103f60 2024-05-22T23:01:28.251 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.247+0000 7f450dca1700 1 -- 172.21.15.84:0/3399223386 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4508069050 con 0x7f4508103f60 2024-05-22T23:01:28.252 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.248+0000 7f4504ff9700 1 -- 172.21.15.84:0/3399223386 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4500024070 con 0x7f4508103f60 2024-05-22T23:01:28.252 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.249+0000 7f4504ff9700 1 --2- 172.21.15.84:0/3399223386 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f44f00a2c20 0x7f44f00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:28.252 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.249+0000 7f45077fe700 1 --2- 172.21.15.84:0/3399223386 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f44f00a2c20 0x7f44f00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:28.253 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.250+0000 7f4504ff9700 1 -- 172.21.15.84:0/3399223386 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f45000d0120 con 0x7f4508103f60 2024-05-22T23:01:28.253 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.250+0000 7f45077fe700 1 --2- 172.21.15.84:0/3399223386 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f44f00a2c20 0x7f44f00a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f44f8005950 tx=0x7f44f800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:28.255 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.254+0000 7f4504ff9700 1 -- 172.21.15.84:0/3399223386 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f450009d0f0 con 0x7f4508103f60 2024-05-22T23:01:28.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.454+0000 7f450dca1700 1 -- 172.21.15.84:0/3399223386 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f45080627a0 con 0x7f44f00a2c20 2024-05-22T23:01:28.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.456+0000 7f4504ff9700 1 -- 172.21.15.84:0/3399223386 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f45080627a0 con 0x7f44f00a2c20 2024-05-22T23:01:28.457 INFO:teuthology.orchestra.run.smithi084.stdout:HOST ADDR LABELS STATUS 2024-05-22T23:01:28.458 INFO:teuthology.orchestra.run.smithi084.stdout:smithi084 172.21.15.84 2024-05-22T23:01:28.458 INFO:teuthology.orchestra.run.smithi084.stdout:smithi096 172.21.15.96 2024-05-22T23:01:28.458 INFO:teuthology.orchestra.run.smithi084.stdout:smithi204 172.21.15.204 2024-05-22T23:01:28.458 INFO:teuthology.orchestra.run.smithi084.stdout:3 hosts in cluster 2024-05-22T23:01:28.460 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.458+0000 7f450dca1700 1 -- 172.21.15.84:0/3399223386 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f44f00a2c20 msgr2=0x7f44f00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:28.460 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.458+0000 7f450dca1700 1 --2- 172.21.15.84:0/3399223386 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f44f00a2c20 0x7f44f00a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f44f8005950 tx=0x7f44f800a300 comp rx=0 tx=0).stop 2024-05-22T23:01:28.460 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.458+0000 7f450dca1700 1 -- 172.21.15.84:0/3399223386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4508103f60 msgr2=0x7f450819fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:28.460 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.458+0000 7f450dca1700 1 --2- 172.21.15.84:0/3399223386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4508103f60 0x7f450819fb20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4500006070 tx=0x7f4500004e40 comp rx=0 tx=0).stop 2024-05-22T23:01:28.460 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.458+0000 7f450dca1700 1 -- 172.21.15.84:0/3399223386 shutdown_connections 2024-05-22T23:01:28.460 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.458+0000 7f450dca1700 1 --2- 172.21.15.84:0/3399223386 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f44f00a2c20 0x7f44f00a50e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:28.460 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.458+0000 7f450dca1700 1 --2- 172.21.15.84:0/3399223386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4508103f60 0x7f450819fb20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:28.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.458+0000 7f450dca1700 1 --2- 172.21.15.84:0/3399223386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45081023c0 0x7f4508198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:28.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.458+0000 7f450dca1700 1 --2- 172.21.15.84:0/3399223386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45081035c0 0x7f4508198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:28.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.458+0000 7f450dca1700 1 -- 172.21.15.84:0/3399223386 >> 172.21.15.84:0/3399223386 conn(0x7f45080fd960 msgr2=0x7f4508106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:28.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.458+0000 7f450dca1700 1 -- 172.21.15.84:0/3399223386 shutdown_connections 2024-05-22T23:01:28.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:28.458+0000 7f450dca1700 1 -- 172.21.15.84:0/3399223386 wait complete. 2024-05-22T23:01:28.819 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:28 smithi084 ceph-mon[32186]: pgmap v339: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-22T23:01:28.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:28 smithi204 ceph-mon[36211]: pgmap v339: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-22T23:01:29.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:28 smithi096 ceph-mon[36392]: pgmap v339: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-22T23:01:29.137 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-22T23:01:29.383 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:01:29.757 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:29 smithi084 ceph-mon[32186]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:29.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:29 smithi204 ceph-mon[36211]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:30.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:29 smithi096 ceph-mon[36392]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:30.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:30 smithi084 ceph-mon[32186]: pgmap v340: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-05-22T23:01:30.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:30 smithi204 ceph-mon[36211]: pgmap v340: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-05-22T23:01:31.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:30 smithi096 ceph-mon[36392]: pgmap v340: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-05-22T23:01:31.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.173+0000 7f192a17a700 1 -- 172.21.15.84:0/177029393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1924102e70 msgr2=0x7f1924109720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:31.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.173+0000 7f192a17a700 1 --2- 172.21.15.84:0/177029393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1924102e70 0x7f1924109720 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f191c0099c0 tx=0x7f191c009cd0 comp rx=0 tx=0).stop 2024-05-22T23:01:31.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.173+0000 7f192a17a700 1 -- 172.21.15.84:0/177029393 shutdown_connections 2024-05-22T23:01:31.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.173+0000 7f192a17a700 1 --2- 172.21.15.84:0/177029393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1924102e70 0x7f1924109720 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:31.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.173+0000 7f192a17a700 1 --2- 172.21.15.84:0/177029393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19241012d0 0x7f19241016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:31.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.173+0000 7f192a17a700 1 --2- 172.21.15.84:0/177029393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19241024d0 0x7f1924102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:31.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.173+0000 7f192a17a700 1 -- 172.21.15.84:0/177029393 >> 172.21.15.84:0/177029393 conn(0x7f19240fc890 msgr2=0x7f19240fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:31.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.173+0000 7f192a17a700 1 -- 172.21.15.84:0/177029393 shutdown_connections 2024-05-22T23:01:31.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.173+0000 7f192a17a700 1 -- 172.21.15.84:0/177029393 wait complete. 2024-05-22T23:01:31.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.173+0000 7f192a17a700 1 Processor -- start 2024-05-22T23:01:31.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.174+0000 7f192a17a700 1 -- start start 2024-05-22T23:01:31.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.174+0000 7f192a17a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19241012d0 0x7f1924197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:31.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.174+0000 7f192a17a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19241024d0 0x7f19241979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:31.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.174+0000 7f192a17a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1924102e70 0x7f192419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:31.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.174+0000 7f192a17a700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f192410cf30 con 0x7f19241024d0 2024-05-22T23:01:31.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.174+0000 7f192a17a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f192410cdb0 con 0x7f19241012d0 2024-05-22T23:01:31.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.174+0000 7f192a17a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f192410d0b0 con 0x7f1924102e70 2024-05-22T23:01:31.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.174+0000 7f1922ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19241024d0 0x7f19241979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:31.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.174+0000 7f1922ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19241024d0 0x7f19241979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50134/0 (socket says 172.21.15.84:50134) 2024-05-22T23:01:31.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.174+0000 7f1922ffd700 1 -- 172.21.15.84:0/3774695470 learned_addr learned my addr 172.21.15.84:0/3774695470 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:31.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.174+0000 7f19237fe700 1 --2- 172.21.15.84:0/3774695470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19241012d0 0x7f1924197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:31.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.174+0000 7f1923fff700 1 --2- 172.21.15.84:0/3774695470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1924102e70 0x7f192419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:31.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.174+0000 7f1922ffd700 1 -- 172.21.15.84:0/3774695470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1924102e70 msgr2=0x7f192419ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:31.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.174+0000 7f1922ffd700 1 --2- 172.21.15.84:0/3774695470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1924102e70 0x7f192419ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:31.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.174+0000 7f1922ffd700 1 -- 172.21.15.84:0/3774695470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19241012d0 msgr2=0x7f1924197490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:31.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.174+0000 7f1922ffd700 1 --2- 172.21.15.84:0/3774695470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19241012d0 0x7f1924197490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:31.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.174+0000 7f1922ffd700 1 -- 172.21.15.84:0/3774695470 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f192419f0f0 con 0x7f19241024d0 2024-05-22T23:01:31.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.175+0000 7f1922ffd700 1 --2- 172.21.15.84:0/3774695470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19241024d0 0x7f19241979d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f191800c9e0 tx=0x7f191800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:31.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.175+0000 7f1920ff9700 1 -- 172.21.15.84:0/3774695470 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1918007ac0 con 0x7f19241024d0 2024-05-22T23:01:31.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.175+0000 7f192a17a700 1 -- 172.21.15.84:0/3774695470 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f192419f3e0 con 0x7f19241024d0 2024-05-22T23:01:31.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.175+0000 7f1920ff9700 1 -- 172.21.15.84:0/3774695470 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1918007c60 con 0x7f19241024d0 2024-05-22T23:01:31.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.175+0000 7f1920ff9700 1 -- 172.21.15.84:0/3774695470 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1918018730 con 0x7f19241024d0 2024-05-22T23:01:31.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.175+0000 7f192a17a700 1 -- 172.21.15.84:0/3774695470 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f192419f9f0 con 0x7f19241024d0 2024-05-22T23:01:31.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.176+0000 7f192a17a700 1 -- 172.21.15.84:0/3774695470 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1924069050 con 0x7f19241024d0 2024-05-22T23:01:31.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.177+0000 7f1920ff9700 1 -- 172.21.15.84:0/3774695470 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f19180188d0 con 0x7f19241024d0 2024-05-22T23:01:31.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.178+0000 7f1920ff9700 1 --2- 172.21.15.84:0/3774695470 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f190c0a2c20 0x7f190c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:31.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.178+0000 7f19237fe700 1 --2- 172.21.15.84:0/3774695470 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f190c0a2c20 0x7f190c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:31.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.178+0000 7f1920ff9700 1 -- 172.21.15.84:0/3774695470 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f19180cf320 con 0x7f19241024d0 2024-05-22T23:01:31.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.178+0000 7f19237fe700 1 --2- 172.21.15.84:0/3774695470 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f190c0a2c20 0x7f190c0a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1924102330 tx=0x7f191400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:31.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.181+0000 7f1920ff9700 1 -- 172.21.15.84:0/3774695470 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1918014030 con 0x7f19241024d0 2024-05-22T23:01:31.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.383+0000 7f192a17a700 1 -- 172.21.15.84:0/3774695470 --> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f19240627a0 con 0x7f190c0a2c20 2024-05-22T23:01:31.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.385+0000 7f1920ff9700 1 -- 172.21.15.84:0/3774695470 <== mgr.14152 v2:172.21.15.84:6800/4223589140 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f19240627a0 con 0x7f190c0a2c20 2024-05-22T23:01:31.387 INFO:teuthology.orchestra.run.smithi084.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-22T23:01:31.387 INFO:teuthology.orchestra.run.smithi084.stdout:smithi084 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400WF400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-22T23:01:31.387 INFO:teuthology.orchestra.run.smithi084.stdout:smithi096 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NH400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-22T23:01:31.387 INFO:teuthology.orchestra.run.smithi084.stdout:smithi204 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GD400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-22T23:01:31.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.387+0000 7f192a17a700 1 -- 172.21.15.84:0/3774695470 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f190c0a2c20 msgr2=0x7f190c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:31.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.387+0000 7f192a17a700 1 --2- 172.21.15.84:0/3774695470 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f190c0a2c20 0x7f190c0a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1924102330 tx=0x7f191400d040 comp rx=0 tx=0).stop 2024-05-22T23:01:31.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.387+0000 7f192a17a700 1 -- 172.21.15.84:0/3774695470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19241024d0 msgr2=0x7f19241979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:31.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.387+0000 7f192a17a700 1 --2- 172.21.15.84:0/3774695470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19241024d0 0x7f19241979d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f191800c9e0 tx=0x7f191800ccf0 comp rx=0 tx=0).stop 2024-05-22T23:01:31.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.387+0000 7f192a17a700 1 -- 172.21.15.84:0/3774695470 shutdown_connections 2024-05-22T23:01:31.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.388+0000 7f192a17a700 1 --2- 172.21.15.84:0/3774695470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1924102e70 0x7f192419ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:31.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.388+0000 7f192a17a700 1 --2- 172.21.15.84:0/3774695470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19241012d0 0x7f1924197490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:31.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.388+0000 7f192a17a700 1 --2- 172.21.15.84:0/3774695470 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f190c0a2c20 0x7f190c0a50e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:31.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.388+0000 7f192a17a700 1 --2- 172.21.15.84:0/3774695470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19241024d0 0x7f19241979d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:31.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.388+0000 7f192a17a700 1 -- 172.21.15.84:0/3774695470 >> 172.21.15.84:0/3774695470 conn(0x7f19240fc890 msgr2=0x7f1924105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:31.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.388+0000 7f192a17a700 1 -- 172.21.15.84:0/3774695470 shutdown_connections 2024-05-22T23:01:31.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:31.388+0000 7f192a17a700 1 -- 172.21.15.84:0/3774695470 wait complete. 2024-05-22T23:01:31.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:31 smithi204 ceph-mon[36211]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:31.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:31 smithi204 ceph-mon[36211]: pgmap v341: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-05-22T23:01:32.022 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:31 smithi084 ceph-mon[32186]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:32.022 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:31 smithi084 ceph-mon[32186]: pgmap v341: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-05-22T23:01:32.049 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-22T23:01:32.058 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi084.front.sepia.ceph.com 2024-05-22T23:01:32.059 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-22T23:01:32.060 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:31 smithi096 ceph-mon[36392]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:32.060 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:31 smithi096 ceph-mon[36392]: pgmap v341: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-05-22T23:01:32.301 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:01:34.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.069+0000 7f26d219e700 1 -- 172.21.15.84:0/3690516992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26cc101300 msgr2=0x7f26cc101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:34.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.069+0000 7f26d219e700 1 --2- 172.21.15.84:0/3690516992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26cc101300 0x7f26cc101720 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f26bc0099c0 tx=0x7f26bc009cd0 comp rx=0 tx=0).stop 2024-05-22T23:01:34.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.069+0000 7f26d219e700 1 -- 172.21.15.84:0/3690516992 shutdown_connections 2024-05-22T23:01:34.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.069+0000 7f26d219e700 1 --2- 172.21.15.84:0/3690516992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26cc102ea0 0x7f26cc109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:34.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.069+0000 7f26d219e700 1 --2- 172.21.15.84:0/3690516992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26cc101300 0x7f26cc101720 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:34.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.069+0000 7f26d219e700 1 --2- 172.21.15.84:0/3690516992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26cc102500 0x7f26cc102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:34.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.069+0000 7f26d219e700 1 -- 172.21.15.84:0/3690516992 >> 172.21.15.84:0/3690516992 conn(0x7f26cc0fc8c0 msgr2=0x7f26cc0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:34.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.069+0000 7f26d219e700 1 -- 172.21.15.84:0/3690516992 shutdown_connections 2024-05-22T23:01:34.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.069+0000 7f26d219e700 1 -- 172.21.15.84:0/3690516992 wait complete. 2024-05-22T23:01:34.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.069+0000 7f26d219e700 1 Processor -- start 2024-05-22T23:01:34.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.069+0000 7f26d219e700 1 -- start start 2024-05-22T23:01:34.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.070+0000 7f26d219e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26cc101300 0x7f26cc06b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:34.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.070+0000 7f26d219e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26cc102500 0x7f26cc19d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:34.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.070+0000 7f26d219e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26cc102ea0 0x7f26cc19fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:34.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.070+0000 7f26d219e700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc10cf40 con 0x7f26cc102ea0 2024-05-22T23:01:34.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.070+0000 7f26d219e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc10cdc0 con 0x7f26cc101300 2024-05-22T23:01:34.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.070+0000 7f26d219e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc10d0c0 con 0x7f26cc102500 2024-05-22T23:01:34.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.070+0000 7f26cbfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26cc102ea0 0x7f26cc19fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:34.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.070+0000 7f26cbfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26cc102ea0 0x7f26cc19fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33686/0 (socket says 172.21.15.84:33686) 2024-05-22T23:01:34.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.070+0000 7f26cbfff700 1 -- 172.21.15.84:0/1110608982 learned_addr learned my addr 172.21.15.84:0/1110608982 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:34.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.070+0000 7f26cb7fe700 1 --2- 172.21.15.84:0/1110608982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26cc101300 0x7f26cc06b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:34.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.070+0000 7f26c2ffd700 1 --2- 172.21.15.84:0/1110608982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26cc102500 0x7f26cc19d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:34.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.070+0000 7f26cbfff700 1 -- 172.21.15.84:0/1110608982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26cc102500 msgr2=0x7f26cc19d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:34.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.070+0000 7f26cbfff700 1 --2- 172.21.15.84:0/1110608982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26cc102500 0x7f26cc19d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:34.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.070+0000 7f26cbfff700 1 -- 172.21.15.84:0/1110608982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26cc101300 msgr2=0x7f26cc06b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:34.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.070+0000 7f26cbfff700 1 --2- 172.21.15.84:0/1110608982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26cc101300 0x7f26cc06b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:34.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.070+0000 7f26cbfff700 1 -- 172.21.15.84:0/1110608982 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26cc1a02e0 con 0x7f26cc102ea0 2024-05-22T23:01:34.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.070+0000 7f26cbfff700 1 --2- 172.21.15.84:0/1110608982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26cc102ea0 0x7f26cc19fa10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f26c400b970 tx=0x7f26c400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:34.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.070+0000 7f26c97fa700 1 -- 172.21.15.84:0/1110608982 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4019070 con 0x7f26cc102ea0 2024-05-22T23:01:34.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.070+0000 7f26d219e700 1 -- 172.21.15.84:0/1110608982 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26cc1a05d0 con 0x7f26cc102ea0 2024-05-22T23:01:34.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.070+0000 7f26c97fa700 1 -- 172.21.15.84:0/1110608982 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26c400ce10 con 0x7f26cc102ea0 2024-05-22T23:01:34.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.071+0000 7f26c97fa700 1 -- 172.21.15.84:0/1110608982 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4007370 con 0x7f26cc102ea0 2024-05-22T23:01:34.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.071+0000 7f26d219e700 1 -- 172.21.15.84:0/1110608982 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26cc1a0be0 con 0x7f26cc102ea0 2024-05-22T23:01:34.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.072+0000 7f26d219e700 1 -- 172.21.15.84:0/1110608982 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26cc069050 con 0x7f26cc102ea0 2024-05-22T23:01:34.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.073+0000 7f26c97fa700 1 -- 172.21.15.84:0/1110608982 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f26c401c070 con 0x7f26cc102ea0 2024-05-22T23:01:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.073+0000 7f26c97fa700 1 --2- 172.21.15.84:0/1110608982 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f26b80a2c20 0x7f26b80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.073+0000 7f26cb7fe700 1 --2- 172.21.15.84:0/1110608982 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f26b80a2c20 0x7f26b80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.074+0000 7f26c97fa700 1 -- 172.21.15.84:0/1110608982 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f26c40ce530 con 0x7f26cc102ea0 2024-05-22T23:01:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.074+0000 7f26cb7fe700 1 --2- 172.21.15.84:0/1110608982 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f26b80a2c20 0x7f26b80a50e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f26bc000c00 tx=0x7f26bc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:34.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.078+0000 7f26c97fa700 1 -- 172.21.15.84:0/1110608982 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f26c409b470 con 0x7f26cc102ea0 2024-05-22T23:01:34.322 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.320+0000 7f26d219e700 1 -- 172.21.15.84:0/1110608982 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f26cc062f70 con 0x7f26cc102ea0 2024-05-22T23:01:34.322 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.321+0000 7f26c97fa700 1 -- 172.21.15.84:0/1110608982 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) ==== 76+0+3008 (secure 0 0 0) 0x7f26c409ad70 con 0x7f26cc102ea0 2024-05-22T23:01:34.323 INFO:teuthology.orchestra.run.smithi084.stdout:e18 2024-05-22T23:01:34.323 INFO:teuthology.orchestra.run.smithi084.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-22T23:01:34.323 INFO:teuthology.orchestra.run.smithi084.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-22T23:01:34.323 INFO:teuthology.orchestra.run.smithi084.stdout:legacy client fscid: 1 2024-05-22T23:01:34.323 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:34.323 INFO:teuthology.orchestra.run.smithi084.stdout:Filesystem 'cephfs' (1) 2024-05-22T23:01:34.323 INFO:teuthology.orchestra.run.smithi084.stdout:fs_name cephfs 2024-05-22T23:01:34.323 INFO:teuthology.orchestra.run.smithi084.stdout:epoch 18 2024-05-22T23:01:34.323 INFO:teuthology.orchestra.run.smithi084.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-22T23:01:34.323 INFO:teuthology.orchestra.run.smithi084.stdout:created 2024-05-22T22:59:51.926923+0000 2024-05-22T23:01:34.323 INFO:teuthology.orchestra.run.smithi084.stdout:modified 2024-05-22T22:59:54.945583+0000 2024-05-22T23:01:34.324 INFO:teuthology.orchestra.run.smithi084.stdout:tableserver 0 2024-05-22T23:01:34.324 INFO:teuthology.orchestra.run.smithi084.stdout:root 0 2024-05-22T23:01:34.324 INFO:teuthology.orchestra.run.smithi084.stdout:session_timeout 300 2024-05-22T23:01:34.324 INFO:teuthology.orchestra.run.smithi084.stdout:session_autoclose 300 2024-05-22T23:01:34.324 INFO:teuthology.orchestra.run.smithi084.stdout:max_file_size 1099511627776 2024-05-22T23:01:34.324 INFO:teuthology.orchestra.run.smithi084.stdout:max_xattr_size 65536 2024-05-22T23:01:34.324 INFO:teuthology.orchestra.run.smithi084.stdout:required_client_features {} 2024-05-22T23:01:34.324 INFO:teuthology.orchestra.run.smithi084.stdout:last_failure 0 2024-05-22T23:01:34.324 INFO:teuthology.orchestra.run.smithi084.stdout:last_failure_osd_epoch 0 2024-05-22T23:01:34.324 INFO:teuthology.orchestra.run.smithi084.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-22T23:01:34.324 INFO:teuthology.orchestra.run.smithi084.stdout:max_mds 1 2024-05-22T23:01:34.324 INFO:teuthology.orchestra.run.smithi084.stdout:in 0 2024-05-22T23:01:34.324 INFO:teuthology.orchestra.run.smithi084.stdout:up {0=24530} 2024-05-22T23:01:34.324 INFO:teuthology.orchestra.run.smithi084.stdout:failed 2024-05-22T23:01:34.324 INFO:teuthology.orchestra.run.smithi084.stdout:damaged 2024-05-22T23:01:34.325 INFO:teuthology.orchestra.run.smithi084.stdout:stopped 2024-05-22T23:01:34.325 INFO:teuthology.orchestra.run.smithi084.stdout:data_pools [3] 2024-05-22T23:01:34.325 INFO:teuthology.orchestra.run.smithi084.stdout:metadata_pool 2 2024-05-22T23:01:34.325 INFO:teuthology.orchestra.run.smithi084.stdout:inline_data disabled 2024-05-22T23:01:34.325 INFO:teuthology.orchestra.run.smithi084.stdout:balancer 2024-05-22T23:01:34.325 INFO:teuthology.orchestra.run.smithi084.stdout:bal_rank_mask -1 2024-05-22T23:01:34.325 INFO:teuthology.orchestra.run.smithi084.stdout:standby_count_wanted 0 2024-05-22T23:01:34.325 INFO:teuthology.orchestra.run.smithi084.stdout:qdb_cluster leader: 24530 members: 24530 2024-05-22T23:01:34.325 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.b{0:24530} state up:active seq 3 addr [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] compat {c=[1],r=[1],i=[1fff]}] 2024-05-22T23:01:34.325 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.h{0:24494} state up:standby-replay seq 1 addr [v2:172.21.15.96:6836/1191862158,v1:172.21.15.96:6837/1191862158] compat {c=[1],r=[1],i=[1fff]}] 2024-05-22T23:01:34.325 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:34.325 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:34.325 INFO:teuthology.orchestra.run.smithi084.stdout:Standby daemons: 2024-05-22T23:01:34.325 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:34.325 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.j{-1:14523} state up:standby seq 1 addr [v2:172.21.15.84:6834/2391815590,v1:172.21.15.84:6835/2391815590] compat {c=[1],r=[1],i=[1fff]}] 2024-05-22T23:01:34.326 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.d{-1:14541} state up:standby seq 1 addr [v2:172.21.15.84:6836/4284308638,v1:172.21.15.84:6837/4284308638] compat {c=[1],r=[1],i=[1fff]}] 2024-05-22T23:01:34.326 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.g{-1:14598} state up:standby seq 1 addr [v2:172.21.15.84:6838/669165452,v1:172.21.15.84:6839/669165452] compat {c=[1],r=[1],i=[1fff]}] 2024-05-22T23:01:34.326 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.a{-1:14610} state up:standby seq 1 addr [v2:172.21.15.84:6840/2709540650,v1:172.21.15.84:6841/2709540650] compat {c=[1],r=[1],i=[1fff]}] 2024-05-22T23:01:34.326 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.l{-1:24382} state up:standby seq 1 addr [v2:172.21.15.204:6832/3932120262,v1:172.21.15.204:6833/3932120262] compat {c=[1],r=[1],i=[1fff]}] 2024-05-22T23:01:34.326 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.f{-1:24424} state up:standby seq 1 addr [v2:172.21.15.204:6834/2890595273,v1:172.21.15.204:6835/2890595273] compat {c=[1],r=[1],i=[1fff]}] 2024-05-22T23:01:34.326 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.k{-1:24431} state up:standby seq 1 addr [v2:172.21.15.96:6832/3792500684,v1:172.21.15.96:6833/3792500684] compat {c=[1],r=[1],i=[1fff]}] 2024-05-22T23:01:34.326 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.c{-1:24436} state up:standby seq 1 addr [v2:172.21.15.204:6836/3662981631,v1:172.21.15.204:6837/3662981631] compat {c=[1],r=[1],i=[1fff]}] 2024-05-22T23:01:34.326 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.i{-1:24457} state up:standby seq 1 addr [v2:172.21.15.204:6838/3328293038,v1:172.21.15.204:6839/3328293038] compat {c=[1],r=[1],i=[1fff]}] 2024-05-22T23:01:34.326 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.e{-1:24461} state up:standby seq 1 addr [v2:172.21.15.96:6834/1115134082,v1:172.21.15.96:6835/1115134082] compat {c=[1],r=[1],i=[1fff]}] 2024-05-22T23:01:34.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.323+0000 7f26d219e700 1 -- 172.21.15.84:0/1110608982 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f26b80a2c20 msgr2=0x7f26b80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:34.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.323+0000 7f26d219e700 1 --2- 172.21.15.84:0/1110608982 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f26b80a2c20 0x7f26b80a50e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f26bc000c00 tx=0x7f26bc005fb0 comp rx=0 tx=0).stop 2024-05-22T23:01:34.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.323+0000 7f26d219e700 1 -- 172.21.15.84:0/1110608982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26cc102ea0 msgr2=0x7f26cc19fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:34.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.323+0000 7f26d219e700 1 --2- 172.21.15.84:0/1110608982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26cc102ea0 0x7f26cc19fa10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f26c400b970 tx=0x7f26c400bc80 comp rx=0 tx=0).stop 2024-05-22T23:01:34.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.323+0000 7f26d219e700 1 -- 172.21.15.84:0/1110608982 shutdown_connections 2024-05-22T23:01:34.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.323+0000 7f26d219e700 1 --2- 172.21.15.84:0/1110608982 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f26b80a2c20 0x7f26b80a50e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:34.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.323+0000 7f26d219e700 1 --2- 172.21.15.84:0/1110608982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26cc102ea0 0x7f26cc19fa10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:34.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.323+0000 7f26d219e700 1 --2- 172.21.15.84:0/1110608982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26cc101300 0x7f26cc06b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:34.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.323+0000 7f26d219e700 1 --2- 172.21.15.84:0/1110608982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26cc102500 0x7f26cc19d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:34.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.323+0000 7f26d219e700 1 -- 172.21.15.84:0/1110608982 >> 172.21.15.84:0/1110608982 conn(0x7f26cc0fc8c0 msgr2=0x7f26cc105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:34.328 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.323+0000 7f26d219e700 1 -- 172.21.15.84:0/1110608982 shutdown_connections 2024-05-22T23:01:34.328 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:34.323+0000 7f26d219e700 1 -- 172.21.15.84:0/1110608982 wait complete. 2024-05-22T23:01:34.328 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:01:34.572 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:34 smithi084 ceph-mon[32186]: pgmap v342: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:01:34.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:34 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1110608982' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-22T23:01:34.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:34 smithi204 ceph-mon[36211]: pgmap v342: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:01:34.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:34 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1110608982' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-22T23:01:34.960 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-22T23:01:35.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:34 smithi096 ceph-mon[36392]: pgmap v342: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:01:35.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:34 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1110608982' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-22T23:01:35.198 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:01:36.601 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:36 smithi084 ceph-mon[32186]: pgmap v343: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:01:36.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:36 smithi204 ceph-mon[36211]: pgmap v343: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:01:36.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.947+0000 7f95a93bb700 1 -- 172.21.15.84:0/2651359975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95a41028c0 msgr2=0x7f95a4102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:36.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.947+0000 7f95a93bb700 1 --2- 172.21.15.84:0/2651359975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95a41028c0 0x7f95a4102ce0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9594009990 tx=0x7f9594009ca0 comp rx=0 tx=0).stop 2024-05-22T23:01:36.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.947+0000 7f95a93bb700 1 -- 172.21.15.84:0/2651359975 shutdown_connections 2024-05-22T23:01:36.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.947+0000 7f95a93bb700 1 --2- 172.21.15.84:0/2651359975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a4101ee0 0x7f95a4109780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:36.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.947+0000 7f95a93bb700 1 --2- 172.21.15.84:0/2651359975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95a41028c0 0x7f95a4102ce0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:36.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.947+0000 7f95a93bb700 1 --2- 172.21.15.84:0/2651359975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95a4101520 0x7f95a41019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:36.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.947+0000 7f95a93bb700 1 -- 172.21.15.84:0/2651359975 >> 172.21.15.84:0/2651359975 conn(0x7f95a40fbff0 msgr2=0x7f95a40fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:36.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.947+0000 7f95a93bb700 1 -- 172.21.15.84:0/2651359975 shutdown_connections 2024-05-22T23:01:36.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.947+0000 7f95a93bb700 1 -- 172.21.15.84:0/2651359975 wait complete. 2024-05-22T23:01:36.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.947+0000 7f95a93bb700 1 Processor -- start 2024-05-22T23:01:36.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.948+0000 7f95a93bb700 1 -- start start 2024-05-22T23:01:36.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.948+0000 7f95a93bb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a4101520 0x7f95a4100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:36.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.948+0000 7f95a93bb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95a4101ee0 0x7f95a419d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:36.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.948+0000 7f95a93bb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95a41028c0 0x7f95a419f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:36.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.948+0000 7f95a93bb700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f95a410cea0 con 0x7f95a4101ee0 2024-05-22T23:01:36.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.948+0000 7f95a93bb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f95a410cd20 con 0x7f95a41028c0 2024-05-22T23:01:36.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.948+0000 7f95a93bb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f95a410d020 con 0x7f95a4101520 2024-05-22T23:01:36.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.948+0000 7f95a27fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95a4101ee0 0x7f95a419d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:36.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.948+0000 7f95a27fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95a4101ee0 0x7f95a419d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33702/0 (socket says 172.21.15.84:33702) 2024-05-22T23:01:36.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.948+0000 7f95a27fc700 1 -- 172.21.15.84:0/1259318197 learned_addr learned my addr 172.21.15.84:0/1259318197 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:36.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.948+0000 7f95a2ffd700 1 --2- 172.21.15.84:0/1259318197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a4101520 0x7f95a4100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:36.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.948+0000 7f95a27fc700 1 -- 172.21.15.84:0/1259318197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a4101520 msgr2=0x7f95a4100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:36.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.948+0000 7f95a27fc700 1 --2- 172.21.15.84:0/1259318197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a4101520 0x7f95a4100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:36.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.948+0000 7f95a27fc700 1 -- 172.21.15.84:0/1259318197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95a41028c0 msgr2=0x7f95a419f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:36.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.948+0000 7f95a37fe700 1 --2- 172.21.15.84:0/1259318197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95a41028c0 0x7f95a419f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:36.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.948+0000 7f95a27fc700 1 --2- 172.21.15.84:0/1259318197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95a41028c0 0x7f95a419f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:36.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.948+0000 7f95a27fc700 1 -- 172.21.15.84:0/1259318197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a41a01e0 con 0x7f95a4101ee0 2024-05-22T23:01:36.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.948+0000 7f95a27fc700 1 --2- 172.21.15.84:0/1259318197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95a4101ee0 0x7f95a419d2d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f959800d950 tx=0x7f959800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:36.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.949+0000 7f958bfff700 1 -- 172.21.15.84:0/1259318197 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598016070 con 0x7f95a4101ee0 2024-05-22T23:01:36.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.949+0000 7f95a93bb700 1 -- 172.21.15.84:0/1259318197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a41a04d0 con 0x7f95a4101ee0 2024-05-22T23:01:36.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.949+0000 7f958bfff700 1 -- 172.21.15.84:0/1259318197 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9598009e20 con 0x7f95a4101ee0 2024-05-22T23:01:36.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.949+0000 7f958bfff700 1 -- 172.21.15.84:0/1259318197 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959800f620 con 0x7f95a4101ee0 2024-05-22T23:01:36.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.949+0000 7f95a93bb700 1 -- 172.21.15.84:0/1259318197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a41a0a10 con 0x7f95a4101ee0 2024-05-22T23:01:36.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.949+0000 7f95a93bb700 1 -- 172.21.15.84:0/1259318197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95a4069050 con 0x7f95a4101ee0 2024-05-22T23:01:36.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.951+0000 7f958bfff700 1 -- 172.21.15.84:0/1259318197 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9598019070 con 0x7f95a4101ee0 2024-05-22T23:01:36.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.952+0000 7f958bfff700 1 --2- 172.21.15.84:0/1259318197 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f958c0a2c20 0x7f958c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:36.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.952+0000 7f95a2ffd700 1 --2- 172.21.15.84:0/1259318197 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f958c0a2c20 0x7f958c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:36.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.952+0000 7f958bfff700 1 -- 172.21.15.84:0/1259318197 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f95980cec20 con 0x7f95a4101ee0 2024-05-22T23:01:36.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.952+0000 7f95a2ffd700 1 --2- 172.21.15.84:0/1259318197 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f958c0a2c20 0x7f958c0a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9594003890 tx=0x7f9594005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:36.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:36.955+0000 7f958bfff700 1 -- 172.21.15.84:0/1259318197 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f959809bb60 con 0x7f95a4101ee0 2024-05-22T23:01:37.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:36 smithi096 ceph-mon[36392]: pgmap v343: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:01:37.152 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.150+0000 7f95a93bb700 1 -- 172.21.15.84:0/1259318197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f95a402dbf0 con 0x7f95a4101ee0 2024-05-22T23:01:37.152 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.151+0000 7f958bfff700 1 -- 172.21.15.84:0/1259318197 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5327 (secure 0 0 0) 0x7f959809b460 con 0x7f95a4101ee0 2024-05-22T23:01:37.153 INFO:teuthology.orchestra.run.smithi084.stdout:epoch 88 2024-05-22T23:01:37.153 INFO:teuthology.orchestra.run.smithi084.stdout:fsid c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T23:01:37.153 INFO:teuthology.orchestra.run.smithi084.stdout:created 2024-05-22T22:51:49.878218+0000 2024-05-22T23:01:37.153 INFO:teuthology.orchestra.run.smithi084.stdout:modified 2024-05-22T23:00:46.572591+0000 2024-05-22T23:01:37.153 INFO:teuthology.orchestra.run.smithi084.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-22T23:01:37.153 INFO:teuthology.orchestra.run.smithi084.stdout:crush_version 26 2024-05-22T23:01:37.153 INFO:teuthology.orchestra.run.smithi084.stdout:full_ratio 0.95 2024-05-22T23:01:37.153 INFO:teuthology.orchestra.run.smithi084.stdout:backfillfull_ratio 0.9 2024-05-22T23:01:37.153 INFO:teuthology.orchestra.run.smithi084.stdout:nearfull_ratio 0.85 2024-05-22T23:01:37.154 INFO:teuthology.orchestra.run.smithi084.stdout:require_min_compat_client luminous 2024-05-22T23:01:37.154 INFO:teuthology.orchestra.run.smithi084.stdout:min_compat_client luminous 2024-05-22T23:01:37.154 INFO:teuthology.orchestra.run.smithi084.stdout:require_osd_release squid 2024-05-22T23:01:37.154 INFO:teuthology.orchestra.run.smithi084.stdout:stretch_mode_enabled false 2024-05-22T23:01:37.154 INFO:teuthology.orchestra.run.smithi084.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-22T23:01:37.154 INFO:teuthology.orchestra.run.smithi084.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-22T23:01:37.154 INFO:teuthology.orchestra.run.smithi084.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-22T23:01:37.154 INFO:teuthology.orchestra.run.smithi084.stdout:max_osd 12 2024-05-22T23:01:37.154 INFO:teuthology.orchestra.run.smithi084.stdout:osd.0 up in weight 1 up_from 8 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6802/1559945826,v1:172.21.15.84:6803/1559945826] [v2:172.21.15.84:6804/1559945826,v1:172.21.15.84:6805/1559945826] exists,up 762df065-fb4f-4c71-aab8-a62d2282c253 2024-05-22T23:01:37.154 INFO:teuthology.orchestra.run.smithi084.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6800/2996505353,v1:172.21.15.96:6801/2996505353] [v2:172.21.15.96:6802/2996505353,v1:172.21.15.96:6803/2996505353] exists,up 3fb2385d-6a34-49e3-88e1-116237beb346 2024-05-22T23:01:37.154 INFO:teuthology.orchestra.run.smithi084.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6800/1499577897,v1:172.21.15.204:6801/1499577897] [v2:172.21.15.204:6802/1499577897,v1:172.21.15.204:6803/1499577897] exists,up e306e88e-a308-4585-ae11-c8af10b84038 2024-05-22T23:01:37.154 INFO:teuthology.orchestra.run.smithi084.stdout:osd.3 up in weight 1 up_from 25 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6810/2869340648,v1:172.21.15.84:6811/2869340648] [v2:172.21.15.84:6812/2869340648,v1:172.21.15.84:6813/2869340648] exists,up d2a5fb97-6f60-4d45-9f18-447cae480974 2024-05-22T23:01:37.155 INFO:teuthology.orchestra.run.smithi084.stdout:osd.4 up in weight 1 up_from 33 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6808/2554265648,v1:172.21.15.96:6809/2554265648] [v2:172.21.15.96:6810/2554265648,v1:172.21.15.96:6811/2554265648] exists,up 4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f 2024-05-22T23:01:37.155 INFO:teuthology.orchestra.run.smithi084.stdout:osd.5 up in weight 1 up_from 38 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6808/4075476986,v1:172.21.15.204:6809/4075476986] [v2:172.21.15.204:6810/4075476986,v1:172.21.15.204:6811/4075476986] exists,up 7996bf5e-93a5-41b0-a9b1-99e7ef4514f3 2024-05-22T23:01:37.155 INFO:teuthology.orchestra.run.smithi084.stdout:osd.6 up in weight 1 up_from 46 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6818/1806202865,v1:172.21.15.84:6819/1806202865] [v2:172.21.15.84:6820/1806202865,v1:172.21.15.84:6821/1806202865] exists,up f4b7c5de-1836-4d16-8603-3c517524d9fc 2024-05-22T23:01:37.155 INFO:teuthology.orchestra.run.smithi084.stdout:osd.7 up in weight 1 up_from 52 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6816/3195792774,v1:172.21.15.96:6817/3195792774] [v2:172.21.15.96:6818/3195792774,v1:172.21.15.96:6819/3195792774] exists,up 69a18c43-5447-480d-9466-190830070c3e 2024-05-22T23:01:37.155 INFO:teuthology.orchestra.run.smithi084.stdout:osd.8 up in weight 1 up_from 60 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6816/606997927,v1:172.21.15.204:6817/606997927] [v2:172.21.15.204:6818/606997927,v1:172.21.15.204:6819/606997927] exists,up 2c5cadca-689a-441e-847c-ba370d400ef1 2024-05-22T23:01:37.155 INFO:teuthology.orchestra.run.smithi084.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6826/2786189012,v1:172.21.15.84:6827/2786189012] [v2:172.21.15.84:6828/2786189012,v1:172.21.15.84:6829/2786189012] exists,up e3984fb2-7c00-4275-a10e-bca7c859f714 2024-05-22T23:01:37.155 INFO:teuthology.orchestra.run.smithi084.stdout:osd.10 up in weight 1 up_from 73 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6824/1985398526,v1:172.21.15.96:6825/1985398526] [v2:172.21.15.96:6826/1985398526,v1:172.21.15.96:6827/1985398526] exists,up b4938160-a808-4ded-91c2-d245c99fff8c 2024-05-22T23:01:37.155 INFO:teuthology.orchestra.run.smithi084.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6824/3106116519,v1:172.21.15.204:6825/3106116519] [v2:172.21.15.204:6826/3106116519,v1:172.21.15.204:6827/3106116519] exists,up 238922ef-d046-40cc-833d-25b7e7a523cb 2024-05-22T23:01:37.155 INFO:teuthology.orchestra.run.smithi084.stdout:pg_upmap_items 2.0 [8,4] 2024-05-22T23:01:37.155 INFO:teuthology.orchestra.run.smithi084.stdout:pg_upmap_items 2.a [2,11] 2024-05-22T23:01:37.155 INFO:teuthology.orchestra.run.smithi084.stdout:pg_upmap_items 2.10 [8,11] 2024-05-22T23:01:37.155 INFO:teuthology.orchestra.run.smithi084.stdout:pg_upmap_items 2.14 [2,4] 2024-05-22T23:01:37.155 INFO:teuthology.orchestra.run.smithi084.stdout:pg_upmap_items 3.1a [5,6] 2024-05-22T23:01:37.155 INFO:teuthology.orchestra.run.smithi084.stdout:pg_upmap_items 3.1b [4,7] 2024-05-22T23:01:37.156 INFO:teuthology.orchestra.run.smithi084.stdout:pg_upmap_items 3.1e [3,9] 2024-05-22T23:01:37.156 INFO:teuthology.orchestra.run.smithi084.stdout:pg_upmap_items 3.24 [4,9] 2024-05-22T23:01:37.156 INFO:teuthology.orchestra.run.smithi084.stdout:pg_upmap_items 3.38 [3,9] 2024-05-22T23:01:37.156 INFO:teuthology.orchestra.run.smithi084.stdout:pg_upmap_items 3.3a [4,8] 2024-05-22T23:01:37.156 INFO:teuthology.orchestra.run.smithi084.stdout:blocklist 172.21.15.84:6801/3278959854 expires 2024-05-23T22:52:45.395791+0000 2024-05-22T23:01:37.156 INFO:teuthology.orchestra.run.smithi084.stdout:blocklist 172.21.15.84:6800/3278959854 expires 2024-05-23T22:52:45.395791+0000 2024-05-22T23:01:37.156 INFO:teuthology.orchestra.run.smithi084.stdout:blocklist 172.21.15.84:0/2766288608 expires 2024-05-23T22:52:45.395791+0000 2024-05-22T23:01:37.156 INFO:teuthology.orchestra.run.smithi084.stdout:blocklist 172.21.15.84:0/950455999 expires 2024-05-23T22:52:45.395791+0000 2024-05-22T23:01:37.156 INFO:teuthology.orchestra.run.smithi084.stdout:blocklist 172.21.15.84:0/4000289608 expires 2024-05-23T22:52:18.615994+0000 2024-05-22T23:01:37.156 INFO:teuthology.orchestra.run.smithi084.stdout:blocklist 172.21.15.84:0/1650467363 expires 2024-05-23T22:52:18.615994+0000 2024-05-22T23:01:37.156 INFO:teuthology.orchestra.run.smithi084.stdout:blocklist 172.21.15.84:0/1052746114 expires 2024-05-23T22:52:18.615994+0000 2024-05-22T23:01:37.156 INFO:teuthology.orchestra.run.smithi084.stdout:blocklist 172.21.15.84:6800/4029123070 expires 2024-05-23T22:52:18.615994+0000 2024-05-22T23:01:37.156 INFO:teuthology.orchestra.run.smithi084.stdout:blocklist 172.21.15.84:6801/4029123070 expires 2024-05-23T22:52:18.615994+0000 2024-05-22T23:01:37.156 INFO:teuthology.orchestra.run.smithi084.stdout:blocklist 172.21.15.84:0/14610813 expires 2024-05-23T22:52:45.395791+0000 2024-05-22T23:01:37.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.153+0000 7f95a93bb700 1 -- 172.21.15.84:0/1259318197 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f958c0a2c20 msgr2=0x7f958c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:37.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.153+0000 7f95a93bb700 1 --2- 172.21.15.84:0/1259318197 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f958c0a2c20 0x7f958c0a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9594003890 tx=0x7f9594005fd0 comp rx=0 tx=0).stop 2024-05-22T23:01:37.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.153+0000 7f95a93bb700 1 -- 172.21.15.84:0/1259318197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95a4101ee0 msgr2=0x7f95a419d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:37.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.153+0000 7f95a93bb700 1 --2- 172.21.15.84:0/1259318197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95a4101ee0 0x7f95a419d2d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f959800d950 tx=0x7f959800dc60 comp rx=0 tx=0).stop 2024-05-22T23:01:37.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.153+0000 7f95a93bb700 1 -- 172.21.15.84:0/1259318197 shutdown_connections 2024-05-22T23:01:37.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.153+0000 7f95a93bb700 1 --2- 172.21.15.84:0/1259318197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95a41028c0 0x7f95a419f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:37.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.153+0000 7f95a93bb700 1 --2- 172.21.15.84:0/1259318197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a4101520 0x7f95a4100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:37.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.153+0000 7f95a93bb700 1 --2- 172.21.15.84:0/1259318197 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f958c0a2c20 0x7f958c0a50e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:37.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.153+0000 7f95a93bb700 1 --2- 172.21.15.84:0/1259318197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95a4101ee0 0x7f95a419d2d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:37.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.153+0000 7f95a93bb700 1 -- 172.21.15.84:0/1259318197 >> 172.21.15.84:0/1259318197 conn(0x7f95a40fbff0 msgr2=0x7f95a4105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:37.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.153+0000 7f95a93bb700 1 -- 172.21.15.84:0/1259318197 shutdown_connections 2024-05-22T23:01:37.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.153+0000 7f95a93bb700 1 -- 172.21.15.84:0/1259318197 wait complete. 2024-05-22T23:01:37.809 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:37 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1259318197' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-22T23:01:37.837 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-22T23:01:37.849 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T23:01:37.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:37 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1259318197' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-22T23:01:37.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.951+0000 7fb5184bd640 1 -- 172.21.15.84:0/683039370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5101007d0 msgr2=0x7fb510100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:37.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.951+0000 7fb5184bd640 1 --2- 172.21.15.84:0/683039370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5101007d0 0x7fb510100bf0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009900 tx=0x7fb50c02a410 comp rx=0 tx=0).stop 2024-05-22T23:01:37.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.951+0000 7fb5184bd640 1 -- 172.21.15.84:0/683039370 shutdown_connections 2024-05-22T23:01:37.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.951+0000 7fb5184bd640 1 --2- 172.21.15.84:0/683039370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb510102380 0x7fb510108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:37.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.951+0000 7fb5184bd640 1 --2- 172.21.15.84:0/683039370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5101019c0 0x7fb510101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:37.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.951+0000 7fb5184bd640 1 --2- 172.21.15.84:0/683039370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5101007d0 0x7fb510100bf0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:37.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.951+0000 7fb5184bd640 1 -- 172.21.15.84:0/683039370 >> 172.21.15.84:0/683039370 conn(0x7fb5100fbd60 msgr2=0x7fb5100fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:37.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.951+0000 7fb5184bd640 1 -- 172.21.15.84:0/683039370 shutdown_connections 2024-05-22T23:01:37.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.951+0000 7fb5184bd640 1 -- 172.21.15.84:0/683039370 wait complete. 2024-05-22T23:01:37.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.951+0000 7fb5184bd640 1 Processor -- start 2024-05-22T23:01:37.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.951+0000 7fb5184bd640 1 -- start start 2024-05-22T23:01:37.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.952+0000 7fb5184bd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5101007d0 0x7fb510197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:37.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.952+0000 7fb5184bd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5101019c0 0x7fb5101980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:37.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.952+0000 7fb5184bd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb510102380 0x7fb51019f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:37.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.952+0000 7fb5184bd640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb51010c440 con 0x7fb5101007d0 2024-05-22T23:01:37.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.952+0000 7fb5184bd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb51010c2c0 con 0x7fb5101019c0 2024-05-22T23:01:37.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.952+0000 7fb5184bd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb51010c5c0 con 0x7fb510102380 2024-05-22T23:01:37.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.952+0000 7fb516259640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5101007d0 0x7fb510197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:37.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.952+0000 7fb516259640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5101007d0 0x7fb510197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33722/0 (socket says 172.21.15.84:33722) 2024-05-22T23:01:37.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.952+0000 7fb516259640 1 -- 172.21.15.84:0/3617842709 learned_addr learned my addr 172.21.15.84:0/3617842709 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:37.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.952+0000 7fb516a5a640 1 --2- 172.21.15.84:0/3617842709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb510102380 0x7fb51019f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:37.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.952+0000 7fb516259640 1 -- 172.21.15.84:0/3617842709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb510102380 msgr2=0x7fb51019f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:37.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.952+0000 7fb516259640 1 --2- 172.21.15.84:0/3617842709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb510102380 0x7fb51019f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:37.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.952+0000 7fb516259640 1 -- 172.21.15.84:0/3617842709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5101019c0 msgr2=0x7fb5101980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:37.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.952+0000 7fb516259640 1 --2- 172.21.15.84:0/3617842709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5101019c0 0x7fb5101980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:37.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.952+0000 7fb516259640 1 -- 172.21.15.84:0/3617842709 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51019f7b0 con 0x7fb5101007d0 2024-05-22T23:01:37.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.952+0000 7fb516a5a640 1 --2- 172.21.15.84:0/3617842709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb510102380 0x7fb51019f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:01:37.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.952+0000 7fb516259640 1 --2- 172.21.15.84:0/3617842709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5101007d0 0x7fb510197b90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb50c031610 tx=0x7fb50c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:37.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.952+0000 7fb5077fe640 1 -- 172.21.15.84:0/3617842709 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c03f030 con 0x7fb5101007d0 2024-05-22T23:01:37.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.952+0000 7fb5077fe640 1 -- 172.21.15.84:0/3617842709 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb50c003ce0 con 0x7fb5101007d0 2024-05-22T23:01:37.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.952+0000 7fb5184bd640 1 -- 172.21.15.84:0/3617842709 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51019fa40 con 0x7fb5101007d0 2024-05-22T23:01:37.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.952+0000 7fb5077fe640 1 -- 172.21.15.84:0/3617842709 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c035070 con 0x7fb5101007d0 2024-05-22T23:01:37.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.952+0000 7fb5184bd640 1 -- 172.21.15.84:0/3617842709 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb51019ff30 con 0x7fb5101007d0 2024-05-22T23:01:37.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.953+0000 7fb5184bd640 1 -- 172.21.15.84:0/3617842709 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5101a0160 con 0x7fb5101007d0 2024-05-22T23:01:37.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.955+0000 7fb5077fe640 1 -- 172.21.15.84:0/3617842709 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb50c042030 con 0x7fb5101007d0 2024-05-22T23:01:37.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.956+0000 7fb5077fe640 1 --2- 172.21.15.84:0/3617842709 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb4fc0a2b90 0x7fb4fc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:37.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.956+0000 7fb515a58640 1 --2- 172.21.15.84:0/3617842709 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb4fc0a2b90 0x7fb4fc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:37.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.956+0000 7fb5077fe640 1 -- 172.21.15.84:0/3617842709 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fb50c0ef8f0 con 0x7fb5101007d0 2024-05-22T23:01:37.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.956+0000 7fb515a58640 1 --2- 172.21.15.84:0/3617842709 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb4fc0a2b90 0x7fb4fc0a5050 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb500005fd0 tx=0x7fb500005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:37.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:37.959+0000 7fb5077fe640 1 -- 172.21.15.84:0/3617842709 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb50c0bc830 con 0x7fb5101007d0 2024-05-22T23:01:38.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:37 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1259318197' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-22T23:01:38.137 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.136+0000 7fb5184bd640 1 -- 172.21.15.84:0/3617842709 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5101a0330 con 0x7fb5101007d0 2024-05-22T23:01:38.139 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.137+0000 7fb5077fe640 1 -- 172.21.15.84:0/3617842709 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22722 (secure 0 0 0) 0x7fb50c0bc130 con 0x7fb5101007d0 2024-05-22T23:01:38.139 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:38.139 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":88,"fsid":"c1135a12-188d-11ef-bc9a-c7b262605968","created":"2024-05-22T22:51:49.878218+0000","modified":"2024-05-22T23:00:46.572591+0000","last_up_change":"2024-05-22T22:58:58.550278+0000","last_in_change":"2024-05-22T22:58:37.322139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T22:55:07.719394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T22:59:50.150511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T22:59:51.111182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762df065-fb4f-4c71-aab8-a62d2282c253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6803","nonce":1559945826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6805","nonce":1559945826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6809","nonce":1559945826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6807","nonce":1559945826}]},"public_addr":"172.21.15.84:6803/1559945826","cluster_addr":"172.21.15.84:6805/1559945826","heartbeat_back_addr":"172.21.15.84:6809/1559945826","heartbeat_front_addr":"172.21.15.84:6807/1559945826","state":["exists","up"]},{"osd":1,"uuid":"3fb2385d-6a34-49e3-88e1-116237beb346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6801","nonce":2996505353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6803","nonce":2996505353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6807","nonce":2996505353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6805","nonce":2996505353}]},"public_addr":"172.21.15.96:6801/2996505353","cluster_addr":"172.21.15.96:6803/2996505353","heartbeat_back_addr":"172.21.15.96:6807/2996505353","heartbeat_front_addr":"172.21.15.96:6805/2996505353","state":["exists","up"]},{"osd":2,"uuid":"e306e88e-a308-4585-ae11-c8af10b84038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6801","nonce":1499577897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6803","nonce":1499577897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6807","nonce":1499577897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6805","nonce":1499577897}]},"public_addr":"172.21.15.204:6801/1499577897","cluster_addr":"172.21.15.204:6803/1499577897","heartbeat_back_addr":"172.21.15.204:6807/1499577897","heartbeat_front_addr":"172.21.15.204:6805/1499577897","state":["exists","up"]},{"osd":3,"uuid":"d2a5fb97-6f60-4d45-9f18-447cae480974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6811","nonce":2869340648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6813","nonce":2869340648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6817","nonce":2869340648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6815","nonce":2869340648}]},"public_addr":"172.21.15.84:6811/2869340648","cluster_addr":"172.21.15.84:6813/2869340648","heartbeat_back_addr":"172.21.15.84:6817/2869340648","heartbeat_front_addr":"172.21.15.84:6815/2869340648","state":["exists","up"]},{"osd":4,"uuid":"4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6809","nonce":2554265648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6811","nonce":2554265648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6815","nonce":2554265648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6813","nonce":2554265648}]},"public_addr":"172.21.15.96:6809/2554265648","cluster_addr":"172.21.15.96:6811/2554265648","heartbeat_back_addr":"172.21.15.96:6815/2554265648","heartbeat_front_addr":"172.21.15.96:6813/2554265648","state":["exists","up"]},{"osd":5,"uuid":"7996bf5e-93a5-41b0-a9b1-99e7ef4514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6809","nonce":4075476986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6811","nonce":4075476986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6815","nonce":4075476986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6813","nonce":4075476986}]},"public_addr":"172.21.15.204:6809/4075476986","cluster_addr":"172.21.15.204:6811/4075476986","heartbeat_back_addr":"172.21.15.204:6815/4075476986","heartbeat_front_addr":"172.21.15.204:6813/4075476986","state":["exists","up"]},{"osd":6,"uuid":"f4b7c5de-1836-4d16-8603-3c517524d9fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6819","nonce":1806202865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6821","nonce":1806202865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6825","nonce":1806202865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6823","nonce":1806202865}]},"public_addr":"172.21.15.84:6819/1806202865","cluster_addr":"172.21.15.84:6821/1806202865","heartbeat_back_addr":"172.21.15.84:6825/1806202865","heartbeat_front_addr":"172.21.15.84:6823/1806202865","state":["exists","up"]},{"osd":7,"uuid":"69a18c43-5447-480d-9466-190830070c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6817","nonce":3195792774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6819","nonce":3195792774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6823","nonce":3195792774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6821","nonce":3195792774}]},"public_addr":"172.21.15.96:6817/3195792774","cluster_addr":"172.21.15.96:6819/3195792774","heartbeat_back_addr":"172.21.15.96:6823/3195792774","heartbeat_front_addr":"172.21.15.96:6821/3195792774","state":["exists","up"]},{"osd":8,"uuid":"2c5cadca-689a-441e-847c-ba370d400ef1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6817","nonce":606997927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6819","nonce":606997927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6823","nonce":606997927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6821","nonce":606997927}]},"public_addr":"172.21.15.204:6817/606997927","cluster_addr":"172.21.15.204:6819/606997927","heartbeat_back_addr":"172.21.15.204:6823/606997927","heartbeat_front_addr":"172.21.15.204:6821/606997927","state":["exists","up"]},{"osd":9,"uuid":"e3984fb2-7c00-4275-a10e-bca7c859f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6827","nonce":2786189012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6829","nonce":2786189012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6833","nonce":2786189012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6831","nonce":2786189012}]},"public_addr":"172.21.15.84:6827/2786189012","cluster_addr":"172.21.15.84:6829/2786189012","heartbeat_back_addr":"172.21.15.84:6833/2786189012","heartbeat_front_addr":"172.21.15.84:6831/2786189012","state":["exists","up"]},{"osd":10,"uuid":"b4938160-a808-4ded-91c2-d245c99fff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6825","nonce":1985398526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6827","nonce":1985398526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6831","nonce":1985398526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6829","nonce":1985398526}]},"public_addr":"172.21.15.96:6825/1985398526","cluster_addr":"172.21.15.96:6827/1985398526","heartbeat_back_addr":"172.21.15.96:6831/1985398526","heartbeat_front_addr":"172.21.15.96:6829/1985398526","state":["exists","up"]},{"osd":11,"uuid":"238922ef-d046-40cc-833d-25b7e7a523cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6825","nonce":3106116519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6827","nonce":3106116519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6831","nonce":3106116519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6829","nonce":3106116519}]},"public_addr":"172.21.15.204:6825/3106116519","cluster_addr":"172.21.15.204:6827/3106116519","heartbeat_back_addr":"172.21.15.204:6831/3106116519","heartbeat_front_addr":"172.21.15.204:6829/3106116519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:16.443480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:39.898133+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:03.664544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:28.838687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:53.829733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:19.269307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:45.166910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:10.771614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:36.678516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:03.150592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:29.327769+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:56.315938+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6801/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:6800/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/2766288608":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/950455999":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/4000289608":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1650467363":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1052746114":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6800/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6801/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/14610813":"2024-05-23T22:52:45.395791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T23:01:38.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.139+0000 7fb5184bd640 1 -- 172.21.15.84:0/3617842709 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb4fc0a2b90 msgr2=0x7fb4fc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:38.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.139+0000 7fb5184bd640 1 --2- 172.21.15.84:0/3617842709 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb4fc0a2b90 0x7fb4fc0a5050 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb500005fd0 tx=0x7fb500005850 comp rx=0 tx=0).stop 2024-05-22T23:01:38.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.139+0000 7fb5184bd640 1 -- 172.21.15.84:0/3617842709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5101007d0 msgr2=0x7fb510197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:38.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.139+0000 7fb5184bd640 1 --2- 172.21.15.84:0/3617842709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5101007d0 0x7fb510197b90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb50c031610 tx=0x7fb50c0043b0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.139+0000 7fb5184bd640 1 -- 172.21.15.84:0/3617842709 shutdown_connections 2024-05-22T23:01:38.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.139+0000 7fb5184bd640 1 --2- 172.21.15.84:0/3617842709 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fb4fc0a2b90 0x7fb4fc0a5050 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.139+0000 7fb5184bd640 1 --2- 172.21.15.84:0/3617842709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb510102380 0x7fb51019f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.139+0000 7fb5184bd640 1 --2- 172.21.15.84:0/3617842709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5101019c0 0x7fb5101980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.139+0000 7fb5184bd640 1 --2- 172.21.15.84:0/3617842709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5101007d0 0x7fb510197b90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.139+0000 7fb5184bd640 1 -- 172.21.15.84:0/3617842709 >> 172.21.15.84:0/3617842709 conn(0x7fb5100fbd60 msgr2=0x7fb510069f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:38.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.139+0000 7fb5184bd640 1 -- 172.21.15.84:0/3617842709 shutdown_connections 2024-05-22T23:01:38.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.139+0000 7fb5184bd640 1 -- 172.21.15.84:0/3617842709 wait complete. 2024-05-22T23:01:38.152 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T22:55:07.719394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T22:59:50.150511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T22:59:51.111182+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T23:01:38.153 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T23:01:38.256 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.254+0000 7f7590ddc640 1 -- 172.21.15.84:0/1872364693 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f758c1007b0 msgr2=0x7f758c100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:38.257 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.254+0000 7f7590ddc640 1 --2- 172.21.15.84:0/1872364693 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f758c1007b0 0x7f758c100bd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f757c009900 tx=0x7f757c02a410 comp rx=0 tx=0).stop 2024-05-22T23:01:38.257 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.254+0000 7f7590ddc640 1 -- 172.21.15.84:0/1872364693 shutdown_connections 2024-05-22T23:01:38.257 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.254+0000 7f7590ddc640 1 --2- 172.21.15.84:0/1872364693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f758c102360 0x7f758c108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.257 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.254+0000 7f7590ddc640 1 --2- 172.21.15.84:0/1872364693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f758c1019a0 0x7f758c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.257 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.254+0000 7f7590ddc640 1 --2- 172.21.15.84:0/1872364693 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f758c1007b0 0x7f758c100bd0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.257 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.254+0000 7f7590ddc640 1 -- 172.21.15.84:0/1872364693 >> 172.21.15.84:0/1872364693 conn(0x7f758c0fbd60 msgr2=0x7f758c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:38.257 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.254+0000 7f7590ddc640 1 -- 172.21.15.84:0/1872364693 shutdown_connections 2024-05-22T23:01:38.257 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.254+0000 7f7590ddc640 1 -- 172.21.15.84:0/1872364693 wait complete. 2024-05-22T23:01:38.257 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.255+0000 7f7590ddc640 1 Processor -- start 2024-05-22T23:01:38.259 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.255+0000 7f7590ddc640 1 -- start start 2024-05-22T23:01:38.259 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.255+0000 7f7590ddc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f758c1007b0 0x7f758c199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:38.259 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.255+0000 7f7590ddc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f758c1019a0 0x7f758c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:38.259 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.255+0000 7f7590ddc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f758c102360 0x7f758c1a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:38.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.255+0000 7f7590ddc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f758c10c4a0 con 0x7f758c102360 2024-05-22T23:01:38.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.255+0000 7f7590ddc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f758c10c320 con 0x7f758c1019a0 2024-05-22T23:01:38.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.255+0000 7f7590ddc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f758c10c620 con 0x7f758c1007b0 2024-05-22T23:01:38.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.256+0000 7f758ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f758c102360 0x7f758c1a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:38.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.256+0000 7f758ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f758c102360 0x7f758c1a1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33738/0 (socket says 172.21.15.84:33738) 2024-05-22T23:01:38.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.256+0000 7f758ad9d640 1 -- 172.21.15.84:0/3981549601 learned_addr learned my addr 172.21.15.84:0/3981549601 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:38.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.256+0000 7f758a59c640 1 --2- 172.21.15.84:0/3981549601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f758c1007b0 0x7f758c199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:38.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.256+0000 7f758ad9d640 1 -- 172.21.15.84:0/3981549601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f758c1007b0 msgr2=0x7f758c199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:38.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.256+0000 7f758ad9d640 1 --2- 172.21.15.84:0/3981549601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f758c1007b0 0x7f758c199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.256+0000 7f7581d9b640 1 --2- 172.21.15.84:0/3981549601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f758c1019a0 0x7f758c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:38.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.256+0000 7f758ad9d640 1 -- 172.21.15.84:0/3981549601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f758c1019a0 msgr2=0x7f758c19a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:38.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.256+0000 7f758ad9d640 1 --2- 172.21.15.84:0/3981549601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f758c1019a0 0x7f758c19a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.256+0000 7f758ad9d640 1 -- 172.21.15.84:0/3981549601 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758c1a1ac0 con 0x7f758c102360 2024-05-22T23:01:38.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.256+0000 7f7581d9b640 1 --2- 172.21.15.84:0/3981549601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f758c1019a0 0x7f758c19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:01:38.261 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.256+0000 7f758ad9d640 1 --2- 172.21.15.84:0/3981549601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f758c102360 0x7f758c1a1450 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f758400b970 tx=0x7f758400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:38.261 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.256+0000 7f7583fff640 1 -- 172.21.15.84:0/3981549601 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584019070 con 0x7f758c102360 2024-05-22T23:01:38.261 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.256+0000 7f7590ddc640 1 -- 172.21.15.84:0/3981549601 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f758c1a1db0 con 0x7f758c102360 2024-05-22T23:01:38.261 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.256+0000 7f7583fff640 1 -- 172.21.15.84:0/3981549601 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f758400cb40 con 0x7f758c102360 2024-05-22T23:01:38.261 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.256+0000 7f7583fff640 1 -- 172.21.15.84:0/3981549601 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584007650 con 0x7f758c102360 2024-05-22T23:01:38.261 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.256+0000 7f7590ddc640 1 -- 172.21.15.84:0/3981549601 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f758c1a22f0 con 0x7f758c102360 2024-05-22T23:01:38.261 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.257+0000 7f7590ddc640 1 -- 172.21.15.84:0/3981549601 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f758c1a25b0 con 0x7f758c102360 2024-05-22T23:01:38.262 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.258+0000 7f7583fff640 1 -- 172.21.15.84:0/3981549601 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f758401c070 con 0x7f758c102360 2024-05-22T23:01:38.262 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.259+0000 7f7583fff640 1 --2- 172.21.15.84:0/3981549601 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f75780a2b90 0x7f75780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:38.262 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.259+0000 7f758a59c640 1 --2- 172.21.15.84:0/3981549601 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f75780a2b90 0x7f75780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:38.262 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.259+0000 7f7583fff640 1 -- 172.21.15.84:0/3981549601 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f75840ce6f0 con 0x7f758c102360 2024-05-22T23:01:38.262 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.259+0000 7f758a59c640 1 --2- 172.21.15.84:0/3981549601 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f75780a2b90 0x7f75780a5050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f757c006010 tx=0x7f757c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:38.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.263+0000 7f7583fff640 1 -- 172.21.15.84:0/3981549601 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f758409b630 con 0x7f758c102360 2024-05-22T23:01:38.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.440+0000 7f7590ddc640 1 -- 172.21.15.84:0/3981549601 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f758c1a2780 con 0x7f758c102360 2024-05-22T23:01:38.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.440+0000 7f7583fff640 1 -- 172.21.15.84:0/3981549601 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f758409af30 con 0x7f758c102360 2024-05-22T23:01:38.443 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2024-05-22T23:01:38.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.442+0000 7f7590ddc640 1 -- 172.21.15.84:0/3981549601 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f75780a2b90 msgr2=0x7f75780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:38.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.442+0000 7f7590ddc640 1 --2- 172.21.15.84:0/3981549601 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f75780a2b90 0x7f75780a5050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f757c006010 tx=0x7f757c000bc0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.442+0000 7f7590ddc640 1 -- 172.21.15.84:0/3981549601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f758c102360 msgr2=0x7f758c1a1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:38.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.442+0000 7f7590ddc640 1 --2- 172.21.15.84:0/3981549601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f758c102360 0x7f758c1a1450 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f758400b970 tx=0x7f758400be30 comp rx=0 tx=0).stop 2024-05-22T23:01:38.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.442+0000 7f7590ddc640 1 -- 172.21.15.84:0/3981549601 shutdown_connections 2024-05-22T23:01:38.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.442+0000 7f7590ddc640 1 --2- 172.21.15.84:0/3981549601 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f75780a2b90 0x7f75780a5050 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.442+0000 7f7590ddc640 1 --2- 172.21.15.84:0/3981549601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f758c102360 0x7f758c1a1450 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.442+0000 7f7590ddc640 1 --2- 172.21.15.84:0/3981549601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f758c1019a0 0x7f758c19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.442+0000 7f7590ddc640 1 --2- 172.21.15.84:0/3981549601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f758c1007b0 0x7f758c199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.442+0000 7f7590ddc640 1 -- 172.21.15.84:0/3981549601 >> 172.21.15.84:0/3981549601 conn(0x7f758c0fbd60 msgr2=0x7f758c0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:38.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.442+0000 7f7590ddc640 1 -- 172.21.15.84:0/3981549601 shutdown_connections 2024-05-22T23:01:38.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.443+0000 7f7590ddc640 1 -- 172.21.15.84:0/3981549601 wait complete. 2024-05-22T23:01:38.455 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T23:01:38.590 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.583+0000 7fbd64177640 1 -- 172.21.15.84:0/440433167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd5c102360 msgr2=0x7fbd5c108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:38.590 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.583+0000 7fbd64177640 1 --2- 172.21.15.84:0/440433167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd5c102360 0x7fbd5c108bf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009ca0 tx=0x7fbd5402a8c0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.590 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.583+0000 7fbd64177640 1 -- 172.21.15.84:0/440433167 shutdown_connections 2024-05-22T23:01:38.590 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.583+0000 7fbd64177640 1 --2- 172.21.15.84:0/440433167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd5c102360 0x7fbd5c108bf0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.590 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.583+0000 7fbd64177640 1 --2- 172.21.15.84:0/440433167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd5c1019a0 0x7fbd5c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.590 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.583+0000 7fbd64177640 1 --2- 172.21.15.84:0/440433167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd5c1007b0 0x7fbd5c100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.590 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.583+0000 7fbd64177640 1 -- 172.21.15.84:0/440433167 >> 172.21.15.84:0/440433167 conn(0x7fbd5c0fbd20 msgr2=0x7fbd5c0fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:38.590 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.584+0000 7fbd64177640 1 -- 172.21.15.84:0/440433167 shutdown_connections 2024-05-22T23:01:38.590 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.584+0000 7fbd64177640 1 -- 172.21.15.84:0/440433167 wait complete. 2024-05-22T23:01:38.590 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.584+0000 7fbd64177640 1 Processor -- start 2024-05-22T23:01:38.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.584+0000 7fbd64177640 1 -- start start 2024-05-22T23:01:38.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.584+0000 7fbd64177640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd5c1007b0 0x7fbd5c195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:38.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.584+0000 7fbd64177640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd5c1019a0 0x7fbd5c195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:38.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.584+0000 7fbd64177640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd5c102360 0x7fbd5c19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:38.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.584+0000 7fbd64177640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c10b380 con 0x7fbd5c1019a0 2024-05-22T23:01:38.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.584+0000 7fbd64177640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c10b200 con 0x7fbd5c102360 2024-05-22T23:01:38.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.584+0000 7fbd64177640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c10b500 con 0x7fbd5c1007b0 2024-05-22T23:01:38.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.584+0000 7fbd61712640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd5c1019a0 0x7fbd5c195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:38.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.584+0000 7fbd61712640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd5c1019a0 0x7fbd5c195eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33752/0 (socket says 172.21.15.84:33752) 2024-05-22T23:01:38.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.584+0000 7fbd61712640 1 -- 172.21.15.84:0/1026366977 learned_addr learned my addr 172.21.15.84:0/1026366977 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:38.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.584+0000 7fbd61f13640 1 --2- 172.21.15.84:0/1026366977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd5c1007b0 0x7fbd5c195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:38.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.585+0000 7fbd62714640 1 --2- 172.21.15.84:0/1026366977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd5c102360 0x7fbd5c19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:38.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.585+0000 7fbd61712640 1 -- 172.21.15.84:0/1026366977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd5c1007b0 msgr2=0x7fbd5c195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:38.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.585+0000 7fbd61712640 1 --2- 172.21.15.84:0/1026366977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd5c1007b0 0x7fbd5c195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.585+0000 7fbd61712640 1 -- 172.21.15.84:0/1026366977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd5c102360 msgr2=0x7fbd5c19cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:38.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.585+0000 7fbd61712640 1 --2- 172.21.15.84:0/1026366977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd5c102360 0x7fbd5c19cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.585+0000 7fbd61712640 1 -- 172.21.15.84:0/1026366977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5c19d590 con 0x7fbd5c1019a0 2024-05-22T23:01:38.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.585+0000 7fbd62714640 1 --2- 172.21.15.84:0/1026366977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd5c102360 0x7fbd5c19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:01:38.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.585+0000 7fbd61712640 1 --2- 172.21.15.84:0/1026366977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd5c1019a0 0x7fbd5c195eb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00e9f0 tx=0x7fbd4c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:38.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.585+0000 7fbd52ffd640 1 -- 172.21.15.84:0/1026366977 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c00cd80 con 0x7fbd5c1019a0 2024-05-22T23:01:38.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.585+0000 7fbd52ffd640 1 -- 172.21.15.84:0/1026366977 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd4c010910 con 0x7fbd5c1019a0 2024-05-22T23:01:38.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.585+0000 7fbd64177640 1 -- 172.21.15.84:0/1026366977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd5c19d880 con 0x7fbd5c1019a0 2024-05-22T23:01:38.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.585+0000 7fbd52ffd640 1 -- 172.21.15.84:0/1026366977 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c018990 con 0x7fbd5c1019a0 2024-05-22T23:01:38.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.585+0000 7fbd64177640 1 -- 172.21.15.84:0/1026366977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd5c19ddd0 con 0x7fbd5c1019a0 2024-05-22T23:01:38.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.586+0000 7fbd64177640 1 -- 172.21.15.84:0/1026366977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd5c107ad0 con 0x7fbd5c1019a0 2024-05-22T23:01:38.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.587+0000 7fbd52ffd640 1 -- 172.21.15.84:0/1026366977 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbd4c010ac0 con 0x7fbd5c1019a0 2024-05-22T23:01:38.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.587+0000 7fbd52ffd640 1 --2- 172.21.15.84:0/1026366977 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fbd480a4da0 0x7fbd480a7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:38.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.588+0000 7fbd61f13640 1 --2- 172.21.15.84:0/1026366977 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fbd480a4da0 0x7fbd480a7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:38.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.588+0000 7fbd52ffd640 1 -- 172.21.15.84:0/1026366977 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fbd4c0cf420 con 0x7fbd5c1019a0 2024-05-22T23:01:38.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.588+0000 7fbd61f13640 1 --2- 172.21.15.84:0/1026366977 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fbd480a4da0 0x7fbd480a7260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c101800 tx=0x7fbd58005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:38.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.592+0000 7fbd52ffd640 1 -- 172.21.15.84:0/1026366977 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbd4c09c360 con 0x7fbd5c1019a0 2024-05-22T23:01:38.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.768+0000 7fbd64177640 1 -- 172.21.15.84:0/1026366977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbd5c062470 con 0x7fbd5c1019a0 2024-05-22T23:01:38.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.768+0000 7fbd52ffd640 1 -- 172.21.15.84:0/1026366977 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fbd4c09bc60 con 0x7fbd5c1019a0 2024-05-22T23:01:38.770 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2024-05-22T23:01:38.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.770+0000 7fbd64177640 1 -- 172.21.15.84:0/1026366977 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fbd480a4da0 msgr2=0x7fbd480a7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:38.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.770+0000 7fbd64177640 1 --2- 172.21.15.84:0/1026366977 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fbd480a4da0 0x7fbd480a7260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c101800 tx=0x7fbd58005f90 comp rx=0 tx=0).stop 2024-05-22T23:01:38.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.770+0000 7fbd64177640 1 -- 172.21.15.84:0/1026366977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd5c1019a0 msgr2=0x7fbd5c195eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:38.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.770+0000 7fbd64177640 1 --2- 172.21.15.84:0/1026366977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd5c1019a0 0x7fbd5c195eb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00e9f0 tx=0x7fbd4c00eeb0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.770+0000 7fbd64177640 1 -- 172.21.15.84:0/1026366977 shutdown_connections 2024-05-22T23:01:38.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.770+0000 7fbd64177640 1 --2- 172.21.15.84:0/1026366977 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fbd480a4da0 0x7fbd480a7260 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.770+0000 7fbd64177640 1 --2- 172.21.15.84:0/1026366977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd5c102360 0x7fbd5c19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.770+0000 7fbd64177640 1 --2- 172.21.15.84:0/1026366977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd5c1019a0 0x7fbd5c195eb0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.770+0000 7fbd64177640 1 --2- 172.21.15.84:0/1026366977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd5c1007b0 0x7fbd5c195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.770+0000 7fbd64177640 1 -- 172.21.15.84:0/1026366977 >> 172.21.15.84:0/1026366977 conn(0x7fbd5c0fbd20 msgr2=0x7fbd5c104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:38.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.770+0000 7fbd64177640 1 -- 172.21.15.84:0/1026366977 shutdown_connections 2024-05-22T23:01:38.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.770+0000 7fbd64177640 1 -- 172.21.15.84:0/1026366977 wait complete. 2024-05-22T23:01:38.782 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:38 smithi084 ceph-mon[32186]: pgmap v344: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:01:38.782 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:38 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3617842709' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:38.782 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:38 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3981549601' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:01:38.783 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T23:01:38.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.887+0000 7f1c2f5a9640 1 -- 172.21.15.84:0/3989691072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c280ffb80 msgr2=0x7f1c280fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:38.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.887+0000 7f1c2f5a9640 1 --2- 172.21.15.84:0/3989691072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c280ffb80 0x7f1c280fffa0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009930 tx=0x7f1c2002a420 comp rx=0 tx=0).stop 2024-05-22T23:01:38.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.887+0000 7f1c2f5a9640 1 -- 172.21.15.84:0/3989691072 shutdown_connections 2024-05-22T23:01:38.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.887+0000 7f1c2f5a9640 1 --2- 172.21.15.84:0/3989691072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c28103510 0x7f1c28109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.887+0000 7f1c2f5a9640 1 --2- 172.21.15.84:0/3989691072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c28102b50 0x7f1c28102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.887+0000 7f1c2f5a9640 1 --2- 172.21.15.84:0/3989691072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c280ffb80 0x7f1c280fffa0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.887+0000 7f1c2f5a9640 1 -- 172.21.15.84:0/3989691072 >> 172.21.15.84:0/3989691072 conn(0x7f1c280fb4b0 msgr2=0x7f1c280fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:38.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.887+0000 7f1c2f5a9640 1 -- 172.21.15.84:0/3989691072 shutdown_connections 2024-05-22T23:01:38.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.887+0000 7f1c2f5a9640 1 -- 172.21.15.84:0/3989691072 wait complete. 2024-05-22T23:01:38.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.887+0000 7f1c2f5a9640 1 Processor -- start 2024-05-22T23:01:38.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.887+0000 7f1c2f5a9640 1 -- start start 2024-05-22T23:01:38.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.887+0000 7f1c2f5a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c280ffb80 0x7f1c28197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:38.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.887+0000 7f1c2f5a9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c28102b50 0x7f1c281981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:38.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.887+0000 7f1c2f5a9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c28103510 0x7f1c2819f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:38.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.887+0000 7f1c2f5a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2810d5b0 con 0x7f1c280ffb80 2024-05-22T23:01:38.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.887+0000 7f1c2f5a9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2810d430 con 0x7f1c28102b50 2024-05-22T23:01:38.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.887+0000 7f1c2f5a9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2810d730 con 0x7f1c28103510 2024-05-22T23:01:38.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.888+0000 7f1c2d345640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c280ffb80 0x7f1c28197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:38.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.888+0000 7f1c2d345640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c280ffb80 0x7f1c28197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33760/0 (socket says 172.21.15.84:33760) 2024-05-22T23:01:38.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.888+0000 7f1c2d345640 1 -- 172.21.15.84:0/3362803409 learned_addr learned my addr 172.21.15.84:0/3362803409 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:38.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.888+0000 7f1c2cb44640 1 --2- 172.21.15.84:0/3362803409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c28102b50 0x7f1c281981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:38.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.888+0000 7f1c2db46640 1 --2- 172.21.15.84:0/3362803409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c28103510 0x7f1c2819f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:38.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.888+0000 7f1c2d345640 1 -- 172.21.15.84:0/3362803409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c28103510 msgr2=0x7f1c2819f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:38.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.888+0000 7f1c2d345640 1 --2- 172.21.15.84:0/3362803409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c28103510 0x7f1c2819f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.888+0000 7f1c2d345640 1 -- 172.21.15.84:0/3362803409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c28102b50 msgr2=0x7f1c281981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:38.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.888+0000 7f1c2d345640 1 --2- 172.21.15.84:0/3362803409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c28102b50 0x7f1c281981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:38.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.888+0000 7f1c2d345640 1 -- 172.21.15.84:0/3362803409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2819f8d0 con 0x7f1c280ffb80 2024-05-22T23:01:38.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.888+0000 7f1c2d345640 1 --2- 172.21.15.84:0/3362803409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c280ffb80 0x7f1c28197c90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1c20031440 tx=0x7f1c200043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:38.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.888+0000 7f1c267fc640 1 -- 172.21.15.84:0/3362803409 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2003f030 con 0x7f1c280ffb80 2024-05-22T23:01:38.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.888+0000 7f1c267fc640 1 -- 172.21.15.84:0/3362803409 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c20003ce0 con 0x7f1c280ffb80 2024-05-22T23:01:38.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.888+0000 7f1c2f5a9640 1 -- 172.21.15.84:0/3362803409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c2819fb60 con 0x7f1c280ffb80 2024-05-22T23:01:38.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.888+0000 7f1c267fc640 1 -- 172.21.15.84:0/3362803409 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20035070 con 0x7f1c280ffb80 2024-05-22T23:01:38.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.888+0000 7f1c2f5a9640 1 -- 172.21.15.84:0/3362803409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c281a0050 con 0x7f1c280ffb80 2024-05-22T23:01:38.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.889+0000 7f1c2f5a9640 1 -- 172.21.15.84:0/3362803409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c28108d00 con 0x7f1c280ffb80 2024-05-22T23:01:38.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.890+0000 7f1c267fc640 1 -- 172.21.15.84:0/3362803409 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1c20042030 con 0x7f1c280ffb80 2024-05-22T23:01:38.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.891+0000 7f1c267fc640 1 --2- 172.21.15.84:0/3362803409 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1c100a2b90 0x7f1c100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:38.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.891+0000 7f1c2cb44640 1 --2- 172.21.15.84:0/3362803409 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1c100a2b90 0x7f1c100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:38.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.891+0000 7f1c267fc640 1 -- 172.21.15.84:0/3362803409 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f1c200ef5b0 con 0x7f1c280ffb80 2024-05-22T23:01:38.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.891+0000 7f1c2cb44640 1 --2- 172.21.15.84:0/3362803409 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1c100a2b90 0x7f1c100a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1c28199190 tx=0x7f1c1c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:38.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:38.895+0000 7f1c267fc640 1 -- 172.21.15.84:0/3362803409 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1c200bc4f0 con 0x7f1c280ffb80 2024-05-22T23:01:38.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:38 smithi204 ceph-mon[36211]: pgmap v344: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:01:38.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:38 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3617842709' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:38.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:38 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3981549601' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:01:39.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:38 smithi096 ceph-mon[36392]: pgmap v344: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:01:39.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:38 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3617842709' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:39.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:38 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3981549601' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:01:39.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.069+0000 7f1c2f5a9640 1 -- 172.21.15.84:0/3362803409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1c2802d200 con 0x7f1c280ffb80 2024-05-22T23:01:39.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.069+0000 7f1c267fc640 1 -- 172.21.15.84:0/3362803409 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f1c200bbdf0 con 0x7f1c280ffb80 2024-05-22T23:01:39.071 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2024-05-22T23:01:39.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.071+0000 7f1c2f5a9640 1 -- 172.21.15.84:0/3362803409 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1c100a2b90 msgr2=0x7f1c100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:39.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.071+0000 7f1c2f5a9640 1 --2- 172.21.15.84:0/3362803409 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1c100a2b90 0x7f1c100a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1c28199190 tx=0x7f1c1c00b390 comp rx=0 tx=0).stop 2024-05-22T23:01:39.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.071+0000 7f1c2f5a9640 1 -- 172.21.15.84:0/3362803409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c280ffb80 msgr2=0x7f1c28197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:39.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.071+0000 7f1c2f5a9640 1 --2- 172.21.15.84:0/3362803409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c280ffb80 0x7f1c28197c90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1c20031440 tx=0x7f1c200043b0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.071+0000 7f1c2f5a9640 1 -- 172.21.15.84:0/3362803409 shutdown_connections 2024-05-22T23:01:39.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.071+0000 7f1c2f5a9640 1 --2- 172.21.15.84:0/3362803409 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1c100a2b90 0x7f1c100a5050 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.071+0000 7f1c2f5a9640 1 --2- 172.21.15.84:0/3362803409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c28103510 0x7f1c2819f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.071+0000 7f1c2f5a9640 1 --2- 172.21.15.84:0/3362803409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c28102b50 0x7f1c281981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.071+0000 7f1c2f5a9640 1 --2- 172.21.15.84:0/3362803409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c280ffb80 0x7f1c28197c90 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.071+0000 7f1c2f5a9640 1 -- 172.21.15.84:0/3362803409 >> 172.21.15.84:0/3362803409 conn(0x7f1c280fb4b0 msgr2=0x7f1c28105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:39.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.071+0000 7f1c2f5a9640 1 -- 172.21.15.84:0/3362803409 shutdown_connections 2024-05-22T23:01:39.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.071+0000 7f1c2f5a9640 1 -- 172.21.15.84:0/3362803409 wait complete. 2024-05-22T23:01:39.084 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:01:39.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.183+0000 7f0bf80ee640 1 -- 172.21.15.84:0/172965806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bf01007d0 msgr2=0x7f0bf0100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:39.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.183+0000 7f0bf80ee640 1 --2- 172.21.15.84:0/172965806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bf01007d0 0x7f0bf0100bf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009900 tx=0x7f0bec02a410 comp rx=0 tx=0).stop 2024-05-22T23:01:39.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.183+0000 7f0bf80ee640 1 -- 172.21.15.84:0/172965806 shutdown_connections 2024-05-22T23:01:39.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.183+0000 7f0bf80ee640 1 --2- 172.21.15.84:0/172965806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bf0102380 0x7f0bf0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.183+0000 7f0bf80ee640 1 --2- 172.21.15.84:0/172965806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bf01019c0 0x7f0bf0101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.183+0000 7f0bf80ee640 1 --2- 172.21.15.84:0/172965806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bf01007d0 0x7f0bf0100bf0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.183+0000 7f0bf80ee640 1 -- 172.21.15.84:0/172965806 >> 172.21.15.84:0/172965806 conn(0x7f0bf00fbd60 msgr2=0x7f0bf00fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:39.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.183+0000 7f0bf80ee640 1 -- 172.21.15.84:0/172965806 shutdown_connections 2024-05-22T23:01:39.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.183+0000 7f0bf80ee640 1 -- 172.21.15.84:0/172965806 wait complete. 2024-05-22T23:01:39.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.184+0000 7f0bf80ee640 1 Processor -- start 2024-05-22T23:01:39.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.184+0000 7f0bf80ee640 1 -- start start 2024-05-22T23:01:39.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.184+0000 7f0bf80ee640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bf01007d0 0x7f0bf0197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:39.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.184+0000 7f0bf80ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bf01019c0 0x7f0bf01980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:39.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.184+0000 7f0bf80ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bf0102380 0x7f0bf019f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:39.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.184+0000 7f0bf80ee640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf010d570 con 0x7f0bf01019c0 2024-05-22T23:01:39.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.184+0000 7f0bf80ee640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf010d3f0 con 0x7f0bf0102380 2024-05-22T23:01:39.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.184+0000 7f0bf80ee640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf010d6f0 con 0x7f0bf01007d0 2024-05-22T23:01:39.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.184+0000 7f0bf5689640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bf01019c0 0x7f0bf01980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:39.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.184+0000 7f0bf5689640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bf01019c0 0x7f0bf01980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33770/0 (socket says 172.21.15.84:33770) 2024-05-22T23:01:39.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.184+0000 7f0bf5689640 1 -- 172.21.15.84:0/81006688 learned_addr learned my addr 172.21.15.84:0/81006688 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:39.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.184+0000 7f0bf5e8a640 1 --2- 172.21.15.84:0/81006688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bf01007d0 0x7f0bf0197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:39.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.184+0000 7f0bf5689640 1 -- 172.21.15.84:0/81006688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bf01007d0 msgr2=0x7f0bf0197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:39.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.184+0000 7f0bf668b640 1 --2- 172.21.15.84:0/81006688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bf0102380 0x7f0bf019f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:39.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.184+0000 7f0bf5689640 1 --2- 172.21.15.84:0/81006688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bf01007d0 0x7f0bf0197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.184+0000 7f0bf5689640 1 -- 172.21.15.84:0/81006688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bf0102380 msgr2=0x7f0bf019f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:39.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.184+0000 7f0bf5689640 1 --2- 172.21.15.84:0/81006688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bf0102380 0x7f0bf019f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.184+0000 7f0bf5689640 1 -- 172.21.15.84:0/81006688 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf019f7c0 con 0x7f0bf01019c0 2024-05-22T23:01:39.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.184+0000 7f0bf5e8a640 1 --2- 172.21.15.84:0/81006688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bf01007d0 0x7f0bf0197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:01:39.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.184+0000 7f0bf668b640 1 --2- 172.21.15.84:0/81006688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bf0102380 0x7f0bf019f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:01:39.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.184+0000 7f0bf5689640 1 --2- 172.21.15.84:0/81006688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bf01019c0 0x7f0bf01980e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0be000c970 tx=0x7f0be000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:39.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.185+0000 7f0be6ffd640 1 -- 172.21.15.84:0/81006688 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be0007bb0 con 0x7f0bf01019c0 2024-05-22T23:01:39.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.185+0000 7f0be6ffd640 1 -- 172.21.15.84:0/81006688 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0be0007d50 con 0x7f0bf01019c0 2024-05-22T23:01:39.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.185+0000 7f0bf80ee640 1 -- 172.21.15.84:0/81006688 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf019fab0 con 0x7f0bf01019c0 2024-05-22T23:01:39.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.185+0000 7f0be6ffd640 1 -- 172.21.15.84:0/81006688 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be000f450 con 0x7f0bf01019c0 2024-05-22T23:01:39.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.185+0000 7f0bf80ee640 1 -- 172.21.15.84:0/81006688 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf01a0000 con 0x7f0bf01019c0 2024-05-22T23:01:39.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.186+0000 7f0bf80ee640 1 -- 172.21.15.84:0/81006688 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bf0107af0 con 0x7f0bf01019c0 2024-05-22T23:01:39.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.187+0000 7f0be6ffd640 1 -- 172.21.15.84:0/81006688 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0be0005020 con 0x7f0bf01019c0 2024-05-22T23:01:39.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.188+0000 7f0be6ffd640 1 --2- 172.21.15.84:0/81006688 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0bdc0a2c10 0x7f0bdc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:39.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.188+0000 7f0bf5e8a640 1 --2- 172.21.15.84:0/81006688 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0bdc0a2c10 0x7f0bdc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:39.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.188+0000 7f0be6ffd640 1 -- 172.21.15.84:0/81006688 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f0be00cee80 con 0x7f0bf01019c0 2024-05-22T23:01:39.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.188+0000 7f0bf5e8a640 1 --2- 172.21.15.84:0/81006688 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0bdc0a2c10 0x7f0bdc0a50d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0bec000c00 tx=0x7f0bec031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:39.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.192+0000 7f0be6ffd640 1 -- 172.21.15.84:0/81006688 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0be0014030 con 0x7f0bf01019c0 2024-05-22T23:01:39.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.405+0000 7f0bf80ee640 1 -- 172.21.15.84:0/81006688 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0bf002d200 con 0x7f0bf01019c0 2024-05-22T23:01:39.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.405+0000 7f0be6ffd640 1 -- 172.21.15.84:0/81006688 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7f0be009bdc0 con 0x7f0bf01019c0 2024-05-22T23:01:39.407 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:39.407 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:01:39.408 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:01:39.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.407+0000 7f0bf80ee640 1 -- 172.21.15.84:0/81006688 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0bdc0a2c10 msgr2=0x7f0bdc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:39.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.407+0000 7f0bf80ee640 1 --2- 172.21.15.84:0/81006688 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0bdc0a2c10 0x7f0bdc0a50d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0bec000c00 tx=0x7f0bec031400 comp rx=0 tx=0).stop 2024-05-22T23:01:39.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.407+0000 7f0bf80ee640 1 -- 172.21.15.84:0/81006688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bf01019c0 msgr2=0x7f0bf01980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:39.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.407+0000 7f0bf80ee640 1 --2- 172.21.15.84:0/81006688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bf01019c0 0x7f0bf01980e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0be000c970 tx=0x7f0be000ce30 comp rx=0 tx=0).stop 2024-05-22T23:01:39.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.407+0000 7f0bf80ee640 1 -- 172.21.15.84:0/81006688 shutdown_connections 2024-05-22T23:01:39.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.407+0000 7f0bf80ee640 1 --2- 172.21.15.84:0/81006688 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f0bdc0a2c10 0x7f0bdc0a50d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.407+0000 7f0bf80ee640 1 --2- 172.21.15.84:0/81006688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bf0102380 0x7f0bf019f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.407+0000 7f0bf80ee640 1 --2- 172.21.15.84:0/81006688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bf01019c0 0x7f0bf01980e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.407+0000 7f0bf80ee640 1 --2- 172.21.15.84:0/81006688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bf01007d0 0x7f0bf0197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.408+0000 7f0bf80ee640 1 -- 172.21.15.84:0/81006688 >> 172.21.15.84:0/81006688 conn(0x7f0bf00fbd60 msgr2=0x7f0bf0104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:39.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.408+0000 7f0bf80ee640 1 -- 172.21.15.84:0/81006688 shutdown_connections 2024-05-22T23:01:39.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.408+0000 7f0bf80ee640 1 -- 172.21.15.84:0/81006688 wait complete. 2024-05-22T23:01:39.421 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T23:01:39.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.520+0000 7fec24d9e640 1 -- 172.21.15.84:0/1377332487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec20102360 msgr2=0x7fec20108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:39.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.520+0000 7fec24d9e640 1 --2- 172.21.15.84:0/1377332487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec20102360 0x7fec20108bf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fec18009ca0 tx=0x7fec1802a8c0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.521+0000 7fec24d9e640 1 -- 172.21.15.84:0/1377332487 shutdown_connections 2024-05-22T23:01:39.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.521+0000 7fec24d9e640 1 --2- 172.21.15.84:0/1377332487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec20102360 0x7fec20108bf0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.521+0000 7fec24d9e640 1 --2- 172.21.15.84:0/1377332487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec201019a0 0x7fec20101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.521+0000 7fec24d9e640 1 --2- 172.21.15.84:0/1377332487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec201007b0 0x7fec20100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.521+0000 7fec24d9e640 1 -- 172.21.15.84:0/1377332487 >> 172.21.15.84:0/1377332487 conn(0x7fec200fbd20 msgr2=0x7fec200fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:39.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.521+0000 7fec24d9e640 1 -- 172.21.15.84:0/1377332487 shutdown_connections 2024-05-22T23:01:39.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.521+0000 7fec24d9e640 1 -- 172.21.15.84:0/1377332487 wait complete. 2024-05-22T23:01:39.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.521+0000 7fec24d9e640 1 Processor -- start 2024-05-22T23:01:39.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.521+0000 7fec24d9e640 1 -- start start 2024-05-22T23:01:39.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.522+0000 7fec24d9e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec201007b0 0x7fec20069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:39.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.522+0000 7fec24d9e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec201019a0 0x7fec2019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:39.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.522+0000 7fec24d9e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec20102360 0x7fec2019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:39.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.522+0000 7fec24d9e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fec2010c430 con 0x7fec201007b0 2024-05-22T23:01:39.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.522+0000 7fec24d9e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fec2010c2b0 con 0x7fec201019a0 2024-05-22T23:01:39.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.522+0000 7fec24d9e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fec2010c5b0 con 0x7fec20102360 2024-05-22T23:01:39.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.522+0000 7fec1e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec201007b0 0x7fec20069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:39.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.522+0000 7fec1e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec201007b0 0x7fec20069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33786/0 (socket says 172.21.15.84:33786) 2024-05-22T23:01:39.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.522+0000 7fec1e59c640 1 -- 172.21.15.84:0/1211728901 learned_addr learned my addr 172.21.15.84:0/1211728901 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:39.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.522+0000 7fec1dd9b640 1 --2- 172.21.15.84:0/1211728901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec201019a0 0x7fec2019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:39.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.522+0000 7fec1e59c640 1 -- 172.21.15.84:0/1211728901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec20102360 msgr2=0x7fec2019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:39.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.522+0000 7fec1e59c640 1 --2- 172.21.15.84:0/1211728901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec20102360 0x7fec2019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.522+0000 7fec1e59c640 1 -- 172.21.15.84:0/1211728901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec201019a0 msgr2=0x7fec2019c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:39.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.522+0000 7fec1e59c640 1 --2- 172.21.15.84:0/1211728901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec201019a0 0x7fec2019c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.522+0000 7fec1e59c640 1 -- 172.21.15.84:0/1211728901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2019f7a0 con 0x7fec201007b0 2024-05-22T23:01:39.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.522+0000 7fec1dd9b640 1 --2- 172.21.15.84:0/1211728901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec201019a0 0x7fec2019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:01:39.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.522+0000 7fec1e59c640 1 --2- 172.21.15.84:0/1211728901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec201007b0 0x7fec20069590 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fec1000d770 tx=0x7fec1000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:39.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.522+0000 7fec177fe640 1 -- 172.21.15.84:0/1211728901 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec10017070 con 0x7fec201007b0 2024-05-22T23:01:39.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.522+0000 7fec24d9e640 1 -- 172.21.15.84:0/1211728901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2019fa90 con 0x7fec201007b0 2024-05-22T23:01:39.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.522+0000 7fec177fe640 1 -- 172.21.15.84:0/1211728901 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec10009c30 con 0x7fec201007b0 2024-05-22T23:01:39.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.522+0000 7fec177fe640 1 -- 172.21.15.84:0/1211728901 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec10003dd0 con 0x7fec201007b0 2024-05-22T23:01:39.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.522+0000 7fec24d9e640 1 -- 172.21.15.84:0/1211728901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec2019ffe0 con 0x7fec201007b0 2024-05-22T23:01:39.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.523+0000 7fec24d9e640 1 -- 172.21.15.84:0/1211728901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec20107ad0 con 0x7fec201007b0 2024-05-22T23:01:39.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.524+0000 7fec177fe640 1 -- 172.21.15.84:0/1211728901 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fec10003f70 con 0x7fec201007b0 2024-05-22T23:01:39.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.525+0000 7fec177fe640 1 --2- 172.21.15.84:0/1211728901 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fec0c0a2b90 0x7fec0c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:39.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.525+0000 7fec1dd9b640 1 --2- 172.21.15.84:0/1211728901 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fec0c0a2b90 0x7fec0c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:39.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.525+0000 7fec177fe640 1 -- 172.21.15.84:0/1211728901 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fec100cead0 con 0x7fec201007b0 2024-05-22T23:01:39.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.525+0000 7fec1dd9b640 1 --2- 172.21.15.84:0/1211728901 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fec0c0a2b90 0x7fec0c0a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fec08005fd0 tx=0x7fec08005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:39.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.529+0000 7fec177fe640 1 -- 172.21.15.84:0/1211728901 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fec1009ba10 con 0x7fec201007b0 2024-05-22T23:01:39.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.706+0000 7fec24d9e640 1 -- 172.21.15.84:0/1211728901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec20062470 con 0x7fec201007b0 2024-05-22T23:01:39.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.708+0000 7fec177fe640 1 -- 172.21.15.84:0/1211728901 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22722 (secure 0 0 0) 0x7fec10009de0 con 0x7fec201007b0 2024-05-22T23:01:39.711 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:39.711 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":88,"fsid":"c1135a12-188d-11ef-bc9a-c7b262605968","created":"2024-05-22T22:51:49.878218+0000","modified":"2024-05-22T23:00:46.572591+0000","last_up_change":"2024-05-22T22:58:58.550278+0000","last_in_change":"2024-05-22T22:58:37.322139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T22:55:07.719394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T22:59:50.150511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T22:59:51.111182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762df065-fb4f-4c71-aab8-a62d2282c253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6803","nonce":1559945826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6805","nonce":1559945826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6809","nonce":1559945826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6807","nonce":1559945826}]},"public_addr":"172.21.15.84:6803/1559945826","cluster_addr":"172.21.15.84:6805/1559945826","heartbeat_back_addr":"172.21.15.84:6809/1559945826","heartbeat_front_addr":"172.21.15.84:6807/1559945826","state":["exists","up"]},{"osd":1,"uuid":"3fb2385d-6a34-49e3-88e1-116237beb346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6801","nonce":2996505353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6803","nonce":2996505353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6807","nonce":2996505353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6805","nonce":2996505353}]},"public_addr":"172.21.15.96:6801/2996505353","cluster_addr":"172.21.15.96:6803/2996505353","heartbeat_back_addr":"172.21.15.96:6807/2996505353","heartbeat_front_addr":"172.21.15.96:6805/2996505353","state":["exists","up"]},{"osd":2,"uuid":"e306e88e-a308-4585-ae11-c8af10b84038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6801","nonce":1499577897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6803","nonce":1499577897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6807","nonce":1499577897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6805","nonce":1499577897}]},"public_addr":"172.21.15.204:6801/1499577897","cluster_addr":"172.21.15.204:6803/1499577897","heartbeat_back_addr":"172.21.15.204:6807/1499577897","heartbeat_front_addr":"172.21.15.204:6805/1499577897","state":["exists","up"]},{"osd":3,"uuid":"d2a5fb97-6f60-4d45-9f18-447cae480974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6811","nonce":2869340648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6813","nonce":2869340648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6817","nonce":2869340648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6815","nonce":2869340648}]},"public_addr":"172.21.15.84:6811/2869340648","cluster_addr":"172.21.15.84:6813/2869340648","heartbeat_back_addr":"172.21.15.84:6817/2869340648","heartbeat_front_addr":"172.21.15.84:6815/2869340648","state":["exists","up"]},{"osd":4,"uuid":"4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6809","nonce":2554265648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6811","nonce":2554265648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6815","nonce":2554265648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6813","nonce":2554265648}]},"public_addr":"172.21.15.96:6809/2554265648","cluster_addr":"172.21.15.96:6811/2554265648","heartbeat_back_addr":"172.21.15.96:6815/2554265648","heartbeat_front_addr":"172.21.15.96:6813/2554265648","state":["exists","up"]},{"osd":5,"uuid":"7996bf5e-93a5-41b0-a9b1-99e7ef4514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6809","nonce":4075476986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6811","nonce":4075476986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6815","nonce":4075476986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6813","nonce":4075476986}]},"public_addr":"172.21.15.204:6809/4075476986","cluster_addr":"172.21.15.204:6811/4075476986","heartbeat_back_addr":"172.21.15.204:6815/4075476986","heartbeat_front_addr":"172.21.15.204:6813/4075476986","state":["exists","up"]},{"osd":6,"uuid":"f4b7c5de-1836-4d16-8603-3c517524d9fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6819","nonce":1806202865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6821","nonce":1806202865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6825","nonce":1806202865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6823","nonce":1806202865}]},"public_addr":"172.21.15.84:6819/1806202865","cluster_addr":"172.21.15.84:6821/1806202865","heartbeat_back_addr":"172.21.15.84:6825/1806202865","heartbeat_front_addr":"172.21.15.84:6823/1806202865","state":["exists","up"]},{"osd":7,"uuid":"69a18c43-5447-480d-9466-190830070c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6817","nonce":3195792774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6819","nonce":3195792774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6823","nonce":3195792774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6821","nonce":3195792774}]},"public_addr":"172.21.15.96:6817/3195792774","cluster_addr":"172.21.15.96:6819/3195792774","heartbeat_back_addr":"172.21.15.96:6823/3195792774","heartbeat_front_addr":"172.21.15.96:6821/3195792774","state":["exists","up"]},{"osd":8,"uuid":"2c5cadca-689a-441e-847c-ba370d400ef1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6817","nonce":606997927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6819","nonce":606997927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6823","nonce":606997927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6821","nonce":606997927}]},"public_addr":"172.21.15.204:6817/606997927","cluster_addr":"172.21.15.204:6819/606997927","heartbeat_back_addr":"172.21.15.204:6823/606997927","heartbeat_front_addr":"172.21.15.204:6821/606997927","state":["exists","up"]},{"osd":9,"uuid":"e3984fb2-7c00-4275-a10e-bca7c859f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6827","nonce":2786189012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6829","nonce":2786189012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6833","nonce":2786189012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6831","nonce":2786189012}]},"public_addr":"172.21.15.84:6827/2786189012","cluster_addr":"172.21.15.84:6829/2786189012","heartbeat_back_addr":"172.21.15.84:6833/2786189012","heartbeat_front_addr":"172.21.15.84:6831/2786189012","state":["exists","up"]},{"osd":10,"uuid":"b4938160-a808-4ded-91c2-d245c99fff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6825","nonce":1985398526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6827","nonce":1985398526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6831","nonce":1985398526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6829","nonce":1985398526}]},"public_addr":"172.21.15.96:6825/1985398526","cluster_addr":"172.21.15.96:6827/1985398526","heartbeat_back_addr":"172.21.15.96:6831/1985398526","heartbeat_front_addr":"172.21.15.96:6829/1985398526","state":["exists","up"]},{"osd":11,"uuid":"238922ef-d046-40cc-833d-25b7e7a523cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6825","nonce":3106116519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6827","nonce":3106116519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6831","nonce":3106116519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6829","nonce":3106116519}]},"public_addr":"172.21.15.204:6825/3106116519","cluster_addr":"172.21.15.204:6827/3106116519","heartbeat_back_addr":"172.21.15.204:6831/3106116519","heartbeat_front_addr":"172.21.15.204:6829/3106116519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:16.443480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:39.898133+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:03.664544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:28.838687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:53.829733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:19.269307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:45.166910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:10.771614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:36.678516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:03.150592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:29.327769+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:56.315938+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6801/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:6800/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/2766288608":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/950455999":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/4000289608":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1650467363":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1052746114":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6800/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6801/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/14610813":"2024-05-23T22:52:45.395791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T23:01:39.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.710+0000 7fec24d9e640 1 -- 172.21.15.84:0/1211728901 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fec0c0a2b90 msgr2=0x7fec0c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:39.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.710+0000 7fec24d9e640 1 --2- 172.21.15.84:0/1211728901 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fec0c0a2b90 0x7fec0c0a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fec08005fd0 tx=0x7fec08005ea0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.710+0000 7fec24d9e640 1 -- 172.21.15.84:0/1211728901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec201007b0 msgr2=0x7fec20069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:39.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.710+0000 7fec24d9e640 1 --2- 172.21.15.84:0/1211728901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec201007b0 0x7fec20069590 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fec1000d770 tx=0x7fec1000dc30 comp rx=0 tx=0).stop 2024-05-22T23:01:39.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.710+0000 7fec24d9e640 1 -- 172.21.15.84:0/1211728901 shutdown_connections 2024-05-22T23:01:39.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.710+0000 7fec24d9e640 1 --2- 172.21.15.84:0/1211728901 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fec0c0a2b90 0x7fec0c0a5050 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.710+0000 7fec24d9e640 1 --2- 172.21.15.84:0/1211728901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec20102360 0x7fec2019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.710+0000 7fec24d9e640 1 --2- 172.21.15.84:0/1211728901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec201019a0 0x7fec2019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.710+0000 7fec24d9e640 1 --2- 172.21.15.84:0/1211728901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec201007b0 0x7fec20069590 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.710+0000 7fec24d9e640 1 -- 172.21.15.84:0/1211728901 >> 172.21.15.84:0/1211728901 conn(0x7fec200fbd20 msgr2=0x7fec20104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:39.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.710+0000 7fec24d9e640 1 -- 172.21.15.84:0/1211728901 shutdown_connections 2024-05-22T23:01:39.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.710+0000 7fec24d9e640 1 -- 172.21.15.84:0/1211728901 wait complete. 2024-05-22T23:01:39.722 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:39 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1026366977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:01:39.722 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:39 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3362803409' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:01:39.723 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:39 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/81006688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:01:39.725 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T22:55:07.719394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T22:59:50.150511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T22:59:51.111182+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T23:01:39.725 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T23:01:39.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.828+0000 7f67defb7640 1 -- 172.21.15.84:0/1361479612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67d81019a0 msgr2=0x7f67d8101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:39.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.828+0000 7f67defb7640 1 --2- 172.21.15.84:0/1361479612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67d81019a0 0x7f67d8101e20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009900 tx=0x7f67cc02a410 comp rx=0 tx=0).stop 2024-05-22T23:01:39.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.828+0000 7f67defb7640 1 -- 172.21.15.84:0/1361479612 shutdown_connections 2024-05-22T23:01:39.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.828+0000 7f67defb7640 1 --2- 172.21.15.84:0/1361479612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67d8102360 0x7f67d8108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.828+0000 7f67defb7640 1 --2- 172.21.15.84:0/1361479612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67d81019a0 0x7f67d8101e20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.828+0000 7f67defb7640 1 --2- 172.21.15.84:0/1361479612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d81007b0 0x7f67d8100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.828+0000 7f67defb7640 1 -- 172.21.15.84:0/1361479612 >> 172.21.15.84:0/1361479612 conn(0x7f67d80fbd60 msgr2=0x7f67d80fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:39.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.828+0000 7f67defb7640 1 -- 172.21.15.84:0/1361479612 shutdown_connections 2024-05-22T23:01:39.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.828+0000 7f67defb7640 1 -- 172.21.15.84:0/1361479612 wait complete. 2024-05-22T23:01:39.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.829+0000 7f67defb7640 1 Processor -- start 2024-05-22T23:01:39.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.829+0000 7f67defb7640 1 -- start start 2024-05-22T23:01:39.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.829+0000 7f67defb7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67d81007b0 0x7f67d8104860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:39.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.829+0000 7f67defb7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67d81019a0 0x7f67d8102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:39.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.829+0000 7f67defb7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d8102360 0x7f67d81033e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:39.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.829+0000 7f67defb7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f67d810c430 con 0x7f67d81019a0 2024-05-22T23:01:39.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.829+0000 7f67defb7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f67d810c2b0 con 0x7f67d8102360 2024-05-22T23:01:39.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.829+0000 7f67defb7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f67d810c5b0 con 0x7f67d81007b0 2024-05-22T23:01:39.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.829+0000 7f67d7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67d81019a0 0x7f67d8102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:39.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.829+0000 7f67d7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67d81019a0 0x7f67d8102ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33810/0 (socket says 172.21.15.84:33810) 2024-05-22T23:01:39.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.829+0000 7f67d7fff640 1 -- 172.21.15.84:0/807492899 learned_addr learned my addr 172.21.15.84:0/807492899 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:39.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.829+0000 7f67dcd53640 1 --2- 172.21.15.84:0/807492899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67d81007b0 0x7f67d8104860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:39.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.829+0000 7f67d7fff640 1 -- 172.21.15.84:0/807492899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67d81007b0 msgr2=0x7f67d8104860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:39.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.829+0000 7f67d7fff640 1 --2- 172.21.15.84:0/807492899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67d81007b0 0x7f67d8104860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.829+0000 7f67d7fff640 1 -- 172.21.15.84:0/807492899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d8102360 msgr2=0x7f67d81033e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:39.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.829+0000 7f67d7fff640 1 --2- 172.21.15.84:0/807492899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d8102360 0x7f67d81033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:39.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.829+0000 7f67d7fff640 1 -- 172.21.15.84:0/807492899 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d8103c70 con 0x7f67d81019a0 2024-05-22T23:01:39.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.829+0000 7f67d7fff640 1 --2- 172.21.15.84:0/807492899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67d81019a0 0x7f67d8102ea0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009230 tx=0x7f67cc004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:39.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.829+0000 7f67d5ffb640 1 -- 172.21.15.84:0/807492899 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67cc03f030 con 0x7f67d81019a0 2024-05-22T23:01:39.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.830+0000 7f67d5ffb640 1 -- 172.21.15.84:0/807492899 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67cc003ce0 con 0x7f67d81019a0 2024-05-22T23:01:39.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.830+0000 7f67defb7640 1 -- 172.21.15.84:0/807492899 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d81a1cb0 con 0x7f67d81019a0 2024-05-22T23:01:39.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.830+0000 7f67d5ffb640 1 -- 172.21.15.84:0/807492899 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67cc035070 con 0x7f67d81019a0 2024-05-22T23:01:39.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.830+0000 7f67defb7640 1 -- 172.21.15.84:0/807492899 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d81a21b0 con 0x7f67d81019a0 2024-05-22T23:01:39.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.831+0000 7f67defb7640 1 -- 172.21.15.84:0/807492899 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67d81a2380 con 0x7f67d81019a0 2024-05-22T23:01:39.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.832+0000 7f67d5ffb640 1 -- 172.21.15.84:0/807492899 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f67cc042030 con 0x7f67d81019a0 2024-05-22T23:01:39.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.833+0000 7f67d5ffb640 1 --2- 172.21.15.84:0/807492899 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f67c00a2b90 0x7f67c00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:39.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.833+0000 7f67dcd53640 1 --2- 172.21.15.84:0/807492899 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f67c00a2b90 0x7f67c00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:39.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.833+0000 7f67d5ffb640 1 -- 172.21.15.84:0/807492899 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f67cc0efc90 con 0x7f67d81019a0 2024-05-22T23:01:39.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.833+0000 7f67dcd53640 1 --2- 172.21.15.84:0/807492899 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f67c00a2b90 0x7f67c00a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f67d000a300 tx=0x7f67d000a290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:39.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:39.836+0000 7f67d5ffb640 1 -- 172.21.15.84:0/807492899 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f67cc0bcbd0 con 0x7f67d81019a0 2024-05-22T23:01:39.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:39 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1026366977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:01:39.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:39 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3362803409' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:01:39.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:39 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/81006688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:01:40.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.013+0000 7f67defb7640 1 -- 172.21.15.84:0/807492899 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f67d81a2550 con 0x7f67d81019a0 2024-05-22T23:01:40.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.013+0000 7f67d5ffb640 1 -- 172.21.15.84:0/807492899 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f67cc0bc4d0 con 0x7f67d81019a0 2024-05-22T23:01:40.016 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2024-05-22T23:01:40.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.015+0000 7f67defb7640 1 -- 172.21.15.84:0/807492899 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f67c00a2b90 msgr2=0x7f67c00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:40.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.015+0000 7f67defb7640 1 --2- 172.21.15.84:0/807492899 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f67c00a2b90 0x7f67c00a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f67d000a300 tx=0x7f67d000a290 comp rx=0 tx=0).stop 2024-05-22T23:01:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.015+0000 7f67defb7640 1 -- 172.21.15.84:0/807492899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67d81019a0 msgr2=0x7f67d8102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.015+0000 7f67defb7640 1 --2- 172.21.15.84:0/807492899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67d81019a0 0x7f67d8102ea0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009230 tx=0x7f67cc004430 comp rx=0 tx=0).stop 2024-05-22T23:01:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.015+0000 7f67defb7640 1 -- 172.21.15.84:0/807492899 shutdown_connections 2024-05-22T23:01:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.015+0000 7f67defb7640 1 --2- 172.21.15.84:0/807492899 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f67c00a2b90 0x7f67c00a5050 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.015+0000 7f67defb7640 1 --2- 172.21.15.84:0/807492899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d8102360 0x7f67d81033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.015+0000 7f67defb7640 1 --2- 172.21.15.84:0/807492899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67d81019a0 0x7f67d8102ea0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.015+0000 7f67defb7640 1 --2- 172.21.15.84:0/807492899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67d81007b0 0x7f67d8104860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.015+0000 7f67defb7640 1 -- 172.21.15.84:0/807492899 >> 172.21.15.84:0/807492899 conn(0x7f67d80fbd60 msgr2=0x7f67d8069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.015+0000 7f67defb7640 1 -- 172.21.15.84:0/807492899 shutdown_connections 2024-05-22T23:01:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.016+0000 7f67defb7640 1 -- 172.21.15.84:0/807492899 wait complete. 2024-05-22T23:01:40.029 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T23:01:40.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:39 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1026366977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:01:40.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:39 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3362803409' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:01:40.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:39 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/81006688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:01:40.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.131+0000 7f9b682a5640 1 -- 172.21.15.84:0/1875141881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b601001c0 msgr2=0x7f9b601005e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:40.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.131+0000 7f9b682a5640 1 --2- 172.21.15.84:0/1875141881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b601001c0 0x7f9b601005e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009900 tx=0x7f9b5c02a480 comp rx=0 tx=0).stop 2024-05-22T23:01:40.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.131+0000 7f9b682a5640 1 -- 172.21.15.84:0/1875141881 shutdown_connections 2024-05-22T23:01:40.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.131+0000 7f9b682a5640 1 --2- 172.21.15.84:0/1875141881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b601014c0 0x7f9b60101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.131+0000 7f9b682a5640 1 --2- 172.21.15.84:0/1875141881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b60100b20 0x7f9b60100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.131+0000 7f9b682a5640 1 --2- 172.21.15.84:0/1875141881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b601001c0 0x7f9b601005e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.131+0000 7f9b682a5640 1 -- 172.21.15.84:0/1875141881 >> 172.21.15.84:0/1875141881 conn(0x7f9b600fbd40 msgr2=0x7f9b600fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:40.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.131+0000 7f9b682a5640 1 -- 172.21.15.84:0/1875141881 shutdown_connections 2024-05-22T23:01:40.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.131+0000 7f9b682a5640 1 -- 172.21.15.84:0/1875141881 wait complete. 2024-05-22T23:01:40.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.132+0000 7f9b682a5640 1 Processor -- start 2024-05-22T23:01:40.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.132+0000 7f9b682a5640 1 -- start start 2024-05-22T23:01:40.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.132+0000 7f9b682a5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b601001c0 0x7f9b60199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:40.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.132+0000 7f9b682a5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b60100b20 0x7f9b6019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:40.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.132+0000 7f9b682a5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b601014c0 0x7f9b601a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:40.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.132+0000 7f9b682a5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6010f820 con 0x7f9b60100b20 2024-05-22T23:01:40.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.132+0000 7f9b682a5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6010f6a0 con 0x7f9b601001c0 2024-05-22T23:01:40.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.132+0000 7f9b682a5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6010f9a0 con 0x7f9b601014c0 2024-05-22T23:01:40.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.132+0000 7f9b65840640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b60100b20 0x7f9b6019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:40.137 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.132+0000 7f9b65840640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b60100b20 0x7f9b6019a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33828/0 (socket says 172.21.15.84:33828) 2024-05-22T23:01:40.137 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.132+0000 7f9b65840640 1 -- 172.21.15.84:0/3432435721 learned_addr learned my addr 172.21.15.84:0/3432435721 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:40.137 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.132+0000 7f9b65840640 1 -- 172.21.15.84:0/3432435721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b601014c0 msgr2=0x7f9b601a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:40.137 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.132+0000 7f9b65840640 1 --2- 172.21.15.84:0/3432435721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b601014c0 0x7f9b601a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.137 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.132+0000 7f9b65840640 1 -- 172.21.15.84:0/3432435721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b601001c0 msgr2=0x7f9b60199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:40.137 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.132+0000 7f9b66041640 1 --2- 172.21.15.84:0/3432435721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b601001c0 0x7f9b60199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:40.137 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.132+0000 7f9b65840640 1 --2- 172.21.15.84:0/3432435721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b601001c0 0x7f9b60199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.137 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.132+0000 7f9b65840640 1 -- 172.21.15.84:0/3432435721 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b601a1b70 con 0x7f9b60100b20 2024-05-22T23:01:40.137 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.132+0000 7f9b65840640 1 --2- 172.21.15.84:0/3432435721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b60100b20 0x7f9b6019a490 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000b4a0 tx=0x7f9b5000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:40.137 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.132+0000 7f9b577fe640 1 -- 172.21.15.84:0/3432435721 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b50014070 con 0x7f9b60100b20 2024-05-22T23:01:40.137 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.132+0000 7f9b577fe640 1 -- 172.21.15.84:0/3432435721 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b50010af0 con 0x7f9b60100b20 2024-05-22T23:01:40.137 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.132+0000 7f9b682a5640 1 -- 172.21.15.84:0/3432435721 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b601a1e60 con 0x7f9b60100b20 2024-05-22T23:01:40.137 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.132+0000 7f9b577fe640 1 -- 172.21.15.84:0/3432435721 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b50017d10 con 0x7f9b60100b20 2024-05-22T23:01:40.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.133+0000 7f9b682a5640 1 -- 172.21.15.84:0/3432435721 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b601a23a0 con 0x7f9b60100b20 2024-05-22T23:01:40.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.133+0000 7f9b682a5640 1 -- 172.21.15.84:0/3432435721 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b6010b1f0 con 0x7f9b60100b20 2024-05-22T23:01:40.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.134+0000 7f9b577fe640 1 -- 172.21.15.84:0/3432435721 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9b50017070 con 0x7f9b60100b20 2024-05-22T23:01:40.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.135+0000 7f9b577fe640 1 --2- 172.21.15.84:0/3432435721 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9b4c0a2b90 0x7f9b4c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:40.139 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.135+0000 7f9b66041640 1 --2- 172.21.15.84:0/3432435721 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9b4c0a2b90 0x7f9b4c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:40.139 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.135+0000 7f9b577fe640 1 -- 172.21.15.84:0/3432435721 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f9b500ce480 con 0x7f9b60100b20 2024-05-22T23:01:40.139 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.136+0000 7f9b66041640 1 --2- 172.21.15.84:0/3432435721 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9b4c0a2b90 0x7f9b4c0a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c031610 tx=0x7f9b5c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:40.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.139+0000 7f9b577fe640 1 -- 172.21.15.84:0/3432435721 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9b5009b3c0 con 0x7f9b60100b20 2024-05-22T23:01:40.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.316+0000 7f9b682a5640 1 -- 172.21.15.84:0/3432435721 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9b60062510 con 0x7f9b60100b20 2024-05-22T23:01:40.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.316+0000 7f9b577fe640 1 -- 172.21.15.84:0/3432435721 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f9b5009acc0 con 0x7f9b60100b20 2024-05-22T23:01:40.318 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2024-05-22T23:01:40.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.318+0000 7f9b682a5640 1 -- 172.21.15.84:0/3432435721 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9b4c0a2b90 msgr2=0x7f9b4c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:40.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.318+0000 7f9b682a5640 1 --2- 172.21.15.84:0/3432435721 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9b4c0a2b90 0x7f9b4c0a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c031610 tx=0x7f9b5c000bc0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.318+0000 7f9b682a5640 1 -- 172.21.15.84:0/3432435721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b60100b20 msgr2=0x7f9b6019a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:40.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.318+0000 7f9b682a5640 1 --2- 172.21.15.84:0/3432435721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b60100b20 0x7f9b6019a490 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000b4a0 tx=0x7f9b5000b960 comp rx=0 tx=0).stop 2024-05-22T23:01:40.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.318+0000 7f9b682a5640 1 -- 172.21.15.84:0/3432435721 shutdown_connections 2024-05-22T23:01:40.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.318+0000 7f9b682a5640 1 --2- 172.21.15.84:0/3432435721 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f9b4c0a2b90 0x7f9b4c0a5050 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.318+0000 7f9b682a5640 1 --2- 172.21.15.84:0/3432435721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b601014c0 0x7f9b601a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.318+0000 7f9b682a5640 1 --2- 172.21.15.84:0/3432435721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b60100b20 0x7f9b6019a490 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.318+0000 7f9b682a5640 1 --2- 172.21.15.84:0/3432435721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b601001c0 0x7f9b60199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.318+0000 7f9b682a5640 1 -- 172.21.15.84:0/3432435721 >> 172.21.15.84:0/3432435721 conn(0x7f9b600fbd40 msgr2=0x7f9b600fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:40.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.318+0000 7f9b682a5640 1 -- 172.21.15.84:0/3432435721 shutdown_connections 2024-05-22T23:01:40.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.318+0000 7f9b682a5640 1 -- 172.21.15.84:0/3432435721 wait complete. 2024-05-22T23:01:40.331 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T23:01:40.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.432+0000 7f1da5693640 1 -- 172.21.15.84:0/3094235354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1da01019c0 msgr2=0x7f1da0101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:40.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.432+0000 7f1da5693640 1 --2- 172.21.15.84:0/3094235354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1da01019c0 0x7f1da0101e40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009900 tx=0x7f1d9402a410 comp rx=0 tx=0).stop 2024-05-22T23:01:40.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.432+0000 7f1da5693640 1 -- 172.21.15.84:0/3094235354 shutdown_connections 2024-05-22T23:01:40.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.432+0000 7f1da5693640 1 --2- 172.21.15.84:0/3094235354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1da0102380 0x7f1da0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.432+0000 7f1da5693640 1 --2- 172.21.15.84:0/3094235354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1da01019c0 0x7f1da0101e40 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.432+0000 7f1da5693640 1 --2- 172.21.15.84:0/3094235354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1da01007d0 0x7f1da0100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.432+0000 7f1da5693640 1 -- 172.21.15.84:0/3094235354 >> 172.21.15.84:0/3094235354 conn(0x7f1da00fbd60 msgr2=0x7f1da00fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:40.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.432+0000 7f1da5693640 1 -- 172.21.15.84:0/3094235354 shutdown_connections 2024-05-22T23:01:40.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.432+0000 7f1da5693640 1 -- 172.21.15.84:0/3094235354 wait complete. 2024-05-22T23:01:40.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.432+0000 7f1da5693640 1 Processor -- start 2024-05-22T23:01:40.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.433+0000 7f1da5693640 1 -- start start 2024-05-22T23:01:40.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.433+0000 7f1da5693640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1da01007d0 0x7f1da0199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:40.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.433+0000 7f1da5693640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1da01019c0 0x7f1da019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:40.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.433+0000 7f1da5693640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1da0102380 0x7f1da01a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:40.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.433+0000 7f1da5693640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1da010d570 con 0x7f1da0102380 2024-05-22T23:01:40.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.433+0000 7f1da5693640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1da010d3f0 con 0x7f1da01007d0 2024-05-22T23:01:40.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.433+0000 7f1da5693640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1da010d6f0 con 0x7f1da01019c0 2024-05-22T23:01:40.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.433+0000 7f1d9f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1da0102380 0x7f1da01a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:40.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.433+0000 7f1d9f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1da0102380 0x7f1da01a13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33850/0 (socket says 172.21.15.84:33850) 2024-05-22T23:01:40.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.433+0000 7f1d9f7fe640 1 -- 172.21.15.84:0/3353489246 learned_addr learned my addr 172.21.15.84:0/3353489246 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:40.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.433+0000 7f1d9effd640 1 --2- 172.21.15.84:0/3353489246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1da01007d0 0x7f1da0199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:40.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.433+0000 7f1d9f7fe640 1 -- 172.21.15.84:0/3353489246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1da01019c0 msgr2=0x7f1da019a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:40.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.433+0000 7f1d9f7fe640 1 --2- 172.21.15.84:0/3353489246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1da01019c0 0x7f1da019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.433+0000 7f1d9f7fe640 1 -- 172.21.15.84:0/3353489246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1da01007d0 msgr2=0x7f1da0199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:40.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.433+0000 7f1d9f7fe640 1 --2- 172.21.15.84:0/3353489246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1da01007d0 0x7f1da0199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.433+0000 7f1d9f7fe640 1 -- 172.21.15.84:0/3353489246 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da01a1a60 con 0x7f1da0102380 2024-05-22T23:01:40.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.433+0000 7f1d9f7fe640 1 --2- 172.21.15.84:0/3353489246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1da0102380 0x7f1da01a13f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800cad0 tx=0x7f1d9800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:40.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.434+0000 7f1d87fff640 1 -- 172.21.15.84:0/3353489246 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9800ae20 con 0x7f1da0102380 2024-05-22T23:01:40.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.434+0000 7f1d87fff640 1 -- 172.21.15.84:0/3353489246 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d98005020 con 0x7f1da0102380 2024-05-22T23:01:40.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.434+0000 7f1da5693640 1 -- 172.21.15.84:0/3353489246 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da01a1d50 con 0x7f1da0102380 2024-05-22T23:01:40.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.434+0000 7f1d87fff640 1 -- 172.21.15.84:0/3353489246 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9800f6f0 con 0x7f1da0102380 2024-05-22T23:01:40.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.434+0000 7f1da5693640 1 -- 172.21.15.84:0/3353489246 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da01a2250 con 0x7f1da0102380 2024-05-22T23:01:40.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.435+0000 7f1da5693640 1 -- 172.21.15.84:0/3353489246 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da01a24e0 con 0x7f1da0102380 2024-05-22T23:01:40.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.436+0000 7f1d87fff640 1 -- 172.21.15.84:0/3353489246 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1d980051d0 con 0x7f1da0102380 2024-05-22T23:01:40.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.436+0000 7f1d87fff640 1 --2- 172.21.15.84:0/3353489246 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1d880a2b90 0x7f1d880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:40.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.436+0000 7f1d9effd640 1 --2- 172.21.15.84:0/3353489246 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1d880a2b90 0x7f1d880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:40.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.437+0000 7f1d87fff640 1 -- 172.21.15.84:0/3353489246 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f1d980cf510 con 0x7f1da0102380 2024-05-22T23:01:40.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.437+0000 7f1d9effd640 1 --2- 172.21.15.84:0/3353489246 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1d880a2b90 0x7f1d880a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1d90005fd0 tx=0x7f1d90005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:40.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.440+0000 7f1d87fff640 1 -- 172.21.15.84:0/3353489246 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1d98014030 con 0x7f1da0102380 2024-05-22T23:01:40.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.620+0000 7f1da5693640 1 -- 172.21.15.84:0/3353489246 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1da01a26b0 con 0x7f1da0102380 2024-05-22T23:01:40.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.621+0000 7f1d87fff640 1 -- 172.21.15.84:0/3353489246 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f1d9809c450 con 0x7f1da0102380 2024-05-22T23:01:40.623 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2024-05-22T23:01:40.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.623+0000 7f1da5693640 1 -- 172.21.15.84:0/3353489246 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1d880a2b90 msgr2=0x7f1d880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:40.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.623+0000 7f1da5693640 1 --2- 172.21.15.84:0/3353489246 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1d880a2b90 0x7f1d880a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1d90005fd0 tx=0x7f1d90005850 comp rx=0 tx=0).stop 2024-05-22T23:01:40.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.623+0000 7f1da5693640 1 -- 172.21.15.84:0/3353489246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1da0102380 msgr2=0x7f1da01a13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:40.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.623+0000 7f1da5693640 1 --2- 172.21.15.84:0/3353489246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1da0102380 0x7f1da01a13f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800cad0 tx=0x7f1d9800cf90 comp rx=0 tx=0).stop 2024-05-22T23:01:40.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.623+0000 7f1da5693640 1 -- 172.21.15.84:0/3353489246 shutdown_connections 2024-05-22T23:01:40.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.623+0000 7f1da5693640 1 --2- 172.21.15.84:0/3353489246 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1d880a2b90 0x7f1d880a5050 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.623+0000 7f1da5693640 1 --2- 172.21.15.84:0/3353489246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1da0102380 0x7f1da01a13f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.623+0000 7f1da5693640 1 --2- 172.21.15.84:0/3353489246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1da01019c0 0x7f1da019a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.623+0000 7f1da5693640 1 --2- 172.21.15.84:0/3353489246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1da01007d0 0x7f1da0199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.623+0000 7f1da5693640 1 -- 172.21.15.84:0/3353489246 >> 172.21.15.84:0/3353489246 conn(0x7f1da00fbd60 msgr2=0x7f1da01095d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:40.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.623+0000 7f1da5693640 1 -- 172.21.15.84:0/3353489246 shutdown_connections 2024-05-22T23:01:40.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.623+0000 7f1da5693640 1 -- 172.21.15.84:0/3353489246 wait complete. 2024-05-22T23:01:40.636 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:01:40.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.735+0000 7fae94f10640 1 -- 172.21.15.84:0/3236488614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae901007d0 msgr2=0x7fae90100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:40.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.735+0000 7fae94f10640 1 --2- 172.21.15.84:0/3236488614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae901007d0 0x7fae90100bf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fae80009900 tx=0x7fae8002a410 comp rx=0 tx=0).stop 2024-05-22T23:01:40.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.735+0000 7fae94f10640 1 -- 172.21.15.84:0/3236488614 shutdown_connections 2024-05-22T23:01:40.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.735+0000 7fae94f10640 1 --2- 172.21.15.84:0/3236488614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae90102380 0x7fae90108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.735+0000 7fae94f10640 1 --2- 172.21.15.84:0/3236488614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae901019c0 0x7fae90101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.735+0000 7fae94f10640 1 --2- 172.21.15.84:0/3236488614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae901007d0 0x7fae90100bf0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.735+0000 7fae94f10640 1 -- 172.21.15.84:0/3236488614 >> 172.21.15.84:0/3236488614 conn(0x7fae900fbd60 msgr2=0x7fae900fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:40.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.735+0000 7fae94f10640 1 -- 172.21.15.84:0/3236488614 shutdown_connections 2024-05-22T23:01:40.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.735+0000 7fae94f10640 1 -- 172.21.15.84:0/3236488614 wait complete. 2024-05-22T23:01:40.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.736+0000 7fae94f10640 1 Processor -- start 2024-05-22T23:01:40.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.736+0000 7fae94f10640 1 -- start start 2024-05-22T23:01:40.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.736+0000 7fae94f10640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae901007d0 0x7fae90104900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:40.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.736+0000 7fae94f10640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae901019c0 0x7fae90102f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:40.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.736+0000 7fae94f10640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae90102380 0x7fae90103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:40.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.736+0000 7fae94f10640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fae9010c520 con 0x7fae901019c0 2024-05-22T23:01:40.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.736+0000 7fae94f10640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fae9010c3a0 con 0x7fae90102380 2024-05-22T23:01:40.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.736+0000 7fae94f10640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fae9010c6a0 con 0x7fae901007d0 2024-05-22T23:01:40.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.736+0000 7fae87fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae901019c0 0x7fae90102f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:40.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.736+0000 7fae87fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae901019c0 0x7fae90102f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33866/0 (socket says 172.21.15.84:33866) 2024-05-22T23:01:40.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.736+0000 7fae87fff640 1 -- 172.21.15.84:0/3640436965 learned_addr learned my addr 172.21.15.84:0/3640436965 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:40.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.737+0000 7fae8ed9d640 1 --2- 172.21.15.84:0/3640436965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae90102380 0x7fae90103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:40.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.737+0000 7fae8e59c640 1 --2- 172.21.15.84:0/3640436965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae901007d0 0x7fae90104900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:40.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.737+0000 7fae87fff640 1 -- 172.21.15.84:0/3640436965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae901007d0 msgr2=0x7fae90104900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:40.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.737+0000 7fae87fff640 1 --2- 172.21.15.84:0/3640436965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae901007d0 0x7fae90104900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.737+0000 7fae87fff640 1 -- 172.21.15.84:0/3640436965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae90102380 msgr2=0x7fae90103480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:40.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.737+0000 7fae87fff640 1 --2- 172.21.15.84:0/3640436965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae90102380 0x7fae90103480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.737+0000 7fae87fff640 1 -- 172.21.15.84:0/3640436965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae90103d10 con 0x7fae901019c0 2024-05-22T23:01:40.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.737+0000 7fae8ed9d640 1 --2- 172.21.15.84:0/3640436965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae90102380 0x7fae90103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:01:40.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.737+0000 7fae87fff640 1 --2- 172.21.15.84:0/3640436965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae901019c0 0x7fae90102f40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fae7800b4a0 tx=0x7fae7800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:40.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.737+0000 7fae877fe640 1 -- 172.21.15.84:0/3640436965 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78014070 con 0x7fae901019c0 2024-05-22T23:01:40.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.737+0000 7fae94f10640 1 -- 172.21.15.84:0/3640436965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae901a1e70 con 0x7fae901019c0 2024-05-22T23:01:40.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.737+0000 7fae877fe640 1 -- 172.21.15.84:0/3640436965 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae78010af0 con 0x7fae901019c0 2024-05-22T23:01:40.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.737+0000 7fae877fe640 1 -- 172.21.15.84:0/3640436965 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78017d10 con 0x7fae901019c0 2024-05-22T23:01:40.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.737+0000 7fae94f10640 1 -- 172.21.15.84:0/3640436965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae901a2350 con 0x7fae901019c0 2024-05-22T23:01:40.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.738+0000 7fae94f10640 1 -- 172.21.15.84:0/3640436965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae901a2640 con 0x7fae901019c0 2024-05-22T23:01:40.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.739+0000 7fae877fe640 1 -- 172.21.15.84:0/3640436965 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fae78017070 con 0x7fae901019c0 2024-05-22T23:01:40.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.740+0000 7fae877fe640 1 --2- 172.21.15.84:0/3640436965 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fae7c0a2b90 0x7fae7c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:40.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.740+0000 7fae8e59c640 1 --2- 172.21.15.84:0/3640436965 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fae7c0a2b90 0x7fae7c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:40.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.740+0000 7fae877fe640 1 -- 172.21.15.84:0/3640436965 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fae780ce450 con 0x7fae901019c0 2024-05-22T23:01:40.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.740+0000 7fae8e59c640 1 --2- 172.21.15.84:0/3640436965 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fae7c0a2b90 0x7fae7c0a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fae80006010 tx=0x7fae80000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:40.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.744+0000 7fae877fe640 1 -- 172.21.15.84:0/3640436965 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fae7809b390 con 0x7fae901019c0 2024-05-22T23:01:40.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:40 smithi084 ceph-mon[32186]: pgmap v345: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:01:40.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:40 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1211728901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:40.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:40 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/807492899' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:01:40.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:40 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3432435721' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:01:40.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:40 smithi204 ceph-mon[36211]: pgmap v345: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:01:40.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:40 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1211728901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:40.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:40 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/807492899' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:01:40.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:40 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3432435721' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:01:40.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.958+0000 7fae94f10640 1 -- 172.21.15.84:0/3640436965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae901a2810 con 0x7fae901019c0 2024-05-22T23:01:40.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.959+0000 7fae877fe640 1 -- 172.21.15.84:0/3640436965 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7fae780b2e30 con 0x7fae901019c0 2024-05-22T23:01:40.961 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:40.961 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:01:40.962 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:01:40.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.961+0000 7fae94f10640 1 -- 172.21.15.84:0/3640436965 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fae7c0a2b90 msgr2=0x7fae7c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:40.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.961+0000 7fae94f10640 1 --2- 172.21.15.84:0/3640436965 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fae7c0a2b90 0x7fae7c0a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fae80006010 tx=0x7fae80000bc0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.961+0000 7fae94f10640 1 -- 172.21.15.84:0/3640436965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae901019c0 msgr2=0x7fae90102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:40.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.961+0000 7fae94f10640 1 --2- 172.21.15.84:0/3640436965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae901019c0 0x7fae90102f40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fae7800b4a0 tx=0x7fae7800b960 comp rx=0 tx=0).stop 2024-05-22T23:01:40.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.961+0000 7fae94f10640 1 -- 172.21.15.84:0/3640436965 shutdown_connections 2024-05-22T23:01:40.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.961+0000 7fae94f10640 1 --2- 172.21.15.84:0/3640436965 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fae7c0a2b90 0x7fae7c0a5050 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.961+0000 7fae94f10640 1 --2- 172.21.15.84:0/3640436965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae90102380 0x7fae90103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.961+0000 7fae94f10640 1 --2- 172.21.15.84:0/3640436965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae901019c0 0x7fae90102f40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.961+0000 7fae94f10640 1 --2- 172.21.15.84:0/3640436965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae901007d0 0x7fae90104900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:40.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.961+0000 7fae94f10640 1 -- 172.21.15.84:0/3640436965 >> 172.21.15.84:0/3640436965 conn(0x7fae900fbd60 msgr2=0x7fae90069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:40.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.961+0000 7fae94f10640 1 -- 172.21.15.84:0/3640436965 shutdown_connections 2024-05-22T23:01:40.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:40.961+0000 7fae94f10640 1 -- 172.21.15.84:0/3640436965 wait complete. 2024-05-22T23:01:40.975 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T23:01:41.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:40 smithi096 ceph-mon[36392]: pgmap v345: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:01:41.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:40 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1211728901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:41.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:40 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/807492899' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:01:41.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:40 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3432435721' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:01:41.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.074+0000 7f3e1c529640 1 -- 172.21.15.84:0/1675815493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e141075b0 msgr2=0x7f3e14109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:41.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.074+0000 7f3e1c529640 1 --2- 172.21.15.84:0/1675815493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e141075b0 0x7f3e14109a00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009ca0 tx=0x7f3e0c02a8c0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.074+0000 7f3e1c529640 1 -- 172.21.15.84:0/1675815493 shutdown_connections 2024-05-22T23:01:41.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.074+0000 7f3e1c529640 1 --2- 172.21.15.84:0/1675815493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e141075b0 0x7f3e14109a00 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.074+0000 7f3e1c529640 1 --2- 172.21.15.84:0/1675815493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e14104c80 0x7f3e14107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.074+0000 7f3e1c529640 1 --2- 172.21.15.84:0/1675815493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e14102350 0x7f3e14104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.074+0000 7f3e1c529640 1 -- 172.21.15.84:0/1675815493 >> 172.21.15.84:0/1675815493 conn(0x7f3e140fbd60 msgr2=0x7f3e140fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:41.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.074+0000 7f3e1c529640 1 -- 172.21.15.84:0/1675815493 shutdown_connections 2024-05-22T23:01:41.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.074+0000 7f3e1c529640 1 -- 172.21.15.84:0/1675815493 wait complete. 2024-05-22T23:01:41.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.075+0000 7f3e1c529640 1 Processor -- start 2024-05-22T23:01:41.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.075+0000 7f3e1c529640 1 -- start start 2024-05-22T23:01:41.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.075+0000 7f3e1c529640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e14102350 0x7f3e14197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.075+0000 7f3e1c529640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e14104c80 0x7f3e14198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.075+0000 7f3e1c529640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e141075b0 0x7f3e1419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.075+0000 7f3e1c529640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1410d570 con 0x7f3e141075b0 2024-05-22T23:01:41.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.075+0000 7f3e1c529640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1410d3f0 con 0x7f3e14104c80 2024-05-22T23:01:41.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.075+0000 7f3e1c529640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1410d6f0 con 0x7f3e14102350 2024-05-22T23:01:41.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.075+0000 7f3e1aac6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e141075b0 0x7f3e1419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:41.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.075+0000 7f3e1aac6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e141075b0 0x7f3e1419f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33894/0 (socket says 172.21.15.84:33894) 2024-05-22T23:01:41.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.075+0000 7f3e1aac6640 1 -- 172.21.15.84:0/415782083 learned_addr learned my addr 172.21.15.84:0/415782083 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:41.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.075+0000 7f3e1a2c5640 1 --2- 172.21.15.84:0/415782083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e14102350 0x7f3e14197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:41.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.075+0000 7f3e19ac4640 1 --2- 172.21.15.84:0/415782083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e14104c80 0x7f3e14198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:41.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.075+0000 7f3e1aac6640 1 -- 172.21.15.84:0/415782083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e14102350 msgr2=0x7f3e14197c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:41.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.075+0000 7f3e1aac6640 1 --2- 172.21.15.84:0/415782083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e14102350 0x7f3e14197c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.075+0000 7f3e1aac6640 1 -- 172.21.15.84:0/415782083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e14104c80 msgr2=0x7f3e14198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:41.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.075+0000 7f3e1aac6640 1 --2- 172.21.15.84:0/415782083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e14104c80 0x7f3e14198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.075+0000 7f3e1aac6640 1 -- 172.21.15.84:0/415782083 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1419f7e0 con 0x7f3e141075b0 2024-05-22T23:01:41.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.076+0000 7f3e1aac6640 1 --2- 172.21.15.84:0/415782083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e141075b0 0x7f3e1419f170 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c031ae0 tx=0x7f3e0c031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:41.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.076+0000 7f3e0b7fe640 1 -- 172.21.15.84:0/415782083 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c003c30 con 0x7f3e141075b0 2024-05-22T23:01:41.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.076+0000 7f3e0b7fe640 1 -- 172.21.15.84:0/415782083 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e0c003dd0 con 0x7f3e141075b0 2024-05-22T23:01:41.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.076+0000 7f3e1c529640 1 -- 172.21.15.84:0/415782083 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e1419fa70 con 0x7f3e141075b0 2024-05-22T23:01:41.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.076+0000 7f3e0b7fe640 1 -- 172.21.15.84:0/415782083 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c02fcb0 con 0x7f3e141075b0 2024-05-22T23:01:41.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.076+0000 7f3e1c529640 1 -- 172.21.15.84:0/415782083 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e1419ffc0 con 0x7f3e141075b0 2024-05-22T23:01:41.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.077+0000 7f3e1c529640 1 -- 172.21.15.84:0/415782083 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e140fd850 con 0x7f3e141075b0 2024-05-22T23:01:41.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.078+0000 7f3e0b7fe640 1 -- 172.21.15.84:0/415782083 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3e0c004050 con 0x7f3e141075b0 2024-05-22T23:01:41.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.079+0000 7f3e0b7fe640 1 --2- 172.21.15.84:0/415782083 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f3e000a2b90 0x7f3e000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.079+0000 7f3e1a2c5640 1 --2- 172.21.15.84:0/415782083 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f3e000a2b90 0x7f3e000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:41.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.079+0000 7f3e0b7fe640 1 -- 172.21.15.84:0/415782083 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f3e0c0f0330 con 0x7f3e141075b0 2024-05-22T23:01:41.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.079+0000 7f3e1a2c5640 1 --2- 172.21.15.84:0/415782083 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f3e000a2b90 0x7f3e000a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009bd0 tx=0x7f3e10005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:41.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.083+0000 7f3e0b7fe640 1 -- 172.21.15.84:0/415782083 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3e0c0bd270 con 0x7f3e141075b0 2024-05-22T23:01:41.259 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.258+0000 7f3e1c529640 1 -- 172.21.15.84:0/415782083 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e14062470 con 0x7f3e141075b0 2024-05-22T23:01:41.261 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.259+0000 7f3e0b7fe640 1 -- 172.21.15.84:0/415782083 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22722 (secure 0 0 0) 0x7f3e0c0bcb70 con 0x7f3e141075b0 2024-05-22T23:01:41.261 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:41.261 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":88,"fsid":"c1135a12-188d-11ef-bc9a-c7b262605968","created":"2024-05-22T22:51:49.878218+0000","modified":"2024-05-22T23:00:46.572591+0000","last_up_change":"2024-05-22T22:58:58.550278+0000","last_in_change":"2024-05-22T22:58:37.322139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T22:55:07.719394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T22:59:50.150511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T22:59:51.111182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762df065-fb4f-4c71-aab8-a62d2282c253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6803","nonce":1559945826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6805","nonce":1559945826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6809","nonce":1559945826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6807","nonce":1559945826}]},"public_addr":"172.21.15.84:6803/1559945826","cluster_addr":"172.21.15.84:6805/1559945826","heartbeat_back_addr":"172.21.15.84:6809/1559945826","heartbeat_front_addr":"172.21.15.84:6807/1559945826","state":["exists","up"]},{"osd":1,"uuid":"3fb2385d-6a34-49e3-88e1-116237beb346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6801","nonce":2996505353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6803","nonce":2996505353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6807","nonce":2996505353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6805","nonce":2996505353}]},"public_addr":"172.21.15.96:6801/2996505353","cluster_addr":"172.21.15.96:6803/2996505353","heartbeat_back_addr":"172.21.15.96:6807/2996505353","heartbeat_front_addr":"172.21.15.96:6805/2996505353","state":["exists","up"]},{"osd":2,"uuid":"e306e88e-a308-4585-ae11-c8af10b84038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6801","nonce":1499577897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6803","nonce":1499577897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6807","nonce":1499577897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6805","nonce":1499577897}]},"public_addr":"172.21.15.204:6801/1499577897","cluster_addr":"172.21.15.204:6803/1499577897","heartbeat_back_addr":"172.21.15.204:6807/1499577897","heartbeat_front_addr":"172.21.15.204:6805/1499577897","state":["exists","up"]},{"osd":3,"uuid":"d2a5fb97-6f60-4d45-9f18-447cae480974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6811","nonce":2869340648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6813","nonce":2869340648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6817","nonce":2869340648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6815","nonce":2869340648}]},"public_addr":"172.21.15.84:6811/2869340648","cluster_addr":"172.21.15.84:6813/2869340648","heartbeat_back_addr":"172.21.15.84:6817/2869340648","heartbeat_front_addr":"172.21.15.84:6815/2869340648","state":["exists","up"]},{"osd":4,"uuid":"4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6809","nonce":2554265648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6811","nonce":2554265648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6815","nonce":2554265648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6813","nonce":2554265648}]},"public_addr":"172.21.15.96:6809/2554265648","cluster_addr":"172.21.15.96:6811/2554265648","heartbeat_back_addr":"172.21.15.96:6815/2554265648","heartbeat_front_addr":"172.21.15.96:6813/2554265648","state":["exists","up"]},{"osd":5,"uuid":"7996bf5e-93a5-41b0-a9b1-99e7ef4514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6809","nonce":4075476986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6811","nonce":4075476986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6815","nonce":4075476986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6813","nonce":4075476986}]},"public_addr":"172.21.15.204:6809/4075476986","cluster_addr":"172.21.15.204:6811/4075476986","heartbeat_back_addr":"172.21.15.204:6815/4075476986","heartbeat_front_addr":"172.21.15.204:6813/4075476986","state":["exists","up"]},{"osd":6,"uuid":"f4b7c5de-1836-4d16-8603-3c517524d9fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6819","nonce":1806202865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6821","nonce":1806202865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6825","nonce":1806202865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6823","nonce":1806202865}]},"public_addr":"172.21.15.84:6819/1806202865","cluster_addr":"172.21.15.84:6821/1806202865","heartbeat_back_addr":"172.21.15.84:6825/1806202865","heartbeat_front_addr":"172.21.15.84:6823/1806202865","state":["exists","up"]},{"osd":7,"uuid":"69a18c43-5447-480d-9466-190830070c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6817","nonce":3195792774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6819","nonce":3195792774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6823","nonce":3195792774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6821","nonce":3195792774}]},"public_addr":"172.21.15.96:6817/3195792774","cluster_addr":"172.21.15.96:6819/3195792774","heartbeat_back_addr":"172.21.15.96:6823/3195792774","heartbeat_front_addr":"172.21.15.96:6821/3195792774","state":["exists","up"]},{"osd":8,"uuid":"2c5cadca-689a-441e-847c-ba370d400ef1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6817","nonce":606997927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6819","nonce":606997927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6823","nonce":606997927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6821","nonce":606997927}]},"public_addr":"172.21.15.204:6817/606997927","cluster_addr":"172.21.15.204:6819/606997927","heartbeat_back_addr":"172.21.15.204:6823/606997927","heartbeat_front_addr":"172.21.15.204:6821/606997927","state":["exists","up"]},{"osd":9,"uuid":"e3984fb2-7c00-4275-a10e-bca7c859f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6827","nonce":2786189012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6829","nonce":2786189012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6833","nonce":2786189012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6831","nonce":2786189012}]},"public_addr":"172.21.15.84:6827/2786189012","cluster_addr":"172.21.15.84:6829/2786189012","heartbeat_back_addr":"172.21.15.84:6833/2786189012","heartbeat_front_addr":"172.21.15.84:6831/2786189012","state":["exists","up"]},{"osd":10,"uuid":"b4938160-a808-4ded-91c2-d245c99fff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6825","nonce":1985398526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6827","nonce":1985398526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6831","nonce":1985398526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6829","nonce":1985398526}]},"public_addr":"172.21.15.96:6825/1985398526","cluster_addr":"172.21.15.96:6827/1985398526","heartbeat_back_addr":"172.21.15.96:6831/1985398526","heartbeat_front_addr":"172.21.15.96:6829/1985398526","state":["exists","up"]},{"osd":11,"uuid":"238922ef-d046-40cc-833d-25b7e7a523cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6825","nonce":3106116519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6827","nonce":3106116519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6831","nonce":3106116519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6829","nonce":3106116519}]},"public_addr":"172.21.15.204:6825/3106116519","cluster_addr":"172.21.15.204:6827/3106116519","heartbeat_back_addr":"172.21.15.204:6831/3106116519","heartbeat_front_addr":"172.21.15.204:6829/3106116519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:16.443480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:39.898133+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:03.664544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:28.838687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:53.829733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:19.269307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:45.166910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:10.771614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:36.678516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:03.150592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:29.327769+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:56.315938+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6801/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:6800/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/2766288608":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/950455999":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/4000289608":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1650467363":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1052746114":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6800/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6801/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/14610813":"2024-05-23T22:52:45.395791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T23:01:41.263 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.261+0000 7f3e1c529640 1 -- 172.21.15.84:0/415782083 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f3e000a2b90 msgr2=0x7f3e000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:41.263 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.261+0000 7f3e1c529640 1 --2- 172.21.15.84:0/415782083 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f3e000a2b90 0x7f3e000a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009bd0 tx=0x7f3e10005f90 comp rx=0 tx=0).stop 2024-05-22T23:01:41.263 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.261+0000 7f3e1c529640 1 -- 172.21.15.84:0/415782083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e141075b0 msgr2=0x7f3e1419f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:41.263 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.261+0000 7f3e1c529640 1 --2- 172.21.15.84:0/415782083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e141075b0 0x7f3e1419f170 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c031ae0 tx=0x7f3e0c031ee0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.261+0000 7f3e1c529640 1 -- 172.21.15.84:0/415782083 shutdown_connections 2024-05-22T23:01:41.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.261+0000 7f3e1c529640 1 --2- 172.21.15.84:0/415782083 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f3e000a2b90 0x7f3e000a5050 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.261+0000 7f3e1c529640 1 --2- 172.21.15.84:0/415782083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e141075b0 0x7f3e1419f170 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.261+0000 7f3e1c529640 1 --2- 172.21.15.84:0/415782083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e14104c80 0x7f3e14198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.261+0000 7f3e1c529640 1 --2- 172.21.15.84:0/415782083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e14102350 0x7f3e14197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.261+0000 7f3e1c529640 1 -- 172.21.15.84:0/415782083 >> 172.21.15.84:0/415782083 conn(0x7f3e140fbd60 msgr2=0x7f3e14102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:41.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.261+0000 7f3e1c529640 1 -- 172.21.15.84:0/415782083 shutdown_connections 2024-05-22T23:01:41.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.261+0000 7f3e1c529640 1 -- 172.21.15.84:0/415782083 wait complete. 2024-05-22T23:01:41.275 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-22T22:59:51.926923+0000', 'modified': '2024-05-22T22:59:54.945583+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24530}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24494': {'gid': 24494, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6837/1191862158', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6836', 'nonce': 1191862158}, {'type': 'v1', 'addr': '172.21.15.96:6837', 'nonce': 1191862158}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24530': {'gid': 24530, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6839/3272422729', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6838', 'nonce': 3272422729}, {'type': 'v1', 'addr': '172.21.15.96:6839', 'nonce': 3272422729}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24530, 'qdb_cluster': [24530]} 2024-05-22T23:01:41.275 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-22T23:01:41.275 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-22T23:01:41.379 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.375+0000 7f1d5d2fc640 1 -- 172.21.15.84:0/998056780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d58104cc0 msgr2=0x7f1d581070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:41.379 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.375+0000 7f1d5d2fc640 1 --2- 172.21.15.84:0/998056780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d58104cc0 0x7f1d581070b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1d40009900 tx=0x7f1d4002a410 comp rx=0 tx=0).stop 2024-05-22T23:01:41.379 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.375+0000 7f1d5d2fc640 1 -- 172.21.15.84:0/998056780 shutdown_connections 2024-05-22T23:01:41.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.375+0000 7f1d5d2fc640 1 --2- 172.21.15.84:0/998056780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d581075f0 0x7f1d581099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.375+0000 7f1d5d2fc640 1 --2- 172.21.15.84:0/998056780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d58104cc0 0x7f1d581070b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.375+0000 7f1d5d2fc640 1 --2- 172.21.15.84:0/998056780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d58102390 0x7f1d58104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.375+0000 7f1d5d2fc640 1 -- 172.21.15.84:0/998056780 >> 172.21.15.84:0/998056780 conn(0x7f1d580fbd20 msgr2=0x7f1d580fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:41.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.375+0000 7f1d5d2fc640 1 -- 172.21.15.84:0/998056780 shutdown_connections 2024-05-22T23:01:41.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.375+0000 7f1d5d2fc640 1 -- 172.21.15.84:0/998056780 wait complete. 2024-05-22T23:01:41.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.376+0000 7f1d5d2fc640 1 Processor -- start 2024-05-22T23:01:41.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.376+0000 7f1d5d2fc640 1 -- start start 2024-05-22T23:01:41.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.376+0000 7f1d5d2fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d58102390 0x7f1d58197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.376+0000 7f1d5d2fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d58104cc0 0x7f1d581981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.376+0000 7f1d5d2fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d581075f0 0x7f1d5819f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.376+0000 7f1d5d2fc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5810d580 con 0x7f1d58104cc0 2024-05-22T23:01:41.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.376+0000 7f1d5d2fc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5810d400 con 0x7f1d581075f0 2024-05-22T23:01:41.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.376+0000 7f1d5d2fc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5810d700 con 0x7f1d58102390 2024-05-22T23:01:41.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.376+0000 7f1d4ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d58104cc0 0x7f1d581981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:41.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.376+0000 7f1d4ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d58104cc0 0x7f1d581981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33910/0 (socket says 172.21.15.84:33910) 2024-05-22T23:01:41.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.376+0000 7f1d4ffff640 1 -- 172.21.15.84:0/2716103330 learned_addr learned my addr 172.21.15.84:0/2716103330 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:41.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.376+0000 7f1d56ffd640 1 --2- 172.21.15.84:0/2716103330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d58102390 0x7f1d58197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:41.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.376+0000 7f1d577fe640 1 --2- 172.21.15.84:0/2716103330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d581075f0 0x7f1d5819f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:41.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.377+0000 7f1d4ffff640 1 -- 172.21.15.84:0/2716103330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d58102390 msgr2=0x7f1d58197cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:41.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.377+0000 7f1d4ffff640 1 --2- 172.21.15.84:0/2716103330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d58102390 0x7f1d58197cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.377+0000 7f1d4ffff640 1 -- 172.21.15.84:0/2716103330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d581075f0 msgr2=0x7f1d5819f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:41.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.377+0000 7f1d4ffff640 1 --2- 172.21.15.84:0/2716103330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d581075f0 0x7f1d5819f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.377+0000 7f1d4ffff640 1 -- 172.21.15.84:0/2716103330 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5819f8f0 con 0x7f1d58104cc0 2024-05-22T23:01:41.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.377+0000 7f1d56ffd640 1 --2- 172.21.15.84:0/2716103330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d58102390 0x7f1d58197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:01:41.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.377+0000 7f1d577fe640 1 --2- 172.21.15.84:0/2716103330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d581075f0 0x7f1d5819f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:01:41.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.377+0000 7f1d4ffff640 1 --2- 172.21.15.84:0/2716103330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d58104cc0 0x7f1d581981f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1d40031610 tx=0x7f1d400043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:41.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.377+0000 7f1d54ff9640 1 -- 172.21.15.84:0/2716103330 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4003f030 con 0x7f1d58104cc0 2024-05-22T23:01:41.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.377+0000 7f1d5d2fc640 1 -- 172.21.15.84:0/2716103330 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d5819fb80 con 0x7f1d58104cc0 2024-05-22T23:01:41.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.377+0000 7f1d54ff9640 1 -- 172.21.15.84:0/2716103330 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d40003ce0 con 0x7f1d58104cc0 2024-05-22T23:01:41.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.377+0000 7f1d54ff9640 1 -- 172.21.15.84:0/2716103330 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40035070 con 0x7f1d58104cc0 2024-05-22T23:01:41.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.377+0000 7f1d5d2fc640 1 -- 172.21.15.84:0/2716103330 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d581a0070 con 0x7f1d58104cc0 2024-05-22T23:01:41.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.379+0000 7f1d54ff9640 1 -- 172.21.15.84:0/2716103330 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1d40042030 con 0x7f1d58104cc0 2024-05-22T23:01:41.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.379+0000 7f1d6000a740 1 Processor -- start 2024-05-22T23:01:41.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.379+0000 7f1d6000a740 1 -- start start 2024-05-22T23:01:41.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.380+0000 7f1d6000a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558372709b0 0x555837270dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.380+0000 7f1d6000a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558372713a0 0x555837275ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.380+0000 7f1d6000a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555837276510 0x5558372789a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.380+0000 7f1d6000a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5558371d3680 con 0x5558372713a0 2024-05-22T23:01:41.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.380+0000 7f1d6000a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555837172250 con 0x555837276510 2024-05-22T23:01:41.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.380+0000 7f1d6000a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5558371cf640 con 0x5558372709b0 2024-05-22T23:01:41.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.380+0000 7f1d56ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558372713a0 0x555837275ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:41.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.380+0000 7f1d56ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558372713a0 0x555837275ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33920/0 (socket says 172.21.15.84:33920) 2024-05-22T23:01:41.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.380+0000 7f1d56ffd640 1 -- 172.21.15.84:0/2404995362 learned_addr learned my addr 172.21.15.84:0/2404995362 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:41.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.380+0000 7f1d577fe640 1 --2- 172.21.15.84:0/2404995362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558372709b0 0x555837270dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:41.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.380+0000 7f1d54ff9640 1 --2- 172.21.15.84:0/2716103330 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1d440a2b90 0x7f1d440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.380+0000 7f1d4ffff640 1 --2- 172.21.15.84:0/2404995362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555837276510 0x5558372789a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:41.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.380+0000 7f1d56ffd640 1 -- 172.21.15.84:0/2404995362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558372709b0 msgr2=0x555837270dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:41.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.380+0000 7f1d56ffd640 1 --2- 172.21.15.84:0/2404995362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558372709b0 0x555837270dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.380+0000 7f1d56ffd640 1 -- 172.21.15.84:0/2404995362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555837276510 msgr2=0x5558372789a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:41.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.380+0000 7f1d56ffd640 1 --2- 172.21.15.84:0/2404995362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555837276510 0x5558372789a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.380+0000 7f1d56ffd640 1 -- 172.21.15.84:0/2404995362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555836ee72d0 con 0x5558372713a0 2024-05-22T23:01:41.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.380+0000 7f1d4ffff640 1 --2- 172.21.15.84:0/2404995362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555837276510 0x5558372789a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:01:41.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.380+0000 7f1d577fe640 1 --2- 172.21.15.84:0/2716103330 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1d440a2b90 0x7f1d440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:41.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.380+0000 7f1d54ff9640 1 -- 172.21.15.84:0/2716103330 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f1d400ef820 con 0x7f1d58104cc0 2024-05-22T23:01:41.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.380+0000 7f1d56ffd640 1 --2- 172.21.15.84:0/2404995362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558372713a0 0x555837275ea0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800a9d0 tx=0x7f1d4800ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f221b54dbd891d0d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:41.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.381+0000 7f1d4d7fa640 1 -- 172.21.15.84:0/2404995362 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48015070 con 0x5558372713a0 2024-05-22T23:01:41.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.381+0000 7f1d4d7fa640 1 -- 172.21.15.84:0/2404995362 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d4800cc60 con 0x5558372713a0 2024-05-22T23:01:41.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.381+0000 7f1d4d7fa640 1 -- 172.21.15.84:0/2404995362 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48018650 con 0x5558372713a0 2024-05-22T23:01:41.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.381+0000 7f1d6000a740 1 -- 172.21.15.84:0/2404995362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558372713a0 msgr2=0x555837275ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:41.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.381+0000 7f1d6000a740 1 --2- 172.21.15.84:0/2404995362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558372713a0 0x555837275ea0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800a9d0 tx=0x7f1d4800ae90 comp rx=0 tx=0).stop 2024-05-22T23:01:41.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.381+0000 7f1d577fe640 1 --2- 172.21.15.84:0/2716103330 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1d440a2b90 0x7f1d440a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x555837271c00 tx=0x7f1d50004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:41.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.381+0000 7f1d6000a740 1 -- 172.21.15.84:0/2404995362 shutdown_connections 2024-05-22T23:01:41.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.381+0000 7f1d6000a740 1 --2- 172.21.15.84:0/2404995362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555837276510 0x5558372789a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.381+0000 7f1d6000a740 1 --2- 172.21.15.84:0/2404995362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558372713a0 0x555837275ea0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.381+0000 7f1d6000a740 1 --2- 172.21.15.84:0/2404995362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558372709b0 0x555837270dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.381+0000 7f1d6000a740 1 -- 172.21.15.84:0/2404995362 >> 172.21.15.84:0/2404995362 conn(0x55583726c020 msgr2=0x55583726c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:41.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.381+0000 7f1d6000a740 1 -- 172.21.15.84:0/2404995362 shutdown_connections 2024-05-22T23:01:41.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.382+0000 7f1d6000a740 1 -- 172.21.15.84:0/2404995362 wait complete. 2024-05-22T23:01:41.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.382+0000 7f1d6000a740 1 -- start start 2024-05-22T23:01:41.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.382+0000 7f1d6000a740 1 Processor -- start 2024-05-22T23:01:41.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.382+0000 7f1d6000a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558372709b0 0x55583727ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.382+0000 7f1d6000a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558372713a0 0x55583727b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.382+0000 7f1d6000a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555837276510 0x555837282380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.382+0000 7f1d6000a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5558372794c0 con 0x5558372713a0 2024-05-22T23:01:41.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.382+0000 7f1d6000a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55583715d3f0 con 0x5558372709b0 2024-05-22T23:01:41.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.382+0000 7f1d6000a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555837279640 con 0x555837276510 2024-05-22T23:01:41.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.382+0000 7f1d6000a740 10 client.0 _ll_register_callbacks cb 0x555837192fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:01:41.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.382+0000 7f1d4ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558372713a0 0x55583727b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:41.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.382+0000 7f1d4ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558372713a0 0x55583727b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33932/0 (socket says 172.21.15.84:33932) 2024-05-22T23:01:41.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.382+0000 7f1d4ffff640 1 -- 172.21.15.84:0/2393917294 learned_addr learned my addr 172.21.15.84:0/2393917294 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:41.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.382+0000 7f1d56ffd640 1 --2- 172.21.15.84:0/2393917294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558372709b0 0x55583727ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:41.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.382+0000 7f1d577fe640 1 --2- 172.21.15.84:0/2393917294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555837276510 0x555837282380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:41.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.382+0000 7f1d4ffff640 1 -- 172.21.15.84:0/2393917294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555837276510 msgr2=0x555837282380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:41.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.382+0000 7f1d4ffff640 1 --2- 172.21.15.84:0/2393917294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555837276510 0x555837282380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.382+0000 7f1d4ffff640 1 -- 172.21.15.84:0/2393917294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558372709b0 msgr2=0x55583727ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:41.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.382+0000 7f1d4ffff640 1 --2- 172.21.15.84:0/2393917294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558372709b0 0x55583727ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.382+0000 7f1d4ffff640 1 -- 172.21.15.84:0/2393917294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555836fb16b0 con 0x5558372713a0 2024-05-22T23:01:41.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.382+0000 7f1d577fe640 1 --2- 172.21.15.84:0/2393917294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555837276510 0x555837282380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:01:41.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.382+0000 7f1d56ffd640 1 --2- 172.21.15.84:0/2393917294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558372709b0 0x55583727ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:01:41.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.383+0000 7f1d4ffff640 1 --2- 172.21.15.84:0/2393917294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558372713a0 0x55583727b340 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1d400b89d0 tx=0x7f1d400bba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1638a1d9675b7c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:41.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.383+0000 7f1d3e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:01:41.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.383+0000 7f1d3e7fc640 1 -- 172.21.15.84:0/2393917294 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40049070 con 0x5558372713a0 2024-05-22T23:01:41.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.383+0000 7f1d3e7fc640 1 -- 172.21.15.84:0/2393917294 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f1d400b8670 con 0x5558372713a0 2024-05-22T23:01:41.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.383+0000 7f1d6000a740 1 -- 172.21.15.84:0/2393917294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d4800cc60 con 0x5558372713a0 2024-05-22T23:01:41.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.383+0000 7f1d3e7fc640 1 -- 172.21.15.84:0/2393917294 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f1d400c03b0 con 0x5558372713a0 2024-05-22T23:01:41.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.383+0000 7f1d6000a740 10 client.15156 fetch_fsmap learned FSMap version 18 2024-05-22T23:01:41.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.383+0000 7f1d6000a740 1 -- 172.21.15.84:0/2393917294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x555837285660 con 0x5558372713a0 2024-05-22T23:01:41.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.383+0000 7f1d3e7fc640 1 -- 172.21.15.84:0/2393917294 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d4800cc60 con 0x5558372713a0 2024-05-22T23:01:41.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.383+0000 7f1d3e7fc640 1 -- 172.21.15.84:0/2393917294 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f1d40031bd0 con 0x5558372713a0 2024-05-22T23:01:41.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.383+0000 7f1d6000a740 10 client.15156 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:01:41.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.383+0000 7f1d6000a740 10 client.15156 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-22T23:01:41.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.383+0000 7f1d6000a740 20 client.15156 populate_metadata read hostname 'smithi084' 2024-05-22T23:01:41.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.383+0000 7f1d6000a740 1 --2- 172.21.15.84:0/2393917294 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x555837287d70 0x5558372891e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.383+0000 7f1d6000a740 4 client.15156 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:01:41.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.383+0000 7f1d6000a740 1 -- 172.21.15.84:0/2393917294 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558371e0db0 con 0x555837287d70 2024-05-22T23:01:41.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.383+0000 7f1d56ffd640 1 --2- 172.21.15.84:0/2393917294 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x555837287d70 0x5558372891e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:41.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.384+0000 7f1d56ffd640 1 --2- 172.21.15.84:0/2393917294 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x555837287d70 0x5558372891e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0c08f0100db155 server_cookie=1d627e81653535ba in_seq=0 out_seq=0 2024-05-22T23:01:41.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.384+0000 7f1d3e7fc640 10 client.15156 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:01:41.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.384+0000 7f1d3e7fc640 1 -- 172.21.15.84:0/2393917294 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5558371e0db0 con 0x555837287d70 2024-05-22T23:01:41.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.384+0000 7f1d3e7fc640 10 client.15156 handle_command_reply: tid=0 2024-05-22T23:01:41.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.384+0000 7f1d6000a740 1 client.15156 shutdown 2024-05-22T23:01:41.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.385+0000 7f1d6000a740 1 -- 172.21.15.84:0/2393917294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558372713a0 msgr2=0x55583727b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:41.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.385+0000 7f1d6000a740 1 --2- 172.21.15.84:0/2393917294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558372713a0 0x55583727b340 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1d400b89d0 tx=0x7f1d400bba20 comp rx=0 tx=0).stop 2024-05-22T23:01:41.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.385+0000 7f1d6000a740 1 -- 172.21.15.84:0/2393917294 shutdown_connections 2024-05-22T23:01:41.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.385+0000 7f1d6000a740 1 --2- 172.21.15.84:0/2393917294 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x555837287d70 0x5558372891e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.385+0000 7f1d6000a740 1 --2- 172.21.15.84:0/2393917294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555837276510 0x555837282380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.385+0000 7f1d6000a740 1 --2- 172.21.15.84:0/2393917294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558372713a0 0x55583727b340 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.385+0000 7f1d3e7fc640 0 client.15156 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:01:41.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.385+0000 7f1d6000a740 1 --2- 172.21.15.84:0/2393917294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558372709b0 0x55583727ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.385+0000 7f1d6000a740 1 -- 172.21.15.84:0/2393917294 >> 172.21.15.84:0/2393917294 conn(0x55583726c020 msgr2=0x55583726b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:41.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.385+0000 7f1d6000a740 1 -- 172.21.15.84:0/2393917294 shutdown_connections 2024-05-22T23:01:41.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.385+0000 7f1d6000a740 1 -- 172.21.15.84:0/2393917294 wait complete. 2024-05-22T23:01:41.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.385+0000 7f1d6000a740 20 client.15156 trim_cache size 0 max 16384 2024-05-22T23:01:41.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.402+0000 7f1d6000a740 1 Processor -- start 2024-05-22T23:01:41.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.402+0000 7f1d6000a740 1 -- start start 2024-05-22T23:01:41.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.403+0000 7f1d6000a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558372709b0 0x555837278900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.403+0000 7f1d6000a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558372713a0 0x555837282340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.403+0000 7f1d6000a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555837276510 0x5558372869b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.403+0000 7f1d6000a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555837287520 con 0x5558372713a0 2024-05-22T23:01:41.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.403+0000 7f1d6000a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555837287bf0 con 0x5558372709b0 2024-05-22T23:01:41.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.403+0000 7f1d6000a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5558372882c0 con 0x555837276510 2024-05-22T23:01:41.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.403+0000 7f1d4ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558372713a0 0x555837282340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:41.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.403+0000 7f1d4ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558372713a0 0x555837282340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33934/0 (socket says 172.21.15.84:33934) 2024-05-22T23:01:41.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.403+0000 7f1d4ffff640 1 -- 172.21.15.84:0/1996029236 learned_addr learned my addr 172.21.15.84:0/1996029236 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:41.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.403+0000 7f1d577fe640 1 --2- 172.21.15.84:0/1996029236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555837276510 0x5558372869b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:41.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.403+0000 7f1d4ffff640 1 -- 172.21.15.84:0/1996029236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555837276510 msgr2=0x5558372869b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:41.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.403+0000 7f1d4ffff640 1 --2- 172.21.15.84:0/1996029236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555837276510 0x5558372869b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.403+0000 7f1d4ffff640 1 -- 172.21.15.84:0/1996029236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558372709b0 msgr2=0x555837278900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:41.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.403+0000 7f1d4ffff640 1 --2- 172.21.15.84:0/1996029236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558372709b0 0x555837278900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.403+0000 7f1d4ffff640 1 -- 172.21.15.84:0/1996029236 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555837271c30 con 0x5558372713a0 2024-05-22T23:01:41.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.403+0000 7f1d577fe640 1 --2- 172.21.15.84:0/1996029236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555837276510 0x5558372869b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:01:41.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.403+0000 7f1d4ffff640 1 --2- 172.21.15.84:0/1996029236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558372713a0 0x555837282340 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1d40000c00 tx=0x7f1d400ba510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2383dbe6bda52bdb server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:41.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.403+0000 7f1d4cff9640 1 -- 172.21.15.84:0/1996029236 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40049070 con 0x5558372713a0 2024-05-22T23:01:41.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.403+0000 7f1d4cff9640 1 -- 172.21.15.84:0/1996029236 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d400bc260 con 0x5558372713a0 2024-05-22T23:01:41.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.403+0000 7f1d4cff9640 1 -- 172.21.15.84:0/1996029236 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40031bd0 con 0x5558372713a0 2024-05-22T23:01:41.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.404+0000 7f1d6000a740 1 -- 172.21.15.84:0/1996029236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558372713a0 msgr2=0x555837282340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:41.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.404+0000 7f1d6000a740 1 --2- 172.21.15.84:0/1996029236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558372713a0 0x555837282340 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1d40000c00 tx=0x7f1d400ba510 comp rx=0 tx=0).stop 2024-05-22T23:01:41.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.404+0000 7f1d6000a740 1 -- 172.21.15.84:0/1996029236 shutdown_connections 2024-05-22T23:01:41.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.404+0000 7f1d6000a740 1 --2- 172.21.15.84:0/1996029236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555837276510 0x5558372869b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.404+0000 7f1d6000a740 1 --2- 172.21.15.84:0/1996029236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558372713a0 0x555837282340 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.404+0000 7f1d6000a740 1 --2- 172.21.15.84:0/1996029236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558372709b0 0x555837278900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.404+0000 7f1d6000a740 1 -- 172.21.15.84:0/1996029236 >> 172.21.15.84:0/1996029236 conn(0x55583726c020 msgr2=0x55583727ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:41.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.404+0000 7f1d6000a740 1 -- 172.21.15.84:0/1996029236 shutdown_connections 2024-05-22T23:01:41.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.404+0000 7f1d6000a740 1 -- 172.21.15.84:0/1996029236 wait complete. 2024-05-22T23:01:41.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.404+0000 7f1d6000a740 1 -- start start 2024-05-22T23:01:41.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.404+0000 7f1d6000a740 1 Processor -- start 2024-05-22T23:01:41.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.405+0000 7f1d6000a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558372709b0 0x55583727db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.405+0000 7f1d6000a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558372713a0 0x55583727e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.405+0000 7f1d6000a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555837276510 0x5558372810b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.405+0000 7f1d6000a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555837289060 con 0x555837276510 2024-05-22T23:01:41.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.405+0000 7f1d6000a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555837288990 con 0x5558372713a0 2024-05-22T23:01:41.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.405+0000 7f1d6000a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555837289730 con 0x5558372709b0 2024-05-22T23:01:41.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.405+0000 7f1d6000a740 10 client.0 _ll_register_callbacks cb 0x7f1d40005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:01:41.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.405+0000 7f1d577fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555837276510 0x5558372810b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:41.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.405+0000 7f1d577fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555837276510 0x5558372810b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33944/0 (socket says 172.21.15.84:33944) 2024-05-22T23:01:41.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.405+0000 7f1d577fe640 1 -- 172.21.15.84:0/3285586357 learned_addr learned my addr 172.21.15.84:0/3285586357 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:41.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.405+0000 7f1d56ffd640 1 --2- 172.21.15.84:0/3285586357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558372709b0 0x55583727db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:41.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.405+0000 7f1d577fe640 1 -- 172.21.15.84:0/3285586357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558372709b0 msgr2=0x55583727db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:41.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.405+0000 7f1d577fe640 1 --2- 172.21.15.84:0/3285586357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558372709b0 0x55583727db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.405+0000 7f1d577fe640 1 -- 172.21.15.84:0/3285586357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558372713a0 msgr2=0x55583727e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:41.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.405+0000 7f1d577fe640 1 --2- 172.21.15.84:0/3285586357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558372713a0 0x55583727e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.405+0000 7f1d577fe640 1 -- 172.21.15.84:0/3285586357 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558372816a0 con 0x555837276510 2024-05-22T23:01:41.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.405+0000 7f1d577fe640 1 --2- 172.21.15.84:0/3285586357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555837276510 0x5558372810b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1d50012e20 tx=0x7f1d50004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccac5b695c9ccb6e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:41.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.405+0000 7f1d3e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:01:41.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.405+0000 7f1d3e7fc640 1 -- 172.21.15.84:0/3285586357 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50029060 con 0x555837276510 2024-05-22T23:01:41.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.405+0000 7f1d3e7fc640 1 -- 172.21.15.84:0/3285586357 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f1d50024600 con 0x555837276510 2024-05-22T23:01:41.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.405+0000 7f1d6000a740 1 -- 172.21.15.84:0/3285586357 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5558371db100 con 0x555837276510 2024-05-22T23:01:41.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.405+0000 7f1d3e7fc640 1 -- 172.21.15.84:0/3285586357 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f1d50024800 con 0x555837276510 2024-05-22T23:01:41.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.405+0000 7f1d6000a740 10 client.15162 fetch_fsmap learned FSMap version 18 2024-05-22T23:01:41.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.405+0000 7f1d6000a740 1 -- 172.21.15.84:0/3285586357 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5558372edeb0 con 0x555837276510 2024-05-22T23:01:41.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.406+0000 7f1d3e7fc640 1 -- 172.21.15.84:0/3285586357 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5558371db100 con 0x555837276510 2024-05-22T23:01:41.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.406+0000 7f1d3e7fc640 1 -- 172.21.15.84:0/3285586357 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f1d5002fbc0 con 0x555837276510 2024-05-22T23:01:41.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.406+0000 7f1d6000a740 10 client.15162 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:01:41.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.406+0000 7f1d6000a740 10 client.15162 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-22T23:01:41.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.406+0000 7f1d6000a740 20 client.15162 populate_metadata read hostname 'smithi084' 2024-05-22T23:01:41.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.406+0000 7f1d6000a740 1 --2- 172.21.15.84:0/3285586357 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5558372f2b50 0x5558372eef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.406+0000 7f1d6000a740 4 client.15162 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-22T23:01:41.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.406+0000 7f1d6000a740 1 -- 172.21.15.84:0/3285586357 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "status"}) -- 0x555837158b80 con 0x5558372f2b50 2024-05-22T23:01:41.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.406+0000 7f1d56ffd640 1 --2- 172.21.15.84:0/3285586357 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5558372f2b50 0x5558372eef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:41.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.406+0000 7f1d3e7fc640 10 client.15162 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:01:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.406+0000 7f1d56ffd640 1 --2- 172.21.15.84:0/3285586357 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5558372f2b50 0x5558372eef90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ed9f492732911bd server_cookie=9a9e9d8ddc7b9f31 in_seq=0 out_seq=0 2024-05-22T23:01:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d3e7fc640 1 -- 172.21.15.84:0/3285586357 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x555837158b80 con 0x5558372f2b50 2024-05-22T23:01:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d3e7fc640 10 client.15162 handle_command_reply: tid=0 2024-05-22T23:01:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d6000a740 1 client.15162 shutdown 2024-05-22T23:01:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d6000a740 1 -- 172.21.15.84:0/3285586357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555837276510 msgr2=0x5558372810b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d6000a740 1 --2- 172.21.15.84:0/3285586357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555837276510 0x5558372810b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1d50012e20 tx=0x7f1d50004ab0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d6000a740 1 -- 172.21.15.84:0/3285586357 shutdown_connections 2024-05-22T23:01:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d6000a740 1 --2- 172.21.15.84:0/3285586357 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5558372f2b50 0x5558372eef90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d6000a740 1 --2- 172.21.15.84:0/3285586357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555837276510 0x5558372810b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d6000a740 1 --2- 172.21.15.84:0/3285586357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558372713a0 0x55583727e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d3e7fc640 0 client.15162 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:01:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d6000a740 1 --2- 172.21.15.84:0/3285586357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558372709b0 0x55583727db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d6000a740 1 -- 172.21.15.84:0/3285586357 >> 172.21.15.84:0/3285586357 conn(0x55583726c020 msgr2=0x5558372869b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d6000a740 1 -- 172.21.15.84:0/3285586357 shutdown_connections 2024-05-22T23:01:41.416 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d6000a740 1 -- 172.21.15.84:0/3285586357 wait complete. 2024-05-22T23:01:41.416 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d6000a740 20 client.15162 trim_cache size 0 max 16384 2024-05-22T23:01:41.416 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:01:41.416 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "c1135a12-188d-11ef-bc9a-c7b262605968", 2024-05-22T23:01:41.416 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2024-05-22T23:01:41.416 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 24530, 2024-05-22T23:01:41.416 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2024-05-22T23:01:41.416 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2024-05-22T23:01:41.416 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2024-05-22T23:01:41.416 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 108.821235829, 2024-05-22T23:01:41.416 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 18, 2024-05-22T23:01:41.417 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 88, 2024-05-22T23:01:41.417 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 86, 2024-05-22T23:01:41.417 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 115.442425594 2024-05-22T23:01:41.417 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:01:41.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d5d2fc640 1 -- 172.21.15.84:0/2716103330 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1d440a2b90 msgr2=0x7f1d440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:41.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d5d2fc640 1 --2- 172.21.15.84:0/2716103330 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1d440a2b90 0x7f1d440a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x555837271c00 tx=0x7f1d50004a40 comp rx=0 tx=0).stop 2024-05-22T23:01:41.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d5d2fc640 1 -- 172.21.15.84:0/2716103330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d58104cc0 msgr2=0x7f1d581981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:41.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d5d2fc640 1 --2- 172.21.15.84:0/2716103330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d58104cc0 0x7f1d581981f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1d40031610 tx=0x7f1d400043b0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d5d2fc640 1 -- 172.21.15.84:0/2716103330 shutdown_connections 2024-05-22T23:01:41.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d5d2fc640 1 --2- 172.21.15.84:0/2716103330 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1d440a2b90 0x7f1d440a5050 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d5d2fc640 1 --2- 172.21.15.84:0/2716103330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d581075f0 0x7f1d5819f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d5d2fc640 1 --2- 172.21.15.84:0/2716103330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d58104cc0 0x7f1d581981f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d5d2fc640 1 --2- 172.21.15.84:0/2716103330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d58102390 0x7f1d58197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.407+0000 7f1d5d2fc640 1 -- 172.21.15.84:0/2716103330 >> 172.21.15.84:0/2716103330 conn(0x7f1d580fbd20 msgr2=0x7f1d58102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:41.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.408+0000 7f1d5d2fc640 1 -- 172.21.15.84:0/2716103330 shutdown_connections 2024-05-22T23:01:41.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.408+0000 7f1d5d2fc640 1 -- 172.21.15.84:0/2716103330 wait complete. 2024-05-22T23:01:41.420 INFO:teuthology.run_tasks:Running task sequential... 2024-05-22T23:01:41.430 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-22T23:01:41.430 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-22T23:01:41.431 INFO:teuthology.task.print:Enabling mgr modules 2024-05-22T23:01:41.431 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-22T23:01:41.432 INFO:teuthology.task.exec:Executing custom commands... 2024-05-22T23:01:41.432 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi084.front.sepia.ceph.com 2024-05-22T23:01:41.432 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-22T23:01:41.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.531+0000 7f367bc4c640 1 -- 172.21.15.84:0/455777879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3674102380 msgr2=0x7f3674108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:41.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.531+0000 7f367bc4c640 1 --2- 172.21.15.84:0/455777879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3674102380 0x7f3674108c10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f366c009980 tx=0x7f366c02e820 comp rx=0 tx=0).stop 2024-05-22T23:01:41.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.531+0000 7f367bc4c640 1 -- 172.21.15.84:0/455777879 shutdown_connections 2024-05-22T23:01:41.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.531+0000 7f367bc4c640 1 --2- 172.21.15.84:0/455777879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3674102380 0x7f3674108c10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.531+0000 7f367bc4c640 1 --2- 172.21.15.84:0/455777879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36741019c0 0x7f3674101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.531+0000 7f367bc4c640 1 --2- 172.21.15.84:0/455777879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36741007d0 0x7f3674100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.531+0000 7f367bc4c640 1 -- 172.21.15.84:0/455777879 >> 172.21.15.84:0/455777879 conn(0x7f36740fbd40 msgr2=0x7f36740fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:41.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.531+0000 7f367bc4c640 1 -- 172.21.15.84:0/455777879 shutdown_connections 2024-05-22T23:01:41.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.531+0000 7f367bc4c640 1 -- 172.21.15.84:0/455777879 wait complete. 2024-05-22T23:01:41.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.532+0000 7f367bc4c640 1 Processor -- start 2024-05-22T23:01:41.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.532+0000 7f367bc4c640 1 -- start start 2024-05-22T23:01:41.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.532+0000 7f367bc4c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36741007d0 0x7f3674195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.532+0000 7f367bc4c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36741019c0 0x7f3674195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.532+0000 7f367bc4c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3674102380 0x7f367419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.532+0000 7f367bc4c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f367410b3b0 con 0x7f36741007d0 2024-05-22T23:01:41.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.532+0000 7f367bc4c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f367410b230 con 0x7f36741019c0 2024-05-22T23:01:41.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.532+0000 7f367bc4c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f367410b530 con 0x7f3674102380 2024-05-22T23:01:41.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.532+0000 7f36799e8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36741007d0 0x7f3674195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:41.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.532+0000 7f36799e8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36741007d0 0x7f3674195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37996/0 (socket says 172.21.15.84:37996) 2024-05-22T23:01:41.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.532+0000 7f36799e8640 1 -- 172.21.15.84:0/3435873392 learned_addr learned my addr 172.21.15.84:0/3435873392 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:41.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.532+0000 7f36791e7640 1 --2- 172.21.15.84:0/3435873392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36741019c0 0x7f3674195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:41.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.533+0000 7f367a1e9640 1 --2- 172.21.15.84:0/3435873392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3674102380 0x7f367419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:41.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.533+0000 7f36799e8640 1 -- 172.21.15.84:0/3435873392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3674102380 msgr2=0x7f367419ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:41.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.533+0000 7f36799e8640 1 --2- 172.21.15.84:0/3435873392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3674102380 0x7f367419ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.533+0000 7f36799e8640 1 -- 172.21.15.84:0/3435873392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36741019c0 msgr2=0x7f3674195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:41.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.533+0000 7f36799e8640 1 --2- 172.21.15.84:0/3435873392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36741019c0 0x7f3674195e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:41.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.533+0000 7f36799e8640 1 -- 172.21.15.84:0/3435873392 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367419d520 con 0x7f36741007d0 2024-05-22T23:01:41.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.533+0000 7f36799e8640 1 --2- 172.21.15.84:0/3435873392 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36741007d0 0x7f3674195900 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f366400b9c0 tx=0x7f366400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:41.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.533+0000 7f3672ffd640 1 -- 172.21.15.84:0/3435873392 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3664019070 con 0x7f36741007d0 2024-05-22T23:01:41.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.533+0000 7f3672ffd640 1 -- 172.21.15.84:0/3435873392 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f366400cba0 con 0x7f36741007d0 2024-05-22T23:01:41.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.533+0000 7f367bc4c640 1 -- 172.21.15.84:0/3435873392 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f367419d810 con 0x7f36741007d0 2024-05-22T23:01:41.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.533+0000 7f3672ffd640 1 -- 172.21.15.84:0/3435873392 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3664012640 con 0x7f36741007d0 2024-05-22T23:01:41.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.533+0000 7f367bc4c640 1 -- 172.21.15.84:0/3435873392 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f367419dd90 con 0x7f36741007d0 2024-05-22T23:01:41.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.534+0000 7f367bc4c640 1 -- 172.21.15.84:0/3435873392 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3674107a20 con 0x7f36741007d0 2024-05-22T23:01:41.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.535+0000 7f3672ffd640 1 -- 172.21.15.84:0/3435873392 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f366401c070 con 0x7f36741007d0 2024-05-22T23:01:41.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.536+0000 7f3672ffd640 1 --2- 172.21.15.84:0/3435873392 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f365c0a2c30 0x7f365c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:41.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.536+0000 7f36791e7640 1 --2- 172.21.15.84:0/3435873392 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f365c0a2c30 0x7f365c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:41.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.536+0000 7f3672ffd640 1 -- 172.21.15.84:0/3435873392 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f36640ce6d0 con 0x7f36741007d0 2024-05-22T23:01:41.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.536+0000 7f36791e7640 1 --2- 172.21.15.84:0/3435873392 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f365c0a2c30 0x7f365c0a50f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3668005fd0 tx=0x7f3668005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:41.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.540+0000 7f3672ffd640 1 -- 172.21.15.84:0/3435873392 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f366409b610 con 0x7f36741007d0 2024-05-22T23:01:41.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:41.765+0000 7f367bc4c640 1 -- 172.21.15.84:0/3435873392 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f3674062470 con 0x7f36741007d0 2024-05-22T23:01:41.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:41 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3353489246' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:01:41.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:41 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3640436965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:01:41.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:41 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/415782083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:41.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:41 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3353489246' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:01:41.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:41 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3640436965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:01:41.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:41 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/415782083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:42.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:41 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3353489246' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:01:42.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:41 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3640436965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:01:42.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:41 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/415782083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:42.602 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:42.601+0000 7f3672ffd640 1 -- 172.21.15.84:0/3435873392 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f366409af10 con 0x7f36741007d0 2024-05-22T23:01:42.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:42.604+0000 7f367bc4c640 1 -- 172.21.15.84:0/3435873392 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f365c0a2c30 msgr2=0x7f365c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:42.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:42.604+0000 7f367bc4c640 1 --2- 172.21.15.84:0/3435873392 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f365c0a2c30 0x7f365c0a50f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3668005fd0 tx=0x7f3668005850 comp rx=0 tx=0).stop 2024-05-22T23:01:42.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:42.604+0000 7f367bc4c640 1 -- 172.21.15.84:0/3435873392 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36741007d0 msgr2=0x7f3674195900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:42.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:42.604+0000 7f367bc4c640 1 --2- 172.21.15.84:0/3435873392 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36741007d0 0x7f3674195900 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f366400b9c0 tx=0x7f366400be80 comp rx=0 tx=0).stop 2024-05-22T23:01:42.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:42.604+0000 7f367bc4c640 1 -- 172.21.15.84:0/3435873392 shutdown_connections 2024-05-22T23:01:42.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:42.604+0000 7f367bc4c640 1 --2- 172.21.15.84:0/3435873392 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f365c0a2c30 0x7f365c0a50f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:42.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:42.604+0000 7f367bc4c640 1 --2- 172.21.15.84:0/3435873392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3674102380 0x7f367419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:42.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:42.604+0000 7f367bc4c640 1 --2- 172.21.15.84:0/3435873392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36741019c0 0x7f3674195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:42.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:42.604+0000 7f367bc4c640 1 --2- 172.21.15.84:0/3435873392 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36741007d0 0x7f3674195900 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:42.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:42.604+0000 7f367bc4c640 1 -- 172.21.15.84:0/3435873392 >> 172.21.15.84:0/3435873392 conn(0x7f36740fbd40 msgr2=0x7f3674104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:42.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:42.604+0000 7f367bc4c640 1 -- 172.21.15.84:0/3435873392 shutdown_connections 2024-05-22T23:01:42.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:42.604+0000 7f367bc4c640 1 -- 172.21.15.84:0/3435873392 wait complete. 2024-05-22T23:01:42.623 INFO:teuthology.run_tasks:Running task kclient... 2024-05-22T23:01:42.651 INFO:tasks.kclient:Mounting kernel clients... 2024-05-22T23:01:42.651 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-22T23:01:42.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:01:42.652 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2024-05-22T23:01:42.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:01:42.670 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link delete ceph-brx 2024-05-22T23:01:42.738 INFO:teuthology.orchestra.run.smithi084.stderr:Cannot find device "ceph-brx" 2024-05-22T23:01:42.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T23:01:42.740 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-05-22T23:01:42.740 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-22T23:01:42.740 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-22T23:01:42.740 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph fs ls 2024-05-22T23:01:42.875 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:42 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: ignoring --setuser ceph since I am not root 2024-05-22T23:01:42.875 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:42 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: ignoring --setgroup ceph since I am not root 2024-05-22T23:01:42.875 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:42 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:42.790+0000 7f447aae9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-22T23:01:42.876 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:42 smithi084 ceph-mon[32186]: pgmap v346: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:01:42.876 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:42 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3435873392' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-22T23:01:42.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:42 smithi204 ceph-mon[36211]: pgmap v346: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:01:42.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:42 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3435873392' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-22T23:01:42.912 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:42 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: ignoring --setuser ceph since I am not root 2024-05-22T23:01:42.912 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:42 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: ignoring --setgroup ceph since I am not root 2024-05-22T23:01:42.912 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:42 smithi204 ceph-mgr[38067]: -- 172.21.15.204:0/3143302229 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b51f2685a0 con 0x55b51f23cc00 2024-05-22T23:01:42.912 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:42 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:42.783+0000 7f4715c6d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-22T23:01:42.965 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:42 smithi096 ceph-mon[36392]: pgmap v346: 97 pgs: 97 active+clean; 635 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:01:42.965 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:42 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3435873392' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-22T23:01:42.965 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:42 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: ignoring --setuser ceph since I am not root 2024-05-22T23:01:42.965 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:42 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: ignoring --setgroup ceph since I am not root 2024-05-22T23:01:42.965 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:42 smithi096 ceph-mgr[37770]: -- 172.21.15.96:0/337761047 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5576699505a0 con 0x557669925000 2024-05-22T23:01:42.965 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:42 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:42.778+0000 7f50b97ca200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-22T23:01:42.965 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:42 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:42.963+0000 7f50b97ca200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-22T23:01:43.025 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:01:43.266 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:42 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:42.988+0000 7f447aae9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-22T23:01:43.266 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:43 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:43.099+0000 7f447aae9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-22T23:01:43.299 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:43 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:43.065+0000 7f50b97ca200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-22T23:01:43.378 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:42 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:42.968+0000 7f4715c6d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-22T23:01:43.378 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:43 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:43.071+0000 7f4715c6d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-22T23:01:43.603 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:43 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:43.424+0000 7f447aae9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-22T23:01:43.604 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:43 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:43.511+0000 7f447aae9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-22T23:01:43.661 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:43 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:43.377+0000 7f4715c6d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-22T23:01:43.661 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:43 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:43.463+0000 7f4715c6d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-22T23:01:43.661 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:43 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3435873392' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-22T23:01:43.661 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:43 smithi204 ceph-mon[36211]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-05-22T23:01:43.800 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:43 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3435873392' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-22T23:01:43.800 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:43 smithi096 ceph-mon[36392]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-05-22T23:01:43.800 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:43 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:43.371+0000 7f50b97ca200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-22T23:01:43.800 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:43 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:43.456+0000 7f50b97ca200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-22T23:01:43.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:43 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3435873392' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-22T23:01:43.875 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:43 smithi084 ceph-mon[32186]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-05-22T23:01:44.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.809+0000 7f1f7075f700 1 -- 172.21.15.84:0/1975213526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f68101260 msgr2=0x7f1f681016e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:44.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.809+0000 7f1f7075f700 1 --2- 172.21.15.84:0/1975213526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f68101260 0x7f1f681016e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1f58009990 tx=0x7f1f58009ca0 comp rx=0 tx=0).stop 2024-05-22T23:01:44.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.810+0000 7f1f7075f700 1 -- 172.21.15.84:0/1975213526 shutdown_connections 2024-05-22T23:01:44.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.810+0000 7f1f7075f700 1 --2- 172.21.15.84:0/1975213526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f68101c20 0x7f1f6810e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:44.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.810+0000 7f1f7075f700 1 --2- 172.21.15.84:0/1975213526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f681072d0 0x7f1f681076b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:44.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.810+0000 7f1f7075f700 1 --2- 172.21.15.84:0/1975213526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f68101260 0x7f1f681016e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:44.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.810+0000 7f1f7075f700 1 -- 172.21.15.84:0/1975213526 >> 172.21.15.84:0/1975213526 conn(0x7f1f68078210 msgr2=0x7f1f68078620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:44.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.810+0000 7f1f7075f700 1 -- 172.21.15.84:0/1975213526 shutdown_connections 2024-05-22T23:01:44.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.810+0000 7f1f7075f700 1 -- 172.21.15.84:0/1975213526 wait complete. 2024-05-22T23:01:44.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.810+0000 7f1f7075f700 1 Processor -- start 2024-05-22T23:01:44.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.811+0000 7f1f7075f700 1 -- start start 2024-05-22T23:01:44.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.811+0000 7f1f7075f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f68101260 0x7f1f68199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:44.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.811+0000 7f1f7075f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f68101c20 0x7f1f68199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:44.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.811+0000 7f1f7075f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f681072d0 0x7f1f681a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:44.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.811+0000 7f1f7075f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1f68111540 con 0x7f1f681072d0 2024-05-22T23:01:44.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.811+0000 7f1f7075f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1f681113c0 con 0x7f1f68101c20 2024-05-22T23:01:44.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.811+0000 7f1f7075f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1f681116c0 con 0x7f1f68101260 2024-05-22T23:01:44.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.811+0000 7f1f6ecfc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f681072d0 0x7f1f681a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:44.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.811+0000 7f1f6ecfc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f681072d0 0x7f1f681a4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38034/0 (socket says 172.21.15.84:38034) 2024-05-22T23:01:44.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.811+0000 7f1f6ecfc700 1 -- 172.21.15.84:0/297948607 learned_addr learned my addr 172.21.15.84:0/297948607 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:44.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.811+0000 7f1f6e4fb700 1 --2- 172.21.15.84:0/297948607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f68101260 0x7f1f68199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:44.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.811+0000 7f1f6dcfa700 1 --2- 172.21.15.84:0/297948607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f68101c20 0x7f1f68199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:44.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.811+0000 7f1f6ecfc700 1 -- 172.21.15.84:0/297948607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f68101260 msgr2=0x7f1f68199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:44.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.811+0000 7f1f6ecfc700 1 --2- 172.21.15.84:0/297948607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f68101260 0x7f1f68199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:44.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.811+0000 7f1f6ecfc700 1 -- 172.21.15.84:0/297948607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f68101c20 msgr2=0x7f1f68199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:44.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.811+0000 7f1f6ecfc700 1 --2- 172.21.15.84:0/297948607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f68101c20 0x7f1f68199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:44.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.811+0000 7f1f6ecfc700 1 -- 172.21.15.84:0/297948607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f681a4ec0 con 0x7f1f681072d0 2024-05-22T23:01:44.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.811+0000 7f1f6ecfc700 1 --2- 172.21.15.84:0/297948607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f681072d0 0x7f1f681a4980 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1f60007ee0 tx=0x7f1f6000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:44.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.812+0000 7f1f5f7fe700 1 -- 172.21.15.84:0/297948607 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f60004bb0 con 0x7f1f681072d0 2024-05-22T23:01:44.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.812+0000 7f1f5f7fe700 1 -- 172.21.15.84:0/297948607 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f60004d50 con 0x7f1f681072d0 2024-05-22T23:01:44.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.812+0000 7f1f7075f700 1 -- 172.21.15.84:0/297948607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f681a50f0 con 0x7f1f681072d0 2024-05-22T23:01:44.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.812+0000 7f1f5f7fe700 1 -- 172.21.15.84:0/297948607 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f60005960 con 0x7f1f681072d0 2024-05-22T23:01:44.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.812+0000 7f1f7075f700 1 -- 172.21.15.84:0/297948607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f681a55b0 con 0x7f1f681072d0 2024-05-22T23:01:44.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.813+0000 7f1f7075f700 1 -- 172.21.15.84:0/297948607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f68069050 con 0x7f1f681072d0 2024-05-22T23:01:44.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.814+0000 7f1f5f7fe700 1 -- 172.21.15.84:0/297948607 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f1f6001a020 con 0x7f1f681072d0 2024-05-22T23:01:44.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.815+0000 7f1f5f7fe700 1 --2- 172.21.15.84:0/297948607 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1f540a2c70 0x7f1f540a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:44.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.815+0000 7f1f6e4fb700 1 -- 172.21.15.84:0/297948607 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1f540a2c70 msgr2=0x7f1f540a5130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/4223589140 2024-05-22T23:01:44.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.815+0000 7f1f5f7fe700 1 -- 172.21.15.84:0/297948607 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f1f600ce800 con 0x7f1f681072d0 2024-05-22T23:01:44.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.815+0000 7f1f6e4fb700 1 --2- 172.21.15.84:0/297948607 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1f540a2c70 0x7f1f540a5130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-22T23:01:44.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:44.819+0000 7f1f5f7fe700 1 -- 172.21.15.84:0/297948607 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1f60010070 con 0x7f1f681072d0 2024-05-22T23:01:44.911 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:44 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:44.657+0000 7f4715c6d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-22T23:01:44.911 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:44 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:44.828+0000 7f4715c6d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-22T23:01:44.977 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:44 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:44.636+0000 7f50b97ca200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-22T23:01:44.977 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:44 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:44.806+0000 7f50b97ca200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-22T23:01:44.977 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:44 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:44.975+0000 7f50b97ca200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-22T23:01:45.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:45.016+0000 7f1f6e4fb700 1 -- 172.21.15.84:0/297948607 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1f540a2c70 msgr2=0x7f1f540a5130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/4223589140 2024-05-22T23:01:45.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:45.016+0000 7f1f6e4fb700 1 --2- 172.21.15.84:0/297948607 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1f540a2c70 0x7f1f540a5130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-22T23:01:45.018 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:44 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:44.746+0000 7f447aae9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-22T23:01:45.018 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:44 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:44.927+0000 7f447aae9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-22T23:01:45.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:45.067+0000 7f1f7075f700 1 -- 172.21.15.84:0/297948607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1f6802dbc0 con 0x7f1f681072d0 2024-05-22T23:01:45.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:45.068+0000 7f1f5f7fe700 1 -- 172.21.15.84:0/297948607 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7f1f6009b740 con 0x7f1f681072d0 2024-05-22T23:01:45.070 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T23:01:45.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:45.071+0000 7f1f7075f700 1 -- 172.21.15.84:0/297948607 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1f540a2c70 msgr2=0x7f1f540a5130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-22T23:01:45.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:45.071+0000 7f1f7075f700 1 --2- 172.21.15.84:0/297948607 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1f540a2c70 0x7f1f540a5130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:45.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:45.071+0000 7f1f7075f700 1 -- 172.21.15.84:0/297948607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f681072d0 msgr2=0x7f1f681a4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:45.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:45.071+0000 7f1f7075f700 1 --2- 172.21.15.84:0/297948607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f681072d0 0x7f1f681a4980 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1f60007ee0 tx=0x7f1f6000d3b0 comp rx=0 tx=0).stop 2024-05-22T23:01:45.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:45.071+0000 7f1f7075f700 1 -- 172.21.15.84:0/297948607 shutdown_connections 2024-05-22T23:01:45.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:45.071+0000 7f1f7075f700 1 --2- 172.21.15.84:0/297948607 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f1f540a2c70 0x7f1f540a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:45.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:45.071+0000 7f1f7075f700 1 --2- 172.21.15.84:0/297948607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f681072d0 0x7f1f681a4980 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:45.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:45.071+0000 7f1f7075f700 1 --2- 172.21.15.84:0/297948607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f68101260 0x7f1f68199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:45.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:45.071+0000 7f1f7075f700 1 --2- 172.21.15.84:0/297948607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f68101c20 0x7f1f68199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:45.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:45.071+0000 7f1f7075f700 1 -- 172.21.15.84:0/297948607 >> 172.21.15.84:0/297948607 conn(0x7f1f68078210 msgr2=0x7f1f680ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:45.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:45.072+0000 7f1f7075f700 1 -- 172.21.15.84:0/297948607 shutdown_connections 2024-05-22T23:01:45.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:45.072+0000 7f1f7075f700 1 -- 172.21.15.84:0/297948607 wait complete. 2024-05-22T23:01:45.266 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:45 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/297948607' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-22T23:01:45.296 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:45 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:45.001+0000 7f4715c6d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-22T23:01:45.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:45 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/297948607' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-22T23:01:45.375 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:45 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:45.108+0000 7f447aae9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-22T23:01:45.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:45 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/297948607' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-22T23:01:45.550 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:45 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:45.265+0000 7f50b97ca200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-22T23:01:45.550 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:45 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:45.357+0000 7f50b97ca200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-22T23:01:45.594 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:45 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:45.294+0000 7f4715c6d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-22T23:01:45.595 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:45 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:45.388+0000 7f4715c6d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-22T23:01:45.711 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:45 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:45.408+0000 7f447aae9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-22T23:01:45.711 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:45 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:45.502+0000 7f447aae9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-22T23:01:45.807 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-22T23:01:45.808 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T23:01:45.808 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2024-05-22T23:01:45.808 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-22T23:01:45.808 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-22T23:01:45.808 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-22T23:01:45.808 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-22T23:01:45.808 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T23:01:45.808 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-22T23:01:45.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:01:45.808 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2024-05-22T23:01:45.824 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T23:01:45.824 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T23:01:45.825 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T23:01:45.825 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2024-05-22T23:01:45.825 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2024-05-22T23:01:45.825 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2024-05-22T23:01:45.825 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-22T23:01:45.825 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2024-05-22T23:01:45.825 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2024-05-22T23:01:45.825 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-22T23:01:45.825 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2024-05-22T23:01:45.825 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2024-05-22T23:01:45.825 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-22T23:01:45.825 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2024-05-22T23:01:45.825 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T23:01:45.826 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2024-05-22T23:01:45.826 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-22T23:01:45.826 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 42145sec preferred_lft 42145sec 2024-05-22T23:01:45.826 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2024-05-22T23:01:45.826 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2024-05-22T23:01:45.826 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-22T23:01:45.826 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-22T23:01:45.826 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2024-05-22T23:01:45.827 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link add name ceph-brx type bridge 2024-05-22T23:01:45.827 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip addr flush dev ceph-brx 2024-05-22T23:01:45.827 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set ceph-brx up 2024-05-22T23:01:45.827 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-22T23:01:45.827 DEBUG:teuthology.orchestra.run.smithi084:> ') 2024-05-22T23:01:45.911 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:45 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:45.593+0000 7f4715c6d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-22T23:01:45.911 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:45 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:45.789+0000 7f4715c6d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-22T23:01:45.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-22T23:01:45.975 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:45 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:45.709+0000 7f447aae9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-22T23:01:45.975 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:45 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:45.923+0000 7f447aae9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-22T23:01:46.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-22T23:01:46.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:01:46.018 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T23:01:46.043 INFO:teuthology.orchestra.run.smithi084.stdout:1 2024-05-22T23:01:46.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:01:46.044 DEBUG:teuthology.orchestra.run.smithi084:> route 2024-05-22T23:01:46.050 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:45 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:45.558+0000 7f50b97ca200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-22T23:01:46.050 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:45 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:45.750+0000 7f50b97ca200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-22T23:01:46.116 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2024-05-22T23:01:46.116 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T23:01:46.116 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T23:01:46.116 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T23:01:46.116 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T23:01:46.116 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-22T23:01:46.117 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2024-05-22T23:01:46.117 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T23:01:46.117 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T23:01:46.117 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T23:01:46.117 DEBUG:teuthology.orchestra.run.smithi084:> ') 2024-05-22T23:01:46.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-22T23:01:46.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-22T23:01:46.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:01:46.420 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2024-05-22T23:01:46.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:01:46.436 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list-id 2024-05-22T23:01:46.493 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-22T23:01:46.493 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2024-05-22T23:01:46.493 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-22T23:01:46.493 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-22T23:01:46.493 DEBUG:teuthology.orchestra.run.smithi084:> ') 2024-05-22T23:01:46.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-22T23:01:46.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-22T23:01:46.612 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-22T23:01:46.612 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-22T23:01:46.612 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2024-05-22T23:01:46.613 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-22T23:01:46.613 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-22T23:01:46.613 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-22T23:01:46.613 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-22T23:01:46.613 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-22T23:01:46.613 DEBUG:teuthology.orchestra.run.smithi084:> ') 2024-05-22T23:01:46.661 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:46 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:46.394+0000 7f4715c6d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-22T23:01:46.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-22T23:01:46.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-22T23:01:46.741 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-22T23:01:46.741 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2024-05-22T23:01:46.741 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2024-05-22T23:01:46.741 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set dev brx.0 master ceph-brx 2024-05-22T23:01:46.741 DEBUG:teuthology.orchestra.run.smithi084:> ') 2024-05-22T23:01:46.767 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:46 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:46.345+0000 7f50b97ca200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-22T23:01:46.791 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-22T23:01:46.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-22T23:01:46.823 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T23:01:46.823 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-22T23:01:46.839 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-22T23:01:46.839 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T23:01:46.839 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-22T23:01:46.875 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:46 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:46.570+0000 7f447aae9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-22T23:01:46.890 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-22T23:01:46.890 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-22T23:01:46.890 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/3ffdedc9-b28a-44c9-9c20-ef01dee53af2 2024-05-22T23:01:46.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:01:46.890 DEBUG:teuthology.orchestra.run.smithi084:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/3ffdedc9-b28a-44c9-9c20-ef01dee53af2 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-05-22T23:01:46.985 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-05-22T23:01:46.985 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-05-22T23:01:47.050 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:46 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:46.765+0000 7f50b97ca200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-22T23:01:47.050 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:46 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:46.864+0000 7f50b97ca200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-22T23:01:47.096 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-05-22T23:01:47.096 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-05-22T23:01:47.096 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: resolved to: "172.21.15.84:6789,172.21.15.96:6789,172.21.15.204:6789" 2024-05-22T23:01:47.097 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: trying mount with new device syntax: 0@c1135a12-188d-11ef-bc9a-c7b262605968.cephfs=/volumes/qa/sv_1/3ffdedc9-b28a-44c9-9c20-ef01dee53af2 2024-05-22T23:01:47.097 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.84:6789/172.21.15.96:6789/172.21.15.204:6789" will pass to kernel 2024-05-22T23:01:47.097 INFO:teuthology.orchestra.run.smithi084.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-22T23:01:47.098 INFO:teuthology.orchestra.run.smithi084.stdout: You just mounted a file system that supports labels which does not 2024-05-22T23:01:47.098 INFO:teuthology.orchestra.run.smithi084.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-22T23:01:47.098 INFO:teuthology.orchestra.run.smithi084.stdout: applications will generate AVC messages and not be allowed access to 2024-05-22T23:01:47.098 INFO:teuthology.orchestra.run.smithi084.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-22T23:01:47.100 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T23:01:47.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:01:47.100 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-22T23:01:47.126 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T23:01:47.127 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:46 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:46.822+0000 7f4715c6d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-22T23:01:47.127 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:46 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:46.920+0000 7f4715c6d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-22T23:01:47.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-22T23:01:47.219 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-22T23:01:47.225 INFO:teuthology.orchestra.run.smithi084.stdout:client24772 2024-05-22T23:01:47.225 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-22T23:01:47.225 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T23:01:47.241 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T23:01:47.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-22T23:01:47.333 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-22T23:01:47.339 INFO:teuthology.orchestra.run.smithi084.stdout:client24772 2024-05-22T23:01:47.340 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T23:01:47.371 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:47 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:47.043+0000 7f447aae9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-22T23:01:47.371 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:47 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:47.149+0000 7f447aae9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-22T23:01:47.372 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:47 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:47.369+0000 7f447aae9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-22T23:01:47.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-22T23:01:47.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-22T23:01:47.398 INFO:teuthology.orchestra.run.smithi084.stdout:client24772 2024-05-22T23:01:47.398 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c1135a12-188d-11ef-bc9a-c7b262605968.client24772/status') 2024-05-22T23:01:47.411 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:47 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:47.125+0000 7f4715c6d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-22T23:01:47.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-22T23:01:47.499 INFO:teuthology.orchestra.run.smithi084.stdout:instance: client.24772 (3)172.21.15.84:0/2310917964 2024-05-22T23:01:47.499 INFO:teuthology.orchestra.run.smithi084.stdout:blocklisted: false 2024-05-22T23:01:47.499 INFO:teuthology.orchestra.run.smithi084.stderr:0+1 records in 2024-05-22T23:01:47.500 INFO:teuthology.orchestra.run.smithi084.stderr:0+1 records out 2024-05-22T23:01:47.500 INFO:teuthology.orchestra.run.smithi084.stderr:71 bytes copied, 2.3424e-05 s, 3.0 MB/s 2024-05-22T23:01:47.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-22T23:01:47.510 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-22T23:01:47.522 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-22T23:01:47.532 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T23:01:47.549 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:47 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:47.065+0000 7f50b97ca200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-22T23:01:47.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.636+0000 7fa8f08d4640 1 -- 172.21.15.84:0/4081647525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8ec1007b0 msgr2=0x7fa8ec100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:47.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.636+0000 7fa8f08d4640 1 --2- 172.21.15.84:0/4081647525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8ec1007b0 0x7fa8ec100bd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc009900 tx=0x7fa8dc02a410 comp rx=0 tx=0).stop 2024-05-22T23:01:47.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.636+0000 7fa8f08d4640 1 -- 172.21.15.84:0/4081647525 shutdown_connections 2024-05-22T23:01:47.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.636+0000 7fa8f08d4640 1 --2- 172.21.15.84:0/4081647525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8ec102360 0x7fa8ec108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:47.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.636+0000 7fa8f08d4640 1 --2- 172.21.15.84:0/4081647525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8ec1019a0 0x7fa8ec101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:47.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.636+0000 7fa8f08d4640 1 --2- 172.21.15.84:0/4081647525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8ec1007b0 0x7fa8ec100bd0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:47.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.636+0000 7fa8f08d4640 1 -- 172.21.15.84:0/4081647525 >> 172.21.15.84:0/4081647525 conn(0x7fa8ec0fbd20 msgr2=0x7fa8ec0fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:47.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.636+0000 7fa8f08d4640 1 -- 172.21.15.84:0/4081647525 shutdown_connections 2024-05-22T23:01:47.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.636+0000 7fa8f08d4640 1 -- 172.21.15.84:0/4081647525 wait complete. 2024-05-22T23:01:47.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.637+0000 7fa8f08d4640 1 Processor -- start 2024-05-22T23:01:47.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.637+0000 7fa8f08d4640 1 -- start start 2024-05-22T23:01:47.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.637+0000 7fa8f08d4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8ec1007b0 0x7fa8ec10b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:47.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.637+0000 7fa8f08d4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8ec1019a0 0x7fa8ec109770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:47.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.637+0000 7fa8f08d4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8ec102360 0x7fa8ec109cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:47.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.637+0000 7fa8f08d4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec10d5b0 con 0x7fa8ec1019a0 2024-05-22T23:01:47.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.637+0000 7fa8f08d4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec10d430 con 0x7fa8ec102360 2024-05-22T23:01:47.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.637+0000 7fa8f08d4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec10d730 con 0x7fa8ec1007b0 2024-05-22T23:01:47.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.637+0000 7fa8e9d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8ec1019a0 0x7fa8ec109770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:47.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.637+0000 7fa8e9d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8ec1019a0 0x7fa8ec109770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38048/0 (socket says 172.21.15.84:38048) 2024-05-22T23:01:47.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.637+0000 7fa8ead9d640 1 --2- 172.21.15.84:0/1986310027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8ec102360 0x7fa8ec109cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:47.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.637+0000 7fa8e9d9b640 1 -- 172.21.15.84:0/1986310027 learned_addr learned my addr 172.21.15.84:0/1986310027 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:47.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.637+0000 7fa8ea59c640 1 --2- 172.21.15.84:0/1986310027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8ec1007b0 0x7fa8ec10b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:47.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.638+0000 7fa8ead9d640 1 -- 172.21.15.84:0/1986310027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8ec1007b0 msgr2=0x7fa8ec10b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:47.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.638+0000 7fa8ead9d640 1 --2- 172.21.15.84:0/1986310027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8ec1007b0 0x7fa8ec10b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:47.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.638+0000 7fa8ead9d640 1 -- 172.21.15.84:0/1986310027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8ec1019a0 msgr2=0x7fa8ec109770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:47.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.638+0000 7fa8ead9d640 1 --2- 172.21.15.84:0/1986310027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8ec1019a0 0x7fa8ec109770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:47.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.638+0000 7fa8ead9d640 1 -- 172.21.15.84:0/1986310027 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8ec10a540 con 0x7fa8ec102360 2024-05-22T23:01:47.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.638+0000 7fa8e9d9b640 1 --2- 172.21.15.84:0/1986310027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8ec1019a0 0x7fa8ec109770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:01:47.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.638+0000 7fa8ea59c640 1 --2- 172.21.15.84:0/1986310027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8ec1007b0 0x7fa8ec10b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:01:47.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.638+0000 7fa8ead9d640 1 --2- 172.21.15.84:0/1986310027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8ec102360 0x7fa8ec109cb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009fd0 tx=0x7fa8e400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:47.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.638+0000 7fa8db7fe640 1 -- 172.21.15.84:0/1986310027 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e400ce20 con 0x7fa8ec102360 2024-05-22T23:01:47.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.638+0000 7fa8db7fe640 1 -- 172.21.15.84:0/1986310027 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8e4005020 con 0x7fa8ec102360 2024-05-22T23:01:47.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.638+0000 7fa8f08d4640 1 -- 172.21.15.84:0/1986310027 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8ec1a1d30 con 0x7fa8ec102360 2024-05-22T23:01:47.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.638+0000 7fa8db7fe640 1 -- 172.21.15.84:0/1986310027 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4010720 con 0x7fa8ec102360 2024-05-22T23:01:47.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.638+0000 7fa8f08d4640 1 -- 172.21.15.84:0/1986310027 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8ec1a21d0 con 0x7fa8ec102360 2024-05-22T23:01:47.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.639+0000 7fa8f08d4640 1 -- 172.21.15.84:0/1986310027 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8ec107ad0 con 0x7fa8ec102360 2024-05-22T23:01:47.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.640+0000 7fa8db7fe640 1 -- 172.21.15.84:0/1986310027 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fa8e4005540 con 0x7fa8ec102360 2024-05-22T23:01:47.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.641+0000 7fa8db7fe640 1 --2- 172.21.15.84:0/1986310027 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa8d40a2be0 0x7fa8d40a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:47.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.641+0000 7fa8ea59c640 1 -- 172.21.15.84:0/1986310027 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa8d40a2be0 msgr2=0x7fa8d40a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/4223589140 2024-05-22T23:01:47.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.641+0000 7fa8db7fe640 1 -- 172.21.15.84:0/1986310027 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fa8e40cf950 con 0x7fa8ec102360 2024-05-22T23:01:47.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.641+0000 7fa8ea59c640 1 --2- 172.21.15.84:0/1986310027 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa8d40a2be0 0x7fa8d40a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-22T23:01:47.646 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.644+0000 7fa8db7fe640 1 -- 172.21.15.84:0/1986310027 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa8e409c890 con 0x7fa8ec102360 2024-05-22T23:01:47.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.827+0000 7fa8f08d4640 1 -- 172.21.15.84:0/1986310027 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8ec02d450 con 0x7fa8ec102360 2024-05-22T23:01:47.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.828+0000 7fa8db7fe640 1 -- 172.21.15.84:0/1986310027 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22722 (secure 0 0 0) 0x7fa8e409c190 con 0x7fa8ec102360 2024-05-22T23:01:47.830 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:47.831 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":88,"fsid":"c1135a12-188d-11ef-bc9a-c7b262605968","created":"2024-05-22T22:51:49.878218+0000","modified":"2024-05-22T23:00:46.572591+0000","last_up_change":"2024-05-22T22:58:58.550278+0000","last_in_change":"2024-05-22T22:58:37.322139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T22:55:07.719394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T22:59:50.150511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T22:59:51.111182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762df065-fb4f-4c71-aab8-a62d2282c253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6803","nonce":1559945826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6805","nonce":1559945826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6809","nonce":1559945826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6807","nonce":1559945826}]},"public_addr":"172.21.15.84:6803/1559945826","cluster_addr":"172.21.15.84:6805/1559945826","heartbeat_back_addr":"172.21.15.84:6809/1559945826","heartbeat_front_addr":"172.21.15.84:6807/1559945826","state":["exists","up"]},{"osd":1,"uuid":"3fb2385d-6a34-49e3-88e1-116237beb346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6801","nonce":2996505353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6803","nonce":2996505353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6807","nonce":2996505353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6805","nonce":2996505353}]},"public_addr":"172.21.15.96:6801/2996505353","cluster_addr":"172.21.15.96:6803/2996505353","heartbeat_back_addr":"172.21.15.96:6807/2996505353","heartbeat_front_addr":"172.21.15.96:6805/2996505353","state":["exists","up"]},{"osd":2,"uuid":"e306e88e-a308-4585-ae11-c8af10b84038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6801","nonce":1499577897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6803","nonce":1499577897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6807","nonce":1499577897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6805","nonce":1499577897}]},"public_addr":"172.21.15.204:6801/1499577897","cluster_addr":"172.21.15.204:6803/1499577897","heartbeat_back_addr":"172.21.15.204:6807/1499577897","heartbeat_front_addr":"172.21.15.204:6805/1499577897","state":["exists","up"]},{"osd":3,"uuid":"d2a5fb97-6f60-4d45-9f18-447cae480974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6811","nonce":2869340648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6813","nonce":2869340648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6817","nonce":2869340648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6815","nonce":2869340648}]},"public_addr":"172.21.15.84:6811/2869340648","cluster_addr":"172.21.15.84:6813/2869340648","heartbeat_back_addr":"172.21.15.84:6817/2869340648","heartbeat_front_addr":"172.21.15.84:6815/2869340648","state":["exists","up"]},{"osd":4,"uuid":"4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6809","nonce":2554265648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6811","nonce":2554265648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6815","nonce":2554265648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6813","nonce":2554265648}]},"public_addr":"172.21.15.96:6809/2554265648","cluster_addr":"172.21.15.96:6811/2554265648","heartbeat_back_addr":"172.21.15.96:6815/2554265648","heartbeat_front_addr":"172.21.15.96:6813/2554265648","state":["exists","up"]},{"osd":5,"uuid":"7996bf5e-93a5-41b0-a9b1-99e7ef4514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6809","nonce":4075476986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6811","nonce":4075476986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6815","nonce":4075476986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6813","nonce":4075476986}]},"public_addr":"172.21.15.204:6809/4075476986","cluster_addr":"172.21.15.204:6811/4075476986","heartbeat_back_addr":"172.21.15.204:6815/4075476986","heartbeat_front_addr":"172.21.15.204:6813/4075476986","state":["exists","up"]},{"osd":6,"uuid":"f4b7c5de-1836-4d16-8603-3c517524d9fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6819","nonce":1806202865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6821","nonce":1806202865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6825","nonce":1806202865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6823","nonce":1806202865}]},"public_addr":"172.21.15.84:6819/1806202865","cluster_addr":"172.21.15.84:6821/1806202865","heartbeat_back_addr":"172.21.15.84:6825/1806202865","heartbeat_front_addr":"172.21.15.84:6823/1806202865","state":["exists","up"]},{"osd":7,"uuid":"69a18c43-5447-480d-9466-190830070c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6817","nonce":3195792774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6819","nonce":3195792774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6823","nonce":3195792774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6821","nonce":3195792774}]},"public_addr":"172.21.15.96:6817/3195792774","cluster_addr":"172.21.15.96:6819/3195792774","heartbeat_back_addr":"172.21.15.96:6823/3195792774","heartbeat_front_addr":"172.21.15.96:6821/3195792774","state":["exists","up"]},{"osd":8,"uuid":"2c5cadca-689a-441e-847c-ba370d400ef1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6817","nonce":606997927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6819","nonce":606997927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6823","nonce":606997927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6821","nonce":606997927}]},"public_addr":"172.21.15.204:6817/606997927","cluster_addr":"172.21.15.204:6819/606997927","heartbeat_back_addr":"172.21.15.204:6823/606997927","heartbeat_front_addr":"172.21.15.204:6821/606997927","state":["exists","up"]},{"osd":9,"uuid":"e3984fb2-7c00-4275-a10e-bca7c859f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6827","nonce":2786189012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6829","nonce":2786189012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6833","nonce":2786189012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6831","nonce":2786189012}]},"public_addr":"172.21.15.84:6827/2786189012","cluster_addr":"172.21.15.84:6829/2786189012","heartbeat_back_addr":"172.21.15.84:6833/2786189012","heartbeat_front_addr":"172.21.15.84:6831/2786189012","state":["exists","up"]},{"osd":10,"uuid":"b4938160-a808-4ded-91c2-d245c99fff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6825","nonce":1985398526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6827","nonce":1985398526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6831","nonce":1985398526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6829","nonce":1985398526}]},"public_addr":"172.21.15.96:6825/1985398526","cluster_addr":"172.21.15.96:6827/1985398526","heartbeat_back_addr":"172.21.15.96:6831/1985398526","heartbeat_front_addr":"172.21.15.96:6829/1985398526","state":["exists","up"]},{"osd":11,"uuid":"238922ef-d046-40cc-833d-25b7e7a523cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6825","nonce":3106116519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6827","nonce":3106116519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6831","nonce":3106116519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6829","nonce":3106116519}]},"public_addr":"172.21.15.204:6825/3106116519","cluster_addr":"172.21.15.204:6827/3106116519","heartbeat_back_addr":"172.21.15.204:6831/3106116519","heartbeat_front_addr":"172.21.15.204:6829/3106116519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:16.443480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:39.898133+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:03.664544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:28.838687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:53.829733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:19.269307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:45.166910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:10.771614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:36.678516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:03.150592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:29.327769+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:56.315938+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6801/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:6800/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/2766288608":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/950455999":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/4000289608":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1650467363":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1052746114":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6800/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6801/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/14610813":"2024-05-23T22:52:45.395791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T23:01:47.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.830+0000 7fa8f08d4640 1 -- 172.21.15.84:0/1986310027 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa8d40a2be0 msgr2=0x7fa8d40a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-22T23:01:47.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.830+0000 7fa8f08d4640 1 --2- 172.21.15.84:0/1986310027 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa8d40a2be0 0x7fa8d40a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:47.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.830+0000 7fa8f08d4640 1 -- 172.21.15.84:0/1986310027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8ec102360 msgr2=0x7fa8ec109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:47.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.830+0000 7fa8f08d4640 1 --2- 172.21.15.84:0/1986310027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8ec102360 0x7fa8ec109cb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009fd0 tx=0x7fa8e400ef10 comp rx=0 tx=0).stop 2024-05-22T23:01:47.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.830+0000 7fa8f08d4640 1 -- 172.21.15.84:0/1986310027 shutdown_connections 2024-05-22T23:01:47.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.830+0000 7fa8f08d4640 1 --2- 172.21.15.84:0/1986310027 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fa8d40a2be0 0x7fa8d40a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:47.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.830+0000 7fa8f08d4640 1 --2- 172.21.15.84:0/1986310027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8ec102360 0x7fa8ec109cb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:47.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.830+0000 7fa8f08d4640 1 --2- 172.21.15.84:0/1986310027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8ec1019a0 0x7fa8ec109770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:47.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.830+0000 7fa8f08d4640 1 --2- 172.21.15.84:0/1986310027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8ec1007b0 0x7fa8ec10b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:47.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.830+0000 7fa8f08d4640 1 -- 172.21.15.84:0/1986310027 >> 172.21.15.84:0/1986310027 conn(0x7fa8ec0fbd20 msgr2=0x7fa8ec104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:47.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.830+0000 7fa8f08d4640 1 -- 172.21.15.84:0/1986310027 shutdown_connections 2024-05-22T23:01:47.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.830+0000 7fa8f08d4640 1 -- 172.21.15.84:0/1986310027 wait complete. 2024-05-22T23:01:47.844 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T22:55:07.719394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T22:59:50.150511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T22:59:51.111182+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T23:01:47.844 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T23:01:47.882 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:47 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:47.588+0000 7f50b97ca200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-22T23:01:47.882 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:47 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:47.682+0000 7f50b97ca200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-22T23:01:47.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:47 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1986310027' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:47.911 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:47 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:47.654+0000 7f4715c6d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-22T23:01:47.911 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:47 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:47.748+0000 7f4715c6d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-22T23:01:47.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.949+0000 7f8bba8a5640 1 -- 172.21.15.84:0/2488382125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bb41007b0 msgr2=0x7f8bb4100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:47.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.949+0000 7f8bba8a5640 1 --2- 172.21.15.84:0/2488382125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bb41007b0 0x7f8bb4100bd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4009900 tx=0x7f8ba402a410 comp rx=0 tx=0).stop 2024-05-22T23:01:47.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.949+0000 7f8bba8a5640 1 -- 172.21.15.84:0/2488382125 shutdown_connections 2024-05-22T23:01:47.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.949+0000 7f8bba8a5640 1 --2- 172.21.15.84:0/2488382125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bb4102360 0x7f8bb4108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:47.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.949+0000 7f8bba8a5640 1 --2- 172.21.15.84:0/2488382125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bb41019a0 0x7f8bb4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:47.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.949+0000 7f8bba8a5640 1 --2- 172.21.15.84:0/2488382125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bb41007b0 0x7f8bb4100bd0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:47.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.949+0000 7f8bba8a5640 1 -- 172.21.15.84:0/2488382125 >> 172.21.15.84:0/2488382125 conn(0x7f8bb40fbd20 msgr2=0x7f8bb40fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:47.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.949+0000 7f8bba8a5640 1 -- 172.21.15.84:0/2488382125 shutdown_connections 2024-05-22T23:01:47.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.949+0000 7f8bba8a5640 1 -- 172.21.15.84:0/2488382125 wait complete. 2024-05-22T23:01:47.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.950+0000 7f8bba8a5640 1 Processor -- start 2024-05-22T23:01:47.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.950+0000 7f8bba8a5640 1 -- start start 2024-05-22T23:01:47.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.950+0000 7f8bba8a5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bb41007b0 0x7f8bb4197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:47.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.950+0000 7f8bba8a5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bb41019a0 0x7f8bb4198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:47.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.950+0000 7f8bba8a5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bb4102360 0x7f8bb419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:47.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.950+0000 7f8bba8a5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb410c430 con 0x7f8bb4102360 2024-05-22T23:01:47.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.950+0000 7f8bba8a5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb410c2b0 con 0x7f8bb41007b0 2024-05-22T23:01:47.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.950+0000 7f8bba8a5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb410c5b0 con 0x7f8bb41019a0 2024-05-22T23:01:47.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.950+0000 7f8bb3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bb41007b0 0x7f8bb4197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:47.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.950+0000 7f8bb8e42640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bb4102360 0x7f8bb419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:47.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.950+0000 7f8bb3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bb41007b0 0x7f8bb4197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:53494/0 (socket says 172.21.15.84:53494) 2024-05-22T23:01:47.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.950+0000 7f8babfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bb41019a0 0x7f8bb4198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:47.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.950+0000 7f8bb3fff640 1 -- 172.21.15.84:0/1704620564 learned_addr learned my addr 172.21.15.84:0/1704620564 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:47.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.950+0000 7f8bb8e42640 1 -- 172.21.15.84:0/1704620564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bb41019a0 msgr2=0x7f8bb4198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:47.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.950+0000 7f8bb8e42640 1 --2- 172.21.15.84:0/1704620564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bb41019a0 0x7f8bb4198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:47.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.950+0000 7f8bb8e42640 1 -- 172.21.15.84:0/1704620564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bb41007b0 msgr2=0x7f8bb4197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:47.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.950+0000 7f8bb8e42640 1 --2- 172.21.15.84:0/1704620564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bb41007b0 0x7f8bb4197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:47.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.950+0000 7f8bb8e42640 1 -- 172.21.15.84:0/1704620564 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb419f830 con 0x7f8bb4102360 2024-05-22T23:01:47.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.950+0000 7f8bb3fff640 1 --2- 172.21.15.84:0/1704620564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bb41007b0 0x7f8bb4197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:01:47.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.951+0000 7f8bb8e42640 1 --2- 172.21.15.84:0/1704620564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bb4102360 0x7f8bb419f190 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009fd0 tx=0x7f8bac00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:47.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.951+0000 7f8bb1ffb640 1 -- 172.21.15.84:0/1704620564 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac00ce20 con 0x7f8bb4102360 2024-05-22T23:01:47.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.951+0000 7f8bba8a5640 1 -- 172.21.15.84:0/1704620564 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb419fb20 con 0x7f8bb4102360 2024-05-22T23:01:47.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.951+0000 7f8bb1ffb640 1 -- 172.21.15.84:0/1704620564 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bac005020 con 0x7f8bb4102360 2024-05-22T23:01:47.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.951+0000 7f8bb1ffb640 1 -- 172.21.15.84:0/1704620564 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac010720 con 0x7f8bb4102360 2024-05-22T23:01:47.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.951+0000 7f8bba8a5640 1 -- 172.21.15.84:0/1704620564 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb41a0070 con 0x7f8bb4102360 2024-05-22T23:01:47.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.952+0000 7f8bba8a5640 1 -- 172.21.15.84:0/1704620564 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bb41a0240 con 0x7f8bb4102360 2024-05-22T23:01:47.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.953+0000 7f8bb1ffb640 1 -- 172.21.15.84:0/1704620564 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f8bac005540 con 0x7f8bb4102360 2024-05-22T23:01:47.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.953+0000 7f8bb1ffb640 1 --2- 172.21.15.84:0/1704620564 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8ba00a2be0 0x7f8ba00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:47.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.953+0000 7f8bb3fff640 1 -- 172.21.15.84:0/1704620564 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8ba00a2be0 msgr2=0x7f8ba00a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/4223589140 2024-05-22T23:01:47.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.954+0000 7f8bb1ffb640 1 -- 172.21.15.84:0/1704620564 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f8bac0cf880 con 0x7f8bb4102360 2024-05-22T23:01:47.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.954+0000 7f8bb3fff640 1 --2- 172.21.15.84:0/1704620564 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8ba00a2be0 0x7f8ba00a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-22T23:01:47.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:47.957+0000 7f8bb1ffb640 1 -- 172.21.15.84:0/1704620564 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8bac09c760 con 0x7f8bb4102360 2024-05-22T23:01:48.125 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:47 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:47.928+0000 7f447aae9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-22T23:01:48.125 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:48 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:48.028+0000 7f447aae9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-22T23:01:48.126 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:47 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1986310027' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:48.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.137+0000 7f8bba8a5640 1 -- 172.21.15.84:0/1704620564 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8bb41a0410 con 0x7f8bb4102360 2024-05-22T23:01:48.139 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.137+0000 7f8bb1ffb640 1 -- 172.21.15.84:0/1704620564 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f8bac09c060 con 0x7f8bb4102360 2024-05-22T23:01:48.139 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2024-05-22T23:01:48.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.139+0000 7f8bba8a5640 1 -- 172.21.15.84:0/1704620564 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8ba00a2be0 msgr2=0x7f8ba00a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-22T23:01:48.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.139+0000 7f8bba8a5640 1 --2- 172.21.15.84:0/1704620564 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8ba00a2be0 0x7f8ba00a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.139+0000 7f8bba8a5640 1 -- 172.21.15.84:0/1704620564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bb4102360 msgr2=0x7f8bb419f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:48.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.139+0000 7f8bba8a5640 1 --2- 172.21.15.84:0/1704620564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bb4102360 0x7f8bb419f190 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009fd0 tx=0x7f8bac00ef10 comp rx=0 tx=0).stop 2024-05-22T23:01:48.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.139+0000 7f8bba8a5640 1 -- 172.21.15.84:0/1704620564 shutdown_connections 2024-05-22T23:01:48.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.139+0000 7f8bba8a5640 1 --2- 172.21.15.84:0/1704620564 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f8ba00a2be0 0x7f8ba00a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.139+0000 7f8bba8a5640 1 --2- 172.21.15.84:0/1704620564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bb4102360 0x7f8bb419f190 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.139+0000 7f8bba8a5640 1 --2- 172.21.15.84:0/1704620564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bb41019a0 0x7f8bb4198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.139+0000 7f8bba8a5640 1 --2- 172.21.15.84:0/1704620564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bb41007b0 0x7f8bb4197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.139+0000 7f8bba8a5640 1 -- 172.21.15.84:0/1704620564 >> 172.21.15.84:0/1704620564 conn(0x7f8bb40fbd20 msgr2=0x7f8bb4068a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:48.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.139+0000 7f8bba8a5640 1 -- 172.21.15.84:0/1704620564 shutdown_connections 2024-05-22T23:01:48.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.139+0000 7f8bba8a5640 1 -- 172.21.15.84:0/1704620564 wait complete. 2024-05-22T23:01:48.152 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T23:01:48.157 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:47 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1986310027' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:48.157 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:47 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:47.926+0000 7f50b97ca200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-22T23:01:48.157 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:48 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:48.061+0000 7f50b97ca200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-22T23:01:48.259 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.257+0000 7f342033b640 1 -- 172.21.15.84:0/459918134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34181007b0 msgr2=0x7f3418100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:48.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.257+0000 7f342033b640 1 --2- 172.21.15.84:0/459918134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34181007b0 0x7f3418100bd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3414009900 tx=0x7f341402a410 comp rx=0 tx=0).stop 2024-05-22T23:01:48.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.257+0000 7f342033b640 1 -- 172.21.15.84:0/459918134 shutdown_connections 2024-05-22T23:01:48.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.257+0000 7f342033b640 1 --2- 172.21.15.84:0/459918134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3418102360 0x7f3418108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.257+0000 7f342033b640 1 --2- 172.21.15.84:0/459918134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34181019a0 0x7f3418101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.257+0000 7f342033b640 1 --2- 172.21.15.84:0/459918134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34181007b0 0x7f3418100bd0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.257+0000 7f342033b640 1 -- 172.21.15.84:0/459918134 >> 172.21.15.84:0/459918134 conn(0x7f34180fbd20 msgr2=0x7f34180fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:48.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.257+0000 7f342033b640 1 -- 172.21.15.84:0/459918134 shutdown_connections 2024-05-22T23:01:48.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.257+0000 7f342033b640 1 -- 172.21.15.84:0/459918134 wait complete. 2024-05-22T23:01:48.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.258+0000 7f342033b640 1 Processor -- start 2024-05-22T23:01:48.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.258+0000 7f342033b640 1 -- start start 2024-05-22T23:01:48.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.258+0000 7f342033b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34181007b0 0x7f3418069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:48.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.258+0000 7f342033b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34181019a0 0x7f341819c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:48.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.258+0000 7f342033b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3418102360 0x7f341819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:48.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.258+0000 7f342033b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f341810c420 con 0x7f34181007b0 2024-05-22T23:01:48.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.258+0000 7f342033b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f341810c2a0 con 0x7f3418102360 2024-05-22T23:01:48.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.258+0000 7f342033b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f341810c5a0 con 0x7f34181019a0 2024-05-22T23:01:48.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.258+0000 7f341e0d7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34181007b0 0x7f3418069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:48.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.258+0000 7f341e0d7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34181007b0 0x7f3418069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38064/0 (socket says 172.21.15.84:38064) 2024-05-22T23:01:48.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.258+0000 7f341e0d7640 1 -- 172.21.15.84:0/672587213 learned_addr learned my addr 172.21.15.84:0/672587213 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:48.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.258+0000 7f341d8d6640 1 --2- 172.21.15.84:0/672587213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34181019a0 0x7f341819c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:48.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.258+0000 7f341e8d8640 1 --2- 172.21.15.84:0/672587213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3418102360 0x7f341819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:48.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.259+0000 7f341e0d7640 1 -- 172.21.15.84:0/672587213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34181019a0 msgr2=0x7f341819c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:48.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.259+0000 7f341e0d7640 1 --2- 172.21.15.84:0/672587213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34181019a0 0x7f341819c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.259+0000 7f341e0d7640 1 -- 172.21.15.84:0/672587213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3418102360 msgr2=0x7f341819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:48.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.259+0000 7f341e0d7640 1 --2- 172.21.15.84:0/672587213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3418102360 0x7f341819eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.259+0000 7f341e0d7640 1 -- 172.21.15.84:0/672587213 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f341819f8b0 con 0x7f34181007b0 2024-05-22T23:01:48.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.259+0000 7f341e0d7640 1 --2- 172.21.15.84:0/672587213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34181007b0 0x7f3418069590 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3414031610 tx=0x7f3414004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:48.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.259+0000 7f340f7fe640 1 -- 172.21.15.84:0/672587213 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341403f030 con 0x7f34181007b0 2024-05-22T23:01:48.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.259+0000 7f340f7fe640 1 -- 172.21.15.84:0/672587213 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3414003ad0 con 0x7f34181007b0 2024-05-22T23:01:48.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.259+0000 7f340f7fe640 1 -- 172.21.15.84:0/672587213 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3414035070 con 0x7f34181007b0 2024-05-22T23:01:48.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.259+0000 7f342033b640 1 -- 172.21.15.84:0/672587213 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f341819fb40 con 0x7f34181007b0 2024-05-22T23:01:48.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.259+0000 7f342033b640 1 -- 172.21.15.84:0/672587213 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34181a0030 con 0x7f34181007b0 2024-05-22T23:01:48.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.260+0000 7f342033b640 1 -- 172.21.15.84:0/672587213 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3418107bf0 con 0x7f34181007b0 2024-05-22T23:01:48.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.261+0000 7f340f7fe640 1 -- 172.21.15.84:0/672587213 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f3414042030 con 0x7f34181007b0 2024-05-22T23:01:48.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.262+0000 7f340f7fe640 1 --2- 172.21.15.84:0/672587213 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f34040a2be0 0x7f34040a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:48.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.262+0000 7f341d8d6640 1 -- 172.21.15.84:0/672587213 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f34040a2be0 msgr2=0x7f34040a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/4223589140 2024-05-22T23:01:48.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.262+0000 7f340f7fe640 1 -- 172.21.15.84:0/672587213 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f34140ef780 con 0x7f34181007b0 2024-05-22T23:01:48.267 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.262+0000 7f341d8d6640 1 --2- 172.21.15.84:0/672587213 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f34040a2be0 0x7f34040a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-22T23:01:48.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.266+0000 7f340f7fe640 1 -- 172.21.15.84:0/672587213 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f34140bc010 con 0x7f34181007b0 2024-05-22T23:01:48.347 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:47 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:47.995+0000 7f4715c6d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-22T23:01:48.347 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:48 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:48.130+0000 7f4715c6d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-22T23:01:48.347 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:48 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:48.227+0000 7f4715c6d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-22T23:01:48.347 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:48 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:48.345+0000 7f4715c6d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-22T23:01:48.435 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:48 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:48.291+0000 7f447aae9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-22T23:01:48.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.448+0000 7f342033b640 1 -- 172.21.15.84:0/672587213 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3418062470 con 0x7f34181007b0 2024-05-22T23:01:48.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.449+0000 7f340f7fe640 1 -- 172.21.15.84:0/672587213 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f34140c0310 con 0x7f34181007b0 2024-05-22T23:01:48.451 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2024-05-22T23:01:48.453 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.451+0000 7f342033b640 1 -- 172.21.15.84:0/672587213 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f34040a2be0 msgr2=0x7f34040a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-22T23:01:48.453 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.451+0000 7f342033b640 1 --2- 172.21.15.84:0/672587213 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f34040a2be0 0x7f34040a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.453 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.451+0000 7f342033b640 1 -- 172.21.15.84:0/672587213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34181007b0 msgr2=0x7f3418069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:48.453 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.451+0000 7f342033b640 1 --2- 172.21.15.84:0/672587213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34181007b0 0x7f3418069590 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3414031610 tx=0x7f3414004270 comp rx=0 tx=0).stop 2024-05-22T23:01:48.453 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.451+0000 7f342033b640 1 -- 172.21.15.84:0/672587213 shutdown_connections 2024-05-22T23:01:48.453 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.451+0000 7f342033b640 1 --2- 172.21.15.84:0/672587213 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f34040a2be0 0x7f34040a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.453 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.451+0000 7f342033b640 1 --2- 172.21.15.84:0/672587213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3418102360 0x7f341819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.453 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.451+0000 7f342033b640 1 --2- 172.21.15.84:0/672587213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34181019a0 0x7f341819c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.451+0000 7f342033b640 1 --2- 172.21.15.84:0/672587213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34181007b0 0x7f3418069590 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.451+0000 7f342033b640 1 -- 172.21.15.84:0/672587213 >> 172.21.15.84:0/672587213 conn(0x7f34180fbd20 msgr2=0x7f3418104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:48.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.451+0000 7f342033b640 1 -- 172.21.15.84:0/672587213 shutdown_connections 2024-05-22T23:01:48.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.451+0000 7f342033b640 1 -- 172.21.15.84:0/672587213 wait complete. 2024-05-22T23:01:48.464 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T23:01:48.550 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:48 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:48.155+0000 7f50b97ca200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-22T23:01:48.550 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:48 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:48.272+0000 7f50b97ca200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-22T23:01:48.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.569+0000 7f7a3a9b0640 1 -- 172.21.15.84:0/3398266026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a34102330 msgr2=0x7f7a34104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:48.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.569+0000 7f7a3a9b0640 1 --2- 172.21.15.84:0/3398266026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a34102330 0x7f7a34104720 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7a24009900 tx=0x7f7a2402a410 comp rx=0 tx=0).stop 2024-05-22T23:01:48.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.569+0000 7f7a3a9b0640 1 -- 172.21.15.84:0/3398266026 shutdown_connections 2024-05-22T23:01:48.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.569+0000 7f7a3a9b0640 1 --2- 172.21.15.84:0/3398266026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a34107590 0x7f7a341099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.569+0000 7f7a3a9b0640 1 --2- 172.21.15.84:0/3398266026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a34104c60 0x7f7a34107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.569+0000 7f7a3a9b0640 1 --2- 172.21.15.84:0/3398266026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a34102330 0x7f7a34104720 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.569+0000 7f7a3a9b0640 1 -- 172.21.15.84:0/3398266026 >> 172.21.15.84:0/3398266026 conn(0x7f7a340fbd20 msgr2=0x7f7a340fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:48.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.569+0000 7f7a3a9b0640 1 -- 172.21.15.84:0/3398266026 shutdown_connections 2024-05-22T23:01:48.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.569+0000 7f7a3a9b0640 1 -- 172.21.15.84:0/3398266026 wait complete. 2024-05-22T23:01:48.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.569+0000 7f7a3a9b0640 1 Processor -- start 2024-05-22T23:01:48.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.569+0000 7f7a3a9b0640 1 -- start start 2024-05-22T23:01:48.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.570+0000 7f7a3a9b0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a34102330 0x7f7a34101ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:48.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.570+0000 7f7a3a9b0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a34104c60 0x7f7a341002e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:48.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.570+0000 7f7a3a9b0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a34107590 0x7f7a34100820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:48.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.570+0000 7f7a3a9b0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3410d560 con 0x7f7a34102330 2024-05-22T23:01:48.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.570+0000 7f7a3a9b0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3410d3e0 con 0x7f7a34107590 2024-05-22T23:01:48.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.570+0000 7f7a3a9b0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3410d6e0 con 0x7f7a34104c60 2024-05-22T23:01:48.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.570+0000 7f7a33fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a34102330 0x7f7a34101ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:48.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.570+0000 7f7a33fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a34102330 0x7f7a34101ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38074/0 (socket says 172.21.15.84:38074) 2024-05-22T23:01:48.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.570+0000 7f7a33fff640 1 -- 172.21.15.84:0/982997484 learned_addr learned my addr 172.21.15.84:0/982997484 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:48.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.570+0000 7f7a38f4d640 1 --2- 172.21.15.84:0/982997484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a34107590 0x7f7a34100820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:48.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.570+0000 7f7a337fe640 1 --2- 172.21.15.84:0/982997484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a34104c60 0x7f7a341002e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:48.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.570+0000 7f7a33fff640 1 -- 172.21.15.84:0/982997484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a34104c60 msgr2=0x7f7a341002e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:48.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.570+0000 7f7a33fff640 1 --2- 172.21.15.84:0/982997484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a34104c60 0x7f7a341002e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.570+0000 7f7a33fff640 1 -- 172.21.15.84:0/982997484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a34107590 msgr2=0x7f7a34100820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:48.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.570+0000 7f7a33fff640 1 --2- 172.21.15.84:0/982997484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a34107590 0x7f7a34100820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.570+0000 7f7a33fff640 1 -- 172.21.15.84:0/982997484 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a341010b0 con 0x7f7a34102330 2024-05-22T23:01:48.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.570+0000 7f7a33fff640 1 --2- 172.21.15.84:0/982997484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a34102330 0x7f7a34101ca0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7a24009230 tx=0x7f7a240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:48.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.570+0000 7f7a317fa640 1 -- 172.21.15.84:0/982997484 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2403f030 con 0x7f7a34102330 2024-05-22T23:01:48.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.570+0000 7f7a3a9b0640 1 -- 172.21.15.84:0/982997484 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a341a1cf0 con 0x7f7a34102330 2024-05-22T23:01:48.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.570+0000 7f7a317fa640 1 -- 172.21.15.84:0/982997484 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a24003ce0 con 0x7f7a34102330 2024-05-22T23:01:48.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.570+0000 7f7a317fa640 1 -- 172.21.15.84:0/982997484 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a24035070 con 0x7f7a34102330 2024-05-22T23:01:48.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.570+0000 7f7a3a9b0640 1 -- 172.21.15.84:0/982997484 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a341a2180 con 0x7f7a34102330 2024-05-22T23:01:48.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.572+0000 7f7a3a9b0640 1 -- 172.21.15.84:0/982997484 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a340fd810 con 0x7f7a34102330 2024-05-22T23:01:48.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.572+0000 7f7a317fa640 1 -- 172.21.15.84:0/982997484 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f7a24042030 con 0x7f7a34102330 2024-05-22T23:01:48.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.573+0000 7f7a317fa640 1 --2- 172.21.15.84:0/982997484 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f7a1c0a2be0 0x7f7a1c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:48.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.573+0000 7f7a337fe640 1 -- 172.21.15.84:0/982997484 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f7a1c0a2be0 msgr2=0x7f7a1c0a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/4223589140 2024-05-22T23:01:48.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.573+0000 7f7a317fa640 1 -- 172.21.15.84:0/982997484 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f7a240ef590 con 0x7f7a34102330 2024-05-22T23:01:48.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.573+0000 7f7a337fe640 1 --2- 172.21.15.84:0/982997484 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f7a1c0a2be0 0x7f7a1c0a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-22T23:01:48.579 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.577+0000 7f7a317fa640 1 -- 172.21.15.84:0/982997484 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7a240bbe80 con 0x7f7a34102330 2024-05-22T23:01:48.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.760+0000 7f7a3a9b0640 1 -- 172.21.15.84:0/982997484 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7a34062470 con 0x7f7a34102330 2024-05-22T23:01:48.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.760+0000 7f7a317fa640 1 -- 172.21.15.84:0/982997484 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f7a240c0180 con 0x7f7a34102330 2024-05-22T23:01:48.762 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2024-05-22T23:01:48.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.763+0000 7f7a3a9b0640 1 -- 172.21.15.84:0/982997484 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f7a1c0a2be0 msgr2=0x7f7a1c0a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-22T23:01:48.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.763+0000 7f7a3a9b0640 1 --2- 172.21.15.84:0/982997484 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f7a1c0a2be0 0x7f7a1c0a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.763+0000 7f7a3a9b0640 1 -- 172.21.15.84:0/982997484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a34102330 msgr2=0x7f7a34101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:48.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.763+0000 7f7a3a9b0640 1 --2- 172.21.15.84:0/982997484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a34102330 0x7f7a34101ca0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7a24009230 tx=0x7f7a240042f0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.763+0000 7f7a3a9b0640 1 -- 172.21.15.84:0/982997484 shutdown_connections 2024-05-22T23:01:48.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.763+0000 7f7a3a9b0640 1 --2- 172.21.15.84:0/982997484 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f7a1c0a2be0 0x7f7a1c0a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.763+0000 7f7a3a9b0640 1 --2- 172.21.15.84:0/982997484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a34107590 0x7f7a34100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.763+0000 7f7a3a9b0640 1 --2- 172.21.15.84:0/982997484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a34104c60 0x7f7a341002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.763+0000 7f7a3a9b0640 1 --2- 172.21.15.84:0/982997484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a34102330 0x7f7a34101ca0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.763+0000 7f7a3a9b0640 1 -- 172.21.15.84:0/982997484 >> 172.21.15.84:0/982997484 conn(0x7f7a340fbd20 msgr2=0x7f7a34102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:48.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.763+0000 7f7a3a9b0640 1 -- 172.21.15.84:0/982997484 shutdown_connections 2024-05-22T23:01:48.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.763+0000 7f7a3a9b0640 1 -- 172.21.15.84:0/982997484 wait complete. 2024-05-22T23:01:48.776 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:48 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:48.433+0000 7f447aae9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-22T23:01:48.776 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:48 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:48.540+0000 7f447aae9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-22T23:01:48.776 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:48 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:48.667+0000 7f447aae9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-22T23:01:48.777 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T23:01:48.883 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:48 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:48.569+0000 7f50b97ca200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-22T23:01:48.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.887+0000 7fc7d1cdd640 1 -- 172.21.15.84:0/771385535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7cc103490 msgr2=0x7fc7cc109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:48.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.887+0000 7fc7d1cdd640 1 --2- 172.21.15.84:0/771385535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7cc103490 0x7fc7cc109d20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009ca0 tx=0x7fc7c402a8c0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.887+0000 7fc7d1cdd640 1 -- 172.21.15.84:0/771385535 shutdown_connections 2024-05-22T23:01:48.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.887+0000 7fc7d1cdd640 1 --2- 172.21.15.84:0/771385535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7cc103490 0x7fc7cc109d20 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.887+0000 7fc7d1cdd640 1 --2- 172.21.15.84:0/771385535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7cc102ad0 0x7fc7cc102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.887+0000 7fc7d1cdd640 1 --2- 172.21.15.84:0/771385535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7cc1018e0 0x7fc7cc101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.887+0000 7fc7d1cdd640 1 -- 172.21.15.84:0/771385535 >> 172.21.15.84:0/771385535 conn(0x7fc7cc0fce70 msgr2=0x7fc7cc0ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:48.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.887+0000 7fc7d1cdd640 1 -- 172.21.15.84:0/771385535 shutdown_connections 2024-05-22T23:01:48.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.887+0000 7fc7d1cdd640 1 -- 172.21.15.84:0/771385535 wait complete. 2024-05-22T23:01:48.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.887+0000 7fc7d1cdd640 1 Processor -- start 2024-05-22T23:01:48.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.887+0000 7fc7d1cdd640 1 -- start start 2024-05-22T23:01:48.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.887+0000 7fc7d1cdd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7cc1018e0 0x7fc7cc197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:48.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.887+0000 7fc7d1cdd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7cc102ad0 0x7fc7cc198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:48.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.887+0000 7fc7d1cdd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7cc103490 0x7fc7cc19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:48.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.887+0000 7fc7d1cdd640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc10d5b0 con 0x7fc7cc102ad0 2024-05-22T23:01:48.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.887+0000 7fc7d1cdd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc10d430 con 0x7fc7cc103490 2024-05-22T23:01:48.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.887+0000 7fc7d1cdd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc10d730 con 0x7fc7cc1018e0 2024-05-22T23:01:48.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.888+0000 7fc7c3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7cc102ad0 0x7fc7cc198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:48.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.888+0000 7fc7c3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7cc102ad0 0x7fc7cc198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38102/0 (socket says 172.21.15.84:38102) 2024-05-22T23:01:48.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.888+0000 7fc7c3fff640 1 -- 172.21.15.84:0/1953813225 learned_addr learned my addr 172.21.15.84:0/1953813225 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:48.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.888+0000 7fc7cb7fe640 1 --2- 172.21.15.84:0/1953813225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7cc1018e0 0x7fc7cc197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:48.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.888+0000 7fc7cbfff640 1 --2- 172.21.15.84:0/1953813225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7cc103490 0x7fc7cc19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:48.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.888+0000 7fc7c3fff640 1 -- 172.21.15.84:0/1953813225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7cc1018e0 msgr2=0x7fc7cc197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:48.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.888+0000 7fc7c3fff640 1 --2- 172.21.15.84:0/1953813225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7cc1018e0 0x7fc7cc197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.888+0000 7fc7c3fff640 1 -- 172.21.15.84:0/1953813225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7cc103490 msgr2=0x7fc7cc19f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:48.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.888+0000 7fc7c3fff640 1 --2- 172.21.15.84:0/1953813225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7cc103490 0x7fc7cc19f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:48.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.888+0000 7fc7c3fff640 1 -- 172.21.15.84:0/1953813225 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7cc19f860 con 0x7fc7cc102ad0 2024-05-22T23:01:48.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.888+0000 7fc7c3fff640 1 --2- 172.21.15.84:0/1953813225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7cc102ad0 0x7fc7cc198190 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc7b400ea60 tx=0x7fc7b400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:48.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.888+0000 7fc7c97fa640 1 -- 172.21.15.84:0/1953813225 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b400cde0 con 0x7fc7cc102ad0 2024-05-22T23:01:48.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.888+0000 7fc7d1cdd640 1 -- 172.21.15.84:0/1953813225 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7cc19fb50 con 0x7fc7cc102ad0 2024-05-22T23:01:48.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.888+0000 7fc7c97fa640 1 -- 172.21.15.84:0/1953813225 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7b40035f0 con 0x7fc7cc102ad0 2024-05-22T23:01:48.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.888+0000 7fc7c97fa640 1 -- 172.21.15.84:0/1953813225 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4010670 con 0x7fc7cc102ad0 2024-05-22T23:01:48.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.888+0000 7fc7d1cdd640 1 -- 172.21.15.84:0/1953813225 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7cc1a00a0 con 0x7fc7cc102ad0 2024-05-22T23:01:48.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.889+0000 7fc7d1cdd640 1 -- 172.21.15.84:0/1953813225 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7cc108b30 con 0x7fc7cc102ad0 2024-05-22T23:01:48.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.890+0000 7fc7c97fa640 1 -- 172.21.15.84:0/1953813225 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fc7b40037a0 con 0x7fc7cc102ad0 2024-05-22T23:01:48.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.891+0000 7fc7c97fa640 1 --2- 172.21.15.84:0/1953813225 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc7b80a2be0 0x7fc7b80a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:48.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.891+0000 7fc7cb7fe640 1 -- 172.21.15.84:0/1953813225 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc7b80a2be0 msgr2=0x7fc7b80a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/4223589140 2024-05-22T23:01:48.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.891+0000 7fc7c97fa640 1 -- 172.21.15.84:0/1953813225 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fc7b40cf3a0 con 0x7fc7cc102ad0 2024-05-22T23:01:48.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.891+0000 7fc7cb7fe640 1 --2- 172.21.15.84:0/1953813225 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc7b80a2be0 0x7fc7b80a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-22T23:01:48.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:48.896+0000 7fc7c97fa640 1 -- 172.21.15.84:0/1953813225 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc7b409c2e0 con 0x7fc7cc102ad0 2024-05-22T23:01:48.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:48 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1704620564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:01:48.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:48 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/672587213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:01:48.911 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:48 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/982997484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:01:48.911 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:48 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:48.648+0000 7f4715c6d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-22T23:01:49.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.080+0000 7fc7d1cdd640 1 -- 172.21.15.84:0/1953813225 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7cc02d420 con 0x7fc7cc102ad0 2024-05-22T23:01:49.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.081+0000 7fc7c97fa640 1 -- 172.21.15.84:0/1953813225 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22722 (secure 0 0 0) 0x7fc7b409bbe0 con 0x7fc7cc102ad0 2024-05-22T23:01:49.083 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:49.083 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":88,"fsid":"c1135a12-188d-11ef-bc9a-c7b262605968","created":"2024-05-22T22:51:49.878218+0000","modified":"2024-05-22T23:00:46.572591+0000","last_up_change":"2024-05-22T22:58:58.550278+0000","last_in_change":"2024-05-22T22:58:37.322139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T22:55:07.719394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T22:59:50.150511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T22:59:51.111182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762df065-fb4f-4c71-aab8-a62d2282c253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6803","nonce":1559945826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6805","nonce":1559945826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6809","nonce":1559945826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6807","nonce":1559945826}]},"public_addr":"172.21.15.84:6803/1559945826","cluster_addr":"172.21.15.84:6805/1559945826","heartbeat_back_addr":"172.21.15.84:6809/1559945826","heartbeat_front_addr":"172.21.15.84:6807/1559945826","state":["exists","up"]},{"osd":1,"uuid":"3fb2385d-6a34-49e3-88e1-116237beb346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6801","nonce":2996505353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6803","nonce":2996505353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6807","nonce":2996505353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6805","nonce":2996505353}]},"public_addr":"172.21.15.96:6801/2996505353","cluster_addr":"172.21.15.96:6803/2996505353","heartbeat_back_addr":"172.21.15.96:6807/2996505353","heartbeat_front_addr":"172.21.15.96:6805/2996505353","state":["exists","up"]},{"osd":2,"uuid":"e306e88e-a308-4585-ae11-c8af10b84038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6801","nonce":1499577897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6803","nonce":1499577897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6807","nonce":1499577897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6805","nonce":1499577897}]},"public_addr":"172.21.15.204:6801/1499577897","cluster_addr":"172.21.15.204:6803/1499577897","heartbeat_back_addr":"172.21.15.204:6807/1499577897","heartbeat_front_addr":"172.21.15.204:6805/1499577897","state":["exists","up"]},{"osd":3,"uuid":"d2a5fb97-6f60-4d45-9f18-447cae480974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6811","nonce":2869340648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6813","nonce":2869340648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6817","nonce":2869340648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6815","nonce":2869340648}]},"public_addr":"172.21.15.84:6811/2869340648","cluster_addr":"172.21.15.84:6813/2869340648","heartbeat_back_addr":"172.21.15.84:6817/2869340648","heartbeat_front_addr":"172.21.15.84:6815/2869340648","state":["exists","up"]},{"osd":4,"uuid":"4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6809","nonce":2554265648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6811","nonce":2554265648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6815","nonce":2554265648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6813","nonce":2554265648}]},"public_addr":"172.21.15.96:6809/2554265648","cluster_addr":"172.21.15.96:6811/2554265648","heartbeat_back_addr":"172.21.15.96:6815/2554265648","heartbeat_front_addr":"172.21.15.96:6813/2554265648","state":["exists","up"]},{"osd":5,"uuid":"7996bf5e-93a5-41b0-a9b1-99e7ef4514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6809","nonce":4075476986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6811","nonce":4075476986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6815","nonce":4075476986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6813","nonce":4075476986}]},"public_addr":"172.21.15.204:6809/4075476986","cluster_addr":"172.21.15.204:6811/4075476986","heartbeat_back_addr":"172.21.15.204:6815/4075476986","heartbeat_front_addr":"172.21.15.204:6813/4075476986","state":["exists","up"]},{"osd":6,"uuid":"f4b7c5de-1836-4d16-8603-3c517524d9fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6819","nonce":1806202865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6821","nonce":1806202865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6825","nonce":1806202865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6823","nonce":1806202865}]},"public_addr":"172.21.15.84:6819/1806202865","cluster_addr":"172.21.15.84:6821/1806202865","heartbeat_back_addr":"172.21.15.84:6825/1806202865","heartbeat_front_addr":"172.21.15.84:6823/1806202865","state":["exists","up"]},{"osd":7,"uuid":"69a18c43-5447-480d-9466-190830070c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6817","nonce":3195792774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6819","nonce":3195792774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6823","nonce":3195792774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6821","nonce":3195792774}]},"public_addr":"172.21.15.96:6817/3195792774","cluster_addr":"172.21.15.96:6819/3195792774","heartbeat_back_addr":"172.21.15.96:6823/3195792774","heartbeat_front_addr":"172.21.15.96:6821/3195792774","state":["exists","up"]},{"osd":8,"uuid":"2c5cadca-689a-441e-847c-ba370d400ef1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6817","nonce":606997927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6819","nonce":606997927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6823","nonce":606997927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6821","nonce":606997927}]},"public_addr":"172.21.15.204:6817/606997927","cluster_addr":"172.21.15.204:6819/606997927","heartbeat_back_addr":"172.21.15.204:6823/606997927","heartbeat_front_addr":"172.21.15.204:6821/606997927","state":["exists","up"]},{"osd":9,"uuid":"e3984fb2-7c00-4275-a10e-bca7c859f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6827","nonce":2786189012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6829","nonce":2786189012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6833","nonce":2786189012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6831","nonce":2786189012}]},"public_addr":"172.21.15.84:6827/2786189012","cluster_addr":"172.21.15.84:6829/2786189012","heartbeat_back_addr":"172.21.15.84:6833/2786189012","heartbeat_front_addr":"172.21.15.84:6831/2786189012","state":["exists","up"]},{"osd":10,"uuid":"b4938160-a808-4ded-91c2-d245c99fff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6825","nonce":1985398526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6827","nonce":1985398526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6831","nonce":1985398526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6829","nonce":1985398526}]},"public_addr":"172.21.15.96:6825/1985398526","cluster_addr":"172.21.15.96:6827/1985398526","heartbeat_back_addr":"172.21.15.96:6831/1985398526","heartbeat_front_addr":"172.21.15.96:6829/1985398526","state":["exists","up"]},{"osd":11,"uuid":"238922ef-d046-40cc-833d-25b7e7a523cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6825","nonce":3106116519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6827","nonce":3106116519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6831","nonce":3106116519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6829","nonce":3106116519}]},"public_addr":"172.21.15.204:6825/3106116519","cluster_addr":"172.21.15.204:6827/3106116519","heartbeat_back_addr":"172.21.15.204:6831/3106116519","heartbeat_front_addr":"172.21.15.204:6829/3106116519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:16.443480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:39.898133+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:03.664544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:28.838687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:53.829733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:19.269307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:45.166910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:10.771614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:36.678516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:03.150592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:29.327769+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:56.315938+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6801/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:6800/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/2766288608":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/950455999":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/4000289608":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1650467363":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1052746114":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6800/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6801/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/14610813":"2024-05-23T22:52:45.395791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T23:01:49.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.083+0000 7fc7d1cdd640 1 -- 172.21.15.84:0/1953813225 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc7b80a2be0 msgr2=0x7fc7b80a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-22T23:01:49.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.083+0000 7fc7d1cdd640 1 --2- 172.21.15.84:0/1953813225 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc7b80a2be0 0x7fc7b80a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.083+0000 7fc7d1cdd640 1 -- 172.21.15.84:0/1953813225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7cc102ad0 msgr2=0x7fc7cc198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:49.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.083+0000 7fc7d1cdd640 1 --2- 172.21.15.84:0/1953813225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7cc102ad0 0x7fc7cc198190 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc7b400ea60 tx=0x7fc7b400ef20 comp rx=0 tx=0).stop 2024-05-22T23:01:49.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.083+0000 7fc7d1cdd640 1 -- 172.21.15.84:0/1953813225 shutdown_connections 2024-05-22T23:01:49.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.083+0000 7fc7d1cdd640 1 --2- 172.21.15.84:0/1953813225 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7fc7b80a2be0 0x7fc7b80a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.083+0000 7fc7d1cdd640 1 --2- 172.21.15.84:0/1953813225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7cc103490 0x7fc7cc19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.083+0000 7fc7d1cdd640 1 --2- 172.21.15.84:0/1953813225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7cc102ad0 0x7fc7cc198190 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.083+0000 7fc7d1cdd640 1 --2- 172.21.15.84:0/1953813225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7cc1018e0 0x7fc7cc197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.083+0000 7fc7d1cdd640 1 -- 172.21.15.84:0/1953813225 >> 172.21.15.84:0/1953813225 conn(0x7fc7cc0fce70 msgr2=0x7fc7cc105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:49.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.083+0000 7fc7d1cdd640 1 -- 172.21.15.84:0/1953813225 shutdown_connections 2024-05-22T23:01:49.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.083+0000 7fc7d1cdd640 1 -- 172.21.15.84:0/1953813225 wait complete. 2024-05-22T23:01:49.095 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:48 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:48.983+0000 7f447aae9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-22T23:01:49.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:48 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1704620564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:01:49.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:48 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/672587213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:01:49.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:48 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/982997484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:01:49.097 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T22:55:07.719394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T22:59:50.150511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T22:59:51.111182+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T23:01:49.097 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T23:01:49.171 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:48 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1704620564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:01:49.171 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:48 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/672587213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:01:49.172 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:48 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/982997484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:01:49.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.199+0000 7f923b934640 1 -- 172.21.15.84:0/2271370349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92341019a0 msgr2=0x7f9234101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:49.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.199+0000 7f923b934640 1 --2- 172.21.15.84:0/2271370349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92341019a0 0x7f9234101e20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9230009900 tx=0x7f923002a410 comp rx=0 tx=0).stop 2024-05-22T23:01:49.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.199+0000 7f923b934640 1 -- 172.21.15.84:0/2271370349 shutdown_connections 2024-05-22T23:01:49.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.199+0000 7f923b934640 1 --2- 172.21.15.84:0/2271370349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9234102360 0x7f9234108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.199+0000 7f923b934640 1 --2- 172.21.15.84:0/2271370349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92341019a0 0x7f9234101e20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.199+0000 7f923b934640 1 --2- 172.21.15.84:0/2271370349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92341007b0 0x7f9234100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.199+0000 7f923b934640 1 -- 172.21.15.84:0/2271370349 >> 172.21.15.84:0/2271370349 conn(0x7f92340fbd20 msgr2=0x7f92340fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:49.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.200+0000 7f923b934640 1 -- 172.21.15.84:0/2271370349 shutdown_connections 2024-05-22T23:01:49.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.200+0000 7f923b934640 1 -- 172.21.15.84:0/2271370349 wait complete. 2024-05-22T23:01:49.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.200+0000 7f923b934640 1 Processor -- start 2024-05-22T23:01:49.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.200+0000 7f923b934640 1 -- start start 2024-05-22T23:01:49.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.200+0000 7f923b934640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92341007b0 0x7f9234199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:49.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.200+0000 7f923b934640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92341019a0 0x7f923419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:49.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.200+0000 7f923b934640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9234102360 0x7f92341a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:49.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.200+0000 7f923b934640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f923410c410 con 0x7f92341019a0 2024-05-22T23:01:49.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.200+0000 7f923b934640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f923410c290 con 0x7f9234102360 2024-05-22T23:01:49.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.200+0000 7f923b934640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f923410c590 con 0x7f92341007b0 2024-05-22T23:01:49.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.200+0000 7f9238ecf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92341019a0 0x7f923419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:49.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.200+0000 7f92396d0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92341007b0 0x7f9234199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:49.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.200+0000 7f92396d0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92341007b0 0x7f9234199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:48386/0 (socket says 172.21.15.84:48386) 2024-05-22T23:01:49.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.200+0000 7f92396d0640 1 -- 172.21.15.84:0/195749175 learned_addr learned my addr 172.21.15.84:0/195749175 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:49.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.200+0000 7f9239ed1640 1 --2- 172.21.15.84:0/195749175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9234102360 0x7f92341a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:49.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.201+0000 7f9238ecf640 1 -- 172.21.15.84:0/195749175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92341007b0 msgr2=0x7f9234199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:49.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.201+0000 7f9238ecf640 1 --2- 172.21.15.84:0/195749175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92341007b0 0x7f9234199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.201+0000 7f9238ecf640 1 -- 172.21.15.84:0/195749175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9234102360 msgr2=0x7f92341a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:49.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.201+0000 7f9238ecf640 1 --2- 172.21.15.84:0/195749175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9234102360 0x7f92341a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.201+0000 7f9238ecf640 1 -- 172.21.15.84:0/195749175 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92341a1a10 con 0x7f92341019a0 2024-05-22T23:01:49.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.201+0000 7f9239ed1640 1 --2- 172.21.15.84:0/195749175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9234102360 0x7f92341a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:01:49.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.201+0000 7f92396d0640 1 --2- 172.21.15.84:0/195749175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92341007b0 0x7f9234199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:01:49.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.201+0000 7f9238ecf640 1 --2- 172.21.15.84:0/195749175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92341019a0 0x7f923419a330 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9230031440 tx=0x7f9230031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:49.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.201+0000 7f92267fc640 1 -- 172.21.15.84:0/195749175 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923003f070 con 0x7f92341019a0 2024-05-22T23:01:49.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.201+0000 7f92267fc640 1 -- 172.21.15.84:0/195749175 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9230003cd0 con 0x7f92341019a0 2024-05-22T23:01:49.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.201+0000 7f923b934640 1 -- 172.21.15.84:0/195749175 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92341a1ca0 con 0x7f92341019a0 2024-05-22T23:01:49.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.201+0000 7f92267fc640 1 -- 172.21.15.84:0/195749175 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230035070 con 0x7f92341019a0 2024-05-22T23:01:49.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.201+0000 7f923b934640 1 -- 172.21.15.84:0/195749175 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92341a2190 con 0x7f92341019a0 2024-05-22T23:01:49.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.202+0000 7f923b934640 1 -- 172.21.15.84:0/195749175 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92341a2360 con 0x7f92341019a0 2024-05-22T23:01:49.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.203+0000 7f92267fc640 1 -- 172.21.15.84:0/195749175 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f923003b560 con 0x7f92341019a0 2024-05-22T23:01:49.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.204+0000 7f92267fc640 1 --2- 172.21.15.84:0/195749175 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f92200a2be0 0x7f92200a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:49.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.204+0000 7f92396d0640 1 -- 172.21.15.84:0/195749175 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f92200a2be0 msgr2=0x7f92200a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/4223589140 2024-05-22T23:01:49.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.204+0000 7f92267fc640 1 -- 172.21.15.84:0/195749175 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f92300ef500 con 0x7f92341019a0 2024-05-22T23:01:49.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.204+0000 7f92396d0640 1 --2- 172.21.15.84:0/195749175 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f92200a2be0 0x7f92200a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-22T23:01:49.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.208+0000 7f92267fc640 1 -- 172.21.15.84:0/195749175 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f92300bbdf0 con 0x7f92341019a0 2024-05-22T23:01:49.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.393+0000 7f923b934640 1 -- 172.21.15.84:0/195749175 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f92341a26f0 con 0x7f92341019a0 2024-05-22T23:01:49.396 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.394+0000 7f92267fc640 1 -- 172.21.15.84:0/195749175 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f92300c00f0 con 0x7f92341019a0 2024-05-22T23:01:49.396 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2024-05-22T23:01:49.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.396+0000 7f923b934640 1 -- 172.21.15.84:0/195749175 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f92200a2be0 msgr2=0x7f92200a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-22T23:01:49.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.396+0000 7f923b934640 1 --2- 172.21.15.84:0/195749175 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f92200a2be0 0x7f92200a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.396+0000 7f923b934640 1 -- 172.21.15.84:0/195749175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92341019a0 msgr2=0x7f923419a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:49.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.396+0000 7f923b934640 1 --2- 172.21.15.84:0/195749175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92341019a0 0x7f923419a330 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9230031440 tx=0x7f9230031b60 comp rx=0 tx=0).stop 2024-05-22T23:01:49.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.396+0000 7f923b934640 1 -- 172.21.15.84:0/195749175 shutdown_connections 2024-05-22T23:01:49.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.396+0000 7f923b934640 1 --2- 172.21.15.84:0/195749175 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f92200a2be0 0x7f92200a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.396+0000 7f923b934640 1 --2- 172.21.15.84:0/195749175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9234102360 0x7f92341a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.396+0000 7f923b934640 1 --2- 172.21.15.84:0/195749175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92341019a0 0x7f923419a330 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.396+0000 7f923b934640 1 --2- 172.21.15.84:0/195749175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92341007b0 0x7f9234199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.396+0000 7f923b934640 1 -- 172.21.15.84:0/195749175 >> 172.21.15.84:0/195749175 conn(0x7f92340fbd20 msgr2=0x7f9234068a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:49.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.396+0000 7f923b934640 1 -- 172.21.15.84:0/195749175 shutdown_connections 2024-05-22T23:01:49.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.396+0000 7f923b934640 1 -- 172.21.15.84:0/195749175 wait complete. 2024-05-22T23:01:49.409 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T23:01:49.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.513+0000 7f7281709640 1 -- 172.21.15.84:0/1863149033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f727c104c80 msgr2=0x7f727c107070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:49.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.513+0000 7f7281709640 1 --2- 172.21.15.84:0/1863149033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f727c104c80 0x7f727c107070 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7264009900 tx=0x7f726402a410 comp rx=0 tx=0).stop 2024-05-22T23:01:49.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.513+0000 7f7281709640 1 -- 172.21.15.84:0/1863149033 shutdown_connections 2024-05-22T23:01:49.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.513+0000 7f7281709640 1 --2- 172.21.15.84:0/1863149033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f727c1075b0 0x7f727c109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.513+0000 7f7281709640 1 --2- 172.21.15.84:0/1863149033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f727c104c80 0x7f727c107070 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.513+0000 7f7281709640 1 --2- 172.21.15.84:0/1863149033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f727c102350 0x7f727c104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.513+0000 7f7281709640 1 -- 172.21.15.84:0/1863149033 >> 172.21.15.84:0/1863149033 conn(0x7f727c0fbd60 msgr2=0x7f727c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:49.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.513+0000 7f7281709640 1 -- 172.21.15.84:0/1863149033 shutdown_connections 2024-05-22T23:01:49.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.513+0000 7f7281709640 1 -- 172.21.15.84:0/1863149033 wait complete. 2024-05-22T23:01:49.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.514+0000 7f7281709640 1 Processor -- start 2024-05-22T23:01:49.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.514+0000 7f7281709640 1 -- start start 2024-05-22T23:01:49.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.514+0000 7f7281709640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f727c102350 0x7f727c101cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:49.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.514+0000 7f7281709640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f727c104c80 0x7f727c100300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:49.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.514+0000 7f7281709640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f727c1075b0 0x7f727c100840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:49.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.514+0000 7f7281709640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f727c10d560 con 0x7f727c104c80 2024-05-22T23:01:49.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.514+0000 7f7281709640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f727c10d3e0 con 0x7f727c102350 2024-05-22T23:01:49.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.514+0000 7f7281709640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f727c10d6e0 con 0x7f727c1075b0 2024-05-22T23:01:49.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.514+0000 7f7273fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f727c104c80 0x7f727c100300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:49.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.514+0000 7f7273fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f727c104c80 0x7f727c100300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38132/0 (socket says 172.21.15.84:38132) 2024-05-22T23:01:49.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.514+0000 7f7273fff640 1 -- 172.21.15.84:0/1811826945 learned_addr learned my addr 172.21.15.84:0/1811826945 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:49.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.514+0000 7f727b7fe640 1 --2- 172.21.15.84:0/1811826945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f727c1075b0 0x7f727c100840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:49.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.514+0000 7f727affd640 1 --2- 172.21.15.84:0/1811826945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f727c102350 0x7f727c101cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:49.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.514+0000 7f7273fff640 1 -- 172.21.15.84:0/1811826945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f727c1075b0 msgr2=0x7f727c100840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:49.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.514+0000 7f7273fff640 1 --2- 172.21.15.84:0/1811826945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f727c1075b0 0x7f727c100840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.514+0000 7f7273fff640 1 -- 172.21.15.84:0/1811826945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f727c102350 msgr2=0x7f727c101cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:49.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.514+0000 7f7273fff640 1 --2- 172.21.15.84:0/1811826945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f727c102350 0x7f727c101cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.514+0000 7f7273fff640 1 -- 172.21.15.84:0/1811826945 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727c1010d0 con 0x7f727c104c80 2024-05-22T23:01:49.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.515+0000 7f7273fff640 1 --2- 172.21.15.84:0/1811826945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f727c104c80 0x7f727c100300 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7264031610 tx=0x7f7264004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:49.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.515+0000 7f7278ff9640 1 -- 172.21.15.84:0/1811826945 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726403f030 con 0x7f727c104c80 2024-05-22T23:01:49.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.515+0000 7f7281709640 1 -- 172.21.15.84:0/1811826945 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f727c1a1d20 con 0x7f727c104c80 2024-05-22T23:01:49.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.515+0000 7f7278ff9640 1 -- 172.21.15.84:0/1811826945 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7264003ad0 con 0x7f727c104c80 2024-05-22T23:01:49.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.515+0000 7f7278ff9640 1 -- 172.21.15.84:0/1811826945 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264035070 con 0x7f727c104c80 2024-05-22T23:01:49.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.515+0000 7f7281709640 1 -- 172.21.15.84:0/1811826945 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f727c1a21c0 con 0x7f727c104c80 2024-05-22T23:01:49.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.516+0000 7f7281709640 1 -- 172.21.15.84:0/1811826945 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f727c0fd850 con 0x7f727c104c80 2024-05-22T23:01:49.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.517+0000 7f7278ff9640 1 -- 172.21.15.84:0/1811826945 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f7264042030 con 0x7f727c104c80 2024-05-22T23:01:49.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.518+0000 7f7278ff9640 1 --2- 172.21.15.84:0/1811826945 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f72680a2be0 0x7f72680a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:49.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.518+0000 7f727affd640 1 -- 172.21.15.84:0/1811826945 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f72680a2be0 msgr2=0x7f72680a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/4223589140 2024-05-22T23:01:49.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.519+0000 7f7278ff9640 1 -- 172.21.15.84:0/1811826945 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f72640efa30 con 0x7f727c104c80 2024-05-22T23:01:49.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.519+0000 7f727affd640 1 --2- 172.21.15.84:0/1811826945 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f72680a2be0 0x7f72680a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-22T23:01:49.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.522+0000 7f7278ff9640 1 -- 172.21.15.84:0/1811826945 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f72640bc970 con 0x7f727c104c80 2024-05-22T23:01:49.550 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:49 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:49.170+0000 7f50b97ca200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-22T23:01:49.550 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:01:49 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y[37766]: 2024-05-22T23:01:49.389+0000 7f50b97ca200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-22T23:01:49.661 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:49 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:49.256+0000 7f4715c6d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-22T23:01:49.661 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:01:49 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z[38063]: 2024-05-22T23:01:49.477+0000 7f4715c6d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-22T23:01:49.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.703+0000 7f7281709640 1 -- 172.21.15.84:0/1811826945 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f727c02d3b0 con 0x7f727c104c80 2024-05-22T23:01:49.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.703+0000 7f7278ff9640 1 -- 172.21.15.84:0/1811826945 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f72640bc270 con 0x7f727c104c80 2024-05-22T23:01:49.705 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2024-05-22T23:01:49.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.705+0000 7f7281709640 1 -- 172.21.15.84:0/1811826945 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f72680a2be0 msgr2=0x7f72680a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-22T23:01:49.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.705+0000 7f7281709640 1 --2- 172.21.15.84:0/1811826945 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f72680a2be0 0x7f72680a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.705+0000 7f7281709640 1 -- 172.21.15.84:0/1811826945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f727c104c80 msgr2=0x7f727c100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:49.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.705+0000 7f7281709640 1 --2- 172.21.15.84:0/1811826945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f727c104c80 0x7f727c100300 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7264031610 tx=0x7f7264004270 comp rx=0 tx=0).stop 2024-05-22T23:01:49.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.706+0000 7f7281709640 1 -- 172.21.15.84:0/1811826945 shutdown_connections 2024-05-22T23:01:49.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.706+0000 7f7281709640 1 --2- 172.21.15.84:0/1811826945 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f72680a2be0 0x7f72680a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.706+0000 7f7281709640 1 --2- 172.21.15.84:0/1811826945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f727c1075b0 0x7f727c100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.706+0000 7f7281709640 1 --2- 172.21.15.84:0/1811826945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f727c104c80 0x7f727c100300 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.706+0000 7f7281709640 1 --2- 172.21.15.84:0/1811826945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f727c102350 0x7f727c101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.706+0000 7f7281709640 1 -- 172.21.15.84:0/1811826945 >> 172.21.15.84:0/1811826945 conn(0x7f727c0fbd60 msgr2=0x7f727c102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:49.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.706+0000 7f7281709640 1 -- 172.21.15.84:0/1811826945 shutdown_connections 2024-05-22T23:01:49.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.706+0000 7f7281709640 1 -- 172.21.15.84:0/1811826945 wait complete. 2024-05-22T23:01:49.717 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:49 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:49.629+0000 7f447aae9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-22T23:01:49.719 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T23:01:49.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.823+0000 7f7f185c8640 1 -- 172.21.15.84:0/1394754739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f10102390 msgr2=0x7f7f10104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:49.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.823+0000 7f7f185c8640 1 --2- 172.21.15.84:0/1394754739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f10102390 0x7f7f10104780 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009900 tx=0x7f7f0c02a410 comp rx=0 tx=0).stop 2024-05-22T23:01:49.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.823+0000 7f7f185c8640 1 -- 172.21.15.84:0/1394754739 shutdown_connections 2024-05-22T23:01:49.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.823+0000 7f7f185c8640 1 --2- 172.21.15.84:0/1394754739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f101075f0 0x7f7f101099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.823+0000 7f7f185c8640 1 --2- 172.21.15.84:0/1394754739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f10104cc0 0x7f7f101070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.823+0000 7f7f185c8640 1 --2- 172.21.15.84:0/1394754739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f10102390 0x7f7f10104780 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.823+0000 7f7f185c8640 1 -- 172.21.15.84:0/1394754739 >> 172.21.15.84:0/1394754739 conn(0x7f7f100fbd20 msgr2=0x7f7f100fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:49.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.824+0000 7f7f185c8640 1 -- 172.21.15.84:0/1394754739 shutdown_connections 2024-05-22T23:01:49.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.824+0000 7f7f185c8640 1 -- 172.21.15.84:0/1394754739 wait complete. 2024-05-22T23:01:49.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.824+0000 7f7f185c8640 1 Processor -- start 2024-05-22T23:01:49.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.824+0000 7f7f185c8640 1 -- start start 2024-05-22T23:01:49.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.824+0000 7f7f185c8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f10102390 0x7f7f10197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:49.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.824+0000 7f7f185c8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f10104cc0 0x7f7f101981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:49.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.824+0000 7f7f185c8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f101075f0 0x7f7f1019f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:49.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.824+0000 7f7f185c8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1010d550 con 0x7f7f101075f0 2024-05-22T23:01:49.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.824+0000 7f7f185c8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1010d3d0 con 0x7f7f10102390 2024-05-22T23:01:49.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.824+0000 7f7f185c8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1010d6d0 con 0x7f7f10104cc0 2024-05-22T23:01:49.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.824+0000 7f7f16b65640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f101075f0 0x7f7f1019f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:49.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.824+0000 7f7f16364640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f10102390 0x7f7f10197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:49.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.824+0000 7f7f16364640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f10102390 0x7f7f10197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:53600/0 (socket says 172.21.15.84:53600) 2024-05-22T23:01:49.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.824+0000 7f7f16b65640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f101075f0 0x7f7f1019f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38158/0 (socket says 172.21.15.84:38158) 2024-05-22T23:01:49.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.824+0000 7f7f16364640 1 -- 172.21.15.84:0/3031200119 learned_addr learned my addr 172.21.15.84:0/3031200119 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:49.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.825+0000 7f7f15b63640 1 --2- 172.21.15.84:0/3031200119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f10104cc0 0x7f7f101981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:49.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.825+0000 7f7f16364640 1 -- 172.21.15.84:0/3031200119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f10104cc0 msgr2=0x7f7f101981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:49.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.825+0000 7f7f16364640 1 --2- 172.21.15.84:0/3031200119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f10104cc0 0x7f7f101981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.825+0000 7f7f16364640 1 -- 172.21.15.84:0/3031200119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f101075f0 msgr2=0x7f7f1019f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:49.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.825+0000 7f7f16364640 1 --2- 172.21.15.84:0/3031200119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f101075f0 0x7f7f1019f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:49.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.825+0000 7f7f16364640 1 -- 172.21.15.84:0/3031200119 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1019f880 con 0x7f7f10102390 2024-05-22T23:01:49.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.825+0000 7f7f16364640 1 --2- 172.21.15.84:0/3031200119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f10102390 0x7f7f10197c70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009230 tx=0x7f7f0c004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:49.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.825+0000 7f7f077fe640 1 -- 172.21.15.84:0/3031200119 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c03f030 con 0x7f7f10102390 2024-05-22T23:01:49.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.825+0000 7f7f185c8640 1 -- 172.21.15.84:0/3031200119 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f1019fb10 con 0x7f7f10102390 2024-05-22T23:01:49.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.825+0000 7f7f077fe640 1 -- 172.21.15.84:0/3031200119 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f0c003ce0 con 0x7f7f10102390 2024-05-22T23:01:49.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.825+0000 7f7f077fe640 1 -- 172.21.15.84:0/3031200119 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c035070 con 0x7f7f10102390 2024-05-22T23:01:49.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.825+0000 7f7f185c8640 1 -- 172.21.15.84:0/3031200119 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f101a0000 con 0x7f7f10102390 2024-05-22T23:01:49.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.826+0000 7f7f185c8640 1 -- 172.21.15.84:0/3031200119 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f100fd810 con 0x7f7f10102390 2024-05-22T23:01:49.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.827+0000 7f7f077fe640 1 -- 172.21.15.84:0/3031200119 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f7f0c042030 con 0x7f7f10102390 2024-05-22T23:01:49.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.828+0000 7f7f077fe640 1 --2- 172.21.15.84:0/3031200119 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f7efc0a2be0 0x7f7efc0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:49.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.828+0000 7f7f15b63640 1 -- 172.21.15.84:0/3031200119 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f7efc0a2be0 msgr2=0x7f7efc0a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/4223589140 2024-05-22T23:01:49.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.828+0000 7f7f077fe640 1 -- 172.21.15.84:0/3031200119 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f7f0c0efbb0 con 0x7f7f10102390 2024-05-22T23:01:49.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.828+0000 7f7f15b63640 1 --2- 172.21.15.84:0/3031200119 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f7efc0a2be0 0x7f7efc0a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-22T23:01:49.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.831+0000 7f7f077fe640 1 -- 172.21.15.84:0/3031200119 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7f0c0bc4a0 con 0x7f7f10102390 2024-05-22T23:01:49.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.881+0000 7f7f077fe640 1 -- 172.21.15.84:0/3031200119 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 20) ==== 137518+0+0 (secure 0 0 0) 0x7f7f0c0b8a80 con 0x7f7f10102390 2024-05-22T23:01:49.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.881+0000 7f7f077fe640 1 -- 172.21.15.84:0/3031200119 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f7efc0a2be0 msgr2=0x7f7efc0a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-22T23:01:49.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:49.881+0000 7f7f077fe640 1 --2- 172.21.15.84:0/3031200119 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f7efc0a2be0 0x7f7efc0a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:50.045 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:49 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:49.866+0000 7f447aae9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-22T23:01:50.046 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:49 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1953813225' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:50.046 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:49 smithi084 ceph-mon[32186]: Standby manager daemon y restarted 2024-05-22T23:01:50.046 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:49 smithi084 ceph-mon[32186]: Standby manager daemon y started 2024-05-22T23:01:50.046 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:49 smithi084 ceph-mon[32186]: from='mgr.? 172.21.15.96:0/498551474' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-22T23:01:50.046 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:49 smithi084 ceph-mon[32186]: from='mgr.? 172.21.15.96:0/498551474' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-22T23:01:50.046 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:49 smithi084 ceph-mon[32186]: from='mgr.? 172.21.15.96:0/498551474' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-22T23:01:50.047 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:49 smithi084 ceph-mon[32186]: from='mgr.? 172.21.15.96:0/498551474' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-22T23:01:50.047 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:49 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/195749175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:01:50.047 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:49 smithi084 ceph-mon[32186]: Standby manager daemon z restarted 2024-05-22T23:01:50.047 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:49 smithi084 ceph-mon[32186]: Standby manager daemon z started 2024-05-22T23:01:50.047 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:49 smithi084 ceph-mon[32186]: from='mgr.? 172.21.15.204:0/3028437192' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-22T23:01:50.047 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:49 smithi084 ceph-mon[32186]: from='mgr.? 172.21.15.204:0/3028437192' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-22T23:01:50.047 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:49 smithi084 ceph-mon[32186]: from='mgr.? 172.21.15.204:0/3028437192' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-22T23:01:50.047 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:49 smithi084 ceph-mon[32186]: from='mgr.? 172.21.15.204:0/3028437192' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-22T23:01:50.047 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:49 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1811826945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:01:50.047 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:49 smithi084 ceph-mon[32186]: Active manager daemon x restarted 2024-05-22T23:01:50.047 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:49 smithi084 ceph-mon[32186]: Activating manager daemon x 2024-05-22T23:01:50.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.066+0000 7f7f185c8640 1 -- 172.21.15.84:0/3031200119 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7f1002d450 con 0x7f7f10102390 2024-05-22T23:01:50.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.067+0000 7f7f077fe640 1 -- 172.21.15.84:0/3031200119 <== mon.2 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f7f0c0e9f20 con 0x7f7f10102390 2024-05-22T23:01:50.068 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2024-05-22T23:01:50.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.069+0000 7f7f185c8640 1 -- 172.21.15.84:0/3031200119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f10102390 msgr2=0x7f7f10197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:50.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.069+0000 7f7f185c8640 1 --2- 172.21.15.84:0/3031200119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f10102390 0x7f7f10197c70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009230 tx=0x7f7f0c004360 comp rx=0 tx=0).stop 2024-05-22T23:01:50.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.069+0000 7f7f185c8640 1 -- 172.21.15.84:0/3031200119 shutdown_connections 2024-05-22T23:01:50.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.069+0000 7f7f185c8640 1 --2- 172.21.15.84:0/3031200119 >> [v2:172.21.15.84:6800/4223589140,v1:172.21.15.84:6801/4223589140] conn(0x7f7efc0a2be0 0x7f7efc0a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:50.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.069+0000 7f7f185c8640 1 --2- 172.21.15.84:0/3031200119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f101075f0 0x7f7f1019f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:50.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.069+0000 7f7f185c8640 1 --2- 172.21.15.84:0/3031200119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f10104cc0 0x7f7f101981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:50.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.069+0000 7f7f185c8640 1 --2- 172.21.15.84:0/3031200119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f10102390 0x7f7f10197c70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:50.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.069+0000 7f7f185c8640 1 -- 172.21.15.84:0/3031200119 >> 172.21.15.84:0/3031200119 conn(0x7f7f100fbd20 msgr2=0x7f7f10102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:50.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.069+0000 7f7f185c8640 1 -- 172.21.15.84:0/3031200119 shutdown_connections 2024-05-22T23:01:50.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.069+0000 7f7f185c8640 1 -- 172.21.15.84:0/3031200119 wait complete. 2024-05-22T23:01:50.083 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-22T23:01:50.084 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:01:50.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:49 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1953813225' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:50.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:49 smithi204 ceph-mon[36211]: Standby manager daemon y restarted 2024-05-22T23:01:50.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:49 smithi204 ceph-mon[36211]: Standby manager daemon y started 2024-05-22T23:01:50.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:49 smithi204 ceph-mon[36211]: from='mgr.? 172.21.15.96:0/498551474' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-22T23:01:50.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:49 smithi204 ceph-mon[36211]: from='mgr.? 172.21.15.96:0/498551474' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-22T23:01:50.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:49 smithi204 ceph-mon[36211]: from='mgr.? 172.21.15.96:0/498551474' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-22T23:01:50.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:49 smithi204 ceph-mon[36211]: from='mgr.? 172.21.15.96:0/498551474' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-22T23:01:50.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:49 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/195749175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:01:50.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:49 smithi204 ceph-mon[36211]: Standby manager daemon z restarted 2024-05-22T23:01:50.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:49 smithi204 ceph-mon[36211]: Standby manager daemon z started 2024-05-22T23:01:50.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:49 smithi204 ceph-mon[36211]: from='mgr.? 172.21.15.204:0/3028437192' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-22T23:01:50.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:49 smithi204 ceph-mon[36211]: from='mgr.? 172.21.15.204:0/3028437192' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-22T23:01:50.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:49 smithi204 ceph-mon[36211]: from='mgr.? 172.21.15.204:0/3028437192' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-22T23:01:50.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:49 smithi204 ceph-mon[36211]: from='mgr.? 172.21.15.204:0/3028437192' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-22T23:01:50.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:49 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1811826945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:01:50.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:49 smithi204 ceph-mon[36211]: Active manager daemon x restarted 2024-05-22T23:01:50.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:49 smithi204 ceph-mon[36211]: Activating manager daemon x 2024-05-22T23:01:50.147 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:49 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1953813225' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:50.148 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:49 smithi096 ceph-mon[36392]: Standby manager daemon y restarted 2024-05-22T23:01:50.148 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:49 smithi096 ceph-mon[36392]: Standby manager daemon y started 2024-05-22T23:01:50.148 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:49 smithi096 ceph-mon[36392]: from='mgr.? 172.21.15.96:0/498551474' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-22T23:01:50.148 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:49 smithi096 ceph-mon[36392]: from='mgr.? 172.21.15.96:0/498551474' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-22T23:01:50.148 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:49 smithi096 ceph-mon[36392]: from='mgr.? 172.21.15.96:0/498551474' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-22T23:01:50.148 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:49 smithi096 ceph-mon[36392]: from='mgr.? 172.21.15.96:0/498551474' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-22T23:01:50.148 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:49 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/195749175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:01:50.148 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:49 smithi096 ceph-mon[36392]: Standby manager daemon z restarted 2024-05-22T23:01:50.148 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:49 smithi096 ceph-mon[36392]: Standby manager daemon z started 2024-05-22T23:01:50.148 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:49 smithi096 ceph-mon[36392]: from='mgr.? 172.21.15.204:0/3028437192' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-22T23:01:50.148 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:49 smithi096 ceph-mon[36392]: from='mgr.? 172.21.15.204:0/3028437192' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-22T23:01:50.148 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:49 smithi096 ceph-mon[36392]: from='mgr.? 172.21.15.204:0/3028437192' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-22T23:01:50.149 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:49 smithi096 ceph-mon[36392]: from='mgr.? 172.21.15.204:0/3028437192' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-22T23:01:50.149 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:49 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1811826945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:01:50.149 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:49 smithi096 ceph-mon[36392]: Active manager daemon x restarted 2024-05-22T23:01:50.149 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:49 smithi096 ceph-mon[36392]: Activating manager daemon x 2024-05-22T23:01:50.219 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.217+0000 7f61ed631640 1 -- 172.21.15.84:0/1396235287 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e8104cc0 msgr2=0x7f61e81070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:50.219 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.217+0000 7f61ed631640 1 --2- 172.21.15.84:0/1396235287 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e8104cc0 0x7f61e81070b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009900 tx=0x7f61dc02a410 comp rx=0 tx=0).stop 2024-05-22T23:01:50.219 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.217+0000 7f61ed631640 1 -- 172.21.15.84:0/1396235287 shutdown_connections 2024-05-22T23:01:50.219 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.217+0000 7f61ed631640 1 --2- 172.21.15.84:0/1396235287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61e81075f0 0x7f61e81099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:50.219 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.217+0000 7f61ed631640 1 --2- 172.21.15.84:0/1396235287 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e8104cc0 0x7f61e81070b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:50.219 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.217+0000 7f61ed631640 1 --2- 172.21.15.84:0/1396235287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61e8102390 0x7f61e8104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:50.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.217+0000 7f61ed631640 1 -- 172.21.15.84:0/1396235287 >> 172.21.15.84:0/1396235287 conn(0x7f61e80fbd20 msgr2=0x7f61e80fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:50.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.217+0000 7f61ed631640 1 -- 172.21.15.84:0/1396235287 shutdown_connections 2024-05-22T23:01:50.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.217+0000 7f61ed631640 1 -- 172.21.15.84:0/1396235287 wait complete. 2024-05-22T23:01:50.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.217+0000 7f61ed631640 1 Processor -- start 2024-05-22T23:01:50.222 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.218+0000 7f61ed631640 1 -- start start 2024-05-22T23:01:50.222 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.218+0000 7f61ed631640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61e8102390 0x7f61e8199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:50.222 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.218+0000 7f61ed631640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e8104cc0 0x7f61e819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:50.222 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.218+0000 7f61ed631640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61e81075f0 0x7f61e81a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:50.222 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.218+0000 7f61ed631640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f61e810d550 con 0x7f61e8104cc0 2024-05-22T23:01:50.222 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.218+0000 7f61ed631640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f61e810d3d0 con 0x7f61e81075f0 2024-05-22T23:01:50.222 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.218+0000 7f61ed631640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f61e810d6d0 con 0x7f61e8102390 2024-05-22T23:01:50.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.218+0000 7f61e67fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e8104cc0 0x7f61e819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:50.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.218+0000 7f61e67fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e8104cc0 0x7f61e819a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38284/0 (socket says 172.21.15.84:38284) 2024-05-22T23:01:50.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.218+0000 7f61e67fc640 1 -- 172.21.15.84:0/773261103 learned_addr learned my addr 172.21.15.84:0/773261103 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:50.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.218+0000 7f61e77fe640 1 --2- 172.21.15.84:0/773261103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61e81075f0 0x7f61e81a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:50.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.218+0000 7f61e6ffd640 1 --2- 172.21.15.84:0/773261103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61e8102390 0x7f61e8199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:50.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.218+0000 7f61e67fc640 1 -- 172.21.15.84:0/773261103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61e8102390 msgr2=0x7f61e8199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:50.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.218+0000 7f61e67fc640 1 --2- 172.21.15.84:0/773261103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61e8102390 0x7f61e8199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:50.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.218+0000 7f61e67fc640 1 -- 172.21.15.84:0/773261103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61e81075f0 msgr2=0x7f61e81a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:50.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.218+0000 7f61e67fc640 1 --2- 172.21.15.84:0/773261103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61e81075f0 0x7f61e81a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:50.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.218+0000 7f61e67fc640 1 -- 172.21.15.84:0/773261103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e81a1a60 con 0x7f61e8104cc0 2024-05-22T23:01:50.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.218+0000 7f61e77fe640 1 --2- 172.21.15.84:0/773261103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61e81075f0 0x7f61e81a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:01:50.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.219+0000 7f61e67fc640 1 --2- 172.21.15.84:0/773261103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e8104cc0 0x7f61e819a380 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f61dc031440 tx=0x7f61dc032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:50.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.219+0000 7f61cffff640 1 -- 172.21.15.84:0/773261103 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc03f030 con 0x7f61e8104cc0 2024-05-22T23:01:50.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.219+0000 7f61cffff640 1 -- 172.21.15.84:0/773261103 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61dc003ce0 con 0x7f61e8104cc0 2024-05-22T23:01:50.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.219+0000 7f61ed631640 1 -- 172.21.15.84:0/773261103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e81a1cf0 con 0x7f61e8104cc0 2024-05-22T23:01:50.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.219+0000 7f61cffff640 1 -- 172.21.15.84:0/773261103 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc035070 con 0x7f61e8104cc0 2024-05-22T23:01:50.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.219+0000 7f61ed631640 1 -- 172.21.15.84:0/773261103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e81a2120 con 0x7f61e8104cc0 2024-05-22T23:01:50.225 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.220+0000 7f61ed631640 1 -- 172.21.15.84:0/773261103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e81090e0 con 0x7f61e8104cc0 2024-05-22T23:01:50.225 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.221+0000 7f61cffff640 1 -- 172.21.15.84:0/773261103 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 20) ==== 137518+0+0 (secure 0 0 0) 0x7f61dc042030 con 0x7f61e8104cc0 2024-05-22T23:01:50.225 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.222+0000 7f61cffff640 1 -- 172.21.15.84:0/773261103 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f61dc0ef3a0 con 0x7f61e8104cc0 2024-05-22T23:01:50.228 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.226+0000 7f61cffff640 1 -- 172.21.15.84:0/773261103 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f61dc0bbf90 con 0x7f61e8104cc0 2024-05-22T23:01:50.375 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:50 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:50.142+0000 7f43d9be3700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-22T23:01:50.375 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:50 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:50.142+0000 7f43d9be3700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-22T23:01:50.376 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:50 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:50.142+0000 7f43d9be3700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-22T23:01:50.376 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:50 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:50.142+0000 7f43d9be3700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-22T23:01:50.376 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:50 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:50.142+0000 7f43d9be3700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-22T23:01:50.376 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:50 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:50.142+0000 7f43dd3ea700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-22T23:01:50.376 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:50 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:50.142+0000 7f43dd3ea700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-22T23:01:50.376 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:50 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:50.142+0000 7f43dd3ea700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-22T23:01:50.376 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:50 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:50.142+0000 7f43dd3ea700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-22T23:01:50.376 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:01:50 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:01:50.142+0000 7f43dd3ea700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-22T23:01:50.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.478+0000 7f61ed631640 1 -- 172.21.15.84:0/773261103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61e8062470 con 0x7f61e8104cc0 2024-05-22T23:01:50.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.479+0000 7f61cffff640 1 -- 172.21.15.84:0/773261103 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7f61dc0c0290 con 0x7f61e8104cc0 2024-05-22T23:01:50.481 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:50.481 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:01:50.481 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:01:50.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.482+0000 7f61ed631640 1 -- 172.21.15.84:0/773261103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e8104cc0 msgr2=0x7f61e819a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:50.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.482+0000 7f61ed631640 1 --2- 172.21.15.84:0/773261103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e8104cc0 0x7f61e819a380 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f61dc031440 tx=0x7f61dc032ce0 comp rx=0 tx=0).stop 2024-05-22T23:01:50.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.482+0000 7f61ed631640 1 -- 172.21.15.84:0/773261103 shutdown_connections 2024-05-22T23:01:50.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.482+0000 7f61ed631640 1 --2- 172.21.15.84:0/773261103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61e81075f0 0x7f61e81a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:50.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.482+0000 7f61ed631640 1 --2- 172.21.15.84:0/773261103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e8104cc0 0x7f61e819a380 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:50.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.482+0000 7f61ed631640 1 --2- 172.21.15.84:0/773261103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61e8102390 0x7f61e8199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:50.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.482+0000 7f61ed631640 1 -- 172.21.15.84:0/773261103 >> 172.21.15.84:0/773261103 conn(0x7f61e80fbd20 msgr2=0x7f61e8101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:50.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.482+0000 7f61ed631640 1 -- 172.21.15.84:0/773261103 shutdown_connections 2024-05-22T23:01:50.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.482+0000 7f61ed631640 1 -- 172.21.15.84:0/773261103 wait complete. 2024-05-22T23:01:50.499 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-22T23:01:50.499 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-22T23:01:50.500 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T23:01:50.605 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.601+0000 7f4c5b0f5640 1 -- 172.21.15.84:0/1307873292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c541007b0 msgr2=0x7f4c54100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:50.605 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.601+0000 7f4c5b0f5640 1 --2- 172.21.15.84:0/1307873292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c541007b0 0x7f4c54100bd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4c48009900 tx=0x7f4c4802a410 comp rx=0 tx=0).stop 2024-05-22T23:01:50.605 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.601+0000 7f4c5b0f5640 1 -- 172.21.15.84:0/1307873292 shutdown_connections 2024-05-22T23:01:50.605 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.601+0000 7f4c5b0f5640 1 --2- 172.21.15.84:0/1307873292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c54102360 0x7f4c54108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:50.605 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.601+0000 7f4c5b0f5640 1 --2- 172.21.15.84:0/1307873292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c541019a0 0x7f4c54101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:50.605 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.601+0000 7f4c5b0f5640 1 --2- 172.21.15.84:0/1307873292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c541007b0 0x7f4c54100bd0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:50.605 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.601+0000 7f4c5b0f5640 1 -- 172.21.15.84:0/1307873292 >> 172.21.15.84:0/1307873292 conn(0x7f4c540fbd20 msgr2=0x7f4c540fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:50.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.601+0000 7f4c5b0f5640 1 -- 172.21.15.84:0/1307873292 shutdown_connections 2024-05-22T23:01:50.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.601+0000 7f4c5b0f5640 1 -- 172.21.15.84:0/1307873292 wait complete. 2024-05-22T23:01:50.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.602+0000 7f4c5b0f5640 1 Processor -- start 2024-05-22T23:01:50.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.602+0000 7f4c5b0f5640 1 -- start start 2024-05-22T23:01:50.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.602+0000 7f4c5b0f5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c541007b0 0x7f4c54069570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:50.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.602+0000 7f4c5b0f5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c541019a0 0x7f4c5419c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:50.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.602+0000 7f4c5b0f5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c54102360 0x7f4c5419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:50.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.602+0000 7f4c5b0f5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5410c430 con 0x7f4c54102360 2024-05-22T23:01:50.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.602+0000 7f4c5b0f5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5410c2b0 con 0x7f4c541007b0 2024-05-22T23:01:50.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.602+0000 7f4c5b0f5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5410c5b0 con 0x7f4c541019a0 2024-05-22T23:01:50.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.602+0000 7f4c59692640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c54102360 0x7f4c5419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:50.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.602+0000 7f4c58e91640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c541007b0 0x7f4c54069570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:50.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.602+0000 7f4c58e91640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c541007b0 0x7f4c54069570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:53772/0 (socket says 172.21.15.84:53772) 2024-05-22T23:01:50.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.602+0000 7f4c59692640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c54102360 0x7f4c5419ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38308/0 (socket says 172.21.15.84:38308) 2024-05-22T23:01:50.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.602+0000 7f4c58e91640 1 -- 172.21.15.84:0/4086020409 learned_addr learned my addr 172.21.15.84:0/4086020409 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:50.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.602+0000 7f4c53fff640 1 --2- 172.21.15.84:0/4086020409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c541019a0 0x7f4c5419c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:50.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.602+0000 7f4c59692640 1 -- 172.21.15.84:0/4086020409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c541019a0 msgr2=0x7f4c5419c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:50.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.602+0000 7f4c59692640 1 --2- 172.21.15.84:0/4086020409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c541019a0 0x7f4c5419c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:50.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.602+0000 7f4c59692640 1 -- 172.21.15.84:0/4086020409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c541007b0 msgr2=0x7f4c54069570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:50.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.602+0000 7f4c59692640 1 --2- 172.21.15.84:0/4086020409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c541007b0 0x7f4c54069570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:50.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.602+0000 7f4c59692640 1 -- 172.21.15.84:0/4086020409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5419f820 con 0x7f4c54102360 2024-05-22T23:01:50.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.603+0000 7f4c59692640 1 --2- 172.21.15.84:0/4086020409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c54102360 0x7f4c5419ef10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009fd0 tx=0x7f4c4c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:50.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.603+0000 7f4c51ffb640 1 -- 172.21.15.84:0/4086020409 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c00ce20 con 0x7f4c54102360 2024-05-22T23:01:50.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.603+0000 7f4c5b0f5640 1 -- 172.21.15.84:0/4086020409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c5419fab0 con 0x7f4c54102360 2024-05-22T23:01:50.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.603+0000 7f4c51ffb640 1 -- 172.21.15.84:0/4086020409 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c4c005020 con 0x7f4c54102360 2024-05-22T23:01:50.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.603+0000 7f4c51ffb640 1 -- 172.21.15.84:0/4086020409 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c010720 con 0x7f4c54102360 2024-05-22T23:01:50.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.603+0000 7f4c5b0f5640 1 -- 172.21.15.84:0/4086020409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c541a0000 con 0x7f4c54102360 2024-05-22T23:01:50.608 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.604+0000 7f4c5b0f5640 1 -- 172.21.15.84:0/4086020409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c54107ad0 con 0x7f4c54102360 2024-05-22T23:01:50.608 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.605+0000 7f4c51ffb640 1 -- 172.21.15.84:0/4086020409 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 20) ==== 137518+0+0 (secure 0 0 0) 0x7f4c4c005540 con 0x7f4c54102360 2024-05-22T23:01:50.608 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.606+0000 7f4c51ffb640 1 -- 172.21.15.84:0/4086020409 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f4c4c0cf1e0 con 0x7f4c54102360 2024-05-22T23:01:50.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.611+0000 7f4c51ffb640 1 -- 172.21.15.84:0/4086020409 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4c4c09c420 con 0x7f4c54102360 2024-05-22T23:01:50.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.902+0000 7f4c51ffb640 1 -- 172.21.15.84:0/4086020409 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f4c4c097a70 con 0x7f4c54102360 2024-05-22T23:01:50.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.903+0000 7f4c51ffb640 1 --2- 172.21.15.84:0/4086020409 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f4c440a9980 0x7f4c440abd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:50.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.903+0000 7f4c51ffb640 1 -- 172.21.15.84:0/4086020409 --> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4c4c01ee50 con 0x7f4c440a9980 2024-05-22T23:01:50.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.905+0000 7f4c58e91640 1 --2- 172.21.15.84:0/4086020409 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f4c440a9980 0x7f4c440abd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:50.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.909+0000 7f4c58e91640 1 --2- 172.21.15.84:0/4086020409 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f4c440a9980 0x7f4c440abd70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4c48000c00 tx=0x7f4c480046c0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:50.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.918+0000 7f4c51ffb640 1 -- 172.21.15.84:0/4086020409 <== mgr.24851 v2:172.21.15.84:6800/3358259585 1 ==== mgr_command_reply(tid 0: 0 Warning: due to ceph-mgr restart, some PG states may not be up to date 2024-05-22T23:01:50.920 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all) ==== 89+0+230503 (secure 0 0 0) 0x7f4c4c01ee50 con 0x7f4c440a9980 2024-05-22T23:01:50.922 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:50.923 INFO:teuthology.orchestra.run.smithi084.stderr:Warning: due to ceph-mgr restart, some PG states may not be up to date 2024-05-22T23:01:50.923 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2024-05-22T23:01:50.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.921+0000 7f4c5b0f5640 1 -- 172.21.15.84:0/4086020409 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f4c440a9980 msgr2=0x7f4c440abd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:50.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.921+0000 7f4c5b0f5640 1 --2- 172.21.15.84:0/4086020409 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f4c440a9980 0x7f4c440abd70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4c48000c00 tx=0x7f4c480046c0 comp rx=0 tx=0).stop 2024-05-22T23:01:50.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.921+0000 7f4c5b0f5640 1 -- 172.21.15.84:0/4086020409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c54102360 msgr2=0x7f4c5419ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:50.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.921+0000 7f4c5b0f5640 1 --2- 172.21.15.84:0/4086020409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c54102360 0x7f4c5419ef10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009fd0 tx=0x7f4c4c00ef10 comp rx=0 tx=0).stop 2024-05-22T23:01:50.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.921+0000 7f4c5b0f5640 1 -- 172.21.15.84:0/4086020409 shutdown_connections 2024-05-22T23:01:50.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.921+0000 7f4c5b0f5640 1 --2- 172.21.15.84:0/4086020409 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f4c440a9980 0x7f4c440abd70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:50.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.921+0000 7f4c5b0f5640 1 --2- 172.21.15.84:0/4086020409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c54102360 0x7f4c5419ef10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:50.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.921+0000 7f4c5b0f5640 1 --2- 172.21.15.84:0/4086020409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c541019a0 0x7f4c5419c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:50.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.921+0000 7f4c5b0f5640 1 --2- 172.21.15.84:0/4086020409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c541007b0 0x7f4c54069570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:50.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.921+0000 7f4c5b0f5640 1 -- 172.21.15.84:0/4086020409 >> 172.21.15.84:0/4086020409 conn(0x7f4c540fbd20 msgr2=0x7f4c54104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:50.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.921+0000 7f4c5b0f5640 1 -- 172.21.15.84:0/4086020409 shutdown_connections 2024-05-22T23:01:50.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:50.921+0000 7f4c5b0f5640 1 -- 172.21.15.84:0/4086020409 wait complete. 2024-05-22T23:01:50.935 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":false,"pg_map":{"version":2,"stamp":"2024-05-22T23:01:49.896588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T23:01:49.869890+0000","last_change":"2024-05-22T23:01:49.869890+0000","last_active":"2024-05-22T23:01:49.869890+0000","last_peered":"2024-05-22T23:01:49.869890+0000","last_clean":"2024-05-22T23:01:49.869890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T23:01:49.869890+0000","last_undegraded":"2024-05-22T23:01:49.869890+0000","last_fullsized":"2024-05-22T23:01:49.869890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T23:01:49.869890+0000","last_clean_scrub_stamp":"2024-05-22T23:01:49.869890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2024-05-22T23:01:50.939 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T23:01:51.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.074+0000 7f2ef7627640 1 -- 172.21.15.84:0/465561784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ef01007b0 msgr2=0x7f2ef0100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:51.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.074+0000 7f2ef7627640 1 --2- 172.21.15.84:0/465561784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ef01007b0 0x7f2ef0100bd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009900 tx=0x7f2eec02a410 comp rx=0 tx=0).stop 2024-05-22T23:01:51.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.074+0000 7f2ef7627640 1 -- 172.21.15.84:0/465561784 shutdown_connections 2024-05-22T23:01:51.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.074+0000 7f2ef7627640 1 --2- 172.21.15.84:0/465561784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ef0102360 0x7f2ef0108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.074+0000 7f2ef7627640 1 --2- 172.21.15.84:0/465561784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ef01019a0 0x7f2ef0101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.074+0000 7f2ef7627640 1 --2- 172.21.15.84:0/465561784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ef01007b0 0x7f2ef0100bd0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.074+0000 7f2ef7627640 1 -- 172.21.15.84:0/465561784 >> 172.21.15.84:0/465561784 conn(0x7f2ef00fbd20 msgr2=0x7f2ef00fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:51.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.075+0000 7f2ef7627640 1 -- 172.21.15.84:0/465561784 shutdown_connections 2024-05-22T23:01:51.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.075+0000 7f2ef7627640 1 -- 172.21.15.84:0/465561784 wait complete. 2024-05-22T23:01:51.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.075+0000 7f2ef7627640 1 Processor -- start 2024-05-22T23:01:51.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.075+0000 7f2ef7627640 1 -- start start 2024-05-22T23:01:51.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.075+0000 7f2ef7627640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ef01007b0 0x7f2ef0197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:51.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.075+0000 7f2ef7627640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ef01019a0 0x7f2ef0198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:51.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.075+0000 7f2ef7627640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ef0102360 0x7f2ef019f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:51.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.075+0000 7f2ef7627640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef010c400 con 0x7f2ef0102360 2024-05-22T23:01:51.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.075+0000 7f2ef7627640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef010c280 con 0x7f2ef01019a0 2024-05-22T23:01:51.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.075+0000 7f2ef7627640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef010c580 con 0x7f2ef01007b0 2024-05-22T23:01:51.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.075+0000 7f2ef5bc4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ef0102360 0x7f2ef019f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:51.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.075+0000 7f2ef5bc4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ef0102360 0x7f2ef019f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38326/0 (socket says 172.21.15.84:38326) 2024-05-22T23:01:51.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.075+0000 7f2ef5bc4640 1 -- 172.21.15.84:0/2196849691 learned_addr learned my addr 172.21.15.84:0/2196849691 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:51.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.075+0000 7f2ef4bc2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ef01019a0 0x7f2ef0198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:51.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.076+0000 7f2ef53c3640 1 --2- 172.21.15.84:0/2196849691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ef01007b0 0x7f2ef0197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:51.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.076+0000 7f2ef5bc4640 1 -- 172.21.15.84:0/2196849691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ef01007b0 msgr2=0x7f2ef0197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:51.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.076+0000 7f2ef5bc4640 1 --2- 172.21.15.84:0/2196849691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ef01007b0 0x7f2ef0197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.076+0000 7f2ef5bc4640 1 -- 172.21.15.84:0/2196849691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ef01019a0 msgr2=0x7f2ef0198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:51.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.076+0000 7f2ef5bc4640 1 --2- 172.21.15.84:0/2196849691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ef01019a0 0x7f2ef0198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.076+0000 7f2ef5bc4640 1 -- 172.21.15.84:0/2196849691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef019f840 con 0x7f2ef0102360 2024-05-22T23:01:51.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.076+0000 7f2ef53c3640 1 --2- 172.21.15.84:0/2196849691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ef01007b0 0x7f2ef0197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:01:51.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.076+0000 7f2ef5bc4640 1 --2- 172.21.15.84:0/2196849691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ef0102360 0x7f2ef019f1a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800cad0 tx=0x7f2ee800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:51.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.076+0000 7f2ee67fc640 1 -- 172.21.15.84:0/2196849691 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee800ae20 con 0x7f2ef0102360 2024-05-22T23:01:51.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.076+0000 7f2ee67fc640 1 -- 172.21.15.84:0/2196849691 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ee8005020 con 0x7f2ef0102360 2024-05-22T23:01:51.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.076+0000 7f2ef7627640 1 -- 172.21.15.84:0/2196849691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef019fb30 con 0x7f2ef0102360 2024-05-22T23:01:51.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.076+0000 7f2ee67fc640 1 -- 172.21.15.84:0/2196849691 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee800f6f0 con 0x7f2ef0102360 2024-05-22T23:01:51.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.076+0000 7f2ef7627640 1 -- 172.21.15.84:0/2196849691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef01a0030 con 0x7f2ef0102360 2024-05-22T23:01:51.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.077+0000 7f2ef7627640 1 -- 172.21.15.84:0/2196849691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ef01a0200 con 0x7f2ef0102360 2024-05-22T23:01:51.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.078+0000 7f2ee67fc640 1 -- 172.21.15.84:0/2196849691 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f2ee80051d0 con 0x7f2ef0102360 2024-05-22T23:01:51.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.079+0000 7f2ee67fc640 1 --2- 172.21.15.84:0/2196849691 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f2edc0a2990 0x7f2edc0a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:51.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.079+0000 7f2ef53c3640 1 --2- 172.21.15.84:0/2196849691 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f2edc0a2990 0x7f2edc0a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:51.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.079+0000 7f2ee67fc640 1 -- 172.21.15.84:0/2196849691 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f2ee80cf200 con 0x7f2ef0102360 2024-05-22T23:01:51.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.079+0000 7f2ef53c3640 1 --2- 172.21.15.84:0/2196849691 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f2edc0a2990 0x7f2edc0a4e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2eec0314e0 tx=0x7f2eec032dd0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:51.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.082+0000 7f2ee67fc640 1 -- 172.21.15.84:0/2196849691 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2ee8014030 con 0x7f2ef0102360 2024-05-22T23:01:51.160 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: osdmap e89: 12 total, 12 up, 12 in 2024-05-22T23:01:51.160 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: mgrmap e20: x(active, starting, since 0.0131659s), standbys: z, y 2024-05-22T23:01:51.160 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-22T23:01:51.160 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T23:01:51.160 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-22T23:01:51.160 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-22T23:01:51.160 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-22T23:01:51.160 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-22T23:01:51.160 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-22T23:01:51.160 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-22T23:01:51.160 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-22T23:01:51.161 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-22T23:01:51.161 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-22T23:01:51.161 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-22T23:01:51.161 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-22T23:01:51.161 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-22T23:01:51.161 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-22T23:01:51.161 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-22T23:01:51.161 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-22T23:01:51.161 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-22T23:01:51.161 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-22T23:01:51.161 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-22T23:01:51.161 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-22T23:01:51.161 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-22T23:01:51.162 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-22T23:01:51.162 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-22T23:01:51.162 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-22T23:01:51.162 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-22T23:01:51.162 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-22T23:01:51.162 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-22T23:01:51.162 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-22T23:01:51.162 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-22T23:01:51.162 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-22T23:01:51.162 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-22T23:01:51.162 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-22T23:01:51.162 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: Manager daemon x is now available 2024-05-22T23:01:51.162 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:01:51.163 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-22T23:01:51.163 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-22T23:01:51.163 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3031200119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:01:51.163 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: [22/May/2024:23:01:50] ENGINE Bus STARTING 2024-05-22T23:01:51.163 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/773261103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:01:51.163 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: [22/May/2024:23:01:50] ENGINE Serving on https://172.21.15.84:7150 2024-05-22T23:01:51.163 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: [22/May/2024:23:01:50] ENGINE Serving on http://172.21.15.84:8765 2024-05-22T23:01:51.163 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:50 smithi096 ceph-mon[36392]: [22/May/2024:23:01:50] ENGINE Bus STARTED 2024-05-22T23:01:51.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: osdmap e89: 12 total, 12 up, 12 in 2024-05-22T23:01:51.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: mgrmap e20: x(active, starting, since 0.0131659s), standbys: z, y 2024-05-22T23:01:51.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-22T23:01:51.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T23:01:51.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-22T23:01:51.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-22T23:01:51.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-22T23:01:51.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-22T23:01:51.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-22T23:01:51.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-22T23:01:51.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-22T23:01:51.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-22T23:01:51.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-22T23:01:51.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-22T23:01:51.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-22T23:01:51.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-22T23:01:51.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-22T23:01:51.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-22T23:01:51.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-22T23:01:51.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-22T23:01:51.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-22T23:01:51.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-22T23:01:51.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-22T23:01:51.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-22T23:01:51.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-22T23:01:51.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-22T23:01:51.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-22T23:01:51.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-22T23:01:51.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-22T23:01:51.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-22T23:01:51.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-22T23:01:51.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-22T23:01:51.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-22T23:01:51.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-22T23:01:51.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-22T23:01:51.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: Manager daemon x is now available 2024-05-22T23:01:51.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:01:51.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-22T23:01:51.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-22T23:01:51.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3031200119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:01:51.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: [22/May/2024:23:01:50] ENGINE Bus STARTING 2024-05-22T23:01:51.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/773261103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:01:51.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: [22/May/2024:23:01:50] ENGINE Serving on https://172.21.15.84:7150 2024-05-22T23:01:51.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: [22/May/2024:23:01:50] ENGINE Serving on http://172.21.15.84:8765 2024-05-22T23:01:51.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:50 smithi204 ceph-mon[36211]: [22/May/2024:23:01:50] ENGINE Bus STARTED 2024-05-22T23:01:51.259 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.258+0000 7f2ef7627640 1 -- 172.21.15.84:0/2196849691 --> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2ef01a03d0 con 0x7f2edc0a2990 2024-05-22T23:01:51.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.265+0000 7f2ee67fc640 1 -- 172.21.15.84:0/2196849691 <== mgr.24851 v2:172.21.15.84:6800/3358259585 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257962 (secure 0 0 0) 0x7f2ef01a03d0 con 0x7f2edc0a2990 2024-05-22T23:01:51.270 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:51.271 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2024-05-22T23:01:51.273 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.268+0000 7f2ef7627640 1 -- 172.21.15.84:0/2196849691 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f2edc0a2990 msgr2=0x7f2edc0a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:51.273 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.268+0000 7f2ef7627640 1 --2- 172.21.15.84:0/2196849691 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f2edc0a2990 0x7f2edc0a4e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2eec0314e0 tx=0x7f2eec032dd0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.273 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.268+0000 7f2ef7627640 1 -- 172.21.15.84:0/2196849691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ef0102360 msgr2=0x7f2ef019f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:51.273 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.268+0000 7f2ef7627640 1 --2- 172.21.15.84:0/2196849691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ef0102360 0x7f2ef019f1a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800cad0 tx=0x7f2ee800cf90 comp rx=0 tx=0).stop 2024-05-22T23:01:51.273 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.268+0000 7f2ef7627640 1 -- 172.21.15.84:0/2196849691 shutdown_connections 2024-05-22T23:01:51.273 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.268+0000 7f2ef7627640 1 --2- 172.21.15.84:0/2196849691 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f2edc0a2990 0x7f2edc0a4e50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.273 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.268+0000 7f2ef7627640 1 --2- 172.21.15.84:0/2196849691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ef0102360 0x7f2ef019f1a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.268+0000 7f2ef7627640 1 --2- 172.21.15.84:0/2196849691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ef01019a0 0x7f2ef0198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.268+0000 7f2ef7627640 1 --2- 172.21.15.84:0/2196849691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ef01007b0 0x7f2ef0197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.268+0000 7f2ef7627640 1 -- 172.21.15.84:0/2196849691 >> 172.21.15.84:0/2196849691 conn(0x7f2ef00fbd20 msgr2=0x7f2ef0068a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:51.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.268+0000 7f2ef7627640 1 -- 172.21.15.84:0/2196849691 shutdown_connections 2024-05-22T23:01:51.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.268+0000 7f2ef7627640 1 -- 172.21.15.84:0/2196849691 wait complete. 2024-05-22T23:01:51.280 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: osdmap e89: 12 total, 12 up, 12 in 2024-05-22T23:01:51.280 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: mgrmap e20: x(active, starting, since 0.0131659s), standbys: z, y 2024-05-22T23:01:51.280 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-22T23:01:51.280 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-22T23:01:51.280 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-22T23:01:51.280 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-22T23:01:51.281 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-22T23:01:51.281 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-22T23:01:51.281 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-22T23:01:51.281 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-22T23:01:51.281 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-22T23:01:51.281 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-22T23:01:51.281 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-22T23:01:51.281 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-22T23:01:51.281 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-22T23:01:51.281 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-22T23:01:51.281 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-22T23:01:51.281 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-22T23:01:51.282 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-22T23:01:51.282 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-22T23:01:51.282 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-22T23:01:51.282 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-22T23:01:51.282 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-22T23:01:51.282 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-22T23:01:51.282 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-22T23:01:51.282 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-22T23:01:51.282 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-22T23:01:51.283 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-22T23:01:51.283 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-22T23:01:51.283 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-22T23:01:51.283 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-22T23:01:51.283 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-22T23:01:51.283 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-22T23:01:51.283 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-22T23:01:51.283 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-22T23:01:51.284 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: Manager daemon x is now available 2024-05-22T23:01:51.284 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:01:51.284 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-22T23:01:51.284 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-22T23:01:51.284 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3031200119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:01:51.284 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: [22/May/2024:23:01:50] ENGINE Bus STARTING 2024-05-22T23:01:51.284 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/773261103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:01:51.284 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: [22/May/2024:23:01:50] ENGINE Serving on https://172.21.15.84:7150 2024-05-22T23:01:51.285 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: [22/May/2024:23:01:50] ENGINE Serving on http://172.21.15.84:8765 2024-05-22T23:01:51.285 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:50 smithi084 ceph-mon[32186]: [22/May/2024:23:01:50] ENGINE Bus STARTED 2024-05-22T23:01:51.286 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":3,"stamp":"2024-05-22T23:01:50.919930+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1181837,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":460,"num_read_kb":375,"num_write":376,"num_write_kb":4339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1239647,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":782,"ondisk_log_size":782,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385932,"kb_used_data":11780,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124555892,"statfs":{"total":1151940427776,"available":1151545233408,"internally_reserved":0,"allocated":12062720,"data_stored":9169317,"data_compressed":294247,"data_compressed_allocated":393216,"data_compressed_original":3670016,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.214487+0000","last_change":"2024-05-22T22:59:51.951625+0000","last_active":"2024-05-22T23:01:50.214487+0000","last_peered":"2024-05-22T23:01:50.214487+0000","last_clean":"2024-05-22T23:01:50.214487+0000","last_became_active":"2024-05-22T22:59:51.951225+0000","last_became_peered":"2024-05-22T22:59:51.951225+0000","last_unstale":"2024-05-22T23:01:50.214487+0000","last_undegraded":"2024-05-22T23:01:50.214487+0000","last_fullsized":"2024-05-22T23:01:50.214487+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:53:32.616083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.709529+0000","last_change":"2024-05-22T22:59:51.956925+0000","last_active":"2024-05-22T23:01:50.709529+0000","last_peered":"2024-05-22T23:01:50.709529+0000","last_clean":"2024-05-22T23:01:50.709529+0000","last_became_active":"2024-05-22T22:59:51.956557+0000","last_became_peered":"2024-05-22T22:59:51.956557+0000","last_unstale":"2024-05-22T23:01:50.709529+0000","last_undegraded":"2024-05-22T23:01:50.709529+0000","last_fullsized":"2024-05-22T23:01:50.709529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:18:14.609682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.709546+0000","last_change":"2024-05-22T22:59:51.951981+0000","last_active":"2024-05-22T23:01:50.709546+0000","last_peered":"2024-05-22T23:01:50.709546+0000","last_clean":"2024-05-22T23:01:50.709546+0000","last_became_active":"2024-05-22T22:59:51.951522+0000","last_became_peered":"2024-05-22T22:59:51.951522+0000","last_unstale":"2024-05-22T23:01:50.709546+0000","last_undegraded":"2024-05-22T23:01:50.709546+0000","last_fullsized":"2024-05-22T23:01:50.709546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:56:23.357290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.149669+0000","last_change":"2024-05-22T22:59:51.966141+0000","last_active":"2024-05-22T23:01:50.149669+0000","last_peered":"2024-05-22T23:01:50.149669+0000","last_clean":"2024-05-22T23:01:50.149669+0000","last_became_active":"2024-05-22T22:59:51.965798+0000","last_became_peered":"2024-05-22T22:59:51.965798+0000","last_unstale":"2024-05-22T23:01:50.149669+0000","last_undegraded":"2024-05-22T23:01:50.149669+0000","last_fullsized":"2024-05-22T23:01:50.149669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:23:58.282078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987121+0000","last_change":"2024-05-22T22:59:51.950425+0000","last_active":"2024-05-22T23:01:49.987121+0000","last_peered":"2024-05-22T23:01:49.987121+0000","last_clean":"2024-05-22T23:01:49.987121+0000","last_became_active":"2024-05-22T22:59:51.950071+0000","last_became_peered":"2024-05-22T22:59:51.950071+0000","last_unstale":"2024-05-22T23:01:49.987121+0000","last_undegraded":"2024-05-22T23:01:49.987121+0000","last_fullsized":"2024-05-22T23:01:49.987121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:42:43.913816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.962773+0000","last_change":"2024-05-22T23:00:46.582890+0000","last_active":"2024-05-22T23:01:49.962773+0000","last_peered":"2024-05-22T23:01:49.962773+0000","last_clean":"2024-05-22T23:01:49.962773+0000","last_became_active":"2024-05-22T23:00:46.582497+0000","last_became_peered":"2024-05-22T23:00:46.582497+0000","last_unstale":"2024-05-22T23:01:49.962773+0000","last_undegraded":"2024-05-22T23:01:49.962773+0000","last_fullsized":"2024-05-22T23:01:49.962773+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:37:39.420650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876005+0000","last_change":"2024-05-22T22:59:51.959583+0000","last_active":"2024-05-22T23:01:49.876005+0000","last_peered":"2024-05-22T23:01:49.876005+0000","last_clean":"2024-05-22T23:01:49.876005+0000","last_became_active":"2024-05-22T22:59:51.959260+0000","last_became_peered":"2024-05-22T22:59:51.959260+0000","last_unstale":"2024-05-22T23:01:49.876005+0000","last_undegraded":"2024-05-22T23:01:49.876005+0000","last_fullsized":"2024-05-22T23:01:49.876005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:55:48.960861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876284+0000","last_change":"2024-05-22T23:00:46.584888+0000","last_active":"2024-05-22T23:01:49.876284+0000","last_peered":"2024-05-22T23:01:49.876284+0000","last_clean":"2024-05-22T23:01:49.876284+0000","last_became_active":"2024-05-22T23:00:46.584534+0000","last_became_peered":"2024-05-22T23:00:46.584534+0000","last_unstale":"2024-05-22T23:01:49.876284+0000","last_undegraded":"2024-05-22T23:01:49.876284+0000","last_fullsized":"2024-05-22T23:01:49.876284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:01:14.551458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.149983+0000","last_change":"2024-05-22T22:59:51.956954+0000","last_active":"2024-05-22T23:01:50.149983+0000","last_peered":"2024-05-22T23:01:50.149983+0000","last_clean":"2024-05-22T23:01:50.149983+0000","last_became_active":"2024-05-22T22:59:51.956570+0000","last_became_peered":"2024-05-22T22:59:51.956570+0000","last_unstale":"2024-05-22T23:01:50.149983+0000","last_undegraded":"2024-05-22T23:01:50.149983+0000","last_fullsized":"2024-05-22T23:01:50.149983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:55:18.828023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.963250+0000","last_change":"2024-05-22T22:59:51.962371+0000","last_active":"2024-05-22T23:01:49.963250+0000","last_peered":"2024-05-22T23:01:49.963250+0000","last_clean":"2024-05-22T23:01:49.963250+0000","last_became_active":"2024-05-22T22:59:51.962036+0000","last_became_peered":"2024-05-22T22:59:51.962036+0000","last_unstale":"2024-05-22T23:01:49.963250+0000","last_undegraded":"2024-05-22T23:01:49.963250+0000","last_fullsized":"2024-05-22T23:01:49.963250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:27:21.745563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876335+0000","last_change":"2024-05-22T22:59:51.952446+0000","last_active":"2024-05-22T23:01:49.876335+0000","last_peered":"2024-05-22T23:01:49.876335+0000","last_clean":"2024-05-22T23:01:49.876335+0000","last_became_active":"2024-05-22T22:59:51.952115+0000","last_became_peered":"2024-05-22T22:59:51.952115+0000","last_unstale":"2024-05-22T23:01:49.876335+0000","last_undegraded":"2024-05-22T23:01:49.876335+0000","last_fullsized":"2024-05-22T23:01:49.876335+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:33:56.960951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.149532+0000","last_change":"2024-05-22T22:59:51.968457+0000","last_active":"2024-05-22T23:01:50.149532+0000","last_peered":"2024-05-22T23:01:50.149532+0000","last_clean":"2024-05-22T23:01:50.149532+0000","last_became_active":"2024-05-22T22:59:51.968070+0000","last_became_peered":"2024-05-22T22:59:51.968070+0000","last_unstale":"2024-05-22T23:01:50.149532+0000","last_undegraded":"2024-05-22T23:01:50.149532+0000","last_fullsized":"2024-05-22T23:01:50.149532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:07:31.595119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.709986+0000","last_change":"2024-05-22T22:59:51.957357+0000","last_active":"2024-05-22T23:01:50.709986+0000","last_peered":"2024-05-22T23:01:50.709986+0000","last_clean":"2024-05-22T23:01:50.709986+0000","last_became_active":"2024-05-22T22:59:51.957013+0000","last_became_peered":"2024-05-22T22:59:51.957013+0000","last_unstale":"2024-05-22T23:01:50.709986+0000","last_undegraded":"2024-05-22T23:01:50.709986+0000","last_fullsized":"2024-05-22T23:01:50.709986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:38:48.118178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.708864+0000","last_change":"2024-05-22T22:59:51.957084+0000","last_active":"2024-05-22T23:01:50.708864+0000","last_peered":"2024-05-22T23:01:50.708864+0000","last_clean":"2024-05-22T23:01:50.708864+0000","last_became_active":"2024-05-22T22:59:51.956787+0000","last_became_peered":"2024-05-22T22:59:51.956787+0000","last_unstale":"2024-05-22T23:01:50.708864+0000","last_undegraded":"2024-05-22T23:01:50.708864+0000","last_fullsized":"2024-05-22T23:01:50.708864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:30:40.578286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.708794+0000","last_change":"2024-05-22T22:59:51.956375+0000","last_active":"2024-05-22T23:01:50.708794+0000","last_peered":"2024-05-22T23:01:50.708794+0000","last_clean":"2024-05-22T23:01:50.708794+0000","last_became_active":"2024-05-22T22:59:51.956012+0000","last_became_peered":"2024-05-22T22:59:51.956012+0000","last_unstale":"2024-05-22T23:01:50.708794+0000","last_undegraded":"2024-05-22T23:01:50.708794+0000","last_fullsized":"2024-05-22T23:01:50.708794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:33:49.920327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.148419+0000","last_change":"2024-05-22T22:59:51.950287+0000","last_active":"2024-05-22T23:01:50.148419+0000","last_peered":"2024-05-22T23:01:50.148419+0000","last_clean":"2024-05-22T23:01:50.148419+0000","last_became_active":"2024-05-22T22:59:51.949962+0000","last_became_peered":"2024-05-22T22:59:51.949962+0000","last_unstale":"2024-05-22T23:01:50.148419+0000","last_undegraded":"2024-05-22T23:01:50.148419+0000","last_fullsized":"2024-05-22T23:01:50.148419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:33:20.674713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.214674+0000","last_change":"2024-05-22T22:59:51.952523+0000","last_active":"2024-05-22T23:01:50.214674+0000","last_peered":"2024-05-22T23:01:50.214674+0000","last_clean":"2024-05-22T23:01:50.214674+0000","last_became_active":"2024-05-22T22:59:51.952152+0000","last_became_peered":"2024-05-22T22:59:51.952152+0000","last_unstale":"2024-05-22T23:01:50.214674+0000","last_undegraded":"2024-05-22T23:01:50.214674+0000","last_fullsized":"2024-05-22T23:01:50.214674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:33:01.332667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876557+0000","last_change":"2024-05-22T22:59:51.960919+0000","last_active":"2024-05-22T23:01:49.876557+0000","last_peered":"2024-05-22T23:01:49.876557+0000","last_clean":"2024-05-22T23:01:49.876557+0000","last_became_active":"2024-05-22T22:59:51.960603+0000","last_became_peered":"2024-05-22T22:59:51.960603+0000","last_unstale":"2024-05-22T23:01:49.876557+0000","last_undegraded":"2024-05-22T23:01:49.876557+0000","last_fullsized":"2024-05-22T23:01:49.876557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:28:05.977612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876196+0000","last_change":"2024-05-22T22:59:51.949186+0000","last_active":"2024-05-22T23:01:49.876196+0000","last_peered":"2024-05-22T23:01:49.876196+0000","last_clean":"2024-05-22T23:01:49.876196+0000","last_became_active":"2024-05-22T22:59:51.948819+0000","last_became_peered":"2024-05-22T22:59:51.948819+0000","last_unstale":"2024-05-22T23:01:49.876196+0000","last_undegraded":"2024-05-22T23:01:49.876196+0000","last_fullsized":"2024-05-22T23:01:49.876196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:32:16.153838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876505+0000","last_change":"2024-05-22T22:59:51.592520+0000","last_active":"2024-05-22T23:01:49.876505+0000","last_peered":"2024-05-22T23:01:49.876505+0000","last_clean":"2024-05-22T23:01:49.876505+0000","last_became_active":"2024-05-22T22:59:51.592201+0000","last_became_peered":"2024-05-22T22:59:51.592201+0000","last_unstale":"2024-05-22T23:01:49.876505+0000","last_undegraded":"2024-05-22T23:01:49.876505+0000","last_fullsized":"2024-05-22T23:01:49.876505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:45:06.500085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876469+0000","last_change":"2024-05-22T23:00:46.585147+0000","last_active":"2024-05-22T23:01:49.876469+0000","last_peered":"2024-05-22T23:01:49.876469+0000","last_clean":"2024-05-22T23:01:49.876469+0000","last_became_active":"2024-05-22T23:00:46.584828+0000","last_became_peered":"2024-05-22T23:00:46.584828+0000","last_unstale":"2024-05-22T23:01:49.876469+0000","last_undegraded":"2024-05-22T23:01:49.876469+0000","last_fullsized":"2024-05-22T23:01:49.876469+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:28:29.641493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987420+0000","last_change":"2024-05-22T22:59:51.595383+0000","last_active":"2024-05-22T23:01:49.987420+0000","last_peered":"2024-05-22T23:01:49.987420+0000","last_clean":"2024-05-22T23:01:49.987420+0000","last_became_active":"2024-05-22T22:59:51.595076+0000","last_became_peered":"2024-05-22T22:59:51.595076+0000","last_unstale":"2024-05-22T23:01:49.987420+0000","last_undegraded":"2024-05-22T23:01:49.987420+0000","last_fullsized":"2024-05-22T23:01:49.987420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:29:57.205196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.709776+0000","last_change":"2024-05-22T22:59:51.952447+0000","last_active":"2024-05-22T23:01:50.709776+0000","last_peered":"2024-05-22T23:01:50.709776+0000","last_clean":"2024-05-22T23:01:50.709776+0000","last_became_active":"2024-05-22T22:59:51.951927+0000","last_became_peered":"2024-05-22T22:59:51.951927+0000","last_unstale":"2024-05-22T23:01:50.709776+0000","last_undegraded":"2024-05-22T23:01:50.709776+0000","last_fullsized":"2024-05-22T23:01:50.709776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:14:59.161404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987479+0000","last_change":"2024-05-22T22:59:51.597705+0000","last_active":"2024-05-22T23:01:49.987479+0000","last_peered":"2024-05-22T23:01:49.987479+0000","last_clean":"2024-05-22T23:01:49.987479+0000","last_became_active":"2024-05-22T22:59:51.597319+0000","last_became_peered":"2024-05-22T22:59:51.597319+0000","last_unstale":"2024-05-22T23:01:49.987479+0000","last_undegraded":"2024-05-22T23:01:49.987479+0000","last_fullsized":"2024-05-22T23:01:49.987479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:59:59.307499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.149017+0000","last_change":"2024-05-22T22:59:51.955477+0000","last_active":"2024-05-22T23:01:50.149017+0000","last_peered":"2024-05-22T23:01:50.149017+0000","last_clean":"2024-05-22T23:01:50.149017+0000","last_became_active":"2024-05-22T22:59:51.953608+0000","last_became_peered":"2024-05-22T22:59:51.953608+0000","last_unstale":"2024-05-22T23:01:50.149017+0000","last_undegraded":"2024-05-22T23:01:50.149017+0000","last_fullsized":"2024-05-22T23:01:50.149017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:27:35.192556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987286+0000","last_change":"2024-05-22T22:59:51.595417+0000","last_active":"2024-05-22T23:01:49.987286+0000","last_peered":"2024-05-22T23:01:49.987286+0000","last_clean":"2024-05-22T23:01:49.987286+0000","last_became_active":"2024-05-22T22:59:51.595051+0000","last_became_peered":"2024-05-22T22:59:51.595051+0000","last_unstale":"2024-05-22T23:01:49.987286+0000","last_undegraded":"2024-05-22T23:01:49.987286+0000","last_fullsized":"2024-05-22T23:01:49.987286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:13:47.936348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.968175+0000","last_change":"2024-05-22T22:59:51.958471+0000","last_active":"2024-05-22T23:01:49.968175+0000","last_peered":"2024-05-22T23:01:49.968175+0000","last_clean":"2024-05-22T23:01:49.968175+0000","last_became_active":"2024-05-22T22:59:51.958017+0000","last_became_peered":"2024-05-22T22:59:51.958017+0000","last_unstale":"2024-05-22T23:01:49.968175+0000","last_undegraded":"2024-05-22T23:01:49.968175+0000","last_fullsized":"2024-05-22T23:01:49.968175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:42:52.196892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.968040+0000","last_change":"2024-05-22T22:59:51.591909+0000","last_active":"2024-05-22T23:01:49.968040+0000","last_peered":"2024-05-22T23:01:49.968040+0000","last_clean":"2024-05-22T23:01:49.968040+0000","last_became_active":"2024-05-22T22:59:51.591531+0000","last_became_peered":"2024-05-22T22:59:51.591531+0000","last_unstale":"2024-05-22T23:01:49.968040+0000","last_undegraded":"2024-05-22T23:01:49.968040+0000","last_fullsized":"2024-05-22T23:01:49.968040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:07:22.997156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987176+0000","last_change":"2024-05-22T23:00:46.581083+0000","last_active":"2024-05-22T23:01:49.987176+0000","last_peered":"2024-05-22T23:01:49.987176+0000","last_clean":"2024-05-22T23:01:49.987176+0000","last_became_active":"2024-05-22T23:00:46.580458+0000","last_became_peered":"2024-05-22T23:00:46.580458+0000","last_unstale":"2024-05-22T23:01:49.987176+0000","last_undegraded":"2024-05-22T23:01:49.987176+0000","last_fullsized":"2024-05-22T23:01:49.987176+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:38:53.136828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.962696+0000","last_change":"2024-05-22T22:59:51.590124+0000","last_active":"2024-05-22T23:01:49.962696+0000","last_peered":"2024-05-22T23:01:49.962696+0000","last_clean":"2024-05-22T23:01:49.962696+0000","last_became_active":"2024-05-22T22:59:51.589661+0000","last_became_peered":"2024-05-22T22:59:51.589661+0000","last_unstale":"2024-05-22T23:01:49.962696+0000","last_undegraded":"2024-05-22T23:01:49.962696+0000","last_fullsized":"2024-05-22T23:01:49.962696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:33:14.293487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.968174+0000","last_change":"2024-05-22T23:00:46.873993+0000","last_active":"2024-05-22T23:01:49.968174+0000","last_peered":"2024-05-22T23:01:49.968174+0000","last_clean":"2024-05-22T23:01:49.968174+0000","last_became_active":"2024-05-22T23:00:46.873523+0000","last_became_peered":"2024-05-22T23:00:46.873523+0000","last_unstale":"2024-05-22T23:01:49.968174+0000","last_undegraded":"2024-05-22T23:01:49.968174+0000","last_fullsized":"2024-05-22T23:01:49.968174+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:39:20.586352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.709814+0000","last_change":"2024-05-22T22:59:51.595510+0000","last_active":"2024-05-22T23:01:50.709814+0000","last_peered":"2024-05-22T23:01:50.709814+0000","last_clean":"2024-05-22T23:01:50.709814+0000","last_became_active":"2024-05-22T22:59:51.595208+0000","last_became_peered":"2024-05-22T22:59:51.595208+0000","last_unstale":"2024-05-22T23:01:50.709814+0000","last_undegraded":"2024-05-22T23:01:50.709814+0000","last_fullsized":"2024-05-22T23:01:50.709814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:47:46.957965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987160+0000","last_change":"2024-05-22T22:59:51.950684+0000","last_active":"2024-05-22T23:01:49.987160+0000","last_peered":"2024-05-22T23:01:49.987160+0000","last_clean":"2024-05-22T23:01:49.987160+0000","last_became_active":"2024-05-22T22:59:51.950380+0000","last_became_peered":"2024-05-22T22:59:51.950380+0000","last_unstale":"2024-05-22T23:01:49.987160+0000","last_undegraded":"2024-05-22T23:01:49.987160+0000","last_fullsized":"2024-05-22T23:01:49.987160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:58:02.035245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.967882+0000","last_change":"2024-05-22T22:59:51.592958+0000","last_active":"2024-05-22T23:01:49.967882+0000","last_peered":"2024-05-22T23:01:49.967882+0000","last_clean":"2024-05-22T23:01:49.967882+0000","last_became_active":"2024-05-22T22:59:51.592586+0000","last_became_peered":"2024-05-22T22:59:51.592586+0000","last_unstale":"2024-05-22T23:01:49.967882+0000","last_undegraded":"2024-05-22T23:01:49.967882+0000","last_fullsized":"2024-05-22T23:01:49.967882+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:44:43.859724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876289+0000","last_change":"2024-05-22T22:59:51.960863+0000","last_active":"2024-05-22T23:01:49.876289+0000","last_peered":"2024-05-22T23:01:49.876289+0000","last_clean":"2024-05-22T23:01:49.876289+0000","last_became_active":"2024-05-22T22:59:51.960537+0000","last_became_peered":"2024-05-22T22:59:51.960537+0000","last_unstale":"2024-05-22T23:01:49.876289+0000","last_undegraded":"2024-05-22T23:01:49.876289+0000","last_fullsized":"2024-05-22T23:01:49.876289+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:08:19.735397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.963270+0000","last_change":"2024-05-22T22:59:51.596150+0000","last_active":"2024-05-22T23:01:49.963270+0000","last_peered":"2024-05-22T23:01:49.963270+0000","last_clean":"2024-05-22T23:01:49.963270+0000","last_became_active":"2024-05-22T22:59:51.595791+0000","last_became_peered":"2024-05-22T22:59:51.595791+0000","last_unstale":"2024-05-22T23:01:49.963270+0000","last_undegraded":"2024-05-22T23:01:49.963270+0000","last_fullsized":"2024-05-22T23:01:49.963270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:47:46.144045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.149914+0000","last_change":"2024-05-22T22:59:51.951907+0000","last_active":"2024-05-22T23:01:50.149914+0000","last_peered":"2024-05-22T23:01:50.149914+0000","last_clean":"2024-05-22T23:01:50.149914+0000","last_became_active":"2024-05-22T22:59:51.951264+0000","last_became_peered":"2024-05-22T22:59:51.951264+0000","last_unstale":"2024-05-22T23:01:50.149914+0000","last_undegraded":"2024-05-22T23:01:50.149914+0000","last_fullsized":"2024-05-22T23:01:50.149914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:26:44.024848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876292+0000","last_change":"2024-05-22T22:59:51.594883+0000","last_active":"2024-05-22T23:01:49.876292+0000","last_peered":"2024-05-22T23:01:49.876292+0000","last_clean":"2024-05-22T23:01:49.876292+0000","last_became_active":"2024-05-22T22:59:51.594458+0000","last_became_peered":"2024-05-22T22:59:51.594458+0000","last_unstale":"2024-05-22T23:01:49.876292+0000","last_undegraded":"2024-05-22T23:01:49.876292+0000","last_fullsized":"2024-05-22T23:01:49.876292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:39:27.366961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.709769+0000","last_change":"2024-05-22T22:59:51.958180+0000","last_active":"2024-05-22T23:01:50.709769+0000","last_peered":"2024-05-22T23:01:50.709769+0000","last_clean":"2024-05-22T23:01:50.709769+0000","last_became_active":"2024-05-22T22:59:51.957877+0000","last_became_peered":"2024-05-22T22:59:51.957877+0000","last_unstale":"2024-05-22T23:01:50.709769+0000","last_undegraded":"2024-05-22T23:01:50.709769+0000","last_fullsized":"2024-05-22T23:01:50.709769+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:33:42.478340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.968255+0000","last_change":"2024-05-22T22:59:51.592953+0000","last_active":"2024-05-22T23:01:49.968255+0000","last_peered":"2024-05-22T23:01:49.968255+0000","last_clean":"2024-05-22T23:01:49.968255+0000","last_became_active":"2024-05-22T22:59:51.592525+0000","last_became_peered":"2024-05-22T22:59:51.592525+0000","last_unstale":"2024-05-22T23:01:49.968255+0000","last_undegraded":"2024-05-22T23:01:49.968255+0000","last_fullsized":"2024-05-22T23:01:49.968255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:18:03.512123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.149250+0000","last_change":"2024-05-22T22:59:51.959510+0000","last_active":"2024-05-22T23:01:50.149250+0000","last_peered":"2024-05-22T23:01:50.149250+0000","last_clean":"2024-05-22T23:01:50.149250+0000","last_became_active":"2024-05-22T22:59:51.959197+0000","last_became_peered":"2024-05-22T22:59:51.959197+0000","last_unstale":"2024-05-22T23:01:50.149250+0000","last_undegraded":"2024-05-22T23:01:50.149250+0000","last_fullsized":"2024-05-22T23:01:50.149250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:33:22.616668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'30","reported_seq":305,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.148971+0000","last_change":"2024-05-22T23:00:46.882566+0000","last_active":"2024-05-22T23:01:50.148971+0000","last_peered":"2024-05-22T23:01:50.148971+0000","last_clean":"2024-05-22T23:01:50.148971+0000","last_became_active":"2024-05-22T23:00:46.874658+0000","last_became_peered":"2024-05-22T23:00:46.874658+0000","last_unstale":"2024-05-22T23:01:50.148971+0000","last_undegraded":"2024-05-22T23:01:50.148971+0000","last_fullsized":"2024-05-22T23:01:50.148971+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:20:09.011734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65621,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":226,"num_read_kb":171,"num_write":30,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":58911,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987236+0000","last_change":"2024-05-22T22:59:51.952486+0000","last_active":"2024-05-22T23:01:49.987236+0000","last_peered":"2024-05-22T23:01:49.987236+0000","last_clean":"2024-05-22T23:01:49.987236+0000","last_became_active":"2024-05-22T22:59:51.952096+0000","last_became_peered":"2024-05-22T22:59:51.952096+0000","last_unstale":"2024-05-22T23:01:49.987236+0000","last_undegraded":"2024-05-22T23:01:49.987236+0000","last_fullsized":"2024-05-22T23:01:49.987236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:52:33.877655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.875961+0000","last_change":"2024-05-22T22:59:51.595446+0000","last_active":"2024-05-22T23:01:49.875961+0000","last_peered":"2024-05-22T23:01:49.875961+0000","last_clean":"2024-05-22T23:01:49.875961+0000","last_became_active":"2024-05-22T22:59:51.595097+0000","last_became_peered":"2024-05-22T22:59:51.595097+0000","last_unstale":"2024-05-22T23:01:49.875961+0000","last_undegraded":"2024-05-22T23:01:49.875961+0000","last_fullsized":"2024-05-22T23:01:49.875961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:03:29.722773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.709400+0000","last_change":"2024-05-22T22:59:51.953304+0000","last_active":"2024-05-22T23:01:50.709400+0000","last_peered":"2024-05-22T23:01:50.709400+0000","last_clean":"2024-05-22T23:01:50.709400+0000","last_became_active":"2024-05-22T22:59:51.952974+0000","last_became_peered":"2024-05-22T22:59:51.952974+0000","last_unstale":"2024-05-22T23:01:50.709400+0000","last_undegraded":"2024-05-22T23:01:50.709400+0000","last_fullsized":"2024-05-22T23:01:50.709400+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:21:06.116285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.967906+0000","last_change":"2024-05-22T22:59:51.593684+0000","last_active":"2024-05-22T23:01:49.967906+0000","last_peered":"2024-05-22T23:01:49.967906+0000","last_clean":"2024-05-22T23:01:49.967906+0000","last_became_active":"2024-05-22T22:59:51.593335+0000","last_became_peered":"2024-05-22T22:59:51.593335+0000","last_unstale":"2024-05-22T23:01:49.967906+0000","last_undegraded":"2024-05-22T23:01:49.967906+0000","last_fullsized":"2024-05-22T23:01:49.967906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:36:24.405609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876610+0000","last_change":"2024-05-22T22:59:51.960930+0000","last_active":"2024-05-22T23:01:49.876610+0000","last_peered":"2024-05-22T23:01:49.876610+0000","last_clean":"2024-05-22T23:01:49.876610+0000","last_became_active":"2024-05-22T22:59:51.960571+0000","last_became_peered":"2024-05-22T22:59:51.960571+0000","last_unstale":"2024-05-22T23:01:49.876610+0000","last_undegraded":"2024-05-22T23:01:49.876610+0000","last_fullsized":"2024-05-22T23:01:49.876610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:50:31.472449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.149029+0000","last_change":"2024-05-22T22:59:51.592876+0000","last_active":"2024-05-22T23:01:50.149029+0000","last_peered":"2024-05-22T23:01:50.149029+0000","last_clean":"2024-05-22T23:01:50.149029+0000","last_became_active":"2024-05-22T22:59:51.592488+0000","last_became_peered":"2024-05-22T22:59:51.592488+0000","last_unstale":"2024-05-22T23:01:50.149029+0000","last_undegraded":"2024-05-22T23:01:50.149029+0000","last_fullsized":"2024-05-22T23:01:50.149029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:37:16.517302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.875938+0000","last_change":"2024-05-22T22:59:51.962336+0000","last_active":"2024-05-22T23:01:49.875938+0000","last_peered":"2024-05-22T23:01:49.875938+0000","last_clean":"2024-05-22T23:01:49.875938+0000","last_became_active":"2024-05-22T22:59:51.962002+0000","last_became_peered":"2024-05-22T22:59:51.962002+0000","last_unstale":"2024-05-22T23:01:49.875938+0000","last_undegraded":"2024-05-22T23:01:49.875938+0000","last_fullsized":"2024-05-22T23:01:49.875938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:23:19.409418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.214265+0000","last_change":"2024-05-22T23:00:46.591734+0000","last_active":"2024-05-22T23:01:50.214265+0000","last_peered":"2024-05-22T23:01:50.214265+0000","last_clean":"2024-05-22T23:01:50.214265+0000","last_became_active":"2024-05-22T23:00:46.584075+0000","last_became_peered":"2024-05-22T23:00:46.584075+0000","last_unstale":"2024-05-22T23:01:50.214265+0000","last_undegraded":"2024-05-22T23:01:50.214265+0000","last_fullsized":"2024-05-22T23:01:50.214265+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:32:13.548197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.967837+0000","last_change":"2024-05-22T22:59:51.961600+0000","last_active":"2024-05-22T23:01:49.967837+0000","last_peered":"2024-05-22T23:01:49.967837+0000","last_clean":"2024-05-22T23:01:49.967837+0000","last_became_active":"2024-05-22T22:59:51.961286+0000","last_became_peered":"2024-05-22T22:59:51.961286+0000","last_unstale":"2024-05-22T23:01:49.967837+0000","last_undegraded":"2024-05-22T23:01:49.967837+0000","last_fullsized":"2024-05-22T23:01:49.967837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:04:26.992865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987338+0000","last_change":"2024-05-22T22:59:51.594461+0000","last_active":"2024-05-22T23:01:49.987338+0000","last_peered":"2024-05-22T23:01:49.987338+0000","last_clean":"2024-05-22T23:01:49.987338+0000","last_became_active":"2024-05-22T22:59:51.594128+0000","last_became_peered":"2024-05-22T22:59:51.594128+0000","last_unstale":"2024-05-22T23:01:49.987338+0000","last_undegraded":"2024-05-22T23:01:49.987338+0000","last_fullsized":"2024-05-22T23:01:49.987338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:52:05.095814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.214429+0000","last_change":"2024-05-22T22:59:51.958851+0000","last_active":"2024-05-22T23:01:50.214429+0000","last_peered":"2024-05-22T23:01:50.214429+0000","last_clean":"2024-05-22T23:01:50.214429+0000","last_became_active":"2024-05-22T22:59:51.957412+0000","last_became_peered":"2024-05-22T22:59:51.957412+0000","last_unstale":"2024-05-22T23:01:50.214429+0000","last_undegraded":"2024-05-22T23:01:50.214429+0000","last_fullsized":"2024-05-22T23:01:50.214429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:13:02.761205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876410+0000","last_change":"2024-05-22T22:59:51.588750+0000","last_active":"2024-05-22T23:01:49.876410+0000","last_peered":"2024-05-22T23:01:49.876410+0000","last_clean":"2024-05-22T23:01:49.876410+0000","last_became_active":"2024-05-22T22:59:51.588364+0000","last_became_peered":"2024-05-22T22:59:51.588364+0000","last_unstale":"2024-05-22T23:01:49.876410+0000","last_undegraded":"2024-05-22T23:01:49.876410+0000","last_fullsized":"2024-05-22T23:01:49.876410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:28:41.605406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.962928+0000","last_change":"2024-05-22T22:59:51.964676+0000","last_active":"2024-05-22T23:01:49.962928+0000","last_peered":"2024-05-22T23:01:49.962928+0000","last_clean":"2024-05-22T23:01:49.962928+0000","last_became_active":"2024-05-22T22:59:51.960472+0000","last_became_peered":"2024-05-22T22:59:51.960472+0000","last_unstale":"2024-05-22T23:01:49.962928+0000","last_undegraded":"2024-05-22T23:01:49.962928+0000","last_fullsized":"2024-05-22T23:01:49.962928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:24:55.042637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876125+0000","last_change":"2024-05-22T22:59:51.600873+0000","last_active":"2024-05-22T23:01:49.876125+0000","last_peered":"2024-05-22T23:01:49.876125+0000","last_clean":"2024-05-22T23:01:49.876125+0000","last_became_active":"2024-05-22T22:59:51.599785+0000","last_became_peered":"2024-05-22T22:59:51.599785+0000","last_unstale":"2024-05-22T23:01:49.876125+0000","last_undegraded":"2024-05-22T23:01:49.876125+0000","last_fullsized":"2024-05-22T23:01:49.876125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:20:09.590765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.962927+0000","last_change":"2024-05-22T22:59:51.964942+0000","last_active":"2024-05-22T23:01:49.962927+0000","last_peered":"2024-05-22T23:01:49.962927+0000","last_clean":"2024-05-22T23:01:49.962927+0000","last_became_active":"2024-05-22T22:59:51.964333+0000","last_became_peered":"2024-05-22T22:59:51.964333+0000","last_unstale":"2024-05-22T23:01:49.962927+0000","last_undegraded":"2024-05-22T23:01:49.962927+0000","last_fullsized":"2024-05-22T23:01:49.962927+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:01:07.215075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.968033+0000","last_change":"2024-05-22T22:59:51.957291+0000","last_active":"2024-05-22T23:01:49.968033+0000","last_peered":"2024-05-22T23:01:49.968033+0000","last_clean":"2024-05-22T23:01:49.968033+0000","last_became_active":"2024-05-22T22:59:51.956987+0000","last_became_peered":"2024-05-22T22:59:51.956987+0000","last_unstale":"2024-05-22T23:01:49.968033+0000","last_undegraded":"2024-05-22T23:01:49.968033+0000","last_fullsized":"2024-05-22T23:01:49.968033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:14:10.420756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.709738+0000","last_change":"2024-05-22T22:59:51.590255+0000","last_active":"2024-05-22T23:01:50.709738+0000","last_peered":"2024-05-22T23:01:50.709738+0000","last_clean":"2024-05-22T23:01:50.709738+0000","last_became_active":"2024-05-22T22:59:51.589261+0000","last_became_peered":"2024-05-22T22:59:51.589261+0000","last_unstale":"2024-05-22T23:01:50.709738+0000","last_undegraded":"2024-05-22T23:01:50.709738+0000","last_fullsized":"2024-05-22T23:01:50.709738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:15:14.171311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987623+0000","last_change":"2024-05-22T22:59:51.961453+0000","last_active":"2024-05-22T23:01:49.987623+0000","last_peered":"2024-05-22T23:01:49.987623+0000","last_clean":"2024-05-22T23:01:49.987623+0000","last_became_active":"2024-05-22T22:59:51.961082+0000","last_became_peered":"2024-05-22T22:59:51.961082+0000","last_unstale":"2024-05-22T23:01:49.987623+0000","last_undegraded":"2024-05-22T23:01:49.987623+0000","last_fullsized":"2024-05-22T23:01:49.987623+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:05:50.310688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.149332+0000","last_change":"2024-05-22T22:59:51.595913+0000","last_active":"2024-05-22T23:01:50.149332+0000","last_peered":"2024-05-22T23:01:50.149332+0000","last_clean":"2024-05-22T23:01:50.149332+0000","last_became_active":"2024-05-22T22:59:51.595561+0000","last_became_peered":"2024-05-22T22:59:51.595561+0000","last_unstale":"2024-05-22T23:01:50.149332+0000","last_undegraded":"2024-05-22T23:01:50.149332+0000","last_fullsized":"2024-05-22T23:01:50.149332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:37:04.701575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":357,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876032+0000","last_change":"2024-05-22T22:58:34.927654+0000","last_active":"2024-05-22T23:01:49.876032+0000","last_peered":"2024-05-22T23:01:49.876032+0000","last_clean":"2024-05-22T23:01:49.876032+0000","last_became_active":"2024-05-22T22:58:34.927090+0000","last_became_peered":"2024-05-22T22:58:34.927090+0000","last_unstale":"2024-05-22T23:01:49.876032+0000","last_undegraded":"2024-05-22T23:01:49.876032+0000","last_fullsized":"2024-05-22T23:01:49.876032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:55:07.719852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:55:07.719852+0000","last_clean_scrub_stamp":"2024-05-22T22:55:07.719852+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:40:53.763146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.709675+0000","last_change":"2024-05-22T22:59:51.956422+0000","last_active":"2024-05-22T23:01:50.709675+0000","last_peered":"2024-05-22T23:01:50.709675+0000","last_clean":"2024-05-22T23:01:50.709675+0000","last_became_active":"2024-05-22T22:59:51.956127+0000","last_became_peered":"2024-05-22T22:59:51.956127+0000","last_unstale":"2024-05-22T23:01:50.709675+0000","last_undegraded":"2024-05-22T23:01:50.709675+0000","last_fullsized":"2024-05-22T23:01:50.709675+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:04:36.955644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876146+0000","last_change":"2024-05-22T23:00:46.874231+0000","last_active":"2024-05-22T23:01:49.876146+0000","last_peered":"2024-05-22T23:01:49.876146+0000","last_clean":"2024-05-22T23:01:49.876146+0000","last_became_active":"2024-05-22T23:00:46.873728+0000","last_became_peered":"2024-05-22T23:00:46.873728+0000","last_unstale":"2024-05-22T23:01:49.876146+0000","last_undegraded":"2024-05-22T23:01:49.876146+0000","last_fullsized":"2024-05-22T23:01:49.876146+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:49:12.372179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987349+0000","last_change":"2024-05-22T22:59:51.598772+0000","last_active":"2024-05-22T23:01:49.987349+0000","last_peered":"2024-05-22T23:01:49.987349+0000","last_clean":"2024-05-22T23:01:49.987349+0000","last_became_active":"2024-05-22T22:59:51.598192+0000","last_became_peered":"2024-05-22T22:59:51.598192+0000","last_unstale":"2024-05-22T23:01:49.987349+0000","last_undegraded":"2024-05-22T23:01:49.987349+0000","last_fullsized":"2024-05-22T23:01:49.987349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:42:47.069554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.710017+0000","last_change":"2024-05-22T22:59:51.942337+0000","last_active":"2024-05-22T23:01:50.710017+0000","last_peered":"2024-05-22T23:01:50.710017+0000","last_clean":"2024-05-22T23:01:50.710017+0000","last_became_active":"2024-05-22T22:59:51.941939+0000","last_became_peered":"2024-05-22T22:59:51.941939+0000","last_unstale":"2024-05-22T23:01:50.710017+0000","last_undegraded":"2024-05-22T23:01:50.710017+0000","last_fullsized":"2024-05-22T23:01:50.710017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:35:42.360128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'43","reported_seq":87,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.110236+0000","last_change":"2024-05-22T22:59:51.589640+0000","last_active":"2024-05-22T23:01:50.110236+0000","last_peered":"2024-05-22T23:01:50.110236+0000","last_clean":"2024-05-22T23:01:50.110236+0000","last_became_active":"2024-05-22T22:59:51.589310+0000","last_became_peered":"2024-05-22T22:59:51.589310+0000","last_unstale":"2024-05-22T23:01:50.110236+0000","last_undegraded":"2024-05-22T23:01:50.110236+0000","last_fullsized":"2024-05-22T23:01:50.110236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:54:39.182002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":77,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987475+0000","last_change":"2024-05-22T22:59:51.952423+0000","last_active":"2024-05-22T23:01:49.987475+0000","last_peered":"2024-05-22T23:01:49.987475+0000","last_clean":"2024-05-22T23:01:49.987475+0000","last_became_active":"2024-05-22T22:59:51.952015+0000","last_became_peered":"2024-05-22T22:59:51.952015+0000","last_unstale":"2024-05-22T23:01:49.987475+0000","last_undegraded":"2024-05-22T23:01:49.987475+0000","last_fullsized":"2024-05-22T23:01:49.987475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:06:23.705248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.149938+0000","last_change":"2024-05-22T22:59:51.594882+0000","last_active":"2024-05-22T23:01:50.149938+0000","last_peered":"2024-05-22T23:01:50.149938+0000","last_clean":"2024-05-22T23:01:50.149938+0000","last_became_active":"2024-05-22T22:59:51.594552+0000","last_became_peered":"2024-05-22T22:59:51.594552+0000","last_unstale":"2024-05-22T23:01:50.149938+0000","last_undegraded":"2024-05-22T23:01:50.149938+0000","last_fullsized":"2024-05-22T23:01:50.149938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:06:40.771969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.709570+0000","last_change":"2024-05-22T22:59:51.956534+0000","last_active":"2024-05-22T23:01:50.709570+0000","last_peered":"2024-05-22T23:01:50.709570+0000","last_clean":"2024-05-22T23:01:50.709570+0000","last_became_active":"2024-05-22T22:59:51.956037+0000","last_became_peered":"2024-05-22T22:59:51.956037+0000","last_unstale":"2024-05-22T23:01:50.709570+0000","last_undegraded":"2024-05-22T23:01:50.709570+0000","last_fullsized":"2024-05-22T23:01:50.709570+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:24:15.289570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.148587+0000","last_change":"2024-05-22T22:59:51.592916+0000","last_active":"2024-05-22T23:01:50.148587+0000","last_peered":"2024-05-22T23:01:50.148587+0000","last_clean":"2024-05-22T23:01:50.148587+0000","last_became_active":"2024-05-22T22:59:51.592530+0000","last_became_peered":"2024-05-22T22:59:51.592530+0000","last_unstale":"2024-05-22T23:01:50.148587+0000","last_undegraded":"2024-05-22T23:01:50.148587+0000","last_fullsized":"2024-05-22T23:01:50.148587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:56:18.660120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987022+0000","last_change":"2024-05-22T22:59:51.956188+0000","last_active":"2024-05-22T23:01:49.987022+0000","last_peered":"2024-05-22T23:01:49.987022+0000","last_clean":"2024-05-22T23:01:49.987022+0000","last_became_active":"2024-05-22T22:59:51.955844+0000","last_became_peered":"2024-05-22T22:59:51.955844+0000","last_unstale":"2024-05-22T23:01:49.987022+0000","last_undegraded":"2024-05-22T23:01:49.987022+0000","last_fullsized":"2024-05-22T23:01:49.987022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:09:12.848822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987300+0000","last_change":"2024-05-22T22:59:51.590641+0000","last_active":"2024-05-22T23:01:49.987300+0000","last_peered":"2024-05-22T23:01:49.987300+0000","last_clean":"2024-05-22T23:01:49.987300+0000","last_became_active":"2024-05-22T22:59:51.590321+0000","last_became_peered":"2024-05-22T22:59:51.590321+0000","last_unstale":"2024-05-22T23:01:49.987300+0000","last_undegraded":"2024-05-22T23:01:49.987300+0000","last_fullsized":"2024-05-22T23:01:49.987300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:46:15.157160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.214400+0000","last_change":"2024-05-22T22:59:51.950256+0000","last_active":"2024-05-22T23:01:50.214400+0000","last_peered":"2024-05-22T23:01:50.214400+0000","last_clean":"2024-05-22T23:01:50.214400+0000","last_became_active":"2024-05-22T22:59:51.949746+0000","last_became_peered":"2024-05-22T22:59:51.949746+0000","last_unstale":"2024-05-22T23:01:50.214400+0000","last_undegraded":"2024-05-22T23:01:50.214400+0000","last_fullsized":"2024-05-22T23:01:50.214400+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:33:12.146425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.214297+0000","last_change":"2024-05-22T23:00:46.874312+0000","last_active":"2024-05-22T23:01:50.214297+0000","last_peered":"2024-05-22T23:01:50.214297+0000","last_clean":"2024-05-22T23:01:50.214297+0000","last_became_active":"2024-05-22T23:00:46.873803+0000","last_became_peered":"2024-05-22T23:00:46.873803+0000","last_unstale":"2024-05-22T23:01:50.214297+0000","last_undegraded":"2024-05-22T23:01:50.214297+0000","last_fullsized":"2024-05-22T23:01:50.214297+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:44:48.915692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987564+0000","last_change":"2024-05-22T22:59:51.968432+0000","last_active":"2024-05-22T23:01:49.987564+0000","last_peered":"2024-05-22T23:01:49.987564+0000","last_clean":"2024-05-22T23:01:49.987564+0000","last_became_active":"2024-05-22T22:59:51.968077+0000","last_became_peered":"2024-05-22T22:59:51.968077+0000","last_unstale":"2024-05-22T23:01:49.987564+0000","last_undegraded":"2024-05-22T23:01:49.987564+0000","last_fullsized":"2024-05-22T23:01:49.987564+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:08:16.281671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987132+0000","last_change":"2024-05-22T22:59:51.601616+0000","last_active":"2024-05-22T23:01:49.987132+0000","last_peered":"2024-05-22T23:01:49.987132+0000","last_clean":"2024-05-22T23:01:49.987132+0000","last_became_active":"2024-05-22T22:59:51.601249+0000","last_became_peered":"2024-05-22T22:59:51.601249+0000","last_unstale":"2024-05-22T23:01:49.987132+0000","last_undegraded":"2024-05-22T23:01:49.987132+0000","last_fullsized":"2024-05-22T23:01:49.987132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:07:07.030142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876196+0000","last_change":"2024-05-22T22:59:51.951578+0000","last_active":"2024-05-22T23:01:49.876196+0000","last_peered":"2024-05-22T23:01:49.876196+0000","last_clean":"2024-05-22T23:01:49.876196+0000","last_became_active":"2024-05-22T22:59:51.951233+0000","last_became_peered":"2024-05-22T22:59:51.951233+0000","last_unstale":"2024-05-22T23:01:49.876196+0000","last_undegraded":"2024-05-22T23:01:49.876196+0000","last_fullsized":"2024-05-22T23:01:49.876196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:25.882943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876127+0000","last_change":"2024-05-22T22:59:51.597819+0000","last_active":"2024-05-22T23:01:49.876127+0000","last_peered":"2024-05-22T23:01:49.876127+0000","last_clean":"2024-05-22T23:01:49.876127+0000","last_became_active":"2024-05-22T22:59:51.597262+0000","last_became_peered":"2024-05-22T22:59:51.597262+0000","last_unstale":"2024-05-22T23:01:49.876127+0000","last_undegraded":"2024-05-22T23:01:49.876127+0000","last_fullsized":"2024-05-22T23:01:49.876127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:05:02.849623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876421+0000","last_change":"2024-05-22T22:59:51.951668+0000","last_active":"2024-05-22T23:01:49.876421+0000","last_peered":"2024-05-22T23:01:49.876421+0000","last_clean":"2024-05-22T23:01:49.876421+0000","last_became_active":"2024-05-22T22:59:51.951176+0000","last_became_peered":"2024-05-22T22:59:51.951176+0000","last_unstale":"2024-05-22T23:01:49.876421+0000","last_undegraded":"2024-05-22T23:01:49.876421+0000","last_fullsized":"2024-05-22T23:01:49.876421+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:01:46.010858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'39","reported_seq":68,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.104832+0000","last_change":"2024-05-22T22:59:51.593144+0000","last_active":"2024-05-22T23:01:50.104832+0000","last_peered":"2024-05-22T23:01:50.104832+0000","last_clean":"2024-05-22T23:01:50.104832+0000","last_became_active":"2024-05-22T22:59:51.592557+0000","last_became_peered":"2024-05-22T22:59:51.592557+0000","last_unstale":"2024-05-22T23:01:50.104832+0000","last_undegraded":"2024-05-22T23:01:50.104832+0000","last_fullsized":"2024-05-22T23:01:50.104832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:28:19.403419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876262+0000","last_change":"2024-05-22T22:59:51.958866+0000","last_active":"2024-05-22T23:01:49.876262+0000","last_peered":"2024-05-22T23:01:49.876262+0000","last_clean":"2024-05-22T23:01:49.876262+0000","last_became_active":"2024-05-22T22:59:51.958505+0000","last_became_peered":"2024-05-22T22:59:51.958505+0000","last_unstale":"2024-05-22T23:01:49.876262+0000","last_undegraded":"2024-05-22T23:01:49.876262+0000","last_fullsized":"2024-05-22T23:01:49.876262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:54:24.601966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.710067+0000","last_change":"2024-05-22T22:59:51.587391+0000","last_active":"2024-05-22T23:01:50.710067+0000","last_peered":"2024-05-22T23:01:50.710067+0000","last_clean":"2024-05-22T23:01:50.710067+0000","last_became_active":"2024-05-22T22:59:51.587056+0000","last_became_peered":"2024-05-22T22:59:51.587056+0000","last_unstale":"2024-05-22T23:01:50.710067+0000","last_undegraded":"2024-05-22T23:01:50.710067+0000","last_fullsized":"2024-05-22T23:01:50.710067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:04:40.614853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876301+0000","last_change":"2024-05-22T22:59:51.953466+0000","last_active":"2024-05-22T23:01:49.876301+0000","last_peered":"2024-05-22T23:01:49.876301+0000","last_clean":"2024-05-22T23:01:49.876301+0000","last_became_active":"2024-05-22T22:59:51.952830+0000","last_became_peered":"2024-05-22T22:59:51.952830+0000","last_unstale":"2024-05-22T23:01:49.876301+0000","last_undegraded":"2024-05-22T23:01:49.876301+0000","last_fullsized":"2024-05-22T23:01:49.876301+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:36:06.314680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.214453+0000","last_change":"2024-05-22T22:59:51.962993+0000","last_active":"2024-05-22T23:01:50.214453+0000","last_peered":"2024-05-22T23:01:50.214453+0000","last_clean":"2024-05-22T23:01:50.214453+0000","last_became_active":"2024-05-22T22:59:51.962591+0000","last_became_peered":"2024-05-22T22:59:51.962591+0000","last_unstale":"2024-05-22T23:01:50.214453+0000","last_undegraded":"2024-05-22T23:01:50.214453+0000","last_fullsized":"2024-05-22T23:01:50.214453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:55:50.902537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.875914+0000","last_change":"2024-05-22T22:59:51.956304+0000","last_active":"2024-05-22T23:01:49.875914+0000","last_peered":"2024-05-22T23:01:49.875914+0000","last_clean":"2024-05-22T23:01:49.875914+0000","last_became_active":"2024-05-22T22:59:51.955933+0000","last_became_peered":"2024-05-22T22:59:51.955933+0000","last_unstale":"2024-05-22T23:01:49.875914+0000","last_undegraded":"2024-05-22T23:01:49.875914+0000","last_fullsized":"2024-05-22T23:01:49.875914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:28:17.225851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.875841+0000","last_change":"2024-05-22T22:59:51.946806+0000","last_active":"2024-05-22T23:01:49.875841+0000","last_peered":"2024-05-22T23:01:49.875841+0000","last_clean":"2024-05-22T23:01:49.875841+0000","last_became_active":"2024-05-22T22:59:51.946433+0000","last_became_peered":"2024-05-22T22:59:51.946433+0000","last_unstale":"2024-05-22T23:01:49.875841+0000","last_undegraded":"2024-05-22T23:01:49.875841+0000","last_fullsized":"2024-05-22T23:01:49.875841+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:24:06.013688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.709394+0000","last_change":"2024-05-22T22:59:51.956466+0000","last_active":"2024-05-22T23:01:50.709394+0000","last_peered":"2024-05-22T23:01:50.709394+0000","last_clean":"2024-05-22T23:01:50.709394+0000","last_became_active":"2024-05-22T22:59:51.956144+0000","last_became_peered":"2024-05-22T22:59:51.956144+0000","last_unstale":"2024-05-22T23:01:50.709394+0000","last_undegraded":"2024-05-22T23:01:50.709394+0000","last_fullsized":"2024-05-22T23:01:50.709394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:08:28.674824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.709256+0000","last_change":"2024-05-22T23:00:46.582147+0000","last_active":"2024-05-22T23:01:50.709256+0000","last_peered":"2024-05-22T23:01:50.709256+0000","last_clean":"2024-05-22T23:01:50.709256+0000","last_became_active":"2024-05-22T23:00:46.581804+0000","last_became_peered":"2024-05-22T23:00:46.581804+0000","last_unstale":"2024-05-22T23:01:50.709256+0000","last_undegraded":"2024-05-22T23:01:50.709256+0000","last_fullsized":"2024-05-22T23:01:50.709256+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:02:15.933895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.214476+0000","last_change":"2024-05-22T22:59:51.961971+0000","last_active":"2024-05-22T23:01:50.214476+0000","last_peered":"2024-05-22T23:01:50.214476+0000","last_clean":"2024-05-22T23:01:50.214476+0000","last_became_active":"2024-05-22T22:59:51.961604+0000","last_became_peered":"2024-05-22T22:59:51.961604+0000","last_unstale":"2024-05-22T23:01:50.214476+0000","last_undegraded":"2024-05-22T23:01:50.214476+0000","last_fullsized":"2024-05-22T23:01:50.214476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:32:14.669631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876247+0000","last_change":"2024-05-22T22:59:51.948173+0000","last_active":"2024-05-22T23:01:49.876247+0000","last_peered":"2024-05-22T23:01:49.876247+0000","last_clean":"2024-05-22T23:01:49.876247+0000","last_became_active":"2024-05-22T22:59:51.947839+0000","last_became_peered":"2024-05-22T22:59:51.947839+0000","last_unstale":"2024-05-22T23:01:49.876247+0000","last_undegraded":"2024-05-22T23:01:49.876247+0000","last_fullsized":"2024-05-22T23:01:49.876247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:55:07.748445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987341+0000","last_change":"2024-05-22T22:59:51.955627+0000","last_active":"2024-05-22T23:01:49.987341+0000","last_peered":"2024-05-22T23:01:49.987341+0000","last_clean":"2024-05-22T23:01:49.987341+0000","last_became_active":"2024-05-22T22:59:51.955284+0000","last_became_peered":"2024-05-22T22:59:51.955284+0000","last_unstale":"2024-05-22T23:01:49.987341+0000","last_undegraded":"2024-05-22T23:01:49.987341+0000","last_fullsized":"2024-05-22T23:01:49.987341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:57:13.029317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.967969+0000","last_change":"2024-05-22T22:59:51.955889+0000","last_active":"2024-05-22T23:01:49.967969+0000","last_peered":"2024-05-22T23:01:49.967969+0000","last_clean":"2024-05-22T23:01:49.967969+0000","last_became_active":"2024-05-22T22:59:51.954930+0000","last_became_peered":"2024-05-22T22:59:51.954930+0000","last_unstale":"2024-05-22T23:01:49.967969+0000","last_undegraded":"2024-05-22T23:01:49.967969+0000","last_fullsized":"2024-05-22T23:01:49.967969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:47.536072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987300+0000","last_change":"2024-05-22T22:59:51.954148+0000","last_active":"2024-05-22T23:01:49.987300+0000","last_peered":"2024-05-22T23:01:49.987300+0000","last_clean":"2024-05-22T23:01:49.987300+0000","last_became_active":"2024-05-22T22:59:51.952336+0000","last_became_peered":"2024-05-22T22:59:51.952336+0000","last_unstale":"2024-05-22T23:01:49.987300+0000","last_undegraded":"2024-05-22T23:01:49.987300+0000","last_fullsized":"2024-05-22T23:01:49.987300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:07:27.928910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.149653+0000","last_change":"2024-05-22T22:59:51.954300+0000","last_active":"2024-05-22T23:01:50.149653+0000","last_peered":"2024-05-22T23:01:50.149653+0000","last_clean":"2024-05-22T23:01:50.149653+0000","last_became_active":"2024-05-22T22:59:51.953779+0000","last_became_peered":"2024-05-22T22:59:51.953779+0000","last_unstale":"2024-05-22T23:01:50.149653+0000","last_undegraded":"2024-05-22T23:01:50.149653+0000","last_fullsized":"2024-05-22T23:01:50.149653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:24:59.363634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.710011+0000","last_change":"2024-05-22T22:59:51.947860+0000","last_active":"2024-05-22T23:01:50.710011+0000","last_peered":"2024-05-22T23:01:50.710011+0000","last_clean":"2024-05-22T23:01:50.710011+0000","last_became_active":"2024-05-22T22:59:51.947420+0000","last_became_peered":"2024-05-22T22:59:51.947420+0000","last_unstale":"2024-05-22T23:01:50.710011+0000","last_undegraded":"2024-05-22T23:01:50.710011+0000","last_fullsized":"2024-05-22T23:01:50.710011+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:20:40.629285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.214320+0000","last_change":"2024-05-22T22:59:51.957867+0000","last_active":"2024-05-22T23:01:50.214320+0000","last_peered":"2024-05-22T23:01:50.214320+0000","last_clean":"2024-05-22T23:01:50.214320+0000","last_became_active":"2024-05-22T22:59:51.957522+0000","last_became_peered":"2024-05-22T22:59:51.957522+0000","last_unstale":"2024-05-22T23:01:50.214320+0000","last_undegraded":"2024-05-22T23:01:50.214320+0000","last_fullsized":"2024-05-22T23:01:50.214320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:41:18.631970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107052,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":591235,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":324,"num_read_kb":256,"num_write":193,"num_write_kb":1399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":58911,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":364544,"data_stored":1773705,"data_compressed":89155,"data_compressed_allocated":118784,"data_compressed_original":1622016,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":962560,"data_stored":470148,"data_compressed":8653,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612651,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1118208,"data_stored":1258048,"data_compressed":52350,"data_compressed_allocated":69632,"data_compressed_original":745472,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776176,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":863946,"data_compressed":27799,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037812,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":863968,"data_compressed":27799,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299450,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1019904,"data_stored":1061215,"data_compressed":44337,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495679,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":461280,"data_compressed":5755,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1032192,"data_stored":535769,"data_compressed":10510,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":954368,"data_stored":461253,"data_compressed":5755,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":966656,"data_stored":470182,"data_compressed":8653,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":995302,"data_compressed":33955,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":535791,"data_compressed":18188,"data_compressed_allocated":24576,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757317,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1048576,"data_stored":1192415,"data_compressed":50493,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":65621,"data_compressed":1857,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":524920,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":65643,"data_compressed":9535,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":131782,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":393820,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":393798,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":197415,"data_compressed":8013,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-22T23:01:51.291 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-22T23:01:51.291 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T23:01:51.431 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.427+0000 7f397a9c1640 1 -- 172.21.15.84:0/4196774578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39741007b0 msgr2=0x7f3974100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:51.431 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.427+0000 7f397a9c1640 1 --2- 172.21.15.84:0/4196774578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39741007b0 0x7f3974100bd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3964009900 tx=0x7f396402a410 comp rx=0 tx=0).stop 2024-05-22T23:01:51.431 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.427+0000 7f397a9c1640 1 -- 172.21.15.84:0/4196774578 shutdown_connections 2024-05-22T23:01:51.431 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.427+0000 7f397a9c1640 1 --2- 172.21.15.84:0/4196774578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3974102360 0x7f3974108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.431 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.427+0000 7f397a9c1640 1 --2- 172.21.15.84:0/4196774578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39741019a0 0x7f3974101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.427+0000 7f397a9c1640 1 --2- 172.21.15.84:0/4196774578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39741007b0 0x7f3974100bd0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.427+0000 7f397a9c1640 1 -- 172.21.15.84:0/4196774578 >> 172.21.15.84:0/4196774578 conn(0x7f39740fbd20 msgr2=0x7f39740fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:51.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.427+0000 7f397a9c1640 1 -- 172.21.15.84:0/4196774578 shutdown_connections 2024-05-22T23:01:51.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.427+0000 7f397a9c1640 1 -- 172.21.15.84:0/4196774578 wait complete. 2024-05-22T23:01:51.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.427+0000 7f397a9c1640 1 Processor -- start 2024-05-22T23:01:51.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.428+0000 7f397a9c1640 1 -- start start 2024-05-22T23:01:51.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.428+0000 7f397a9c1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39741007b0 0x7f3974104740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:51.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.428+0000 7f397a9c1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39741019a0 0x7f3974102d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:51.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.428+0000 7f397a9c1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3974102360 0x7f39741032c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:51.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.428+0000 7f397a9c1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f397410c430 con 0x7f39741019a0 2024-05-22T23:01:51.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.428+0000 7f397a9c1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f397410c2b0 con 0x7f3974102360 2024-05-22T23:01:51.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.428+0000 7f397a9c1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f397410c5b0 con 0x7f39741007b0 2024-05-22T23:01:51.433 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.428+0000 7f39737fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39741019a0 0x7f3974102d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:51.433 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.428+0000 7f39737fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39741019a0 0x7f3974102d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38336/0 (socket says 172.21.15.84:38336) 2024-05-22T23:01:51.433 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.428+0000 7f39737fe640 1 -- 172.21.15.84:0/1933545643 learned_addr learned my addr 172.21.15.84:0/1933545643 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:51.433 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.428+0000 7f3973fff640 1 --2- 172.21.15.84:0/1933545643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39741007b0 0x7f3974104740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:51.433 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.428+0000 7f3978f5e640 1 --2- 172.21.15.84:0/1933545643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3974102360 0x7f39741032c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:51.433 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.428+0000 7f39737fe640 1 -- 172.21.15.84:0/1933545643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39741007b0 msgr2=0x7f3974104740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:51.433 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.428+0000 7f39737fe640 1 --2- 172.21.15.84:0/1933545643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39741007b0 0x7f3974104740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.433 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.428+0000 7f39737fe640 1 -- 172.21.15.84:0/1933545643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3974102360 msgr2=0x7f39741032c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:51.433 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.428+0000 7f39737fe640 1 --2- 172.21.15.84:0/1933545643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3974102360 0x7f39741032c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.433 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.428+0000 7f39737fe640 1 -- 172.21.15.84:0/1933545643 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3974103b50 con 0x7f39741019a0 2024-05-22T23:01:51.433 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.428+0000 7f3973fff640 1 --2- 172.21.15.84:0/1933545643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39741007b0 0x7f3974104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:01:51.433 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.428+0000 7f3978f5e640 1 --2- 172.21.15.84:0/1933545643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3974102360 0x7f39741032c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:01:51.433 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.429+0000 7f39737fe640 1 --2- 172.21.15.84:0/1933545643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39741019a0 0x7f3974102d80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f396800c920 tx=0x7f396800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:51.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.429+0000 7f39717fa640 1 -- 172.21.15.84:0/1933545643 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968007c70 con 0x7f39741019a0 2024-05-22T23:01:51.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.429+0000 7f397a9c1640 1 -- 172.21.15.84:0/1933545643 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39741a1cf0 con 0x7f39741019a0 2024-05-22T23:01:51.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.429+0000 7f39717fa640 1 -- 172.21.15.84:0/1933545643 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3968007e10 con 0x7f39741019a0 2024-05-22T23:01:51.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.429+0000 7f397a9c1640 1 -- 172.21.15.84:0/1933545643 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39741a21d0 con 0x7f39741019a0 2024-05-22T23:01:51.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.429+0000 7f39717fa640 1 -- 172.21.15.84:0/1933545643 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968018890 con 0x7f39741019a0 2024-05-22T23:01:51.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.430+0000 7f397a9c1640 1 -- 172.21.15.84:0/1933545643 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39741a24c0 con 0x7f39741019a0 2024-05-22T23:01:51.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.432+0000 7f39717fa640 1 -- 172.21.15.84:0/1933545643 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f3968018b10 con 0x7f39741019a0 2024-05-22T23:01:51.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.433+0000 7f39717fa640 1 --2- 172.21.15.84:0/1933545643 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f395c0a2990 0x7f395c0a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:51.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.433+0000 7f3973fff640 1 --2- 172.21.15.84:0/1933545643 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f395c0a2990 0x7f395c0a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:51.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.433+0000 7f39717fa640 1 -- 172.21.15.84:0/1933545643 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f39680cee90 con 0x7f39741019a0 2024-05-22T23:01:51.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.433+0000 7f3973fff640 1 --2- 172.21.15.84:0/1933545643 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f395c0a2990 0x7f395c0a4e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3964009230 tx=0x7f3964031400 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:51.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.437+0000 7f39717fa640 1 -- 172.21.15.84:0/1933545643 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3968014030 con 0x7f39741019a0 2024-05-22T23:01:51.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.617+0000 7f397a9c1640 1 -- 172.21.15.84:0/1933545643 --> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f39740fde70 con 0x7f395c0a2990 2024-05-22T23:01:51.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.624+0000 7f39717fa640 1 -- 172.21.15.84:0/1933545643 <== mgr.24851 v2:172.21.15.84:6800/3358259585 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257962 (secure 0 0 0) 0x7f39740fde70 con 0x7f395c0a2990 2024-05-22T23:01:51.628 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:51.629 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2024-05-22T23:01:51.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.627+0000 7f397a9c1640 1 -- 172.21.15.84:0/1933545643 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f395c0a2990 msgr2=0x7f395c0a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:51.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.627+0000 7f397a9c1640 1 --2- 172.21.15.84:0/1933545643 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f395c0a2990 0x7f395c0a4e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3964009230 tx=0x7f3964031400 comp rx=0 tx=0).stop 2024-05-22T23:01:51.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.627+0000 7f397a9c1640 1 -- 172.21.15.84:0/1933545643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39741019a0 msgr2=0x7f3974102d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:51.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.627+0000 7f397a9c1640 1 --2- 172.21.15.84:0/1933545643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39741019a0 0x7f3974102d80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f396800c920 tx=0x7f396800cde0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.627+0000 7f397a9c1640 1 -- 172.21.15.84:0/1933545643 shutdown_connections 2024-05-22T23:01:51.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.627+0000 7f397a9c1640 1 --2- 172.21.15.84:0/1933545643 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f395c0a2990 0x7f395c0a4e50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.627+0000 7f397a9c1640 1 --2- 172.21.15.84:0/1933545643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3974102360 0x7f39741032c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.627+0000 7f397a9c1640 1 --2- 172.21.15.84:0/1933545643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39741019a0 0x7f3974102d80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.627+0000 7f397a9c1640 1 --2- 172.21.15.84:0/1933545643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39741007b0 0x7f3974104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.627+0000 7f397a9c1640 1 -- 172.21.15.84:0/1933545643 >> 172.21.15.84:0/1933545643 conn(0x7f39740fbd20 msgr2=0x7f39740fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:51.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.627+0000 7f397a9c1640 1 -- 172.21.15.84:0/1933545643 shutdown_connections 2024-05-22T23:01:51.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.627+0000 7f397a9c1640 1 -- 172.21.15.84:0/1933545643 wait complete. 2024-05-22T23:01:51.640 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":3,"stamp":"2024-05-22T23:01:50.919930+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1181837,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":460,"num_read_kb":375,"num_write":376,"num_write_kb":4339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1239647,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":782,"ondisk_log_size":782,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385932,"kb_used_data":11780,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124555892,"statfs":{"total":1151940427776,"available":1151545233408,"internally_reserved":0,"allocated":12062720,"data_stored":9169317,"data_compressed":294247,"data_compressed_allocated":393216,"data_compressed_original":3670016,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.214487+0000","last_change":"2024-05-22T22:59:51.951625+0000","last_active":"2024-05-22T23:01:50.214487+0000","last_peered":"2024-05-22T23:01:50.214487+0000","last_clean":"2024-05-22T23:01:50.214487+0000","last_became_active":"2024-05-22T22:59:51.951225+0000","last_became_peered":"2024-05-22T22:59:51.951225+0000","last_unstale":"2024-05-22T23:01:50.214487+0000","last_undegraded":"2024-05-22T23:01:50.214487+0000","last_fullsized":"2024-05-22T23:01:50.214487+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:53:32.616083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.709529+0000","last_change":"2024-05-22T22:59:51.956925+0000","last_active":"2024-05-22T23:01:50.709529+0000","last_peered":"2024-05-22T23:01:50.709529+0000","last_clean":"2024-05-22T23:01:50.709529+0000","last_became_active":"2024-05-22T22:59:51.956557+0000","last_became_peered":"2024-05-22T22:59:51.956557+0000","last_unstale":"2024-05-22T23:01:50.709529+0000","last_undegraded":"2024-05-22T23:01:50.709529+0000","last_fullsized":"2024-05-22T23:01:50.709529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:18:14.609682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.709546+0000","last_change":"2024-05-22T22:59:51.951981+0000","last_active":"2024-05-22T23:01:50.709546+0000","last_peered":"2024-05-22T23:01:50.709546+0000","last_clean":"2024-05-22T23:01:50.709546+0000","last_became_active":"2024-05-22T22:59:51.951522+0000","last_became_peered":"2024-05-22T22:59:51.951522+0000","last_unstale":"2024-05-22T23:01:50.709546+0000","last_undegraded":"2024-05-22T23:01:50.709546+0000","last_fullsized":"2024-05-22T23:01:50.709546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:56:23.357290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.149669+0000","last_change":"2024-05-22T22:59:51.966141+0000","last_active":"2024-05-22T23:01:50.149669+0000","last_peered":"2024-05-22T23:01:50.149669+0000","last_clean":"2024-05-22T23:01:50.149669+0000","last_became_active":"2024-05-22T22:59:51.965798+0000","last_became_peered":"2024-05-22T22:59:51.965798+0000","last_unstale":"2024-05-22T23:01:50.149669+0000","last_undegraded":"2024-05-22T23:01:50.149669+0000","last_fullsized":"2024-05-22T23:01:50.149669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:23:58.282078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987121+0000","last_change":"2024-05-22T22:59:51.950425+0000","last_active":"2024-05-22T23:01:49.987121+0000","last_peered":"2024-05-22T23:01:49.987121+0000","last_clean":"2024-05-22T23:01:49.987121+0000","last_became_active":"2024-05-22T22:59:51.950071+0000","last_became_peered":"2024-05-22T22:59:51.950071+0000","last_unstale":"2024-05-22T23:01:49.987121+0000","last_undegraded":"2024-05-22T23:01:49.987121+0000","last_fullsized":"2024-05-22T23:01:49.987121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:42:43.913816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.962773+0000","last_change":"2024-05-22T23:00:46.582890+0000","last_active":"2024-05-22T23:01:49.962773+0000","last_peered":"2024-05-22T23:01:49.962773+0000","last_clean":"2024-05-22T23:01:49.962773+0000","last_became_active":"2024-05-22T23:00:46.582497+0000","last_became_peered":"2024-05-22T23:00:46.582497+0000","last_unstale":"2024-05-22T23:01:49.962773+0000","last_undegraded":"2024-05-22T23:01:49.962773+0000","last_fullsized":"2024-05-22T23:01:49.962773+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:37:39.420650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876005+0000","last_change":"2024-05-22T22:59:51.959583+0000","last_active":"2024-05-22T23:01:49.876005+0000","last_peered":"2024-05-22T23:01:49.876005+0000","last_clean":"2024-05-22T23:01:49.876005+0000","last_became_active":"2024-05-22T22:59:51.959260+0000","last_became_peered":"2024-05-22T22:59:51.959260+0000","last_unstale":"2024-05-22T23:01:49.876005+0000","last_undegraded":"2024-05-22T23:01:49.876005+0000","last_fullsized":"2024-05-22T23:01:49.876005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:55:48.960861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876284+0000","last_change":"2024-05-22T23:00:46.584888+0000","last_active":"2024-05-22T23:01:49.876284+0000","last_peered":"2024-05-22T23:01:49.876284+0000","last_clean":"2024-05-22T23:01:49.876284+0000","last_became_active":"2024-05-22T23:00:46.584534+0000","last_became_peered":"2024-05-22T23:00:46.584534+0000","last_unstale":"2024-05-22T23:01:49.876284+0000","last_undegraded":"2024-05-22T23:01:49.876284+0000","last_fullsized":"2024-05-22T23:01:49.876284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:01:14.551458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.149983+0000","last_change":"2024-05-22T22:59:51.956954+0000","last_active":"2024-05-22T23:01:50.149983+0000","last_peered":"2024-05-22T23:01:50.149983+0000","last_clean":"2024-05-22T23:01:50.149983+0000","last_became_active":"2024-05-22T22:59:51.956570+0000","last_became_peered":"2024-05-22T22:59:51.956570+0000","last_unstale":"2024-05-22T23:01:50.149983+0000","last_undegraded":"2024-05-22T23:01:50.149983+0000","last_fullsized":"2024-05-22T23:01:50.149983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:55:18.828023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.963250+0000","last_change":"2024-05-22T22:59:51.962371+0000","last_active":"2024-05-22T23:01:49.963250+0000","last_peered":"2024-05-22T23:01:49.963250+0000","last_clean":"2024-05-22T23:01:49.963250+0000","last_became_active":"2024-05-22T22:59:51.962036+0000","last_became_peered":"2024-05-22T22:59:51.962036+0000","last_unstale":"2024-05-22T23:01:49.963250+0000","last_undegraded":"2024-05-22T23:01:49.963250+0000","last_fullsized":"2024-05-22T23:01:49.963250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:27:21.745563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876335+0000","last_change":"2024-05-22T22:59:51.952446+0000","last_active":"2024-05-22T23:01:49.876335+0000","last_peered":"2024-05-22T23:01:49.876335+0000","last_clean":"2024-05-22T23:01:49.876335+0000","last_became_active":"2024-05-22T22:59:51.952115+0000","last_became_peered":"2024-05-22T22:59:51.952115+0000","last_unstale":"2024-05-22T23:01:49.876335+0000","last_undegraded":"2024-05-22T23:01:49.876335+0000","last_fullsized":"2024-05-22T23:01:49.876335+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:33:56.960951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.149532+0000","last_change":"2024-05-22T22:59:51.968457+0000","last_active":"2024-05-22T23:01:50.149532+0000","last_peered":"2024-05-22T23:01:50.149532+0000","last_clean":"2024-05-22T23:01:50.149532+0000","last_became_active":"2024-05-22T22:59:51.968070+0000","last_became_peered":"2024-05-22T22:59:51.968070+0000","last_unstale":"2024-05-22T23:01:50.149532+0000","last_undegraded":"2024-05-22T23:01:50.149532+0000","last_fullsized":"2024-05-22T23:01:50.149532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:07:31.595119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.709986+0000","last_change":"2024-05-22T22:59:51.957357+0000","last_active":"2024-05-22T23:01:50.709986+0000","last_peered":"2024-05-22T23:01:50.709986+0000","last_clean":"2024-05-22T23:01:50.709986+0000","last_became_active":"2024-05-22T22:59:51.957013+0000","last_became_peered":"2024-05-22T22:59:51.957013+0000","last_unstale":"2024-05-22T23:01:50.709986+0000","last_undegraded":"2024-05-22T23:01:50.709986+0000","last_fullsized":"2024-05-22T23:01:50.709986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:38:48.118178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.708864+0000","last_change":"2024-05-22T22:59:51.957084+0000","last_active":"2024-05-22T23:01:50.708864+0000","last_peered":"2024-05-22T23:01:50.708864+0000","last_clean":"2024-05-22T23:01:50.708864+0000","last_became_active":"2024-05-22T22:59:51.956787+0000","last_became_peered":"2024-05-22T22:59:51.956787+0000","last_unstale":"2024-05-22T23:01:50.708864+0000","last_undegraded":"2024-05-22T23:01:50.708864+0000","last_fullsized":"2024-05-22T23:01:50.708864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:30:40.578286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.708794+0000","last_change":"2024-05-22T22:59:51.956375+0000","last_active":"2024-05-22T23:01:50.708794+0000","last_peered":"2024-05-22T23:01:50.708794+0000","last_clean":"2024-05-22T23:01:50.708794+0000","last_became_active":"2024-05-22T22:59:51.956012+0000","last_became_peered":"2024-05-22T22:59:51.956012+0000","last_unstale":"2024-05-22T23:01:50.708794+0000","last_undegraded":"2024-05-22T23:01:50.708794+0000","last_fullsized":"2024-05-22T23:01:50.708794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:33:49.920327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.148419+0000","last_change":"2024-05-22T22:59:51.950287+0000","last_active":"2024-05-22T23:01:50.148419+0000","last_peered":"2024-05-22T23:01:50.148419+0000","last_clean":"2024-05-22T23:01:50.148419+0000","last_became_active":"2024-05-22T22:59:51.949962+0000","last_became_peered":"2024-05-22T22:59:51.949962+0000","last_unstale":"2024-05-22T23:01:50.148419+0000","last_undegraded":"2024-05-22T23:01:50.148419+0000","last_fullsized":"2024-05-22T23:01:50.148419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:33:20.674713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.214674+0000","last_change":"2024-05-22T22:59:51.952523+0000","last_active":"2024-05-22T23:01:50.214674+0000","last_peered":"2024-05-22T23:01:50.214674+0000","last_clean":"2024-05-22T23:01:50.214674+0000","last_became_active":"2024-05-22T22:59:51.952152+0000","last_became_peered":"2024-05-22T22:59:51.952152+0000","last_unstale":"2024-05-22T23:01:50.214674+0000","last_undegraded":"2024-05-22T23:01:50.214674+0000","last_fullsized":"2024-05-22T23:01:50.214674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:33:01.332667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876557+0000","last_change":"2024-05-22T22:59:51.960919+0000","last_active":"2024-05-22T23:01:49.876557+0000","last_peered":"2024-05-22T23:01:49.876557+0000","last_clean":"2024-05-22T23:01:49.876557+0000","last_became_active":"2024-05-22T22:59:51.960603+0000","last_became_peered":"2024-05-22T22:59:51.960603+0000","last_unstale":"2024-05-22T23:01:49.876557+0000","last_undegraded":"2024-05-22T23:01:49.876557+0000","last_fullsized":"2024-05-22T23:01:49.876557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:28:05.977612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876196+0000","last_change":"2024-05-22T22:59:51.949186+0000","last_active":"2024-05-22T23:01:49.876196+0000","last_peered":"2024-05-22T23:01:49.876196+0000","last_clean":"2024-05-22T23:01:49.876196+0000","last_became_active":"2024-05-22T22:59:51.948819+0000","last_became_peered":"2024-05-22T22:59:51.948819+0000","last_unstale":"2024-05-22T23:01:49.876196+0000","last_undegraded":"2024-05-22T23:01:49.876196+0000","last_fullsized":"2024-05-22T23:01:49.876196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:32:16.153838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876505+0000","last_change":"2024-05-22T22:59:51.592520+0000","last_active":"2024-05-22T23:01:49.876505+0000","last_peered":"2024-05-22T23:01:49.876505+0000","last_clean":"2024-05-22T23:01:49.876505+0000","last_became_active":"2024-05-22T22:59:51.592201+0000","last_became_peered":"2024-05-22T22:59:51.592201+0000","last_unstale":"2024-05-22T23:01:49.876505+0000","last_undegraded":"2024-05-22T23:01:49.876505+0000","last_fullsized":"2024-05-22T23:01:49.876505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:45:06.500085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876469+0000","last_change":"2024-05-22T23:00:46.585147+0000","last_active":"2024-05-22T23:01:49.876469+0000","last_peered":"2024-05-22T23:01:49.876469+0000","last_clean":"2024-05-22T23:01:49.876469+0000","last_became_active":"2024-05-22T23:00:46.584828+0000","last_became_peered":"2024-05-22T23:00:46.584828+0000","last_unstale":"2024-05-22T23:01:49.876469+0000","last_undegraded":"2024-05-22T23:01:49.876469+0000","last_fullsized":"2024-05-22T23:01:49.876469+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:28:29.641493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987420+0000","last_change":"2024-05-22T22:59:51.595383+0000","last_active":"2024-05-22T23:01:49.987420+0000","last_peered":"2024-05-22T23:01:49.987420+0000","last_clean":"2024-05-22T23:01:49.987420+0000","last_became_active":"2024-05-22T22:59:51.595076+0000","last_became_peered":"2024-05-22T22:59:51.595076+0000","last_unstale":"2024-05-22T23:01:49.987420+0000","last_undegraded":"2024-05-22T23:01:49.987420+0000","last_fullsized":"2024-05-22T23:01:49.987420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:29:57.205196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.709776+0000","last_change":"2024-05-22T22:59:51.952447+0000","last_active":"2024-05-22T23:01:50.709776+0000","last_peered":"2024-05-22T23:01:50.709776+0000","last_clean":"2024-05-22T23:01:50.709776+0000","last_became_active":"2024-05-22T22:59:51.951927+0000","last_became_peered":"2024-05-22T22:59:51.951927+0000","last_unstale":"2024-05-22T23:01:50.709776+0000","last_undegraded":"2024-05-22T23:01:50.709776+0000","last_fullsized":"2024-05-22T23:01:50.709776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:14:59.161404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987479+0000","last_change":"2024-05-22T22:59:51.597705+0000","last_active":"2024-05-22T23:01:49.987479+0000","last_peered":"2024-05-22T23:01:49.987479+0000","last_clean":"2024-05-22T23:01:49.987479+0000","last_became_active":"2024-05-22T22:59:51.597319+0000","last_became_peered":"2024-05-22T22:59:51.597319+0000","last_unstale":"2024-05-22T23:01:49.987479+0000","last_undegraded":"2024-05-22T23:01:49.987479+0000","last_fullsized":"2024-05-22T23:01:49.987479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:59:59.307499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.149017+0000","last_change":"2024-05-22T22:59:51.955477+0000","last_active":"2024-05-22T23:01:50.149017+0000","last_peered":"2024-05-22T23:01:50.149017+0000","last_clean":"2024-05-22T23:01:50.149017+0000","last_became_active":"2024-05-22T22:59:51.953608+0000","last_became_peered":"2024-05-22T22:59:51.953608+0000","last_unstale":"2024-05-22T23:01:50.149017+0000","last_undegraded":"2024-05-22T23:01:50.149017+0000","last_fullsized":"2024-05-22T23:01:50.149017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:27:35.192556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987286+0000","last_change":"2024-05-22T22:59:51.595417+0000","last_active":"2024-05-22T23:01:49.987286+0000","last_peered":"2024-05-22T23:01:49.987286+0000","last_clean":"2024-05-22T23:01:49.987286+0000","last_became_active":"2024-05-22T22:59:51.595051+0000","last_became_peered":"2024-05-22T22:59:51.595051+0000","last_unstale":"2024-05-22T23:01:49.987286+0000","last_undegraded":"2024-05-22T23:01:49.987286+0000","last_fullsized":"2024-05-22T23:01:49.987286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:13:47.936348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.968175+0000","last_change":"2024-05-22T22:59:51.958471+0000","last_active":"2024-05-22T23:01:49.968175+0000","last_peered":"2024-05-22T23:01:49.968175+0000","last_clean":"2024-05-22T23:01:49.968175+0000","last_became_active":"2024-05-22T22:59:51.958017+0000","last_became_peered":"2024-05-22T22:59:51.958017+0000","last_unstale":"2024-05-22T23:01:49.968175+0000","last_undegraded":"2024-05-22T23:01:49.968175+0000","last_fullsized":"2024-05-22T23:01:49.968175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:42:52.196892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.968040+0000","last_change":"2024-05-22T22:59:51.591909+0000","last_active":"2024-05-22T23:01:49.968040+0000","last_peered":"2024-05-22T23:01:49.968040+0000","last_clean":"2024-05-22T23:01:49.968040+0000","last_became_active":"2024-05-22T22:59:51.591531+0000","last_became_peered":"2024-05-22T22:59:51.591531+0000","last_unstale":"2024-05-22T23:01:49.968040+0000","last_undegraded":"2024-05-22T23:01:49.968040+0000","last_fullsized":"2024-05-22T23:01:49.968040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:07:22.997156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987176+0000","last_change":"2024-05-22T23:00:46.581083+0000","last_active":"2024-05-22T23:01:49.987176+0000","last_peered":"2024-05-22T23:01:49.987176+0000","last_clean":"2024-05-22T23:01:49.987176+0000","last_became_active":"2024-05-22T23:00:46.580458+0000","last_became_peered":"2024-05-22T23:00:46.580458+0000","last_unstale":"2024-05-22T23:01:49.987176+0000","last_undegraded":"2024-05-22T23:01:49.987176+0000","last_fullsized":"2024-05-22T23:01:49.987176+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:38:53.136828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.962696+0000","last_change":"2024-05-22T22:59:51.590124+0000","last_active":"2024-05-22T23:01:49.962696+0000","last_peered":"2024-05-22T23:01:49.962696+0000","last_clean":"2024-05-22T23:01:49.962696+0000","last_became_active":"2024-05-22T22:59:51.589661+0000","last_became_peered":"2024-05-22T22:59:51.589661+0000","last_unstale":"2024-05-22T23:01:49.962696+0000","last_undegraded":"2024-05-22T23:01:49.962696+0000","last_fullsized":"2024-05-22T23:01:49.962696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:33:14.293487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.968174+0000","last_change":"2024-05-22T23:00:46.873993+0000","last_active":"2024-05-22T23:01:49.968174+0000","last_peered":"2024-05-22T23:01:49.968174+0000","last_clean":"2024-05-22T23:01:49.968174+0000","last_became_active":"2024-05-22T23:00:46.873523+0000","last_became_peered":"2024-05-22T23:00:46.873523+0000","last_unstale":"2024-05-22T23:01:49.968174+0000","last_undegraded":"2024-05-22T23:01:49.968174+0000","last_fullsized":"2024-05-22T23:01:49.968174+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:39:20.586352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.709814+0000","last_change":"2024-05-22T22:59:51.595510+0000","last_active":"2024-05-22T23:01:50.709814+0000","last_peered":"2024-05-22T23:01:50.709814+0000","last_clean":"2024-05-22T23:01:50.709814+0000","last_became_active":"2024-05-22T22:59:51.595208+0000","last_became_peered":"2024-05-22T22:59:51.595208+0000","last_unstale":"2024-05-22T23:01:50.709814+0000","last_undegraded":"2024-05-22T23:01:50.709814+0000","last_fullsized":"2024-05-22T23:01:50.709814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:47:46.957965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987160+0000","last_change":"2024-05-22T22:59:51.950684+0000","last_active":"2024-05-22T23:01:49.987160+0000","last_peered":"2024-05-22T23:01:49.987160+0000","last_clean":"2024-05-22T23:01:49.987160+0000","last_became_active":"2024-05-22T22:59:51.950380+0000","last_became_peered":"2024-05-22T22:59:51.950380+0000","last_unstale":"2024-05-22T23:01:49.987160+0000","last_undegraded":"2024-05-22T23:01:49.987160+0000","last_fullsized":"2024-05-22T23:01:49.987160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:58:02.035245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.967882+0000","last_change":"2024-05-22T22:59:51.592958+0000","last_active":"2024-05-22T23:01:49.967882+0000","last_peered":"2024-05-22T23:01:49.967882+0000","last_clean":"2024-05-22T23:01:49.967882+0000","last_became_active":"2024-05-22T22:59:51.592586+0000","last_became_peered":"2024-05-22T22:59:51.592586+0000","last_unstale":"2024-05-22T23:01:49.967882+0000","last_undegraded":"2024-05-22T23:01:49.967882+0000","last_fullsized":"2024-05-22T23:01:49.967882+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:44:43.859724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876289+0000","last_change":"2024-05-22T22:59:51.960863+0000","last_active":"2024-05-22T23:01:49.876289+0000","last_peered":"2024-05-22T23:01:49.876289+0000","last_clean":"2024-05-22T23:01:49.876289+0000","last_became_active":"2024-05-22T22:59:51.960537+0000","last_became_peered":"2024-05-22T22:59:51.960537+0000","last_unstale":"2024-05-22T23:01:49.876289+0000","last_undegraded":"2024-05-22T23:01:49.876289+0000","last_fullsized":"2024-05-22T23:01:49.876289+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:08:19.735397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.963270+0000","last_change":"2024-05-22T22:59:51.596150+0000","last_active":"2024-05-22T23:01:49.963270+0000","last_peered":"2024-05-22T23:01:49.963270+0000","last_clean":"2024-05-22T23:01:49.963270+0000","last_became_active":"2024-05-22T22:59:51.595791+0000","last_became_peered":"2024-05-22T22:59:51.595791+0000","last_unstale":"2024-05-22T23:01:49.963270+0000","last_undegraded":"2024-05-22T23:01:49.963270+0000","last_fullsized":"2024-05-22T23:01:49.963270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:47:46.144045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.149914+0000","last_change":"2024-05-22T22:59:51.951907+0000","last_active":"2024-05-22T23:01:50.149914+0000","last_peered":"2024-05-22T23:01:50.149914+0000","last_clean":"2024-05-22T23:01:50.149914+0000","last_became_active":"2024-05-22T22:59:51.951264+0000","last_became_peered":"2024-05-22T22:59:51.951264+0000","last_unstale":"2024-05-22T23:01:50.149914+0000","last_undegraded":"2024-05-22T23:01:50.149914+0000","last_fullsized":"2024-05-22T23:01:50.149914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:26:44.024848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876292+0000","last_change":"2024-05-22T22:59:51.594883+0000","last_active":"2024-05-22T23:01:49.876292+0000","last_peered":"2024-05-22T23:01:49.876292+0000","last_clean":"2024-05-22T23:01:49.876292+0000","last_became_active":"2024-05-22T22:59:51.594458+0000","last_became_peered":"2024-05-22T22:59:51.594458+0000","last_unstale":"2024-05-22T23:01:49.876292+0000","last_undegraded":"2024-05-22T23:01:49.876292+0000","last_fullsized":"2024-05-22T23:01:49.876292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:39:27.366961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.709769+0000","last_change":"2024-05-22T22:59:51.958180+0000","last_active":"2024-05-22T23:01:50.709769+0000","last_peered":"2024-05-22T23:01:50.709769+0000","last_clean":"2024-05-22T23:01:50.709769+0000","last_became_active":"2024-05-22T22:59:51.957877+0000","last_became_peered":"2024-05-22T22:59:51.957877+0000","last_unstale":"2024-05-22T23:01:50.709769+0000","last_undegraded":"2024-05-22T23:01:50.709769+0000","last_fullsized":"2024-05-22T23:01:50.709769+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:33:42.478340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.968255+0000","last_change":"2024-05-22T22:59:51.592953+0000","last_active":"2024-05-22T23:01:49.968255+0000","last_peered":"2024-05-22T23:01:49.968255+0000","last_clean":"2024-05-22T23:01:49.968255+0000","last_became_active":"2024-05-22T22:59:51.592525+0000","last_became_peered":"2024-05-22T22:59:51.592525+0000","last_unstale":"2024-05-22T23:01:49.968255+0000","last_undegraded":"2024-05-22T23:01:49.968255+0000","last_fullsized":"2024-05-22T23:01:49.968255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:18:03.512123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.149250+0000","last_change":"2024-05-22T22:59:51.959510+0000","last_active":"2024-05-22T23:01:50.149250+0000","last_peered":"2024-05-22T23:01:50.149250+0000","last_clean":"2024-05-22T23:01:50.149250+0000","last_became_active":"2024-05-22T22:59:51.959197+0000","last_became_peered":"2024-05-22T22:59:51.959197+0000","last_unstale":"2024-05-22T23:01:50.149250+0000","last_undegraded":"2024-05-22T23:01:50.149250+0000","last_fullsized":"2024-05-22T23:01:50.149250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:33:22.616668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'30","reported_seq":305,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.148971+0000","last_change":"2024-05-22T23:00:46.882566+0000","last_active":"2024-05-22T23:01:50.148971+0000","last_peered":"2024-05-22T23:01:50.148971+0000","last_clean":"2024-05-22T23:01:50.148971+0000","last_became_active":"2024-05-22T23:00:46.874658+0000","last_became_peered":"2024-05-22T23:00:46.874658+0000","last_unstale":"2024-05-22T23:01:50.148971+0000","last_undegraded":"2024-05-22T23:01:50.148971+0000","last_fullsized":"2024-05-22T23:01:50.148971+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:20:09.011734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65621,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":226,"num_read_kb":171,"num_write":30,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":58911,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987236+0000","last_change":"2024-05-22T22:59:51.952486+0000","last_active":"2024-05-22T23:01:49.987236+0000","last_peered":"2024-05-22T23:01:49.987236+0000","last_clean":"2024-05-22T23:01:49.987236+0000","last_became_active":"2024-05-22T22:59:51.952096+0000","last_became_peered":"2024-05-22T22:59:51.952096+0000","last_unstale":"2024-05-22T23:01:49.987236+0000","last_undegraded":"2024-05-22T23:01:49.987236+0000","last_fullsized":"2024-05-22T23:01:49.987236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:52:33.877655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.875961+0000","last_change":"2024-05-22T22:59:51.595446+0000","last_active":"2024-05-22T23:01:49.875961+0000","last_peered":"2024-05-22T23:01:49.875961+0000","last_clean":"2024-05-22T23:01:49.875961+0000","last_became_active":"2024-05-22T22:59:51.595097+0000","last_became_peered":"2024-05-22T22:59:51.595097+0000","last_unstale":"2024-05-22T23:01:49.875961+0000","last_undegraded":"2024-05-22T23:01:49.875961+0000","last_fullsized":"2024-05-22T23:01:49.875961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:03:29.722773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.709400+0000","last_change":"2024-05-22T22:59:51.953304+0000","last_active":"2024-05-22T23:01:50.709400+0000","last_peered":"2024-05-22T23:01:50.709400+0000","last_clean":"2024-05-22T23:01:50.709400+0000","last_became_active":"2024-05-22T22:59:51.952974+0000","last_became_peered":"2024-05-22T22:59:51.952974+0000","last_unstale":"2024-05-22T23:01:50.709400+0000","last_undegraded":"2024-05-22T23:01:50.709400+0000","last_fullsized":"2024-05-22T23:01:50.709400+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:21:06.116285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.967906+0000","last_change":"2024-05-22T22:59:51.593684+0000","last_active":"2024-05-22T23:01:49.967906+0000","last_peered":"2024-05-22T23:01:49.967906+0000","last_clean":"2024-05-22T23:01:49.967906+0000","last_became_active":"2024-05-22T22:59:51.593335+0000","last_became_peered":"2024-05-22T22:59:51.593335+0000","last_unstale":"2024-05-22T23:01:49.967906+0000","last_undegraded":"2024-05-22T23:01:49.967906+0000","last_fullsized":"2024-05-22T23:01:49.967906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:36:24.405609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876610+0000","last_change":"2024-05-22T22:59:51.960930+0000","last_active":"2024-05-22T23:01:49.876610+0000","last_peered":"2024-05-22T23:01:49.876610+0000","last_clean":"2024-05-22T23:01:49.876610+0000","last_became_active":"2024-05-22T22:59:51.960571+0000","last_became_peered":"2024-05-22T22:59:51.960571+0000","last_unstale":"2024-05-22T23:01:49.876610+0000","last_undegraded":"2024-05-22T23:01:49.876610+0000","last_fullsized":"2024-05-22T23:01:49.876610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:50:31.472449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.149029+0000","last_change":"2024-05-22T22:59:51.592876+0000","last_active":"2024-05-22T23:01:50.149029+0000","last_peered":"2024-05-22T23:01:50.149029+0000","last_clean":"2024-05-22T23:01:50.149029+0000","last_became_active":"2024-05-22T22:59:51.592488+0000","last_became_peered":"2024-05-22T22:59:51.592488+0000","last_unstale":"2024-05-22T23:01:50.149029+0000","last_undegraded":"2024-05-22T23:01:50.149029+0000","last_fullsized":"2024-05-22T23:01:50.149029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:37:16.517302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.875938+0000","last_change":"2024-05-22T22:59:51.962336+0000","last_active":"2024-05-22T23:01:49.875938+0000","last_peered":"2024-05-22T23:01:49.875938+0000","last_clean":"2024-05-22T23:01:49.875938+0000","last_became_active":"2024-05-22T22:59:51.962002+0000","last_became_peered":"2024-05-22T22:59:51.962002+0000","last_unstale":"2024-05-22T23:01:49.875938+0000","last_undegraded":"2024-05-22T23:01:49.875938+0000","last_fullsized":"2024-05-22T23:01:49.875938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:23:19.409418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.214265+0000","last_change":"2024-05-22T23:00:46.591734+0000","last_active":"2024-05-22T23:01:50.214265+0000","last_peered":"2024-05-22T23:01:50.214265+0000","last_clean":"2024-05-22T23:01:50.214265+0000","last_became_active":"2024-05-22T23:00:46.584075+0000","last_became_peered":"2024-05-22T23:00:46.584075+0000","last_unstale":"2024-05-22T23:01:50.214265+0000","last_undegraded":"2024-05-22T23:01:50.214265+0000","last_fullsized":"2024-05-22T23:01:50.214265+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:32:13.548197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.967837+0000","last_change":"2024-05-22T22:59:51.961600+0000","last_active":"2024-05-22T23:01:49.967837+0000","last_peered":"2024-05-22T23:01:49.967837+0000","last_clean":"2024-05-22T23:01:49.967837+0000","last_became_active":"2024-05-22T22:59:51.961286+0000","last_became_peered":"2024-05-22T22:59:51.961286+0000","last_unstale":"2024-05-22T23:01:49.967837+0000","last_undegraded":"2024-05-22T23:01:49.967837+0000","last_fullsized":"2024-05-22T23:01:49.967837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:04:26.992865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987338+0000","last_change":"2024-05-22T22:59:51.594461+0000","last_active":"2024-05-22T23:01:49.987338+0000","last_peered":"2024-05-22T23:01:49.987338+0000","last_clean":"2024-05-22T23:01:49.987338+0000","last_became_active":"2024-05-22T22:59:51.594128+0000","last_became_peered":"2024-05-22T22:59:51.594128+0000","last_unstale":"2024-05-22T23:01:49.987338+0000","last_undegraded":"2024-05-22T23:01:49.987338+0000","last_fullsized":"2024-05-22T23:01:49.987338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:52:05.095814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.214429+0000","last_change":"2024-05-22T22:59:51.958851+0000","last_active":"2024-05-22T23:01:50.214429+0000","last_peered":"2024-05-22T23:01:50.214429+0000","last_clean":"2024-05-22T23:01:50.214429+0000","last_became_active":"2024-05-22T22:59:51.957412+0000","last_became_peered":"2024-05-22T22:59:51.957412+0000","last_unstale":"2024-05-22T23:01:50.214429+0000","last_undegraded":"2024-05-22T23:01:50.214429+0000","last_fullsized":"2024-05-22T23:01:50.214429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:13:02.761205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876410+0000","last_change":"2024-05-22T22:59:51.588750+0000","last_active":"2024-05-22T23:01:49.876410+0000","last_peered":"2024-05-22T23:01:49.876410+0000","last_clean":"2024-05-22T23:01:49.876410+0000","last_became_active":"2024-05-22T22:59:51.588364+0000","last_became_peered":"2024-05-22T22:59:51.588364+0000","last_unstale":"2024-05-22T23:01:49.876410+0000","last_undegraded":"2024-05-22T23:01:49.876410+0000","last_fullsized":"2024-05-22T23:01:49.876410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:28:41.605406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.962928+0000","last_change":"2024-05-22T22:59:51.964676+0000","last_active":"2024-05-22T23:01:49.962928+0000","last_peered":"2024-05-22T23:01:49.962928+0000","last_clean":"2024-05-22T23:01:49.962928+0000","last_became_active":"2024-05-22T22:59:51.960472+0000","last_became_peered":"2024-05-22T22:59:51.960472+0000","last_unstale":"2024-05-22T23:01:49.962928+0000","last_undegraded":"2024-05-22T23:01:49.962928+0000","last_fullsized":"2024-05-22T23:01:49.962928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:24:55.042637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876125+0000","last_change":"2024-05-22T22:59:51.600873+0000","last_active":"2024-05-22T23:01:49.876125+0000","last_peered":"2024-05-22T23:01:49.876125+0000","last_clean":"2024-05-22T23:01:49.876125+0000","last_became_active":"2024-05-22T22:59:51.599785+0000","last_became_peered":"2024-05-22T22:59:51.599785+0000","last_unstale":"2024-05-22T23:01:49.876125+0000","last_undegraded":"2024-05-22T23:01:49.876125+0000","last_fullsized":"2024-05-22T23:01:49.876125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:20:09.590765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.962927+0000","last_change":"2024-05-22T22:59:51.964942+0000","last_active":"2024-05-22T23:01:49.962927+0000","last_peered":"2024-05-22T23:01:49.962927+0000","last_clean":"2024-05-22T23:01:49.962927+0000","last_became_active":"2024-05-22T22:59:51.964333+0000","last_became_peered":"2024-05-22T22:59:51.964333+0000","last_unstale":"2024-05-22T23:01:49.962927+0000","last_undegraded":"2024-05-22T23:01:49.962927+0000","last_fullsized":"2024-05-22T23:01:49.962927+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:01:07.215075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.968033+0000","last_change":"2024-05-22T22:59:51.957291+0000","last_active":"2024-05-22T23:01:49.968033+0000","last_peered":"2024-05-22T23:01:49.968033+0000","last_clean":"2024-05-22T23:01:49.968033+0000","last_became_active":"2024-05-22T22:59:51.956987+0000","last_became_peered":"2024-05-22T22:59:51.956987+0000","last_unstale":"2024-05-22T23:01:49.968033+0000","last_undegraded":"2024-05-22T23:01:49.968033+0000","last_fullsized":"2024-05-22T23:01:49.968033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:14:10.420756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.709738+0000","last_change":"2024-05-22T22:59:51.590255+0000","last_active":"2024-05-22T23:01:50.709738+0000","last_peered":"2024-05-22T23:01:50.709738+0000","last_clean":"2024-05-22T23:01:50.709738+0000","last_became_active":"2024-05-22T22:59:51.589261+0000","last_became_peered":"2024-05-22T22:59:51.589261+0000","last_unstale":"2024-05-22T23:01:50.709738+0000","last_undegraded":"2024-05-22T23:01:50.709738+0000","last_fullsized":"2024-05-22T23:01:50.709738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:15:14.171311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987623+0000","last_change":"2024-05-22T22:59:51.961453+0000","last_active":"2024-05-22T23:01:49.987623+0000","last_peered":"2024-05-22T23:01:49.987623+0000","last_clean":"2024-05-22T23:01:49.987623+0000","last_became_active":"2024-05-22T22:59:51.961082+0000","last_became_peered":"2024-05-22T22:59:51.961082+0000","last_unstale":"2024-05-22T23:01:49.987623+0000","last_undegraded":"2024-05-22T23:01:49.987623+0000","last_fullsized":"2024-05-22T23:01:49.987623+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:05:50.310688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.149332+0000","last_change":"2024-05-22T22:59:51.595913+0000","last_active":"2024-05-22T23:01:50.149332+0000","last_peered":"2024-05-22T23:01:50.149332+0000","last_clean":"2024-05-22T23:01:50.149332+0000","last_became_active":"2024-05-22T22:59:51.595561+0000","last_became_peered":"2024-05-22T22:59:51.595561+0000","last_unstale":"2024-05-22T23:01:50.149332+0000","last_undegraded":"2024-05-22T23:01:50.149332+0000","last_fullsized":"2024-05-22T23:01:50.149332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:37:04.701575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":357,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876032+0000","last_change":"2024-05-22T22:58:34.927654+0000","last_active":"2024-05-22T23:01:49.876032+0000","last_peered":"2024-05-22T23:01:49.876032+0000","last_clean":"2024-05-22T23:01:49.876032+0000","last_became_active":"2024-05-22T22:58:34.927090+0000","last_became_peered":"2024-05-22T22:58:34.927090+0000","last_unstale":"2024-05-22T23:01:49.876032+0000","last_undegraded":"2024-05-22T23:01:49.876032+0000","last_fullsized":"2024-05-22T23:01:49.876032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:55:07.719852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:55:07.719852+0000","last_clean_scrub_stamp":"2024-05-22T22:55:07.719852+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:40:53.763146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.709675+0000","last_change":"2024-05-22T22:59:51.956422+0000","last_active":"2024-05-22T23:01:50.709675+0000","last_peered":"2024-05-22T23:01:50.709675+0000","last_clean":"2024-05-22T23:01:50.709675+0000","last_became_active":"2024-05-22T22:59:51.956127+0000","last_became_peered":"2024-05-22T22:59:51.956127+0000","last_unstale":"2024-05-22T23:01:50.709675+0000","last_undegraded":"2024-05-22T23:01:50.709675+0000","last_fullsized":"2024-05-22T23:01:50.709675+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:04:36.955644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876146+0000","last_change":"2024-05-22T23:00:46.874231+0000","last_active":"2024-05-22T23:01:49.876146+0000","last_peered":"2024-05-22T23:01:49.876146+0000","last_clean":"2024-05-22T23:01:49.876146+0000","last_became_active":"2024-05-22T23:00:46.873728+0000","last_became_peered":"2024-05-22T23:00:46.873728+0000","last_unstale":"2024-05-22T23:01:49.876146+0000","last_undegraded":"2024-05-22T23:01:49.876146+0000","last_fullsized":"2024-05-22T23:01:49.876146+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:49:12.372179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987349+0000","last_change":"2024-05-22T22:59:51.598772+0000","last_active":"2024-05-22T23:01:49.987349+0000","last_peered":"2024-05-22T23:01:49.987349+0000","last_clean":"2024-05-22T23:01:49.987349+0000","last_became_active":"2024-05-22T22:59:51.598192+0000","last_became_peered":"2024-05-22T22:59:51.598192+0000","last_unstale":"2024-05-22T23:01:49.987349+0000","last_undegraded":"2024-05-22T23:01:49.987349+0000","last_fullsized":"2024-05-22T23:01:49.987349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:42:47.069554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.710017+0000","last_change":"2024-05-22T22:59:51.942337+0000","last_active":"2024-05-22T23:01:50.710017+0000","last_peered":"2024-05-22T23:01:50.710017+0000","last_clean":"2024-05-22T23:01:50.710017+0000","last_became_active":"2024-05-22T22:59:51.941939+0000","last_became_peered":"2024-05-22T22:59:51.941939+0000","last_unstale":"2024-05-22T23:01:50.710017+0000","last_undegraded":"2024-05-22T23:01:50.710017+0000","last_fullsized":"2024-05-22T23:01:50.710017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:35:42.360128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'43","reported_seq":87,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.110236+0000","last_change":"2024-05-22T22:59:51.589640+0000","last_active":"2024-05-22T23:01:50.110236+0000","last_peered":"2024-05-22T23:01:50.110236+0000","last_clean":"2024-05-22T23:01:50.110236+0000","last_became_active":"2024-05-22T22:59:51.589310+0000","last_became_peered":"2024-05-22T22:59:51.589310+0000","last_unstale":"2024-05-22T23:01:50.110236+0000","last_undegraded":"2024-05-22T23:01:50.110236+0000","last_fullsized":"2024-05-22T23:01:50.110236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:54:39.182002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":77,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987475+0000","last_change":"2024-05-22T22:59:51.952423+0000","last_active":"2024-05-22T23:01:49.987475+0000","last_peered":"2024-05-22T23:01:49.987475+0000","last_clean":"2024-05-22T23:01:49.987475+0000","last_became_active":"2024-05-22T22:59:51.952015+0000","last_became_peered":"2024-05-22T22:59:51.952015+0000","last_unstale":"2024-05-22T23:01:49.987475+0000","last_undegraded":"2024-05-22T23:01:49.987475+0000","last_fullsized":"2024-05-22T23:01:49.987475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:06:23.705248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.149938+0000","last_change":"2024-05-22T22:59:51.594882+0000","last_active":"2024-05-22T23:01:50.149938+0000","last_peered":"2024-05-22T23:01:50.149938+0000","last_clean":"2024-05-22T23:01:50.149938+0000","last_became_active":"2024-05-22T22:59:51.594552+0000","last_became_peered":"2024-05-22T22:59:51.594552+0000","last_unstale":"2024-05-22T23:01:50.149938+0000","last_undegraded":"2024-05-22T23:01:50.149938+0000","last_fullsized":"2024-05-22T23:01:50.149938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:06:40.771969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.709570+0000","last_change":"2024-05-22T22:59:51.956534+0000","last_active":"2024-05-22T23:01:50.709570+0000","last_peered":"2024-05-22T23:01:50.709570+0000","last_clean":"2024-05-22T23:01:50.709570+0000","last_became_active":"2024-05-22T22:59:51.956037+0000","last_became_peered":"2024-05-22T22:59:51.956037+0000","last_unstale":"2024-05-22T23:01:50.709570+0000","last_undegraded":"2024-05-22T23:01:50.709570+0000","last_fullsized":"2024-05-22T23:01:50.709570+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:24:15.289570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.148587+0000","last_change":"2024-05-22T22:59:51.592916+0000","last_active":"2024-05-22T23:01:50.148587+0000","last_peered":"2024-05-22T23:01:50.148587+0000","last_clean":"2024-05-22T23:01:50.148587+0000","last_became_active":"2024-05-22T22:59:51.592530+0000","last_became_peered":"2024-05-22T22:59:51.592530+0000","last_unstale":"2024-05-22T23:01:50.148587+0000","last_undegraded":"2024-05-22T23:01:50.148587+0000","last_fullsized":"2024-05-22T23:01:50.148587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:56:18.660120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987022+0000","last_change":"2024-05-22T22:59:51.956188+0000","last_active":"2024-05-22T23:01:49.987022+0000","last_peered":"2024-05-22T23:01:49.987022+0000","last_clean":"2024-05-22T23:01:49.987022+0000","last_became_active":"2024-05-22T22:59:51.955844+0000","last_became_peered":"2024-05-22T22:59:51.955844+0000","last_unstale":"2024-05-22T23:01:49.987022+0000","last_undegraded":"2024-05-22T23:01:49.987022+0000","last_fullsized":"2024-05-22T23:01:49.987022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:09:12.848822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987300+0000","last_change":"2024-05-22T22:59:51.590641+0000","last_active":"2024-05-22T23:01:49.987300+0000","last_peered":"2024-05-22T23:01:49.987300+0000","last_clean":"2024-05-22T23:01:49.987300+0000","last_became_active":"2024-05-22T22:59:51.590321+0000","last_became_peered":"2024-05-22T22:59:51.590321+0000","last_unstale":"2024-05-22T23:01:49.987300+0000","last_undegraded":"2024-05-22T23:01:49.987300+0000","last_fullsized":"2024-05-22T23:01:49.987300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:46:15.157160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.214400+0000","last_change":"2024-05-22T22:59:51.950256+0000","last_active":"2024-05-22T23:01:50.214400+0000","last_peered":"2024-05-22T23:01:50.214400+0000","last_clean":"2024-05-22T23:01:50.214400+0000","last_became_active":"2024-05-22T22:59:51.949746+0000","last_became_peered":"2024-05-22T22:59:51.949746+0000","last_unstale":"2024-05-22T23:01:50.214400+0000","last_undegraded":"2024-05-22T23:01:50.214400+0000","last_fullsized":"2024-05-22T23:01:50.214400+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:33:12.146425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.214297+0000","last_change":"2024-05-22T23:00:46.874312+0000","last_active":"2024-05-22T23:01:50.214297+0000","last_peered":"2024-05-22T23:01:50.214297+0000","last_clean":"2024-05-22T23:01:50.214297+0000","last_became_active":"2024-05-22T23:00:46.873803+0000","last_became_peered":"2024-05-22T23:00:46.873803+0000","last_unstale":"2024-05-22T23:01:50.214297+0000","last_undegraded":"2024-05-22T23:01:50.214297+0000","last_fullsized":"2024-05-22T23:01:50.214297+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:44:48.915692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987564+0000","last_change":"2024-05-22T22:59:51.968432+0000","last_active":"2024-05-22T23:01:49.987564+0000","last_peered":"2024-05-22T23:01:49.987564+0000","last_clean":"2024-05-22T23:01:49.987564+0000","last_became_active":"2024-05-22T22:59:51.968077+0000","last_became_peered":"2024-05-22T22:59:51.968077+0000","last_unstale":"2024-05-22T23:01:49.987564+0000","last_undegraded":"2024-05-22T23:01:49.987564+0000","last_fullsized":"2024-05-22T23:01:49.987564+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:08:16.281671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987132+0000","last_change":"2024-05-22T22:59:51.601616+0000","last_active":"2024-05-22T23:01:49.987132+0000","last_peered":"2024-05-22T23:01:49.987132+0000","last_clean":"2024-05-22T23:01:49.987132+0000","last_became_active":"2024-05-22T22:59:51.601249+0000","last_became_peered":"2024-05-22T22:59:51.601249+0000","last_unstale":"2024-05-22T23:01:49.987132+0000","last_undegraded":"2024-05-22T23:01:49.987132+0000","last_fullsized":"2024-05-22T23:01:49.987132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:07:07.030142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876196+0000","last_change":"2024-05-22T22:59:51.951578+0000","last_active":"2024-05-22T23:01:49.876196+0000","last_peered":"2024-05-22T23:01:49.876196+0000","last_clean":"2024-05-22T23:01:49.876196+0000","last_became_active":"2024-05-22T22:59:51.951233+0000","last_became_peered":"2024-05-22T22:59:51.951233+0000","last_unstale":"2024-05-22T23:01:49.876196+0000","last_undegraded":"2024-05-22T23:01:49.876196+0000","last_fullsized":"2024-05-22T23:01:49.876196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:25.882943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876127+0000","last_change":"2024-05-22T22:59:51.597819+0000","last_active":"2024-05-22T23:01:49.876127+0000","last_peered":"2024-05-22T23:01:49.876127+0000","last_clean":"2024-05-22T23:01:49.876127+0000","last_became_active":"2024-05-22T22:59:51.597262+0000","last_became_peered":"2024-05-22T22:59:51.597262+0000","last_unstale":"2024-05-22T23:01:49.876127+0000","last_undegraded":"2024-05-22T23:01:49.876127+0000","last_fullsized":"2024-05-22T23:01:49.876127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:05:02.849623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876421+0000","last_change":"2024-05-22T22:59:51.951668+0000","last_active":"2024-05-22T23:01:49.876421+0000","last_peered":"2024-05-22T23:01:49.876421+0000","last_clean":"2024-05-22T23:01:49.876421+0000","last_became_active":"2024-05-22T22:59:51.951176+0000","last_became_peered":"2024-05-22T22:59:51.951176+0000","last_unstale":"2024-05-22T23:01:49.876421+0000","last_undegraded":"2024-05-22T23:01:49.876421+0000","last_fullsized":"2024-05-22T23:01:49.876421+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:01:46.010858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'39","reported_seq":68,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.104832+0000","last_change":"2024-05-22T22:59:51.593144+0000","last_active":"2024-05-22T23:01:50.104832+0000","last_peered":"2024-05-22T23:01:50.104832+0000","last_clean":"2024-05-22T23:01:50.104832+0000","last_became_active":"2024-05-22T22:59:51.592557+0000","last_became_peered":"2024-05-22T22:59:51.592557+0000","last_unstale":"2024-05-22T23:01:50.104832+0000","last_undegraded":"2024-05-22T23:01:50.104832+0000","last_fullsized":"2024-05-22T23:01:50.104832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:28:19.403419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876262+0000","last_change":"2024-05-22T22:59:51.958866+0000","last_active":"2024-05-22T23:01:49.876262+0000","last_peered":"2024-05-22T23:01:49.876262+0000","last_clean":"2024-05-22T23:01:49.876262+0000","last_became_active":"2024-05-22T22:59:51.958505+0000","last_became_peered":"2024-05-22T22:59:51.958505+0000","last_unstale":"2024-05-22T23:01:49.876262+0000","last_undegraded":"2024-05-22T23:01:49.876262+0000","last_fullsized":"2024-05-22T23:01:49.876262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:54:24.601966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.710067+0000","last_change":"2024-05-22T22:59:51.587391+0000","last_active":"2024-05-22T23:01:50.710067+0000","last_peered":"2024-05-22T23:01:50.710067+0000","last_clean":"2024-05-22T23:01:50.710067+0000","last_became_active":"2024-05-22T22:59:51.587056+0000","last_became_peered":"2024-05-22T22:59:51.587056+0000","last_unstale":"2024-05-22T23:01:50.710067+0000","last_undegraded":"2024-05-22T23:01:50.710067+0000","last_fullsized":"2024-05-22T23:01:50.710067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:50.560359+0000","last_clean_scrub_stamp":"2024-05-22T22:59:50.560359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:04:40.614853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876301+0000","last_change":"2024-05-22T22:59:51.953466+0000","last_active":"2024-05-22T23:01:49.876301+0000","last_peered":"2024-05-22T23:01:49.876301+0000","last_clean":"2024-05-22T23:01:49.876301+0000","last_became_active":"2024-05-22T22:59:51.952830+0000","last_became_peered":"2024-05-22T22:59:51.952830+0000","last_unstale":"2024-05-22T23:01:49.876301+0000","last_undegraded":"2024-05-22T23:01:49.876301+0000","last_fullsized":"2024-05-22T23:01:49.876301+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:36:06.314680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.214453+0000","last_change":"2024-05-22T22:59:51.962993+0000","last_active":"2024-05-22T23:01:50.214453+0000","last_peered":"2024-05-22T23:01:50.214453+0000","last_clean":"2024-05-22T23:01:50.214453+0000","last_became_active":"2024-05-22T22:59:51.962591+0000","last_became_peered":"2024-05-22T22:59:51.962591+0000","last_unstale":"2024-05-22T23:01:50.214453+0000","last_undegraded":"2024-05-22T23:01:50.214453+0000","last_fullsized":"2024-05-22T23:01:50.214453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:55:50.902537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.875914+0000","last_change":"2024-05-22T22:59:51.956304+0000","last_active":"2024-05-22T23:01:49.875914+0000","last_peered":"2024-05-22T23:01:49.875914+0000","last_clean":"2024-05-22T23:01:49.875914+0000","last_became_active":"2024-05-22T22:59:51.955933+0000","last_became_peered":"2024-05-22T22:59:51.955933+0000","last_unstale":"2024-05-22T23:01:49.875914+0000","last_undegraded":"2024-05-22T23:01:49.875914+0000","last_fullsized":"2024-05-22T23:01:49.875914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:28:17.225851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.875841+0000","last_change":"2024-05-22T22:59:51.946806+0000","last_active":"2024-05-22T23:01:49.875841+0000","last_peered":"2024-05-22T23:01:49.875841+0000","last_clean":"2024-05-22T23:01:49.875841+0000","last_became_active":"2024-05-22T22:59:51.946433+0000","last_became_peered":"2024-05-22T22:59:51.946433+0000","last_unstale":"2024-05-22T23:01:49.875841+0000","last_undegraded":"2024-05-22T23:01:49.875841+0000","last_fullsized":"2024-05-22T23:01:49.875841+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:24:06.013688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.709394+0000","last_change":"2024-05-22T22:59:51.956466+0000","last_active":"2024-05-22T23:01:50.709394+0000","last_peered":"2024-05-22T23:01:50.709394+0000","last_clean":"2024-05-22T23:01:50.709394+0000","last_became_active":"2024-05-22T22:59:51.956144+0000","last_became_peered":"2024-05-22T22:59:51.956144+0000","last_unstale":"2024-05-22T23:01:50.709394+0000","last_undegraded":"2024-05-22T23:01:50.709394+0000","last_fullsized":"2024-05-22T23:01:50.709394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:08:28.674824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.709256+0000","last_change":"2024-05-22T23:00:46.582147+0000","last_active":"2024-05-22T23:01:50.709256+0000","last_peered":"2024-05-22T23:01:50.709256+0000","last_clean":"2024-05-22T23:01:50.709256+0000","last_became_active":"2024-05-22T23:00:46.581804+0000","last_became_peered":"2024-05-22T23:00:46.581804+0000","last_unstale":"2024-05-22T23:01:50.709256+0000","last_undegraded":"2024-05-22T23:01:50.709256+0000","last_fullsized":"2024-05-22T23:01:50.709256+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:02:15.933895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.214476+0000","last_change":"2024-05-22T22:59:51.961971+0000","last_active":"2024-05-22T23:01:50.214476+0000","last_peered":"2024-05-22T23:01:50.214476+0000","last_clean":"2024-05-22T23:01:50.214476+0000","last_became_active":"2024-05-22T22:59:51.961604+0000","last_became_peered":"2024-05-22T22:59:51.961604+0000","last_unstale":"2024-05-22T23:01:50.214476+0000","last_undegraded":"2024-05-22T23:01:50.214476+0000","last_fullsized":"2024-05-22T23:01:50.214476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:32:14.669631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.876247+0000","last_change":"2024-05-22T22:59:51.948173+0000","last_active":"2024-05-22T23:01:49.876247+0000","last_peered":"2024-05-22T23:01:49.876247+0000","last_clean":"2024-05-22T23:01:49.876247+0000","last_became_active":"2024-05-22T22:59:51.947839+0000","last_became_peered":"2024-05-22T22:59:51.947839+0000","last_unstale":"2024-05-22T23:01:49.876247+0000","last_undegraded":"2024-05-22T23:01:49.876247+0000","last_fullsized":"2024-05-22T23:01:49.876247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:55:07.748445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987341+0000","last_change":"2024-05-22T22:59:51.955627+0000","last_active":"2024-05-22T23:01:49.987341+0000","last_peered":"2024-05-22T23:01:49.987341+0000","last_clean":"2024-05-22T23:01:49.987341+0000","last_became_active":"2024-05-22T22:59:51.955284+0000","last_became_peered":"2024-05-22T22:59:51.955284+0000","last_unstale":"2024-05-22T23:01:49.987341+0000","last_undegraded":"2024-05-22T23:01:49.987341+0000","last_fullsized":"2024-05-22T23:01:49.987341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:57:13.029317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.967969+0000","last_change":"2024-05-22T22:59:51.955889+0000","last_active":"2024-05-22T23:01:49.967969+0000","last_peered":"2024-05-22T23:01:49.967969+0000","last_clean":"2024-05-22T23:01:49.967969+0000","last_became_active":"2024-05-22T22:59:51.954930+0000","last_became_peered":"2024-05-22T22:59:51.954930+0000","last_unstale":"2024-05-22T23:01:49.967969+0000","last_undegraded":"2024-05-22T23:01:49.967969+0000","last_fullsized":"2024-05-22T23:01:49.967969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:47.536072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:49.987300+0000","last_change":"2024-05-22T22:59:51.954148+0000","last_active":"2024-05-22T23:01:49.987300+0000","last_peered":"2024-05-22T23:01:49.987300+0000","last_clean":"2024-05-22T23:01:49.987300+0000","last_became_active":"2024-05-22T22:59:51.952336+0000","last_became_peered":"2024-05-22T22:59:51.952336+0000","last_unstale":"2024-05-22T23:01:49.987300+0000","last_undegraded":"2024-05-22T23:01:49.987300+0000","last_fullsized":"2024-05-22T23:01:49.987300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T10:07:27.928910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.149653+0000","last_change":"2024-05-22T22:59:51.954300+0000","last_active":"2024-05-22T23:01:50.149653+0000","last_peered":"2024-05-22T23:01:50.149653+0000","last_clean":"2024-05-22T23:01:50.149653+0000","last_became_active":"2024-05-22T22:59:51.953779+0000","last_became_peered":"2024-05-22T22:59:51.953779+0000","last_unstale":"2024-05-22T23:01:50.149653+0000","last_undegraded":"2024-05-22T23:01:50.149653+0000","last_fullsized":"2024-05-22T23:01:50.149653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:24:59.363634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.710011+0000","last_change":"2024-05-22T22:59:51.947860+0000","last_active":"2024-05-22T23:01:50.710011+0000","last_peered":"2024-05-22T23:01:50.710011+0000","last_clean":"2024-05-22T23:01:50.710011+0000","last_became_active":"2024-05-22T22:59:51.947420+0000","last_became_peered":"2024-05-22T22:59:51.947420+0000","last_unstale":"2024-05-22T23:01:50.710011+0000","last_undegraded":"2024-05-22T23:01:50.710011+0000","last_fullsized":"2024-05-22T23:01:50.710011+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:20:40.629285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-22T23:01:50.214320+0000","last_change":"2024-05-22T22:59:51.957867+0000","last_active":"2024-05-22T23:01:50.214320+0000","last_peered":"2024-05-22T23:01:50.214320+0000","last_clean":"2024-05-22T23:01:50.214320+0000","last_became_active":"2024-05-22T22:59:51.957522+0000","last_became_peered":"2024-05-22T22:59:51.957522+0000","last_unstale":"2024-05-22T23:01:50.214320+0000","last_undegraded":"2024-05-22T23:01:50.214320+0000","last_fullsized":"2024-05-22T23:01:50.214320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T22:59:51.567644+0000","last_clean_scrub_stamp":"2024-05-22T22:59:51.567644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:41:18.631970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107052,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":591235,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":324,"num_read_kb":256,"num_write":193,"num_write_kb":1399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":58911,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":364544,"data_stored":1773705,"data_compressed":89155,"data_compressed_allocated":118784,"data_compressed_original":1622016,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":962560,"data_stored":470148,"data_compressed":8653,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612651,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1118208,"data_stored":1258048,"data_compressed":52350,"data_compressed_allocated":69632,"data_compressed_original":745472,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776176,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":863946,"data_compressed":27799,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037812,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":863968,"data_compressed":27799,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299450,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1019904,"data_stored":1061215,"data_compressed":44337,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495679,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":461280,"data_compressed":5755,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1032192,"data_stored":535769,"data_compressed":10510,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":954368,"data_stored":461253,"data_compressed":5755,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":966656,"data_stored":470182,"data_compressed":8653,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":995302,"data_compressed":33955,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":535791,"data_compressed":18188,"data_compressed_allocated":24576,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757317,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1048576,"data_stored":1192415,"data_compressed":50493,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35684,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":65621,"data_compressed":1857,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":524920,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":65643,"data_compressed":9535,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":131782,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":393820,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":393798,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":197415,"data_compressed":8013,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-22T23:01:51.644 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T23:01:51.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.779+0000 7f53de941640 1 -- 172.21.15.84:0/133352355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53d8102360 msgr2=0x7f53d8108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:51.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.779+0000 7f53de941640 1 --2- 172.21.15.84:0/133352355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53d8102360 0x7f53d8108bf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009ca0 tx=0x7f53d002a8c0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.779+0000 7f53de941640 1 -- 172.21.15.84:0/133352355 shutdown_connections 2024-05-22T23:01:51.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.779+0000 7f53de941640 1 --2- 172.21.15.84:0/133352355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53d8102360 0x7f53d8108bf0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.779+0000 7f53de941640 1 --2- 172.21.15.84:0/133352355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53d81019a0 0x7f53d8101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.779+0000 7f53de941640 1 --2- 172.21.15.84:0/133352355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53d81007b0 0x7f53d8100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.779+0000 7f53de941640 1 -- 172.21.15.84:0/133352355 >> 172.21.15.84:0/133352355 conn(0x7f53d80fbd20 msgr2=0x7f53d80fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:51.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.780+0000 7f53de941640 1 -- 172.21.15.84:0/133352355 shutdown_connections 2024-05-22T23:01:51.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.780+0000 7f53de941640 1 -- 172.21.15.84:0/133352355 wait complete. 2024-05-22T23:01:51.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.780+0000 7f53de941640 1 Processor -- start 2024-05-22T23:01:51.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.780+0000 7f53de941640 1 -- start start 2024-05-22T23:01:51.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.780+0000 7f53de941640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53d81007b0 0x7f53d8197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:51.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.780+0000 7f53de941640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53d81019a0 0x7f53d8198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:51.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.780+0000 7f53de941640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53d8102360 0x7f53d819f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:51.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.780+0000 7f53de941640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f53d810c410 con 0x7f53d81019a0 2024-05-22T23:01:51.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.780+0000 7f53de941640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f53d810c290 con 0x7f53d81007b0 2024-05-22T23:01:51.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.780+0000 7f53de941640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f53d810c590 con 0x7f53d8102360 2024-05-22T23:01:51.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.780+0000 7f53cffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53d81019a0 0x7f53d8198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:51.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.780+0000 7f53cffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53d81019a0 0x7f53d8198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58702/0 (socket says 172.21.15.84:58702) 2024-05-22T23:01:51.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.780+0000 7f53cffff640 1 -- 172.21.15.84:0/3972378284 learned_addr learned my addr 172.21.15.84:0/3972378284 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:51.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.780+0000 7f53d7fff640 1 --2- 172.21.15.84:0/3972378284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53d81007b0 0x7f53d8197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:51.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.780+0000 7f53dcede640 1 --2- 172.21.15.84:0/3972378284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53d8102360 0x7f53d819f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:51.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.781+0000 7f53cffff640 1 -- 172.21.15.84:0/3972378284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53d8102360 msgr2=0x7f53d819f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:51.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.781+0000 7f53cffff640 1 --2- 172.21.15.84:0/3972378284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53d8102360 0x7f53d819f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.781+0000 7f53cffff640 1 -- 172.21.15.84:0/3972378284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53d81007b0 msgr2=0x7f53d8197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:51.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.781+0000 7f53cffff640 1 --2- 172.21.15.84:0/3972378284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53d81007b0 0x7f53d8197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:51.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.781+0000 7f53cffff640 1 -- 172.21.15.84:0/3972378284 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d819f740 con 0x7f53d81019a0 2024-05-22T23:01:51.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.781+0000 7f53cffff640 1 --2- 172.21.15.84:0/3972378284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53d81019a0 0x7f53d8198060 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f53c000e9f0 tx=0x7f53c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:51.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.781+0000 7f53d5ffb640 1 -- 172.21.15.84:0/3972378284 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c000cd80 con 0x7f53d81019a0 2024-05-22T23:01:51.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.781+0000 7f53d5ffb640 1 -- 172.21.15.84:0/3972378284 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53c0010910 con 0x7f53d81019a0 2024-05-22T23:01:51.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.781+0000 7f53de941640 1 -- 172.21.15.84:0/3972378284 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d819fa30 con 0x7f53d81019a0 2024-05-22T23:01:51.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.781+0000 7f53d5ffb640 1 -- 172.21.15.84:0/3972378284 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c0018990 con 0x7f53d81019a0 2024-05-22T23:01:51.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.781+0000 7f53de941640 1 -- 172.21.15.84:0/3972378284 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d819ff80 con 0x7f53d81019a0 2024-05-22T23:01:51.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.782+0000 7f53de941640 1 -- 172.21.15.84:0/3972378284 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d8068690 con 0x7f53d81019a0 2024-05-22T23:01:51.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.784+0000 7f53d5ffb640 1 -- 172.21.15.84:0/3972378284 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f53c0010ac0 con 0x7f53d81019a0 2024-05-22T23:01:51.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.784+0000 7f53d5ffb640 1 --2- 172.21.15.84:0/3972378284 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f53c40a2990 0x7f53c40a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:51.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.784+0000 7f53d7fff640 1 --2- 172.21.15.84:0/3972378284 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f53c40a2990 0x7f53c40a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:51.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.784+0000 7f53d5ffb640 1 -- 172.21.15.84:0/3972378284 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f53c00cf4b0 con 0x7f53d81019a0 2024-05-22T23:01:51.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.785+0000 7f53d7fff640 1 --2- 172.21.15.84:0/3972378284 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f53c40a2990 0x7f53c40a4e50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f53d8101800 tx=0x7f53c8005f90 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:51.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.788+0000 7f53d5ffb640 1 -- 172.21.15.84:0/3972378284 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f53c009bf90 con 0x7f53d81019a0 2024-05-22T23:01:51.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:51.908+0000 7f53d5ffb640 1 -- 172.21.15.84:0/3972378284 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f53c0098600 con 0x7f53d81019a0 2024-05-22T23:01:52.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.011+0000 7f53de941640 1 -- 172.21.15.84:0/3972378284 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53d81a0400 con 0x7f53d81019a0 2024-05-22T23:01:52.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.013+0000 7f53d5ffb640 1 -- 172.21.15.84:0/3972378284 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23100 (secure 0 0 0) 0x7f53c009cfd0 con 0x7f53d81019a0 2024-05-22T23:01:52.015 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:52.015 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":89,"fsid":"c1135a12-188d-11ef-bc9a-c7b262605968","created":"2024-05-22T22:51:49.878218+0000","modified":"2024-05-22T23:01:49.869890+0000","last_up_change":"2024-05-22T22:58:58.550278+0000","last_in_change":"2024-05-22T22:58:37.322139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T22:55:07.719394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T22:59:50.150511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T22:59:51.111182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762df065-fb4f-4c71-aab8-a62d2282c253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6803","nonce":1559945826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6805","nonce":1559945826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6809","nonce":1559945826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6807","nonce":1559945826}]},"public_addr":"172.21.15.84:6803/1559945826","cluster_addr":"172.21.15.84:6805/1559945826","heartbeat_back_addr":"172.21.15.84:6809/1559945826","heartbeat_front_addr":"172.21.15.84:6807/1559945826","state":["exists","up"]},{"osd":1,"uuid":"3fb2385d-6a34-49e3-88e1-116237beb346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6801","nonce":2996505353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6803","nonce":2996505353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6807","nonce":2996505353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6805","nonce":2996505353}]},"public_addr":"172.21.15.96:6801/2996505353","cluster_addr":"172.21.15.96:6803/2996505353","heartbeat_back_addr":"172.21.15.96:6807/2996505353","heartbeat_front_addr":"172.21.15.96:6805/2996505353","state":["exists","up"]},{"osd":2,"uuid":"e306e88e-a308-4585-ae11-c8af10b84038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6801","nonce":1499577897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6803","nonce":1499577897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6807","nonce":1499577897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6805","nonce":1499577897}]},"public_addr":"172.21.15.204:6801/1499577897","cluster_addr":"172.21.15.204:6803/1499577897","heartbeat_back_addr":"172.21.15.204:6807/1499577897","heartbeat_front_addr":"172.21.15.204:6805/1499577897","state":["exists","up"]},{"osd":3,"uuid":"d2a5fb97-6f60-4d45-9f18-447cae480974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6811","nonce":2869340648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6813","nonce":2869340648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6817","nonce":2869340648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6815","nonce":2869340648}]},"public_addr":"172.21.15.84:6811/2869340648","cluster_addr":"172.21.15.84:6813/2869340648","heartbeat_back_addr":"172.21.15.84:6817/2869340648","heartbeat_front_addr":"172.21.15.84:6815/2869340648","state":["exists","up"]},{"osd":4,"uuid":"4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6809","nonce":2554265648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6811","nonce":2554265648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6815","nonce":2554265648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6813","nonce":2554265648}]},"public_addr":"172.21.15.96:6809/2554265648","cluster_addr":"172.21.15.96:6811/2554265648","heartbeat_back_addr":"172.21.15.96:6815/2554265648","heartbeat_front_addr":"172.21.15.96:6813/2554265648","state":["exists","up"]},{"osd":5,"uuid":"7996bf5e-93a5-41b0-a9b1-99e7ef4514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6809","nonce":4075476986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6811","nonce":4075476986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6815","nonce":4075476986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6813","nonce":4075476986}]},"public_addr":"172.21.15.204:6809/4075476986","cluster_addr":"172.21.15.204:6811/4075476986","heartbeat_back_addr":"172.21.15.204:6815/4075476986","heartbeat_front_addr":"172.21.15.204:6813/4075476986","state":["exists","up"]},{"osd":6,"uuid":"f4b7c5de-1836-4d16-8603-3c517524d9fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6819","nonce":1806202865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6821","nonce":1806202865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6825","nonce":1806202865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6823","nonce":1806202865}]},"public_addr":"172.21.15.84:6819/1806202865","cluster_addr":"172.21.15.84:6821/1806202865","heartbeat_back_addr":"172.21.15.84:6825/1806202865","heartbeat_front_addr":"172.21.15.84:6823/1806202865","state":["exists","up"]},{"osd":7,"uuid":"69a18c43-5447-480d-9466-190830070c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6817","nonce":3195792774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6819","nonce":3195792774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6823","nonce":3195792774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6821","nonce":3195792774}]},"public_addr":"172.21.15.96:6817/3195792774","cluster_addr":"172.21.15.96:6819/3195792774","heartbeat_back_addr":"172.21.15.96:6823/3195792774","heartbeat_front_addr":"172.21.15.96:6821/3195792774","state":["exists","up"]},{"osd":8,"uuid":"2c5cadca-689a-441e-847c-ba370d400ef1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6817","nonce":606997927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6819","nonce":606997927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6823","nonce":606997927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6821","nonce":606997927}]},"public_addr":"172.21.15.204:6817/606997927","cluster_addr":"172.21.15.204:6819/606997927","heartbeat_back_addr":"172.21.15.204:6823/606997927","heartbeat_front_addr":"172.21.15.204:6821/606997927","state":["exists","up"]},{"osd":9,"uuid":"e3984fb2-7c00-4275-a10e-bca7c859f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6827","nonce":2786189012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6829","nonce":2786189012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6833","nonce":2786189012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6831","nonce":2786189012}]},"public_addr":"172.21.15.84:6827/2786189012","cluster_addr":"172.21.15.84:6829/2786189012","heartbeat_back_addr":"172.21.15.84:6833/2786189012","heartbeat_front_addr":"172.21.15.84:6831/2786189012","state":["exists","up"]},{"osd":10,"uuid":"b4938160-a808-4ded-91c2-d245c99fff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6825","nonce":1985398526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6827","nonce":1985398526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6831","nonce":1985398526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6829","nonce":1985398526}]},"public_addr":"172.21.15.96:6825/1985398526","cluster_addr":"172.21.15.96:6827/1985398526","heartbeat_back_addr":"172.21.15.96:6831/1985398526","heartbeat_front_addr":"172.21.15.96:6829/1985398526","state":["exists","up"]},{"osd":11,"uuid":"238922ef-d046-40cc-833d-25b7e7a523cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6825","nonce":3106116519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6827","nonce":3106116519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6831","nonce":3106116519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6829","nonce":3106116519}]},"public_addr":"172.21.15.204:6825/3106116519","cluster_addr":"172.21.15.204:6827/3106116519","heartbeat_back_addr":"172.21.15.204:6831/3106116519","heartbeat_front_addr":"172.21.15.204:6829/3106116519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:16.443480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:39.898133+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:03.664544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:28.838687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:53.829733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:19.269307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:45.166910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:10.771614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:36.678516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:03.150592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:29.327769+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:56.315938+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2317224739":"2024-05-23T23:01:49.869856+0000","172.21.15.84:0/4170743349":"2024-05-23T23:01:49.869856+0000","172.21.15.84:6801/4223589140":"2024-05-23T23:01:49.869856+0000","172.21.15.84:6800/4223589140":"2024-05-23T23:01:49.869856+0000","172.21.15.84:6801/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:6800/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/2766288608":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/950455999":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/3490117794":"2024-05-23T23:01:49.869856+0000","172.21.15.84:0/4000289608":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1650467363":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/4077200298":"2024-05-23T23:01:49.869856+0000","172.21.15.84:0/1052746114":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6800/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6801/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/14610813":"2024-05-23T22:52:45.395791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T23:01:52.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.016+0000 7f53de941640 1 -- 172.21.15.84:0/3972378284 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f53c40a2990 msgr2=0x7f53c40a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:52.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.016+0000 7f53de941640 1 --2- 172.21.15.84:0/3972378284 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f53c40a2990 0x7f53c40a4e50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f53d8101800 tx=0x7f53c8005f90 comp rx=0 tx=0).stop 2024-05-22T23:01:52.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.016+0000 7f53de941640 1 -- 172.21.15.84:0/3972378284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53d81019a0 msgr2=0x7f53d8198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:52.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.016+0000 7f53de941640 1 --2- 172.21.15.84:0/3972378284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53d81019a0 0x7f53d8198060 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f53c000e9f0 tx=0x7f53c000eeb0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.016+0000 7f53de941640 1 -- 172.21.15.84:0/3972378284 shutdown_connections 2024-05-22T23:01:52.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.016+0000 7f53de941640 1 --2- 172.21.15.84:0/3972378284 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f53c40a2990 0x7f53c40a4e50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.016+0000 7f53de941640 1 --2- 172.21.15.84:0/3972378284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53d8102360 0x7f53d819f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.016+0000 7f53de941640 1 --2- 172.21.15.84:0/3972378284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53d81019a0 0x7f53d8198060 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.016+0000 7f53de941640 1 --2- 172.21.15.84:0/3972378284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53d81007b0 0x7f53d8197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.016+0000 7f53de941640 1 -- 172.21.15.84:0/3972378284 >> 172.21.15.84:0/3972378284 conn(0x7f53d80fbd20 msgr2=0x7f53d8068a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:52.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.016+0000 7f53de941640 1 -- 172.21.15.84:0/3972378284 shutdown_connections 2024-05-22T23:01:52.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.016+0000 7f53de941640 1 -- 172.21.15.84:0/3972378284 wait complete. 2024-05-22T23:01:52.033 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T22:55:07.719394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T22:59:50.150511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T22:59:51.111182+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T23:01:52.033 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T23:01:52.161 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:51 smithi204 ceph-mon[36211]: mgrmap e21: x(active, since 1.03638s), standbys: z, y 2024-05-22T23:01:52.161 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:51 smithi204 ceph-mon[36211]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-22T23:01:52.161 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:51 smithi204 ceph-mon[36211]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T23:01:52.161 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:51 smithi204 ceph-mon[36211]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-22T23:01:52.161 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:51 smithi204 ceph-mon[36211]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-22T23:01:52.163 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.161+0000 7f4251b9e640 1 -- 172.21.15.84:0/740615535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f424c101870 msgr2=0x7f424c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:52.163 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.161+0000 7f4251b9e640 1 --2- 172.21.15.84:0/740615535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f424c101870 0x7f424c101cf0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4238009900 tx=0x7f423802a410 comp rx=0 tx=0).stop 2024-05-22T23:01:52.163 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.161+0000 7f4251b9e640 1 -- 172.21.15.84:0/740615535 shutdown_connections 2024-05-22T23:01:52.163 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.161+0000 7f4251b9e640 1 --2- 172.21.15.84:0/740615535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f424c102230 0x7f424c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.164 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.161+0000 7f4251b9e640 1 --2- 172.21.15.84:0/740615535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f424c101870 0x7f424c101cf0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.164 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.161+0000 7f4251b9e640 1 --2- 172.21.15.84:0/740615535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f424c100680 0x7f424c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.164 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.161+0000 7f4251b9e640 1 -- 172.21.15.84:0/740615535 >> 172.21.15.84:0/740615535 conn(0x7f424c0fbc10 msgr2=0x7f424c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:52.164 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.161+0000 7f4251b9e640 1 -- 172.21.15.84:0/740615535 shutdown_connections 2024-05-22T23:01:52.164 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.161+0000 7f4251b9e640 1 -- 172.21.15.84:0/740615535 wait complete. 2024-05-22T23:01:52.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.162+0000 7f4251b9e640 1 Processor -- start 2024-05-22T23:01:52.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.162+0000 7f4251b9e640 1 -- start start 2024-05-22T23:01:52.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.162+0000 7f4251b9e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f424c100680 0x7f424c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:52.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.162+0000 7f4251b9e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f424c101870 0x7f424c19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:52.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.162+0000 7f4251b9e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f424c102230 0x7f424c19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:52.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.162+0000 7f4251b9e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f424c10c2e0 con 0x7f424c102230 2024-05-22T23:01:52.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.162+0000 7f4251b9e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f424c10c160 con 0x7f424c100680 2024-05-22T23:01:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.162+0000 7f4251b9e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f424c10c460 con 0x7f424c101870 2024-05-22T23:01:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.162+0000 7f425139d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f424c102230 0x7f424c19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.162+0000 7f4250b9c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f424c100680 0x7f424c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.162+0000 7f424bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f424c101870 0x7f424c19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.162+0000 7f4250b9c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f424c100680 0x7f424c069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:55028/0 (socket says 172.21.15.84:55028) 2024-05-22T23:01:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.162+0000 7f425139d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f424c102230 0x7f424c19edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58740/0 (socket says 172.21.15.84:58740) 2024-05-22T23:01:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.162+0000 7f4250b9c640 1 -- 172.21.15.84:0/512847767 learned_addr learned my addr 172.21.15.84:0/512847767 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.163+0000 7f4250b9c640 1 -- 172.21.15.84:0/512847767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f424c101870 msgr2=0x7f424c19c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.163+0000 7f4250b9c640 1 --2- 172.21.15.84:0/512847767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f424c101870 0x7f424c19c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.163+0000 7f4250b9c640 1 -- 172.21.15.84:0/512847767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f424c102230 msgr2=0x7f424c19edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.163+0000 7f4250b9c640 1 --2- 172.21.15.84:0/512847767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f424c102230 0x7f424c19edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.163+0000 7f4250b9c640 1 -- 172.21.15.84:0/512847767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424c19f690 con 0x7f424c100680 2024-05-22T23:01:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.163+0000 7f425139d640 1 --2- 172.21.15.84:0/512847767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f424c102230 0x7f424c19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:01:52.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.163+0000 7f4250b9c640 1 --2- 172.21.15.84:0/512847767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f424c100680 0x7f424c069590 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f424000d950 tx=0x7f424000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:52.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.163+0000 7f4249ffb640 1 -- 172.21.15.84:0/512847767 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240017020 con 0x7f424c100680 2024-05-22T23:01:52.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.163+0000 7f4251b9e640 1 -- 172.21.15.84:0/512847767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f424c19f980 con 0x7f424c100680 2024-05-22T23:01:52.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.163+0000 7f4249ffb640 1 -- 172.21.15.84:0/512847767 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4240009c30 con 0x7f424c100680 2024-05-22T23:01:52.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.163+0000 7f4249ffb640 1 -- 172.21.15.84:0/512847767 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424000f720 con 0x7f424c100680 2024-05-22T23:01:52.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.163+0000 7f4251b9e640 1 -- 172.21.15.84:0/512847767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f424c19fea0 con 0x7f424c100680 2024-05-22T23:01:52.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.164+0000 7f4251b9e640 1 -- 172.21.15.84:0/512847767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f424c0685b0 con 0x7f424c100680 2024-05-22T23:01:52.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.165+0000 7f4249ffb640 1 -- 172.21.15.84:0/512847767 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f424001a020 con 0x7f424c100680 2024-05-22T23:01:52.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.166+0000 7f4249ffb640 1 --2- 172.21.15.84:0/512847767 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f423c0a2d00 0x7f423c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:52.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.166+0000 7f4249ffb640 1 -- 172.21.15.84:0/512847767 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f42400cf300 con 0x7f424c100680 2024-05-22T23:01:52.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.166+0000 7f424bfff640 1 --2- 172.21.15.84:0/512847767 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f423c0a2d00 0x7f423c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:52.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.167+0000 7f424bfff640 1 --2- 172.21.15.84:0/512847767 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f423c0a2d00 0x7f423c0a51c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f42380314e0 tx=0x7f4238000bc0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:52.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.170+0000 7f4249ffb640 1 -- 172.21.15.84:0/512847767 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4240021020 con 0x7f424c100680 2024-05-22T23:01:52.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:51 smithi096 ceph-mon[36392]: mgrmap e21: x(active, since 1.03638s), standbys: z, y 2024-05-22T23:01:52.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:51 smithi096 ceph-mon[36392]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-22T23:01:52.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:51 smithi096 ceph-mon[36392]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T23:01:52.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:51 smithi096 ceph-mon[36392]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-22T23:01:52.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:51 smithi096 ceph-mon[36392]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-22T23:01:52.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.360+0000 7f4251b9e640 1 -- 172.21.15.84:0/512847767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f424c02d310 con 0x7f424c100680 2024-05-22T23:01:52.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.361+0000 7f4249ffb640 1 -- 172.21.15.84:0/512847767 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f4240009de0 con 0x7f424c100680 2024-05-22T23:01:52.363 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2024-05-22T23:01:52.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.363+0000 7f4251b9e640 1 -- 172.21.15.84:0/512847767 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f423c0a2d00 msgr2=0x7f423c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:52.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.363+0000 7f4251b9e640 1 --2- 172.21.15.84:0/512847767 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f423c0a2d00 0x7f423c0a51c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f42380314e0 tx=0x7f4238000bc0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.363+0000 7f4251b9e640 1 -- 172.21.15.84:0/512847767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f424c100680 msgr2=0x7f424c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:52.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.363+0000 7f4251b9e640 1 --2- 172.21.15.84:0/512847767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f424c100680 0x7f424c069590 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f424000d950 tx=0x7f424000de10 comp rx=0 tx=0).stop 2024-05-22T23:01:52.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.363+0000 7f4251b9e640 1 -- 172.21.15.84:0/512847767 shutdown_connections 2024-05-22T23:01:52.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.363+0000 7f4251b9e640 1 --2- 172.21.15.84:0/512847767 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f423c0a2d00 0x7f423c0a51c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.363+0000 7f4251b9e640 1 --2- 172.21.15.84:0/512847767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f424c102230 0x7f424c19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.363+0000 7f4251b9e640 1 --2- 172.21.15.84:0/512847767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f424c101870 0x7f424c19c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.363+0000 7f4251b9e640 1 --2- 172.21.15.84:0/512847767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f424c100680 0x7f424c069590 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.363+0000 7f4251b9e640 1 -- 172.21.15.84:0/512847767 >> 172.21.15.84:0/512847767 conn(0x7f424c0fbc10 msgr2=0x7f424c104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:52.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.363+0000 7f4251b9e640 1 -- 172.21.15.84:0/512847767 shutdown_connections 2024-05-22T23:01:52.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.363+0000 7f4251b9e640 1 -- 172.21.15.84:0/512847767 wait complete. 2024-05-22T23:01:52.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:51 smithi084 ceph-mon[32186]: mgrmap e21: x(active, since 1.03638s), standbys: z, y 2024-05-22T23:01:52.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:51 smithi084 ceph-mon[32186]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-22T23:01:52.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:51 smithi084 ceph-mon[32186]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T23:01:52.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:51 smithi084 ceph-mon[32186]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-22T23:01:52.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:51 smithi084 ceph-mon[32186]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-22T23:01:52.376 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T23:01:52.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.479+0000 7feef085d640 1 -- 172.21.15.84:0/479720182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feeec1007b0 msgr2=0x7feeec100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:52.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.479+0000 7feef085d640 1 --2- 172.21.15.84:0/479720182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feeec1007b0 0x7feeec100bd0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7feedc009900 tx=0x7feedc02a410 comp rx=0 tx=0).stop 2024-05-22T23:01:52.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.480+0000 7feef085d640 1 -- 172.21.15.84:0/479720182 shutdown_connections 2024-05-22T23:01:52.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.480+0000 7feef085d640 1 --2- 172.21.15.84:0/479720182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeec102360 0x7feeec108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.480+0000 7feef085d640 1 --2- 172.21.15.84:0/479720182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feeec1019a0 0x7feeec101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.480+0000 7feef085d640 1 --2- 172.21.15.84:0/479720182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feeec1007b0 0x7feeec100bd0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.480+0000 7feef085d640 1 -- 172.21.15.84:0/479720182 >> 172.21.15.84:0/479720182 conn(0x7feeec0fbd20 msgr2=0x7feeec0fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:52.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.480+0000 7feef085d640 1 -- 172.21.15.84:0/479720182 shutdown_connections 2024-05-22T23:01:52.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.480+0000 7feef085d640 1 -- 172.21.15.84:0/479720182 wait complete. 2024-05-22T23:01:52.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.480+0000 7feef085d640 1 Processor -- start 2024-05-22T23:01:52.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.480+0000 7feef085d640 1 -- start start 2024-05-22T23:01:52.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.480+0000 7feef085d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feeec1007b0 0x7feeec197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:52.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.480+0000 7feef085d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeec1019a0 0x7feeec198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:52.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.480+0000 7feef085d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feeec102360 0x7feeec19f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:52.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.481+0000 7feef085d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7feeec10c400 con 0x7feeec102360 2024-05-22T23:01:52.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.481+0000 7feef085d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feeec10c280 con 0x7feeec1019a0 2024-05-22T23:01:52.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.481+0000 7feef085d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feeec10c580 con 0x7feeec1007b0 2024-05-22T23:01:52.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.481+0000 7feeead9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feeec102360 0x7feeec19f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:52.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.481+0000 7feeead9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feeec102360 0x7feeec19f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58770/0 (socket says 172.21.15.84:58770) 2024-05-22T23:01:52.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.481+0000 7feeead9d640 1 -- 172.21.15.84:0/3042456089 learned_addr learned my addr 172.21.15.84:0/3042456089 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:52.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.481+0000 7feeea59c640 1 --2- 172.21.15.84:0/3042456089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feeec1007b0 0x7feeec197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:52.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.481+0000 7feee9d9b640 1 --2- 172.21.15.84:0/3042456089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeec1019a0 0x7feeec198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:52.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.481+0000 7feeead9d640 1 -- 172.21.15.84:0/3042456089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feeec1007b0 msgr2=0x7feeec197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:52.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.481+0000 7feeead9d640 1 --2- 172.21.15.84:0/3042456089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feeec1007b0 0x7feeec197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.481+0000 7feeead9d640 1 -- 172.21.15.84:0/3042456089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeec1019a0 msgr2=0x7feeec198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:52.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.481+0000 7feeead9d640 1 --2- 172.21.15.84:0/3042456089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeec1019a0 0x7feeec198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.481+0000 7feeead9d640 1 -- 172.21.15.84:0/3042456089 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeec19f850 con 0x7feeec102360 2024-05-22T23:01:52.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.481+0000 7feeead9d640 1 --2- 172.21.15.84:0/3042456089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feeec102360 0x7feeec19f1b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7feee400eb10 tx=0x7feee400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:52.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.481+0000 7feee37fe640 1 -- 172.21.15.84:0/3042456089 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4004b20 con 0x7feeec102360 2024-05-22T23:01:52.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.481+0000 7feef085d640 1 -- 172.21.15.84:0/3042456089 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeec19fb40 con 0x7feeec102360 2024-05-22T23:01:52.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.481+0000 7feee37fe640 1 -- 172.21.15.84:0/3042456089 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feee4004cc0 con 0x7feeec102360 2024-05-22T23:01:52.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.481+0000 7feee37fe640 1 -- 172.21.15.84:0/3042456089 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4010760 con 0x7feeec102360 2024-05-22T23:01:52.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.481+0000 7feef085d640 1 -- 172.21.15.84:0/3042456089 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeec1a0060 con 0x7feeec102360 2024-05-22T23:01:52.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.482+0000 7feef085d640 1 -- 172.21.15.84:0/3042456089 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeec1a0230 con 0x7feeec102360 2024-05-22T23:01:52.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.484+0000 7feee37fe640 1 -- 172.21.15.84:0/3042456089 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7feee4005020 con 0x7feeec102360 2024-05-22T23:01:52.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.485+0000 7feee37fe640 1 --2- 172.21.15.84:0/3042456089 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7feed80a2d80 0x7feed80a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:52.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.485+0000 7feee37fe640 1 -- 172.21.15.84:0/3042456089 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7feee40cfa10 con 0x7feeec102360 2024-05-22T23:01:52.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.485+0000 7feeea59c640 1 --2- 172.21.15.84:0/3042456089 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7feed80a2d80 0x7feed80a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:52.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.486+0000 7feeea59c640 1 --2- 172.21.15.84:0/3042456089 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7feed80a2d80 0x7feed80a5240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feedc005ab0 tx=0x7feedc000bc0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:52.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.490+0000 7feee37fe640 1 -- 172.21.15.84:0/3042456089 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7feee409c630 con 0x7feeec102360 2024-05-22T23:01:52.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.671+0000 7feef085d640 1 -- 172.21.15.84:0/3042456089 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7feeec1a0400 con 0x7feeec102360 2024-05-22T23:01:52.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.671+0000 7feee37fe640 1 -- 172.21.15.84:0/3042456089 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7feee409bf30 con 0x7feeec102360 2024-05-22T23:01:52.673 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2024-05-22T23:01:52.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.673+0000 7feef085d640 1 -- 172.21.15.84:0/3042456089 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7feed80a2d80 msgr2=0x7feed80a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:52.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.673+0000 7feef085d640 1 --2- 172.21.15.84:0/3042456089 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7feed80a2d80 0x7feed80a5240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feedc005ab0 tx=0x7feedc000bc0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.673+0000 7feef085d640 1 -- 172.21.15.84:0/3042456089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feeec102360 msgr2=0x7feeec19f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:52.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.673+0000 7feef085d640 1 --2- 172.21.15.84:0/3042456089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feeec102360 0x7feeec19f1b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7feee400eb10 tx=0x7feee400c2d0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.673+0000 7feef085d640 1 -- 172.21.15.84:0/3042456089 shutdown_connections 2024-05-22T23:01:52.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.673+0000 7feef085d640 1 --2- 172.21.15.84:0/3042456089 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7feed80a2d80 0x7feed80a5240 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.673+0000 7feef085d640 1 --2- 172.21.15.84:0/3042456089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feeec102360 0x7feeec19f1b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.673+0000 7feef085d640 1 --2- 172.21.15.84:0/3042456089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeec1019a0 0x7feeec198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.673+0000 7feef085d640 1 --2- 172.21.15.84:0/3042456089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feeec1007b0 0x7feeec197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.673+0000 7feef085d640 1 -- 172.21.15.84:0/3042456089 >> 172.21.15.84:0/3042456089 conn(0x7feeec0fbd20 msgr2=0x7feeec0fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:52.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.673+0000 7feef085d640 1 -- 172.21.15.84:0/3042456089 shutdown_connections 2024-05-22T23:01:52.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.673+0000 7feef085d640 1 -- 172.21.15.84:0/3042456089 wait complete. 2024-05-22T23:01:52.687 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T23:01:52.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.792+0000 7f32073d5640 1 -- 172.21.15.84:0/351212186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3200102390 msgr2=0x7f3200104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:52.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.792+0000 7f32073d5640 1 --2- 172.21.15.84:0/351212186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3200102390 0x7f3200104780 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009900 tx=0x7f31fc02a410 comp rx=0 tx=0).stop 2024-05-22T23:01:52.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.792+0000 7f32073d5640 1 -- 172.21.15.84:0/351212186 shutdown_connections 2024-05-22T23:01:52.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.792+0000 7f32073d5640 1 --2- 172.21.15.84:0/351212186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32001075f0 0x7f32001099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.792+0000 7f32073d5640 1 --2- 172.21.15.84:0/351212186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3200104cc0 0x7f32001070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.792+0000 7f32073d5640 1 --2- 172.21.15.84:0/351212186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3200102390 0x7f3200104780 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.792+0000 7f32073d5640 1 -- 172.21.15.84:0/351212186 >> 172.21.15.84:0/351212186 conn(0x7f32000fbd20 msgr2=0x7f32000fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:52.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.792+0000 7f32073d5640 1 -- 172.21.15.84:0/351212186 shutdown_connections 2024-05-22T23:01:52.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.792+0000 7f32073d5640 1 -- 172.21.15.84:0/351212186 wait complete. 2024-05-22T23:01:52.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.793+0000 7f32073d5640 1 Processor -- start 2024-05-22T23:01:52.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.793+0000 7f32073d5640 1 -- start start 2024-05-22T23:01:52.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.793+0000 7f32073d5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3200102390 0x7f3200197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:52.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.793+0000 7f32073d5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3200104cc0 0x7f3200198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:52.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.793+0000 7f32073d5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32001075f0 0x7f320019f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:52.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.793+0000 7f32073d5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f320010d550 con 0x7f32001075f0 2024-05-22T23:01:52.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.793+0000 7f32073d5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f320010d3d0 con 0x7f3200102390 2024-05-22T23:01:52.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.793+0000 7f32073d5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f320010d6d0 con 0x7f3200104cc0 2024-05-22T23:01:52.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.793+0000 7f3205972640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32001075f0 0x7f320019f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:52.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.793+0000 7f3205972640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32001075f0 0x7f320019f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58792/0 (socket says 172.21.15.84:58792) 2024-05-22T23:01:52.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.793+0000 7f3205972640 1 -- 172.21.15.84:0/870372379 learned_addr learned my addr 172.21.15.84:0/870372379 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:52.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.793+0000 7f3205171640 1 --2- 172.21.15.84:0/870372379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3200102390 0x7f3200197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:52.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.793+0000 7f3204970640 1 --2- 172.21.15.84:0/870372379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3200104cc0 0x7f3200198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:52.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.793+0000 7f3205972640 1 -- 172.21.15.84:0/870372379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3200104cc0 msgr2=0x7f3200198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:52.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.793+0000 7f3205972640 1 --2- 172.21.15.84:0/870372379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3200104cc0 0x7f3200198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.793+0000 7f3205972640 1 -- 172.21.15.84:0/870372379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3200102390 msgr2=0x7f3200197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:52.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.793+0000 7f3205972640 1 --2- 172.21.15.84:0/870372379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3200102390 0x7f3200197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.793+0000 7f3205972640 1 -- 172.21.15.84:0/870372379 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320019f850 con 0x7f32001075f0 2024-05-22T23:01:52.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.794+0000 7f3205972640 1 --2- 172.21.15.84:0/870372379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32001075f0 0x7f320019f1b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f31f800eaa0 tx=0x7f31f800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:52.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.794+0000 7f31f67fc640 1 -- 172.21.15.84:0/870372379 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f800cdb0 con 0x7f32001075f0 2024-05-22T23:01:52.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.794+0000 7f32073d5640 1 -- 172.21.15.84:0/870372379 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320019fb40 con 0x7f32001075f0 2024-05-22T23:01:52.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.794+0000 7f31f67fc640 1 -- 172.21.15.84:0/870372379 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31f8005020 con 0x7f32001075f0 2024-05-22T23:01:52.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.794+0000 7f31f67fc640 1 -- 172.21.15.84:0/870372379 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f80116d0 con 0x7f32001075f0 2024-05-22T23:01:52.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.794+0000 7f32073d5640 1 -- 172.21.15.84:0/870372379 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32001a00e0 con 0x7f32001075f0 2024-05-22T23:01:52.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.795+0000 7f32073d5640 1 -- 172.21.15.84:0/870372379 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32000fd810 con 0x7f32001075f0 2024-05-22T23:01:52.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.797+0000 7f31f67fc640 1 -- 172.21.15.84:0/870372379 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f31f80054d0 con 0x7f32001075f0 2024-05-22T23:01:52.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.797+0000 7f31f67fc640 1 --2- 172.21.15.84:0/870372379 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f31ec0a2d00 0x7f31ec0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:52.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.798+0000 7f31f67fc640 1 -- 172.21.15.84:0/870372379 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f31f80cfa00 con 0x7f32001075f0 2024-05-22T23:01:52.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.798+0000 7f3205171640 1 --2- 172.21.15.84:0/870372379 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f31ec0a2d00 0x7f31ec0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:52.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.798+0000 7f3205171640 1 --2- 172.21.15.84:0/870372379 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f31ec0a2d00 0x7f31ec0a51c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3200199110 tx=0x7f31fc000bc0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:52.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.801+0000 7f31f67fc640 1 -- 172.21.15.84:0/870372379 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f31f8016030 con 0x7f32001075f0 2024-05-22T23:01:52.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.986+0000 7f32073d5640 1 -- 172.21.15.84:0/870372379 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f320002d200 con 0x7f32001075f0 2024-05-22T23:01:52.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.987+0000 7f31f67fc640 1 -- 172.21.15.84:0/870372379 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f31f809bfd0 con 0x7f32001075f0 2024-05-22T23:01:52.989 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2024-05-22T23:01:52.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.989+0000 7f32073d5640 1 -- 172.21.15.84:0/870372379 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f31ec0a2d00 msgr2=0x7f31ec0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:52.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.989+0000 7f32073d5640 1 --2- 172.21.15.84:0/870372379 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f31ec0a2d00 0x7f31ec0a51c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3200199110 tx=0x7f31fc000bc0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.989+0000 7f32073d5640 1 -- 172.21.15.84:0/870372379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32001075f0 msgr2=0x7f320019f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:52.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.989+0000 7f32073d5640 1 --2- 172.21.15.84:0/870372379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32001075f0 0x7f320019f1b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f31f800eaa0 tx=0x7f31f800ef60 comp rx=0 tx=0).stop 2024-05-22T23:01:52.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.989+0000 7f32073d5640 1 -- 172.21.15.84:0/870372379 shutdown_connections 2024-05-22T23:01:52.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.989+0000 7f32073d5640 1 --2- 172.21.15.84:0/870372379 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f31ec0a2d00 0x7f31ec0a51c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.989+0000 7f32073d5640 1 --2- 172.21.15.84:0/870372379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32001075f0 0x7f320019f1b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.989+0000 7f32073d5640 1 --2- 172.21.15.84:0/870372379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3200104cc0 0x7f3200198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.989+0000 7f32073d5640 1 --2- 172.21.15.84:0/870372379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3200102390 0x7f3200197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:52.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.989+0000 7f32073d5640 1 -- 172.21.15.84:0/870372379 >> 172.21.15.84:0/870372379 conn(0x7f32000fbd20 msgr2=0x7f3200102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:52.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.989+0000 7f32073d5640 1 -- 172.21.15.84:0/870372379 shutdown_connections 2024-05-22T23:01:52.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:52.989+0000 7f32073d5640 1 -- 172.21.15.84:0/870372379 wait complete. 2024-05-22T23:01:53.001 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:52 smithi084 ceph-mon[32186]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T23:01:53.001 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:52 smithi084 ceph-mon[32186]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-05-22T23:01:53.001 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:52 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3972378284' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:53.001 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:52 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:53.001 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:52 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:53.001 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:52 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:53.001 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:52 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:53.001 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:52 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:53.002 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:52 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:53.002 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:52 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/512847767' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:01:53.002 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:52 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3042456089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:01:53.003 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:01:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.105+0000 7f88be812640 1 -- 172.21.15.84:0/249761433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88b8102360 msgr2=0x7f88b8108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.105+0000 7f88be812640 1 --2- 172.21.15.84:0/249761433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88b8102360 0x7f88b8108bf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f88b0009ca0 tx=0x7f88b002a8c0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.105+0000 7f88be812640 1 -- 172.21.15.84:0/249761433 shutdown_connections 2024-05-22T23:01:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.105+0000 7f88be812640 1 --2- 172.21.15.84:0/249761433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88b8102360 0x7f88b8108bf0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.105+0000 7f88be812640 1 --2- 172.21.15.84:0/249761433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88b81019a0 0x7f88b8101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.105+0000 7f88be812640 1 --2- 172.21.15.84:0/249761433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b81007b0 0x7f88b8100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.105+0000 7f88be812640 1 -- 172.21.15.84:0/249761433 >> 172.21.15.84:0/249761433 conn(0x7f88b80fbd20 msgr2=0x7f88b80fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.105+0000 7f88be812640 1 -- 172.21.15.84:0/249761433 shutdown_connections 2024-05-22T23:01:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.105+0000 7f88be812640 1 -- 172.21.15.84:0/249761433 wait complete. 2024-05-22T23:01:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.105+0000 7f88be812640 1 Processor -- start 2024-05-22T23:01:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.105+0000 7f88be812640 1 -- start start 2024-05-22T23:01:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.106+0000 7f88be812640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88b81007b0 0x7f88b8197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.106+0000 7f88be812640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b81019a0 0x7f88b8198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.106+0000 7f88be812640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88b8102360 0x7f88b819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.106+0000 7f88be812640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f88b810d510 con 0x7f88b8102360 2024-05-22T23:01:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.106+0000 7f88be812640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f88b810d390 con 0x7f88b81007b0 2024-05-22T23:01:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.106+0000 7f88be812640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88b810d690 con 0x7f88b81019a0 2024-05-22T23:01:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.106+0000 7f88bcdaf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88b8102360 0x7f88b819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.106+0000 7f88bcdaf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88b8102360 0x7f88b819f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58806/0 (socket says 172.21.15.84:58806) 2024-05-22T23:01:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.106+0000 7f88bcdaf640 1 -- 172.21.15.84:0/3264661064 learned_addr learned my addr 172.21.15.84:0/3264661064 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.106+0000 7f88b7fff640 1 --2- 172.21.15.84:0/3264661064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88b81007b0 0x7f88b8197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.106+0000 7f88b77fe640 1 --2- 172.21.15.84:0/3264661064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b81019a0 0x7f88b8198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.106+0000 7f88bcdaf640 1 -- 172.21.15.84:0/3264661064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b81019a0 msgr2=0x7f88b8198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.106+0000 7f88bcdaf640 1 --2- 172.21.15.84:0/3264661064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b81019a0 0x7f88b8198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.106+0000 7f88bcdaf640 1 -- 172.21.15.84:0/3264661064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88b81007b0 msgr2=0x7f88b8197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.106+0000 7f88bcdaf640 1 --2- 172.21.15.84:0/3264661064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88b81007b0 0x7f88b8197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.106+0000 7f88bcdaf640 1 -- 172.21.15.84:0/3264661064 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b819f770 con 0x7f88b8102360 2024-05-22T23:01:53.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.106+0000 7f88bcdaf640 1 --2- 172.21.15.84:0/3264661064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88b8102360 0x7f88b819f100 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f88b0031ae0 tx=0x7f88b0031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:53.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.106+0000 7f88b57fa640 1 -- 172.21.15.84:0/3264661064 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b0003c30 con 0x7f88b8102360 2024-05-22T23:01:53.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.107+0000 7f88b57fa640 1 -- 172.21.15.84:0/3264661064 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b0003dd0 con 0x7f88b8102360 2024-05-22T23:01:53.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.107+0000 7f88be812640 1 -- 172.21.15.84:0/3264661064 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b819fa00 con 0x7f88b8102360 2024-05-22T23:01:53.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.107+0000 7f88b57fa640 1 -- 172.21.15.84:0/3264661064 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b002fcb0 con 0x7f88b8102360 2024-05-22T23:01:53.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.107+0000 7f88be812640 1 -- 172.21.15.84:0/3264661064 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b819ffa0 con 0x7f88b8102360 2024-05-22T23:01:53.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.108+0000 7f88be812640 1 -- 172.21.15.84:0/3264661064 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88b8107a70 con 0x7f88b8102360 2024-05-22T23:01:53.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.109+0000 7f88b57fa640 1 -- 172.21.15.84:0/3264661064 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f88b0004050 con 0x7f88b8102360 2024-05-22T23:01:53.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.109+0000 7f88b57fa640 1 --2- 172.21.15.84:0/3264661064 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f88a00a2d80 0x7f88a00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.110+0000 7f88b7fff640 1 --2- 172.21.15.84:0/3264661064 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f88a00a2d80 0x7f88a00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.110+0000 7f88b57fa640 1 -- 172.21.15.84:0/3264661064 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f88b00f0780 con 0x7f88b8102360 2024-05-22T23:01:53.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.110+0000 7f88b7fff640 1 --2- 172.21.15.84:0/3264661064 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f88a00a2d80 0x7f88a00a5240 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f88a8005fd0 tx=0x7f88a800b410 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:53.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.113+0000 7f88b57fa640 1 -- 172.21.15.84:0/3264661064 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f88b00bd4b0 con 0x7f88b8102360 2024-05-22T23:01:53.161 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:52 smithi204 ceph-mon[36211]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T23:01:53.161 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:52 smithi204 ceph-mon[36211]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-05-22T23:01:53.161 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:52 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3972378284' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:53.161 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:52 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:53.161 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:52 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:53.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:52 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:53.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:52 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:53.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:52 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:53.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:52 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:53.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:52 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/512847767' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:01:53.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:52 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3042456089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:01:53.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:52 smithi096 ceph-mon[36392]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T23:01:53.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:52 smithi096 ceph-mon[36392]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-05-22T23:01:53.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:52 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3972378284' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:53.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:52 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:53.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:52 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:53.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:52 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:53.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:52 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:53.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:52 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:53.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:52 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:53.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:52 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/512847767' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:01:53.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:52 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3042456089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:01:53.341 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.339+0000 7f88be812640 1 -- 172.21.15.84:0/3264661064 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88b802d3c0 con 0x7f88b8102360 2024-05-22T23:01:53.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.340+0000 7f88b57fa640 1 -- 172.21.15.84:0/3264661064 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7f88b00bcdb0 con 0x7f88b8102360 2024-05-22T23:01:53.342 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:53.342 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:01:53.343 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:01:53.344 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.342+0000 7f88be812640 1 -- 172.21.15.84:0/3264661064 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f88a00a2d80 msgr2=0x7f88a00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.344 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.342+0000 7f88be812640 1 --2- 172.21.15.84:0/3264661064 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f88a00a2d80 0x7f88a00a5240 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f88a8005fd0 tx=0x7f88a800b410 comp rx=0 tx=0).stop 2024-05-22T23:01:53.344 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.342+0000 7f88be812640 1 -- 172.21.15.84:0/3264661064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88b8102360 msgr2=0x7f88b819f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.344 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.342+0000 7f88be812640 1 --2- 172.21.15.84:0/3264661064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88b8102360 0x7f88b819f100 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f88b0031ae0 tx=0x7f88b0031ee0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.344 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.342+0000 7f88be812640 1 -- 172.21.15.84:0/3264661064 shutdown_connections 2024-05-22T23:01:53.344 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.342+0000 7f88be812640 1 --2- 172.21.15.84:0/3264661064 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f88a00a2d80 0x7f88a00a5240 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.344 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.342+0000 7f88be812640 1 --2- 172.21.15.84:0/3264661064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88b8102360 0x7f88b819f100 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.345 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.342+0000 7f88be812640 1 --2- 172.21.15.84:0/3264661064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b81019a0 0x7f88b8198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.345 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.342+0000 7f88be812640 1 --2- 172.21.15.84:0/3264661064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88b81007b0 0x7f88b8197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.345 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.342+0000 7f88be812640 1 -- 172.21.15.84:0/3264661064 >> 172.21.15.84:0/3264661064 conn(0x7f88b80fbd20 msgr2=0x7f88b8104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:53.345 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.342+0000 7f88be812640 1 -- 172.21.15.84:0/3264661064 shutdown_connections 2024-05-22T23:01:53.345 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.342+0000 7f88be812640 1 -- 172.21.15.84:0/3264661064 wait complete. 2024-05-22T23:01:53.355 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T23:01:53.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.460+0000 7f7612f14640 1 -- 172.21.15.84:0/2539361700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f760c1023d0 msgr2=0x7f760c1047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.460+0000 7f7612f14640 1 --2- 172.21.15.84:0/2539361700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f760c1023d0 0x7f760c1047c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7600009900 tx=0x7f760002a410 comp rx=0 tx=0).stop 2024-05-22T23:01:53.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.460+0000 7f7612f14640 1 -- 172.21.15.84:0/2539361700 shutdown_connections 2024-05-22T23:01:53.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.460+0000 7f7612f14640 1 --2- 172.21.15.84:0/2539361700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f760c107630 0x7f760c109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.460+0000 7f7612f14640 1 --2- 172.21.15.84:0/2539361700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f760c104d00 0x7f760c1070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.460+0000 7f7612f14640 1 --2- 172.21.15.84:0/2539361700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f760c1023d0 0x7f760c1047c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.460+0000 7f7612f14640 1 -- 172.21.15.84:0/2539361700 >> 172.21.15.84:0/2539361700 conn(0x7f760c0fbd20 msgr2=0x7f760c0fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:53.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.460+0000 7f7612f14640 1 -- 172.21.15.84:0/2539361700 shutdown_connections 2024-05-22T23:01:53.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.460+0000 7f7612f14640 1 -- 172.21.15.84:0/2539361700 wait complete. 2024-05-22T23:01:53.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.461+0000 7f7612f14640 1 Processor -- start 2024-05-22T23:01:53.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.461+0000 7f7612f14640 1 -- start start 2024-05-22T23:01:53.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.461+0000 7f7612f14640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f760c1023d0 0x7f760c199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.461+0000 7f7612f14640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f760c104d00 0x7f760c19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.461+0000 7f7612f14640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f760c107630 0x7f760c1a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.461+0000 7f7612f14640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f760c10d570 con 0x7f760c1023d0 2024-05-22T23:01:53.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.461+0000 7f7612f14640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f760c10d3f0 con 0x7f760c104d00 2024-05-22T23:01:53.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.461+0000 7f7612f14640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f760c10d6f0 con 0x7f760c107630 2024-05-22T23:01:53.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.461+0000 7f7610cb0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f760c1023d0 0x7f760c199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.461+0000 7f7610cb0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f760c1023d0 0x7f760c199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58828/0 (socket says 172.21.15.84:58828) 2024-05-22T23:01:53.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.461+0000 7f7610cb0640 1 -- 172.21.15.84:0/2548137022 learned_addr learned my addr 172.21.15.84:0/2548137022 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:53.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.461+0000 7f760bfff640 1 --2- 172.21.15.84:0/2548137022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f760c104d00 0x7f760c19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.461+0000 7f76114b1640 1 --2- 172.21.15.84:0/2548137022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f760c107630 0x7f760c1a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.461+0000 7f760bfff640 1 -- 172.21.15.84:0/2548137022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f760c107630 msgr2=0x7f760c1a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.461+0000 7f760bfff640 1 --2- 172.21.15.84:0/2548137022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f760c107630 0x7f760c1a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.462+0000 7f760bfff640 1 -- 172.21.15.84:0/2548137022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f760c1023d0 msgr2=0x7f760c199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.462+0000 7f760bfff640 1 --2- 172.21.15.84:0/2548137022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f760c1023d0 0x7f760c199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.462+0000 7f760bfff640 1 -- 172.21.15.84:0/2548137022 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760c1a19b0 con 0x7f760c104d00 2024-05-22T23:01:53.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.462+0000 7f7610cb0640 1 --2- 172.21.15.84:0/2548137022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f760c1023d0 0x7f760c199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:01:53.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.462+0000 7f76114b1640 1 --2- 172.21.15.84:0/2548137022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f760c107630 0x7f760c1a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:01:53.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.462+0000 7f760bfff640 1 --2- 172.21.15.84:0/2548137022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f760c104d00 0x7f760c19a2e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f75f800e9c0 tx=0x7f75f800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:53.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.462+0000 7f7609ffb640 1 -- 172.21.15.84:0/2548137022 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f800ccc0 con 0x7f760c104d00 2024-05-22T23:01:53.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.462+0000 7f7612f14640 1 -- 172.21.15.84:0/2548137022 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f760c1a1ca0 con 0x7f760c104d00 2024-05-22T23:01:53.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.462+0000 7f7609ffb640 1 -- 172.21.15.84:0/2548137022 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75f8010910 con 0x7f760c104d00 2024-05-22T23:01:53.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.462+0000 7f7609ffb640 1 -- 172.21.15.84:0/2548137022 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f8018880 con 0x7f760c104d00 2024-05-22T23:01:53.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.462+0000 7f7612f14640 1 -- 172.21.15.84:0/2548137022 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f760c1a21a0 con 0x7f760c104d00 2024-05-22T23:01:53.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.463+0000 7f7612f14640 1 -- 172.21.15.84:0/2548137022 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f760c109120 con 0x7f760c104d00 2024-05-22T23:01:53.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.464+0000 7f7609ffb640 1 -- 172.21.15.84:0/2548137022 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f75f8010ac0 con 0x7f760c104d00 2024-05-22T23:01:53.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.465+0000 7f7609ffb640 1 --2- 172.21.15.84:0/2548137022 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f75fc0a2d00 0x7f75fc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.465+0000 7f7610cb0640 1 --2- 172.21.15.84:0/2548137022 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f75fc0a2d00 0x7f75fc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.465+0000 7f7609ffb640 1 -- 172.21.15.84:0/2548137022 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f75f80cfc80 con 0x7f760c104d00 2024-05-22T23:01:53.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.465+0000 7f7610cb0640 1 --2- 172.21.15.84:0/2548137022 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f75fc0a2d00 0x7f75fc0a51c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7600000c00 tx=0x7f76000315b0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:53.470 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.468+0000 7f7609ffb640 1 -- 172.21.15.84:0/2548137022 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f75f809c840 con 0x7f760c104d00 2024-05-22T23:01:53.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.658+0000 7f7612f14640 1 -- 172.21.15.84:0/2548137022 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f760c02d420 con 0x7f760c104d00 2024-05-22T23:01:53.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.660+0000 7f7609ffb640 1 -- 172.21.15.84:0/2548137022 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23100 (secure 0 0 0) 0x7f75f809c140 con 0x7f760c104d00 2024-05-22T23:01:53.662 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:01:53.662 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":89,"fsid":"c1135a12-188d-11ef-bc9a-c7b262605968","created":"2024-05-22T22:51:49.878218+0000","modified":"2024-05-22T23:01:49.869890+0000","last_up_change":"2024-05-22T22:58:58.550278+0000","last_in_change":"2024-05-22T22:58:37.322139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T22:55:07.719394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T22:59:50.150511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T22:59:51.111182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762df065-fb4f-4c71-aab8-a62d2282c253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6803","nonce":1559945826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6805","nonce":1559945826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6809","nonce":1559945826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6807","nonce":1559945826}]},"public_addr":"172.21.15.84:6803/1559945826","cluster_addr":"172.21.15.84:6805/1559945826","heartbeat_back_addr":"172.21.15.84:6809/1559945826","heartbeat_front_addr":"172.21.15.84:6807/1559945826","state":["exists","up"]},{"osd":1,"uuid":"3fb2385d-6a34-49e3-88e1-116237beb346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6801","nonce":2996505353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6803","nonce":2996505353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6807","nonce":2996505353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6805","nonce":2996505353}]},"public_addr":"172.21.15.96:6801/2996505353","cluster_addr":"172.21.15.96:6803/2996505353","heartbeat_back_addr":"172.21.15.96:6807/2996505353","heartbeat_front_addr":"172.21.15.96:6805/2996505353","state":["exists","up"]},{"osd":2,"uuid":"e306e88e-a308-4585-ae11-c8af10b84038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6801","nonce":1499577897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6803","nonce":1499577897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6807","nonce":1499577897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6805","nonce":1499577897}]},"public_addr":"172.21.15.204:6801/1499577897","cluster_addr":"172.21.15.204:6803/1499577897","heartbeat_back_addr":"172.21.15.204:6807/1499577897","heartbeat_front_addr":"172.21.15.204:6805/1499577897","state":["exists","up"]},{"osd":3,"uuid":"d2a5fb97-6f60-4d45-9f18-447cae480974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6811","nonce":2869340648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6813","nonce":2869340648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6817","nonce":2869340648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6815","nonce":2869340648}]},"public_addr":"172.21.15.84:6811/2869340648","cluster_addr":"172.21.15.84:6813/2869340648","heartbeat_back_addr":"172.21.15.84:6817/2869340648","heartbeat_front_addr":"172.21.15.84:6815/2869340648","state":["exists","up"]},{"osd":4,"uuid":"4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6809","nonce":2554265648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6811","nonce":2554265648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6815","nonce":2554265648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6813","nonce":2554265648}]},"public_addr":"172.21.15.96:6809/2554265648","cluster_addr":"172.21.15.96:6811/2554265648","heartbeat_back_addr":"172.21.15.96:6815/2554265648","heartbeat_front_addr":"172.21.15.96:6813/2554265648","state":["exists","up"]},{"osd":5,"uuid":"7996bf5e-93a5-41b0-a9b1-99e7ef4514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6809","nonce":4075476986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6811","nonce":4075476986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6815","nonce":4075476986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6813","nonce":4075476986}]},"public_addr":"172.21.15.204:6809/4075476986","cluster_addr":"172.21.15.204:6811/4075476986","heartbeat_back_addr":"172.21.15.204:6815/4075476986","heartbeat_front_addr":"172.21.15.204:6813/4075476986","state":["exists","up"]},{"osd":6,"uuid":"f4b7c5de-1836-4d16-8603-3c517524d9fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6819","nonce":1806202865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6821","nonce":1806202865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6825","nonce":1806202865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6823","nonce":1806202865}]},"public_addr":"172.21.15.84:6819/1806202865","cluster_addr":"172.21.15.84:6821/1806202865","heartbeat_back_addr":"172.21.15.84:6825/1806202865","heartbeat_front_addr":"172.21.15.84:6823/1806202865","state":["exists","up"]},{"osd":7,"uuid":"69a18c43-5447-480d-9466-190830070c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6817","nonce":3195792774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6819","nonce":3195792774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6823","nonce":3195792774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6821","nonce":3195792774}]},"public_addr":"172.21.15.96:6817/3195792774","cluster_addr":"172.21.15.96:6819/3195792774","heartbeat_back_addr":"172.21.15.96:6823/3195792774","heartbeat_front_addr":"172.21.15.96:6821/3195792774","state":["exists","up"]},{"osd":8,"uuid":"2c5cadca-689a-441e-847c-ba370d400ef1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6817","nonce":606997927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6819","nonce":606997927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6823","nonce":606997927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6821","nonce":606997927}]},"public_addr":"172.21.15.204:6817/606997927","cluster_addr":"172.21.15.204:6819/606997927","heartbeat_back_addr":"172.21.15.204:6823/606997927","heartbeat_front_addr":"172.21.15.204:6821/606997927","state":["exists","up"]},{"osd":9,"uuid":"e3984fb2-7c00-4275-a10e-bca7c859f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6827","nonce":2786189012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6829","nonce":2786189012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6833","nonce":2786189012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6831","nonce":2786189012}]},"public_addr":"172.21.15.84:6827/2786189012","cluster_addr":"172.21.15.84:6829/2786189012","heartbeat_back_addr":"172.21.15.84:6833/2786189012","heartbeat_front_addr":"172.21.15.84:6831/2786189012","state":["exists","up"]},{"osd":10,"uuid":"b4938160-a808-4ded-91c2-d245c99fff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6825","nonce":1985398526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6827","nonce":1985398526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6831","nonce":1985398526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6829","nonce":1985398526}]},"public_addr":"172.21.15.96:6825/1985398526","cluster_addr":"172.21.15.96:6827/1985398526","heartbeat_back_addr":"172.21.15.96:6831/1985398526","heartbeat_front_addr":"172.21.15.96:6829/1985398526","state":["exists","up"]},{"osd":11,"uuid":"238922ef-d046-40cc-833d-25b7e7a523cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6825","nonce":3106116519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6827","nonce":3106116519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6831","nonce":3106116519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6829","nonce":3106116519}]},"public_addr":"172.21.15.204:6825/3106116519","cluster_addr":"172.21.15.204:6827/3106116519","heartbeat_back_addr":"172.21.15.204:6831/3106116519","heartbeat_front_addr":"172.21.15.204:6829/3106116519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:16.443480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:39.898133+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:03.664544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:28.838687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:53.829733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:19.269307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:45.166910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:10.771614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:36.678516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:03.150592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:29.327769+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:56.315938+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2317224739":"2024-05-23T23:01:49.869856+0000","172.21.15.84:0/4170743349":"2024-05-23T23:01:49.869856+0000","172.21.15.84:6801/4223589140":"2024-05-23T23:01:49.869856+0000","172.21.15.84:6800/4223589140":"2024-05-23T23:01:49.869856+0000","172.21.15.84:6801/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:6800/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/2766288608":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/950455999":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/3490117794":"2024-05-23T23:01:49.869856+0000","172.21.15.84:0/4000289608":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1650467363":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/4077200298":"2024-05-23T23:01:49.869856+0000","172.21.15.84:0/1052746114":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6800/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6801/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/14610813":"2024-05-23T22:52:45.395791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T23:01:53.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.662+0000 7f7612f14640 1 -- 172.21.15.84:0/2548137022 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f75fc0a2d00 msgr2=0x7f75fc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.662+0000 7f7612f14640 1 --2- 172.21.15.84:0/2548137022 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f75fc0a2d00 0x7f75fc0a51c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7600000c00 tx=0x7f76000315b0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.662+0000 7f7612f14640 1 -- 172.21.15.84:0/2548137022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f760c104d00 msgr2=0x7f760c19a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.662+0000 7f7612f14640 1 --2- 172.21.15.84:0/2548137022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f760c104d00 0x7f760c19a2e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f75f800e9c0 tx=0x7f75f800ee80 comp rx=0 tx=0).stop 2024-05-22T23:01:53.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.662+0000 7f7612f14640 1 -- 172.21.15.84:0/2548137022 shutdown_connections 2024-05-22T23:01:53.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.662+0000 7f7612f14640 1 --2- 172.21.15.84:0/2548137022 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f75fc0a2d00 0x7f75fc0a51c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.662+0000 7f7612f14640 1 --2- 172.21.15.84:0/2548137022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f760c107630 0x7f760c1a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.662+0000 7f7612f14640 1 --2- 172.21.15.84:0/2548137022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f760c104d00 0x7f760c19a2e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.662+0000 7f7612f14640 1 --2- 172.21.15.84:0/2548137022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f760c1023d0 0x7f760c199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.662+0000 7f7612f14640 1 -- 172.21.15.84:0/2548137022 >> 172.21.15.84:0/2548137022 conn(0x7f760c0fbd20 msgr2=0x7f760c101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:53.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.662+0000 7f7612f14640 1 -- 172.21.15.84:0/2548137022 shutdown_connections 2024-05-22T23:01:53.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.662+0000 7f7612f14640 1 -- 172.21.15.84:0/2548137022 wait complete. 2024-05-22T23:01:53.676 DEBUG:tasks.fwd_scrub:Yielding 2024-05-22T23:01:53.676 INFO:teuthology.run_tasks:Running task exec... 2024-05-22T23:01:53.686 INFO:teuthology.task.exec:Executing custom commands... 2024-05-22T23:01:53.686 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi084.front.sepia.ceph.com 2024-05-22T23:01:53.686 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-05-22T23:01:53.687 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-22T23:01:53.687 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-22T23:01:53.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:01:53.687 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-22T23:01:53.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.792+0000 7f5583cda640 1 -- 172.21.15.84:0/47430919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f557c102380 msgr2=0x7f557c108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.792+0000 7f5583cda640 1 --2- 172.21.15.84:0/47430919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f557c102380 0x7f557c108c10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5574009ca0 tx=0x7f557402a8c0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.792+0000 7f5583cda640 1 -- 172.21.15.84:0/47430919 shutdown_connections 2024-05-22T23:01:53.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.792+0000 7f5583cda640 1 --2- 172.21.15.84:0/47430919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f557c102380 0x7f557c108c10 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.792+0000 7f5583cda640 1 --2- 172.21.15.84:0/47430919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f557c1019c0 0x7f557c101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.792+0000 7f5583cda640 1 --2- 172.21.15.84:0/47430919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f557c1007d0 0x7f557c100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.792+0000 7f5583cda640 1 -- 172.21.15.84:0/47430919 >> 172.21.15.84:0/47430919 conn(0x7f557c0fbd40 msgr2=0x7f557c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:53.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.792+0000 7f5583cda640 1 -- 172.21.15.84:0/47430919 shutdown_connections 2024-05-22T23:01:53.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.792+0000 7f5583cda640 1 -- 172.21.15.84:0/47430919 wait complete. 2024-05-22T23:01:53.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.793+0000 7f5583cda640 1 Processor -- start 2024-05-22T23:01:53.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.793+0000 7f5583cda640 1 -- start start 2024-05-22T23:01:53.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.793+0000 7f5583cda640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f557c1007d0 0x7f557c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.793+0000 7f5583cda640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f557c1019c0 0x7f557c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.793+0000 7f5583cda640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f557c102380 0x7f557c19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.793+0000 7f5583cda640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f557c10c450 con 0x7f557c1019c0 2024-05-22T23:01:53.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.793+0000 7f5583cda640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f557c10c2d0 con 0x7f557c102380 2024-05-22T23:01:53.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.793+0000 7f5583cda640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f557c10c5d0 con 0x7f557c1007d0 2024-05-22T23:01:53.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.793+0000 7f5581275640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f557c1019c0 0x7f557c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.793+0000 7f5581275640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f557c1019c0 0x7f557c198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58836/0 (socket says 172.21.15.84:58836) 2024-05-22T23:01:53.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.793+0000 7f5581275640 1 -- 172.21.15.84:0/691040664 learned_addr learned my addr 172.21.15.84:0/691040664 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:53.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.793+0000 7f5582277640 1 --2- 172.21.15.84:0/691040664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f557c102380 0x7f557c19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.793+0000 7f5581275640 1 -- 172.21.15.84:0/691040664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f557c1007d0 msgr2=0x7f557c197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.793+0000 7f5581a76640 1 --2- 172.21.15.84:0/691040664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f557c1007d0 0x7f557c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.793+0000 7f5581275640 1 --2- 172.21.15.84:0/691040664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f557c1007d0 0x7f557c197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.793+0000 7f5581275640 1 -- 172.21.15.84:0/691040664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f557c102380 msgr2=0x7f557c19f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.793+0000 7f5581275640 1 --2- 172.21.15.84:0/691040664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f557c102380 0x7f557c19f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.793+0000 7f5581275640 1 -- 172.21.15.84:0/691040664 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557c19f740 con 0x7f557c1019c0 2024-05-22T23:01:53.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.793+0000 7f5582277640 1 --2- 172.21.15.84:0/691040664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f557c102380 0x7f557c19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:01:53.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.794+0000 7f5581275640 1 --2- 172.21.15.84:0/691040664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f557c1019c0 0x7f557c198060 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f556c00e9f0 tx=0x7f556c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:53.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.794+0000 7f5572ffd640 1 -- 172.21.15.84:0/691040664 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c00cd80 con 0x7f557c1019c0 2024-05-22T23:01:53.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.794+0000 7f5572ffd640 1 -- 172.21.15.84:0/691040664 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f556c005500 con 0x7f557c1019c0 2024-05-22T23:01:53.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.794+0000 7f5583cda640 1 -- 172.21.15.84:0/691040664 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f557c19fa30 con 0x7f557c1019c0 2024-05-22T23:01:53.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.794+0000 7f5572ffd640 1 -- 172.21.15.84:0/691040664 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c010690 con 0x7f557c1019c0 2024-05-22T23:01:53.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.794+0000 7f5583cda640 1 -- 172.21.15.84:0/691040664 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f557c19ff80 con 0x7f557c1019c0 2024-05-22T23:01:53.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.795+0000 7f5583cda640 1 -- 172.21.15.84:0/691040664 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f557c1a0270 con 0x7f557c1019c0 2024-05-22T23:01:53.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.796+0000 7f5572ffd640 1 -- 172.21.15.84:0/691040664 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f556c0046c0 con 0x7f557c1019c0 2024-05-22T23:01:53.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.797+0000 7f5572ffd640 1 --2- 172.21.15.84:0/691040664 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f55680a2e20 0x7f55680a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.797+0000 7f5581a76640 1 --2- 172.21.15.84:0/691040664 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f55680a2e20 0x7f55680a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.797+0000 7f5572ffd640 1 -- 172.21.15.84:0/691040664 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f556c0cfb20 con 0x7f557c1019c0 2024-05-22T23:01:53.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.798+0000 7f5581a76640 1 --2- 172.21.15.84:0/691040664 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f55680a2e20 0x7f55680a52e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f557c101820 tx=0x7f5578005f90 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:53.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.799+0000 7f3ecedb6640 1 -- 172.21.15.84:0/1064101256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ec8102380 msgr2=0x7f3ec8108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.799+0000 7f3ecedb6640 1 --2- 172.21.15.84:0/1064101256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ec8102380 0x7f3ec8108c10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009be0 tx=0x7f3ec002c640 comp rx=0 tx=0).stop 2024-05-22T23:01:53.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.799+0000 7f3ecedb6640 1 -- 172.21.15.84:0/1064101256 shutdown_connections 2024-05-22T23:01:53.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.799+0000 7f3ecedb6640 1 --2- 172.21.15.84:0/1064101256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ec8102380 0x7f3ec8108c10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.799+0000 7f3ecedb6640 1 --2- 172.21.15.84:0/1064101256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ec81019c0 0x7f3ec8101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.799+0000 7f3ecedb6640 1 --2- 172.21.15.84:0/1064101256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ec81007d0 0x7f3ec8100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.799+0000 7f3ecedb6640 1 -- 172.21.15.84:0/1064101256 >> 172.21.15.84:0/1064101256 conn(0x7f3ec80fbd60 msgr2=0x7f3ec80fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:53.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.799+0000 7f3ecedb6640 1 -- 172.21.15.84:0/1064101256 shutdown_connections 2024-05-22T23:01:53.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.799+0000 7f3ecedb6640 1 -- 172.21.15.84:0/1064101256 wait complete. 2024-05-22T23:01:53.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.800+0000 7f3ecedb6640 1 Processor -- start 2024-05-22T23:01:53.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.800+0000 7f3ecedb6640 1 -- start start 2024-05-22T23:01:53.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.800+0000 7f3ecedb6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ec81007d0 0x7f3ec8195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.800+0000 7f3ecedb6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ec81019c0 0x7f3ec8195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.800+0000 7f3ecedb6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ec8102380 0x7f3ec819cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.800+0000 7f3ecedb6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec810b350 con 0x7f3ec81007d0 2024-05-22T23:01:53.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.800+0000 7f3ecedb6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec810b1d0 con 0x7f3ec81019c0 2024-05-22T23:01:53.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.800+0000 7f3ecedb6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec810b4d0 con 0x7f3ec8102380 2024-05-22T23:01:53.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.800+0000 7f3eccb52640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ec81007d0 0x7f3ec8195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.801+0000 7f3eccb52640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ec81007d0 0x7f3ec8195a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58854/0 (socket says 172.21.15.84:58854) 2024-05-22T23:01:53.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.801+0000 7f3eccb52640 1 -- 172.21.15.84:0/3505582840 learned_addr learned my addr 172.21.15.84:0/3505582840 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:53.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.801+0000 7f3ec7fff640 1 --2- 172.21.15.84:0/3505582840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ec81019c0 0x7f3ec8195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.801+0000 7f3ecd353640 1 --2- 172.21.15.84:0/3505582840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ec8102380 0x7f3ec819cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.801+0000 7f3eccb52640 1 -- 172.21.15.84:0/3505582840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ec8102380 msgr2=0x7f3ec819cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.801+0000 7f3eccb52640 1 --2- 172.21.15.84:0/3505582840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ec8102380 0x7f3ec819cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.801+0000 7f3eccb52640 1 -- 172.21.15.84:0/3505582840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ec81019c0 msgr2=0x7f3ec8195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.801+0000 7f3eccb52640 1 --2- 172.21.15.84:0/3505582840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ec81019c0 0x7f3ec8195f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.801+0000 7f3eccb52640 1 -- 172.21.15.84:0/3505582840 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec819d620 con 0x7f3ec81007d0 2024-05-22T23:01:53.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.801+0000 7f3ec7fff640 1 --2- 172.21.15.84:0/3505582840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ec81019c0 0x7f3ec8195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:01:53.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.801+0000 7f3eccb52640 1 --2- 172.21.15.84:0/3505582840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ec81007d0 0x7f3ec8195a00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009ec0 tx=0x7f3ebc00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:53.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.801+0000 7f3ec5ffb640 1 -- 172.21.15.84:0/3505582840 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc00ccc0 con 0x7f3ec81007d0 2024-05-22T23:01:53.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.801+0000 7f3ecedb6640 1 -- 172.21.15.84:0/3505582840 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec819d910 con 0x7f3ec81007d0 2024-05-22T23:01:53.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.801+0000 7f3ec5ffb640 1 -- 172.21.15.84:0/3505582840 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ebc0137d0 con 0x7f3ec81007d0 2024-05-22T23:01:53.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.802+0000 7f3ec5ffb640 1 -- 172.21.15.84:0/3505582840 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc003850 con 0x7f3ec81007d0 2024-05-22T23:01:53.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.802+0000 7f5572ffd640 1 -- 172.21.15.84:0/691040664 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f556c053850 con 0x7f557c1019c0 2024-05-22T23:01:53.811 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.802+0000 7f3ecedb6640 1 -- 172.21.15.84:0/3505582840 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec819deb0 con 0x7f3ec81007d0 2024-05-22T23:01:53.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.805+0000 7f3ed1b9b740 1 Processor -- start 2024-05-22T23:01:53.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.805+0000 7f3ed1b9b740 1 -- start start 2024-05-22T23:01:53.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.805+0000 7f3ec5ffb640 1 -- 172.21.15.84:0/3505582840 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f3ebc022020 con 0x7f3ec81007d0 2024-05-22T23:01:53.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.805+0000 7f3ed1b9b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e715a4900 0x564e715a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.805+0000 7f3ed1b9b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e715a52f0 0x564e715a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.805+0000 7f3ed1b9b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e715aa460 0x564e715ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.805+0000 7f3ed1b9b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564e71507a40 con 0x564e715aa460 2024-05-22T23:01:53.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.805+0000 7f3ed1b9b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564e714a6440 con 0x564e715a4900 2024-05-22T23:01:53.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.805+0000 7f3ed1b9b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564e71503a00 con 0x564e715a52f0 2024-05-22T23:01:53.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.805+0000 7f3ecd353640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e715a4900 0x564e715a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.805+0000 7f3eccb52640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e715a52f0 0x564e715a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.805+0000 7f3ecd353640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e715a4900 0x564e715a4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:55182/0 (socket says 172.21.15.84:55182) 2024-05-22T23:01:53.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.805+0000 7f3ecd353640 1 -- 172.21.15.84:0/4215723455 learned_addr learned my addr 172.21.15.84:0/4215723455 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:53.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.805+0000 7f3ec7fff640 1 --2- 172.21.15.84:0/4215723455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e715aa460 0x564e715ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.806+0000 7f3ecd353640 1 -- 172.21.15.84:0/4215723455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e715a52f0 msgr2=0x564e715a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:53.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.806+0000 7f3ecd353640 1 --2- 172.21.15.84:0/4215723455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e715a52f0 0x564e715a9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.806+0000 7f3ecd353640 1 -- 172.21.15.84:0/4215723455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e715aa460 msgr2=0x564e715ac8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:53.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.806+0000 7f3ecd353640 1 --2- 172.21.15.84:0/4215723455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e715aa460 0x564e715ac8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.806+0000 7f3ecd353640 1 -- 172.21.15.84:0/4215723455 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e714b02d0 con 0x564e715a4900 2024-05-22T23:01:53.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.806+0000 7f3ecd353640 1 --2- 172.21.15.84:0/4215723455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e715a4900 0x564e715a4d20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009be0 tx=0x7f3ec002ce40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5f967b13a2a95a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:53.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.806+0000 7f3ec5ffb640 1 --2- 172.21.15.84:0/3505582840 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f3eb80a2d00 0x7f3eb80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.806+0000 7f3eb2ffd640 1 -- 172.21.15.84:0/4215723455 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0042070 con 0x564e715a4900 2024-05-22T23:01:53.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.806+0000 7f3eccb52640 1 --2- 172.21.15.84:0/3505582840 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f3eb80a2d00 0x7f3eb80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.806+0000 7f3eb2ffd640 1 -- 172.21.15.84:0/4215723455 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ec0003970 con 0x564e715a4900 2024-05-22T23:01:53.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.806+0000 7f3eb2ffd640 1 -- 172.21.15.84:0/4215723455 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0038040 con 0x564e715a4900 2024-05-22T23:01:53.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.806+0000 7f3ec5ffb640 1 -- 172.21.15.84:0/3505582840 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f3ebc0d3460 con 0x7f3ec81007d0 2024-05-22T23:01:53.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.807+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/4215723455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e715a4900 msgr2=0x564e715a4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.807+0000 7f3eccb52640 1 --2- 172.21.15.84:0/3505582840 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f3eb80a2d00 0x7f3eb80a51c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0d0d70 tx=0x7f3ebc0d9060 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:53.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.807+0000 7f3ed1b9b740 1 --2- 172.21.15.84:0/4215723455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e715a4900 0x564e715a4d20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009be0 tx=0x7f3ec002ce40 comp rx=0 tx=0).stop 2024-05-22T23:01:53.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.807+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/4215723455 shutdown_connections 2024-05-22T23:01:53.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.807+0000 7f3ed1b9b740 1 --2- 172.21.15.84:0/4215723455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e715aa460 0x564e715ac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.807+0000 7f3ed1b9b740 1 --2- 172.21.15.84:0/4215723455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e715a52f0 0x564e715a9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.807+0000 7f3ed1b9b740 1 --2- 172.21.15.84:0/4215723455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e715a4900 0x564e715a4d20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.807+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/4215723455 >> 172.21.15.84:0/4215723455 conn(0x564e7159ff70 msgr2=0x564e715a0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:53.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.807+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/4215723455 shutdown_connections 2024-05-22T23:01:53.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.807+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/4215723455 wait complete. 2024-05-22T23:01:53.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.807+0000 7f3ed1b9b740 1 -- start start 2024-05-22T23:01:53.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.807+0000 7f3ed1b9b740 1 Processor -- start 2024-05-22T23:01:53.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.808+0000 7f3ed1b9b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e715a4900 0x564e715aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.808+0000 7f3ed1b9b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e715a52f0 0x564e715af200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.808+0000 7f3ed1b9b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e715aa460 0x564e715b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.808+0000 7f3ed1b9b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564e715ad3b0 con 0x564e715a52f0 2024-05-22T23:01:53.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.808+0000 7f3ed1b9b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564e714915f0 con 0x564e715aa460 2024-05-22T23:01:53.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.808+0000 7f3ed1b9b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564e715ad530 con 0x564e715a4900 2024-05-22T23:01:53.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.808+0000 7f3ed1b9b740 10 client.0 _ll_register_callbacks cb 0x564e7148ccf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:01:53.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.808+0000 7f3ec7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e715a52f0 0x564e715af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.808+0000 7f3ecd353640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e715a4900 0x564e715aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.808+0000 7f3ec7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e715a52f0 0x564e715af200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58878/0 (socket says 172.21.15.84:58878) 2024-05-22T23:01:53.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.808+0000 7f3ec7fff640 1 -- 172.21.15.84:0/1050960666 learned_addr learned my addr 172.21.15.84:0/1050960666 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:53.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.808+0000 7f3ecd353640 1 --2- 172.21.15.84:0/1050960666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e715aa460 0x564e715b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.808+0000 7f3ec7fff640 1 -- 172.21.15.84:0/1050960666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e715a4900 msgr2=0x564e715aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:53.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.808+0000 7f3ec7fff640 1 --2- 172.21.15.84:0/1050960666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e715a4900 0x564e715aecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.808+0000 7f3ec7fff640 1 -- 172.21.15.84:0/1050960666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e715aa460 msgr2=0x564e715b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:53.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.808+0000 7f3ec7fff640 1 --2- 172.21.15.84:0/1050960666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e715aa460 0x564e715b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.808+0000 7f3ec7fff640 1 -- 172.21.15.84:0/1050960666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e712e37e0 con 0x564e715a52f0 2024-05-22T23:01:53.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.808+0000 7f3ecd353640 1 --2- 172.21.15.84:0/1050960666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e715aa460 0x564e715b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:01:53.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.809+0000 7f3ec7fff640 1 --2- 172.21.15.84:0/1050960666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e715a52f0 0x564e715af200 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4012a80 tx=0x7f3eb4012f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb5c388859ac3a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:53.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.809+0000 7f3ea7fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:01:53.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.809+0000 7f3ea7fff640 1 -- 172.21.15.84:0/1050960666 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb400fb20 con 0x564e715a52f0 2024-05-22T23:01:53.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.809+0000 7f3ea7fff640 1 -- 172.21.15.84:0/1050960666 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f3eb400fcc0 con 0x564e715a52f0 2024-05-22T23:01:53.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.809+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/1050960666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ec0003970 con 0x564e715a52f0 2024-05-22T23:01:53.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.809+0000 7f3ea7fff640 1 -- 172.21.15.84:0/1050960666 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f3eb4004050 con 0x564e715a52f0 2024-05-22T23:01:53.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.809+0000 7f3ed1b9b740 10 client.15360 fetch_fsmap learned FSMap version 18 2024-05-22T23:01:53.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.809+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/1050960666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x564e71343f70 con 0x564e715a52f0 2024-05-22T23:01:53.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.809+0000 7f3ea7fff640 1 -- 172.21.15.84:0/1050960666 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ec0003970 con 0x564e715a52f0 2024-05-22T23:01:53.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.809+0000 7f3ea7fff640 1 -- 172.21.15.84:0/1050960666 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f3eb4025480 con 0x564e715a52f0 2024-05-22T23:01:53.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.810+0000 7f3ed1b9b740 10 client.15360 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:01:53.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.810+0000 7f3ed1b9b740 10 client.15360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:01:53.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.810+0000 7f3ed1b9b740 20 client.15360 populate_metadata read hostname 'smithi084' 2024-05-22T23:01:53.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.810+0000 7f3ed1b9b740 1 --2- 172.21.15.84:0/1050960666 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564e715baba0 0x564e715bcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.810+0000 7f3ed1b9b740 4 client.15360 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:01:53.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.810+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/1050960666 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e7151b460 con 0x564e715baba0 2024-05-22T23:01:53.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.810+0000 7f3ecd353640 1 --2- 172.21.15.84:0/1050960666 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564e715baba0 0x564e715bcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.810+0000 7f3ea7fff640 10 client.15360 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:01:53.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.810+0000 7f3ecd353640 1 --2- 172.21.15.84:0/1050960666 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564e715baba0 0x564e715bcfb0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1036435a742719f1 server_cookie=676227689c8b8fff in_seq=0 out_seq=0 2024-05-22T23:01:53.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.811+0000 7f3ea7fff640 1 -- 172.21.15.84:0/1050960666 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564e7151b460 con 0x564e715baba0 2024-05-22T23:01:53.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.811+0000 7f3ea7fff640 10 client.15360 handle_command_reply: tid=0 2024-05-22T23:01:53.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.811+0000 7f3ed1b9b740 1 client.15360 shutdown 2024-05-22T23:01:53.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.811+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/1050960666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e715a52f0 msgr2=0x564e715af200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.811+0000 7f3ed1b9b740 1 --2- 172.21.15.84:0/1050960666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e715a52f0 0x564e715af200 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4012a80 tx=0x7f3eb4012f40 comp rx=0 tx=0).stop 2024-05-22T23:01:53.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.811+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/1050960666 shutdown_connections 2024-05-22T23:01:53.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.811+0000 7f3ed1b9b740 1 --2- 172.21.15.84:0/1050960666 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564e715baba0 0x564e715bcfb0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.811+0000 7f3ed1b9b740 1 --2- 172.21.15.84:0/1050960666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e715aa460 0x564e715b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.811+0000 7f3ed1b9b740 1 --2- 172.21.15.84:0/1050960666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e715a52f0 0x564e715af200 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.811+0000 7f3ea7fff640 0 client.15360 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:01:53.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.811+0000 7f3ed1b9b740 1 --2- 172.21.15.84:0/1050960666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e715a4900 0x564e715aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.811+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/1050960666 >> 172.21.15.84:0/1050960666 conn(0x564e7159ff70 msgr2=0x564e7159f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:53.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.811+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/1050960666 shutdown_connections 2024-05-22T23:01:53.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.811+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/1050960666 wait complete. 2024-05-22T23:01:53.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.811+0000 7f3ed1b9b740 20 client.15360 trim_cache size 0 max 16384 2024-05-22T23:01:53.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.844+0000 7f3ed1b9b740 1 Processor -- start 2024-05-22T23:01:53.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.844+0000 7f3ed1b9b740 1 -- start start 2024-05-22T23:01:53.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.844+0000 7f3ed1b9b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e7159ff70 0x564e715a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.844+0000 7f3ed1b9b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e715a4900 0x564e715b6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.844+0000 7f3ed1b9b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e715a52f0 0x564e715a3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.844+0000 7f3ed1b9b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec81a4fd0 con 0x564e7159ff70 2024-05-22T23:01:53.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.844+0000 7f3ed1b9b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec81a7850 con 0x564e715a52f0 2024-05-22T23:01:53.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.844+0000 7f3ed1b9b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec81a5640 con 0x564e715a4900 2024-05-22T23:01:53.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.844+0000 7f3ecd353640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e7159ff70 0x564e715a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.844+0000 7f3ecd353640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e7159ff70 0x564e715a2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58890/0 (socket says 172.21.15.84:58890) 2024-05-22T23:01:53.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.844+0000 7f3ecd353640 1 -- 172.21.15.84:0/768797972 learned_addr learned my addr 172.21.15.84:0/768797972 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:53.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.844+0000 7f3ecd353640 1 --2- 172.21.15.84:0/768797972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e715a52f0 0x564e715a3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.844+0000 7f3ec7fff640 1 --2- 172.21.15.84:0/768797972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e715a4900 0x564e715b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.845+0000 7f3ecd353640 1 -- 172.21.15.84:0/768797972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e715a4900 msgr2=0x564e715b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:53.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.845+0000 7f3ecd353640 1 --2- 172.21.15.84:0/768797972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e715a4900 0x564e715b6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.845+0000 7f3ecd353640 1 -- 172.21.15.84:0/768797972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e715a52f0 msgr2=0x564e715a3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:53.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.845+0000 7f3ecd353640 1 --2- 172.21.15.84:0/768797972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e715a52f0 0x564e715a3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.845+0000 7f3ecd353640 1 -- 172.21.15.84:0/768797972 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e711db170 con 0x564e7159ff70 2024-05-22T23:01:53.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.845+0000 7f3ecd353640 1 --2- 172.21.15.84:0/768797972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e7159ff70 0x564e715a2c50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009600 tx=0x7f3ec002c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f5980e73cf6dde server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:53.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.845+0000 7f3eb27fc640 1 -- 172.21.15.84:0/768797972 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0042040 con 0x564e7159ff70 2024-05-22T23:01:53.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.845+0000 7f3eb27fc640 1 -- 172.21.15.84:0/768797972 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ec00573d0 con 0x564e7159ff70 2024-05-22T23:01:53.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.845+0000 7f3eb27fc640 1 -- 172.21.15.84:0/768797972 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0004360 con 0x564e7159ff70 2024-05-22T23:01:53.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.846+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/768797972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e7159ff70 msgr2=0x564e715a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.846+0000 7f3ed1b9b740 1 --2- 172.21.15.84:0/768797972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e7159ff70 0x564e715a2c50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009600 tx=0x7f3ec002c640 comp rx=0 tx=0).stop 2024-05-22T23:01:53.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.846+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/768797972 shutdown_connections 2024-05-22T23:01:53.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.846+0000 7f3ed1b9b740 1 --2- 172.21.15.84:0/768797972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e715a52f0 0x564e715a3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.846+0000 7f3ed1b9b740 1 --2- 172.21.15.84:0/768797972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e715a4900 0x564e715b6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.846+0000 7f3ed1b9b740 1 --2- 172.21.15.84:0/768797972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e7159ff70 0x564e715a2c50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.846+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/768797972 >> 172.21.15.84:0/768797972 conn(0x564e7159b3b0 msgr2=0x564e715a6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:53.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.846+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/768797972 shutdown_connections 2024-05-22T23:01:53.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.846+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/768797972 wait complete. 2024-05-22T23:01:53.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.847+0000 7f3ed1b9b740 1 -- start start 2024-05-22T23:01:53.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.847+0000 7f3ed1b9b740 1 Processor -- start 2024-05-22T23:01:53.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.847+0000 7f3ed1b9b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e7159ff70 0x564e715b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.847+0000 7f3ed1b9b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e715a4900 0x564e715b4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.848+0000 7f3ecd353640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e7159ff70 0x564e715b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.848+0000 7f3ecd353640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e7159ff70 0x564e715b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:60420/0 (socket says 172.21.15.84:60420) 2024-05-22T23:01:53.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.848+0000 7f3ec7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e715a4900 0x564e715b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.848+0000 7f3ec7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e715a4900 0x564e715b4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58892/0 (socket says 172.21.15.84:58892) 2024-05-22T23:01:53.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.848+0000 7f3ecd353640 1 -- 172.21.15.84:0/1626662232 learned_addr learned my addr 172.21.15.84:0/1626662232 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:53.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.848+0000 7f3ed1b9b740 1 --2- 172.21.15.84:0/1626662232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e715a52f0 0x564e715b5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.848+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/1626662232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec81a5d10 con 0x564e715a4900 2024-05-22T23:01:53.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.848+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/1626662232 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec81a7180 con 0x564e715a52f0 2024-05-22T23:01:53.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.848+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/1626662232 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec81a63e0 con 0x564e7159ff70 2024-05-22T23:01:53.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.848+0000 7f3ed1b9b740 10 client.0 _ll_register_callbacks cb 0x7f3eb4009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:01:53.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.848+0000 7f3ecd353640 1 --2- 172.21.15.84:0/1626662232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e715a52f0 0x564e715b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.848+0000 7f3ec7fff640 1 -- 172.21.15.84:0/1626662232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e7159ff70 msgr2=0x564e715b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:53.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.848+0000 7f3ec7fff640 1 --2- 172.21.15.84:0/1626662232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e7159ff70 0x564e715b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.848+0000 7f3ec7fff640 1 -- 172.21.15.84:0/1626662232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e715a52f0 msgr2=0x564e715b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:53.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.848+0000 7f3ec7fff640 1 --2- 172.21.15.84:0/1626662232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e715a52f0 0x564e715b5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.848+0000 7f3ec7fff640 1 -- 172.21.15.84:0/1626662232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e714d0930 con 0x564e715a4900 2024-05-22T23:01:53.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.848+0000 7f3ecd353640 1 --2- 172.21.15.84:0/1626662232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e7159ff70 0x564e715b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:01:53.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.849+0000 7f3ec7fff640 1 --2- 172.21.15.84:0/1626662232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e715a4900 0x564e715b4250 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4025a60 tx=0x7f3eb4012990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd861389a501bff server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:53.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.849+0000 7f3ea7fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:01:53.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.849+0000 7f3ea7fff640 1 -- 172.21.15.84:0/1626662232 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4025ae0 con 0x564e715a4900 2024-05-22T23:01:53.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.849+0000 7f3ea7fff640 1 -- 172.21.15.84:0/1626662232 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f3eb4025c80 con 0x564e715a4900 2024-05-22T23:01:53.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.849+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/1626662232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ec00573d0 con 0x564e715a4900 2024-05-22T23:01:53.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.849+0000 7f3ea7fff640 1 -- 172.21.15.84:0/1626662232 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f3eb4005260 con 0x564e715a4900 2024-05-22T23:01:53.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.849+0000 7f3ed1b9b740 10 client.15366 fetch_fsmap learned FSMap version 18 2024-05-22T23:01:53.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.849+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/1626662232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x564e71621330 con 0x564e715a4900 2024-05-22T23:01:53.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.849+0000 7f3ea7fff640 1 -- 172.21.15.84:0/1626662232 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3eb402c040 con 0x564e715a4900 2024-05-22T23:01:53.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.849+0000 7f3ea7fff640 1 -- 172.21.15.84:0/1626662232 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f3eb4003b40 con 0x564e715a4900 2024-05-22T23:01:53.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.850+0000 7f3ed1b9b740 10 client.15366 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:01:53.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.850+0000 7f3ed1b9b740 10 client.15366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:01:53.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.850+0000 7f3ed1b9b740 20 client.15366 populate_metadata read hostname 'smithi084' 2024-05-22T23:01:53.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.850+0000 7f3ed1b9b740 1 --2- 172.21.15.84:0/1626662232 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564e715aa460 0x564e71623da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.850+0000 7f3ed1b9b740 4 client.15366 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-22T23:01:53.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.850+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/1626662232 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f3ec00573d0 con 0x564e715aa460 2024-05-22T23:01:53.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.850+0000 7f3ecd353640 1 --2- 172.21.15.84:0/1626662232 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564e715aa460 0x564e71623da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.850+0000 7f3ea7fff640 10 client.15366 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:01:53.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.850+0000 7f3ecd353640 1 --2- 172.21.15.84:0/1626662232 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564e715aa460 0x564e71623da0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a0355db79e7e959 server_cookie=304a4531f6e14b49 in_seq=0 out_seq=0 2024-05-22T23:01:53.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.851+0000 7f3ea7fff640 1 -- 172.21.15.84:0/1626662232 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f3ec00573d0 con 0x564e715aa460 2024-05-22T23:01:53.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.851+0000 7f3ea7fff640 10 client.15366 handle_command_reply: tid=0 2024-05-22T23:01:53.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.851+0000 7f3ed1b9b740 1 client.15366 shutdown 2024-05-22T23:01:53.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.851+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/1626662232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e715a4900 msgr2=0x564e715b4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.851+0000 7f3ed1b9b740 1 --2- 172.21.15.84:0/1626662232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e715a4900 0x564e715b4250 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4025a60 tx=0x7f3eb4012990 comp rx=0 tx=0).stop 2024-05-22T23:01:53.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.852+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/1626662232 shutdown_connections 2024-05-22T23:01:53.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.852+0000 7f3ed1b9b740 1 --2- 172.21.15.84:0/1626662232 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564e715aa460 0x564e71623da0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.852+0000 7f3ed1b9b740 1 --2- 172.21.15.84:0/1626662232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e715a52f0 0x564e715b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.852+0000 7f3ed1b9b740 1 --2- 172.21.15.84:0/1626662232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e715a4900 0x564e715b4250 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.852+0000 7f3ed1b9b740 1 --2- 172.21.15.84:0/1626662232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e7159ff70 0x564e715b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.852+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/1626662232 >> 172.21.15.84:0/1626662232 conn(0x564e7159b3b0 msgr2=0x564e7159e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:53.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.852+0000 7f3ea7fff640 0 client.15366 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:01:53.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.852+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/1626662232 shutdown_connections 2024-05-22T23:01:53.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.852+0000 7f3ed1b9b740 1 -- 172.21.15.84:0/1626662232 wait complete. 2024-05-22T23:01:53.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.852+0000 7f3ed1b9b740 20 client.15366 trim_cache size 0 max 16384 2024-05-22T23:01:53.858 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:01:53.858 INFO:teuthology.orchestra.run.smithi084.stdout: "return_code": 0, 2024-05-22T23:01:53.858 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_tag": "d1ecc66c-b50c-4f10-846a-692654a2436d", 2024-05-22T23:01:53.858 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": "asynchronous" 2024-05-22T23:01:53.858 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:01:53.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.853+0000 7f3ecedb6640 1 -- 172.21.15.84:0/3505582840 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f3eb80a2d00 msgr2=0x7f3eb80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.853+0000 7f3ecedb6640 1 --2- 172.21.15.84:0/3505582840 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f3eb80a2d00 0x7f3eb80a51c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0d0d70 tx=0x7f3ebc0d9060 comp rx=0 tx=0).stop 2024-05-22T23:01:53.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.853+0000 7f3ecedb6640 1 -- 172.21.15.84:0/3505582840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ec81007d0 msgr2=0x7f3ec8195a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.853+0000 7f3ecedb6640 1 --2- 172.21.15.84:0/3505582840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ec81007d0 0x7f3ec8195a00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009ec0 tx=0x7f3ebc00ed10 comp rx=0 tx=0).stop 2024-05-22T23:01:53.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.853+0000 7f3ecedb6640 1 -- 172.21.15.84:0/3505582840 shutdown_connections 2024-05-22T23:01:53.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.853+0000 7f3ecedb6640 1 --2- 172.21.15.84:0/3505582840 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f3eb80a2d00 0x7f3eb80a51c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.853+0000 7f3ecedb6640 1 --2- 172.21.15.84:0/3505582840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ec8102380 0x7f3ec819cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.853+0000 7f3ecedb6640 1 --2- 172.21.15.84:0/3505582840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ec81019c0 0x7f3ec8195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.853+0000 7f3ecedb6640 1 --2- 172.21.15.84:0/3505582840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ec81007d0 0x7f3ec8195a00 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.853+0000 7f3ecedb6640 1 -- 172.21.15.84:0/3505582840 >> 172.21.15.84:0/3505582840 conn(0x7f3ec80fbd60 msgr2=0x7f3ec8104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:53.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.853+0000 7f3ecedb6640 1 -- 172.21.15.84:0/3505582840 shutdown_connections 2024-05-22T23:01:53.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.853+0000 7f3ecedb6640 1 -- 172.21.15.84:0/3505582840 wait complete. 2024-05-22T23:01:53.870 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:01:53.871 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:01:53.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.980+0000 7f68b7714640 1 -- 172.21.15.84:0/3090978754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68b0104c60 msgr2=0x7f68b0107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.980+0000 7f68b7714640 1 --2- 172.21.15.84:0/3090978754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68b0104c60 0x7f68b0107050 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f68a0009900 tx=0x7f68a002a410 comp rx=0 tx=0).stop 2024-05-22T23:01:53.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.980+0000 7f68b7714640 1 -- 172.21.15.84:0/3090978754 shutdown_connections 2024-05-22T23:01:53.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.980+0000 7f68b7714640 1 --2- 172.21.15.84:0/3090978754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68b0107590 0x7f68b01099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.980+0000 7f68b7714640 1 --2- 172.21.15.84:0/3090978754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68b0104c60 0x7f68b0107050 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.980+0000 7f68b7714640 1 --2- 172.21.15.84:0/3090978754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b0102330 0x7f68b0104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.980+0000 7f68b7714640 1 -- 172.21.15.84:0/3090978754 >> 172.21.15.84:0/3090978754 conn(0x7f68b00fbd20 msgr2=0x7f68b00fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:53.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.980+0000 7f68b7714640 1 -- 172.21.15.84:0/3090978754 shutdown_connections 2024-05-22T23:01:53.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.980+0000 7f68b7714640 1 -- 172.21.15.84:0/3090978754 wait complete. 2024-05-22T23:01:53.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.981+0000 7f68b7714640 1 Processor -- start 2024-05-22T23:01:53.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.981+0000 7f68b7714640 1 -- start start 2024-05-22T23:01:53.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.981+0000 7f68b7714640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68b0102330 0x7f68b0101ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.981+0000 7f68b7714640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68b0104c60 0x7f68b01002e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.981+0000 7f68b7714640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b0107590 0x7f68b0100820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.981+0000 7f68b7714640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f68b010d520 con 0x7f68b0102330 2024-05-22T23:01:53.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.981+0000 7f68b7714640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f68b010d3a0 con 0x7f68b0107590 2024-05-22T23:01:53.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.981+0000 7f68b7714640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f68b010d6a0 con 0x7f68b0104c60 2024-05-22T23:01:53.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.981+0000 7f68b54b0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68b0102330 0x7f68b0101ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.981+0000 7f68b54b0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68b0102330 0x7f68b0101ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58902/0 (socket says 172.21.15.84:58902) 2024-05-22T23:01:53.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.981+0000 7f68b54b0640 1 -- 172.21.15.84:0/4252216315 learned_addr learned my addr 172.21.15.84:0/4252216315 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:53.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.981+0000 7f68b4caf640 1 --2- 172.21.15.84:0/4252216315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68b0104c60 0x7f68b01002e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.981+0000 7f68b5cb1640 1 --2- 172.21.15.84:0/4252216315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b0107590 0x7f68b0100820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.982+0000 7f68b54b0640 1 -- 172.21.15.84:0/4252216315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68b0104c60 msgr2=0x7f68b01002e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.982+0000 7f68b54b0640 1 --2- 172.21.15.84:0/4252216315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68b0104c60 0x7f68b01002e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.982+0000 7f68b54b0640 1 -- 172.21.15.84:0/4252216315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b0107590 msgr2=0x7f68b0100820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.982+0000 7f68b54b0640 1 --2- 172.21.15.84:0/4252216315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b0107590 0x7f68b0100820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.982+0000 7f68b54b0640 1 -- 172.21.15.84:0/4252216315 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b01010b0 con 0x7f68b0102330 2024-05-22T23:01:53.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.982+0000 7f68b54b0640 1 --2- 172.21.15.84:0/4252216315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68b0102330 0x7f68b0101ca0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00e9f0 tx=0x7f68ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:53.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.982+0000 7f68a67fc640 1 -- 172.21.15.84:0/4252216315 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac00cd80 con 0x7f68b0102330 2024-05-22T23:01:53.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.982+0000 7f68b7714640 1 -- 172.21.15.84:0/4252216315 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b01a1d40 con 0x7f68b0102330 2024-05-22T23:01:53.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.982+0000 7f68a67fc640 1 -- 172.21.15.84:0/4252216315 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68ac010910 con 0x7f68b0102330 2024-05-22T23:01:53.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.982+0000 7f68a67fc640 1 -- 172.21.15.84:0/4252216315 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac018990 con 0x7f68b0102330 2024-05-22T23:01:53.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.982+0000 7f68b7714640 1 -- 172.21.15.84:0/4252216315 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b01a21b0 con 0x7f68b0102330 2024-05-22T23:01:53.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.985+0000 7f68ba423740 1 Processor -- start 2024-05-22T23:01:53.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.985+0000 7f68ba423740 1 -- start start 2024-05-22T23:01:53.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.985+0000 7f68ba423740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557fdba38b0 0x5557fdba3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.985+0000 7f68ba423740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fdba42a0 0x5557fdba8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.985+0000 7f68ba423740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557fdba9410 0x5557fdbab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.985+0000 7f68a67fc640 1 -- 172.21.15.84:0/4252216315 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f68ac010ac0 con 0x7f68b0102330 2024-05-22T23:01:53.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.985+0000 7f68ba423740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5557fdb06db0 con 0x5557fdba9410 2024-05-22T23:01:53.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.985+0000 7f68ba423740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5557fdaa5360 con 0x5557fdba42a0 2024-05-22T23:01:53.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.985+0000 7f68ba423740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5557fdb02d70 con 0x5557fdba38b0 2024-05-22T23:01:53.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.985+0000 7f68b5cb1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557fdba38b0 0x5557fdba3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.985+0000 7f68b5cb1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557fdba38b0 0x5557fdba3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:60458/0 (socket says 172.21.15.84:60458) 2024-05-22T23:01:53.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.985+0000 7f68b4caf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557fdba9410 0x5557fdbab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.985+0000 7f68b5cb1640 1 -- 172.21.15.84:0/958588104 learned_addr learned my addr 172.21.15.84:0/958588104 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:53.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.985+0000 7f68b54b0640 1 --2- 172.21.15.84:0/958588104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fdba42a0 0x5557fdba8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.986+0000 7f68b5cb1640 1 -- 172.21.15.84:0/958588104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fdba42a0 msgr2=0x5557fdba8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:53.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.986+0000 7f68b5cb1640 1 --2- 172.21.15.84:0/958588104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fdba42a0 0x5557fdba8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.986+0000 7f68b5cb1640 1 -- 172.21.15.84:0/958588104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557fdba9410 msgr2=0x5557fdbab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:53.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.986+0000 7f68b5cb1640 1 --2- 172.21.15.84:0/958588104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557fdba9410 0x5557fdbab8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.986+0000 7f68b5cb1640 1 -- 172.21.15.84:0/958588104 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557fdac8a60 con 0x5557fdba38b0 2024-05-22T23:01:53.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.986+0000 7f68b5cb1640 1 --2- 172.21.15.84:0/958588104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557fdba38b0 0x5557fdba3cd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f68a80039f0 tx=0x7f68a8003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=578ea156cbcb888f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:53.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.986+0000 7f68a67fc640 1 --2- 172.21.15.84:0/4252216315 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f689c0a2d00 0x7f689c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.986+0000 7f689b7fe640 1 -- 172.21.15.84:0/958588104 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a801a470 con 0x5557fdba38b0 2024-05-22T23:01:53.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.986+0000 7f68b54b0640 1 --2- 172.21.15.84:0/4252216315 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f689c0a2d00 0x7f689c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.986+0000 7f68a67fc640 1 -- 172.21.15.84:0/4252216315 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f68ac0cf770 con 0x7f68b0102330 2024-05-22T23:01:53.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.986+0000 7f689b7fe640 1 -- 172.21.15.84:0/958588104 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a8015070 con 0x5557fdba38b0 2024-05-22T23:01:53.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.986+0000 7f689b7fe640 1 -- 172.21.15.84:0/958588104 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8010880 con 0x5557fdba38b0 2024-05-22T23:01:53.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.987+0000 7f68ba423740 1 -- 172.21.15.84:0/958588104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557fdba38b0 msgr2=0x5557fdba3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.987+0000 7f68b54b0640 1 --2- 172.21.15.84:0/4252216315 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f689c0a2d00 0x7f689c0a51c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0ccf70 tx=0x7f68ac018420 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:53.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.987+0000 7f68ba423740 1 --2- 172.21.15.84:0/958588104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557fdba38b0 0x5557fdba3cd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f68a80039f0 tx=0x7f68a8003eb0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.987+0000 7f68ba423740 1 -- 172.21.15.84:0/958588104 shutdown_connections 2024-05-22T23:01:53.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.987+0000 7f68ba423740 1 --2- 172.21.15.84:0/958588104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557fdba9410 0x5557fdbab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.987+0000 7f68ba423740 1 --2- 172.21.15.84:0/958588104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fdba42a0 0x5557fdba8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.987+0000 7f68ba423740 1 --2- 172.21.15.84:0/958588104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557fdba38b0 0x5557fdba3cd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.987+0000 7f68ba423740 1 -- 172.21.15.84:0/958588104 >> 172.21.15.84:0/958588104 conn(0x5557fdb6b8e0 msgr2=0x5557fdb9f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:53.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.987+0000 7f68ba423740 1 -- 172.21.15.84:0/958588104 shutdown_connections 2024-05-22T23:01:53.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.987+0000 7f68ba423740 1 -- 172.21.15.84:0/958588104 wait complete. 2024-05-22T23:01:53.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.988+0000 7f68ba423740 1 -- start start 2024-05-22T23:01:53.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.988+0000 7f68ba423740 1 Processor -- start 2024-05-22T23:01:53.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.988+0000 7f68ba423740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557fdba38b0 0x5557fdbadc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.988+0000 7f68ba423740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fdba42a0 0x5557fdbae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.988+0000 7f68ba423740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557fdba9410 0x5557fdbb51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.988+0000 7f68ba423740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5557fdbac3c0 con 0x5557fdba9410 2024-05-22T23:01:53.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.988+0000 7f68ba423740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5557fdac10f0 con 0x5557fdba42a0 2024-05-22T23:01:53.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.988+0000 7f68ba423740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5557fdbac540 con 0x5557fdba38b0 2024-05-22T23:01:53.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.988+0000 7f68ba423740 10 client.0 _ll_register_callbacks cb 0x5557fdac5f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:01:53.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.988+0000 7f68b4caf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fdba42a0 0x5557fdbae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.988+0000 7f68b4caf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fdba42a0 0x5557fdbae1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:55230/0 (socket says 172.21.15.84:55230) 2024-05-22T23:01:53.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.988+0000 7f68b4caf640 1 -- 172.21.15.84:0/1409987916 learned_addr learned my addr 172.21.15.84:0/1409987916 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:53.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.988+0000 7f68b5cb1640 1 --2- 172.21.15.84:0/1409987916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557fdba38b0 0x5557fdbadc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.989+0000 7f68b5cb1640 1 --2- 172.21.15.84:0/1409987916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557fdba9410 0x5557fdbb51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.989+0000 7f68b4caf640 1 -- 172.21.15.84:0/1409987916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557fdba38b0 msgr2=0x5557fdbadc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:53.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.989+0000 7f68b4caf640 1 --2- 172.21.15.84:0/1409987916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557fdba38b0 0x5557fdbadc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.989+0000 7f68b4caf640 1 -- 172.21.15.84:0/1409987916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557fdba9410 msgr2=0x5557fdbb51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:53.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.989+0000 7f68b4caf640 1 --2- 172.21.15.84:0/1409987916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557fdba9410 0x5557fdbb51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.989+0000 7f68b4caf640 1 -- 172.21.15.84:0/1409987916 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557fd8e47d0 con 0x5557fdba42a0 2024-05-22T23:01:53.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.989+0000 7f68b5cb1640 1 --2- 172.21.15.84:0/1409987916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557fdba38b0 0x5557fdbadc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:01:53.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.989+0000 7f68b4caf640 1 --2- 172.21.15.84:0/1409987916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fdba42a0 0x5557fdbae1b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f68a00314e0 tx=0x7f68a00318e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d939e7d16c60a65e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:53.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.989+0000 7f6898ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:01:53.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.989+0000 7f6898ff9640 1 -- 172.21.15.84:0/1409987916 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a0035040 con 0x5557fdba42a0 2024-05-22T23:01:53.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.989+0000 7f6898ff9640 1 -- 172.21.15.84:0/1409987916 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f68a0004040 con 0x5557fdba42a0 2024-05-22T23:01:53.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.989+0000 7f68ba423740 1 -- 172.21.15.84:0/1409987916 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68a8015070 con 0x5557fdba42a0 2024-05-22T23:01:53.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.989+0000 7f6898ff9640 1 -- 172.21.15.84:0/1409987916 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f68a0004240 con 0x5557fdba42a0 2024-05-22T23:01:53.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.989+0000 7f68ba423740 10 client.25028 fetch_fsmap learned FSMap version 18 2024-05-22T23:01:53.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.989+0000 7f68ba423740 1 -- 172.21.15.84:0/1409987916 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5557fd81a3f0 con 0x5557fdba42a0 2024-05-22T23:01:53.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.990+0000 7f6898ff9640 1 -- 172.21.15.84:0/1409987916 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a8015070 con 0x5557fdba42a0 2024-05-22T23:01:53.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.990+0000 7f6898ff9640 1 -- 172.21.15.84:0/1409987916 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f68a0004a80 con 0x5557fdba42a0 2024-05-22T23:01:53.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.990+0000 7f68ba423740 10 client.25028 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:01:53.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.990+0000 7f68ba423740 10 client.25028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:01:53.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.990+0000 7f68ba423740 20 client.25028 populate_metadata read hostname 'smithi084' 2024-05-22T23:01:53.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.990+0000 7f68ba423740 1 --2- 172.21.15.84:0/1409987916 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5557fdbb9b10 0x5557fdbbbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:53.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.990+0000 7f68ba423740 4 client.25028 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:01:53.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.990+0000 7f68ba423740 1 -- 172.21.15.84:0/1409987916 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557fdb144e0 con 0x5557fdbb9b10 2024-05-22T23:01:53.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.990+0000 7f68b5cb1640 1 --2- 172.21.15.84:0/1409987916 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5557fdbb9b10 0x5557fdbbbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:53.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.990+0000 7f68b5cb1640 1 --2- 172.21.15.84:0/1409987916 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5557fdbb9b10 0x5557fdbbbf20 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0735603665e1e3b server_cookie=5164eb83abb1b899 in_seq=0 out_seq=0 2024-05-22T23:01:53.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.990+0000 7f6898ff9640 10 client.25028 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:01:53.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.991+0000 7f6898ff9640 1 -- 172.21.15.84:0/1409987916 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557fdb144e0 con 0x5557fdbb9b10 2024-05-22T23:01:53.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.991+0000 7f6898ff9640 10 client.25028 handle_command_reply: tid=0 2024-05-22T23:01:53.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.991+0000 7f68ba423740 1 client.25028 shutdown 2024-05-22T23:01:53.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.991+0000 7f68ba423740 1 -- 172.21.15.84:0/1409987916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fdba42a0 msgr2=0x5557fdbae1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:53.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.991+0000 7f68ba423740 1 --2- 172.21.15.84:0/1409987916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fdba42a0 0x5557fdbae1b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f68a00314e0 tx=0x7f68a00318e0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.991+0000 7f68ba423740 1 -- 172.21.15.84:0/1409987916 shutdown_connections 2024-05-22T23:01:53.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.991+0000 7f68ba423740 1 --2- 172.21.15.84:0/1409987916 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5557fdbb9b10 0x5557fdbbbf20 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.991+0000 7f68ba423740 1 --2- 172.21.15.84:0/1409987916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557fdba9410 0x5557fdbb51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.991+0000 7f68ba423740 1 --2- 172.21.15.84:0/1409987916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fdba42a0 0x5557fdbae1b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.991+0000 7f6898ff9640 0 client.25028 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:01:53.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.991+0000 7f68ba423740 1 --2- 172.21.15.84:0/1409987916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557fdba38b0 0x5557fdbadc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:53.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.991+0000 7f68ba423740 1 -- 172.21.15.84:0/1409987916 >> 172.21.15.84:0/1409987916 conn(0x5557fdb6b8e0 msgr2=0x5557fdb9e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:53.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.991+0000 7f68ba423740 1 -- 172.21.15.84:0/1409987916 shutdown_connections 2024-05-22T23:01:53.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.991+0000 7f68ba423740 1 -- 172.21.15.84:0/1409987916 wait complete. 2024-05-22T23:01:53.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:53.992+0000 7f68ba423740 20 client.25028 trim_cache size 0 max 16384 2024-05-22T23:01:54.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.007+0000 7f5583cda640 1 -- 172.21.15.84:0/691040664 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f557c1a0440 con 0x7f557c1019c0 2024-05-22T23:01:54.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.010+0000 7f68ba423740 1 Processor -- start 2024-05-22T23:01:54.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.010+0000 7f68ba423740 1 -- start start 2024-05-22T23:01:54.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.010+0000 7f68ba423740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557fdba38b0 0x5557fdba1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:54.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.010+0000 7f68ba423740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fdba42a0 0x5557fdbb51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:54.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.010+0000 7f68ba423740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557fdba9410 0x5557fdba2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:54.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.010+0000 7f68ba423740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f68b01a9320 con 0x5557fdba9410 2024-05-22T23:01:54.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.010+0000 7f68ba423740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f68b01abba0 con 0x5557fdba42a0 2024-05-22T23:01:54.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.010+0000 7f68ba423740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f68b01a9990 con 0x5557fdba38b0 2024-05-22T23:01:54.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.011+0000 7f68b5cb1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557fdba9410 0x5557fdba2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:54.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.011+0000 7f68b5cb1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557fdba9410 0x5557fdba2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58916/0 (socket says 172.21.15.84:58916) 2024-05-22T23:01:54.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.011+0000 7f68b5cb1640 1 -- 172.21.15.84:0/2204717345 learned_addr learned my addr 172.21.15.84:0/2204717345 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:54.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.011+0000 7f68b4caf640 1 --2- 172.21.15.84:0/2204717345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fdba42a0 0x5557fdbb51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:54.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.011+0000 7f68b5cb1640 1 --2- 172.21.15.84:0/2204717345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557fdba38b0 0x5557fdba1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:54.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.011+0000 7f68b5cb1640 1 -- 172.21.15.84:0/2204717345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557fdba38b0 msgr2=0x5557fdba1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:54.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.011+0000 7f68b5cb1640 1 --2- 172.21.15.84:0/2204717345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557fdba38b0 0x5557fdba1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.011+0000 7f68b5cb1640 1 -- 172.21.15.84:0/2204717345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fdba42a0 msgr2=0x5557fdbb51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:54.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.011+0000 7f68b5cb1640 1 --2- 172.21.15.84:0/2204717345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fdba42a0 0x5557fdbb51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.011+0000 7f68b5cb1640 1 -- 172.21.15.84:0/2204717345 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557fd7da0f0 con 0x5557fdba9410 2024-05-22T23:01:54.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.011+0000 7f5572ffd640 1 -- 172.21.15.84:0/691040664 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v32)=0 v32) ==== 129+0+0 (secure 0 0 0) 0x7f556c09c740 con 0x7f557c1019c0 2024-05-22T23:01:54.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.011+0000 7f68b5cb1640 1 --2- 172.21.15.84:0/2204717345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557fdba9410 0x5557fdba2530 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f68a8002230 tx=0x7f68a80096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadee75b54d623f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:54.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.013+0000 7f689affd640 1 -- 172.21.15.84:0/2204717345 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a802b0d0 con 0x5557fdba9410 2024-05-22T23:01:54.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.013+0000 7f689affd640 1 -- 172.21.15.84:0/2204717345 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a8015070 con 0x5557fdba9410 2024-05-22T23:01:54.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.013+0000 7f689affd640 1 -- 172.21.15.84:0/2204717345 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a801a910 con 0x5557fdba9410 2024-05-22T23:01:54.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.013+0000 7f5583cda640 1 -- 172.21.15.84:0/691040664 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f55680a2e20 msgr2=0x7f55680a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:54.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.013+0000 7f5583cda640 1 --2- 172.21.15.84:0/691040664 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f55680a2e20 0x7f55680a52e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f557c101820 tx=0x7f5578005f90 comp rx=0 tx=0).stop 2024-05-22T23:01:54.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.013+0000 7f5583cda640 1 -- 172.21.15.84:0/691040664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f557c1019c0 msgr2=0x7f557c198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:54.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.013+0000 7f5583cda640 1 --2- 172.21.15.84:0/691040664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f557c1019c0 0x7f557c198060 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f556c00e9f0 tx=0x7f556c00edf0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.014+0000 7f5583cda640 1 -- 172.21.15.84:0/691040664 shutdown_connections 2024-05-22T23:01:54.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.014+0000 7f5583cda640 1 --2- 172.21.15.84:0/691040664 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f55680a2e20 0x7f55680a52e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.014+0000 7f5583cda640 1 --2- 172.21.15.84:0/691040664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f557c102380 0x7f557c19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.014+0000 7f5583cda640 1 --2- 172.21.15.84:0/691040664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f557c1019c0 0x7f557c198060 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.014+0000 7f5583cda640 1 --2- 172.21.15.84:0/691040664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f557c1007d0 0x7f557c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.014+0000 7f5583cda640 1 -- 172.21.15.84:0/691040664 >> 172.21.15.84:0/691040664 conn(0x7f557c0fbd40 msgr2=0x7f557c0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:54.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.014+0000 7f5583cda640 1 -- 172.21.15.84:0/691040664 shutdown_connections 2024-05-22T23:01:54.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.014+0000 7f5583cda640 1 -- 172.21.15.84:0/691040664 wait complete. 2024-05-22T23:01:54.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.014+0000 7f68ba423740 1 -- 172.21.15.84:0/2204717345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557fdba9410 msgr2=0x5557fdba2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:54.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.014+0000 7f68ba423740 1 --2- 172.21.15.84:0/2204717345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557fdba9410 0x5557fdba2530 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f68a8002230 tx=0x7f68a80096a0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.014+0000 7f68ba423740 1 -- 172.21.15.84:0/2204717345 shutdown_connections 2024-05-22T23:01:54.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.014+0000 7f68ba423740 1 --2- 172.21.15.84:0/2204717345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557fdba9410 0x5557fdba2530 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.014+0000 7f68ba423740 1 --2- 172.21.15.84:0/2204717345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fdba42a0 0x5557fdbb51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.014+0000 7f68ba423740 1 --2- 172.21.15.84:0/2204717345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557fdba38b0 0x5557fdba1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.014+0000 7f68ba423740 1 -- 172.21.15.84:0/2204717345 >> 172.21.15.84:0/2204717345 conn(0x5557fdb6b8e0 msgr2=0x5557fdba5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:54.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.014+0000 7f68ba423740 1 -- 172.21.15.84:0/2204717345 shutdown_connections 2024-05-22T23:01:54.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.014+0000 7f68ba423740 1 -- 172.21.15.84:0/2204717345 wait complete. 2024-05-22T23:01:54.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.014+0000 7f68ba423740 1 -- start start 2024-05-22T23:01:54.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.014+0000 7f68ba423740 1 Processor -- start 2024-05-22T23:01:54.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.015+0000 7f68ba423740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557fdba38b0 0x5557fdbb23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:54.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.015+0000 7f68ba423740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fdba42a0 0x5557fdbb28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:54.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.015+0000 7f68ba423740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557fdba9410 0x5557fdbb3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:54.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.015+0000 7f68ba423740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f68b01aa060 con 0x5557fdba38b0 2024-05-22T23:01:54.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.015+0000 7f68ba423740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f68b01ab4d0 con 0x5557fdba42a0 2024-05-22T23:01:54.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.015+0000 7f68b5cb1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557fdba38b0 0x5557fdbb23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:54.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.015+0000 7f68ba423740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f68b01aa730 con 0x5557fdba9410 2024-05-22T23:01:54.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.015+0000 7f68b5cb1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557fdba38b0 0x5557fdbb23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58918/0 (socket says 172.21.15.84:58918) 2024-05-22T23:01:54.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.015+0000 7f68b4caf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fdba42a0 0x5557fdbb28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:54.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.015+0000 7f68b5cb1640 1 -- 172.21.15.84:0/3234206348 learned_addr learned my addr 172.21.15.84:0/3234206348 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:54.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.015+0000 7f68ba423740 10 client.0 _ll_register_callbacks cb 0x7f68a0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:01:54.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.015+0000 7f68b4caf640 1 -- 172.21.15.84:0/3234206348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557fdba9410 msgr2=0x5557fdbb3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:54.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.015+0000 7f68b4caf640 1 --2- 172.21.15.84:0/3234206348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557fdba9410 0x5557fdbb3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.015+0000 7f68b4caf640 1 -- 172.21.15.84:0/3234206348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557fdba38b0 msgr2=0x5557fdbb23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:01:54.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.015+0000 7f68b4caf640 1 --2- 172.21.15.84:0/3234206348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557fdba38b0 0x5557fdbb23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.015+0000 7f68b4caf640 1 -- 172.21.15.84:0/3234206348 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557fdac5fe0 con 0x5557fdba42a0 2024-05-22T23:01:54.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.015+0000 7f68b4caf640 1 --2- 172.21.15.84:0/3234206348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fdba42a0 0x5557fdbb28f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f68a0003640 tx=0x7f68a0004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ee32a884c6fdef2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:54.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.015+0000 7f6898ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:01:54.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.016+0000 7f6898ff9640 1 -- 172.21.15.84:0/3234206348 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a0035030 con 0x5557fdba42a0 2024-05-22T23:01:54.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.016+0000 7f68ba423740 1 -- 172.21.15.84:0/3234206348 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68a8015070 con 0x5557fdba42a0 2024-05-22T23:01:54.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.016+0000 7f6898ff9640 1 -- 172.21.15.84:0/3234206348 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f68a0031d10 con 0x5557fdba42a0 2024-05-22T23:01:54.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.016+0000 7f6898ff9640 1 -- 172.21.15.84:0/3234206348 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f68a0053040 con 0x5557fdba42a0 2024-05-22T23:01:54.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.016+0000 7f68ba423740 10 client.25034 fetch_fsmap learned FSMap version 18 2024-05-22T23:01:54.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.016+0000 7f68ba423740 1 -- 172.21.15.84:0/3234206348 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5557fda971c0 con 0x5557fdba42a0 2024-05-22T23:01:54.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.016+0000 7f6898ff9640 1 -- 172.21.15.84:0/3234206348 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a8015070 con 0x5557fdba42a0 2024-05-22T23:01:54.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.016+0000 7f6898ff9640 1 -- 172.21.15.84:0/3234206348 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f68a0004fc0 con 0x5557fdba42a0 2024-05-22T23:01:54.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.016+0000 7f68ba423740 10 client.25034 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:01:54.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.016+0000 7f68ba423740 10 client.25034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:01:54.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.016+0000 7f68ba423740 20 client.25034 populate_metadata read hostname 'smithi084' 2024-05-22T23:01:54.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.016+0000 7f68ba423740 1 --2- 172.21.15.84:0/3234206348 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5557fdbb9b10 0x5557fdc221d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:54.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.016+0000 7f68ba423740 4 client.25034 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:01:54.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.016+0000 7f68ba423740 1 -- 172.21.15.84:0/3234206348 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5557fdb0e830 con 0x5557fdbb9b10 2024-05-22T23:01:54.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.017+0000 7f68b5cb1640 1 --2- 172.21.15.84:0/3234206348 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5557fdbb9b10 0x5557fdc221d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:54.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.017+0000 7f6898ff9640 10 client.25034 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:01:54.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.017+0000 7f68b5cb1640 1 --2- 172.21.15.84:0/3234206348 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5557fdbb9b10 0x5557fdc221d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b95c6b885fc0ba48 server_cookie=36b852ff23c13623 in_seq=0 out_seq=0 2024-05-22T23:01:54.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.017+0000 7f6898ff9640 1 -- 172.21.15.84:0/3234206348 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5557fdb0e830 con 0x5557fdbb9b10 2024-05-22T23:01:54.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.017+0000 7f6898ff9640 10 client.25034 handle_command_reply: tid=0 2024-05-22T23:01:54.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.017+0000 7f68ba423740 1 client.25034 shutdown 2024-05-22T23:01:54.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.017+0000 7f68ba423740 1 -- 172.21.15.84:0/3234206348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fdba42a0 msgr2=0x5557fdbb28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:54.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.017+0000 7f68ba423740 1 --2- 172.21.15.84:0/3234206348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fdba42a0 0x5557fdbb28f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f68a0003640 tx=0x7f68a0004610 comp rx=0 tx=0).stop 2024-05-22T23:01:54.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.018+0000 7f68ba423740 1 -- 172.21.15.84:0/3234206348 shutdown_connections 2024-05-22T23:01:54.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.018+0000 7f68ba423740 1 --2- 172.21.15.84:0/3234206348 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5557fdbb9b10 0x5557fdc221d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.018+0000 7f68ba423740 1 --2- 172.21.15.84:0/3234206348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557fdba9410 0x5557fdbb3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.018+0000 7f6898ff9640 0 client.25034 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:01:54.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.018+0000 7f68ba423740 1 --2- 172.21.15.84:0/3234206348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fdba42a0 0x5557fdbb28f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.018+0000 7f68ba423740 1 --2- 172.21.15.84:0/3234206348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557fdba38b0 0x5557fdbb23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.018+0000 7f68ba423740 1 -- 172.21.15.84:0/3234206348 >> 172.21.15.84:0/3234206348 conn(0x5557fdb6b8e0 msgr2=0x5557fdb9dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:54.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.018+0000 7f68ba423740 1 -- 172.21.15.84:0/3234206348 shutdown_connections 2024-05-22T23:01:54.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.018+0000 7f68ba423740 1 -- 172.21.15.84:0/3234206348 wait complete. 2024-05-22T23:01:54.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.018+0000 7f68ba423740 20 client.25034 trim_cache size 0 max 16384 2024-05-22T23:01:54.026 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:01:54.026 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-22T23:01:54.026 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": { 2024-05-22T23:01:54.026 INFO:teuthology.orchestra.run.smithi084.stdout: "d1ecc66c-b50c-4f10-846a-692654a2436d": { 2024-05-22T23:01:54.026 INFO:teuthology.orchestra.run.smithi084.stdout: "path": "/", 2024-05-22T23:01:54.026 INFO:teuthology.orchestra.run.smithi084.stdout: "tag": "d1ecc66c-b50c-4f10-846a-692654a2436d", 2024-05-22T23:01:54.026 INFO:teuthology.orchestra.run.smithi084.stdout: "options": "recursive,force" 2024-05-22T23:01:54.026 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:01:54.026 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:01:54.027 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:01:54.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.019+0000 7f68b7714640 1 -- 172.21.15.84:0/4252216315 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f689c0a2d00 msgr2=0x7f689c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:54.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.019+0000 7f68b7714640 1 --2- 172.21.15.84:0/4252216315 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f689c0a2d00 0x7f689c0a51c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0ccf70 tx=0x7f68ac018420 comp rx=0 tx=0).stop 2024-05-22T23:01:54.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.019+0000 7f68b7714640 1 -- 172.21.15.84:0/4252216315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68b0102330 msgr2=0x7f68b0101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:54.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.019+0000 7f68b7714640 1 --2- 172.21.15.84:0/4252216315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68b0102330 0x7f68b0101ca0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00e9f0 tx=0x7f68ac00eeb0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.019+0000 7f68b7714640 1 -- 172.21.15.84:0/4252216315 shutdown_connections 2024-05-22T23:01:54.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.019+0000 7f68b7714640 1 --2- 172.21.15.84:0/4252216315 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f689c0a2d00 0x7f689c0a51c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.019+0000 7f68b7714640 1 --2- 172.21.15.84:0/4252216315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b0107590 0x7f68b0100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.019+0000 7f68b7714640 1 --2- 172.21.15.84:0/4252216315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68b0104c60 0x7f68b01002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.019+0000 7f68b7714640 1 --2- 172.21.15.84:0/4252216315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68b0102330 0x7f68b0101ca0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.019+0000 7f68b7714640 1 -- 172.21.15.84:0/4252216315 >> 172.21.15.84:0/4252216315 conn(0x7f68b00fbd20 msgr2=0x7f68b0102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:54.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.019+0000 7f68b7714640 1 -- 172.21.15.84:0/4252216315 shutdown_connections 2024-05-22T23:01:54.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.019+0000 7f68b7714640 1 -- 172.21.15.84:0/4252216315 wait complete. 2024-05-22T23:01:54.030 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/870372379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:01:54.030 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:54.030 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:54.030 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.030 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:54.030 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.030 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:54.030 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.030 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.031 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.031 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.031 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: Adjusting osd_memory_target on smithi204 to 176.2M 2024-05-22T23:01:54.031 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.031 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: Unable to set osd_memory_target on smithi204 to 184759500: error parsing value: Value '184759500' is below minimum 939524096 2024-05-22T23:01:54.031 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.031 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-05-22T23:01:54.031 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: Unable to set osd_memory_target on smithi096 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-05-22T23:01:54.031 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3264661064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:01:54.031 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:54.032 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:54.032 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.032 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.032 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.032 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.032 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: Adjusting osd_memory_target on smithi084 to 176.2M 2024-05-22T23:01:54.032 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: Unable to set osd_memory_target on smithi084 to 184766668: error parsing value: Value '184766668' is below minimum 939524096 2024-05-22T23:01:54.032 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:01:54.032 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:01:54.032 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: Updating smithi084:/etc/ceph/ceph.conf 2024-05-22T23:01:54.032 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: Updating smithi096:/etc/ceph/ceph.conf 2024-05-22T23:01:54.033 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: Updating smithi204:/etc/ceph/ceph.conf 2024-05-22T23:01:54.033 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:53 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2548137022' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:54.034 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-22T23:01:54.035 INFO:tasks.cephfs.filesystem:scrub status for tag:d1ecc66c-b50c-4f10-846a-692654a2436d - {'path': '/', 'tag': 'd1ecc66c-b50c-4f10-846a-692654a2436d', 'options': 'recursive,force'} 2024-05-22T23:01:54.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/870372379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:01:54.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:54.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:54.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:54.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:54.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: Adjusting osd_memory_target on smithi204 to 176.2M 2024-05-22T23:01:54.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: Unable to set osd_memory_target on smithi204 to 184759500: error parsing value: Value '184759500' is below minimum 939524096 2024-05-22T23:01:54.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-05-22T23:01:54.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: Unable to set osd_memory_target on smithi096 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-05-22T23:01:54.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3264661064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:01:54.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:54.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:54.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: Adjusting osd_memory_target on smithi084 to 176.2M 2024-05-22T23:01:54.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: Unable to set osd_memory_target on smithi084 to 184766668: error parsing value: Value '184766668' is below minimum 939524096 2024-05-22T23:01:54.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:01:54.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:01:54.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: Updating smithi084:/etc/ceph/ceph.conf 2024-05-22T23:01:54.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: Updating smithi096:/etc/ceph/ceph.conf 2024-05-22T23:01:54.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: Updating smithi204:/etc/ceph/ceph.conf 2024-05-22T23:01:54.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:53 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2548137022' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:54.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.179+0000 7f8aa7c64640 1 -- 172.21.15.84:0/3064779085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8aa01019c0 msgr2=0x7f8aa0101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:54.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.179+0000 7f8aa7c64640 1 --2- 172.21.15.84:0/3064779085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8aa01019c0 0x7f8aa0101e40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009900 tx=0x7f8a9002a410 comp rx=0 tx=0).stop 2024-05-22T23:01:54.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.179+0000 7f8aa7c64640 1 -- 172.21.15.84:0/3064779085 shutdown_connections 2024-05-22T23:01:54.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.179+0000 7f8aa7c64640 1 --2- 172.21.15.84:0/3064779085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8aa0102380 0x7f8aa0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.179+0000 7f8aa7c64640 1 --2- 172.21.15.84:0/3064779085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8aa01019c0 0x7f8aa0101e40 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.179+0000 7f8aa7c64640 1 --2- 172.21.15.84:0/3064779085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8aa01007d0 0x7f8aa0100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.179+0000 7f8aa7c64640 1 -- 172.21.15.84:0/3064779085 >> 172.21.15.84:0/3064779085 conn(0x7f8aa00fbd40 msgr2=0x7f8aa00fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:54.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.179+0000 7f8aa7c64640 1 -- 172.21.15.84:0/3064779085 shutdown_connections 2024-05-22T23:01:54.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.179+0000 7f8aa7c64640 1 -- 172.21.15.84:0/3064779085 wait complete. 2024-05-22T23:01:54.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.179+0000 7f8aa7c64640 1 Processor -- start 2024-05-22T23:01:54.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.179+0000 7f8aa7c64640 1 -- start start 2024-05-22T23:01:54.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.180+0000 7f8aa7c64640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8aa01007d0 0x7f8aa010b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:54.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.180+0000 7f8aa7c64640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8aa01019c0 0x7f8aa0109790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:54.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.180+0000 7f8aa7c64640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8aa0102380 0x7f8aa0109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:54.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.180+0000 7f8aa7c64640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa010d570 con 0x7f8aa01007d0 2024-05-22T23:01:54.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.180+0000 7f8aa7c64640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa010d3f0 con 0x7f8aa0102380 2024-05-22T23:01:54.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.180+0000 7f8aa7c64640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa010d6f0 con 0x7f8aa01019c0 2024-05-22T23:01:54.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.180+0000 7f8aa5a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8aa01007d0 0x7f8aa010b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:54.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.180+0000 7f8aa5a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8aa01007d0 0x7f8aa010b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58930/0 (socket says 172.21.15.84:58930) 2024-05-22T23:01:54.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.180+0000 7f8aa5a00640 1 -- 172.21.15.84:0/1527307894 learned_addr learned my addr 172.21.15.84:0/1527307894 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:54.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.180+0000 7f8aa51ff640 1 --2- 172.21.15.84:0/1527307894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8aa01019c0 0x7f8aa0109790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:54.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.180+0000 7f8aa6201640 1 --2- 172.21.15.84:0/1527307894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8aa0102380 0x7f8aa0109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:54.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.180+0000 7f8aa5a00640 1 -- 172.21.15.84:0/1527307894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8aa01019c0 msgr2=0x7f8aa0109790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:54.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.180+0000 7f8aa5a00640 1 --2- 172.21.15.84:0/1527307894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8aa01019c0 0x7f8aa0109790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.180+0000 7f8aa5a00640 1 -- 172.21.15.84:0/1527307894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8aa0102380 msgr2=0x7f8aa0109cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:54.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.180+0000 7f8aa5a00640 1 --2- 172.21.15.84:0/1527307894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8aa0102380 0x7f8aa0109cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.180+0000 7f8aa5a00640 1 -- 172.21.15.84:0/1527307894 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa010a560 con 0x7f8aa01007d0 2024-05-22T23:01:54.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.180+0000 7f8aa5a00640 1 --2- 172.21.15.84:0/1527307894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8aa01007d0 0x7f8aa010b150 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00ebc0 tx=0x7f8a9c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:54.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.180+0000 7f8a96ffd640 1 -- 172.21.15.84:0/1527307894 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c003b70 con 0x7f8aa01007d0 2024-05-22T23:01:54.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.180+0000 7f8aa7c64640 1 -- 172.21.15.84:0/1527307894 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa01a1c40 con 0x7f8aa01007d0 2024-05-22T23:01:54.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.180+0000 7f8a96ffd640 1 -- 172.21.15.84:0/1527307894 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a9c003d10 con 0x7f8aa01007d0 2024-05-22T23:01:54.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.180+0000 7f8a96ffd640 1 -- 172.21.15.84:0/1527307894 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c019980 con 0x7f8aa01007d0 2024-05-22T23:01:54.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.180+0000 7f8aa7c64640 1 -- 172.21.15.84:0/1527307894 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa01a2160 con 0x7f8aa01007d0 2024-05-22T23:01:54.186 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.182+0000 7f8aa7c64640 1 -- 172.21.15.84:0/1527307894 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aa004f630 con 0x7f8aa01007d0 2024-05-22T23:01:54.186 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.183+0000 7f8a96ffd640 1 -- 172.21.15.84:0/1527307894 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f8a9c019b20 con 0x7f8aa01007d0 2024-05-22T23:01:54.186 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.183+0000 7f8a96ffd640 1 --2- 172.21.15.84:0/1527307894 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f8a8c0a2da0 0x7f8a8c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:54.186 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.183+0000 7f8a96ffd640 1 -- 172.21.15.84:0/1527307894 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f8a9c0cfc90 con 0x7f8aa01007d0 2024-05-22T23:01:54.186 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.183+0000 7f8aa51ff640 1 --2- 172.21.15.84:0/1527307894 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f8a8c0a2da0 0x7f8a8c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:54.186 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.184+0000 7f8aa51ff640 1 --2- 172.21.15.84:0/1527307894 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f8a8c0a2da0 0x7f8a8c0a5260 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8a900314e0 tx=0x7f8a90000bc0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:54.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.188+0000 7f8a96ffd640 1 -- 172.21.15.84:0/1527307894 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8a9c09c8b0 con 0x7f8aa01007d0 2024-05-22T23:01:54.201 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/870372379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:01:54.201 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:54.201 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:54.201 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.201 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:54.201 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.201 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:54.201 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.201 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.202 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.202 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.202 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: Adjusting osd_memory_target on smithi204 to 176.2M 2024-05-22T23:01:54.202 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.202 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: Unable to set osd_memory_target on smithi204 to 184759500: error parsing value: Value '184759500' is below minimum 939524096 2024-05-22T23:01:54.202 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.202 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-05-22T23:01:54.202 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: Unable to set osd_memory_target on smithi096 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-05-22T23:01:54.202 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3264661064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:01:54.202 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:54.202 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:54.202 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.202 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.203 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.203 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:01:54.203 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: Adjusting osd_memory_target on smithi084 to 176.2M 2024-05-22T23:01:54.203 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: Unable to set osd_memory_target on smithi084 to 184766668: error parsing value: Value '184766668' is below minimum 939524096 2024-05-22T23:01:54.203 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:01:54.203 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:01:54.203 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: Updating smithi084:/etc/ceph/ceph.conf 2024-05-22T23:01:54.203 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: Updating smithi096:/etc/ceph/ceph.conf 2024-05-22T23:01:54.203 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: Updating smithi204:/etc/ceph/ceph.conf 2024-05-22T23:01:54.203 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:53 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2548137022' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:01:54.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.376+0000 7f8aa7c64640 1 -- 172.21.15.84:0/1527307894 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f8aa0107a90 con 0x7f8aa01007d0 2024-05-22T23:01:54.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.380+0000 7f8a96ffd640 1 -- 172.21.15.84:0/1527307894 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v33)=0 v33) ==== 138+0+0 (secure 0 0 0) 0x7f8aa0107a90 con 0x7f8aa01007d0 2024-05-22T23:01:54.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.383+0000 7f8aa7c64640 1 -- 172.21.15.84:0/1527307894 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f8a8c0a2da0 msgr2=0x7f8a8c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:54.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.383+0000 7f8aa7c64640 1 --2- 172.21.15.84:0/1527307894 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f8a8c0a2da0 0x7f8a8c0a5260 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8a900314e0 tx=0x7f8a90000bc0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.383+0000 7f8aa7c64640 1 -- 172.21.15.84:0/1527307894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8aa01007d0 msgr2=0x7f8aa010b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:54.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.383+0000 7f8aa7c64640 1 --2- 172.21.15.84:0/1527307894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8aa01007d0 0x7f8aa010b150 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00ebc0 tx=0x7f8a9c00c6a0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.383+0000 7f8aa7c64640 1 -- 172.21.15.84:0/1527307894 shutdown_connections 2024-05-22T23:01:54.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.383+0000 7f8aa7c64640 1 --2- 172.21.15.84:0/1527307894 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f8a8c0a2da0 0x7f8a8c0a5260 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.383+0000 7f8aa7c64640 1 --2- 172.21.15.84:0/1527307894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8aa0102380 0x7f8aa0109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.383+0000 7f8aa7c64640 1 --2- 172.21.15.84:0/1527307894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8aa01019c0 0x7f8aa0109790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.383+0000 7f8aa7c64640 1 --2- 172.21.15.84:0/1527307894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8aa01007d0 0x7f8aa010b150 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.383+0000 7f8aa7c64640 1 -- 172.21.15.84:0/1527307894 >> 172.21.15.84:0/1527307894 conn(0x7f8aa00fbd40 msgr2=0x7f8aa0104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:54.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.383+0000 7f8aa7c64640 1 -- 172.21.15.84:0/1527307894 shutdown_connections 2024-05-22T23:01:54.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.383+0000 7f8aa7c64640 1 -- 172.21.15.84:0/1527307894 wait complete. 2024-05-22T23:01:54.398 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-22T23:01:54.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.512+0000 7ff121f4f640 1 -- 172.21.15.84:0/466353984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11c107860 msgr2=0x7ff11c107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:54.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.512+0000 7ff121f4f640 1 --2- 172.21.15.84:0/466353984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11c107860 0x7ff11c107c40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009930 tx=0x7ff10c02a420 comp rx=0 tx=0).stop 2024-05-22T23:01:54.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.512+0000 7ff121f4f640 1 -- 172.21.15.84:0/466353984 shutdown_connections 2024-05-22T23:01:54.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.512+0000 7ff121f4f640 1 --2- 172.21.15.84:0/466353984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff11c102210 0x7ff11c10e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.512+0000 7ff121f4f640 1 --2- 172.21.15.84:0/466353984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff11c101850 0x7ff11c101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.512+0000 7ff121f4f640 1 --2- 172.21.15.84:0/466353984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11c107860 0x7ff11c107c40 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.512+0000 7ff121f4f640 1 -- 172.21.15.84:0/466353984 >> 172.21.15.84:0/466353984 conn(0x7ff11c0fd320 msgr2=0x7ff11c0ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:54.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.512+0000 7ff121f4f640 1 -- 172.21.15.84:0/466353984 shutdown_connections 2024-05-22T23:01:54.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.512+0000 7ff121f4f640 1 -- 172.21.15.84:0/466353984 wait complete. 2024-05-22T23:01:54.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.513+0000 7ff121f4f640 1 Processor -- start 2024-05-22T23:01:54.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.513+0000 7ff121f4f640 1 -- start start 2024-05-22T23:01:54.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.513+0000 7ff121f4f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff11c101850 0x7ff11c073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:54.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.513+0000 7ff121f4f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff11c102210 0x7ff11c0740b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:54.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.513+0000 7ff121f4f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11c107860 0x7ff11c0745f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:54.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.513+0000 7ff121f4f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff11c111cc0 con 0x7ff11c107860 2024-05-22T23:01:54.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.513+0000 7ff121f4f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff11c111b40 con 0x7ff11c102210 2024-05-22T23:01:54.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.513+0000 7ff121f4f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff11c111e40 con 0x7ff11c101850 2024-05-22T23:01:54.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.513+0000 7ff11bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11c107860 0x7ff11c0745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:54.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.513+0000 7ff11b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff11c101850 0x7ff11c073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:54.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.513+0000 7ff11bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11c107860 0x7ff11c0745f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58950/0 (socket says 172.21.15.84:58950) 2024-05-22T23:01:54.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.513+0000 7ff11affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff11c102210 0x7ff11c0740b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:54.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.513+0000 7ff11bfff640 1 -- 172.21.15.84:0/3921405499 learned_addr learned my addr 172.21.15.84:0/3921405499 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:54.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.514+0000 7ff11bfff640 1 -- 172.21.15.84:0/3921405499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff11c101850 msgr2=0x7ff11c073b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:54.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.514+0000 7ff11bfff640 1 --2- 172.21.15.84:0/3921405499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff11c101850 0x7ff11c073b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.514+0000 7ff11bfff640 1 -- 172.21.15.84:0/3921405499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff11c102210 msgr2=0x7ff11c0740b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:54.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.514+0000 7ff11bfff640 1 --2- 172.21.15.84:0/3921405499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff11c102210 0x7ff11c0740b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.514+0000 7ff11bfff640 1 -- 172.21.15.84:0/3921405499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11c1a65a0 con 0x7ff11c107860 2024-05-22T23:01:54.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.514+0000 7ff11affd640 1 --2- 172.21.15.84:0/3921405499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff11c102210 0x7ff11c0740b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:01:54.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.514+0000 7ff11bfff640 1 --2- 172.21.15.84:0/3921405499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11c107860 0x7ff11c0745f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff11400eb10 tx=0x7ff11400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:54.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.514+0000 7ff118ff9640 1 -- 172.21.15.84:0/3921405499 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114004b20 con 0x7ff11c107860 2024-05-22T23:01:54.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.514+0000 7ff118ff9640 1 -- 172.21.15.84:0/3921405499 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff114004cc0 con 0x7ff11c107860 2024-05-22T23:01:54.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.514+0000 7ff121f4f640 1 -- 172.21.15.84:0/3921405499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff11c1a6890 con 0x7ff11c107860 2024-05-22T23:01:54.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.514+0000 7ff118ff9640 1 -- 172.21.15.84:0/3921405499 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114010760 con 0x7ff11c107860 2024-05-22T23:01:54.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.514+0000 7ff121f4f640 1 -- 172.21.15.84:0/3921405499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff11c1a6de0 con 0x7ff11c107860 2024-05-22T23:01:54.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.515+0000 7ff121f4f640 1 -- 172.21.15.84:0/3921405499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff11c04f630 con 0x7ff11c107860 2024-05-22T23:01:54.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.516+0000 7ff118ff9640 1 -- 172.21.15.84:0/3921405499 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7ff114005020 con 0x7ff11c107860 2024-05-22T23:01:54.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.517+0000 7ff118ff9640 1 --2- 172.21.15.84:0/3921405499 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff1080a2e40 0x7ff1080a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:54.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.517+0000 7ff11b7fe640 1 --2- 172.21.15.84:0/3921405499 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff1080a2e40 0x7ff1080a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:54.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.517+0000 7ff118ff9640 1 -- 172.21.15.84:0/3921405499 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7ff1140cfc30 con 0x7ff11c107860 2024-05-22T23:01:54.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.518+0000 7ff11b7fe640 1 --2- 172.21.15.84:0/3921405499 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff1080a2e40 0x7ff1080a5300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff10c006010 tx=0x7ff10c000bc0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:54.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.522+0000 7ff118ff9640 1 -- 172.21.15.84:0/3921405499 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff11409c6e0 con 0x7ff11c107860 2024-05-22T23:01:54.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.719+0000 7ff121f4f640 1 -- 172.21.15.84:0/3921405499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7ff11c1a7180 con 0x7ff11c107860 2024-05-22T23:01:54.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.723+0000 7ff118ff9640 1 -- 172.21.15.84:0/3921405499 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v34)=0 v34) ==== 133+0+0 (secure 0 0 0) 0x7ff11409bfe0 con 0x7ff11c107860 2024-05-22T23:01:54.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.726+0000 7ff121f4f640 1 -- 172.21.15.84:0/3921405499 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff1080a2e40 msgr2=0x7ff1080a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:54.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.726+0000 7ff121f4f640 1 --2- 172.21.15.84:0/3921405499 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff1080a2e40 0x7ff1080a5300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff10c006010 tx=0x7ff10c000bc0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.726+0000 7ff121f4f640 1 -- 172.21.15.84:0/3921405499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11c107860 msgr2=0x7ff11c0745f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:54.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.726+0000 7ff121f4f640 1 --2- 172.21.15.84:0/3921405499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11c107860 0x7ff11c0745f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff11400eb10 tx=0x7ff11400c2d0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.726+0000 7ff121f4f640 1 -- 172.21.15.84:0/3921405499 shutdown_connections 2024-05-22T23:01:54.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.726+0000 7ff121f4f640 1 --2- 172.21.15.84:0/3921405499 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff1080a2e40 0x7ff1080a5300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.726+0000 7ff121f4f640 1 --2- 172.21.15.84:0/3921405499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11c107860 0x7ff11c0745f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.726+0000 7ff121f4f640 1 --2- 172.21.15.84:0/3921405499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff11c102210 0x7ff11c0740b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.726+0000 7ff121f4f640 1 --2- 172.21.15.84:0/3921405499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff11c101850 0x7ff11c073b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.726+0000 7ff121f4f640 1 -- 172.21.15.84:0/3921405499 >> 172.21.15.84:0/3921405499 conn(0x7ff11c0fd320 msgr2=0x7ff11c0fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:54.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.726+0000 7ff121f4f640 1 -- 172.21.15.84:0/3921405499 shutdown_connections 2024-05-22T23:01:54.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.726+0000 7ff121f4f640 1 -- 172.21.15.84:0/3921405499 wait complete. 2024-05-22T23:01:54.739 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-22T23:01:54.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.842+0000 7f73b922c640 1 -- 172.21.15.84:0/2911821423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73b4103420 msgr2=0x7f73b410b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:54.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.842+0000 7f73b922c640 1 --2- 172.21.15.84:0/2911821423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73b4103420 0x7f73b410b900 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f739c009900 tx=0x7f739c02a510 comp rx=0 tx=0).stop 2024-05-22T23:01:54.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.842+0000 7f73b922c640 1 -- 172.21.15.84:0/2911821423 shutdown_connections 2024-05-22T23:01:54.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.842+0000 7f73b922c640 1 --2- 172.21.15.84:0/2911821423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73b410be40 0x7f73b410e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.842+0000 7f73b922c640 1 --2- 172.21.15.84:0/2911821423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73b4103420 0x7f73b410b900 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.842+0000 7f73b922c640 1 --2- 172.21.15.84:0/2911821423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73b4102b00 0x7f73b4102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.843+0000 7f73b922c640 1 -- 172.21.15.84:0/2911821423 >> 172.21.15.84:0/2911821423 conn(0x7f73b40778f0 msgr2=0x7f73b4077d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:54.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.843+0000 7f73b922c640 1 -- 172.21.15.84:0/2911821423 shutdown_connections 2024-05-22T23:01:54.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.843+0000 7f73b922c640 1 -- 172.21.15.84:0/2911821423 wait complete. 2024-05-22T23:01:54.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.843+0000 7f73b922c640 1 Processor -- start 2024-05-22T23:01:54.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.843+0000 7f73b922c640 1 -- start start 2024-05-22T23:01:54.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.844+0000 7f73b922c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73b4102b00 0x7f73b4100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:54.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.844+0000 7f73b922c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73b4103420 0x7f73b4100c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:54.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.844+0000 7f73b922c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73b410be40 0x7f73b41011d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:54.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.844+0000 7f73b922c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f73b4111c40 con 0x7f73b4102b00 2024-05-22T23:01:54.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.844+0000 7f73b922c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f73b4111ac0 con 0x7f73b410be40 2024-05-22T23:01:54.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.844+0000 7f73b922c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f73b4111dc0 con 0x7f73b4103420 2024-05-22T23:01:54.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.844+0000 7f73b2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73b4102b00 0x7f73b4100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:54.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.844+0000 7f73b2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73b4102b00 0x7f73b4100750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58968/0 (socket says 172.21.15.84:58968) 2024-05-22T23:01:54.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.844+0000 7f73b2d9d640 1 -- 172.21.15.84:0/453286297 learned_addr learned my addr 172.21.15.84:0/453286297 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:54.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.844+0000 7f73abfff640 1 --2- 172.21.15.84:0/453286297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73b4103420 0x7f73b4100c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:54.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.844+0000 7f73b359e640 1 --2- 172.21.15.84:0/453286297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73b410be40 0x7f73b41011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:54.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.844+0000 7f73b2d9d640 1 -- 172.21.15.84:0/453286297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73b4103420 msgr2=0x7f73b4100c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:54.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.844+0000 7f73b2d9d640 1 --2- 172.21.15.84:0/453286297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73b4103420 0x7f73b4100c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.844+0000 7f73b2d9d640 1 -- 172.21.15.84:0/453286297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73b410be40 msgr2=0x7f73b41011d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:54.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.844+0000 7f73b2d9d640 1 --2- 172.21.15.84:0/453286297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73b410be40 0x7f73b41011d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:54.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.844+0000 7f73b2d9d640 1 -- 172.21.15.84:0/453286297 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b41a65e0 con 0x7f73b4102b00 2024-05-22T23:01:54.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.844+0000 7f73b2d9d640 1 --2- 172.21.15.84:0/453286297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73b4102b00 0x7f73b4100750 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f73a400e9f0 tx=0x7f73a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:54.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.844+0000 7f73b0d99640 1 -- 172.21.15.84:0/453286297 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a400cd80 con 0x7f73b4102b00 2024-05-22T23:01:54.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.844+0000 7f73b0d99640 1 -- 172.21.15.84:0/453286297 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a4010910 con 0x7f73b4102b00 2024-05-22T23:01:54.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.844+0000 7f73b922c640 1 -- 172.21.15.84:0/453286297 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b41a68d0 con 0x7f73b4102b00 2024-05-22T23:01:54.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.844+0000 7f73b0d99640 1 -- 172.21.15.84:0/453286297 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4018990 con 0x7f73b4102b00 2024-05-22T23:01:54.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.844+0000 7f73b922c640 1 -- 172.21.15.84:0/453286297 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b41a6df0 con 0x7f73b4102b00 2024-05-22T23:01:54.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.845+0000 7f73b922c640 1 -- 172.21.15.84:0/453286297 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73b404f630 con 0x7f73b4102b00 2024-05-22T23:01:54.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.847+0000 7f73b0d99640 1 -- 172.21.15.84:0/453286297 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f73a4010ac0 con 0x7f73b4102b00 2024-05-22T23:01:54.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.847+0000 7f73b0d99640 1 --2- 172.21.15.84:0/453286297 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f73a00a2e70 0x7f73a00a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:54.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.847+0000 7f73abfff640 1 --2- 172.21.15.84:0/453286297 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f73a00a2e70 0x7f73a00a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:54.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.847+0000 7f73b0d99640 1 -- 172.21.15.84:0/453286297 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f73a40cfcc0 con 0x7f73b4102b00 2024-05-22T23:01:54.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.848+0000 7f73abfff640 1 --2- 172.21.15.84:0/453286297 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f73a00a2e70 0x7f73a00a5330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f739c009230 tx=0x7f739c0058d0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:54.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:54.851+0000 7f73b0d99640 1 -- 172.21.15.84:0/453286297 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f73a409c120 con 0x7f73b4102b00 2024-05-22T23:01:55.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.057+0000 7f73b922c640 1 -- 172.21.15.84:0/453286297 --> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f73b4107030 con 0x7f73a00a2e70 2024-05-22T23:01:55.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.114+0000 7f73b0d99640 1 -- 172.21.15.84:0/453286297 <== mgr.24851 v2:172.21.15.84:6800/3358259585 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f739c031630 con 0x7f73a00a2e70 2024-05-22T23:01:55.116 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path / 2024-05-22T23:01:55.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.116+0000 7f73b922c640 1 -- 172.21.15.84:0/453286297 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f73a00a2e70 msgr2=0x7f73a00a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:55.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.116+0000 7f73b922c640 1 --2- 172.21.15.84:0/453286297 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f73a00a2e70 0x7f73a00a5330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f739c009230 tx=0x7f739c0058d0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.116+0000 7f73b922c640 1 -- 172.21.15.84:0/453286297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73b4102b00 msgr2=0x7f73b4100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:55.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.116+0000 7f73b922c640 1 --2- 172.21.15.84:0/453286297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73b4102b00 0x7f73b4100750 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f73a400e9f0 tx=0x7f73a400eeb0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.116+0000 7f73b922c640 1 -- 172.21.15.84:0/453286297 shutdown_connections 2024-05-22T23:01:55.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.116+0000 7f73b922c640 1 --2- 172.21.15.84:0/453286297 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f73a00a2e70 0x7f73a00a5330 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.116+0000 7f73b922c640 1 --2- 172.21.15.84:0/453286297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73b410be40 0x7f73b41011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.116+0000 7f73b922c640 1 --2- 172.21.15.84:0/453286297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73b4103420 0x7f73b4100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.116+0000 7f73b922c640 1 --2- 172.21.15.84:0/453286297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73b4102b00 0x7f73b4100750 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.116+0000 7f73b922c640 1 -- 172.21.15.84:0/453286297 >> 172.21.15.84:0/453286297 conn(0x7f73b40778f0 msgr2=0x7f73b40fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:55.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.116+0000 7f73b922c640 1 -- 172.21.15.84:0/453286297 shutdown_connections 2024-05-22T23:01:55.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.116+0000 7f73b922c640 1 -- 172.21.15.84:0/453286297 wait complete. 2024-05-22T23:01:55.125 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: Updating smithi096:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-22T23:01:55.125 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: Updating smithi084:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-22T23:01:55.125 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: Updating smithi204:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-22T23:01:55.125 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: scrub queued for path: / 2024-05-22T23:01:55.126 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:01:55.126 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: scrub summary: active paths [/] 2024-05-22T23:01:55.126 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:01:55.126 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T23:01:55.126 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-05-22T23:01:55.126 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/691040664' entity='client.admin' 2024-05-22T23:01:55.126 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-05-22T23:01:55.126 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-05-22T23:01:55.126 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: Updating smithi096:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-22T23:01:55.126 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: Updating smithi084:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-22T23:01:55.126 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: Updating smithi204:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-22T23:01:55.126 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1527307894' entity='client.admin' 2024-05-22T23:01:55.126 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-05-22T23:01:55.127 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.127 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.127 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.127 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.127 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.127 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.127 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.127 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3921405499' entity='client.admin' 2024-05-22T23:01:55.127 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:01:55.127 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:01:55.127 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:01:55.127 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.127 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:01:55.127 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:01:55.128 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:01:55.128 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:54 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.129 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-22T23:01:55.161 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: Updating smithi096:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-22T23:01:55.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: Updating smithi084:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-22T23:01:55.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: Updating smithi204:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-22T23:01:55.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: scrub queued for path: / 2024-05-22T23:01:55.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:01:55.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: scrub summary: active paths [/] 2024-05-22T23:01:55.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:01:55.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T23:01:55.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-05-22T23:01:55.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/691040664' entity='client.admin' 2024-05-22T23:01:55.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-05-22T23:01:55.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-05-22T23:01:55.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: Updating smithi096:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-22T23:01:55.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: Updating smithi084:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-22T23:01:55.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: Updating smithi204:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-22T23:01:55.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1527307894' entity='client.admin' 2024-05-22T23:01:55.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-05-22T23:01:55.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3921405499' entity='client.admin' 2024-05-22T23:01:55.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:01:55.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:01:55.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:01:55.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:01:55.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:01:55.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:01:55.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:54 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.236 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.231+0000 7f12ba163640 1 -- 172.21.15.84:0/4045406565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b4101880 msgr2=0x7f12b4101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:55.236 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.231+0000 7f12ba163640 1 --2- 172.21.15.84:0/4045406565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b4101880 0x7f12b4101d00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009930 tx=0x7f12a802a420 comp rx=0 tx=0).stop 2024-05-22T23:01:55.236 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.231+0000 7f12ba163640 1 -- 172.21.15.84:0/4045406565 shutdown_connections 2024-05-22T23:01:55.236 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.231+0000 7f12ba163640 1 --2- 172.21.15.84:0/4045406565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12b4102240 0x7f12b410e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.236 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.231+0000 7f12ba163640 1 --2- 172.21.15.84:0/4045406565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b4101880 0x7f12b4101d00 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.236 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.231+0000 7f12ba163640 1 --2- 172.21.15.84:0/4045406565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12b4107890 0x7f12b4107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.236 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.231+0000 7f12ba163640 1 -- 172.21.15.84:0/4045406565 >> 172.21.15.84:0/4045406565 conn(0x7f12b4078d00 msgr2=0x7f12b4079110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:55.236 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.231+0000 7f12ba163640 1 -- 172.21.15.84:0/4045406565 shutdown_connections 2024-05-22T23:01:55.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.231+0000 7f12ba163640 1 -- 172.21.15.84:0/4045406565 wait complete. 2024-05-22T23:01:55.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.232+0000 7f12ba163640 1 Processor -- start 2024-05-22T23:01:55.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.232+0000 7f12ba163640 1 -- start start 2024-05-22T23:01:55.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.232+0000 7f12ba163640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12b4101880 0x7f12b4198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:55.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.232+0000 7f12ba163640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b4102240 0x7f12b4198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:55.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.232+0000 7f12ba163640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12b4107890 0x7f12b419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:55.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.232+0000 7f12ba163640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f12b4111ca0 con 0x7f12b4102240 2024-05-22T23:01:55.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.232+0000 7f12ba163640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f12b4111b20 con 0x7f12b4107890 2024-05-22T23:01:55.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.232+0000 7f12ba163640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f12b4111e20 con 0x7f12b4101880 2024-05-22T23:01:55.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.232+0000 7f12b2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b4102240 0x7f12b4198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:55.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.232+0000 7f12b2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b4102240 0x7f12b4198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58988/0 (socket says 172.21.15.84:58988) 2024-05-22T23:01:55.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.232+0000 7f12b2ffd640 1 -- 172.21.15.84:0/1337125960 learned_addr learned my addr 172.21.15.84:0/1337125960 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:55.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.232+0000 7f12b37fe640 1 --2- 172.21.15.84:0/1337125960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12b4101880 0x7f12b4198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:55.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.232+0000 7f12b3fff640 1 --2- 172.21.15.84:0/1337125960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12b4107890 0x7f12b419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:55.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.233+0000 7f12b2ffd640 1 -- 172.21.15.84:0/1337125960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12b4101880 msgr2=0x7f12b4198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:55.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.233+0000 7f12b2ffd640 1 --2- 172.21.15.84:0/1337125960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12b4101880 0x7f12b4198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.233+0000 7f12b2ffd640 1 -- 172.21.15.84:0/1337125960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12b4107890 msgr2=0x7f12b419c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:55.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.233+0000 7f12b2ffd640 1 --2- 172.21.15.84:0/1337125960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12b4107890 0x7f12b419c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.233+0000 7f12b2ffd640 1 -- 172.21.15.84:0/1337125960 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b419d0e0 con 0x7f12b4102240 2024-05-22T23:01:55.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.233+0000 7f12b3fff640 1 --2- 172.21.15.84:0/1337125960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12b4107890 0x7f12b419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:01:55.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.233+0000 7f12b2ffd640 1 --2- 172.21.15.84:0/1337125960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b4102240 0x7f12b4198680 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009230 tx=0x7f12a8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:55.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.233+0000 7f12b0ff9640 1 -- 172.21.15.84:0/1337125960 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a803f030 con 0x7f12b4102240 2024-05-22T23:01:55.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.233+0000 7f12ba163640 1 -- 172.21.15.84:0/1337125960 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b419d370 con 0x7f12b4102240 2024-05-22T23:01:55.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.233+0000 7f12b0ff9640 1 -- 172.21.15.84:0/1337125960 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12a8004440 con 0x7f12b4102240 2024-05-22T23:01:55.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.233+0000 7f12b0ff9640 1 -- 172.21.15.84:0/1337125960 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8035070 con 0x7f12b4102240 2024-05-22T23:01:55.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.233+0000 7f12ba163640 1 -- 172.21.15.84:0/1337125960 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b41a4c60 con 0x7f12b4102240 2024-05-22T23:01:55.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.234+0000 7f12ba163640 1 -- 172.21.15.84:0/1337125960 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12b404f630 con 0x7f12b4102240 2024-05-22T23:01:55.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.235+0000 7f12b0ff9640 1 -- 172.21.15.84:0/1337125960 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f12a8042030 con 0x7f12b4102240 2024-05-22T23:01:55.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.236+0000 7f12b0ff9640 1 --2- 172.21.15.84:0/1337125960 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f129c0a2ec0 0x7f129c0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:55.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.236+0000 7f12b37fe640 1 --2- 172.21.15.84:0/1337125960 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f129c0a2ec0 0x7f129c0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:55.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.236+0000 7f12b0ff9640 1 -- 172.21.15.84:0/1337125960 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f12a80ef930 con 0x7f12b4102240 2024-05-22T23:01:55.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.236+0000 7f12b37fe640 1 --2- 172.21.15.84:0/1337125960 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f129c0a2ec0 0x7f129c0a5380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009b50 tx=0x7f12a4005f90 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:55.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.240+0000 7f12b0ff9640 1 -- 172.21.15.84:0/1337125960 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f12a80bbd30 con 0x7f12b4102240 2024-05-22T23:01:55.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: Updating smithi096:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-22T23:01:55.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: Updating smithi084:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-22T23:01:55.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: Updating smithi204:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-22T23:01:55.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: scrub queued for path: / 2024-05-22T23:01:55.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:01:55.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: scrub summary: active paths [/] 2024-05-22T23:01:55.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:01:55.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T23:01:55.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-05-22T23:01:55.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/691040664' entity='client.admin' 2024-05-22T23:01:55.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-05-22T23:01:55.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-05-22T23:01:55.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: Updating smithi096:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-22T23:01:55.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: Updating smithi084:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-22T23:01:55.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: Updating smithi204:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-22T23:01:55.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1527307894' entity='client.admin' 2024-05-22T23:01:55.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-05-22T23:01:55.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.302 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.302 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.302 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.302 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.302 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.302 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.302 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3921405499' entity='client.admin' 2024-05-22T23:01:55.302 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:01:55.302 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:01:55.302 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:01:55.302 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.302 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:01:55.302 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:01:55.302 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:01:55.303 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:54 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:01:55.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.437+0000 7f12ba163640 1 -- 172.21.15.84:0/1337125960 --> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f12b4061cb0 con 0x7f129c0a2ec0 2024-05-22T23:01:55.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.492+0000 7f12b0ff9640 1 -- 172.21.15.84:0/1337125960 <== mgr.24851 v2:172.21.15.84:6800/3358259585 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f12b4061cb0 con 0x7f129c0a2ec0 2024-05-22T23:01:55.494 INFO:teuthology.orchestra.run.smithi084.stdout:Retention added to path / 2024-05-22T23:01:55.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.494+0000 7f12ba163640 1 -- 172.21.15.84:0/1337125960 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f129c0a2ec0 msgr2=0x7f129c0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:55.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.494+0000 7f12ba163640 1 --2- 172.21.15.84:0/1337125960 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f129c0a2ec0 0x7f129c0a5380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009b50 tx=0x7f12a4005f90 comp rx=0 tx=0).stop 2024-05-22T23:01:55.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.494+0000 7f12ba163640 1 -- 172.21.15.84:0/1337125960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b4102240 msgr2=0x7f12b4198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:55.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.494+0000 7f12ba163640 1 --2- 172.21.15.84:0/1337125960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b4102240 0x7f12b4198680 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009230 tx=0x7f12a8004040 comp rx=0 tx=0).stop 2024-05-22T23:01:55.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.494+0000 7f12ba163640 1 -- 172.21.15.84:0/1337125960 shutdown_connections 2024-05-22T23:01:55.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.494+0000 7f12ba163640 1 --2- 172.21.15.84:0/1337125960 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f129c0a2ec0 0x7f129c0a5380 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.494+0000 7f12ba163640 1 --2- 172.21.15.84:0/1337125960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12b4107890 0x7f12b419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.494+0000 7f12ba163640 1 --2- 172.21.15.84:0/1337125960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b4102240 0x7f12b4198680 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.494+0000 7f12ba163640 1 --2- 172.21.15.84:0/1337125960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12b4101880 0x7f12b4198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.494+0000 7f12ba163640 1 -- 172.21.15.84:0/1337125960 >> 172.21.15.84:0/1337125960 conn(0x7f12b4078d00 msgr2=0x7f12b40ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:55.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.494+0000 7f12ba163640 1 -- 172.21.15.84:0/1337125960 shutdown_connections 2024-05-22T23:01:55.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.494+0000 7f12ba163640 1 -- 172.21.15.84:0/1337125960 wait complete. 2024-05-22T23:01:55.507 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-22T23:01:55.610 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.606+0000 7f29ee072640 1 -- 172.21.15.84:0/734389442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29e8100750 msgr2=0x7f29e8100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:55.610 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.606+0000 7f29ee072640 1 --2- 172.21.15.84:0/734389442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29e8100750 0x7f29e8100bd0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009930 tx=0x7f29dc02a420 comp rx=0 tx=0).stop 2024-05-22T23:01:55.610 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.606+0000 7f29ee072640 1 -- 172.21.15.84:0/734389442 shutdown_connections 2024-05-22T23:01:55.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.606+0000 7f29ee072640 1 --2- 172.21.15.84:0/734389442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29e8101110 0x7f29e810d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.606+0000 7f29ee072640 1 --2- 172.21.15.84:0/734389442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29e8100750 0x7f29e8100bd0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.606+0000 7f29ee072640 1 --2- 172.21.15.84:0/734389442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29e8106760 0x7f29e8106b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.606+0000 7f29ee072640 1 -- 172.21.15.84:0/734389442 >> 172.21.15.84:0/734389442 conn(0x7f29e8076ab0 msgr2=0x7f29e8076ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:55.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.606+0000 7f29ee072640 1 -- 172.21.15.84:0/734389442 shutdown_connections 2024-05-22T23:01:55.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.606+0000 7f29ee072640 1 -- 172.21.15.84:0/734389442 wait complete. 2024-05-22T23:01:55.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.607+0000 7f29ee072640 1 Processor -- start 2024-05-22T23:01:55.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.607+0000 7f29ee072640 1 -- start start 2024-05-22T23:01:55.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.607+0000 7f29ee072640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29e8100750 0x7f29e8198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:55.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.607+0000 7f29ee072640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29e8101110 0x7f29e8199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:55.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.607+0000 7f29ee072640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29e8106760 0x7f29e81a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:55.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.607+0000 7f29ee072640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f29e8110b70 con 0x7f29e8100750 2024-05-22T23:01:55.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.607+0000 7f29ee072640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f29e81109f0 con 0x7f29e8101110 2024-05-22T23:01:55.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.607+0000 7f29ee072640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f29e8110cf0 con 0x7f29e8106760 2024-05-22T23:01:55.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.607+0000 7f29e77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29e8100750 0x7f29e8198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:55.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.607+0000 7f29e77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29e8100750 0x7f29e8198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59008/0 (socket says 172.21.15.84:59008) 2024-05-22T23:01:55.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.607+0000 7f29e77fe640 1 -- 172.21.15.84:0/297625706 learned_addr learned my addr 172.21.15.84:0/297625706 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:55.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.608+0000 7f29e7fff640 1 --2- 172.21.15.84:0/297625706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29e8106760 0x7f29e81a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:55.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.608+0000 7f29e6ffd640 1 --2- 172.21.15.84:0/297625706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29e8101110 0x7f29e8199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:55.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.608+0000 7f29e77fe640 1 -- 172.21.15.84:0/297625706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29e8106760 msgr2=0x7f29e81a3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:55.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.608+0000 7f29e77fe640 1 --2- 172.21.15.84:0/297625706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29e8106760 0x7f29e81a3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.608+0000 7f29e77fe640 1 -- 172.21.15.84:0/297625706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29e8101110 msgr2=0x7f29e8199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:55.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.608+0000 7f29e77fe640 1 --2- 172.21.15.84:0/297625706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29e8101110 0x7f29e8199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.608+0000 7f29e77fe640 1 -- 172.21.15.84:0/297625706 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e81a44d0 con 0x7f29e8100750 2024-05-22T23:01:55.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.608+0000 7f29e7fff640 1 --2- 172.21.15.84:0/297625706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29e8106760 0x7f29e81a3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:01:55.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.608+0000 7f29e6ffd640 1 --2- 172.21.15.84:0/297625706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29e8101110 0x7f29e8199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:01:55.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.608+0000 7f29e77fe640 1 --2- 172.21.15.84:0/297625706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29e8100750 0x7f29e8198f10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f29d800eb90 tx=0x7f29d800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:55.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.608+0000 7f29e4ff9640 1 -- 172.21.15.84:0/297625706 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8004d20 con 0x7f29e8100750 2024-05-22T23:01:55.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.608+0000 7f29ee072640 1 -- 172.21.15.84:0/297625706 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e81a46a0 con 0x7f29e8100750 2024-05-22T23:01:55.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.608+0000 7f29e4ff9640 1 -- 172.21.15.84:0/297625706 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29d800ee90 con 0x7f29e8100750 2024-05-22T23:01:55.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.608+0000 7f29e4ff9640 1 -- 172.21.15.84:0/297625706 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8010830 con 0x7f29e8100750 2024-05-22T23:01:55.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.608+0000 7f29ee072640 1 -- 172.21.15.84:0/297625706 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e81a4b70 con 0x7f29e8100750 2024-05-22T23:01:55.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.609+0000 7f29ee072640 1 -- 172.21.15.84:0/297625706 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29e804f630 con 0x7f29e8100750 2024-05-22T23:01:55.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.610+0000 7f29e4ff9640 1 -- 172.21.15.84:0/297625706 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f29d8005020 con 0x7f29e8100750 2024-05-22T23:01:55.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.611+0000 7f29e4ff9640 1 --2- 172.21.15.84:0/297625706 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f29d00a2e40 0x7f29d00a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:55.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.611+0000 7f29e6ffd640 1 --2- 172.21.15.84:0/297625706 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f29d00a2e40 0x7f29d00a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:55.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.611+0000 7f29e4ff9640 1 -- 172.21.15.84:0/297625706 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f29d80cfd10 con 0x7f29e8100750 2024-05-22T23:01:55.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.611+0000 7f29e6ffd640 1 --2- 172.21.15.84:0/297625706 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f29d00a2e40 0x7f29d00a5300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f29dc000c00 tx=0x7f29dc005a20 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:55.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.615+0000 7f29e4ff9640 1 -- 172.21.15.84:0/297625706 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f29d809c7c0 con 0x7f29e8100750 2024-05-22T23:01:55.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.812+0000 7f29ee072640 1 -- 172.21.15.84:0/297625706 --> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f29e8108d30 con 0x7f29d00a2e40 2024-05-22T23:01:55.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.819+0000 7f29e4ff9640 1 -- 172.21.15.84:0/297625706 <== mgr.24851 v2:172.21.15.84:6800/3358259585 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f29dc0315b0 con 0x7f29d00a2e40 2024-05-22T23:01:55.821 INFO:teuthology.orchestra.run.smithi084.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-22T00:00:00", "created": "2024-05-22T23:01:55", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-22T23:01:55.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.822+0000 7f29ee072640 1 -- 172.21.15.84:0/297625706 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f29d00a2e40 msgr2=0x7f29d00a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:55.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.822+0000 7f29ee072640 1 --2- 172.21.15.84:0/297625706 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f29d00a2e40 0x7f29d00a5300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f29dc000c00 tx=0x7f29dc005a20 comp rx=0 tx=0).stop 2024-05-22T23:01:55.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.822+0000 7f29ee072640 1 -- 172.21.15.84:0/297625706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29e8100750 msgr2=0x7f29e8198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:55.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.822+0000 7f29ee072640 1 --2- 172.21.15.84:0/297625706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29e8100750 0x7f29e8198f10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f29d800eb90 tx=0x7f29d800c6a0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.822+0000 7f29ee072640 1 -- 172.21.15.84:0/297625706 shutdown_connections 2024-05-22T23:01:55.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.822+0000 7f29ee072640 1 --2- 172.21.15.84:0/297625706 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f29d00a2e40 0x7f29d00a5300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.822+0000 7f29ee072640 1 --2- 172.21.15.84:0/297625706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29e8106760 0x7f29e81a3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.822+0000 7f29ee072640 1 --2- 172.21.15.84:0/297625706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29e8101110 0x7f29e8199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.822+0000 7f29ee072640 1 --2- 172.21.15.84:0/297625706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29e8100750 0x7f29e8198f10 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.822+0000 7f29ee072640 1 -- 172.21.15.84:0/297625706 >> 172.21.15.84:0/297625706 conn(0x7f29e8076ab0 msgr2=0x7f29e80fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:55.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.822+0000 7f29ee072640 1 -- 172.21.15.84:0/297625706 shutdown_connections 2024-05-22T23:01:55.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.822+0000 7f29ee072640 1 -- 172.21.15.84:0/297625706 wait complete. 2024-05-22T23:01:55.834 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-22T23:01:55.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.939+0000 7f21a5942640 1 -- 172.21.15.84:0/3421077039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a0103420 msgr2=0x7f21a010b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:55.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.939+0000 7f21a5942640 1 --2- 172.21.15.84:0/3421077039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a0103420 0x7f21a010b900 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2194009900 tx=0x7f219402a510 comp rx=0 tx=0).stop 2024-05-22T23:01:55.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.939+0000 7f21a5942640 1 -- 172.21.15.84:0/3421077039 shutdown_connections 2024-05-22T23:01:55.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.939+0000 7f21a5942640 1 --2- 172.21.15.84:0/3421077039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21a010be40 0x7f21a010e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.939+0000 7f21a5942640 1 --2- 172.21.15.84:0/3421077039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a0103420 0x7f21a010b900 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.939+0000 7f21a5942640 1 --2- 172.21.15.84:0/3421077039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21a0102b00 0x7f21a0102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.939+0000 7f21a5942640 1 -- 172.21.15.84:0/3421077039 >> 172.21.15.84:0/3421077039 conn(0x7f21a00778f0 msgr2=0x7f21a0077d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:55.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.939+0000 7f21a5942640 1 -- 172.21.15.84:0/3421077039 shutdown_connections 2024-05-22T23:01:55.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.939+0000 7f21a5942640 1 -- 172.21.15.84:0/3421077039 wait complete. 2024-05-22T23:01:55.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.940+0000 7f21a5942640 1 Processor -- start 2024-05-22T23:01:55.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.940+0000 7f21a5942640 1 -- start start 2024-05-22T23:01:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.940+0000 7f21a5942640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21a0102b00 0x7f21a0198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.940+0000 7f21a5942640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21a0103420 0x7f21a0198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.940+0000 7f21a5942640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a010be40 0x7f21a019c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.940+0000 7f21a5942640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f21a0111c40 con 0x7f21a010be40 2024-05-22T23:01:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.940+0000 7f21a5942640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f21a0111ac0 con 0x7f21a0103420 2024-05-22T23:01:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.940+0000 7f21a5942640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f21a0111dc0 con 0x7f21a0102b00 2024-05-22T23:01:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.940+0000 7f219f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a010be40 0x7f21a019c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.941+0000 7f219f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a010be40 0x7f21a019c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59030/0 (socket says 172.21.15.84:59030) 2024-05-22T23:01:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.941+0000 7f219f7fe640 1 -- 172.21.15.84:0/3713322955 learned_addr learned my addr 172.21.15.84:0/3713322955 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:01:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.941+0000 7f219e7fc640 1 --2- 172.21.15.84:0/3713322955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21a0103420 0x7f21a0198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.941+0000 7f219effd640 1 --2- 172.21.15.84:0/3713322955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21a0102b00 0x7f21a0198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.941+0000 7f219f7fe640 1 -- 172.21.15.84:0/3713322955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21a0102b00 msgr2=0x7f21a0198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.941+0000 7f219f7fe640 1 --2- 172.21.15.84:0/3713322955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21a0102b00 0x7f21a0198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.941+0000 7f219f7fe640 1 -- 172.21.15.84:0/3713322955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21a0103420 msgr2=0x7f21a0198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:55.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.941+0000 7f219f7fe640 1 --2- 172.21.15.84:0/3713322955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21a0103420 0x7f21a0198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:55.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.941+0000 7f219f7fe640 1 -- 172.21.15.84:0/3713322955 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a019d0f0 con 0x7f21a010be40 2024-05-22T23:01:55.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.941+0000 7f219f7fe640 1 --2- 172.21.15.84:0/3713322955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a010be40 0x7f21a019c9c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f219800d950 tx=0x7f219800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:55.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.941+0000 7f21a4940640 1 -- 172.21.15.84:0/3713322955 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198017030 con 0x7f21a010be40 2024-05-22T23:01:55.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.941+0000 7f21a5942640 1 -- 172.21.15.84:0/3713322955 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a019d3e0 con 0x7f21a010be40 2024-05-22T23:01:55.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.941+0000 7f21a4940640 1 -- 172.21.15.84:0/3713322955 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2198004020 con 0x7f21a010be40 2024-05-22T23:01:55.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.941+0000 7f21a4940640 1 -- 172.21.15.84:0/3713322955 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198003c00 con 0x7f21a010be40 2024-05-22T23:01:55.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.941+0000 7f21a5942640 1 -- 172.21.15.84:0/3713322955 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a01a4c70 con 0x7f21a010be40 2024-05-22T23:01:55.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.942+0000 7f21a5942640 1 -- 172.21.15.84:0/3713322955 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21a0069720 con 0x7f21a010be40 2024-05-22T23:01:55.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.943+0000 7f21a4940640 1 -- 172.21.15.84:0/3713322955 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f219801a070 con 0x7f21a010be40 2024-05-22T23:01:55.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.944+0000 7f21a4940640 1 --2- 172.21.15.84:0/3713322955 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f21880a2e40 0x7f21880a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:01:55.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.944+0000 7f219effd640 1 --2- 172.21.15.84:0/3713322955 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f21880a2e40 0x7f21880a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:01:55.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.944+0000 7f21a4940640 1 -- 172.21.15.84:0/3713322955 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f21980cf140 con 0x7f21a010be40 2024-05-22T23:01:55.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.944+0000 7f219effd640 1 --2- 172.21.15.84:0/3713322955 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f21880a2e40 0x7f21880a5300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2190005fd0 tx=0x7f2190009870 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:01:55.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:55.949+0000 7f21a4940640 1 -- 172.21.15.84:0/3713322955 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f219809bbf0 con 0x7f21a010be40 2024-05-22T23:01:56.125 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:55 smithi084 ceph-mon[32186]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:56.125 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:55 smithi084 ceph-mon[32186]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:56.151 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:56.149+0000 7f21a5942640 1 -- 172.21.15.84:0/3713322955 --> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f21a0061cb0 con 0x7f21880a2e40 2024-05-22T23:01:56.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:56.157+0000 7f21a4940640 1 -- 172.21.15.84:0/3713322955 <== mgr.24851 v2:172.21.15.84:6800/3358259585 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f21a0061cb0 con 0x7f21880a2e40 2024-05-22T23:01:56.159 INFO:teuthology.orchestra.run.smithi084.stdout:/ 1m 6m3h 2024-05-22T23:01:56.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:56.159+0000 7f21a5942640 1 -- 172.21.15.84:0/3713322955 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f21880a2e40 msgr2=0x7f21880a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:56.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:56.159+0000 7f21a5942640 1 --2- 172.21.15.84:0/3713322955 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f21880a2e40 0x7f21880a5300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2190005fd0 tx=0x7f2190009870 comp rx=0 tx=0).stop 2024-05-22T23:01:56.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:56.159+0000 7f21a5942640 1 -- 172.21.15.84:0/3713322955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a010be40 msgr2=0x7f21a019c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:01:56.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:56.159+0000 7f21a5942640 1 --2- 172.21.15.84:0/3713322955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a010be40 0x7f21a019c9c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f219800d950 tx=0x7f219800de10 comp rx=0 tx=0).stop 2024-05-22T23:01:56.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:56.159+0000 7f21a5942640 1 -- 172.21.15.84:0/3713322955 shutdown_connections 2024-05-22T23:01:56.162 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:56.159+0000 7f21a5942640 1 --2- 172.21.15.84:0/3713322955 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f21880a2e40 0x7f21880a5300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:56.162 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:56.159+0000 7f21a5942640 1 --2- 172.21.15.84:0/3713322955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a010be40 0x7f21a019c9c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:56.162 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:56.159+0000 7f21a5942640 1 --2- 172.21.15.84:0/3713322955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21a0103420 0x7f21a0198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:56.162 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:56.159+0000 7f21a5942640 1 --2- 172.21.15.84:0/3713322955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21a0102b00 0x7f21a0198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:01:56.162 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:56.159+0000 7f21a5942640 1 -- 172.21.15.84:0/3713322955 >> 172.21.15.84:0/3713322955 conn(0x7f21a00778f0 msgr2=0x7f21a00fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:01:56.162 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:56.159+0000 7f21a5942640 1 -- 172.21.15.84:0/3713322955 shutdown_connections 2024-05-22T23:01:56.162 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:01:56.159+0000 7f21a5942640 1 -- 172.21.15.84:0/3713322955 wait complete. 2024-05-22T23:01:56.172 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-22T23:01:56.198 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-22T23:01:56.207 INFO:teuthology.run_tasks:Running task workunit... 2024-05-22T23:01:56.250 INFO:tasks.workunit:Pulling workunits from ref 935217f1cb4dc722d5303f5ad364c883e9133376 2024-05-22T23:01:56.250 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-22T23:01:56.250 INFO:tasks.workunit:timeout=3h 2024-05-22T23:01:56.250 INFO:tasks.workunit:cleanup=True 2024-05-22T23:01:56.250 DEBUG:teuthology.orchestra.run.smithi084:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-22T23:01:56.266 INFO:teuthology.orchestra.run.smithi084.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-22T23:01:56.266 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-22T23:01:56.266 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-05-22T23:01:56.266 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-22T23:01:56.267 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-22T23:01:56.267 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-05-22 22:59:56.984046893 +0000 2024-05-22T23:01:56.267 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-05-22 22:59:56.984046893 +0000 2024-05-22T23:01:56.267 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-05-22 23:01:47.122382693 +0000 2024-05-22T23:01:56.267 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-05-22 22:59:56.984046893 +0000 2024-05-22T23:01:56.267 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-22T23:01:56.267 DEBUG:teuthology.orchestra.run.smithi084:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-22T23:01:56.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:55 smithi096 ceph-mon[36392]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:56.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:55 smithi096 ceph-mon[36392]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:56.336 DEBUG:teuthology.orchestra.run.smithi084:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 935217f1cb4dc722d5303f5ad364c883e9133376 2024-05-22T23:01:56.393 INFO:tasks.workunit.client.0.smithi084.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-22T23:01:56.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:55 smithi204 ceph-mon[36211]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:56.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:55 smithi204 ceph-mon[36211]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:57.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:56 smithi096 ceph-mon[36392]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:57.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:56 smithi096 ceph-mon[36392]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T23:01:57.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:56 smithi096 ceph-mon[36392]: scrub summary: idle 2024-05-22T23:01:57.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:56 smithi096 ceph-mon[36392]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:57.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:56 smithi084 ceph-mon[32186]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:57.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:56 smithi084 ceph-mon[32186]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T23:01:57.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:56 smithi084 ceph-mon[32186]: scrub summary: idle 2024-05-22T23:01:57.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:56 smithi084 ceph-mon[32186]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:57.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:56 smithi204 ceph-mon[36211]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:57.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:56 smithi204 ceph-mon[36211]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-22T23:01:57.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:56 smithi204 ceph-mon[36211]: scrub summary: idle 2024-05-22T23:01:57.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:56 smithi204 ceph-mon[36211]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:01:59.177 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:01:58 smithi084 ceph-mon[32186]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 131 KiB/s wr, 71 op/s 2024-05-22T23:01:59.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:01:58 smithi096 ceph-mon[36392]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 131 KiB/s wr, 71 op/s 2024-05-22T23:01:59.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:01:58 smithi204 ceph-mon[36211]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 131 KiB/s wr, 71 op/s 2024-05-22T23:02:00.875 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:02:00 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:02:00.502+0000 7f43b6e9e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-22T23:02:00.875 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:02:00 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:02:00.502+0000 7f43b6e9e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-22T23:02:00.875 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:02:00 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:02:00.502+0000 7f43b6e9e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-22T23:02:00.875 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:02:00 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:02:00.502+0000 7f43b6e9e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-22T23:02:00.876 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:02:00 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:02:00.502+0000 7f43b6e9e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-22T23:02:01.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:00 smithi096 ceph-mon[36392]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 102 KiB/s wr, 55 op/s 2024-05-22T23:02:01.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:00 smithi084 ceph-mon[32186]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 102 KiB/s wr, 55 op/s 2024-05-22T23:02:01.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:00 smithi204 ceph-mon[36211]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 102 KiB/s wr, 55 op/s 2024-05-22T23:02:03.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:02 smithi096 ceph-mon[36392]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 108 KiB/s wr, 64 op/s 2024-05-22T23:02:03.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:02 smithi096 ceph-mon[36392]: mgrmap e24: x(active, since 12s), standbys: z, y 2024-05-22T23:02:03.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:02 smithi084 ceph-mon[32186]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 108 KiB/s wr, 64 op/s 2024-05-22T23:02:03.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:02 smithi084 ceph-mon[32186]: mgrmap e24: x(active, since 12s), standbys: z, y 2024-05-22T23:02:03.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:02 smithi204 ceph-mon[36211]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 108 KiB/s wr, 64 op/s 2024-05-22T23:02:03.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:02 smithi204 ceph-mon[36211]: mgrmap e24: x(active, since 12s), standbys: z, y 2024-05-22T23:02:05.161 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:04 smithi204 ceph-mon[36211]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 99 KiB/s wr, 58 op/s 2024-05-22T23:02:05.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:04 smithi096 ceph-mon[36392]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 99 KiB/s wr, 58 op/s 2024-05-22T23:02:05.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:04 smithi084 ceph-mon[32186]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 99 KiB/s wr, 58 op/s 2024-05-22T23:02:07.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:06 smithi096 ceph-mon[36392]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 99 KiB/s wr, 58 op/s 2024-05-22T23:02:07.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:06 smithi084 ceph-mon[32186]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 99 KiB/s wr, 58 op/s 2024-05-22T23:02:07.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:06 smithi204 ceph-mon[36211]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 99 KiB/s wr, 58 op/s 2024-05-22T23:02:09.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:08 smithi096 ceph-mon[36392]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 99 KiB/s wr, 59 op/s 2024-05-22T23:02:09.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:08 smithi084 ceph-mon[32186]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 99 KiB/s wr, 59 op/s 2024-05-22T23:02:09.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:08 smithi204 ceph-mon[36211]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 99 KiB/s wr, 59 op/s 2024-05-22T23:02:11.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:10 smithi096 ceph-mon[36392]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 18 op/s 2024-05-22T23:02:11.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:10 smithi084 ceph-mon[32186]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 18 op/s 2024-05-22T23:02:11.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:10 smithi204 ceph-mon[36211]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 18 op/s 2024-05-22T23:02:13.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:12 smithi096 ceph-mon[36392]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-05-22T23:02:13.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:12 smithi084 ceph-mon[32186]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-05-22T23:02:13.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:12 smithi204 ceph-mon[36211]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-05-22T23:02:15.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:14 smithi096 ceph-mon[36392]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-22T23:02:15.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:14 smithi084 ceph-mon[32186]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-22T23:02:15.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:14 smithi204 ceph-mon[36211]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-22T23:02:17.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:16 smithi096 ceph-mon[36392]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-22T23:02:17.377 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:16 smithi084 ceph-mon[32186]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-22T23:02:17.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:16 smithi204 ceph-mon[36211]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-22T23:02:19.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:18 smithi096 ceph-mon[36392]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-22T23:02:19.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:18 smithi084 ceph-mon[32186]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-22T23:02:19.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:18 smithi204 ceph-mon[36211]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-22T23:02:21.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:20 smithi096 ceph-mon[36392]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-05-22T23:02:21.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:20 smithi084 ceph-mon[32186]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-05-22T23:02:21.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:20 smithi204 ceph-mon[36211]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-05-22T23:02:23.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:22 smithi096 ceph-mon[36392]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-05-22T23:02:23.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:22 smithi084 ceph-mon[32186]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-05-22T23:02:23.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:22 smithi204 ceph-mon[36211]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-05-22T23:02:24.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:02:24.037 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:02:24.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.152+0000 7fcae2fa9640 1 -- 172.21.15.84:0/332435073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcadc102250 msgr2=0x7fcadc10e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:24.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.152+0000 7fcae2fa9640 1 --2- 172.21.15.84:0/332435073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcadc102250 0x7fcadc10e760 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcad4009cd0 tx=0x7fcad402a8e0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.152+0000 7fcae2fa9640 1 -- 172.21.15.84:0/332435073 shutdown_connections 2024-05-22T23:02:24.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.152+0000 7fcae2fa9640 1 --2- 172.21.15.84:0/332435073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcadc102250 0x7fcadc10e760 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.152+0000 7fcae2fa9640 1 --2- 172.21.15.84:0/332435073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcadc101890 0x7fcadc101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.152+0000 7fcae2fa9640 1 --2- 172.21.15.84:0/332435073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcadc1078a0 0x7fcadc107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.152+0000 7fcae2fa9640 1 -- 172.21.15.84:0/332435073 >> 172.21.15.84:0/332435073 conn(0x7fcadc0fd380 msgr2=0x7fcadc0ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:24.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.152+0000 7fcae2fa9640 1 -- 172.21.15.84:0/332435073 shutdown_connections 2024-05-22T23:02:24.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.152+0000 7fcae2fa9640 1 -- 172.21.15.84:0/332435073 wait complete. 2024-05-22T23:02:24.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.152+0000 7fcae2fa9640 1 Processor -- start 2024-05-22T23:02:24.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.152+0000 7fcae2fa9640 1 -- start start 2024-05-22T23:02:24.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.153+0000 7fcae2fa9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcadc101890 0x7fcadc198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.153+0000 7fcae2fa9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcadc102250 0x7fcadc1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.153+0000 7fcae2fa9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcadc1078a0 0x7fcadc19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.153+0000 7fcae2fa9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc111cb0 con 0x7fcadc101890 2024-05-22T23:02:24.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.153+0000 7fcae2fa9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc111b30 con 0x7fcadc1078a0 2024-05-22T23:02:24.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.153+0000 7fcae2fa9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc111e30 con 0x7fcadc102250 2024-05-22T23:02:24.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.153+0000 7fcae0d45640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcadc101890 0x7fcadc198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.153+0000 7fcae0d45640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcadc101890 0x7fcadc198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35560/0 (socket says 172.21.15.84:35560) 2024-05-22T23:02:24.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.153+0000 7fcae0d45640 1 -- 172.21.15.84:0/1571425774 learned_addr learned my addr 172.21.15.84:0/1571425774 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:24.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.153+0000 7fcadbfff640 1 --2- 172.21.15.84:0/1571425774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcadc102250 0x7fcadc1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.153+0000 7fcae0d45640 1 -- 172.21.15.84:0/1571425774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcadc102250 msgr2=0x7fcadc1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:24.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.153+0000 7fcae0d45640 1 --2- 172.21.15.84:0/1571425774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcadc102250 0x7fcadc1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.153+0000 7fcae0d45640 1 -- 172.21.15.84:0/1571425774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcadc1078a0 msgr2=0x7fcadc19ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:24.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.153+0000 7fcae0d45640 1 --2- 172.21.15.84:0/1571425774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcadc1078a0 0x7fcadc19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.153+0000 7fcae0d45640 1 -- 172.21.15.84:0/1571425774 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcadc19d180 con 0x7fcadc101890 2024-05-22T23:02:24.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.153+0000 7fcae0d45640 1 --2- 172.21.15.84:0/1571425774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcadc101890 0x7fcadc198190 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fcad000d950 tx=0x7fcad000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:24.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.153+0000 7fcad9ffb640 1 -- 172.21.15.84:0/1571425774 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad0017030 con 0x7fcadc101890 2024-05-22T23:02:24.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.153+0000 7fcad9ffb640 1 -- 172.21.15.84:0/1571425774 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcad0004020 con 0x7fcadc101890 2024-05-22T23:02:24.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.153+0000 7fcae2fa9640 1 -- 172.21.15.84:0/1571425774 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcadc19d470 con 0x7fcadc101890 2024-05-22T23:02:24.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.153+0000 7fcad9ffb640 1 -- 172.21.15.84:0/1571425774 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad00047a0 con 0x7fcadc101890 2024-05-22T23:02:24.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.154+0000 7fcae2fa9640 1 -- 172.21.15.84:0/1571425774 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcadc1a4d00 con 0x7fcadc101890 2024-05-22T23:02:24.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.156+0000 7fcad9ffb640 1 -- 172.21.15.84:0/1571425774 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7fcad00041d0 con 0x7fcadc101890 2024-05-22T23:02:24.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.156+0000 7fcae5d96740 1 Processor -- start 2024-05-22T23:02:24.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.156+0000 7fcad9ffb640 1 --2- 172.21.15.84:0/1571425774 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fcacc0a2ea0 0x7fcacc0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.156+0000 7fcae5d96740 1 -- start start 2024-05-22T23:02:24.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.157+0000 7fcae5d96740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f8554a0 0x56156f8537e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.157+0000 7fcae1546640 1 --2- 172.21.15.84:0/1571425774 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fcacc0a2ea0 0x7fcacc0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.157+0000 7fcad9ffb640 1 -- 172.21.15.84:0/1571425774 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fcad00cf090 con 0x7fcadc101890 2024-05-22T23:02:24.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.157+0000 7fcae5d96740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156f853db0 0x56156f854230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.157+0000 7fcae5d96740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56156f85eb30 0x56156f860f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.157+0000 7fcae5d96740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56156f7b7db0 con 0x56156f8554a0 2024-05-22T23:02:24.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.157+0000 7fcae5d96740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56156f756360 con 0x56156f85eb30 2024-05-22T23:02:24.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.157+0000 7fcae5d96740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56156f7b3d70 con 0x56156f853db0 2024-05-22T23:02:24.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.157+0000 7fcae0d45640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f8554a0 0x56156f8537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.157+0000 7fcae0d45640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f8554a0 0x56156f8537e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35576/0 (socket says 172.21.15.84:35576) 2024-05-22T23:02:24.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.157+0000 7fcae0d45640 1 -- 172.21.15.84:0/1956824987 learned_addr learned my addr 172.21.15.84:0/1956824987 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:24.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.157+0000 7fcadbfff640 1 --2- 172.21.15.84:0/1956824987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156f853db0 0x56156f854230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.157+0000 7fcae1546640 1 --2- 172.21.15.84:0/1956824987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56156f85eb30 0x56156f860f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.157+0000 7fcae0d45640 1 -- 172.21.15.84:0/1956824987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156f853db0 msgr2=0x56156f854230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:24.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.157+0000 7fcae0d45640 1 --2- 172.21.15.84:0/1956824987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156f853db0 0x56156f854230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.157+0000 7fcae0d45640 1 -- 172.21.15.84:0/1956824987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56156f85eb30 msgr2=0x56156f860f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:24.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.157+0000 7fcae0d45640 1 --2- 172.21.15.84:0/1956824987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56156f85eb30 0x56156f860f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.157+0000 7fcae0d45640 1 -- 172.21.15.84:0/1956824987 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56156f779a60 con 0x56156f8554a0 2024-05-22T23:02:24.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.157+0000 7fcae1546640 1 --2- 172.21.15.84:0/1571425774 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fcacc0a2ea0 0x7fcacc0a5360 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcad402ae00 tx=0x7fcad4009860 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:24.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.157+0000 7fcae0d45640 1 --2- 172.21.15.84:0/1956824987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f8554a0 0x56156f8537e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fcad009b2f0 tx=0x7fcad009acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615d111cc4aa33da server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:24.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.157+0000 7fcac6ffd640 1 -- 172.21.15.84:0/1956824987 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad009f710 con 0x56156f8554a0 2024-05-22T23:02:24.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.157+0000 7fcac6ffd640 1 -- 172.21.15.84:0/1956824987 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcad00a3fd0 con 0x56156f8554a0 2024-05-22T23:02:24.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.157+0000 7fcac6ffd640 1 -- 172.21.15.84:0/1956824987 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad009a7b0 con 0x56156f8554a0 2024-05-22T23:02:24.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.158+0000 7fcae5d96740 1 -- 172.21.15.84:0/1956824987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f8554a0 msgr2=0x56156f8537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:24.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.158+0000 7fcae5d96740 1 --2- 172.21.15.84:0/1956824987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f8554a0 0x56156f8537e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fcad009b2f0 tx=0x7fcad009acc0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.158+0000 7fcae5d96740 1 -- 172.21.15.84:0/1956824987 shutdown_connections 2024-05-22T23:02:24.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.158+0000 7fcae5d96740 1 --2- 172.21.15.84:0/1956824987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56156f85eb30 0x56156f860f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.158+0000 7fcae5d96740 1 --2- 172.21.15.84:0/1956824987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156f853db0 0x56156f854230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.158+0000 7fcae5d96740 1 --2- 172.21.15.84:0/1956824987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f8554a0 0x56156f8537e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.158+0000 7fcae5d96740 1 -- 172.21.15.84:0/1956824987 >> 172.21.15.84:0/1956824987 conn(0x56156f81c8e0 msgr2=0x56156f850310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:24.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.158+0000 7fcae5d96740 1 -- 172.21.15.84:0/1956824987 shutdown_connections 2024-05-22T23:02:24.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.158+0000 7fcae5d96740 1 -- 172.21.15.84:0/1956824987 wait complete. 2024-05-22T23:02:24.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.158+0000 7fcae5d96740 1 -- start start 2024-05-22T23:02:24.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.159+0000 7fcae5d96740 1 Processor -- start 2024-05-22T23:02:24.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.159+0000 7fcae5d96740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156f853db0 0x56156f85bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.159+0000 7fcae5d96740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56156f8554a0 0x56156f85c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.159+0000 7fcae5d96740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f85eb30 0x56156f85c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.159+0000 7fcae5d96740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56156f861960 con 0x56156f85eb30 2024-05-22T23:02:24.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.159+0000 7fcae5d96740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56156f7720f0 con 0x56156f8554a0 2024-05-22T23:02:24.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.159+0000 7fcae5d96740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56156f861ae0 con 0x56156f853db0 2024-05-22T23:02:24.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.159+0000 7fcae5d96740 10 client.0 _ll_register_callbacks cb 0x56156f776f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:02:24.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.159+0000 7fcae1546640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f85eb30 0x56156f85c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.159+0000 7fcae1546640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f85eb30 0x56156f85c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35586/0 (socket says 172.21.15.84:35586) 2024-05-22T23:02:24.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.159+0000 7fcae1546640 1 -- 172.21.15.84:0/3252601810 learned_addr learned my addr 172.21.15.84:0/3252601810 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:24.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.159+0000 7fcadbfff640 1 --2- 172.21.15.84:0/3252601810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56156f8554a0 0x56156f85c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.159+0000 7fcae1546640 1 -- 172.21.15.84:0/3252601810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156f853db0 msgr2=0x56156f85bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:24.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.159+0000 7fcae0d45640 1 --2- 172.21.15.84:0/3252601810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156f853db0 0x56156f85bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.159+0000 7fcae1546640 1 --2- 172.21.15.84:0/3252601810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156f853db0 0x56156f85bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.159+0000 7fcae1546640 1 -- 172.21.15.84:0/3252601810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56156f8554a0 msgr2=0x56156f85c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:24.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.159+0000 7fcae1546640 1 --2- 172.21.15.84:0/3252601810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56156f8554a0 0x56156f85c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.159+0000 7fcae1546640 1 -- 172.21.15.84:0/3252601810 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56156f5957d0 con 0x56156f85eb30 2024-05-22T23:02:24.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.160+0000 7fcae1546640 1 --2- 172.21.15.84:0/3252601810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f85eb30 0x56156f85c5a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcad4031af0 tx=0x7fcad40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4058adb3fa8658fc server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:24.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.160+0000 7fcabbfff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:02:24.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.160+0000 7fcabbfff640 1 -- 172.21.15.84:0/3252601810 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad4031d20 con 0x56156f85eb30 2024-05-22T23:02:24.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.160+0000 7fcabbfff640 1 -- 172.21.15.84:0/3252601810 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fcad4004540 con 0x56156f85eb30 2024-05-22T23:02:24.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.160+0000 7fcae5d96740 1 -- 172.21.15.84:0/3252601810 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcad00a3fd0 con 0x56156f85eb30 2024-05-22T23:02:24.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.160+0000 7fcabbfff640 1 -- 172.21.15.84:0/3252601810 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fcad4004740 con 0x56156f85eb30 2024-05-22T23:02:24.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.160+0000 7fcae5d96740 10 client.15438 fetch_fsmap learned FSMap version 18 2024-05-22T23:02:24.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.160+0000 7fcae5d96740 1 -- 172.21.15.84:0/3252601810 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x56156f4cb3f0 con 0x56156f85eb30 2024-05-22T23:02:24.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.160+0000 7fcabbfff640 1 -- 172.21.15.84:0/3252601810 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcad00a3fd0 con 0x56156f85eb30 2024-05-22T23:02:24.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.160+0000 7fcabbfff640 1 -- 172.21.15.84:0/3252601810 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fcad4030070 con 0x56156f85eb30 2024-05-22T23:02:24.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.160+0000 7fcae5d96740 10 client.15438 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:02:24.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.160+0000 7fcae5d96740 10 client.15438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:02:24.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.160+0000 7fcae5d96740 20 client.15438 populate_metadata read hostname 'smithi084' 2024-05-22T23:02:24.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.160+0000 7fcae5d96740 1 --2- 172.21.15.84:0/3252601810 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56156f86f0c0 0x56156f8714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.160+0000 7fcae5d96740 4 client.15438 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:02:24.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.160+0000 7fcae5d96740 1 -- 172.21.15.84:0/3252601810 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56156f7c54e0 con 0x56156f86f0c0 2024-05-22T23:02:24.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.161+0000 7fcae0d45640 1 --2- 172.21.15.84:0/3252601810 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56156f86f0c0 0x56156f8714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.161+0000 7fcabbfff640 10 client.15438 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:24.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.161+0000 7fcae0d45640 1 --2- 172.21.15.84:0/3252601810 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56156f86f0c0 0x56156f8714e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=361edaa5eecee0cc server_cookie=1d5f61154c40707f in_seq=0 out_seq=0 2024-05-22T23:02:24.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.162+0000 7fcabbfff640 1 -- 172.21.15.84:0/3252601810 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fcad0080ae0 con 0x56156f86f0c0 2024-05-22T23:02:24.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.162+0000 7fcabbfff640 10 client.15438 handle_command_reply: tid=0 2024-05-22T23:02:24.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.162+0000 7fcae5d96740 1 client.15438 shutdown 2024-05-22T23:02:24.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.162+0000 7fcae5d96740 1 -- 172.21.15.84:0/3252601810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f85eb30 msgr2=0x56156f85c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:24.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.162+0000 7fcae5d96740 1 --2- 172.21.15.84:0/3252601810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f85eb30 0x56156f85c5a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcad4031af0 tx=0x7fcad40095c0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.162+0000 7fcae5d96740 1 -- 172.21.15.84:0/3252601810 shutdown_connections 2024-05-22T23:02:24.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.162+0000 7fcae5d96740 1 --2- 172.21.15.84:0/3252601810 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56156f86f0c0 0x56156f8714e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.162+0000 7fcae5d96740 1 --2- 172.21.15.84:0/3252601810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f85eb30 0x56156f85c5a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.162+0000 7fcae5d96740 1 --2- 172.21.15.84:0/3252601810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56156f8554a0 0x56156f85c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.162+0000 7fcabbfff640 0 client.15438 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:24.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.162+0000 7fcae5d96740 1 --2- 172.21.15.84:0/3252601810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156f853db0 0x56156f85bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.162+0000 7fcae5d96740 1 -- 172.21.15.84:0/3252601810 >> 172.21.15.84:0/3252601810 conn(0x56156f81c8e0 msgr2=0x56156f856180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:24.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.162+0000 7fcae5d96740 1 -- 172.21.15.84:0/3252601810 shutdown_connections 2024-05-22T23:02:24.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.162+0000 7fcae5d96740 1 -- 172.21.15.84:0/3252601810 wait complete. 2024-05-22T23:02:24.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.162+0000 7fcae5d96740 20 client.15438 trim_cache size 0 max 16384 2024-05-22T23:02:24.186 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.179+0000 7fcae5d96740 1 Processor -- start 2024-05-22T23:02:24.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.179+0000 7fcae5d96740 1 -- start start 2024-05-22T23:02:24.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.180+0000 7fcae5d96740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f853db0 0x56156f86a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.180+0000 7fcae5d96740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56156f8554a0 0x56156f85c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.180+0000 7fcae5d96740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156f85eb30 0x56156f85da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.180+0000 7fcae5d96740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc1ab700 con 0x56156f853db0 2024-05-22T23:02:24.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.180+0000 7fcae5d96740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc1adf80 con 0x56156f8554a0 2024-05-22T23:02:24.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.180+0000 7fcae5d96740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc1abd70 con 0x56156f85eb30 2024-05-22T23:02:24.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.180+0000 7fcae0d45640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f853db0 0x56156f86a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.180+0000 7fcae0d45640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f853db0 0x56156f86a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35588/0 (socket says 172.21.15.84:35588) 2024-05-22T23:02:24.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.180+0000 7fcae0d45640 1 -- 172.21.15.84:0/1554805581 learned_addr learned my addr 172.21.15.84:0/1554805581 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:24.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.180+0000 7fcae0d45640 1 -- 172.21.15.84:0/1554805581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156f85eb30 msgr2=0x56156f85da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:24.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.180+0000 7fcae0d45640 1 --2- 172.21.15.84:0/1554805581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156f85eb30 0x56156f85da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.180+0000 7fcae0d45640 1 -- 172.21.15.84:0/1554805581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56156f8554a0 msgr2=0x56156f85c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:24.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.180+0000 7fcae0d45640 1 --2- 172.21.15.84:0/1554805581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56156f8554a0 0x56156f85c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.180+0000 7fcae0d45640 1 -- 172.21.15.84:0/1554805581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56156f86b7e0 con 0x56156f853db0 2024-05-22T23:02:24.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.180+0000 7fcae0d45640 1 --2- 172.21.15.84:0/1554805581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f853db0 0x56156f86a860 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcad0097770 tx=0x7fcad009adf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be10ae3a79fd1c8f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:24.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.180+0000 7fcac67fc640 1 -- 172.21.15.84:0/1554805581 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad00b42a0 con 0x56156f853db0 2024-05-22T23:02:24.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.180+0000 7fcac67fc640 1 -- 172.21.15.84:0/1554805581 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56156f7c54e0 con 0x56156f853db0 2024-05-22T23:02:24.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.180+0000 7fcac67fc640 1 -- 172.21.15.84:0/1554805581 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad00cc940 con 0x56156f853db0 2024-05-22T23:02:24.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.181+0000 7fcae5d96740 1 -- 172.21.15.84:0/1554805581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f853db0 msgr2=0x56156f86a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:24.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.181+0000 7fcae5d96740 1 --2- 172.21.15.84:0/1554805581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f853db0 0x56156f86a860 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcad0097770 tx=0x7fcad009adf0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.181+0000 7fcae5d96740 1 -- 172.21.15.84:0/1554805581 shutdown_connections 2024-05-22T23:02:24.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.181+0000 7fcae5d96740 1 --2- 172.21.15.84:0/1554805581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156f85eb30 0x56156f85da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.181+0000 7fcae5d96740 1 --2- 172.21.15.84:0/1554805581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56156f8554a0 0x56156f85c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.181+0000 7fcae5d96740 1 --2- 172.21.15.84:0/1554805581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f853db0 0x56156f86a860 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.181+0000 7fcae5d96740 1 -- 172.21.15.84:0/1554805581 >> 172.21.15.84:0/1554805581 conn(0x56156f81c8e0 msgr2=0x56156f85d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:24.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.181+0000 7fcae5d96740 1 -- 172.21.15.84:0/1554805581 shutdown_connections 2024-05-22T23:02:24.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.181+0000 7fcae5d96740 1 -- 172.21.15.84:0/1554805581 wait complete. 2024-05-22T23:02:24.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.181+0000 7fcae5d96740 1 -- start start 2024-05-22T23:02:24.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.181+0000 7fcae5d96740 1 Processor -- start 2024-05-22T23:02:24.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcae5d96740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f853db0 0x56156f867070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcae5d96740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156f8554a0 0x56156f85f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcae5d96740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56156f85eb30 0x56156f85fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcae5d96740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc1ac440 con 0x56156f853db0 2024-05-22T23:02:24.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcae5d96740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc1ad8b0 con 0x56156f85eb30 2024-05-22T23:02:24.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcae5d96740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc1acb10 con 0x56156f8554a0 2024-05-22T23:02:24.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcae5d96740 10 client.0 _ll_register_callbacks cb 0x7fcad4041050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:02:24.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcae0d45640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f853db0 0x56156f867070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcae0d45640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f853db0 0x56156f867070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35596/0 (socket says 172.21.15.84:35596) 2024-05-22T23:02:24.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcae0d45640 1 -- 172.21.15.84:0/1391700412 learned_addr learned my addr 172.21.15.84:0/1391700412 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:24.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcadbfff640 1 --2- 172.21.15.84:0/1391700412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156f8554a0 0x56156f85f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcae0d45640 1 -- 172.21.15.84:0/1391700412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156f8554a0 msgr2=0x56156f85f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:24.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcae0d45640 1 --2- 172.21.15.84:0/1391700412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156f8554a0 0x56156f85f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcae0d45640 1 -- 172.21.15.84:0/1391700412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56156f85eb30 msgr2=0x56156f85fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:24.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcae0d45640 1 --2- 172.21.15.84:0/1391700412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56156f85eb30 0x56156f85fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcae0d45640 1 -- 172.21.15.84:0/1391700412 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56156f48b0f0 con 0x56156f853db0 2024-05-22T23:02:24.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcadbfff640 1 --2- 172.21.15.84:0/1391700412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156f8554a0 0x56156f85f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:02:24.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcae0d45640 1 --2- 172.21.15.84:0/1391700412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f853db0 0x56156f867070 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009fd0 tx=0x7fcad00cc680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e2d96f1cdbaf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:24.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcabbfff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:02:24.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcabbfff640 1 -- 172.21.15.84:0/1391700412 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad00a3fd0 con 0x56156f853db0 2024-05-22T23:02:24.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcabbfff640 1 -- 172.21.15.84:0/1391700412 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fcad0099660 con 0x56156f853db0 2024-05-22T23:02:24.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcae5d96740 1 -- 172.21.15.84:0/1391700412 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56156f7c54e0 con 0x56156f853db0 2024-05-22T23:02:24.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcabbfff640 1 -- 172.21.15.84:0/1391700412 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fcad00979d0 con 0x56156f853db0 2024-05-22T23:02:24.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcae5d96740 10 client.15444 fetch_fsmap learned FSMap version 18 2024-05-22T23:02:24.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcae5d96740 1 -- 172.21.15.84:0/1391700412 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x56156f776fe0 con 0x56156f853db0 2024-05-22T23:02:24.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.182+0000 7fcabbfff640 1 -- 172.21.15.84:0/1391700412 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56156f7c54e0 con 0x56156f853db0 2024-05-22T23:02:24.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.183+0000 7fcabbfff640 1 -- 172.21.15.84:0/1391700412 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fcad00a7070 con 0x56156f853db0 2024-05-22T23:02:24.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.183+0000 7fcae5d96740 10 client.15444 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:02:24.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.183+0000 7fcae5d96740 10 client.15444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:02:24.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.183+0000 7fcae5d96740 20 client.15444 populate_metadata read hostname 'smithi084' 2024-05-22T23:02:24.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.183+0000 7fcae5d96740 1 --2- 172.21.15.84:0/1391700412 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56156f8d5600 0x56156f8d79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.183+0000 7fcae5d96740 4 client.15444 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:02:24.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.183+0000 7fcae5d96740 1 -- 172.21.15.84:0/1391700412 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56156f7bf830 con 0x56156f8d5600 2024-05-22T23:02:24.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.183+0000 7fcadbfff640 1 --2- 172.21.15.84:0/1391700412 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56156f8d5600 0x56156f8d79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.183+0000 7fcabbfff640 10 client.15444 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:24.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.183+0000 7fcadbfff640 1 --2- 172.21.15.84:0/1391700412 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56156f8d5600 0x56156f8d79f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a1d27cb97cc586a server_cookie=77ffc27a10746899 in_seq=0 out_seq=0 2024-05-22T23:02:24.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.184+0000 7fcabbfff640 1 -- 172.21.15.84:0/1391700412 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56156f7bf830 con 0x56156f8d5600 2024-05-22T23:02:24.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.184+0000 7fcabbfff640 10 client.15444 handle_command_reply: tid=0 2024-05-22T23:02:24.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.184+0000 7fcae5d96740 1 client.15444 shutdown 2024-05-22T23:02:24.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.184+0000 7fcae5d96740 1 -- 172.21.15.84:0/1391700412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f853db0 msgr2=0x56156f867070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:24.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.184+0000 7fcae5d96740 1 --2- 172.21.15.84:0/1391700412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f853db0 0x56156f867070 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009fd0 tx=0x7fcad00cc680 comp rx=0 tx=0).stop 2024-05-22T23:02:24.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.184+0000 7fcae5d96740 1 -- 172.21.15.84:0/1391700412 shutdown_connections 2024-05-22T23:02:24.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.184+0000 7fcae5d96740 1 --2- 172.21.15.84:0/1391700412 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56156f8d5600 0x56156f8d79f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.184+0000 7fcae5d96740 1 --2- 172.21.15.84:0/1391700412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56156f85eb30 0x56156f85fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.184+0000 7fcae5d96740 1 --2- 172.21.15.84:0/1391700412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156f8554a0 0x56156f85f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.184+0000 7fcabbfff640 0 client.15444 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:24.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.184+0000 7fcae5d96740 1 --2- 172.21.15.84:0/1391700412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56156f853db0 0x56156f867070 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.184+0000 7fcae5d96740 1 -- 172.21.15.84:0/1391700412 >> 172.21.15.84:0/1391700412 conn(0x56156f81c8e0 msgr2=0x56156f856180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:24.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.184+0000 7fcae5d96740 1 -- 172.21.15.84:0/1391700412 shutdown_connections 2024-05-22T23:02:24.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.184+0000 7fcae5d96740 1 -- 172.21.15.84:0/1391700412 wait complete. 2024-05-22T23:02:24.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.184+0000 7fcae5d96740 20 client.15444 trim_cache size 0 max 16384 2024-05-22T23:02:24.193 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:02:24.193 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "no active scrubs running", 2024-05-22T23:02:24.193 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": {} 2024-05-22T23:02:24.193 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:02:24.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.184+0000 7fcae2fa9640 1 -- 172.21.15.84:0/1571425774 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fcacc0a2ea0 msgr2=0x7fcacc0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:24.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.184+0000 7fcae2fa9640 1 --2- 172.21.15.84:0/1571425774 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fcacc0a2ea0 0x7fcacc0a5360 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcad402ae00 tx=0x7fcad4009860 comp rx=0 tx=0).stop 2024-05-22T23:02:24.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.184+0000 7fcae2fa9640 1 -- 172.21.15.84:0/1571425774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcadc101890 msgr2=0x7fcadc198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:24.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.184+0000 7fcae2fa9640 1 --2- 172.21.15.84:0/1571425774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcadc101890 0x7fcadc198190 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fcad000d950 tx=0x7fcad000de10 comp rx=0 tx=0).stop 2024-05-22T23:02:24.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.185+0000 7fcae2fa9640 1 -- 172.21.15.84:0/1571425774 shutdown_connections 2024-05-22T23:02:24.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.185+0000 7fcae2fa9640 1 --2- 172.21.15.84:0/1571425774 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fcacc0a2ea0 0x7fcacc0a5360 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.185+0000 7fcae2fa9640 1 --2- 172.21.15.84:0/1571425774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcadc1078a0 0x7fcadc19ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.185+0000 7fcae2fa9640 1 --2- 172.21.15.84:0/1571425774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcadc102250 0x7fcadc1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.185+0000 7fcae2fa9640 1 --2- 172.21.15.84:0/1571425774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcadc101890 0x7fcadc198190 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.185+0000 7fcae2fa9640 1 -- 172.21.15.84:0/1571425774 >> 172.21.15.84:0/1571425774 conn(0x7fcadc0fd380 msgr2=0x7fcadc0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:24.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.185+0000 7fcae2fa9640 1 -- 172.21.15.84:0/1571425774 shutdown_connections 2024-05-22T23:02:24.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.185+0000 7fcae2fa9640 1 -- 172.21.15.84:0/1571425774 wait complete. 2024-05-22T23:02:24.198 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-22T23:02:24.198 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:02:24.300 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.296+0000 7f5e4de38640 1 -- 172.21.15.84:0/75689749 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e48100b80 msgr2=0x7f5e4810cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:24.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.296+0000 7f5e4de38640 1 --2- 172.21.15.84:0/75689749 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e48100b80 0x7f5e4810cd90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009ca0 tx=0x7f5e4002a950 comp rx=0 tx=0).stop 2024-05-22T23:02:24.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.296+0000 7f5e4de38640 1 -- 172.21.15.84:0/75689749 shutdown_connections 2024-05-22T23:02:24.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.296+0000 7f5e4de38640 1 --2- 172.21.15.84:0/75689749 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e48100b80 0x7f5e4810cd90 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.296+0000 7f5e4de38640 1 --2- 172.21.15.84:0/75689749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e481001c0 0x7f5e48100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.296+0000 7f5e4de38640 1 --2- 172.21.15.84:0/75689749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e48104f00 0x7f5e481052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.297+0000 7f5e4de38640 1 -- 172.21.15.84:0/75689749 >> 172.21.15.84:0/75689749 conn(0x7f5e48075a50 msgr2=0x7f5e48075e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:24.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.297+0000 7f5e4de38640 1 -- 172.21.15.84:0/75689749 shutdown_connections 2024-05-22T23:02:24.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.297+0000 7f5e4de38640 1 -- 172.21.15.84:0/75689749 wait complete. 2024-05-22T23:02:24.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.297+0000 7f5e4de38640 1 Processor -- start 2024-05-22T23:02:24.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.297+0000 7f5e4de38640 1 -- start start 2024-05-22T23:02:24.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.297+0000 7f5e4de38640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e481001c0 0x7f5e48198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.297+0000 7f5e4de38640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e48100b80 0x7f5e48198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.297+0000 7f5e4de38640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e48104f00 0x7f5e4819c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.297+0000 7f5e4de38640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5e48111ce0 con 0x7f5e48104f00 2024-05-22T23:02:24.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.297+0000 7f5e4de38640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5e48111b60 con 0x7f5e48100b80 2024-05-22T23:02:24.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.297+0000 7f5e4de38640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5e48111e60 con 0x7f5e481001c0 2024-05-22T23:02:24.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.297+0000 7f5e47fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e48104f00 0x7f5e4819c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.297+0000 7f5e47fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e48104f00 0x7f5e4819c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35606/0 (socket says 172.21.15.84:35606) 2024-05-22T23:02:24.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.298+0000 7f5e47fff640 1 -- 172.21.15.84:0/3487412453 learned_addr learned my addr 172.21.15.84:0/3487412453 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:24.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.298+0000 7f5e3ffff640 1 --2- 172.21.15.84:0/3487412453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e48100b80 0x7f5e48198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.298+0000 7f5e477fe640 1 --2- 172.21.15.84:0/3487412453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e481001c0 0x7f5e48198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.298+0000 7f5e47fff640 1 -- 172.21.15.84:0/3487412453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e481001c0 msgr2=0x7f5e48198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:24.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.298+0000 7f5e47fff640 1 --2- 172.21.15.84:0/3487412453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e481001c0 0x7f5e48198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.298+0000 7f5e47fff640 1 -- 172.21.15.84:0/3487412453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e48100b80 msgr2=0x7f5e48198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:24.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.298+0000 7f5e47fff640 1 --2- 172.21.15.84:0/3487412453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e48100b80 0x7f5e48198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.298+0000 7f5e47fff640 1 -- 172.21.15.84:0/3487412453 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4819cfe0 con 0x7f5e48104f00 2024-05-22T23:02:24.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.298+0000 7f5e47fff640 1 --2- 172.21.15.84:0/3487412453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e48104f00 0x7f5e4819c8b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5e40004af0 tx=0x7f5e4002f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:24.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.298+0000 7f5e457fa640 1 -- 172.21.15.84:0/3487412453 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e40034040 con 0x7f5e48104f00 2024-05-22T23:02:24.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.298+0000 7f5e457fa640 1 -- 172.21.15.84:0/3487412453 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e40030920 con 0x7f5e48104f00 2024-05-22T23:02:24.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.298+0000 7f5e4de38640 1 -- 172.21.15.84:0/3487412453 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e4819d270 con 0x7f5e48104f00 2024-05-22T23:02:24.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.298+0000 7f5e457fa640 1 -- 172.21.15.84:0/3487412453 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e40004740 con 0x7f5e48104f00 2024-05-22T23:02:24.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.298+0000 7f5e4de38640 1 -- 172.21.15.84:0/3487412453 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e481a4b60 con 0x7f5e48104f00 2024-05-22T23:02:24.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.299+0000 7f5e4de38640 1 -- 172.21.15.84:0/3487412453 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e48068ae0 con 0x7f5e48104f00 2024-05-22T23:02:24.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.300+0000 7f5e457fa640 1 -- 172.21.15.84:0/3487412453 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f5e40037020 con 0x7f5e48104f00 2024-05-22T23:02:24.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.301+0000 7f5e457fa640 1 --2- 172.21.15.84:0/3487412453 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f5e340a2e20 0x7f5e340a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.301+0000 7f5e477fe640 1 --2- 172.21.15.84:0/3487412453 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f5e340a2e20 0x7f5e340a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.301+0000 7f5e457fa640 1 -- 172.21.15.84:0/3487412453 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f5e400f03d0 con 0x7f5e48104f00 2024-05-22T23:02:24.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.301+0000 7f5e477fe640 1 --2- 172.21.15.84:0/3487412453 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f5e340a2e20 0x7f5e340a52e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5e48078280 tx=0x7f5e38005f90 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:24.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.305+0000 7f5e457fa640 1 -- 172.21.15.84:0/3487412453 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5e400bceb0 con 0x7f5e48104f00 2024-05-22T23:02:24.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.519+0000 7f5e4de38640 1 -- 172.21.15.84:0/3487412453 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e481a4d30 con 0x7f5e48104f00 2024-05-22T23:02:24.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.519+0000 7f5e457fa640 1 -- 172.21.15.84:0/3487412453 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7f5e400bc7b0 con 0x7f5e48104f00 2024-05-22T23:02:24.522 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:02:24.522 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:02:24.522 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:02:24.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.521+0000 7f5e4de38640 1 -- 172.21.15.84:0/3487412453 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f5e340a2e20 msgr2=0x7f5e340a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:24.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.521+0000 7f5e4de38640 1 --2- 172.21.15.84:0/3487412453 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f5e340a2e20 0x7f5e340a52e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5e48078280 tx=0x7f5e38005f90 comp rx=0 tx=0).stop 2024-05-22T23:02:24.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.521+0000 7f5e4de38640 1 -- 172.21.15.84:0/3487412453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e48104f00 msgr2=0x7f5e4819c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:24.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.521+0000 7f5e4de38640 1 --2- 172.21.15.84:0/3487412453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e48104f00 0x7f5e4819c8b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5e40004af0 tx=0x7f5e4002f5b0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.521+0000 7f5e4de38640 1 -- 172.21.15.84:0/3487412453 shutdown_connections 2024-05-22T23:02:24.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.521+0000 7f5e4de38640 1 --2- 172.21.15.84:0/3487412453 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f5e340a2e20 0x7f5e340a52e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.521+0000 7f5e4de38640 1 --2- 172.21.15.84:0/3487412453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e48104f00 0x7f5e4819c8b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.521+0000 7f5e4de38640 1 --2- 172.21.15.84:0/3487412453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e48100b80 0x7f5e48198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.521+0000 7f5e4de38640 1 --2- 172.21.15.84:0/3487412453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e481001c0 0x7f5e48198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.521+0000 7f5e4de38640 1 -- 172.21.15.84:0/3487412453 >> 172.21.15.84:0/3487412453 conn(0x7f5e48075a50 msgr2=0x7f5e480fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:24.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.521+0000 7f5e4de38640 1 -- 172.21.15.84:0/3487412453 shutdown_connections 2024-05-22T23:02:24.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.521+0000 7f5e4de38640 1 -- 172.21.15.84:0/3487412453 wait complete. 2024-05-22T23:02:24.535 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-22T23:02:24.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.636+0000 7f594455a640 1 -- 172.21.15.84:0/2851257529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f593c106750 msgr2=0x7f593c106b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:24.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.636+0000 7f594455a640 1 --2- 172.21.15.84:0/2851257529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f593c106750 0x7f593c106b30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5938009930 tx=0x7f593802a420 comp rx=0 tx=0).stop 2024-05-22T23:02:24.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.637+0000 7f594455a640 1 -- 172.21.15.84:0/2851257529 shutdown_connections 2024-05-22T23:02:24.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.637+0000 7f594455a640 1 --2- 172.21.15.84:0/2851257529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c101100 0x7f593c10d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.637+0000 7f594455a640 1 --2- 172.21.15.84:0/2851257529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f593c100740 0x7f593c100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.637+0000 7f594455a640 1 --2- 172.21.15.84:0/2851257529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f593c106750 0x7f593c106b30 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.637+0000 7f594455a640 1 -- 172.21.15.84:0/2851257529 >> 172.21.15.84:0/2851257529 conn(0x7f593c077870 msgr2=0x7f593c077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:24.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.637+0000 7f594455a640 1 -- 172.21.15.84:0/2851257529 shutdown_connections 2024-05-22T23:02:24.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.637+0000 7f594455a640 1 -- 172.21.15.84:0/2851257529 wait complete. 2024-05-22T23:02:24.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.637+0000 7f594455a640 1 Processor -- start 2024-05-22T23:02:24.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.637+0000 7f594455a640 1 -- start start 2024-05-22T23:02:24.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.637+0000 7f594455a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f593c100740 0x7f593c1970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.637+0000 7f594455a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f593c101100 0x7f593c197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.637+0000 7f594455a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c106750 0x7f593c19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.637+0000 7f594455a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f593c110b60 con 0x7f593c100740 2024-05-22T23:02:24.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.637+0000 7f594455a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1109e0 con 0x7f593c106750 2024-05-22T23:02:24.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.637+0000 7f594455a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f593c110ce0 con 0x7f593c101100 2024-05-22T23:02:24.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.637+0000 7f59422f6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f593c100740 0x7f593c1970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.637+0000 7f59422f6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f593c100740 0x7f593c1970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35634/0 (socket says 172.21.15.84:35634) 2024-05-22T23:02:24.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.638+0000 7f59422f6640 1 -- 172.21.15.84:0/3948445055 learned_addr learned my addr 172.21.15.84:0/3948445055 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:24.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.638+0000 7f5941af5640 1 --2- 172.21.15.84:0/3948445055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f593c101100 0x7f593c197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.638+0000 7f5942af7640 1 --2- 172.21.15.84:0/3948445055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c106750 0x7f593c19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.638+0000 7f59422f6640 1 -- 172.21.15.84:0/3948445055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f593c101100 msgr2=0x7f593c197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:24.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.638+0000 7f59422f6640 1 --2- 172.21.15.84:0/3948445055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f593c101100 0x7f593c197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.638+0000 7f59422f6640 1 -- 172.21.15.84:0/3948445055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c106750 msgr2=0x7f593c19b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:24.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.638+0000 7f59422f6640 1 --2- 172.21.15.84:0/3948445055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c106750 0x7f593c19b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.638+0000 7f59422f6640 1 -- 172.21.15.84:0/3948445055 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593c19c0d0 con 0x7f593c100740 2024-05-22T23:02:24.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.638+0000 7f5942af7640 1 --2- 172.21.15.84:0/3948445055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c106750 0x7f593c19b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:02:24.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.638+0000 7f59422f6640 1 --2- 172.21.15.84:0/3948445055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f593c100740 0x7f593c1970e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5938009a50 tx=0x7f5938004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:24.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.638+0000 7f592f7fe640 1 -- 172.21.15.84:0/3948445055 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5938046070 con 0x7f593c100740 2024-05-22T23:02:24.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.638+0000 7f592f7fe640 1 -- 172.21.15.84:0/3948445055 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5938003ad0 con 0x7f593c100740 2024-05-22T23:02:24.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.638+0000 7f594455a640 1 -- 172.21.15.84:0/3948445055 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f593c19c360 con 0x7f593c100740 2024-05-22T23:02:24.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.638+0000 7f592f7fe640 1 -- 172.21.15.84:0/3948445055 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5938035050 con 0x7f593c100740 2024-05-22T23:02:24.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.638+0000 7f594455a640 1 -- 172.21.15.84:0/3948445055 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f593c068a00 con 0x7f593c100740 2024-05-22T23:02:24.649 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.640+0000 7f592f7fe640 1 -- 172.21.15.84:0/3948445055 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f5938049070 con 0x7f593c100740 2024-05-22T23:02:24.649 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.641+0000 7f5947268740 1 Processor -- start 2024-05-22T23:02:24.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.641+0000 7f5947268740 1 -- start start 2024-05-22T23:02:24.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.641+0000 7f5947268740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952236cb0 0x55b952237090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.641+0000 7f5947268740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b952237660 0x55b952245e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.641+0000 7f5947268740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b952246380 0x55b952248770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.641+0000 7f5947268740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b95219bdb0 con 0x55b952236cb0 2024-05-22T23:02:24.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.641+0000 7f5947268740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b95213a400 con 0x55b952237660 2024-05-22T23:02:24.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.641+0000 7f5947268740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b952197d70 con 0x55b952246380 2024-05-22T23:02:24.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.641+0000 7f592f7fe640 1 --2- 172.21.15.84:0/3948445055 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f59280a2e20 0x7f59280a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.641+0000 7f5942af7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952236cb0 0x55b952237090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.641+0000 7f5942af7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952236cb0 0x55b952237090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35644/0 (socket says 172.21.15.84:35644) 2024-05-22T23:02:24.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.641+0000 7f5942af7640 1 -- 172.21.15.84:0/581372336 learned_addr learned my addr 172.21.15.84:0/581372336 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:24.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.641+0000 7f5941af5640 1 --2- 172.21.15.84:0/581372336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b952246380 0x55b952248770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.641+0000 7f592f7fe640 1 -- 172.21.15.84:0/3948445055 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f59380f1290 con 0x7f593c100740 2024-05-22T23:02:24.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.641+0000 7f5942af7640 1 --2- 172.21.15.84:0/3948445055 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f59280a2e20 0x7f59280a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.641+0000 7f59422f6640 1 --2- 172.21.15.84:0/581372336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b952237660 0x55b952245e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.642+0000 7f5942af7640 1 -- 172.21.15.84:0/581372336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b952246380 msgr2=0x55b952248770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:24.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.642+0000 7f5942af7640 1 --2- 172.21.15.84:0/581372336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b952246380 0x55b952248770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.642+0000 7f5942af7640 1 -- 172.21.15.84:0/581372336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b952237660 msgr2=0x55b952245e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:24.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.642+0000 7f5942af7640 1 --2- 172.21.15.84:0/581372336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b952237660 0x55b952245e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.642+0000 7f5942af7640 1 -- 172.21.15.84:0/581372336 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b95212c2f0 con 0x55b952236cb0 2024-05-22T23:02:24.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.642+0000 7f5941af5640 1 --2- 172.21.15.84:0/581372336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b952246380 0x55b952248770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:02:24.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.642+0000 7f5942af7640 1 --2- 172.21.15.84:0/581372336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952236cb0 0x55b952237090 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f59340129e0 tx=0x7f5934012de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe8d3077d4d9ee4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:24.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.642+0000 7f592cbfe640 1 -- 172.21.15.84:0/581372336 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934022070 con 0x55b952236cb0 2024-05-22T23:02:24.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.642+0000 7f592cbfe640 1 -- 172.21.15.84:0/581372336 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5934012460 con 0x55b952236cb0 2024-05-22T23:02:24.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.642+0000 7f592cbfe640 1 -- 172.21.15.84:0/581372336 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934005280 con 0x55b952236cb0 2024-05-22T23:02:24.651 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.643+0000 7f5947268740 1 -- 172.21.15.84:0/581372336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952236cb0 msgr2=0x55b952237090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:24.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.643+0000 7f5942af7640 1 --2- 172.21.15.84:0/3948445055 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f59280a2e20 0x7f59280a52e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5934020000 tx=0x7f593401f000 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:24.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.643+0000 7f5947268740 1 --2- 172.21.15.84:0/581372336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952236cb0 0x55b952237090 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f59340129e0 tx=0x7f5934012de0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.643+0000 7f5947268740 1 -- 172.21.15.84:0/581372336 shutdown_connections 2024-05-22T23:02:24.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.643+0000 7f5947268740 1 --2- 172.21.15.84:0/581372336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b952246380 0x55b952248770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.643+0000 7f5947268740 1 --2- 172.21.15.84:0/581372336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b952237660 0x55b952245e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.643+0000 7f5947268740 1 --2- 172.21.15.84:0/581372336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952236cb0 0x55b952237090 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.643+0000 7f5947268740 1 -- 172.21.15.84:0/581372336 >> 172.21.15.84:0/581372336 conn(0x55b9522332a0 msgr2=0x55b952233680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:24.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.643+0000 7f5947268740 1 -- 172.21.15.84:0/581372336 shutdown_connections 2024-05-22T23:02:24.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.643+0000 7f5947268740 1 -- 172.21.15.84:0/581372336 wait complete. 2024-05-22T23:02:24.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.643+0000 7f5947268740 1 -- start start 2024-05-22T23:02:24.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.643+0000 7f5947268740 1 Processor -- start 2024-05-22T23:02:24.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.643+0000 7f5947268740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b952236cb0 0x55b952243330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.643+0000 7f5947268740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952237660 0x55b952243870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.643+0000 7f5947268740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b952246380 0x55b952243db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.644+0000 7f5947268740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b952249140 con 0x55b952237660 2024-05-22T23:02:24.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.644+0000 7f5947268740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b9521562e0 con 0x55b952246380 2024-05-22T23:02:24.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.644+0000 7f5947268740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b9522492c0 con 0x55b952236cb0 2024-05-22T23:02:24.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.644+0000 7f5947268740 10 client.0 _ll_register_callbacks cb 0x55b95215fa70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:02:24.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.644+0000 7f5941af5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952237660 0x55b952243870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.644+0000 7f5941af5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952237660 0x55b952243870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35646/0 (socket says 172.21.15.84:35646) 2024-05-22T23:02:24.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.644+0000 7f5941af5640 1 -- 172.21.15.84:0/935636994 learned_addr learned my addr 172.21.15.84:0/935636994 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:24.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.644+0000 7f5942af7640 1 --2- 172.21.15.84:0/935636994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b952246380 0x55b952243db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.644+0000 7f59422f6640 1 --2- 172.21.15.84:0/935636994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b952236cb0 0x55b952243330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.644+0000 7f5941af5640 1 -- 172.21.15.84:0/935636994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b952236cb0 msgr2=0x55b952243330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:24.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.644+0000 7f5941af5640 1 --2- 172.21.15.84:0/935636994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b952236cb0 0x55b952243330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.644+0000 7f5941af5640 1 -- 172.21.15.84:0/935636994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b952246380 msgr2=0x55b952243db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:24.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.644+0000 7f5941af5640 1 --2- 172.21.15.84:0/935636994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b952246380 0x55b952243db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.644+0000 7f5941af5640 1 -- 172.21.15.84:0/935636994 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b951eaf3f0 con 0x55b952237660 2024-05-22T23:02:24.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.644+0000 7f5941af5640 1 --2- 172.21.15.84:0/935636994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952237660 0x55b952243870 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f59300115a0 tx=0x7f5930009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1512ab62a519493d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:24.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.644+0000 7f5925ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:02:24.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.644+0000 7f5925ffb640 1 -- 172.21.15.84:0/935636994 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5930013670 con 0x55b952237660 2024-05-22T23:02:24.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.644+0000 7f5925ffb640 1 -- 172.21.15.84:0/935636994 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f5930013810 con 0x55b952237660 2024-05-22T23:02:24.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.644+0000 7f5947268740 1 -- 172.21.15.84:0/935636994 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5934012460 con 0x55b952237660 2024-05-22T23:02:24.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.644+0000 7f5925ffb640 1 -- 172.21.15.84:0/935636994 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f5930013a10 con 0x55b952237660 2024-05-22T23:02:24.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.644+0000 7f5947268740 10 client.15462 fetch_fsmap learned FSMap version 18 2024-05-22T23:02:24.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.644+0000 7f5947268740 1 -- 172.21.15.84:0/935636994 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55b951f797d0 con 0x55b952237660 2024-05-22T23:02:24.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.645+0000 7f5925ffb640 1 -- 172.21.15.84:0/935636994 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5934012460 con 0x55b952237660 2024-05-22T23:02:24.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.645+0000 7f5925ffb640 1 -- 172.21.15.84:0/935636994 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f5930014440 con 0x55b952237660 2024-05-22T23:02:24.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.645+0000 7f5947268740 10 client.15462 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:02:24.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.645+0000 7f5947268740 10 client.15462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:02:24.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.645+0000 7f5947268740 20 client.15462 populate_metadata read hostname 'smithi084' 2024-05-22T23:02:24.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.645+0000 7f5947268740 1 --2- 172.21.15.84:0/935636994 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b952256690 0x55b952258ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.645+0000 7f5947268740 4 client.15462 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:02:24.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.645+0000 7f5947268740 1 -- 172.21.15.84:0/935636994 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9521a94e0 con 0x55b952256690 2024-05-22T23:02:24.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.645+0000 7f59422f6640 1 --2- 172.21.15.84:0/935636994 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b952256690 0x55b952258ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.645+0000 7f5925ffb640 10 client.15462 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:24.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.645+0000 7f59422f6640 1 --2- 172.21.15.84:0/935636994 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b952256690 0x55b952258ad0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=577ecc30ad257983 server_cookie=60230516b5bba6dc in_seq=0 out_seq=0 2024-05-22T23:02:24.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.646+0000 7f5925ffb640 1 -- 172.21.15.84:0/935636994 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b9521a94e0 con 0x55b952256690 2024-05-22T23:02:24.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.646+0000 7f5925ffb640 10 client.15462 handle_command_reply: tid=0 2024-05-22T23:02:24.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.646+0000 7f5947268740 1 client.15462 shutdown 2024-05-22T23:02:24.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.646+0000 7f5947268740 1 -- 172.21.15.84:0/935636994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952237660 msgr2=0x55b952243870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:24.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.646+0000 7f5947268740 1 --2- 172.21.15.84:0/935636994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952237660 0x55b952243870 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f59300115a0 tx=0x7f5930009f90 comp rx=0 tx=0).stop 2024-05-22T23:02:24.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.646+0000 7f5947268740 1 -- 172.21.15.84:0/935636994 shutdown_connections 2024-05-22T23:02:24.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.646+0000 7f5947268740 1 --2- 172.21.15.84:0/935636994 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b952256690 0x55b952258ad0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.646+0000 7f5947268740 1 --2- 172.21.15.84:0/935636994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b952246380 0x55b952243db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.646+0000 7f5947268740 1 --2- 172.21.15.84:0/935636994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952237660 0x55b952243870 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.646+0000 7f5925ffb640 0 client.15462 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:24.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.646+0000 7f5947268740 1 --2- 172.21.15.84:0/935636994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b952236cb0 0x55b952243330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.646+0000 7f5947268740 1 -- 172.21.15.84:0/935636994 >> 172.21.15.84:0/935636994 conn(0x55b9522332a0 msgr2=0x55b952239110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:24.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.646+0000 7f5947268740 1 -- 172.21.15.84:0/935636994 shutdown_connections 2024-05-22T23:02:24.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.646+0000 7f5947268740 1 -- 172.21.15.84:0/935636994 wait complete. 2024-05-22T23:02:24.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.646+0000 7f5947268740 20 client.15462 trim_cache size 0 max 16384 2024-05-22T23:02:24.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.659+0000 7f5947268740 1 Processor -- start 2024-05-22T23:02:24.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.659+0000 7f5947268740 1 -- start start 2024-05-22T23:02:24.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.660+0000 7f5947268740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952236cb0 0x55b952252040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.660+0000 7f5947268740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b952237660 0x55b9522390f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.660+0000 7f5947268740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b952246380 0x55b952243a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.660+0000 7f5947268740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1a9400 con 0x55b952236cb0 2024-05-22T23:02:24.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.660+0000 7f5947268740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1abc80 con 0x55b952237660 2024-05-22T23:02:24.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.660+0000 7f5947268740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1a9a70 con 0x55b952246380 2024-05-22T23:02:24.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.660+0000 7f59422f6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952236cb0 0x55b952252040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.660+0000 7f59422f6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952236cb0 0x55b952252040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35658/0 (socket says 172.21.15.84:35658) 2024-05-22T23:02:24.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.660+0000 7f59422f6640 1 -- 172.21.15.84:0/3860642440 learned_addr learned my addr 172.21.15.84:0/3860642440 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:24.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.660+0000 7f59422f6640 1 -- 172.21.15.84:0/3860642440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b952246380 msgr2=0x55b952243a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:24.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.660+0000 7f59422f6640 1 --2- 172.21.15.84:0/3860642440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b952246380 0x55b952243a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.660+0000 7f59422f6640 1 -- 172.21.15.84:0/3860642440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b952237660 msgr2=0x55b9522390f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:24.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.660+0000 7f59422f6640 1 --2- 172.21.15.84:0/3860642440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b952237660 0x55b9522390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.660+0000 7f59422f6640 1 -- 172.21.15.84:0/3860642440 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b952237b30 con 0x55b952236cb0 2024-05-22T23:02:24.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.660+0000 7f59422f6640 1 --2- 172.21.15.84:0/3860642440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952236cb0 0x55b952252040 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5938009900 tx=0x7f59380a8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854d61fd21851e69 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:24.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.660+0000 7f5927fff640 1 -- 172.21.15.84:0/3860642440 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59380c62a0 con 0x55b952236cb0 2024-05-22T23:02:24.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.660+0000 7f5927fff640 1 -- 172.21.15.84:0/3860642440 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59380bd630 con 0x55b952236cb0 2024-05-22T23:02:24.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.660+0000 7f5927fff640 1 -- 172.21.15.84:0/3860642440 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59380bdaf0 con 0x55b952236cb0 2024-05-22T23:02:24.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.661+0000 7f5947268740 1 -- 172.21.15.84:0/3860642440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952236cb0 msgr2=0x55b952252040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:24.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.661+0000 7f5947268740 1 --2- 172.21.15.84:0/3860642440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952236cb0 0x55b952252040 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5938009900 tx=0x7f59380a8040 comp rx=0 tx=0).stop 2024-05-22T23:02:24.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.661+0000 7f5947268740 1 -- 172.21.15.84:0/3860642440 shutdown_connections 2024-05-22T23:02:24.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.661+0000 7f5947268740 1 --2- 172.21.15.84:0/3860642440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b952246380 0x55b952243a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.661+0000 7f5947268740 1 --2- 172.21.15.84:0/3860642440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b952237660 0x55b9522390f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.661+0000 7f5947268740 1 --2- 172.21.15.84:0/3860642440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952236cb0 0x55b952252040 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.661+0000 7f5947268740 1 -- 172.21.15.84:0/3860642440 >> 172.21.15.84:0/3860642440 conn(0x55b9522332a0 msgr2=0x55b9522432f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:24.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.661+0000 7f5947268740 1 -- 172.21.15.84:0/3860642440 shutdown_connections 2024-05-22T23:02:24.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.661+0000 7f5947268740 1 -- 172.21.15.84:0/3860642440 wait complete. 2024-05-22T23:02:24.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.661+0000 7f5947268740 1 -- start start 2024-05-22T23:02:24.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.661+0000 7f5947268740 1 Processor -- start 2024-05-22T23:02:24.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.661+0000 7f5947268740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952236cb0 0x55b95224e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.662+0000 7f5947268740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b952237660 0x55b95224ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.662+0000 7f5947268740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b952246380 0x55b95224b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.662+0000 7f5947268740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1aa140 con 0x55b952236cb0 2024-05-22T23:02:24.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.662+0000 7f5947268740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1ab5b0 con 0x55b952237660 2024-05-22T23:02:24.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.662+0000 7f5947268740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1aa810 con 0x55b952246380 2024-05-22T23:02:24.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.662+0000 7f5947268740 10 client.0 _ll_register_callbacks cb 0x7f59300095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:02:24.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.662+0000 7f59422f6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952236cb0 0x55b95224e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.662+0000 7f59422f6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952236cb0 0x55b95224e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35664/0 (socket says 172.21.15.84:35664) 2024-05-22T23:02:24.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.662+0000 7f59422f6640 1 -- 172.21.15.84:0/1002406669 learned_addr learned my addr 172.21.15.84:0/1002406669 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:24.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.662+0000 7f5941af5640 1 --2- 172.21.15.84:0/1002406669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b952237660 0x55b95224ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.662+0000 7f59422f6640 1 -- 172.21.15.84:0/1002406669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b952246380 msgr2=0x55b95224b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:24.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.662+0000 7f59422f6640 1 --2- 172.21.15.84:0/1002406669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b952246380 0x55b95224b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.662+0000 7f59422f6640 1 -- 172.21.15.84:0/1002406669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b952237660 msgr2=0x55b95224ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:24.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.662+0000 7f59422f6640 1 --2- 172.21.15.84:0/1002406669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b952237660 0x55b95224ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.662+0000 7f59422f6640 1 -- 172.21.15.84:0/1002406669 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b95224bc90 con 0x55b952236cb0 2024-05-22T23:02:24.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.662+0000 7f59422f6640 1 --2- 172.21.15.84:0/1002406669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952236cb0 0x55b95224e550 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f59380c5d70 tx=0x7f5938003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11c0e46ad6d8a73 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:24.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.662+0000 7f5925ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:02:24.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.662+0000 7f5925ffb640 1 -- 172.21.15.84:0/1002406669 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59380d5070 con 0x55b952236cb0 2024-05-22T23:02:24.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.662+0000 7f5925ffb640 1 -- 172.21.15.84:0/1002406669 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f59380bd890 con 0x55b952236cb0 2024-05-22T23:02:24.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.662+0000 7f5947268740 1 -- 172.21.15.84:0/1002406669 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59380bd630 con 0x55b952236cb0 2024-05-22T23:02:24.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.662+0000 7f5925ffb640 1 -- 172.21.15.84:0/1002406669 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f59380bda90 con 0x55b952236cb0 2024-05-22T23:02:24.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.662+0000 7f5947268740 10 client.15468 fetch_fsmap learned FSMap version 18 2024-05-22T23:02:24.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.662+0000 7f5947268740 1 -- 172.21.15.84:0/1002406669 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55b9522bd680 con 0x55b952236cb0 2024-05-22T23:02:24.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.662+0000 7f5925ffb640 1 -- 172.21.15.84:0/1002406669 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59380bd630 con 0x55b952236cb0 2024-05-22T23:02:24.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.663+0000 7f5925ffb640 1 -- 172.21.15.84:0/1002406669 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f5938031ba0 con 0x55b952236cb0 2024-05-22T23:02:24.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.663+0000 7f5947268740 10 client.15468 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:02:24.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.663+0000 7f5947268740 10 client.15468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:02:24.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.663+0000 7f5947268740 20 client.15468 populate_metadata read hostname 'smithi084' 2024-05-22T23:02:24.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.663+0000 7f5947268740 1 --2- 172.21.15.84:0/1002406669 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b952256690 0x55b9522bfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:24.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.663+0000 7f5947268740 4 client.15468 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-22T23:02:24.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.663+0000 7f5947268740 1 -- 172.21.15.84:0/1002406669 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b9521a3830 con 0x55b952256690 2024-05-22T23:02:24.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.663+0000 7f5941af5640 1 --2- 172.21.15.84:0/1002406669 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b952256690 0x55b9522bfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:24.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.663+0000 7f5925ffb640 10 client.15468 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:24.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.663+0000 7f5941af5640 1 --2- 172.21.15.84:0/1002406669 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b952256690 0x55b9522bfbd0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bab089a84d7ea31 server_cookie=529f55e4eb045e8b in_seq=0 out_seq=0 2024-05-22T23:02:24.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.663+0000 7f5925ffb640 1 -- 172.21.15.84:0/1002406669 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b9521a3830 con 0x55b952256690 2024-05-22T23:02:24.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.663+0000 7f5925ffb640 10 client.15468 handle_command_reply: tid=0 2024-05-22T23:02:24.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.663+0000 7f5947268740 1 client.15468 shutdown 2024-05-22T23:02:24.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.663+0000 7f5947268740 1 -- 172.21.15.84:0/1002406669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952236cb0 msgr2=0x55b95224e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:24.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.663+0000 7f5947268740 1 --2- 172.21.15.84:0/1002406669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952236cb0 0x55b95224e550 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f59380c5d70 tx=0x7f5938003f60 comp rx=0 tx=0).stop 2024-05-22T23:02:24.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.663+0000 7f5947268740 1 -- 172.21.15.84:0/1002406669 shutdown_connections 2024-05-22T23:02:24.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.663+0000 7f5947268740 1 --2- 172.21.15.84:0/1002406669 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b952256690 0x55b9522bfbd0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.663+0000 7f5947268740 1 --2- 172.21.15.84:0/1002406669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b952246380 0x55b95224b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.663+0000 7f5947268740 1 --2- 172.21.15.84:0/1002406669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b952237660 0x55b95224ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.663+0000 7f5925ffb640 0 client.15468 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:24.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.663+0000 7f5947268740 1 --2- 172.21.15.84:0/1002406669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b952236cb0 0x55b95224e550 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.663+0000 7f5947268740 1 -- 172.21.15.84:0/1002406669 >> 172.21.15.84:0/1002406669 conn(0x55b9522332a0 msgr2=0x55b952238f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:24.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.664+0000 7f5947268740 1 -- 172.21.15.84:0/1002406669 shutdown_connections 2024-05-22T23:02:24.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.664+0000 7f5947268740 1 -- 172.21.15.84:0/1002406669 wait complete. 2024-05-22T23:02:24.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.664+0000 7f5947268740 20 client.15468 trim_cache size 0 max 16384 2024-05-22T23:02:24.672 INFO:teuthology.orchestra.run.smithi084.stdout:[] 2024-05-22T23:02:24.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.664+0000 7f594455a640 1 -- 172.21.15.84:0/3948445055 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f59280a2e20 msgr2=0x7f59280a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:24.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.664+0000 7f594455a640 1 --2- 172.21.15.84:0/3948445055 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f59280a2e20 0x7f59280a52e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5934020000 tx=0x7f593401f000 comp rx=0 tx=0).stop 2024-05-22T23:02:24.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.664+0000 7f594455a640 1 -- 172.21.15.84:0/3948445055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f593c100740 msgr2=0x7f593c1970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:24.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.664+0000 7f594455a640 1 --2- 172.21.15.84:0/3948445055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f593c100740 0x7f593c1970e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5938009a50 tx=0x7f5938004270 comp rx=0 tx=0).stop 2024-05-22T23:02:24.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.664+0000 7f594455a640 1 -- 172.21.15.84:0/3948445055 shutdown_connections 2024-05-22T23:02:24.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.664+0000 7f594455a640 1 --2- 172.21.15.84:0/3948445055 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f59280a2e20 0x7f59280a52e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.664+0000 7f594455a640 1 --2- 172.21.15.84:0/3948445055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c106750 0x7f593c19b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.664+0000 7f594455a640 1 --2- 172.21.15.84:0/3948445055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f593c101100 0x7f593c197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.664+0000 7f594455a640 1 --2- 172.21.15.84:0/3948445055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f593c100740 0x7f593c1970e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:24.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.664+0000 7f594455a640 1 -- 172.21.15.84:0/3948445055 >> 172.21.15.84:0/3948445055 conn(0x7f593c077870 msgr2=0x7f593c103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:24.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.664+0000 7f594455a640 1 -- 172.21.15.84:0/3948445055 shutdown_connections 2024-05-22T23:02:24.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:24.664+0000 7f594455a640 1 -- 172.21.15.84:0/3948445055 wait complete. 2024-05-22T23:02:25.214 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:24 smithi204 ceph-mon[36211]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:02:25.214 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:24 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3487412453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:02:25.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:24 smithi096 ceph-mon[36392]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:02:25.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:24 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3487412453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:02:25.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:24 smithi084 ceph-mon[32186]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:02:25.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:24 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3487412453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:02:25.678 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-22T23:02:25.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:02:25.678 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-22T23:02:25.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.791+0000 7f51952a5640 1 -- 172.21.15.84:0/1701888546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f519010be90 msgr2=0x7f519010e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:25.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.791+0000 7f51952a5640 1 --2- 172.21.15.84:0/1701888546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f519010be90 0x7f519010e2e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5188009ca0 tx=0x7f518802a950 comp rx=0 tx=0).stop 2024-05-22T23:02:25.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.791+0000 7f51952a5640 1 -- 172.21.15.84:0/1701888546 shutdown_connections 2024-05-22T23:02:25.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.791+0000 7f51952a5640 1 --2- 172.21.15.84:0/1701888546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f519010be90 0x7f519010e2e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.791+0000 7f51952a5640 1 --2- 172.21.15.84:0/1701888546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5190103290 0x7f519010b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.791+0000 7f51952a5640 1 --2- 172.21.15.84:0/1701888546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5190102970 0x7f5190102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.791+0000 7f51952a5640 1 -- 172.21.15.84:0/1701888546 >> 172.21.15.84:0/1701888546 conn(0x7f51900778d0 msgr2=0x7f5190077ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:25.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.791+0000 7f51952a5640 1 -- 172.21.15.84:0/1701888546 shutdown_connections 2024-05-22T23:02:25.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.791+0000 7f51952a5640 1 -- 172.21.15.84:0/1701888546 wait complete. 2024-05-22T23:02:25.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.792+0000 7f51952a5640 1 Processor -- start 2024-05-22T23:02:25.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.792+0000 7f51952a5640 1 -- start start 2024-05-22T23:02:25.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.792+0000 7f51952a5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5190102970 0x7f51901007a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.792+0000 7f51952a5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5190103290 0x7f5190100ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.792+0000 7f51952a5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f519010be90 0x7f5190101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.792+0000 7f51952a5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5190111c30 con 0x7f5190103290 2024-05-22T23:02:25.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.792+0000 7f51952a5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5190111ab0 con 0x7f519010be90 2024-05-22T23:02:25.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.792+0000 7f51952a5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5190111db0 con 0x7f5190102970 2024-05-22T23:02:25.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.792+0000 7f51867fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5190103290 0x7f5190100ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.792+0000 7f51867fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5190103290 0x7f5190100ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35676/0 (socket says 172.21.15.84:35676) 2024-05-22T23:02:25.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.792+0000 7f51867fc640 1 -- 172.21.15.84:0/3878971368 learned_addr learned my addr 172.21.15.84:0/3878971368 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:25.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.793+0000 7f518effd640 1 --2- 172.21.15.84:0/3878971368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5190102970 0x7f51901007a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.793+0000 7f518f7fe640 1 --2- 172.21.15.84:0/3878971368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f519010be90 0x7f5190101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.793+0000 7f51867fc640 1 -- 172.21.15.84:0/3878971368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5190102970 msgr2=0x7f51901007a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:25.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.793+0000 7f51867fc640 1 --2- 172.21.15.84:0/3878971368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5190102970 0x7f51901007a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.793+0000 7f51867fc640 1 -- 172.21.15.84:0/3878971368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f519010be90 msgr2=0x7f5190101220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:25.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.793+0000 7f51867fc640 1 --2- 172.21.15.84:0/3878971368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f519010be90 0x7f5190101220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.793+0000 7f51867fc640 1 -- 172.21.15.84:0/3878971368 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51901a65f0 con 0x7f5190103290 2024-05-22T23:02:25.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.793+0000 7f518f7fe640 1 --2- 172.21.15.84:0/3878971368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f519010be90 0x7f5190101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:02:25.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.793+0000 7f51867fc640 1 --2- 172.21.15.84:0/3878971368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5190103290 0x7f5190100ce0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f517800e9f0 tx=0x7f517800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:25.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.793+0000 7f518cff9640 1 -- 172.21.15.84:0/3878971368 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517800cd80 con 0x7f5190103290 2024-05-22T23:02:25.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.793+0000 7f518cff9640 1 -- 172.21.15.84:0/3878971368 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5178005500 con 0x7f5190103290 2024-05-22T23:02:25.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.793+0000 7f51952a5640 1 -- 172.21.15.84:0/3878971368 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51901a68e0 con 0x7f5190103290 2024-05-22T23:02:25.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.793+0000 7f518cff9640 1 -- 172.21.15.84:0/3878971368 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178010690 con 0x7f5190103290 2024-05-22T23:02:25.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.793+0000 7f51952a5640 1 -- 172.21.15.84:0/3878971368 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51901a6e30 con 0x7f5190103290 2024-05-22T23:02:25.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.796+0000 7f518cff9640 1 -- 172.21.15.84:0/3878971368 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f51780046c0 con 0x7f5190103290 2024-05-22T23:02:25.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.796+0000 7f5197fb1740 1 Processor -- start 2024-05-22T23:02:25.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.796+0000 7f5197fb1740 1 -- start start 2024-05-22T23:02:25.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.797+0000 7f5197fb1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1577c50 0x55dfb1578030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.797+0000 7f5197fb1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb1578570 0x55dfb1582700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.797+0000 7f5197fb1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfb1582c40 0x55dfb1585030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.797+0000 7f5197fb1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55dfb14dba40 con 0x55dfb1577c50 2024-05-22T23:02:25.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.797+0000 7f5197fb1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dfb147a440 con 0x55dfb1582c40 2024-05-22T23:02:25.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.797+0000 7f5197fb1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55dfb14d7a00 con 0x55dfb1578570 2024-05-22T23:02:25.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.797+0000 7f518f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1577c50 0x55dfb1578030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.797+0000 7f518f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1577c50 0x55dfb1578030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35688/0 (socket says 172.21.15.84:35688) 2024-05-22T23:02:25.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.797+0000 7f518f7fe640 1 -- 172.21.15.84:0/3942572286 learned_addr learned my addr 172.21.15.84:0/3942572286 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:25.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.797+0000 7f518effd640 1 --2- 172.21.15.84:0/3942572286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb1578570 0x55dfb1582700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.797+0000 7f51867fc640 1 --2- 172.21.15.84:0/3942572286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfb1582c40 0x55dfb1585030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.797+0000 7f518cff9640 1 --2- 172.21.15.84:0/3878971368 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f517c0a2ea0 0x7f517c0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.797+0000 7f518f7fe640 1 -- 172.21.15.84:0/3942572286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb1578570 msgr2=0x55dfb1582700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:25.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.797+0000 7f518f7fe640 1 --2- 172.21.15.84:0/3942572286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb1578570 0x55dfb1582700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.797+0000 7f518f7fe640 1 -- 172.21.15.84:0/3942572286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfb1582c40 msgr2=0x55dfb1585030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:25.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.797+0000 7f518f7fe640 1 --2- 172.21.15.84:0/3942572286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfb1582c40 0x55dfb1585030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.797+0000 7f518f7fe640 1 -- 172.21.15.84:0/3942572286 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfb14842d0 con 0x55dfb1577c50 2024-05-22T23:02:25.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.797+0000 7f518effd640 1 --2- 172.21.15.84:0/3942572286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb1578570 0x55dfb1582700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:02:25.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.797+0000 7f518cff9640 1 -- 172.21.15.84:0/3878971368 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f51780cfb10 con 0x7f5190103290 2024-05-22T23:02:25.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.797+0000 7f518f7fe640 1 --2- 172.21.15.84:0/3878971368 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f517c0a2ea0 0x7f517c0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.798+0000 7f518f7fe640 1 --2- 172.21.15.84:0/3942572286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1577c50 0x55dfb1578030 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5188009880 tx=0x7f51880098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd8dc59b97230b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:25.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.798+0000 7f51857fa640 1 -- 172.21.15.84:0/3942572286 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5188040040 con 0x55dfb1577c50 2024-05-22T23:02:25.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.798+0000 7f51857fa640 1 -- 172.21.15.84:0/3942572286 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5188003c30 con 0x55dfb1577c50 2024-05-22T23:02:25.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.798+0000 7f51857fa640 1 -- 172.21.15.84:0/3942572286 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5188034070 con 0x55dfb1577c50 2024-05-22T23:02:25.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.798+0000 7f5197fb1740 1 -- 172.21.15.84:0/3942572286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1577c50 msgr2=0x55dfb1578030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:25.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.798+0000 7f5197fb1740 1 --2- 172.21.15.84:0/3942572286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1577c50 0x55dfb1578030 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5188009880 tx=0x7f51880098b0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.798+0000 7f518f7fe640 1 --2- 172.21.15.84:0/3878971368 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f517c0a2ea0 0x7f517c0a5360 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f518804b000 tx=0x7f518804c000 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:25.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.799+0000 7f5197fb1740 1 -- 172.21.15.84:0/3942572286 shutdown_connections 2024-05-22T23:02:25.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.799+0000 7f5197fb1740 1 --2- 172.21.15.84:0/3942572286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfb1582c40 0x55dfb1585030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.799+0000 7f5197fb1740 1 --2- 172.21.15.84:0/3942572286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb1578570 0x55dfb1582700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.799+0000 7f5197fb1740 1 --2- 172.21.15.84:0/3942572286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1577c50 0x55dfb1578030 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.799+0000 7f5197fb1740 1 -- 172.21.15.84:0/3942572286 >> 172.21.15.84:0/3942572286 conn(0x55dfb1573f70 msgr2=0x55dfb1574350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:25.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.799+0000 7f5197fb1740 1 -- 172.21.15.84:0/3942572286 shutdown_connections 2024-05-22T23:02:25.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.799+0000 7f5197fb1740 1 -- 172.21.15.84:0/3942572286 wait complete. 2024-05-22T23:02:25.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.799+0000 7f5197fb1740 1 -- start start 2024-05-22T23:02:25.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.799+0000 7f5197fb1740 1 Processor -- start 2024-05-22T23:02:25.810 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.799+0000 7f5197fb1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb1577c50 0x55dfb157fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.810 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.799+0000 7f5197fb1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1578570 0x55dfb1580150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.810 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.799+0000 7f5197fb1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfb1582c40 0x55dfb1580690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.810 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.799+0000 7f5197fb1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55dfb1585a30 con 0x55dfb1578570 2024-05-22T23:02:25.810 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.799+0000 7f5197fb1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dfb14655f0 con 0x55dfb1582c40 2024-05-22T23:02:25.810 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.799+0000 7f5197fb1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55dfb1585bb0 con 0x55dfb1577c50 2024-05-22T23:02:25.811 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.799+0000 7f5197fb1740 10 client.0 _ll_register_callbacks cb 0x55dfb1460cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:02:25.811 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.799+0000 7f51867fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1578570 0x55dfb1580150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.811 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.799+0000 7f51867fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1578570 0x55dfb1580150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35702/0 (socket says 172.21.15.84:35702) 2024-05-22T23:02:25.811 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.799+0000 7f51867fc640 1 -- 172.21.15.84:0/2519906117 learned_addr learned my addr 172.21.15.84:0/2519906117 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:25.811 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.799+0000 7f518effd640 1 --2- 172.21.15.84:0/2519906117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb1577c50 0x55dfb157fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.811 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.799+0000 7f518f7fe640 1 --2- 172.21.15.84:0/2519906117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfb1582c40 0x55dfb1580690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.811 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.800+0000 7f51867fc640 1 -- 172.21.15.84:0/2519906117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb1577c50 msgr2=0x55dfb157fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:25.811 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.800+0000 7f51867fc640 1 --2- 172.21.15.84:0/2519906117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb1577c50 0x55dfb157fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.800+0000 7f51867fc640 1 -- 172.21.15.84:0/2519906117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfb1582c40 msgr2=0x55dfb1580690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:25.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.800+0000 7f51867fc640 1 --2- 172.21.15.84:0/2519906117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfb1582c40 0x55dfb1580690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.800+0000 7f51867fc640 1 -- 172.21.15.84:0/2519906117 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfb12b77e0 con 0x55dfb1578570 2024-05-22T23:02:25.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.800+0000 7f51867fc640 1 --2- 172.21.15.84:0/2519906117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1578570 0x55dfb1580150 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f51780987b0 tx=0x7f51780d3210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7ce41fd0d7108b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:25.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.800+0000 7f51767fc640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:02:25.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.800+0000 7f51767fc640 1 -- 172.21.15.84:0/2519906117 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51780d4040 con 0x55dfb1578570 2024-05-22T23:02:25.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.800+0000 7f51767fc640 1 -- 172.21.15.84:0/2519906117 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f51780cd3c0 con 0x55dfb1578570 2024-05-22T23:02:25.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.800+0000 7f5197fb1740 1 -- 172.21.15.84:0/2519906117 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5188003c30 con 0x55dfb1578570 2024-05-22T23:02:25.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.800+0000 7f51767fc640 1 -- 172.21.15.84:0/2519906117 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f517809b920 con 0x55dfb1578570 2024-05-22T23:02:25.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.800+0000 7f5197fb1740 10 client.15480 fetch_fsmap learned FSMap version 18 2024-05-22T23:02:25.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.800+0000 7f5197fb1740 1 -- 172.21.15.84:0/2519906117 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55dfb1317f70 con 0x55dfb1578570 2024-05-22T23:02:25.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.800+0000 7f51767fc640 1 -- 172.21.15.84:0/2519906117 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5188003c30 con 0x55dfb1578570 2024-05-22T23:02:25.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.800+0000 7f51767fc640 1 -- 172.21.15.84:0/2519906117 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f517809b230 con 0x55dfb1578570 2024-05-22T23:02:25.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.800+0000 7f5197fb1740 10 client.15480 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:02:25.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.800+0000 7f5197fb1740 10 client.15480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:02:25.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.801+0000 7f5197fb1740 20 client.15480 populate_metadata read hostname 'smithi084' 2024-05-22T23:02:25.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.801+0000 7f5197fb1740 1 --2- 172.21.15.84:0/2519906117 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55dfb1593060 0x55dfb1595450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.801+0000 7f5197fb1740 4 client.15480 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:02:25.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.801+0000 7f5197fb1740 1 -- 172.21.15.84:0/2519906117 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfb14ef460 con 0x55dfb1593060 2024-05-22T23:02:25.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.801+0000 7f518effd640 1 --2- 172.21.15.84:0/2519906117 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55dfb1593060 0x55dfb1595450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.801+0000 7f51767fc640 10 client.15480 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:25.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.801+0000 7f518effd640 1 --2- 172.21.15.84:0/2519906117 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55dfb1593060 0x55dfb1595450 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dd2232066b99233 server_cookie=dd561495076da598 in_seq=0 out_seq=0 2024-05-22T23:02:25.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.802+0000 7f51767fc640 1 -- 172.21.15.84:0/2519906117 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dfb14ef460 con 0x55dfb1593060 2024-05-22T23:02:25.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.802+0000 7f51767fc640 10 client.15480 handle_command_reply: tid=0 2024-05-22T23:02:25.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.802+0000 7f5197fb1740 1 client.15480 shutdown 2024-05-22T23:02:25.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.802+0000 7f5197fb1740 1 -- 172.21.15.84:0/2519906117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1578570 msgr2=0x55dfb1580150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:25.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.802+0000 7f5197fb1740 1 --2- 172.21.15.84:0/2519906117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1578570 0x55dfb1580150 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f51780987b0 tx=0x7f51780d3210 comp rx=0 tx=0).stop 2024-05-22T23:02:25.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.802+0000 7f5197fb1740 1 -- 172.21.15.84:0/2519906117 shutdown_connections 2024-05-22T23:02:25.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.802+0000 7f5197fb1740 1 --2- 172.21.15.84:0/2519906117 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55dfb1593060 0x55dfb1595450 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.802+0000 7f5197fb1740 1 --2- 172.21.15.84:0/2519906117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfb1582c40 0x55dfb1580690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.802+0000 7f5197fb1740 1 --2- 172.21.15.84:0/2519906117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1578570 0x55dfb1580150 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.802+0000 7f5197fb1740 1 --2- 172.21.15.84:0/2519906117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb1577c50 0x55dfb157fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.802+0000 7f51767fc640 0 client.15480 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:25.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.802+0000 7f5197fb1740 1 -- 172.21.15.84:0/2519906117 >> 172.21.15.84:0/2519906117 conn(0x55dfb1573f70 msgr2=0x55dfb1579d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:25.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.802+0000 7f5197fb1740 1 -- 172.21.15.84:0/2519906117 shutdown_connections 2024-05-22T23:02:25.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.802+0000 7f5197fb1740 1 -- 172.21.15.84:0/2519906117 wait complete. 2024-05-22T23:02:25.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.802+0000 7f5197fb1740 20 client.15480 trim_cache size 0 max 16384 2024-05-22T23:02:25.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.819+0000 7f5197fb1740 1 Processor -- start 2024-05-22T23:02:25.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.819+0000 7f5197fb1740 1 -- start start 2024-05-22T23:02:25.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.820+0000 7f5197fb1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb1577c50 0x55dfb158e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.820+0000 7f5197fb1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1578570 0x55dfb15804b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.820+0000 7f5197fb1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfb1582c40 0x55dfb159ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.820+0000 7f5197fb1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f51901ad700 con 0x55dfb1578570 2024-05-22T23:02:25.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.820+0000 7f5197fb1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f51901aff80 con 0x55dfb1582c40 2024-05-22T23:02:25.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.820+0000 7f5197fb1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f51901add70 con 0x55dfb1577c50 2024-05-22T23:02:25.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.820+0000 7f51867fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1578570 0x55dfb15804b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.820+0000 7f51867fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1578570 0x55dfb15804b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35714/0 (socket says 172.21.15.84:35714) 2024-05-22T23:02:25.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.820+0000 7f51867fc640 1 -- 172.21.15.84:0/2416695540 learned_addr learned my addr 172.21.15.84:0/2416695540 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:25.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.820+0000 7f518effd640 1 --2- 172.21.15.84:0/2416695540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb1577c50 0x55dfb158e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.820+0000 7f51867fc640 1 -- 172.21.15.84:0/2416695540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb1577c50 msgr2=0x55dfb158e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:25.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.820+0000 7f51867fc640 1 --2- 172.21.15.84:0/2416695540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb1577c50 0x55dfb158e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.820+0000 7f51867fc640 1 -- 172.21.15.84:0/2416695540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfb1582c40 msgr2=0x55dfb159ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:25.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.820+0000 7f51867fc640 1 --2- 172.21.15.84:0/2416695540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfb1582c40 0x55dfb159ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.820+0000 7f51867fc640 1 -- 172.21.15.84:0/2416695540 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfb15957c0 con 0x55dfb1578570 2024-05-22T23:02:25.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.820+0000 7f518effd640 1 --2- 172.21.15.84:0/2416695540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb1577c50 0x55dfb158e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:02:25.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.820+0000 7f51867fc640 1 --2- 172.21.15.84:0/2416695540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1578570 0x55dfb15804b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f51780a4520 tx=0x7f5178009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ac78184f53f59 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:25.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.820+0000 7f5184ff9640 1 -- 172.21.15.84:0/2416695540 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517809b740 con 0x55dfb1578570 2024-05-22T23:02:25.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.820+0000 7f5184ff9640 1 -- 172.21.15.84:0/2416695540 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51780a9070 con 0x55dfb1578570 2024-05-22T23:02:25.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.820+0000 7f5184ff9640 1 -- 172.21.15.84:0/2416695540 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517809bde0 con 0x55dfb1578570 2024-05-22T23:02:25.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.821+0000 7f5197fb1740 1 -- 172.21.15.84:0/2416695540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1578570 msgr2=0x55dfb15804b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:25.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.821+0000 7f5197fb1740 1 --2- 172.21.15.84:0/2416695540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1578570 0x55dfb15804b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f51780a4520 tx=0x7f5178009310 comp rx=0 tx=0).stop 2024-05-22T23:02:25.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.821+0000 7f5197fb1740 1 -- 172.21.15.84:0/2416695540 shutdown_connections 2024-05-22T23:02:25.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.821+0000 7f5197fb1740 1 --2- 172.21.15.84:0/2416695540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfb1582c40 0x55dfb159ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.821+0000 7f5197fb1740 1 --2- 172.21.15.84:0/2416695540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1578570 0x55dfb15804b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.821+0000 7f5197fb1740 1 --2- 172.21.15.84:0/2416695540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb1577c50 0x55dfb158e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.821+0000 7f5197fb1740 1 -- 172.21.15.84:0/2416695540 >> 172.21.15.84:0/2416695540 conn(0x55dfb1573f70 msgr2=0x55dfb1581cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:25.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.821+0000 7f5197fb1740 1 -- 172.21.15.84:0/2416695540 shutdown_connections 2024-05-22T23:02:25.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.821+0000 7f5197fb1740 1 -- 172.21.15.84:0/2416695540 wait complete. 2024-05-22T23:02:25.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.821+0000 7f5197fb1740 1 -- start start 2024-05-22T23:02:25.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.821+0000 7f5197fb1740 1 Processor -- start 2024-05-22T23:02:25.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f5197fb1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1577c50 0x55dfb1586c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f5197fb1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfb1578570 0x55dfb1587190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f5197fb1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb1582c40 0x55dfb15f8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f5197fb1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f51901ae440 con 0x55dfb1577c50 2024-05-22T23:02:25.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f5197fb1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f51901af8b0 con 0x55dfb1578570 2024-05-22T23:02:25.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f5197fb1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f51901aeb10 con 0x55dfb1582c40 2024-05-22T23:02:25.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f5197fb1740 10 client.0 _ll_register_callbacks cb 0x7f517800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:02:25.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f518effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1577c50 0x55dfb1586c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f518effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1577c50 0x55dfb1586c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35730/0 (socket says 172.21.15.84:35730) 2024-05-22T23:02:25.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f518effd640 1 -- 172.21.15.84:0/2543561506 learned_addr learned my addr 172.21.15.84:0/2543561506 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:25.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f518effd640 1 -- 172.21.15.84:0/2543561506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb1582c40 msgr2=0x55dfb15f8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:25.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f518f7fe640 1 --2- 172.21.15.84:0/2543561506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb1582c40 0x55dfb15f8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f51867fc640 1 --2- 172.21.15.84:0/2543561506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfb1578570 0x55dfb1587190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f518effd640 1 --2- 172.21.15.84:0/2543561506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb1582c40 0x55dfb15f8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f518effd640 1 -- 172.21.15.84:0/2543561506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfb1578570 msgr2=0x55dfb1587190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:25.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f518effd640 1 --2- 172.21.15.84:0/2543561506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfb1578570 0x55dfb1587190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f518effd640 1 -- 172.21.15.84:0/2543561506 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfb11af170 con 0x55dfb1577c50 2024-05-22T23:02:25.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f518effd640 1 --2- 172.21.15.84:0/2543561506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1577c50 0x55dfb1586c50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f518000b890 tx=0x7f518000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8075f5eb7ef2e7e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:25.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f51767fc640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:02:25.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f51767fc640 1 -- 172.21.15.84:0/2543561506 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5180005220 con 0x55dfb1577c50 2024-05-22T23:02:25.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f51767fc640 1 -- 172.21.15.84:0/2543561506 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f51800053c0 con 0x55dfb1577c50 2024-05-22T23:02:25.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f5197fb1740 1 -- 172.21.15.84:0/2543561506 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51780a9070 con 0x55dfb1577c50 2024-05-22T23:02:25.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f51767fc640 1 -- 172.21.15.84:0/2543561506 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f518002f050 con 0x55dfb1577c50 2024-05-22T23:02:25.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f5197fb1740 10 client.15486 fetch_fsmap learned FSMap version 18 2024-05-22T23:02:25.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.822+0000 7f5197fb1740 1 -- 172.21.15.84:0/2543561506 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55dfb14a4930 con 0x55dfb1577c50 2024-05-22T23:02:25.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.823+0000 7f51767fc640 1 -- 172.21.15.84:0/2543561506 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51780a9070 con 0x55dfb1577c50 2024-05-22T23:02:25.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.823+0000 7f51767fc640 1 -- 172.21.15.84:0/2543561506 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f51800203d0 con 0x55dfb1577c50 2024-05-22T23:02:25.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.823+0000 7f5197fb1740 10 client.15486 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:02:25.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.823+0000 7f5197fb1740 10 client.15486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:02:25.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.823+0000 7f5197fb1740 20 client.15486 populate_metadata read hostname 'smithi084' 2024-05-22T23:02:25.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.823+0000 7f5197fb1740 1 --2- 172.21.15.84:0/2543561506 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55dfb15fa0b0 0x55dfb15fc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.823+0000 7f5197fb1740 4 client.15486 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-22T23:02:25.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.823+0000 7f5197fb1740 1 -- 172.21.15.84:0/2543561506 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dfb14e9170 con 0x55dfb15fa0b0 2024-05-22T23:02:25.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.823+0000 7f51867fc640 1 --2- 172.21.15.84:0/2543561506 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55dfb15fa0b0 0x55dfb15fc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.823+0000 7f51767fc640 10 client.15486 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:25.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.823+0000 7f51867fc640 1 --2- 172.21.15.84:0/2543561506 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55dfb15fa0b0 0x55dfb15fc4a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eee082b7efe6984 server_cookie=552319f0926ceef2 in_seq=0 out_seq=0 2024-05-22T23:02:25.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.825+0000 7f51767fc640 1 -- 172.21.15.84:0/2543561506 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dfb14e9170 con 0x55dfb15fa0b0 2024-05-22T23:02:25.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.825+0000 7f51767fc640 10 client.15486 handle_command_reply: tid=0 2024-05-22T23:02:25.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.825+0000 7f5197fb1740 1 client.15486 shutdown 2024-05-22T23:02:25.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.825+0000 7f5197fb1740 1 -- 172.21.15.84:0/2543561506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1577c50 msgr2=0x55dfb1586c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:25.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.825+0000 7f5197fb1740 1 --2- 172.21.15.84:0/2543561506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1577c50 0x55dfb1586c50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f518000b890 tx=0x7f518000bf90 comp rx=0 tx=0).stop 2024-05-22T23:02:25.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.825+0000 7f5197fb1740 1 -- 172.21.15.84:0/2543561506 shutdown_connections 2024-05-22T23:02:25.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.825+0000 7f5197fb1740 1 --2- 172.21.15.84:0/2543561506 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55dfb15fa0b0 0x55dfb15fc4a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.825+0000 7f5197fb1740 1 --2- 172.21.15.84:0/2543561506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb1582c40 0x55dfb15f8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.825+0000 7f51767fc640 0 client.15486 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:25.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.825+0000 7f5197fb1740 1 --2- 172.21.15.84:0/2543561506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfb1578570 0x55dfb1587190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.825+0000 7f5197fb1740 1 --2- 172.21.15.84:0/2543561506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dfb1577c50 0x55dfb1586c50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.825+0000 7f5197fb1740 1 -- 172.21.15.84:0/2543561506 >> 172.21.15.84:0/2543561506 conn(0x55dfb1573f70 msgr2=0x55dfb1572920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:25.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.825+0000 7f5197fb1740 1 -- 172.21.15.84:0/2543561506 shutdown_connections 2024-05-22T23:02:25.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.825+0000 7f5197fb1740 1 -- 172.21.15.84:0/2543561506 wait complete. 2024-05-22T23:02:25.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.825+0000 7f5197fb1740 20 client.15486 trim_cache size 0 max 16384 2024-05-22T23:02:25.834 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:02:25.834 INFO:teuthology.orchestra.run.smithi084.stdout: "return_code": 0, 2024-05-22T23:02:25.834 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_tag": "4e5b235b-7344-49eb-bed0-3e7acdec755a", 2024-05-22T23:02:25.834 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": "asynchronous" 2024-05-22T23:02:25.834 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:02:25.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.826+0000 7f51952a5640 1 -- 172.21.15.84:0/3878971368 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f517c0a2ea0 msgr2=0x7f517c0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:25.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.826+0000 7f51952a5640 1 --2- 172.21.15.84:0/3878971368 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f517c0a2ea0 0x7f517c0a5360 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f518804b000 tx=0x7f518804c000 comp rx=0 tx=0).stop 2024-05-22T23:02:25.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.826+0000 7f51952a5640 1 -- 172.21.15.84:0/3878971368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5190103290 msgr2=0x7f5190100ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:25.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.826+0000 7f51952a5640 1 --2- 172.21.15.84:0/3878971368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5190103290 0x7f5190100ce0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f517800e9f0 tx=0x7f517800edf0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.826+0000 7f51952a5640 1 -- 172.21.15.84:0/3878971368 shutdown_connections 2024-05-22T23:02:25.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.826+0000 7f51952a5640 1 --2- 172.21.15.84:0/3878971368 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f517c0a2ea0 0x7f517c0a5360 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.826+0000 7f51952a5640 1 --2- 172.21.15.84:0/3878971368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f519010be90 0x7f5190101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.826+0000 7f51952a5640 1 --2- 172.21.15.84:0/3878971368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5190103290 0x7f5190100ce0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.826+0000 7f51952a5640 1 --2- 172.21.15.84:0/3878971368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5190102970 0x7f51901007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.826+0000 7f51952a5640 1 -- 172.21.15.84:0/3878971368 >> 172.21.15.84:0/3878971368 conn(0x7f51900778d0 msgr2=0x7f51900fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:25.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.826+0000 7f51952a5640 1 -- 172.21.15.84:0/3878971368 shutdown_connections 2024-05-22T23:02:25.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.826+0000 7f51952a5640 1 -- 172.21.15.84:0/3878971368 wait complete. 2024-05-22T23:02:25.839 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:02:25.839 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:02:25.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.942+0000 7fd61236a640 1 -- 172.21.15.84:0/3916367900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd60c101100 msgr2=0x7fd60c10d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:25.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.942+0000 7fd61236a640 1 --2- 172.21.15.84:0/3916367900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd60c101100 0x7fd60c10d610 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd604009cd0 tx=0x7fd60402a8e0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.942+0000 7fd61236a640 1 -- 172.21.15.84:0/3916367900 shutdown_connections 2024-05-22T23:02:25.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.942+0000 7fd61236a640 1 --2- 172.21.15.84:0/3916367900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd60c101100 0x7fd60c10d610 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.942+0000 7fd61236a640 1 --2- 172.21.15.84:0/3916367900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd60c100740 0x7fd60c100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.942+0000 7fd61236a640 1 --2- 172.21.15.84:0/3916367900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd60c106750 0x7fd60c106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.942+0000 7fd61236a640 1 -- 172.21.15.84:0/3916367900 >> 172.21.15.84:0/3916367900 conn(0x7fd60c077870 msgr2=0x7fd60c077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:25.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.942+0000 7fd61236a640 1 -- 172.21.15.84:0/3916367900 shutdown_connections 2024-05-22T23:02:25.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.942+0000 7fd61236a640 1 -- 172.21.15.84:0/3916367900 wait complete. 2024-05-22T23:02:25.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.942+0000 7fd61236a640 1 Processor -- start 2024-05-22T23:02:25.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.942+0000 7fd61236a640 1 -- start start 2024-05-22T23:02:25.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.942+0000 7fd61236a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd60c100740 0x7fd60c197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.942+0000 7fd61236a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd60c101100 0x7fd60c1975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.942+0000 7fd61236a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd60c106750 0x7fd60c19b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.943+0000 7fd61236a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c110b60 con 0x7fd60c101100 2024-05-22T23:02:25.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.943+0000 7fd61236a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c1109e0 con 0x7fd60c106750 2024-05-22T23:02:25.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.943+0000 7fd61236a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c110ce0 con 0x7fd60c100740 2024-05-22T23:02:25.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.943+0000 7fd603fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd60c101100 0x7fd60c1975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.943+0000 7fd603fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd60c101100 0x7fd60c1975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35758/0 (socket says 172.21.15.84:35758) 2024-05-22T23:02:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.943+0000 7fd603fff640 1 -- 172.21.15.84:0/1677592854 learned_addr learned my addr 172.21.15.84:0/1677592854 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.943+0000 7fd60bfff640 1 --2- 172.21.15.84:0/1677592854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd60c100740 0x7fd60c197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.943+0000 7fd603fff640 1 -- 172.21.15.84:0/1677592854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd60c100740 msgr2=0x7fd60c197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.943+0000 7fd603fff640 1 --2- 172.21.15.84:0/1677592854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd60c100740 0x7fd60c197060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.943+0000 7fd610907640 1 --2- 172.21.15.84:0/1677592854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd60c106750 0x7fd60c19b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.943+0000 7fd603fff640 1 -- 172.21.15.84:0/1677592854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd60c106750 msgr2=0x7fd60c19b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.943+0000 7fd603fff640 1 --2- 172.21.15.84:0/1677592854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd60c106750 0x7fd60c19b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.943+0000 7fd603fff640 1 -- 172.21.15.84:0/1677592854 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60c19c050 con 0x7fd60c101100 2024-05-22T23:02:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.943+0000 7fd603fff640 1 --2- 172.21.15.84:0/1677592854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd60c101100 0x7fd60c1975a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400c890 tx=0x7fd5f400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.943+0000 7fd609ffb640 1 -- 172.21.15.84:0/1677592854 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4004220 con 0x7fd60c101100 2024-05-22T23:02:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.943+0000 7fd609ffb640 1 -- 172.21.15.84:0/1677592854 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5f40043c0 con 0x7fd60c101100 2024-05-22T23:02:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.943+0000 7fd61236a640 1 -- 172.21.15.84:0/1677592854 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd60c19c340 con 0x7fd60c101100 2024-05-22T23:02:25.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.943+0000 7fd609ffb640 1 -- 172.21.15.84:0/1677592854 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f40049e0 con 0x7fd60c101100 2024-05-22T23:02:25.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.943+0000 7fd61236a640 1 -- 172.21.15.84:0/1677592854 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd60c1a3bd0 con 0x7fd60c101100 2024-05-22T23:02:25.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.945+0000 7fd609ffb640 1 -- 172.21.15.84:0/1677592854 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7fd5f40079b0 con 0x7fd60c101100 2024-05-22T23:02:25.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.946+0000 7fd615071740 1 Processor -- start 2024-05-22T23:02:25.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.946+0000 7fd615071740 1 -- start start 2024-05-22T23:02:25.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.946+0000 7fd615071740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea19ebd4a0 0x55ea19ebb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.946+0000 7fd615071740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ebbdb0 0x55ea19ebc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.946+0000 7fd615071740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea19ec8d40 0x55ea19ecb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.946+0000 7fd615071740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ea19e1fdb0 con 0x55ea19ebbdb0 2024-05-22T23:02:25.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.946+0000 7fd615071740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ea19dbe360 con 0x55ea19ec8d40 2024-05-22T23:02:25.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.946+0000 7fd615071740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ea19e1bd70 con 0x55ea19ebd4a0 2024-05-22T23:02:25.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.946+0000 7fd609ffb640 1 --2- 172.21.15.84:0/1677592854 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd5f80a2e20 0x7fd5f80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.946+0000 7fd60bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ebbdb0 0x55ea19ebc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.946+0000 7fd60bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ebbdb0 0x55ea19ebc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35772/0 (socket says 172.21.15.84:35772) 2024-05-22T23:02:25.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.946+0000 7fd60bfff640 1 -- 172.21.15.84:0/139367590 learned_addr learned my addr 172.21.15.84:0/139367590 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:25.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.946+0000 7fd610907640 1 --2- 172.21.15.84:0/1677592854 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd5f80a2e20 0x7fd5f80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.946+0000 7fd610907640 1 --2- 172.21.15.84:0/139367590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea19ebd4a0 0x55ea19ebb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.946+0000 7fd609ffb640 1 -- 172.21.15.84:0/1677592854 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fd5f40cf820 con 0x7fd60c101100 2024-05-22T23:02:25.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.946+0000 7fd603fff640 1 --2- 172.21.15.84:0/139367590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea19ec8d40 0x55ea19ecb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.946+0000 7fd60bfff640 1 -- 172.21.15.84:0/139367590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea19ebd4a0 msgr2=0x55ea19ebb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:25.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.946+0000 7fd60bfff640 1 --2- 172.21.15.84:0/139367590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea19ebd4a0 0x55ea19ebb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.946+0000 7fd60bfff640 1 -- 172.21.15.84:0/139367590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea19ec8d40 msgr2=0x55ea19ecb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:25.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.946+0000 7fd60bfff640 1 --2- 172.21.15.84:0/139367590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea19ec8d40 0x55ea19ecb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.946+0000 7fd60bfff640 1 -- 172.21.15.84:0/139367590 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea19de1a60 con 0x55ea19ebbdb0 2024-05-22T23:02:25.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.946+0000 7fd603fff640 1 --2- 172.21.15.84:0/139367590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea19ec8d40 0x55ea19ecb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:02:25.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.947+0000 7fd60bfff640 1 --2- 172.21.15.84:0/139367590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ebbdb0 0x55ea19ebc230 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00cb40 tx=0x7fd5fc00e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383cfdfaf799988a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:25.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.947+0000 7fd610907640 1 --2- 172.21.15.84:0/1677592854 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd5f80a2e20 0x7fd5f80a52e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd604006010 tx=0x7fd604005c50 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:25.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.947+0000 7fd6027fc640 1 -- 172.21.15.84:0/139367590 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc010430 con 0x55ea19ebbdb0 2024-05-22T23:02:25.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.947+0000 7fd6027fc640 1 -- 172.21.15.84:0/139367590 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5fc010ab0 con 0x55ea19ebbdb0 2024-05-22T23:02:25.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.947+0000 7fd6027fc640 1 -- 172.21.15.84:0/139367590 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc00fbe0 con 0x55ea19ebbdb0 2024-05-22T23:02:25.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.947+0000 7fd615071740 1 -- 172.21.15.84:0/139367590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ebbdb0 msgr2=0x55ea19ebc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:25.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.947+0000 7fd615071740 1 --2- 172.21.15.84:0/139367590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ebbdb0 0x55ea19ebc230 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00cb40 tx=0x7fd5fc00e350 comp rx=0 tx=0).stop 2024-05-22T23:02:25.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.947+0000 7fd615071740 1 -- 172.21.15.84:0/139367590 shutdown_connections 2024-05-22T23:02:25.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.947+0000 7fd615071740 1 --2- 172.21.15.84:0/139367590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea19ec8d40 0x55ea19ecb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.947+0000 7fd615071740 1 --2- 172.21.15.84:0/139367590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ebbdb0 0x55ea19ebc230 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.947+0000 7fd615071740 1 --2- 172.21.15.84:0/139367590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea19ebd4a0 0x55ea19ebb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.947+0000 7fd615071740 1 -- 172.21.15.84:0/139367590 >> 172.21.15.84:0/139367590 conn(0x55ea19e848e0 msgr2=0x55ea19eb8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:25.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.947+0000 7fd615071740 1 -- 172.21.15.84:0/139367590 shutdown_connections 2024-05-22T23:02:25.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.947+0000 7fd615071740 1 -- 172.21.15.84:0/139367590 wait complete. 2024-05-22T23:02:25.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.948+0000 7fd615071740 1 -- start start 2024-05-22T23:02:25.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.948+0000 7fd615071740 1 Processor -- start 2024-05-22T23:02:25.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.948+0000 7fd615071740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea19ebbdb0 0x55ea19ec5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.948+0000 7fd615071740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ebd4a0 0x55ea19ec61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.948+0000 7fd615071740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea19ec8d40 0x55ea19ec66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.948+0000 7fd615071740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ea19ecbbd0 con 0x55ea19ebd4a0 2024-05-22T23:02:25.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.948+0000 7fd615071740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ea19dda0f0 con 0x55ea19ebbdb0 2024-05-22T23:02:25.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.948+0000 7fd615071740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ea19ecbd50 con 0x55ea19ec8d40 2024-05-22T23:02:25.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.948+0000 7fd615071740 10 client.0 _ll_register_callbacks cb 0x55ea19ddef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:02:25.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.948+0000 7fd603fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ebd4a0 0x55ea19ec61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.948+0000 7fd603fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ebd4a0 0x55ea19ec61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35778/0 (socket says 172.21.15.84:35778) 2024-05-22T23:02:25.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.948+0000 7fd603fff640 1 -- 172.21.15.84:0/1002790879 learned_addr learned my addr 172.21.15.84:0/1002790879 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:25.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.948+0000 7fd60bfff640 1 --2- 172.21.15.84:0/1002790879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea19ebbdb0 0x55ea19ec5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.948+0000 7fd610907640 1 --2- 172.21.15.84:0/1002790879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea19ec8d40 0x55ea19ec66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.949+0000 7fd603fff640 1 -- 172.21.15.84:0/1002790879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea19ec8d40 msgr2=0x55ea19ec66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:25.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.949+0000 7fd603fff640 1 --2- 172.21.15.84:0/1002790879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea19ec8d40 0x55ea19ec66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.949+0000 7fd603fff640 1 -- 172.21.15.84:0/1002790879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea19ebbdb0 msgr2=0x55ea19ec5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:25.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.949+0000 7fd603fff640 1 --2- 172.21.15.84:0/1002790879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea19ebbdb0 0x55ea19ec5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.949+0000 7fd603fff640 1 -- 172.21.15.84:0/1002790879 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea19bfd7d0 con 0x55ea19ebd4a0 2024-05-22T23:02:25.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.949+0000 7fd60bfff640 1 --2- 172.21.15.84:0/1002790879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea19ebbdb0 0x55ea19ec5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:02:25.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.949+0000 7fd603fff640 1 --2- 172.21.15.84:0/1002790879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ebd4a0 0x55ea19ec61b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4005fd0 tx=0x7fd5f409b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517bef08f9260dc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:25.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.949+0000 7fd5eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:02:25.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.949+0000 7fd5eb7fe640 1 -- 172.21.15.84:0/1002790879 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f401d040 con 0x55ea19ebd4a0 2024-05-22T23:02:25.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.949+0000 7fd5eb7fe640 1 -- 172.21.15.84:0/1002790879 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fd5f40a44d0 con 0x55ea19ebd4a0 2024-05-22T23:02:25.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.949+0000 7fd615071740 1 -- 172.21.15.84:0/1002790879 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5fc010ab0 con 0x55ea19ebd4a0 2024-05-22T23:02:25.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.949+0000 7fd5eb7fe640 1 -- 172.21.15.84:0/1002790879 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fd5f40a4760 con 0x55ea19ebd4a0 2024-05-22T23:02:25.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.949+0000 7fd615071740 10 client.15498 fetch_fsmap learned FSMap version 18 2024-05-22T23:02:25.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.949+0000 7fd615071740 1 -- 172.21.15.84:0/1002790879 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55ea19b333f0 con 0x55ea19ebd4a0 2024-05-22T23:02:25.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.949+0000 7fd5eb7fe640 1 -- 172.21.15.84:0/1002790879 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5fc010ab0 con 0x55ea19ebd4a0 2024-05-22T23:02:25.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.949+0000 7fd5eb7fe640 1 -- 172.21.15.84:0/1002790879 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fd5f409b0d0 con 0x55ea19ebd4a0 2024-05-22T23:02:25.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.949+0000 7fd615071740 10 client.15498 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:02:25.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.949+0000 7fd615071740 10 client.15498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:02:25.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.949+0000 7fd615071740 20 client.15498 populate_metadata read hostname 'smithi084' 2024-05-22T23:02:25.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.949+0000 7fd615071740 1 --2- 172.21.15.84:0/1002790879 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55ea19ed9140 0x55ea19edb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.949+0000 7fd615071740 4 client.15498 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:02:25.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.949+0000 7fd615071740 1 -- 172.21.15.84:0/1002790879 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea19e2d4e0 con 0x55ea19ed9140 2024-05-22T23:02:25.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.950+0000 7fd60bfff640 1 --2- 172.21.15.84:0/1002790879 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55ea19ed9140 0x55ea19edb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.950+0000 7fd5eb7fe640 10 client.15498 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:25.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.950+0000 7fd60bfff640 1 --2- 172.21.15.84:0/1002790879 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55ea19ed9140 0x55ea19edb530 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70275f67913545b4 server_cookie=6225bde9af190eea in_seq=0 out_seq=0 2024-05-22T23:02:25.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.951+0000 7fd5eb7fe640 1 -- 172.21.15.84:0/1002790879 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ea19e2d4e0 con 0x55ea19ed9140 2024-05-22T23:02:25.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.951+0000 7fd5eb7fe640 10 client.15498 handle_command_reply: tid=0 2024-05-22T23:02:25.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.951+0000 7fd615071740 1 client.15498 shutdown 2024-05-22T23:02:25.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.951+0000 7fd615071740 1 -- 172.21.15.84:0/1002790879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ebd4a0 msgr2=0x55ea19ec61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:25.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.951+0000 7fd615071740 1 --2- 172.21.15.84:0/1002790879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ebd4a0 0x55ea19ec61b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4005fd0 tx=0x7fd5f409b510 comp rx=0 tx=0).stop 2024-05-22T23:02:25.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.951+0000 7fd615071740 1 -- 172.21.15.84:0/1002790879 shutdown_connections 2024-05-22T23:02:25.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.951+0000 7fd615071740 1 --2- 172.21.15.84:0/1002790879 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55ea19ed9140 0x55ea19edb530 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.951+0000 7fd615071740 1 --2- 172.21.15.84:0/1002790879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea19ec8d40 0x55ea19ec66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.951+0000 7fd615071740 1 --2- 172.21.15.84:0/1002790879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ebd4a0 0x55ea19ec61b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.951+0000 7fd5eb7fe640 0 client.15498 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:25.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.951+0000 7fd615071740 1 --2- 172.21.15.84:0/1002790879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea19ebbdb0 0x55ea19ec5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.951+0000 7fd615071740 1 -- 172.21.15.84:0/1002790879 >> 172.21.15.84:0/1002790879 conn(0x55ea19e848e0 msgr2=0x55ea19ebe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:25.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.951+0000 7fd615071740 1 -- 172.21.15.84:0/1002790879 shutdown_connections 2024-05-22T23:02:25.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.951+0000 7fd615071740 1 -- 172.21.15.84:0/1002790879 wait complete. 2024-05-22T23:02:25.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.951+0000 7fd615071740 20 client.15498 trim_cache size 0 max 16384 2024-05-22T23:02:25.972 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.968+0000 7fd615071740 1 Processor -- start 2024-05-22T23:02:25.972 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.968+0000 7fd615071740 1 -- start start 2024-05-22T23:02:25.972 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.968+0000 7fd615071740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea19ebbdb0 0x55ea19ed4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.972 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.968+0000 7fd615071740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea19ebd4a0 0x55ea19ee0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.973 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.968+0000 7fd615071740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ec8d40 0x55ea19ee1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.973 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.968+0000 7fd615071740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c1a9320 con 0x55ea19ec8d40 2024-05-22T23:02:25.973 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.968+0000 7fd615071740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c1abba0 con 0x55ea19ebbdb0 2024-05-22T23:02:25.973 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.968+0000 7fd615071740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c1a9990 con 0x55ea19ebd4a0 2024-05-22T23:02:25.973 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.969+0000 7fd610907640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ec8d40 0x55ea19ee1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.973 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.969+0000 7fd610907640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ec8d40 0x55ea19ee1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35788/0 (socket says 172.21.15.84:35788) 2024-05-22T23:02:25.973 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.969+0000 7fd610907640 1 -- 172.21.15.84:0/1524337451 learned_addr learned my addr 172.21.15.84:0/1524337451 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:25.973 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.969+0000 7fd610907640 1 -- 172.21.15.84:0/1524337451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea19ebd4a0 msgr2=0x55ea19ee0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:25.973 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.969+0000 7fd603fff640 1 --2- 172.21.15.84:0/1524337451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea19ebd4a0 0x55ea19ee0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.973 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.969+0000 7fd60bfff640 1 --2- 172.21.15.84:0/1524337451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea19ebbdb0 0x55ea19ed4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.973 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.969+0000 7fd610907640 1 --2- 172.21.15.84:0/1524337451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea19ebd4a0 0x55ea19ee0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.973 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.969+0000 7fd610907640 1 -- 172.21.15.84:0/1524337451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea19ebbdb0 msgr2=0x55ea19ed4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:25.973 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.969+0000 7fd610907640 1 --2- 172.21.15.84:0/1524337451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea19ebbdb0 0x55ea19ed4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.974 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.969+0000 7fd610907640 1 -- 172.21.15.84:0/1524337451 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea19af30f0 con 0x55ea19ec8d40 2024-05-22T23:02:25.974 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.969+0000 7fd60bfff640 1 --2- 172.21.15.84:0/1524337451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea19ebbdb0 0x55ea19ed4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:02:25.974 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.969+0000 7fd610907640 1 --2- 172.21.15.84:0/1524337451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ec8d40 0x55ea19ee1320 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd604035000 tx=0x7fd604040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6e670756a34f8b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:25.974 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.969+0000 7fd601ffb640 1 -- 172.21.15.84:0/1524337451 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604003bf0 con 0x55ea19ec8d40 2024-05-22T23:02:25.974 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.969+0000 7fd601ffb640 1 -- 172.21.15.84:0/1524337451 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd604036040 con 0x55ea19ec8d40 2024-05-22T23:02:25.974 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.969+0000 7fd601ffb640 1 -- 172.21.15.84:0/1524337451 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604030a00 con 0x55ea19ec8d40 2024-05-22T23:02:25.974 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.970+0000 7fd615071740 1 -- 172.21.15.84:0/1524337451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ec8d40 msgr2=0x55ea19ee1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:25.974 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.970+0000 7fd615071740 1 --2- 172.21.15.84:0/1524337451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ec8d40 0x55ea19ee1320 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd604035000 tx=0x7fd604040000 comp rx=0 tx=0).stop 2024-05-22T23:02:25.974 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.970+0000 7fd615071740 1 -- 172.21.15.84:0/1524337451 shutdown_connections 2024-05-22T23:02:25.974 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.970+0000 7fd615071740 1 --2- 172.21.15.84:0/1524337451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ec8d40 0x55ea19ee1320 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.974 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.970+0000 7fd615071740 1 --2- 172.21.15.84:0/1524337451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea19ebd4a0 0x55ea19ee0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.974 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.970+0000 7fd615071740 1 --2- 172.21.15.84:0/1524337451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea19ebbdb0 0x55ea19ed4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.974 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.970+0000 7fd615071740 1 -- 172.21.15.84:0/1524337451 >> 172.21.15.84:0/1524337451 conn(0x55ea19e848e0 msgr2=0x55ea19ec7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:25.975 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.970+0000 7fd615071740 1 -- 172.21.15.84:0/1524337451 shutdown_connections 2024-05-22T23:02:25.975 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.970+0000 7fd615071740 1 -- 172.21.15.84:0/1524337451 wait complete. 2024-05-22T23:02:25.975 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.970+0000 7fd615071740 1 -- start start 2024-05-22T23:02:25.975 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.970+0000 7fd615071740 1 Processor -- start 2024-05-22T23:02:25.978 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.971+0000 7fd615071740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea19ebbdb0 0x55ea19ec9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.978 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.971+0000 7fd615071740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea19ebd4a0 0x55ea19eca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.978 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.971+0000 7fd615071740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ec8d40 0x55ea19ecf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.978 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.971+0000 7fd615071740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c1aa060 con 0x55ea19ec8d40 2024-05-22T23:02:25.978 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.971+0000 7fd615071740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c1ab4d0 con 0x55ea19ebd4a0 2024-05-22T23:02:25.978 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.971+0000 7fd615071740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c1aa730 con 0x55ea19ebbdb0 2024-05-22T23:02:25.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.971+0000 7fd615071740 10 client.0 _ll_register_callbacks cb 0x7fd5f40057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:02:25.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.971+0000 7fd610907640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ec8d40 0x55ea19ecf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.971+0000 7fd610907640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ec8d40 0x55ea19ecf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35802/0 (socket says 172.21.15.84:35802) 2024-05-22T23:02:25.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.971+0000 7fd610907640 1 -- 172.21.15.84:0/588802367 learned_addr learned my addr 172.21.15.84:0/588802367 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:25.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.971+0000 7fd60bfff640 1 --2- 172.21.15.84:0/588802367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea19ebbdb0 0x55ea19ec9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.971+0000 7fd603fff640 1 --2- 172.21.15.84:0/588802367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea19ebd4a0 0x55ea19eca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.971+0000 7fd610907640 1 -- 172.21.15.84:0/588802367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea19ebbdb0 msgr2=0x55ea19ec9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:25.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.971+0000 7fd610907640 1 --2- 172.21.15.84:0/588802367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea19ebbdb0 0x55ea19ec9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.971+0000 7fd610907640 1 -- 172.21.15.84:0/588802367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea19ebd4a0 msgr2=0x55ea19eca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:25.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.971+0000 7fd610907640 1 --2- 172.21.15.84:0/588802367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea19ebd4a0 0x55ea19eca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.971+0000 7fd610907640 1 -- 172.21.15.84:0/588802367 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea19ddefe0 con 0x55ea19ec8d40 2024-05-22T23:02:25.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.971+0000 7fd603fff640 1 --2- 172.21.15.84:0/588802367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea19ebd4a0 0x55ea19eca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:02:25.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.971+0000 7fd610907640 1 --2- 172.21.15.84:0/588802367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ec8d40 0x55ea19ecf620 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd6040306a0 tx=0x7fd604009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5beb4349757174a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:25.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.971+0000 7fd5eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:02:25.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.971+0000 7fd5eb7fe640 1 -- 172.21.15.84:0/588802367 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604060060 con 0x55ea19ec8d40 2024-05-22T23:02:25.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.971+0000 7fd5eb7fe640 1 -- 172.21.15.84:0/588802367 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fd604004250 con 0x55ea19ec8d40 2024-05-22T23:02:25.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.971+0000 7fd615071740 1 -- 172.21.15.84:0/588802367 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd604036040 con 0x55ea19ec8d40 2024-05-22T23:02:25.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.971+0000 7fd5eb7fe640 1 -- 172.21.15.84:0/588802367 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fd604004450 con 0x55ea19ec8d40 2024-05-22T23:02:25.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.972+0000 7fd615071740 10 client.15504 fetch_fsmap learned FSMap version 18 2024-05-22T23:02:25.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.972+0000 7fd615071740 1 -- 172.21.15.84:0/588802367 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55ea19db01c0 con 0x55ea19ec8d40 2024-05-22T23:02:25.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.972+0000 7fd5eb7fe640 1 -- 172.21.15.84:0/588802367 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd604036040 con 0x55ea19ec8d40 2024-05-22T23:02:25.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.972+0000 7fd5eb7fe640 1 -- 172.21.15.84:0/588802367 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fd604045040 con 0x55ea19ec8d40 2024-05-22T23:02:25.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.972+0000 7fd615071740 10 client.15504 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:02:25.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.972+0000 7fd615071740 10 client.15504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:02:25.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.972+0000 7fd615071740 20 client.15504 populate_metadata read hostname 'smithi084' 2024-05-22T23:02:25.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.972+0000 7fd615071740 1 --2- 172.21.15.84:0/588802367 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55ea19ed9140 0x55ea19f42190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:25.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.972+0000 7fd615071740 4 client.15504 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:02:25.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.972+0000 7fd615071740 1 -- 172.21.15.84:0/588802367 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ea19e27830 con 0x55ea19ed9140 2024-05-22T23:02:25.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.972+0000 7fd60bfff640 1 --2- 172.21.15.84:0/588802367 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55ea19ed9140 0x55ea19f42190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:25.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.973+0000 7fd5eb7fe640 10 client.15504 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:25.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.973+0000 7fd60bfff640 1 --2- 172.21.15.84:0/588802367 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55ea19ed9140 0x55ea19f42190 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceff90211ca3f15b server_cookie=1b75b1fb2a198305 in_seq=0 out_seq=0 2024-05-22T23:02:25.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.973+0000 7fd5eb7fe640 1 -- 172.21.15.84:0/588802367 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ea19e27830 con 0x55ea19ed9140 2024-05-22T23:02:25.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.973+0000 7fd5eb7fe640 10 client.15504 handle_command_reply: tid=0 2024-05-22T23:02:25.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.973+0000 7fd615071740 1 client.15504 shutdown 2024-05-22T23:02:25.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.973+0000 7fd615071740 1 -- 172.21.15.84:0/588802367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ec8d40 msgr2=0x55ea19ecf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:25.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.973+0000 7fd615071740 1 --2- 172.21.15.84:0/588802367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ec8d40 0x55ea19ecf620 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd6040306a0 tx=0x7fd604009a10 comp rx=0 tx=0).stop 2024-05-22T23:02:25.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.973+0000 7fd615071740 1 -- 172.21.15.84:0/588802367 shutdown_connections 2024-05-22T23:02:25.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.973+0000 7fd615071740 1 --2- 172.21.15.84:0/588802367 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55ea19ed9140 0x55ea19f42190 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.973+0000 7fd615071740 1 --2- 172.21.15.84:0/588802367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea19ec8d40 0x55ea19ecf620 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.973+0000 7fd615071740 1 --2- 172.21.15.84:0/588802367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea19ebd4a0 0x55ea19eca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.973+0000 7fd615071740 1 --2- 172.21.15.84:0/588802367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea19ebbdb0 0x55ea19ec9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.973+0000 7fd5eb7fe640 0 client.15504 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:25.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.973+0000 7fd615071740 1 -- 172.21.15.84:0/588802367 >> 172.21.15.84:0/588802367 conn(0x55ea19e848e0 msgr2=0x55ea19ec5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:25.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.973+0000 7fd615071740 1 -- 172.21.15.84:0/588802367 shutdown_connections 2024-05-22T23:02:25.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.973+0000 7fd615071740 1 -- 172.21.15.84:0/588802367 wait complete. 2024-05-22T23:02:25.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.973+0000 7fd615071740 20 client.15504 trim_cache size 0 max 16384 2024-05-22T23:02:25.982 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:02:25.982 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "no active scrubs running", 2024-05-22T23:02:25.982 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": {} 2024-05-22T23:02:25.982 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:02:25.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.974+0000 7fd61236a640 1 -- 172.21.15.84:0/1677592854 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd5f80a2e20 msgr2=0x7fd5f80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:25.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.974+0000 7fd61236a640 1 --2- 172.21.15.84:0/1677592854 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd5f80a2e20 0x7fd5f80a52e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd604006010 tx=0x7fd604005c50 comp rx=0 tx=0).stop 2024-05-22T23:02:25.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.974+0000 7fd61236a640 1 -- 172.21.15.84:0/1677592854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd60c101100 msgr2=0x7fd60c1975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:25.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.974+0000 7fd61236a640 1 --2- 172.21.15.84:0/1677592854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd60c101100 0x7fd60c1975a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400c890 tx=0x7fd5f400cd50 comp rx=0 tx=0).stop 2024-05-22T23:02:25.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.974+0000 7fd61236a640 1 -- 172.21.15.84:0/1677592854 shutdown_connections 2024-05-22T23:02:25.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.974+0000 7fd61236a640 1 --2- 172.21.15.84:0/1677592854 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd5f80a2e20 0x7fd5f80a52e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.974+0000 7fd61236a640 1 --2- 172.21.15.84:0/1677592854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd60c106750 0x7fd60c19b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.974+0000 7fd61236a640 1 --2- 172.21.15.84:0/1677592854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd60c101100 0x7fd60c1975a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.974+0000 7fd61236a640 1 --2- 172.21.15.84:0/1677592854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd60c100740 0x7fd60c197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:25.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.974+0000 7fd61236a640 1 -- 172.21.15.84:0/1677592854 >> 172.21.15.84:0/1677592854 conn(0x7fd60c077870 msgr2=0x7fd60c102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:25.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.974+0000 7fd61236a640 1 -- 172.21.15.84:0/1677592854 shutdown_connections 2024-05-22T23:02:25.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:25.974+0000 7fd61236a640 1 -- 172.21.15.84:0/1677592854 wait complete. 2024-05-22T23:02:25.987 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-22T23:02:25.987 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:02:26.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.085+0000 7f32d762c640 1 -- 172.21.15.84:0/111282429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32d0104f00 msgr2=0x7f32d01052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:26.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.085+0000 7f32d762c640 1 --2- 172.21.15.84:0/111282429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32d0104f00 0x7f32d01052e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009900 tx=0x7f32cc02a480 comp rx=0 tx=0).stop 2024-05-22T23:02:26.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.085+0000 7f32d762c640 1 -- 172.21.15.84:0/111282429 shutdown_connections 2024-05-22T23:02:26.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.085+0000 7f32d762c640 1 --2- 172.21.15.84:0/111282429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32d0100b80 0x7f32d010cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.085+0000 7f32d762c640 1 --2- 172.21.15.84:0/111282429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32d01001c0 0x7f32d0100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.085+0000 7f32d762c640 1 --2- 172.21.15.84:0/111282429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32d0104f00 0x7f32d01052e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.085+0000 7f32d762c640 1 -- 172.21.15.84:0/111282429 >> 172.21.15.84:0/111282429 conn(0x7f32d0075a50 msgr2=0x7f32d0075e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:26.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.085+0000 7f32d762c640 1 -- 172.21.15.84:0/111282429 shutdown_connections 2024-05-22T23:02:26.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.085+0000 7f32d762c640 1 -- 172.21.15.84:0/111282429 wait complete. 2024-05-22T23:02:26.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.086+0000 7f32d762c640 1 Processor -- start 2024-05-22T23:02:26.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.086+0000 7f32d762c640 1 -- start start 2024-05-22T23:02:26.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.086+0000 7f32d762c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32d01001c0 0x7f32d0198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:26.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.086+0000 7f32d762c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32d0100b80 0x7f32d01994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:26.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.086+0000 7f32d762c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32d0104f00 0x7f32d01a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:26.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.086+0000 7f32d762c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f32d0110290 con 0x7f32d0100b80 2024-05-22T23:02:26.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.086+0000 7f32d762c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f32d0110110 con 0x7f32d01001c0 2024-05-22T23:02:26.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.086+0000 7f32d762c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f32d0110410 con 0x7f32d0104f00 2024-05-22T23:02:26.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.086+0000 7f32d4bc7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32d0100b80 0x7f32d01994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:26.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.086+0000 7f32d4bc7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32d0100b80 0x7f32d01994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35814/0 (socket says 172.21.15.84:35814) 2024-05-22T23:02:26.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.086+0000 7f32d4bc7640 1 -- 172.21.15.84:0/3905720877 learned_addr learned my addr 172.21.15.84:0/3905720877 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:26.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.086+0000 7f32d53c8640 1 --2- 172.21.15.84:0/3905720877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32d01001c0 0x7f32d0198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:26.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.087+0000 7f32d5bc9640 1 --2- 172.21.15.84:0/3905720877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32d0104f00 0x7f32d01a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:26.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.087+0000 7f32d4bc7640 1 -- 172.21.15.84:0/3905720877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32d0104f00 msgr2=0x7f32d01a4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:26.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.087+0000 7f32d4bc7640 1 --2- 172.21.15.84:0/3905720877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32d0104f00 0x7f32d01a4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.087+0000 7f32d4bc7640 1 -- 172.21.15.84:0/3905720877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32d01001c0 msgr2=0x7f32d0198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:26.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.087+0000 7f32d4bc7640 1 --2- 172.21.15.84:0/3905720877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32d01001c0 0x7f32d0198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.087+0000 7f32d4bc7640 1 -- 172.21.15.84:0/3905720877 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d01a4550 con 0x7f32d0100b80 2024-05-22T23:02:26.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.087+0000 7f32d5bc9640 1 --2- 172.21.15.84:0/3905720877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32d0104f00 0x7f32d01a4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:02:26.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.087+0000 7f32d4bc7640 1 --2- 172.21.15.84:0/3905720877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32d0100b80 0x7f32d01994d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f32c000d950 tx=0x7f32c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:26.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.087+0000 7f32c67fc640 1 -- 172.21.15.84:0/3905720877 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c0017030 con 0x7f32d0100b80 2024-05-22T23:02:26.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.087+0000 7f32c67fc640 1 -- 172.21.15.84:0/3905720877 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32c0004020 con 0x7f32d0100b80 2024-05-22T23:02:26.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.087+0000 7f32d762c640 1 -- 172.21.15.84:0/3905720877 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d01a4720 con 0x7f32d0100b80 2024-05-22T23:02:26.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.087+0000 7f32c67fc640 1 -- 172.21.15.84:0/3905720877 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c00047a0 con 0x7f32d0100b80 2024-05-22T23:02:26.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.087+0000 7f32d762c640 1 -- 172.21.15.84:0/3905720877 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d01a4c70 con 0x7f32d0100b80 2024-05-22T23:02:26.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.088+0000 7f32d762c640 1 -- 172.21.15.84:0/3905720877 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32d0101630 con 0x7f32d0100b80 2024-05-22T23:02:26.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.089+0000 7f32c67fc640 1 -- 172.21.15.84:0/3905720877 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f32c00041d0 con 0x7f32d0100b80 2024-05-22T23:02:26.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.090+0000 7f32c67fc640 1 --2- 172.21.15.84:0/3905720877 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f32bc0a2e20 0x7f32bc0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:26.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.090+0000 7f32d53c8640 1 --2- 172.21.15.84:0/3905720877 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f32bc0a2e20 0x7f32bc0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:26.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.090+0000 7f32c67fc640 1 -- 172.21.15.84:0/3905720877 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f32c00cf0f0 con 0x7f32d0100b80 2024-05-22T23:02:26.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.090+0000 7f32d53c8640 1 --2- 172.21.15.84:0/3905720877 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f32bc0a2e20 0x7f32bc0a52e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f32cc004770 tx=0x7f32cc0058d0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:26.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.094+0000 7f32c67fc640 1 -- 172.21.15.84:0/3905720877 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f32c009bac0 con 0x7f32d0100b80 2024-05-22T23:02:26.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.306+0000 7f32d762c640 1 -- 172.21.15.84:0/3905720877 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32d01a4f30 con 0x7f32d0100b80 2024-05-22T23:02:26.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.307+0000 7f32c67fc640 1 -- 172.21.15.84:0/3905720877 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7f32c009b3c0 con 0x7f32d0100b80 2024-05-22T23:02:26.310 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:02:26.310 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:02:26.310 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:02:26.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.309+0000 7f32d762c640 1 -- 172.21.15.84:0/3905720877 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f32bc0a2e20 msgr2=0x7f32bc0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:26.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.309+0000 7f32d762c640 1 --2- 172.21.15.84:0/3905720877 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f32bc0a2e20 0x7f32bc0a52e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f32cc004770 tx=0x7f32cc0058d0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.309+0000 7f32d762c640 1 -- 172.21.15.84:0/3905720877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32d0100b80 msgr2=0x7f32d01994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:26.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.309+0000 7f32d762c640 1 --2- 172.21.15.84:0/3905720877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32d0100b80 0x7f32d01994d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f32c000d950 tx=0x7f32c000de10 comp rx=0 tx=0).stop 2024-05-22T23:02:26.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.309+0000 7f32d762c640 1 -- 172.21.15.84:0/3905720877 shutdown_connections 2024-05-22T23:02:26.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.309+0000 7f32d762c640 1 --2- 172.21.15.84:0/3905720877 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f32bc0a2e20 0x7f32bc0a52e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.309+0000 7f32d762c640 1 --2- 172.21.15.84:0/3905720877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32d0104f00 0x7f32d01a4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.309+0000 7f32d762c640 1 --2- 172.21.15.84:0/3905720877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32d0100b80 0x7f32d01994d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.309+0000 7f32d762c640 1 --2- 172.21.15.84:0/3905720877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32d01001c0 0x7f32d0198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.309+0000 7f32d762c640 1 -- 172.21.15.84:0/3905720877 >> 172.21.15.84:0/3905720877 conn(0x7f32d0075a50 msgr2=0x7f32d00fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:26.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.309+0000 7f32d762c640 1 -- 172.21.15.84:0/3905720877 shutdown_connections 2024-05-22T23:02:26.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.309+0000 7f32d762c640 1 -- 172.21.15.84:0/3905720877 wait complete. 2024-05-22T23:02:26.323 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-22T23:02:26.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.423+0000 7fac87122640 1 -- 172.21.15.84:0/3822217233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac80101100 msgr2=0x7fac8010d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:26.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.423+0000 7fac87122640 1 --2- 172.21.15.84:0/3822217233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac80101100 0x7fac8010d610 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fac78009cd0 tx=0x7fac7802a8e0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.423+0000 7fac87122640 1 -- 172.21.15.84:0/3822217233 shutdown_connections 2024-05-22T23:02:26.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.423+0000 7fac87122640 1 --2- 172.21.15.84:0/3822217233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac80101100 0x7fac8010d610 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.423+0000 7fac87122640 1 --2- 172.21.15.84:0/3822217233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac80100740 0x7fac80100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.423+0000 7fac87122640 1 --2- 172.21.15.84:0/3822217233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac80106750 0x7fac80106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.423+0000 7fac87122640 1 -- 172.21.15.84:0/3822217233 >> 172.21.15.84:0/3822217233 conn(0x7fac80077870 msgr2=0x7fac80077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:26.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.423+0000 7fac87122640 1 -- 172.21.15.84:0/3822217233 shutdown_connections 2024-05-22T23:02:26.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.423+0000 7fac87122640 1 -- 172.21.15.84:0/3822217233 wait complete. 2024-05-22T23:02:26.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.423+0000 7fac87122640 1 Processor -- start 2024-05-22T23:02:26.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.423+0000 7fac87122640 1 -- start start 2024-05-22T23:02:26.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.424+0000 7fac87122640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac80100740 0x7fac80199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:26.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.424+0000 7fac87122640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac80101100 0x7fac80199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:26.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.424+0000 7fac87122640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac80106750 0x7fac801a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:26.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.424+0000 7fac87122640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fac80110b60 con 0x7fac80101100 2024-05-22T23:02:26.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.424+0000 7fac87122640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fac801109e0 con 0x7fac80100740 2024-05-22T23:02:26.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.424+0000 7fac87122640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fac80110ce0 con 0x7fac80106750 2024-05-22T23:02:26.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.424+0000 7fac7ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac80101100 0x7fac80199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:26.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.424+0000 7fac7ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac80101100 0x7fac80199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35830/0 (socket says 172.21.15.84:35830) 2024-05-22T23:02:26.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.424+0000 7fac7ffff640 1 -- 172.21.15.84:0/1135406488 learned_addr learned my addr 172.21.15.84:0/1135406488 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:26.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.424+0000 7fac84ebe640 1 --2- 172.21.15.84:0/1135406488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac80100740 0x7fac80199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:26.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.424+0000 7fac856bf640 1 --2- 172.21.15.84:0/1135406488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac80106750 0x7fac801a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:26.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.424+0000 7fac7ffff640 1 -- 172.21.15.84:0/1135406488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac80106750 msgr2=0x7fac801a40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:26.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.424+0000 7fac7ffff640 1 --2- 172.21.15.84:0/1135406488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac80106750 0x7fac801a40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.424+0000 7fac7ffff640 1 -- 172.21.15.84:0/1135406488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac80100740 msgr2=0x7fac80199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:26.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.424+0000 7fac7ffff640 1 --2- 172.21.15.84:0/1135406488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac80100740 0x7fac80199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.424+0000 7fac7ffff640 1 -- 172.21.15.84:0/1135406488 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac801a45e0 con 0x7fac80101100 2024-05-22T23:02:26.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.424+0000 7fac856bf640 1 --2- 172.21.15.84:0/1135406488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac80106750 0x7fac801a40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:02:26.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.424+0000 7fac7ffff640 1 --2- 172.21.15.84:0/1135406488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac80101100 0x7fac80199560 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00d950 tx=0x7fac6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:26.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.424+0000 7fac7dffb640 1 -- 172.21.15.84:0/1135406488 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c017070 con 0x7fac80101100 2024-05-22T23:02:26.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.424+0000 7fac7dffb640 1 -- 172.21.15.84:0/1135406488 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac6c009c30 con 0x7fac80101100 2024-05-22T23:02:26.430 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.424+0000 7fac87122640 1 -- 172.21.15.84:0/1135406488 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac801a47b0 con 0x7fac80101100 2024-05-22T23:02:26.430 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.424+0000 7fac7dffb640 1 -- 172.21.15.84:0/1135406488 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c00f6b0 con 0x7fac80101100 2024-05-22T23:02:26.430 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.425+0000 7fac87122640 1 -- 172.21.15.84:0/1135406488 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac801a4d00 con 0x7fac80101100 2024-05-22T23:02:26.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.427+0000 7fac7dffb640 1 -- 172.21.15.84:0/1135406488 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7fac6c0109d0 con 0x7fac80101100 2024-05-22T23:02:26.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.427+0000 7fac89e33740 1 Processor -- start 2024-05-22T23:02:26.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.427+0000 7fac89e33740 1 -- start start 2024-05-22T23:02:26.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.427+0000 7fac89e33740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb37b4cf80 0x55fb37b4d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:26.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.427+0000 7fac7dffb640 1 --2- 172.21.15.84:0/1135406488 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fac700a2e20 0x7fac700a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:26.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.427+0000 7fac89e33740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4d8a0 0x55fb37b57a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:26.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.427+0000 7fac89e33740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb37b57f70 0x55fb37b5a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:26.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.427+0000 7fac89e33740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55fb37ab1db0 con 0x55fb37b4d8a0 2024-05-22T23:02:26.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.427+0000 7fac89e33740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fb37a50400 con 0x55fb37b4cf80 2024-05-22T23:02:26.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.427+0000 7fac89e33740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fb37aadd70 con 0x55fb37b57f70 2024-05-22T23:02:26.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.428+0000 7fac7ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4d8a0 0x55fb37b57a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:26.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.428+0000 7fac856bf640 1 --2- 172.21.15.84:0/1135406488 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fac700a2e20 0x7fac700a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:26.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.428+0000 7fac7ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4d8a0 0x55fb37b57a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35846/0 (socket says 172.21.15.84:35846) 2024-05-22T23:02:26.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.428+0000 7fac7ffff640 1 -- 172.21.15.84:0/3271933710 learned_addr learned my addr 172.21.15.84:0/3271933710 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:26.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.428+0000 7fac84ebe640 1 --2- 172.21.15.84:0/3271933710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb37b4cf80 0x55fb37b4d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:26.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.428+0000 7fac7dffb640 1 -- 172.21.15.84:0/1135406488 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fac6c0cf3a0 con 0x7fac80101100 2024-05-22T23:02:26.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.428+0000 7fac856bf640 1 --2- 172.21.15.84:0/3271933710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb37b57f70 0x55fb37b5a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:26.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.428+0000 7fac7ffff640 1 -- 172.21.15.84:0/3271933710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb37b57f70 msgr2=0x55fb37b5a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:26.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.428+0000 7fac7ffff640 1 --2- 172.21.15.84:0/3271933710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb37b57f70 0x55fb37b5a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.428+0000 7fac7ffff640 1 -- 172.21.15.84:0/3271933710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb37b4cf80 msgr2=0x55fb37b4d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:26.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.428+0000 7fac7ffff640 1 --2- 172.21.15.84:0/3271933710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb37b4cf80 0x55fb37b4d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.428+0000 7fac7ffff640 1 -- 172.21.15.84:0/3271933710 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb37a422f0 con 0x55fb37b4d8a0 2024-05-22T23:02:26.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.428+0000 7fac7ffff640 1 --2- 172.21.15.84:0/3271933710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4d8a0 0x55fb37b57a30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fac6c09b370 tx=0x7fac6c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1371f3b8eeabc99 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:26.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.428+0000 7fac856bf640 1 --2- 172.21.15.84:0/1135406488 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fac700a2e20 0x7fac700a52e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fac78006010 tx=0x7fac78031640 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:26.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.428+0000 7fac6affd640 1 -- 172.21.15.84:0/3271933710 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c020070 con 0x55fb37b4d8a0 2024-05-22T23:02:26.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.428+0000 7fac6affd640 1 -- 172.21.15.84:0/3271933710 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac6c09bd70 con 0x55fb37b4d8a0 2024-05-22T23:02:26.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.428+0000 7fac6affd640 1 -- 172.21.15.84:0/3271933710 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c0a0950 con 0x55fb37b4d8a0 2024-05-22T23:02:26.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.429+0000 7fac89e33740 1 -- 172.21.15.84:0/3271933710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4d8a0 msgr2=0x55fb37b57a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:26.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.429+0000 7fac89e33740 1 --2- 172.21.15.84:0/3271933710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4d8a0 0x55fb37b57a30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fac6c09b370 tx=0x7fac6c00bf60 comp rx=0 tx=0).stop 2024-05-22T23:02:26.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.429+0000 7fac89e33740 1 -- 172.21.15.84:0/3271933710 shutdown_connections 2024-05-22T23:02:26.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.429+0000 7fac89e33740 1 --2- 172.21.15.84:0/3271933710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb37b57f70 0x55fb37b5a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.429+0000 7fac89e33740 1 --2- 172.21.15.84:0/3271933710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4d8a0 0x55fb37b57a30 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.429+0000 7fac89e33740 1 --2- 172.21.15.84:0/3271933710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb37b4cf80 0x55fb37b4d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.429+0000 7fac89e33740 1 -- 172.21.15.84:0/3271933710 >> 172.21.15.84:0/3271933710 conn(0x55fb37b492a0 msgr2=0x55fb37b49680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:26.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.429+0000 7fac89e33740 1 -- 172.21.15.84:0/3271933710 shutdown_connections 2024-05-22T23:02:26.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.429+0000 7fac89e33740 1 -- 172.21.15.84:0/3271933710 wait complete. 2024-05-22T23:02:26.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.429+0000 7fac89e33740 1 -- start start 2024-05-22T23:02:26.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.429+0000 7fac89e33740 1 Processor -- start 2024-05-22T23:02:26.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.430+0000 7fac89e33740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb37b4cf80 0x55fb37b54fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:26.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.430+0000 7fac89e33740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4d8a0 0x55fb37b554e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:26.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.430+0000 7fac89e33740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb37b57f70 0x55fb37b55a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:26.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.430+0000 7fac89e33740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55fb37b5aca0 con 0x55fb37b4d8a0 2024-05-22T23:02:26.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.430+0000 7fac89e33740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fb37a6c2e0 con 0x55fb37b4cf80 2024-05-22T23:02:26.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.430+0000 7fac89e33740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fb37b5ae20 con 0x55fb37b57f70 2024-05-22T23:02:26.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.430+0000 7fac89e33740 10 client.0 _ll_register_callbacks cb 0x55fb37a75a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:02:26.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.430+0000 7fac7ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4d8a0 0x55fb37b554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:26.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.430+0000 7fac7ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4d8a0 0x55fb37b554e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35854/0 (socket says 172.21.15.84:35854) 2024-05-22T23:02:26.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.430+0000 7fac7ffff640 1 -- 172.21.15.84:0/3891261902 learned_addr learned my addr 172.21.15.84:0/3891261902 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:26.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.430+0000 7fac84ebe640 1 --2- 172.21.15.84:0/3891261902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb37b4cf80 0x55fb37b54fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:26.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.430+0000 7fac856bf640 1 --2- 172.21.15.84:0/3891261902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb37b57f70 0x55fb37b55a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:26.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.430+0000 7fac7ffff640 1 -- 172.21.15.84:0/3891261902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb37b57f70 msgr2=0x55fb37b55a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:26.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.430+0000 7fac7ffff640 1 --2- 172.21.15.84:0/3891261902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb37b57f70 0x55fb37b55a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.430+0000 7fac7ffff640 1 -- 172.21.15.84:0/3891261902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb37b4cf80 msgr2=0x55fb37b54fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:26.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.430+0000 7fac7ffff640 1 --2- 172.21.15.84:0/3891261902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb37b4cf80 0x55fb37b54fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.430+0000 7fac7ffff640 1 -- 172.21.15.84:0/3891261902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb377c53f0 con 0x55fb37b4d8a0 2024-05-22T23:02:26.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.430+0000 7fac84ebe640 1 --2- 172.21.15.84:0/3891261902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb37b4cf80 0x55fb37b54fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:02:26.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.430+0000 7fac7ffff640 1 --2- 172.21.15.84:0/3891261902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4d8a0 0x55fb37b554e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fac6c097ad0 tx=0x7fac6c086160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5108fac70785da server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:26.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.430+0000 7fac5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:02:26.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.431+0000 7fac5ffff640 1 -- 172.21.15.84:0/3891261902 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c020050 con 0x55fb37b4d8a0 2024-05-22T23:02:26.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.431+0000 7fac5ffff640 1 -- 172.21.15.84:0/3891261902 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fac6c09f7c0 con 0x55fb37b4d8a0 2024-05-22T23:02:26.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.431+0000 7fac89e33740 1 -- 172.21.15.84:0/3891261902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac6c09bd70 con 0x55fb37b4d8a0 2024-05-22T23:02:26.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.431+0000 7fac5ffff640 1 -- 172.21.15.84:0/3891261902 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fac6c009de0 con 0x55fb37b4d8a0 2024-05-22T23:02:26.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.431+0000 7fac89e33740 10 client.15522 fetch_fsmap learned FSMap version 18 2024-05-22T23:02:26.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.431+0000 7fac89e33740 1 -- 172.21.15.84:0/3891261902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55fb3788f7d0 con 0x55fb37b4d8a0 2024-05-22T23:02:26.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.431+0000 7fac5ffff640 1 -- 172.21.15.84:0/3891261902 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac6c09bd70 con 0x55fb37b4d8a0 2024-05-22T23:02:26.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.431+0000 7fac5ffff640 1 -- 172.21.15.84:0/3891261902 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fac6c097ce0 con 0x55fb37b4d8a0 2024-05-22T23:02:26.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.431+0000 7fac89e33740 10 client.15522 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:02:26.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.431+0000 7fac89e33740 10 client.15522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:02:26.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.431+0000 7fac89e33740 20 client.15522 populate_metadata read hostname 'smithi084' 2024-05-22T23:02:26.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.431+0000 7fac89e33740 1 --2- 172.21.15.84:0/3891261902 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55fb37b680c0 0x55fb37b6a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:26.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.431+0000 7fac89e33740 4 client.15522 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:02:26.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.431+0000 7fac89e33740 1 -- 172.21.15.84:0/3891261902 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb37abf4e0 con 0x55fb37b680c0 2024-05-22T23:02:26.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.431+0000 7fac84ebe640 1 --2- 172.21.15.84:0/3891261902 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55fb37b680c0 0x55fb37b6a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:26.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.432+0000 7fac5ffff640 10 client.15522 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:26.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.432+0000 7fac84ebe640 1 --2- 172.21.15.84:0/3891261902 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55fb37b680c0 0x55fb37b6a4b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35924223d42044a8 server_cookie=6a3cdb6ed9e2f04e in_seq=0 out_seq=0 2024-05-22T23:02:26.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.432+0000 7fac5ffff640 1 -- 172.21.15.84:0/3891261902 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fb37abf4e0 con 0x55fb37b680c0 2024-05-22T23:02:26.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.432+0000 7fac5ffff640 10 client.15522 handle_command_reply: tid=0 2024-05-22T23:02:26.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.432+0000 7fac89e33740 1 client.15522 shutdown 2024-05-22T23:02:26.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.433+0000 7fac89e33740 1 -- 172.21.15.84:0/3891261902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4d8a0 msgr2=0x55fb37b554e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:26.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.433+0000 7fac89e33740 1 --2- 172.21.15.84:0/3891261902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4d8a0 0x55fb37b554e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fac6c097ad0 tx=0x7fac6c086160 comp rx=0 tx=0).stop 2024-05-22T23:02:26.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.433+0000 7fac89e33740 1 -- 172.21.15.84:0/3891261902 shutdown_connections 2024-05-22T23:02:26.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.433+0000 7fac89e33740 1 --2- 172.21.15.84:0/3891261902 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55fb37b680c0 0x55fb37b6a4b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.433+0000 7fac89e33740 1 --2- 172.21.15.84:0/3891261902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb37b57f70 0x55fb37b55a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.433+0000 7fac5ffff640 0 client.15522 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:26.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.433+0000 7fac89e33740 1 --2- 172.21.15.84:0/3891261902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4d8a0 0x55fb37b554e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.433+0000 7fac89e33740 1 --2- 172.21.15.84:0/3891261902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb37b4cf80 0x55fb37b54fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.433+0000 7fac89e33740 1 -- 172.21.15.84:0/3891261902 >> 172.21.15.84:0/3891261902 conn(0x55fb37b492a0 msgr2=0x55fb37b484c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:26.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.433+0000 7fac89e33740 1 -- 172.21.15.84:0/3891261902 shutdown_connections 2024-05-22T23:02:26.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.433+0000 7fac89e33740 1 -- 172.21.15.84:0/3891261902 wait complete. 2024-05-22T23:02:26.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.433+0000 7fac89e33740 20 client.15522 trim_cache size 0 max 16384 2024-05-22T23:02:26.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.446+0000 7fac89e33740 1 Processor -- start 2024-05-22T23:02:26.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.446+0000 7fac89e33740 1 -- start start 2024-05-22T23:02:26.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.446+0000 7fac89e33740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4cf80 0x55fb37b576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:26.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.446+0000 7fac89e33740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb37b4d8a0 0x55fb37b6d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:26.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.446+0000 7fac89e33740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb37b57f70 0x55fb37b6da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:26.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.446+0000 7fac89e33740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fac801ab680 con 0x55fb37b4cf80 2024-05-22T23:02:26.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.446+0000 7fac89e33740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fac801adf00 con 0x55fb37b57f70 2024-05-22T23:02:26.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.446+0000 7fac89e33740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fac801abcf0 con 0x55fb37b4d8a0 2024-05-22T23:02:26.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.446+0000 7fac84ebe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4cf80 0x55fb37b576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:26.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.446+0000 7fac84ebe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4cf80 0x55fb37b576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35860/0 (socket says 172.21.15.84:35860) 2024-05-22T23:02:26.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.446+0000 7fac84ebe640 1 -- 172.21.15.84:0/3102457197 learned_addr learned my addr 172.21.15.84:0/3102457197 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:26.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.446+0000 7fac84ebe640 1 -- 172.21.15.84:0/3102457197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb37b4d8a0 msgr2=0x55fb37b6d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:26.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.446+0000 7fac84ebe640 1 --2- 172.21.15.84:0/3102457197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb37b4d8a0 0x55fb37b6d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.446+0000 7fac84ebe640 1 -- 172.21.15.84:0/3102457197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb37b57f70 msgr2=0x55fb37b6da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:26.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.446+0000 7fac84ebe640 1 --2- 172.21.15.84:0/3102457197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb37b57f70 0x55fb37b6da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.446+0000 7fac84ebe640 1 -- 172.21.15.84:0/3102457197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb37b534f0 con 0x55fb37b4cf80 2024-05-22T23:02:26.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.447+0000 7fac84ebe640 1 --2- 172.21.15.84:0/3102457197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4cf80 0x55fb37b576d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x55fb37ab1ec0 tx=0x7fac740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72a2d24674d72b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:26.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.447+0000 7fac6a7fc640 1 -- 172.21.15.84:0/3102457197 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74025070 con 0x55fb37b4cf80 2024-05-22T23:02:26.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.447+0000 7fac6a7fc640 1 -- 172.21.15.84:0/3102457197 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac74003dd0 con 0x55fb37b4cf80 2024-05-22T23:02:26.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.447+0000 7fac6a7fc640 1 -- 172.21.15.84:0/3102457197 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74010a20 con 0x55fb37b4cf80 2024-05-22T23:02:26.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.447+0000 7fac89e33740 1 -- 172.21.15.84:0/3102457197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4cf80 msgr2=0x55fb37b576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:26.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.447+0000 7fac89e33740 1 --2- 172.21.15.84:0/3102457197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4cf80 0x55fb37b576d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x55fb37ab1ec0 tx=0x7fac740046c0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.447+0000 7fac89e33740 1 -- 172.21.15.84:0/3102457197 shutdown_connections 2024-05-22T23:02:26.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.447+0000 7fac89e33740 1 --2- 172.21.15.84:0/3102457197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb37b57f70 0x55fb37b6da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.447+0000 7fac89e33740 1 --2- 172.21.15.84:0/3102457197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb37b4d8a0 0x55fb37b6d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.447+0000 7fac89e33740 1 --2- 172.21.15.84:0/3102457197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4cf80 0x55fb37b576d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.447+0000 7fac89e33740 1 -- 172.21.15.84:0/3102457197 >> 172.21.15.84:0/3102457197 conn(0x55fb37b492a0 msgr2=0x55fb37b557f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:26.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.447+0000 7fac89e33740 1 -- 172.21.15.84:0/3102457197 shutdown_connections 2024-05-22T23:02:26.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.447+0000 7fac89e33740 1 -- 172.21.15.84:0/3102457197 wait complete. 2024-05-22T23:02:26.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.448+0000 7fac89e33740 1 -- start start 2024-05-22T23:02:26.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.448+0000 7fac89e33740 1 Processor -- start 2024-05-22T23:02:26.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.448+0000 7fac89e33740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4cf80 0x55fb37b6b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:26.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.448+0000 7fac89e33740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb37b4d8a0 0x55fb37b6bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:26.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.448+0000 7fac89e33740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb37b57f70 0x55fb37bc72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:26.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.448+0000 7fac89e33740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fac801ac3c0 con 0x55fb37b4cf80 2024-05-22T23:02:26.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.448+0000 7fac89e33740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fac801ad830 con 0x55fb37b4d8a0 2024-05-22T23:02:26.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.448+0000 7fac89e33740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fac801aca90 con 0x55fb37b57f70 2024-05-22T23:02:26.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.448+0000 7fac89e33740 10 client.0 _ll_register_callbacks cb 0x7fac6c005680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:02:26.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.448+0000 7fac84ebe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4cf80 0x55fb37b6b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:26.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.448+0000 7fac84ebe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4cf80 0x55fb37b6b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35862/0 (socket says 172.21.15.84:35862) 2024-05-22T23:02:26.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.448+0000 7fac84ebe640 1 -- 172.21.15.84:0/1387805777 learned_addr learned my addr 172.21.15.84:0/1387805777 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:26.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.448+0000 7fac856bf640 1 --2- 172.21.15.84:0/1387805777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb37b57f70 0x55fb37bc72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:26.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.448+0000 7fac84ebe640 1 -- 172.21.15.84:0/1387805777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb37b57f70 msgr2=0x55fb37bc72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:26.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.448+0000 7fac84ebe640 1 --2- 172.21.15.84:0/1387805777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb37b57f70 0x55fb37bc72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.448+0000 7fac84ebe640 1 -- 172.21.15.84:0/1387805777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb37b4d8a0 msgr2=0x55fb37b6bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:26.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.448+0000 7fac7ffff640 1 --2- 172.21.15.84:0/1387805777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb37b4d8a0 0x55fb37b6bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:26.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.448+0000 7fac84ebe640 1 --2- 172.21.15.84:0/1387805777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb37b4d8a0 0x55fb37b6bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.448+0000 7fac84ebe640 1 -- 172.21.15.84:0/1387805777 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb37bc7920 con 0x55fb37b4cf80 2024-05-22T23:02:26.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.448+0000 7fac856bf640 1 --2- 172.21.15.84:0/1387805777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb37b57f70 0x55fb37bc72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:02:26.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.449+0000 7fac84ebe640 1 --2- 172.21.15.84:0/1387805777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4cf80 0x55fb37b6b750 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fac74009c60 tx=0x7fac74009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f4f23706c0d21b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:26.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.449+0000 7fac5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:02:26.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.449+0000 7fac5ffff640 1 -- 172.21.15.84:0/1387805777 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74025070 con 0x55fb37b4cf80 2024-05-22T23:02:26.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.449+0000 7fac5ffff640 1 -- 172.21.15.84:0/1387805777 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fac74010ab0 con 0x55fb37b4cf80 2024-05-22T23:02:26.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.449+0000 7fac89e33740 1 -- 172.21.15.84:0/1387805777 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac74003dd0 con 0x55fb37b4cf80 2024-05-22T23:02:26.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.449+0000 7fac5ffff640 1 -- 172.21.15.84:0/1387805777 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fac74038050 con 0x55fb37b4cf80 2024-05-22T23:02:26.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.449+0000 7fac89e33740 10 client.15528 fetch_fsmap learned FSMap version 18 2024-05-22T23:02:26.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.449+0000 7fac89e33740 1 -- 172.21.15.84:0/1387805777 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55fb37bcecd0 con 0x55fb37b4cf80 2024-05-22T23:02:26.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.449+0000 7fac5ffff640 1 -- 172.21.15.84:0/1387805777 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac74003dd0 con 0x55fb37b4cf80 2024-05-22T23:02:26.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.449+0000 7fac5ffff640 1 -- 172.21.15.84:0/1387805777 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fac7402a4d0 con 0x55fb37b4cf80 2024-05-22T23:02:26.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.449+0000 7fac89e33740 10 client.15528 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:02:26.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.449+0000 7fac89e33740 10 client.15528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:02:26.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.449+0000 7fac89e33740 20 client.15528 populate_metadata read hostname 'smithi084' 2024-05-22T23:02:26.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.449+0000 7fac89e33740 1 --2- 172.21.15.84:0/1387805777 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55fb37bcf3b0 0x55fb37bd17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:26.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.449+0000 7fac89e33740 4 client.15528 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-22T23:02:26.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.449+0000 7fac89e33740 1 -- 172.21.15.84:0/1387805777 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fb37ab9830 con 0x55fb37bcf3b0 2024-05-22T23:02:26.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.449+0000 7fac7ffff640 1 --2- 172.21.15.84:0/1387805777 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55fb37bcf3b0 0x55fb37bd17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:26.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.450+0000 7fac5ffff640 10 client.15528 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:26.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.450+0000 7fac7ffff640 1 --2- 172.21.15.84:0/1387805777 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55fb37bcf3b0 0x55fb37bd17d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62dd29033fec1185 server_cookie=af6edc3d70c1e36b in_seq=0 out_seq=0 2024-05-22T23:02:26.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.450+0000 7fac5ffff640 1 -- 172.21.15.84:0/1387805777 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fb37ab9830 con 0x55fb37bcf3b0 2024-05-22T23:02:26.459 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.450+0000 7fac5ffff640 10 client.15528 handle_command_reply: tid=0 2024-05-22T23:02:26.459 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.450+0000 7fac89e33740 1 client.15528 shutdown 2024-05-22T23:02:26.459 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.450+0000 7fac89e33740 1 -- 172.21.15.84:0/1387805777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4cf80 msgr2=0x55fb37b6b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:26.459 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.450+0000 7fac89e33740 1 --2- 172.21.15.84:0/1387805777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4cf80 0x55fb37b6b750 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fac74009c60 tx=0x7fac74009df0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.459 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.450+0000 7fac89e33740 1 -- 172.21.15.84:0/1387805777 shutdown_connections 2024-05-22T23:02:26.459 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.450+0000 7fac89e33740 1 --2- 172.21.15.84:0/1387805777 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55fb37bcf3b0 0x55fb37bd17d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.459 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.450+0000 7fac89e33740 1 --2- 172.21.15.84:0/1387805777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb37b57f70 0x55fb37bc72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.459 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.450+0000 7fac5ffff640 0 client.15528 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:26.459 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.450+0000 7fac89e33740 1 --2- 172.21.15.84:0/1387805777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb37b4d8a0 0x55fb37b6bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.459 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.450+0000 7fac89e33740 1 --2- 172.21.15.84:0/1387805777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb37b4cf80 0x55fb37b6b750 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.459 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.450+0000 7fac89e33740 1 -- 172.21.15.84:0/1387805777 >> 172.21.15.84:0/1387805777 conn(0x55fb37b492a0 msgr2=0x55fb37b47c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:26.459 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.450+0000 7fac89e33740 1 -- 172.21.15.84:0/1387805777 shutdown_connections 2024-05-22T23:02:26.459 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.450+0000 7fac89e33740 1 -- 172.21.15.84:0/1387805777 wait complete. 2024-05-22T23:02:26.460 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.450+0000 7fac89e33740 20 client.15528 trim_cache size 0 max 16384 2024-05-22T23:02:26.460 INFO:teuthology.orchestra.run.smithi084.stdout:[] 2024-05-22T23:02:26.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.451+0000 7fac87122640 1 -- 172.21.15.84:0/1135406488 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fac700a2e20 msgr2=0x7fac700a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:26.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.451+0000 7fac87122640 1 --2- 172.21.15.84:0/1135406488 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fac700a2e20 0x7fac700a52e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fac78006010 tx=0x7fac78031640 comp rx=0 tx=0).stop 2024-05-22T23:02:26.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.451+0000 7fac87122640 1 -- 172.21.15.84:0/1135406488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac80101100 msgr2=0x7fac80199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:26.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.451+0000 7fac87122640 1 --2- 172.21.15.84:0/1135406488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac80101100 0x7fac80199560 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00d950 tx=0x7fac6c00de10 comp rx=0 tx=0).stop 2024-05-22T23:02:26.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.451+0000 7fac87122640 1 -- 172.21.15.84:0/1135406488 shutdown_connections 2024-05-22T23:02:26.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.451+0000 7fac87122640 1 --2- 172.21.15.84:0/1135406488 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fac700a2e20 0x7fac700a52e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.451+0000 7fac87122640 1 --2- 172.21.15.84:0/1135406488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac80106750 0x7fac801a40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.451+0000 7fac87122640 1 --2- 172.21.15.84:0/1135406488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac80101100 0x7fac80199560 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.451+0000 7fac87122640 1 --2- 172.21.15.84:0/1135406488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac80100740 0x7fac80199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:26.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.451+0000 7fac87122640 1 -- 172.21.15.84:0/1135406488 >> 172.21.15.84:0/1135406488 conn(0x7fac80077870 msgr2=0x7fac80102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:26.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.451+0000 7fac87122640 1 -- 172.21.15.84:0/1135406488 shutdown_connections 2024-05-22T23:02:26.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:26.451+0000 7fac87122640 1 -- 172.21.15.84:0/1135406488 wait complete. 2024-05-22T23:02:27.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:26 smithi096 ceph-mon[36392]: scrub queued for path: / 2024-05-22T23:02:27.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:26 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:02:27.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:26 smithi096 ceph-mon[36392]: scrub summary: active paths [/] 2024-05-22T23:02:27.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:26 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:02:27.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:26 smithi096 ceph-mon[36392]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:02:27.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:26 smithi096 ceph-mon[36392]: scrub summary: idle 2024-05-22T23:02:27.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:26 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3905720877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:02:27.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:26 smithi084 ceph-mon[32186]: scrub queued for path: / 2024-05-22T23:02:27.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:26 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:02:27.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:26 smithi084 ceph-mon[32186]: scrub summary: active paths [/] 2024-05-22T23:02:27.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:26 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:02:27.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:26 smithi084 ceph-mon[32186]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:02:27.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:26 smithi084 ceph-mon[32186]: scrub summary: idle 2024-05-22T23:02:27.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:26 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3905720877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:02:27.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:26 smithi204 ceph-mon[36211]: scrub queued for path: / 2024-05-22T23:02:27.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:26 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:02:27.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:26 smithi204 ceph-mon[36211]: scrub summary: active paths [/] 2024-05-22T23:02:27.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:26 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:02:27.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:26 smithi204 ceph-mon[36211]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:02:27.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:26 smithi204 ceph-mon[36211]: scrub summary: idle 2024-05-22T23:02:27.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:26 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3905720877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:02:27.464 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-22T23:02:27.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:02:27.465 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-22T23:02:27.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.579+0000 7f4d8fa6b640 1 -- 172.21.15.84:0/1565854539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d88102200 msgr2=0x7f4d8810e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:27.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.579+0000 7f4d8fa6b640 1 --2- 172.21.15.84:0/1565854539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d88102200 0x7f4d8810e710 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009cd0 tx=0x7f4d8002a8e0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.580+0000 7f4d8fa6b640 1 -- 172.21.15.84:0/1565854539 shutdown_connections 2024-05-22T23:02:27.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.580+0000 7f4d8fa6b640 1 --2- 172.21.15.84:0/1565854539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d88102200 0x7f4d8810e710 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.580+0000 7f4d8fa6b640 1 --2- 172.21.15.84:0/1565854539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d88101840 0x7f4d88101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.580+0000 7f4d8fa6b640 1 --2- 172.21.15.84:0/1565854539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d88107850 0x7f4d88107c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.580+0000 7f4d8fa6b640 1 -- 172.21.15.84:0/1565854539 >> 172.21.15.84:0/1565854539 conn(0x7f4d880fd2b0 msgr2=0x7f4d880ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:27.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.580+0000 7f4d8fa6b640 1 -- 172.21.15.84:0/1565854539 shutdown_connections 2024-05-22T23:02:27.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.580+0000 7f4d8fa6b640 1 -- 172.21.15.84:0/1565854539 wait complete. 2024-05-22T23:02:27.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.580+0000 7f4d8fa6b640 1 Processor -- start 2024-05-22T23:02:27.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.580+0000 7f4d8fa6b640 1 -- start start 2024-05-22T23:02:27.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.580+0000 7f4d8fa6b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d88101840 0x7f4d88198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.580+0000 7f4d8fa6b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d88102200 0x7f4d881985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.580+0000 7f4d8fa6b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d88107850 0x7f4d8819c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.580+0000 7f4d8fa6b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4d88111c60 con 0x7f4d88107850 2024-05-22T23:02:27.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.580+0000 7f4d8fa6b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4d88111ae0 con 0x7f4d88101840 2024-05-22T23:02:27.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.580+0000 7f4d8fa6b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4d88111de0 con 0x7f4d88102200 2024-05-22T23:02:27.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.581+0000 7f4d8e008640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d88107850 0x7f4d8819c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.581+0000 7f4d8e008640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d88107850 0x7f4d8819c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35880/0 (socket says 172.21.15.84:35880) 2024-05-22T23:02:27.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.581+0000 7f4d8e008640 1 -- 172.21.15.84:0/1298504802 learned_addr learned my addr 172.21.15.84:0/1298504802 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:27.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.581+0000 7f4d8d006640 1 --2- 172.21.15.84:0/1298504802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d88102200 0x7f4d881985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.581+0000 7f4d8d807640 1 --2- 172.21.15.84:0/1298504802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d88101840 0x7f4d88198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.581+0000 7f4d8e008640 1 -- 172.21.15.84:0/1298504802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d88102200 msgr2=0x7f4d881985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:27.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.581+0000 7f4d8e008640 1 --2- 172.21.15.84:0/1298504802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d88102200 0x7f4d881985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.581+0000 7f4d8e008640 1 -- 172.21.15.84:0/1298504802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d88101840 msgr2=0x7f4d88198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:27.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.581+0000 7f4d8e008640 1 --2- 172.21.15.84:0/1298504802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d88101840 0x7f4d88198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.581+0000 7f4d8e008640 1 -- 172.21.15.84:0/1298504802 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8819d050 con 0x7f4d88107850 2024-05-22T23:02:27.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.581+0000 7f4d8e008640 1 --2- 172.21.15.84:0/1298504802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d88107850 0x7f4d8819c920 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4d80005cc0 tx=0x7f4d80003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:27.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.581+0000 7f4d7effd640 1 -- 172.21.15.84:0/1298504802 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d80004040 con 0x7f4d88107850 2024-05-22T23:02:27.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.581+0000 7f4d7effd640 1 -- 172.21.15.84:0/1298504802 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d800046c0 con 0x7f4d88107850 2024-05-22T23:02:27.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.581+0000 7f4d8fa6b640 1 -- 172.21.15.84:0/1298504802 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d8819d2e0 con 0x7f4d88107850 2024-05-22T23:02:27.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.581+0000 7f4d7effd640 1 -- 172.21.15.84:0/1298504802 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d80046480 con 0x7f4d88107850 2024-05-22T23:02:27.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.581+0000 7f4d8fa6b640 1 -- 172.21.15.84:0/1298504802 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d881a4bd0 con 0x7f4d88107850 2024-05-22T23:02:27.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.583+0000 7f4d7effd640 1 -- 172.21.15.84:0/1298504802 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f4d800305a0 con 0x7f4d88107850 2024-05-22T23:02:27.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.584+0000 7f4d9277b740 1 Processor -- start 2024-05-22T23:02:27.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.584+0000 7f4d9277b740 1 -- start start 2024-05-22T23:02:27.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.584+0000 7f4d7effd640 1 --2- 172.21.15.84:0/1298504802 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f4d740a2e20 0x7f4d740a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.584+0000 7f4d9277b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67864bb30 0x55c67864bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.584+0000 7f4d9277b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67864c450 0x55c67865ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.585+0000 7f4d9277b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67865b050 0x55c67865d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.585+0000 7f4d9277b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c6785afa40 con 0x55c67865b050 2024-05-22T23:02:27.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.585+0000 7f4d9277b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c67854e440 con 0x55c67864bb30 2024-05-22T23:02:27.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.585+0000 7f4d9277b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c6785aba00 con 0x55c67864c450 2024-05-22T23:02:27.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.585+0000 7f4d7effd640 1 -- 172.21.15.84:0/1298504802 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f4d8002e030 con 0x7f4d88107850 2024-05-22T23:02:27.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.585+0000 7f4d8d006640 1 --2- 172.21.15.84:0/1298504802 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f4d740a2e20 0x7f4d740a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.585+0000 7f4d8e008640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67865b050 0x55c67865d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.585+0000 7f4d8e008640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67865b050 0x55c67865d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35884/0 (socket says 172.21.15.84:35884) 2024-05-22T23:02:27.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.585+0000 7f4d8e008640 1 -- 172.21.15.84:0/1183544265 learned_addr learned my addr 172.21.15.84:0/1183544265 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:27.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.585+0000 7f4d8d006640 1 --2- 172.21.15.84:0/1183544265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67864c450 0x55c67865ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.585+0000 7f4d8d807640 1 --2- 172.21.15.84:0/1183544265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67864bb30 0x55c67864bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.585+0000 7f4d8e008640 1 -- 172.21.15.84:0/1183544265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67864c450 msgr2=0x55c67865ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:27.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.585+0000 7f4d8e008640 1 --2- 172.21.15.84:0/1183544265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67864c450 0x55c67865ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.585+0000 7f4d8e008640 1 -- 172.21.15.84:0/1183544265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67864bb30 msgr2=0x55c67864bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:27.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.585+0000 7f4d8e008640 1 --2- 172.21.15.84:0/1183544265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67864bb30 0x55c67864bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.585+0000 7f4d8e008640 1 -- 172.21.15.84:0/1183544265 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6785582d0 con 0x55c67865b050 2024-05-22T23:02:27.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.585+0000 7f4d8d006640 1 --2- 172.21.15.84:0/1298504802 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f4d740a2e20 0x7f4d740a52e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4d78005e10 tx=0x7f4d780128d0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:27.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.585+0000 7f4d8e008640 1 --2- 172.21.15.84:0/1183544265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67865b050 0x55c67865d440 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4d800bcff0 tx=0x7f4d8002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df55a39fe17fa1d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:27.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.585+0000 7f4d73fff640 1 -- 172.21.15.84:0/1183544265 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d800b7e60 con 0x55c67865b050 2024-05-22T23:02:27.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.585+0000 7f4d73fff640 1 -- 172.21.15.84:0/1183544265 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d80036070 con 0x55c67865b050 2024-05-22T23:02:27.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.585+0000 7f4d73fff640 1 -- 172.21.15.84:0/1183544265 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d800c16a0 con 0x55c67865b050 2024-05-22T23:02:27.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.586+0000 7f4d9277b740 1 -- 172.21.15.84:0/1183544265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67865b050 msgr2=0x55c67865d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:27.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.586+0000 7f4d9277b740 1 --2- 172.21.15.84:0/1183544265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67865b050 0x55c67865d440 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4d800bcff0 tx=0x7f4d8002a8e0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.586+0000 7f4d9277b740 1 -- 172.21.15.84:0/1183544265 shutdown_connections 2024-05-22T23:02:27.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.586+0000 7f4d9277b740 1 --2- 172.21.15.84:0/1183544265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67865b050 0x55c67865d440 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.586+0000 7f4d9277b740 1 --2- 172.21.15.84:0/1183544265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67864c450 0x55c67865ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.586+0000 7f4d9277b740 1 --2- 172.21.15.84:0/1183544265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67864bb30 0x55c67864bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.586+0000 7f4d9277b740 1 -- 172.21.15.84:0/1183544265 >> 172.21.15.84:0/1183544265 conn(0x55c678647f70 msgr2=0x55c678648350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:27.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.586+0000 7f4d9277b740 1 -- 172.21.15.84:0/1183544265 shutdown_connections 2024-05-22T23:02:27.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.586+0000 7f4d9277b740 1 -- 172.21.15.84:0/1183544265 wait complete. 2024-05-22T23:02:27.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.587+0000 7f4d9277b740 1 -- start start 2024-05-22T23:02:27.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.587+0000 7f4d9277b740 1 Processor -- start 2024-05-22T23:02:27.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.587+0000 7f4d9277b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67864bb30 0x55c678657fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.587+0000 7f4d9277b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67864c450 0x55c678658520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.587+0000 7f4d9277b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67865b050 0x55c678658a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.587+0000 7f4d9277b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c67865dc50 con 0x55c67865b050 2024-05-22T23:02:27.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.587+0000 7f4d9277b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c6785395f0 con 0x55c67864c450 2024-05-22T23:02:27.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.587+0000 7f4d9277b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c67865ddd0 con 0x55c67864bb30 2024-05-22T23:02:27.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.587+0000 7f4d9277b740 10 client.0 _ll_register_callbacks cb 0x55c678534cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:02:27.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.587+0000 7f4d8e008640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67865b050 0x55c678658a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.587+0000 7f4d8e008640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67865b050 0x55c678658a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35888/0 (socket says 172.21.15.84:35888) 2024-05-22T23:02:27.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.587+0000 7f4d8e008640 1 -- 172.21.15.84:0/330233281 learned_addr learned my addr 172.21.15.84:0/330233281 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:27.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.587+0000 7f4d8d006640 1 --2- 172.21.15.84:0/330233281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67864c450 0x55c678658520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.587+0000 7f4d8d807640 1 --2- 172.21.15.84:0/330233281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67864bb30 0x55c678657fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.588+0000 7f4d8e008640 1 -- 172.21.15.84:0/330233281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67864bb30 msgr2=0x55c678657fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:27.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.588+0000 7f4d8e008640 1 --2- 172.21.15.84:0/330233281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67864bb30 0x55c678657fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.588+0000 7f4d8e008640 1 -- 172.21.15.84:0/330233281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67864c450 msgr2=0x55c678658520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:27.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.588+0000 7f4d8e008640 1 --2- 172.21.15.84:0/330233281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67864c450 0x55c678658520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.588+0000 7f4d8e008640 1 -- 172.21.15.84:0/330233281 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c67838b7e0 con 0x55c67865b050 2024-05-22T23:02:27.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.588+0000 7f4d8d006640 1 --2- 172.21.15.84:0/330233281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67864c450 0x55c678658520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:02:27.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.588+0000 7f4d8d807640 1 --2- 172.21.15.84:0/330233281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67864bb30 0x55c678657fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:02:27.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.588+0000 7f4d8e008640 1 --2- 172.21.15.84:0/330233281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67865b050 0x55c678658a60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4d800ee700 tx=0x7f4d80003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef554f87d69b1e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:27.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.588+0000 7f4d717fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:02:27.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.588+0000 7f4d717fa640 1 -- 172.21.15.84:0/330233281 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d800da030 con 0x55c67865b050 2024-05-22T23:02:27.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.588+0000 7f4d717fa640 1 -- 172.21.15.84:0/330233281 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f4d800d6150 con 0x55c67865b050 2024-05-22T23:02:27.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.588+0000 7f4d9277b740 1 -- 172.21.15.84:0/330233281 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d80036070 con 0x55c67865b050 2024-05-22T23:02:27.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.588+0000 7f4d717fa640 1 -- 172.21.15.84:0/330233281 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f4d800b7e60 con 0x55c67865b050 2024-05-22T23:02:27.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.588+0000 7f4d9277b740 10 client.15540 fetch_fsmap learned FSMap version 18 2024-05-22T23:02:27.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.588+0000 7f4d9277b740 1 -- 172.21.15.84:0/330233281 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55c6783ebf70 con 0x55c67865b050 2024-05-22T23:02:27.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.588+0000 7f4d717fa640 1 -- 172.21.15.84:0/330233281 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d80036070 con 0x55c67865b050 2024-05-22T23:02:27.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.588+0000 7f4d717fa640 1 -- 172.21.15.84:0/330233281 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f4d8004e070 con 0x55c67865b050 2024-05-22T23:02:27.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.588+0000 7f4d9277b740 10 client.15540 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:02:27.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.588+0000 7f4d9277b740 10 client.15540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:02:27.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.588+0000 7f4d9277b740 20 client.15540 populate_metadata read hostname 'smithi084' 2024-05-22T23:02:27.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.588+0000 7f4d9277b740 1 --2- 172.21.15.84:0/330233281 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c67866afe0 0x55c67866d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.588+0000 7f4d9277b740 4 client.15540 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:02:27.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.588+0000 7f4d9277b740 1 -- 172.21.15.84:0/330233281 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6785c3460 con 0x55c67866afe0 2024-05-22T23:02:27.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.589+0000 7f4d8d807640 1 --2- 172.21.15.84:0/330233281 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c67866afe0 0x55c67866d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.589+0000 7f4d717fa640 10 client.15540 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:27.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.589+0000 7f4d8d807640 1 --2- 172.21.15.84:0/330233281 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c67866afe0 0x55c67866d3d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9ac41a3316523f1 server_cookie=785618195739c78d in_seq=0 out_seq=0 2024-05-22T23:02:27.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.590+0000 7f4d717fa640 1 -- 172.21.15.84:0/330233281 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c6785c3460 con 0x55c67866afe0 2024-05-22T23:02:27.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.590+0000 7f4d717fa640 10 client.15540 handle_command_reply: tid=0 2024-05-22T23:02:27.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.590+0000 7f4d9277b740 1 client.15540 shutdown 2024-05-22T23:02:27.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.590+0000 7f4d9277b740 1 -- 172.21.15.84:0/330233281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67865b050 msgr2=0x55c678658a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:27.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.590+0000 7f4d9277b740 1 --2- 172.21.15.84:0/330233281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67865b050 0x55c678658a60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4d800ee700 tx=0x7f4d80003f70 comp rx=0 tx=0).stop 2024-05-22T23:02:27.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.590+0000 7f4d9277b740 1 -- 172.21.15.84:0/330233281 shutdown_connections 2024-05-22T23:02:27.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.590+0000 7f4d9277b740 1 --2- 172.21.15.84:0/330233281 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c67866afe0 0x55c67866d3d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.590+0000 7f4d9277b740 1 --2- 172.21.15.84:0/330233281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67865b050 0x55c678658a60 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.590+0000 7f4d9277b740 1 --2- 172.21.15.84:0/330233281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67864c450 0x55c678658520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.590+0000 7f4d9277b740 1 --2- 172.21.15.84:0/330233281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67864bb30 0x55c678657fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.590+0000 7f4d717fa640 0 client.15540 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:27.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.590+0000 7f4d9277b740 1 -- 172.21.15.84:0/330233281 >> 172.21.15.84:0/330233281 conn(0x55c678647f70 msgr2=0x55c67864dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:27.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.590+0000 7f4d9277b740 1 -- 172.21.15.84:0/330233281 shutdown_connections 2024-05-22T23:02:27.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.590+0000 7f4d9277b740 1 -- 172.21.15.84:0/330233281 wait complete. 2024-05-22T23:02:27.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.590+0000 7f4d9277b740 20 client.15540 trim_cache size 0 max 16384 2024-05-22T23:02:27.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.608+0000 7f4d9277b740 1 Processor -- start 2024-05-22T23:02:27.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.608+0000 7f4d9277b740 1 -- start start 2024-05-22T23:02:27.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.608+0000 7f4d9277b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67864bb30 0x55c678666a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.608+0000 7f4d9277b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67864c450 0x55c678658880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.608+0000 7f4d9277b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67865b050 0x55c678647030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.608+0000 7f4d9277b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881a9320 con 0x55c67865b050 2024-05-22T23:02:27.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.608+0000 7f4d9277b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881abba0 con 0x55c67864bb30 2024-05-22T23:02:27.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.608+0000 7f4d9277b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881a9990 con 0x55c67864c450 2024-05-22T23:02:27.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.608+0000 7f4d8e008640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67865b050 0x55c678647030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.608+0000 7f4d8e008640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67865b050 0x55c678647030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35892/0 (socket says 172.21.15.84:35892) 2024-05-22T23:02:27.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.608+0000 7f4d8e008640 1 -- 172.21.15.84:0/2764153669 learned_addr learned my addr 172.21.15.84:0/2764153669 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:27.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.608+0000 7f4d8e008640 1 -- 172.21.15.84:0/2764153669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67864c450 msgr2=0x55c678658880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:27.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.608+0000 7f4d8e008640 1 --2- 172.21.15.84:0/2764153669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67864c450 0x55c678658880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.608+0000 7f4d8d807640 1 --2- 172.21.15.84:0/2764153669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67864bb30 0x55c678666a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.608+0000 7f4d8e008640 1 -- 172.21.15.84:0/2764153669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67864bb30 msgr2=0x55c678666a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:27.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.608+0000 7f4d8e008640 1 --2- 172.21.15.84:0/2764153669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67864bb30 0x55c678666a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.608+0000 7f4d8e008640 1 -- 172.21.15.84:0/2764153669 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c67864c280 con 0x55c67865b050 2024-05-22T23:02:27.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.608+0000 7f4d8d807640 1 --2- 172.21.15.84:0/2764153669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67864bb30 0x55c678666a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:02:27.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.608+0000 7f4d8e008640 1 --2- 172.21.15.84:0/2764153669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67865b050 0x55c678647030 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4d80006010 tx=0x7f4d8002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ed3ba9941e6351 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:27.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.609+0000 7f4d737fe640 1 -- 172.21.15.84:0/2764153669 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d800ee530 con 0x55c67865b050 2024-05-22T23:02:27.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.609+0000 7f4d737fe640 1 -- 172.21.15.84:0/2764153669 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d80036050 con 0x55c67865b050 2024-05-22T23:02:27.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.609+0000 7f4d737fe640 1 -- 172.21.15.84:0/2764153669 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d800c5f60 con 0x55c67865b050 2024-05-22T23:02:27.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.609+0000 7f4d9277b740 1 -- 172.21.15.84:0/2764153669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67865b050 msgr2=0x55c678647030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:27.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.609+0000 7f4d9277b740 1 --2- 172.21.15.84:0/2764153669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67865b050 0x55c678647030 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4d80006010 tx=0x7f4d8002a8e0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.609+0000 7f4d9277b740 1 -- 172.21.15.84:0/2764153669 shutdown_connections 2024-05-22T23:02:27.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.609+0000 7f4d9277b740 1 --2- 172.21.15.84:0/2764153669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67865b050 0x55c678647030 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.609+0000 7f4d9277b740 1 --2- 172.21.15.84:0/2764153669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67864c450 0x55c678658880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.609+0000 7f4d9277b740 1 --2- 172.21.15.84:0/2764153669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67864bb30 0x55c678666a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.609+0000 7f4d9277b740 1 -- 172.21.15.84:0/2764153669 >> 172.21.15.84:0/2764153669 conn(0x55c678647f70 msgr2=0x55c678646920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:27.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.609+0000 7f4d9277b740 1 -- 172.21.15.84:0/2764153669 shutdown_connections 2024-05-22T23:02:27.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.609+0000 7f4d9277b740 1 -- 172.21.15.84:0/2764153669 wait complete. 2024-05-22T23:02:27.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.609+0000 7f4d9277b740 1 -- start start 2024-05-22T23:02:27.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.610+0000 7f4d9277b740 1 Processor -- start 2024-05-22T23:02:27.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.610+0000 7f4d9277b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67864bb30 0x55c678663490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.610+0000 7f4d9277b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67864c450 0x55c67865e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.610+0000 7f4d9277b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67865b050 0x55c67865ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.610+0000 7f4d9277b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881aa060 con 0x55c67864c450 2024-05-22T23:02:27.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.610+0000 7f4d9277b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881ab4d0 con 0x55c67864bb30 2024-05-22T23:02:27.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.610+0000 7f4d9277b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881aa730 con 0x55c67865b050 2024-05-22T23:02:27.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.610+0000 7f4d9277b740 10 client.0 _ll_register_callbacks cb 0x7f4d7800a980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:02:27.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.610+0000 7f4d8d006640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67864c450 0x55c67865e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.610+0000 7f4d8d006640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67864c450 0x55c67865e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35896/0 (socket says 172.21.15.84:35896) 2024-05-22T23:02:27.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.610+0000 7f4d8d006640 1 -- 172.21.15.84:0/274225712 learned_addr learned my addr 172.21.15.84:0/274225712 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:27.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.610+0000 7f4d8d006640 1 -- 172.21.15.84:0/274225712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67865b050 msgr2=0x55c67865ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:27.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.610+0000 7f4d8d006640 1 --2- 172.21.15.84:0/274225712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67865b050 0x55c67865ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.610+0000 7f4d8d006640 1 -- 172.21.15.84:0/274225712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67864bb30 msgr2=0x55c678663490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:27.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.610+0000 7f4d8d006640 1 --2- 172.21.15.84:0/274225712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67864bb30 0x55c678663490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.610+0000 7f4d8d006640 1 -- 172.21.15.84:0/274225712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c678283170 con 0x55c67864c450 2024-05-22T23:02:27.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.611+0000 7f4d8d006640 1 --2- 172.21.15.84:0/274225712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67864c450 0x55c67865e850 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4d78005fd0 tx=0x7f4d78004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5141a3a415ff3c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:27.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.611+0000 7f4d717fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:02:27.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.611+0000 7f4d717fa640 1 -- 172.21.15.84:0/274225712 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78025070 con 0x55c67864c450 2024-05-22T23:02:27.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.611+0000 7f4d717fa640 1 -- 172.21.15.84:0/274225712 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f4d7801ecc0 con 0x55c67864c450 2024-05-22T23:02:27.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.611+0000 7f4d9277b740 1 -- 172.21.15.84:0/274225712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d80036050 con 0x55c67864c450 2024-05-22T23:02:27.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.611+0000 7f4d717fa640 1 -- 172.21.15.84:0/274225712 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f4d78032460 con 0x55c67864c450 2024-05-22T23:02:27.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.611+0000 7f4d9277b740 10 client.15546 fetch_fsmap learned FSMap version 18 2024-05-22T23:02:27.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.611+0000 7f4d9277b740 1 -- 172.21.15.84:0/274225712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55c678578930 con 0x55c67864c450 2024-05-22T23:02:27.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.611+0000 7f4d717fa640 1 -- 172.21.15.84:0/274225712 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d80036050 con 0x55c67864c450 2024-05-22T23:02:27.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.611+0000 7f4d717fa640 1 -- 172.21.15.84:0/274225712 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f4d7800f040 con 0x55c67864c450 2024-05-22T23:02:27.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.611+0000 7f4d9277b740 10 client.15546 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:02:27.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.611+0000 7f4d9277b740 10 client.15546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:02:27.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.611+0000 7f4d9277b740 20 client.15546 populate_metadata read hostname 'smithi084' 2024-05-22T23:02:27.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.611+0000 7f4d9277b740 1 --2- 172.21.15.84:0/274225712 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c6786d19f0 0x55c6786d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.611+0000 7f4d9277b740 4 client.15546 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-22T23:02:27.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.611+0000 7f4d9277b740 1 -- 172.21.15.84:0/274225712 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c6785bd170 con 0x55c6786d19f0 2024-05-22T23:02:27.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.611+0000 7f4d8d807640 1 --2- 172.21.15.84:0/274225712 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c6786d19f0 0x55c6786d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.612+0000 7f4d717fa640 10 client.15546 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:27.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.612+0000 7f4d8d807640 1 --2- 172.21.15.84:0/274225712 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c6786d19f0 0x55c6786d3e10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f24dee4deb7fecb server_cookie=301e7f668402fbc9 in_seq=0 out_seq=0 2024-05-22T23:02:27.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.613+0000 7f4d717fa640 1 -- 172.21.15.84:0/274225712 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c6785bd170 con 0x55c6786d19f0 2024-05-22T23:02:27.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.613+0000 7f4d717fa640 10 client.15546 handle_command_reply: tid=0 2024-05-22T23:02:27.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.613+0000 7f4d9277b740 1 client.15546 shutdown 2024-05-22T23:02:27.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.613+0000 7f4d9277b740 1 -- 172.21.15.84:0/274225712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67864c450 msgr2=0x55c67865e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:27.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.613+0000 7f4d9277b740 1 --2- 172.21.15.84:0/274225712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67864c450 0x55c67865e850 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4d78005fd0 tx=0x7f4d78004730 comp rx=0 tx=0).stop 2024-05-22T23:02:27.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.613+0000 7f4d9277b740 1 -- 172.21.15.84:0/274225712 shutdown_connections 2024-05-22T23:02:27.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.613+0000 7f4d9277b740 1 --2- 172.21.15.84:0/274225712 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c6786d19f0 0x55c6786d3e10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.613+0000 7f4d9277b740 1 --2- 172.21.15.84:0/274225712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67865b050 0x55c67865ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.613+0000 7f4d9277b740 1 --2- 172.21.15.84:0/274225712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67864c450 0x55c67865e850 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.613+0000 7f4d9277b740 1 --2- 172.21.15.84:0/274225712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67864bb30 0x55c678663490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.613+0000 7f4d717fa640 0 client.15546 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:27.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.613+0000 7f4d9277b740 1 -- 172.21.15.84:0/274225712 >> 172.21.15.84:0/274225712 conn(0x55c678647f70 msgr2=0x55c6786471b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:27.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.613+0000 7f4d9277b740 1 -- 172.21.15.84:0/274225712 shutdown_connections 2024-05-22T23:02:27.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.613+0000 7f4d9277b740 1 -- 172.21.15.84:0/274225712 wait complete. 2024-05-22T23:02:27.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.613+0000 7f4d9277b740 20 client.15546 trim_cache size 0 max 16384 2024-05-22T23:02:27.621 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:02:27.622 INFO:teuthology.orchestra.run.smithi084.stdout: "return_code": 0, 2024-05-22T23:02:27.622 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_tag": "ab83cd97-9994-4866-adc1-d4b616ea165b", 2024-05-22T23:02:27.622 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": "asynchronous" 2024-05-22T23:02:27.622 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:02:27.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.614+0000 7f4d8fa6b640 1 -- 172.21.15.84:0/1298504802 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f4d740a2e20 msgr2=0x7f4d740a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:27.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.614+0000 7f4d8fa6b640 1 --2- 172.21.15.84:0/1298504802 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f4d740a2e20 0x7f4d740a52e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4d78005e10 tx=0x7f4d780128d0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.614+0000 7f4d8fa6b640 1 -- 172.21.15.84:0/1298504802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d88107850 msgr2=0x7f4d8819c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:27.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.614+0000 7f4d8fa6b640 1 --2- 172.21.15.84:0/1298504802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d88107850 0x7f4d8819c920 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4d80005cc0 tx=0x7f4d80003970 comp rx=0 tx=0).stop 2024-05-22T23:02:27.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.614+0000 7f4d8fa6b640 1 -- 172.21.15.84:0/1298504802 shutdown_connections 2024-05-22T23:02:27.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.614+0000 7f4d8fa6b640 1 --2- 172.21.15.84:0/1298504802 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f4d740a2e20 0x7f4d740a52e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.614+0000 7f4d8fa6b640 1 --2- 172.21.15.84:0/1298504802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d88107850 0x7f4d8819c920 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.614+0000 7f4d8fa6b640 1 --2- 172.21.15.84:0/1298504802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d88102200 0x7f4d881985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.614+0000 7f4d8fa6b640 1 --2- 172.21.15.84:0/1298504802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d88101840 0x7f4d88198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.614+0000 7f4d8fa6b640 1 -- 172.21.15.84:0/1298504802 >> 172.21.15.84:0/1298504802 conn(0x7f4d880fd2b0 msgr2=0x7f4d880fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:27.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.614+0000 7f4d8fa6b640 1 -- 172.21.15.84:0/1298504802 shutdown_connections 2024-05-22T23:02:27.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.614+0000 7f4d8fa6b640 1 -- 172.21.15.84:0/1298504802 wait complete. 2024-05-22T23:02:27.627 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:02:27.627 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:02:27.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.728+0000 7f48a1f09640 1 -- 172.21.15.84:0/1727732481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f489c101460 msgr2=0x7f489c10d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:27.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.728+0000 7f48a1f09640 1 --2- 172.21.15.84:0/1727732481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f489c101460 0x7f489c10d590 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4894009900 tx=0x7f489402a410 comp rx=0 tx=0).stop 2024-05-22T23:02:27.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.728+0000 7f48a1f09640 1 -- 172.21.15.84:0/1727732481 shutdown_connections 2024-05-22T23:02:27.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.728+0000 7f48a1f09640 1 --2- 172.21.15.84:0/1727732481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f489c101460 0x7f489c10d590 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.728+0000 7f48a1f09640 1 --2- 172.21.15.84:0/1727732481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f489c100ac0 0x7f489c100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.728+0000 7f48a1f09640 1 --2- 172.21.15.84:0/1727732481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f489c1067c0 0x7f489c106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.728+0000 7f48a1f09640 1 -- 172.21.15.84:0/1727732481 >> 172.21.15.84:0/1727732481 conn(0x7f489c076570 msgr2=0x7f489c076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:27.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.728+0000 7f48a1f09640 1 -- 172.21.15.84:0/1727732481 shutdown_connections 2024-05-22T23:02:27.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.728+0000 7f48a1f09640 1 -- 172.21.15.84:0/1727732481 wait complete. 2024-05-22T23:02:27.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.729+0000 7f48a1f09640 1 Processor -- start 2024-05-22T23:02:27.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.729+0000 7f48a1f09640 1 -- start start 2024-05-22T23:02:27.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.729+0000 7f48a1f09640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f489c100ac0 0x7f489c1021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.729+0000 7f48a1f09640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f489c101460 0x7f489c102730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.729+0000 7f48a1f09640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f489c1067c0 0x7f489c102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.729+0000 7f48a1f09640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f489c110b10 con 0x7f489c1067c0 2024-05-22T23:02:27.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.729+0000 7f48a1f09640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f489c110990 con 0x7f489c101460 2024-05-22T23:02:27.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.729+0000 7f48a1f09640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f489c110c90 con 0x7f489c100ac0 2024-05-22T23:02:27.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.729+0000 7f489bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f489c1067c0 0x7f489c102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.730+0000 7f489bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f489c1067c0 0x7f489c102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35914/0 (socket says 172.21.15.84:35914) 2024-05-22T23:02:27.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.730+0000 7f489bfff640 1 -- 172.21.15.84:0/991318148 learned_addr learned my addr 172.21.15.84:0/991318148 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:27.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.730+0000 7f489b7fe640 1 --2- 172.21.15.84:0/991318148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f489c100ac0 0x7f489c1021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.730+0000 7f489affd640 1 --2- 172.21.15.84:0/991318148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f489c101460 0x7f489c102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.730+0000 7f489bfff640 1 -- 172.21.15.84:0/991318148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f489c100ac0 msgr2=0x7f489c1021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:27.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.730+0000 7f489bfff640 1 --2- 172.21.15.84:0/991318148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f489c100ac0 0x7f489c1021f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.730+0000 7f489bfff640 1 -- 172.21.15.84:0/991318148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f489c101460 msgr2=0x7f489c102730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:27.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.730+0000 7f489bfff640 1 --2- 172.21.15.84:0/991318148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f489c101460 0x7f489c102730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.730+0000 7f489bfff640 1 -- 172.21.15.84:0/991318148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489c1a64e0 con 0x7f489c1067c0 2024-05-22T23:02:27.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.730+0000 7f489bfff640 1 --2- 172.21.15.84:0/991318148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f489c1067c0 0x7f489c102c70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4894005ab0 tx=0x7f48940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:27.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.730+0000 7f4898ff9640 1 -- 172.21.15.84:0/991318148 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489403f030 con 0x7f489c1067c0 2024-05-22T23:02:27.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.730+0000 7f4898ff9640 1 -- 172.21.15.84:0/991318148 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4894003ce0 con 0x7f489c1067c0 2024-05-22T23:02:27.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.730+0000 7f48a1f09640 1 -- 172.21.15.84:0/991318148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489c1a6770 con 0x7f489c1067c0 2024-05-22T23:02:27.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.730+0000 7f4898ff9640 1 -- 172.21.15.84:0/991318148 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894035070 con 0x7f489c1067c0 2024-05-22T23:02:27.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.730+0000 7f48a1f09640 1 -- 172.21.15.84:0/991318148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f489c1a6c60 con 0x7f489c1067c0 2024-05-22T23:02:27.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.732+0000 7f4898ff9640 1 -- 172.21.15.84:0/991318148 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f4894042030 con 0x7f489c1067c0 2024-05-22T23:02:27.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.733+0000 7f48a4c11740 1 Processor -- start 2024-05-22T23:02:27.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.733+0000 7f48a4c11740 1 -- start start 2024-05-22T23:02:27.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.733+0000 7f4898ff9640 1 --2- 172.21.15.84:0/991318148 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f48840a2e20 0x7f48840a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.733+0000 7f48a4c11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8d6fed4a0 0x55f8d6feb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.733+0000 7f48a4c11740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d6febdb0 0x55f8d6fec230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.733+0000 7f48a4c11740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6ff6b30 0x55f8d6ff8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.733+0000 7f48a4c11740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f8d6f4fdb0 con 0x55f8d6ff6b30 2024-05-22T23:02:27.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.733+0000 7f48a4c11740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f8d6eee360 con 0x55f8d6febdb0 2024-05-22T23:02:27.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.733+0000 7f48a4c11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f8d6f4bd70 con 0x55f8d6fed4a0 2024-05-22T23:02:27.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.733+0000 7f489affd640 1 --2- 172.21.15.84:0/991318148 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f48840a2e20 0x7f48840a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.733+0000 7f4898ff9640 1 -- 172.21.15.84:0/991318148 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f48940efda0 con 0x7f489c1067c0 2024-05-22T23:02:27.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.733+0000 7f489bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6ff6b30 0x55f8d6ff8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.733+0000 7f489bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6ff6b30 0x55f8d6ff8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35924/0 (socket says 172.21.15.84:35924) 2024-05-22T23:02:27.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.733+0000 7f489bfff640 1 -- 172.21.15.84:0/3460223011 learned_addr learned my addr 172.21.15.84:0/3460223011 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:27.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.733+0000 7f489b7fe640 1 --2- 172.21.15.84:0/3460223011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8d6fed4a0 0x55f8d6feb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.733+0000 7f489affd640 1 --2- 172.21.15.84:0/3460223011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d6febdb0 0x55f8d6fec230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.733+0000 7f489bfff640 1 -- 172.21.15.84:0/3460223011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8d6fed4a0 msgr2=0x55f8d6feb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:27.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.733+0000 7f489bfff640 1 --2- 172.21.15.84:0/3460223011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8d6fed4a0 0x55f8d6feb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.733+0000 7f489bfff640 1 -- 172.21.15.84:0/3460223011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d6febdb0 msgr2=0x55f8d6fec230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:27.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.733+0000 7f489affd640 1 --2- 172.21.15.84:0/991318148 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f48840a2e20 0x7f48840a52e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f489c100450 tx=0x7f4890014c80 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:27.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.733+0000 7f489bfff640 1 --2- 172.21.15.84:0/3460223011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d6febdb0 0x55f8d6fec230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.733+0000 7f489bfff640 1 -- 172.21.15.84:0/3460223011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8d6f11a60 con 0x55f8d6ff6b30 2024-05-22T23:02:27.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.734+0000 7f489bfff640 1 --2- 172.21.15.84:0/3460223011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6ff6b30 0x55f8d6ff8f20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4894009a20 tx=0x7f48940b8490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da65e77346fdbb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:27.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.734+0000 7f4881ffb640 1 -- 172.21.15.84:0/3460223011 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894049040 con 0x55f8d6ff6b30 2024-05-22T23:02:27.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.734+0000 7f4881ffb640 1 -- 172.21.15.84:0/3460223011 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48940bc1f0 con 0x55f8d6ff6b30 2024-05-22T23:02:27.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.734+0000 7f4881ffb640 1 -- 172.21.15.84:0/3460223011 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48940491e0 con 0x55f8d6ff6b30 2024-05-22T23:02:27.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.734+0000 7f48a4c11740 1 -- 172.21.15.84:0/3460223011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6ff6b30 msgr2=0x55f8d6ff8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:27.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.734+0000 7f48a4c11740 1 --2- 172.21.15.84:0/3460223011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6ff6b30 0x55f8d6ff8f20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4894009a20 tx=0x7f48940b8490 comp rx=0 tx=0).stop 2024-05-22T23:02:27.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.734+0000 7f48a4c11740 1 -- 172.21.15.84:0/3460223011 shutdown_connections 2024-05-22T23:02:27.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.734+0000 7f48a4c11740 1 --2- 172.21.15.84:0/3460223011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6ff6b30 0x55f8d6ff8f20 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.734+0000 7f48a4c11740 1 --2- 172.21.15.84:0/3460223011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d6febdb0 0x55f8d6fec230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.734+0000 7f48a4c11740 1 --2- 172.21.15.84:0/3460223011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8d6fed4a0 0x55f8d6feb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.734+0000 7f48a4c11740 1 -- 172.21.15.84:0/3460223011 >> 172.21.15.84:0/3460223011 conn(0x55f8d6fb48e0 msgr2=0x55f8d6fe8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:27.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.735+0000 7f48a4c11740 1 -- 172.21.15.84:0/3460223011 shutdown_connections 2024-05-22T23:02:27.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.735+0000 7f48a4c11740 1 -- 172.21.15.84:0/3460223011 wait complete. 2024-05-22T23:02:27.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.735+0000 7f48a4c11740 1 -- start start 2024-05-22T23:02:27.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.735+0000 7f48a4c11740 1 Processor -- start 2024-05-22T23:02:27.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.735+0000 7f48a4c11740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d6febdb0 0x55f8d6ff3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.735+0000 7f48a4c11740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6fed4a0 0x55f8d6ff3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.735+0000 7f48a4c11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8d6ff6b30 0x55f8d6ff43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.735+0000 7f48a4c11740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f8d6ff9780 con 0x55f8d6fed4a0 2024-05-22T23:02:27.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.735+0000 7f48a4c11740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f8d6f0a0f0 con 0x55f8d6febdb0 2024-05-22T23:02:27.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.735+0000 7f48a4c11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f8d6ff9900 con 0x55f8d6ff6b30 2024-05-22T23:02:27.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.735+0000 7f48a4c11740 10 client.0 _ll_register_callbacks cb 0x55f8d6f0ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:02:27.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.735+0000 7f489affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6fed4a0 0x55f8d6ff3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.735+0000 7f489affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6fed4a0 0x55f8d6ff3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35938/0 (socket says 172.21.15.84:35938) 2024-05-22T23:02:27.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.735+0000 7f489affd640 1 -- 172.21.15.84:0/152260277 learned_addr learned my addr 172.21.15.84:0/152260277 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:27.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.735+0000 7f489b7fe640 1 --2- 172.21.15.84:0/152260277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d6febdb0 0x55f8d6ff3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.736+0000 7f489bfff640 1 --2- 172.21.15.84:0/152260277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8d6ff6b30 0x55f8d6ff43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.736+0000 7f489affd640 1 -- 172.21.15.84:0/152260277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8d6ff6b30 msgr2=0x55f8d6ff43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:27.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.736+0000 7f489affd640 1 --2- 172.21.15.84:0/152260277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8d6ff6b30 0x55f8d6ff43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.736+0000 7f489affd640 1 -- 172.21.15.84:0/152260277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d6febdb0 msgr2=0x55f8d6ff3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:27.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.736+0000 7f489affd640 1 --2- 172.21.15.84:0/152260277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d6febdb0 0x55f8d6ff3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.736+0000 7f489affd640 1 -- 172.21.15.84:0/152260277 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8d6d2d7d0 con 0x55f8d6fed4a0 2024-05-22T23:02:27.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.736+0000 7f489affd640 1 --2- 172.21.15.84:0/152260277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6fed4a0 0x55f8d6ff3e90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4890012f30 tx=0x7f489001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e2129b80e1addf server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:27.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.736+0000 7f487affd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:02:27.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.736+0000 7f487affd640 1 -- 172.21.15.84:0/152260277 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4890004750 con 0x55f8d6fed4a0 2024-05-22T23:02:27.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.736+0000 7f487affd640 1 -- 172.21.15.84:0/152260277 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f4890020440 con 0x55f8d6fed4a0 2024-05-22T23:02:27.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.736+0000 7f48a4c11740 1 -- 172.21.15.84:0/152260277 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48940bc1f0 con 0x55f8d6fed4a0 2024-05-22T23:02:27.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.736+0000 7f487affd640 1 -- 172.21.15.84:0/152260277 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f4890020640 con 0x55f8d6fed4a0 2024-05-22T23:02:27.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.736+0000 7f48a4c11740 10 client.15558 fetch_fsmap learned FSMap version 18 2024-05-22T23:02:27.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.736+0000 7f48a4c11740 1 -- 172.21.15.84:0/152260277 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55f8d6c633f0 con 0x55f8d6fed4a0 2024-05-22T23:02:27.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.736+0000 7f487affd640 1 -- 172.21.15.84:0/152260277 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48940bc1f0 con 0x55f8d6fed4a0 2024-05-22T23:02:27.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.736+0000 7f487affd640 1 -- 172.21.15.84:0/152260277 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f489000f040 con 0x55f8d6fed4a0 2024-05-22T23:02:27.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.736+0000 7f48a4c11740 10 client.15558 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:02:27.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.736+0000 7f48a4c11740 10 client.15558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:02:27.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.736+0000 7f48a4c11740 20 client.15558 populate_metadata read hostname 'smithi084' 2024-05-22T23:02:27.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.737+0000 7f48a4c11740 1 --2- 172.21.15.84:0/152260277 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55f8d7006bb0 0x55f8d7009000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.737+0000 7f48a4c11740 4 client.15558 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:02:27.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.737+0000 7f48a4c11740 1 -- 172.21.15.84:0/152260277 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8d6f5d4e0 con 0x55f8d7006bb0 2024-05-22T23:02:27.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.737+0000 7f489b7fe640 1 --2- 172.21.15.84:0/152260277 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55f8d7006bb0 0x55f8d7009000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.737+0000 7f487affd640 10 client.15558 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:27.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.737+0000 7f489b7fe640 1 --2- 172.21.15.84:0/152260277 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55f8d7006bb0 0x55f8d7009000 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=669b0eab8155b191 server_cookie=f702135ff6878941 in_seq=0 out_seq=0 2024-05-22T23:02:27.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.738+0000 7f487affd640 1 -- 172.21.15.84:0/152260277 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f8d6f5d4e0 con 0x55f8d7006bb0 2024-05-22T23:02:27.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.738+0000 7f487affd640 10 client.15558 handle_command_reply: tid=0 2024-05-22T23:02:27.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.738+0000 7f48a4c11740 1 client.15558 shutdown 2024-05-22T23:02:27.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.738+0000 7f48a4c11740 1 -- 172.21.15.84:0/152260277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6fed4a0 msgr2=0x55f8d6ff3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:27.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.738+0000 7f48a4c11740 1 --2- 172.21.15.84:0/152260277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6fed4a0 0x55f8d6ff3e90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4890012f30 tx=0x7f489001a000 comp rx=0 tx=0).stop 2024-05-22T23:02:27.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.738+0000 7f48a4c11740 1 -- 172.21.15.84:0/152260277 shutdown_connections 2024-05-22T23:02:27.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.738+0000 7f48a4c11740 1 --2- 172.21.15.84:0/152260277 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55f8d7006bb0 0x55f8d7009000 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.738+0000 7f48a4c11740 1 --2- 172.21.15.84:0/152260277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8d6ff6b30 0x55f8d6ff43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.738+0000 7f48a4c11740 1 --2- 172.21.15.84:0/152260277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6fed4a0 0x55f8d6ff3e90 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.738+0000 7f487affd640 0 client.15558 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:27.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.738+0000 7f48a4c11740 1 --2- 172.21.15.84:0/152260277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d6febdb0 0x55f8d6ff3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.738+0000 7f48a4c11740 1 -- 172.21.15.84:0/152260277 >> 172.21.15.84:0/152260277 conn(0x55f8d6fb48e0 msgr2=0x55f8d6fee180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:27.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.738+0000 7f48a4c11740 1 -- 172.21.15.84:0/152260277 shutdown_connections 2024-05-22T23:02:27.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.738+0000 7f48a4c11740 1 -- 172.21.15.84:0/152260277 wait complete. 2024-05-22T23:02:27.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.738+0000 7f48a4c11740 20 client.15558 trim_cache size 0 max 16384 2024-05-22T23:02:27.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.755+0000 7f48a4c11740 1 Processor -- start 2024-05-22T23:02:27.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.755+0000 7f48a4c11740 1 -- start start 2024-05-22T23:02:27.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.755+0000 7f48a4c11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8d6febdb0 0x55f8d6ff4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.755+0000 7f48a4c11740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d6fed4a0 0x55f8d700e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.755+0000 7f48a4c11740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6ff6b30 0x55f8d700ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.755+0000 7f48a4c11740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f489c1ad7b0 con 0x55f8d6ff6b30 2024-05-22T23:02:27.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.755+0000 7f48a4c11740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f489c1b0030 con 0x55f8d6fed4a0 2024-05-22T23:02:27.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.755+0000 7f48a4c11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f489c1ade20 con 0x55f8d6febdb0 2024-05-22T23:02:27.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.755+0000 7f489bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6ff6b30 0x55f8d700ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.755+0000 7f489bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6ff6b30 0x55f8d700ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35942/0 (socket says 172.21.15.84:35942) 2024-05-22T23:02:27.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.755+0000 7f489bfff640 1 -- 172.21.15.84:0/2419050308 learned_addr learned my addr 172.21.15.84:0/2419050308 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:27.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.756+0000 7f489affd640 1 --2- 172.21.15.84:0/2419050308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d6fed4a0 0x55f8d700e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.756+0000 7f489b7fe640 1 --2- 172.21.15.84:0/2419050308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8d6febdb0 0x55f8d6ff4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.756+0000 7f489bfff640 1 -- 172.21.15.84:0/2419050308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8d6febdb0 msgr2=0x55f8d6ff4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:27.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.756+0000 7f489bfff640 1 --2- 172.21.15.84:0/2419050308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8d6febdb0 0x55f8d6ff4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.756+0000 7f489bfff640 1 -- 172.21.15.84:0/2419050308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d6fed4a0 msgr2=0x55f8d700e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:27.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.756+0000 7f489bfff640 1 --2- 172.21.15.84:0/2419050308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d6fed4a0 0x55f8d700e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.756+0000 7f489bfff640 1 -- 172.21.15.84:0/2419050308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8d6c230f0 con 0x55f8d6ff6b30 2024-05-22T23:02:27.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.756+0000 7f489b7fe640 1 --2- 172.21.15.84:0/2419050308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8d6febdb0 0x55f8d6ff4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:02:27.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.756+0000 7f489bfff640 1 --2- 172.21.15.84:0/2419050308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6ff6b30 0x55f8d700ee10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f489404b000 tx=0x7f4894005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a6dc7ab90f0678 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:27.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.756+0000 7f48817fa640 1 -- 172.21.15.84:0/2419050308 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894049070 con 0x55f8d6ff6b30 2024-05-22T23:02:27.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.756+0000 7f48817fa640 1 -- 172.21.15.84:0/2419050308 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48940bc1f0 con 0x55f8d6ff6b30 2024-05-22T23:02:27.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.756+0000 7f48817fa640 1 -- 172.21.15.84:0/2419050308 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48940b8860 con 0x55f8d6ff6b30 2024-05-22T23:02:27.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.757+0000 7f48a4c11740 1 -- 172.21.15.84:0/2419050308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6ff6b30 msgr2=0x55f8d700ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:27.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.757+0000 7f48a4c11740 1 --2- 172.21.15.84:0/2419050308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6ff6b30 0x55f8d700ee10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f489404b000 tx=0x7f4894005be0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.757+0000 7f48a4c11740 1 -- 172.21.15.84:0/2419050308 shutdown_connections 2024-05-22T23:02:27.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.757+0000 7f48a4c11740 1 --2- 172.21.15.84:0/2419050308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6ff6b30 0x55f8d700ee10 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.757+0000 7f48a4c11740 1 --2- 172.21.15.84:0/2419050308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d6fed4a0 0x55f8d700e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.757+0000 7f48a4c11740 1 --2- 172.21.15.84:0/2419050308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8d6febdb0 0x55f8d6ff4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.757+0000 7f48a4c11740 1 -- 172.21.15.84:0/2419050308 >> 172.21.15.84:0/2419050308 conn(0x55f8d6fb48e0 msgr2=0x55f8d6ff3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:27.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.757+0000 7f48a4c11740 1 -- 172.21.15.84:0/2419050308 shutdown_connections 2024-05-22T23:02:27.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.757+0000 7f48a4c11740 1 -- 172.21.15.84:0/2419050308 wait complete. 2024-05-22T23:02:27.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.757+0000 7f48a4c11740 1 -- start start 2024-05-22T23:02:27.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.757+0000 7f48a4c11740 1 Processor -- start 2024-05-22T23:02:27.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.757+0000 7f48a4c11740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d6febdb0 0x55f8d6ff0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.757+0000 7f48a4c11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8d6fed4a0 0x55f8d6ff0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.757+0000 7f48a4c11740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6ff6b30 0x55f8d6ff10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.757+0000 7f48a4c11740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f489c1ae4f0 con 0x55f8d6ff6b30 2024-05-22T23:02:27.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.757+0000 7f48a4c11740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f489c1af960 con 0x55f8d6febdb0 2024-05-22T23:02:27.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.757+0000 7f48a4c11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f489c1aebc0 con 0x55f8d6fed4a0 2024-05-22T23:02:27.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.757+0000 7f48a4c11740 10 client.0 _ll_register_callbacks cb 0x7f4890012dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:02:27.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.757+0000 7f489bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6ff6b30 0x55f8d6ff10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.758+0000 7f489bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6ff6b30 0x55f8d6ff10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35952/0 (socket says 172.21.15.84:35952) 2024-05-22T23:02:27.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.758+0000 7f489b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d6febdb0 0x55f8d6ff0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.758+0000 7f489bfff640 1 -- 172.21.15.84:0/103625823 learned_addr learned my addr 172.21.15.84:0/103625823 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:27.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.758+0000 7f489affd640 1 --2- 172.21.15.84:0/103625823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8d6fed4a0 0x55f8d6ff0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.758+0000 7f489bfff640 1 -- 172.21.15.84:0/103625823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8d6fed4a0 msgr2=0x55f8d6ff0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:27.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.758+0000 7f489bfff640 1 --2- 172.21.15.84:0/103625823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8d6fed4a0 0x55f8d6ff0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.758+0000 7f489bfff640 1 -- 172.21.15.84:0/103625823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d6febdb0 msgr2=0x55f8d6ff0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:27.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.758+0000 7f489bfff640 1 --2- 172.21.15.84:0/103625823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d6febdb0 0x55f8d6ff0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.758+0000 7f489bfff640 1 -- 172.21.15.84:0/103625823 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8d6f0efe0 con 0x55f8d6ff6b30 2024-05-22T23:02:27.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.758+0000 7f489b7fe640 1 --2- 172.21.15.84:0/103625823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d6febdb0 0x55f8d6ff0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:02:27.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.758+0000 7f489bfff640 1 --2- 172.21.15.84:0/103625823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6ff6b30 0x55f8d6ff10a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f48940bf040 tx=0x7f48940c05b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44302ff45e790512 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:27.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.758+0000 7f487affd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:02:27.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.758+0000 7f487affd640 1 -- 172.21.15.84:0/103625823 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48940d4070 con 0x55f8d6ff6b30 2024-05-22T23:02:27.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.758+0000 7f487affd640 1 -- 172.21.15.84:0/103625823 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f48940b8860 con 0x55f8d6ff6b30 2024-05-22T23:02:27.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.758+0000 7f48a4c11740 1 -- 172.21.15.84:0/103625823 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48940bc1f0 con 0x55f8d6ff6b30 2024-05-22T23:02:27.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.758+0000 7f487affd640 1 -- 172.21.15.84:0/103625823 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f489409e020 con 0x55f8d6ff6b30 2024-05-22T23:02:27.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.758+0000 7f48a4c11740 10 client.15564 fetch_fsmap learned FSMap version 18 2024-05-22T23:02:27.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.758+0000 7f48a4c11740 1 -- 172.21.15.84:0/103625823 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55f8d6ee01c0 con 0x55f8d6ff6b30 2024-05-22T23:02:27.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.758+0000 7f487affd640 1 -- 172.21.15.84:0/103625823 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48940bc1f0 con 0x55f8d6ff6b30 2024-05-22T23:02:27.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.759+0000 7f487affd640 1 -- 172.21.15.84:0/103625823 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f489403bc10 con 0x55f8d6ff6b30 2024-05-22T23:02:27.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.759+0000 7f48a4c11740 10 client.15564 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:02:27.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.759+0000 7f48a4c11740 10 client.15564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:02:27.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.759+0000 7f48a4c11740 20 client.15564 populate_metadata read hostname 'smithi084' 2024-05-22T23:02:27.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.759+0000 7f48a4c11740 1 --2- 172.21.15.84:0/103625823 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55f8d706d310 0x55f8d706f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:27.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.759+0000 7f48a4c11740 4 client.15564 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:02:27.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.759+0000 7f48a4c11740 1 -- 172.21.15.84:0/103625823 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f8d6f57830 con 0x55f8d706d310 2024-05-22T23:02:27.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.759+0000 7f489b7fe640 1 --2- 172.21.15.84:0/103625823 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55f8d706d310 0x55f8d706f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:27.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.759+0000 7f489b7fe640 1 --2- 172.21.15.84:0/103625823 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55f8d706d310 0x55f8d706f700 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea2f0dc445a90901 server_cookie=edfd2414d4a6f353 in_seq=0 out_seq=0 2024-05-22T23:02:27.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.759+0000 7f487affd640 10 client.15564 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:27.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f487affd640 1 -- 172.21.15.84:0/103625823 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f8d6f57830 con 0x55f8d706d310 2024-05-22T23:02:27.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f487affd640 10 client.15564 handle_command_reply: tid=0 2024-05-22T23:02:27.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f48a4c11740 1 client.15564 shutdown 2024-05-22T23:02:27.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f48a4c11740 1 -- 172.21.15.84:0/103625823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6ff6b30 msgr2=0x55f8d6ff10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:27.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f48a4c11740 1 --2- 172.21.15.84:0/103625823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6ff6b30 0x55f8d6ff10a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f48940bf040 tx=0x7f48940c05b0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f48a4c11740 1 -- 172.21.15.84:0/103625823 shutdown_connections 2024-05-22T23:02:27.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f48a4c11740 1 --2- 172.21.15.84:0/103625823 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55f8d706d310 0x55f8d706f700 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f48a4c11740 1 --2- 172.21.15.84:0/103625823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8d6ff6b30 0x55f8d6ff10a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f48a4c11740 1 --2- 172.21.15.84:0/103625823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f8d6fed4a0 0x55f8d6ff0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f487affd640 0 client.15564 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:27.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f48a4c11740 1 --2- 172.21.15.84:0/103625823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d6febdb0 0x55f8d6ff0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f48a4c11740 1 -- 172.21.15.84:0/103625823 >> 172.21.15.84:0/103625823 conn(0x55f8d6fb48e0 msgr2=0x55f8d6fee160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:27.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f48a4c11740 1 -- 172.21.15.84:0/103625823 shutdown_connections 2024-05-22T23:02:27.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f48a4c11740 1 -- 172.21.15.84:0/103625823 wait complete. 2024-05-22T23:02:27.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f48a4c11740 20 client.15564 trim_cache size 0 max 16384 2024-05-22T23:02:27.769 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:02:27.770 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-22T23:02:27.770 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": { 2024-05-22T23:02:27.770 INFO:teuthology.orchestra.run.smithi084.stdout: "ab83cd97-9994-4866-adc1-d4b616ea165b": { 2024-05-22T23:02:27.770 INFO:teuthology.orchestra.run.smithi084.stdout: "path": "/", 2024-05-22T23:02:27.770 INFO:teuthology.orchestra.run.smithi084.stdout: "tag": "ab83cd97-9994-4866-adc1-d4b616ea165b", 2024-05-22T23:02:27.770 INFO:teuthology.orchestra.run.smithi084.stdout: "options": "recursive,force" 2024-05-22T23:02:27.770 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:02:27.770 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:02:27.770 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:02:27.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f48a1f09640 1 -- 172.21.15.84:0/991318148 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f48840a2e20 msgr2=0x7f48840a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:27.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f48a1f09640 1 --2- 172.21.15.84:0/991318148 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f48840a2e20 0x7f48840a52e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f489c100450 tx=0x7f4890014c80 comp rx=0 tx=0).stop 2024-05-22T23:02:27.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f48a1f09640 1 -- 172.21.15.84:0/991318148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f489c1067c0 msgr2=0x7f489c102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:27.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f48a1f09640 1 --2- 172.21.15.84:0/991318148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f489c1067c0 0x7f489c102c70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4894005ab0 tx=0x7f48940042f0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f48a1f09640 1 -- 172.21.15.84:0/991318148 shutdown_connections 2024-05-22T23:02:27.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f48a1f09640 1 --2- 172.21.15.84:0/991318148 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f48840a2e20 0x7f48840a52e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f48a1f09640 1 --2- 172.21.15.84:0/991318148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f489c1067c0 0x7f489c102c70 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f48a1f09640 1 --2- 172.21.15.84:0/991318148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f489c101460 0x7f489c102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f48a1f09640 1 --2- 172.21.15.84:0/991318148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f489c100ac0 0x7f489c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:27.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.760+0000 7f48a1f09640 1 -- 172.21.15.84:0/991318148 >> 172.21.15.84:0/991318148 conn(0x7f489c076570 msgr2=0x7f489c06a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:27.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.761+0000 7f48a1f09640 1 -- 172.21.15.84:0/991318148 shutdown_connections 2024-05-22T23:02:27.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:27.761+0000 7f48a1f09640 1 -- 172.21.15.84:0/991318148 wait complete. 2024-05-22T23:02:27.773 INFO:tasks.cephfs.filesystem:scrub status for tag:ab83cd97-9994-4866-adc1-d4b616ea165b - {'path': '/', 'tag': 'ab83cd97-9994-4866-adc1-d4b616ea165b', 'options': 'recursive,force'} 2024-05-22T23:02:29.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:28 smithi096 ceph-mon[36392]: scrub queued for path: / 2024-05-22T23:02:29.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:28 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:02:29.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:28 smithi096 ceph-mon[36392]: scrub summary: active paths [/] 2024-05-22T23:02:29.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:28 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:02:29.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:28 smithi096 ceph-mon[36392]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-22T23:02:29.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:28 smithi084 ceph-mon[32186]: scrub queued for path: / 2024-05-22T23:02:29.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:28 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:02:29.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:28 smithi084 ceph-mon[32186]: scrub summary: active paths [/] 2024-05-22T23:02:29.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:28 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:02:29.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:28 smithi084 ceph-mon[32186]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-22T23:02:29.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:28 smithi204 ceph-mon[36211]: scrub queued for path: / 2024-05-22T23:02:29.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:28 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:02:29.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:28 smithi204 ceph-mon[36211]: scrub summary: active paths [/] 2024-05-22T23:02:29.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:28 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:02:29.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:28 smithi204 ceph-mon[36211]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-22T23:02:31.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:30 smithi096 ceph-mon[36392]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-22T23:02:31.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:30 smithi084 ceph-mon[32186]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-22T23:02:31.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:30 smithi204 ceph-mon[36211]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-22T23:02:32.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:31 smithi096 ceph-mon[36392]: scrub summary: idle 2024-05-22T23:02:32.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:31 smithi084 ceph-mon[32186]: scrub summary: idle 2024-05-22T23:02:32.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:31 smithi204 ceph-mon[36211]: scrub summary: idle 2024-05-22T23:02:33.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:32 smithi096 ceph-mon[36392]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2 op/s 2024-05-22T23:02:33.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:32 smithi084 ceph-mon[32186]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2 op/s 2024-05-22T23:02:33.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:32 smithi204 ceph-mon[36211]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2 op/s 2024-05-22T23:02:35.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:34 smithi096 ceph-mon[36392]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-22T23:02:35.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:34 smithi084 ceph-mon[32186]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-22T23:02:35.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:34 smithi204 ceph-mon[36211]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-22T23:02:37.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:36 smithi096 ceph-mon[36392]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-22T23:02:37.377 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:36 smithi084 ceph-mon[32186]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-22T23:02:37.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:36 smithi204 ceph-mon[36211]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-22T23:02:39.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:38 smithi096 ceph-mon[36392]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 938 B/s wr, 3 op/s 2024-05-22T23:02:39.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:38 smithi084 ceph-mon[32186]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 938 B/s wr, 3 op/s 2024-05-22T23:02:39.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:38 smithi204 ceph-mon[36211]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 938 B/s wr, 3 op/s 2024-05-22T23:02:41.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:40 smithi096 ceph-mon[36392]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 938 B/s wr, 2 op/s 2024-05-22T23:02:41.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:40 smithi084 ceph-mon[32186]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 938 B/s wr, 2 op/s 2024-05-22T23:02:41.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:40 smithi204 ceph-mon[36211]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 938 B/s wr, 2 op/s 2024-05-22T23:02:43.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:42 smithi096 ceph-mon[36392]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 938 B/s wr, 3 op/s 2024-05-22T23:02:43.375 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:42 smithi084 ceph-mon[32186]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 938 B/s wr, 3 op/s 2024-05-22T23:02:43.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:42 smithi204 ceph-mon[36211]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 938 B/s wr, 3 op/s 2024-05-22T23:02:45.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:44 smithi096 ceph-mon[36392]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-05-22T23:02:45.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:44 smithi084 ceph-mon[32186]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-05-22T23:02:45.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:44 smithi204 ceph-mon[36211]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-05-22T23:02:47.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:46 smithi096 ceph-mon[36392]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-05-22T23:02:47.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:46 smithi084 ceph-mon[32186]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-05-22T23:02:47.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:46 smithi204 ceph-mon[36211]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-05-22T23:02:49.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:48 smithi096 ceph-mon[36392]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 938 B/s wr, 2 op/s 2024-05-22T23:02:49.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:48 smithi084 ceph-mon[32186]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 938 B/s wr, 2 op/s 2024-05-22T23:02:49.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:48 smithi204 ceph-mon[36211]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 938 B/s wr, 2 op/s 2024-05-22T23:02:51.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:50 smithi096 ceph-mon[36392]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:02:51.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:50 smithi084 ceph-mon[32186]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:02:51.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:50 smithi204 ceph-mon[36211]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:02:53.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:52 smithi096 ceph-mon[36392]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:02:53.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:52 smithi084 ceph-mon[32186]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:02:53.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:52 smithi204 ceph-mon[36211]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:02:55.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:54 smithi096 ceph-mon[36392]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:02:55.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:54 smithi084 ceph-mon[32186]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:02:55.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:54 smithi204 ceph-mon[36211]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:02:56.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:56 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:02:56.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:56 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:02:56.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:56 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:02:56.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:56 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:02:56.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:56 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:02:56.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:56 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:02:56.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:56 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:02:56.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:56 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:02:56.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:56 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:02:57.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:57 smithi096 ceph-mon[36392]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:02:57.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:57 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:02:57.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:57 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:02:57.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:57 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:02:57.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:57 smithi084 ceph-mon[32186]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:02:57.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:57 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:02:57.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:57 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:02:57.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:57 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:02:57.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:57 smithi204 ceph-mon[36211]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:02:57.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:57 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:02:57.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:57 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:02:57.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:57 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:02:57.775 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:02:57.775 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:02:57.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.879+0000 7f5e4268c640 1 -- 172.21.15.84:0/126109772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e3c103410 msgr2=0x7f5e3c10b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:57.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.879+0000 7f5e4268c640 1 --2- 172.21.15.84:0/126109772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e3c103410 0x7f5e3c10b8f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009900 tx=0x7f5e3002a510 comp rx=0 tx=0).stop 2024-05-22T23:02:57.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.879+0000 7f5e4268c640 1 -- 172.21.15.84:0/126109772 shutdown_connections 2024-05-22T23:02:57.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.879+0000 7f5e4268c640 1 --2- 172.21.15.84:0/126109772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e3c10be30 0x7f5e3c10e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.879+0000 7f5e4268c640 1 --2- 172.21.15.84:0/126109772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e3c103410 0x7f5e3c10b8f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.879+0000 7f5e4268c640 1 --2- 172.21.15.84:0/126109772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e3c102af0 0x7f5e3c102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.879+0000 7f5e4268c640 1 -- 172.21.15.84:0/126109772 >> 172.21.15.84:0/126109772 conn(0x7f5e3c077870 msgr2=0x7f5e3c077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:57.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.880+0000 7f5e4268c640 1 -- 172.21.15.84:0/126109772 shutdown_connections 2024-05-22T23:02:57.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.880+0000 7f5e4268c640 1 -- 172.21.15.84:0/126109772 wait complete. 2024-05-22T23:02:57.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.880+0000 7f5e4268c640 1 Processor -- start 2024-05-22T23:02:57.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.880+0000 7f5e4268c640 1 -- start start 2024-05-22T23:02:57.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.881+0000 7f5e4268c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e3c102af0 0x7f5e3c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:57.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.881+0000 7f5e4268c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e3c103410 0x7f5e3c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:57.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.881+0000 7f5e3b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e3c103410 0x7f5e3c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:57.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.881+0000 7f5e3b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e3c103410 0x7f5e3c1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44900/0 (socket says 172.21.15.84:44900) 2024-05-22T23:02:57.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.881+0000 7f5e3b7fe640 1 -- 172.21.15.84:0/2942847707 learned_addr learned my addr 172.21.15.84:0/2942847707 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:57.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.881+0000 7f5e4268c640 1 --2- 172.21.15.84:0/2942847707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e3c10be30 0x7f5e3c19ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:57.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.881+0000 7f5e4268c640 1 -- 172.21.15.84:0/2942847707 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c111c30 con 0x7f5e3c103410 2024-05-22T23:02:57.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.881+0000 7f5e3bfff640 1 --2- 172.21.15.84:0/2942847707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e3c102af0 0x7f5e3c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:57.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.881+0000 7f5e4268c640 1 -- 172.21.15.84:0/2942847707 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c111ab0 con 0x7f5e3c102af0 2024-05-22T23:02:57.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.881+0000 7f5e4268c640 1 -- 172.21.15.84:0/2942847707 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c111db0 con 0x7f5e3c10be30 2024-05-22T23:02:57.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.881+0000 7f5e3b7fe640 1 -- 172.21.15.84:0/2942847707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e3c10be30 msgr2=0x7f5e3c19ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:57.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.881+0000 7f5e40c29640 1 --2- 172.21.15.84:0/2942847707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e3c10be30 0x7f5e3c19ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:57.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.882+0000 7f5e3b7fe640 1 --2- 172.21.15.84:0/2942847707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e3c10be30 0x7f5e3c19ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.882+0000 7f5e3b7fe640 1 -- 172.21.15.84:0/2942847707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e3c102af0 msgr2=0x7f5e3c1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:57.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.882+0000 7f5e3b7fe640 1 --2- 172.21.15.84:0/2942847707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e3c102af0 0x7f5e3c1981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.882+0000 7f5e3b7fe640 1 -- 172.21.15.84:0/2942847707 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3c19d150 con 0x7f5e3c103410 2024-05-22T23:02:57.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.882+0000 7f5e3bfff640 1 --2- 172.21.15.84:0/2942847707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e3c102af0 0x7f5e3c1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:02:57.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.882+0000 7f5e40c29640 1 --2- 172.21.15.84:0/2942847707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e3c10be30 0x7f5e3c19ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:02:57.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.882+0000 7f5e3b7fe640 1 --2- 172.21.15.84:0/2942847707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e3c103410 0x7f5e3c1986f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5e30006010 tx=0x7f5e300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:57.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.882+0000 7f5e397fa640 1 -- 172.21.15.84:0/2942847707 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3003f030 con 0x7f5e3c103410 2024-05-22T23:02:57.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.882+0000 7f5e397fa640 1 -- 172.21.15.84:0/2942847707 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e30003ce0 con 0x7f5e3c103410 2024-05-22T23:02:57.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.882+0000 7f5e4268c640 1 -- 172.21.15.84:0/2942847707 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e3c19d3e0 con 0x7f5e3c103410 2024-05-22T23:02:57.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.882+0000 7f5e397fa640 1 -- 172.21.15.84:0/2942847707 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e30035070 con 0x7f5e3c103410 2024-05-22T23:02:57.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.882+0000 7f5e4268c640 1 -- 172.21.15.84:0/2942847707 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e3c1a4cd0 con 0x7f5e3c103410 2024-05-22T23:02:57.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.885+0000 7f5e397fa640 1 -- 172.21.15.84:0/2942847707 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f5e30042030 con 0x7f5e3c103410 2024-05-22T23:02:57.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.885+0000 7f5e397fa640 1 --2- 172.21.15.84:0/2942847707 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f5e240a2ea0 0x7f5e240a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:57.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.886+0000 7f5e397fa640 1 -- 172.21.15.84:0/2942847707 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f5e300efc10 con 0x7f5e3c103410 2024-05-22T23:02:57.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.886+0000 7f5e3bfff640 1 --2- 172.21.15.84:0/2942847707 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f5e240a2ea0 0x7f5e240a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:57.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.886+0000 7f5e3bfff640 1 --2- 172.21.15.84:0/2942847707 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f5e240a2ea0 0x7f5e240a5360 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00bfd0 tx=0x7f5e2c00b390 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:57.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.886+0000 7f5e45397740 1 Processor -- start 2024-05-22T23:02:57.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.886+0000 7f5e45397740 1 -- start start 2024-05-22T23:02:57.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.887+0000 7f5e45397740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a565e7c8b0 0x55a565e7abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:57.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.887+0000 7f5e45397740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e7b1c0 0x55a565e7b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:57.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.887+0000 7f5e45397740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a565e85f40 0x55a565e88330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:57.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.887+0000 7f5e45397740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a565ddadb0 con 0x55a565e7b1c0 2024-05-22T23:02:57.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.887+0000 7f5e45397740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a565d79360 con 0x55a565e7c8b0 2024-05-22T23:02:57.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.887+0000 7f5e45397740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a565dd6d70 con 0x55a565e85f40 2024-05-22T23:02:57.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.887+0000 7f5e3b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e7b1c0 0x55a565e7b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:57.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.887+0000 7f5e3b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e7b1c0 0x55a565e7b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44908/0 (socket says 172.21.15.84:44908) 2024-05-22T23:02:57.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.887+0000 7f5e3b7fe640 1 -- 172.21.15.84:0/4155375261 learned_addr learned my addr 172.21.15.84:0/4155375261 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:57.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.887+0000 7f5e3bfff640 1 --2- 172.21.15.84:0/4155375261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a565e7c8b0 0x55a565e7abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:57.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.887+0000 7f5e40c29640 1 --2- 172.21.15.84:0/4155375261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a565e85f40 0x55a565e88330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:57.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.887+0000 7f5e3b7fe640 1 -- 172.21.15.84:0/4155375261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a565e85f40 msgr2=0x55a565e88330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:57.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.887+0000 7f5e3b7fe640 1 --2- 172.21.15.84:0/4155375261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a565e85f40 0x55a565e88330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.887+0000 7f5e3b7fe640 1 -- 172.21.15.84:0/4155375261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a565e7c8b0 msgr2=0x55a565e7abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:57.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.887+0000 7f5e3b7fe640 1 --2- 172.21.15.84:0/4155375261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a565e7c8b0 0x55a565e7abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.887+0000 7f5e3b7fe640 1 -- 172.21.15.84:0/4155375261 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a565d9ca60 con 0x55a565e7b1c0 2024-05-22T23:02:57.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.887+0000 7f5e3bfff640 1 --2- 172.21.15.84:0/4155375261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a565e7c8b0 0x55a565e7abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:02:57.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.887+0000 7f5e40c29640 1 --2- 172.21.15.84:0/4155375261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a565e85f40 0x55a565e88330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:02:57.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.887+0000 7f5e3b7fe640 1 --2- 172.21.15.84:0/4155375261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e7b1c0 0x55a565e7b640 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5e30005900 tx=0x7f5e300bcde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd66e8d9a90685f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:57.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.888+0000 7f5e227fc640 1 -- 172.21.15.84:0/4155375261 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e30049050 con 0x55a565e7b1c0 2024-05-22T23:02:57.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.888+0000 7f5e227fc640 1 -- 172.21.15.84:0/4155375261 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e300bd650 con 0x55a565e7b1c0 2024-05-22T23:02:57.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.888+0000 7f5e227fc640 1 -- 172.21.15.84:0/4155375261 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e300c5730 con 0x55a565e7b1c0 2024-05-22T23:02:57.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.888+0000 7f5e45397740 1 -- 172.21.15.84:0/4155375261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e7b1c0 msgr2=0x55a565e7b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:57.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.888+0000 7f5e45397740 1 --2- 172.21.15.84:0/4155375261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e7b1c0 0x55a565e7b640 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5e30005900 tx=0x7f5e300bcde0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.889+0000 7f5e45397740 1 -- 172.21.15.84:0/4155375261 shutdown_connections 2024-05-22T23:02:57.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.889+0000 7f5e45397740 1 --2- 172.21.15.84:0/4155375261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a565e85f40 0x55a565e88330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.889+0000 7f5e45397740 1 --2- 172.21.15.84:0/4155375261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e7b1c0 0x55a565e7b640 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.889+0000 7f5e45397740 1 --2- 172.21.15.84:0/4155375261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a565e7c8b0 0x55a565e7abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.889+0000 7f5e45397740 1 -- 172.21.15.84:0/4155375261 >> 172.21.15.84:0/4155375261 conn(0x55a565e3f8e0 msgr2=0x55a565e77720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:57.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.889+0000 7f5e45397740 1 -- 172.21.15.84:0/4155375261 shutdown_connections 2024-05-22T23:02:57.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.889+0000 7f5e45397740 1 -- 172.21.15.84:0/4155375261 wait complete. 2024-05-22T23:02:57.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.889+0000 7f5e45397740 1 -- start start 2024-05-22T23:02:57.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.889+0000 7f5e45397740 1 Processor -- start 2024-05-22T23:02:57.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.890+0000 7f5e45397740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a565e7b1c0 0x55a565e82ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:57.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.890+0000 7f5e45397740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a565e7c8b0 0x55a565e83430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:57.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.890+0000 7f5e45397740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e85f40 0x55a565e83970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:57.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.890+0000 7f5e45397740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a565e88b90 con 0x55a565e85f40 2024-05-22T23:02:57.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.890+0000 7f5e40c29640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e85f40 0x55a565e83970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:57.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.890+0000 7f5e40c29640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e85f40 0x55a565e83970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44914/0 (socket says 172.21.15.84:44914) 2024-05-22T23:02:57.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.890+0000 7f5e40c29640 1 -- 172.21.15.84:0/3521740122 learned_addr learned my addr 172.21.15.84:0/3521740122 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:57.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.890+0000 7f5e45397740 1 -- 172.21.15.84:0/3521740122 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a565d950f0 con 0x55a565e7b1c0 2024-05-22T23:02:57.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.890+0000 7f5e45397740 1 -- 172.21.15.84:0/3521740122 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a565e88d10 con 0x55a565e7c8b0 2024-05-22T23:02:57.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.890+0000 7f5e3b7fe640 1 --2- 172.21.15.84:0/3521740122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a565e7c8b0 0x55a565e83430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:57.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.890+0000 7f5e3bfff640 1 --2- 172.21.15.84:0/3521740122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a565e7b1c0 0x55a565e82ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:57.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.890+0000 7f5e40c29640 1 -- 172.21.15.84:0/3521740122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a565e7c8b0 msgr2=0x55a565e83430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:57.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.890+0000 7f5e40c29640 1 --2- 172.21.15.84:0/3521740122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a565e7c8b0 0x55a565e83430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.891+0000 7f5e40c29640 1 -- 172.21.15.84:0/3521740122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a565e7b1c0 msgr2=0x55a565e82ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:57.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.891+0000 7f5e40c29640 1 --2- 172.21.15.84:0/3521740122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a565e7b1c0 0x55a565e82ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.891+0000 7f5e40c29640 1 -- 172.21.15.84:0/3521740122 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a565bb87d0 con 0x55a565e85f40 2024-05-22T23:02:57.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.891+0000 7f5e45397740 10 client.0 _ll_register_callbacks cb 0x55a565d99f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:02:57.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.891+0000 7f5e40c29640 1 --2- 172.21.15.84:0/3521740122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e85f40 0x55a565e83970 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009c70 tx=0x7f5e3400d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ba2eea55d3fbbd server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:57.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.891+0000 7f5e1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:02:57.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.891+0000 7f5e45397740 1 -- 172.21.15.84:0/3521740122 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e300bd650 con 0x55a565e85f40 2024-05-22T23:02:57.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.891+0000 7f5e1b7fe640 1 -- 172.21.15.84:0/3521740122 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e34004d20 con 0x55a565e85f40 2024-05-22T23:02:57.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.891+0000 7f5e1b7fe640 1 -- 172.21.15.84:0/3521740122 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f5e34003970 con 0x55a565e85f40 2024-05-22T23:02:57.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.891+0000 7f5e1b7fe640 1 -- 172.21.15.84:0/3521740122 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f5e34003b70 con 0x55a565e85f40 2024-05-22T23:02:57.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.891+0000 7f5e45397740 10 client.15576 fetch_fsmap learned FSMap version 18 2024-05-22T23:02:57.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.891+0000 7f5e45397740 1 -- 172.21.15.84:0/3521740122 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({fsmap=18}) -- 0x55a565aee3f0 con 0x55a565e85f40 2024-05-22T23:02:57.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.892+0000 7f5e1b7fe640 1 -- 172.21.15.84:0/3521740122 <== mon.0 v2:172.21.15.84:3300/0 4 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f5e34004050 con 0x55a565e85f40 2024-05-22T23:02:57.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.892+0000 7f5e45397740 10 client.15576 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:02:57.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.892+0000 7f5e45397740 10 client.15576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:02:57.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.892+0000 7f5e45397740 20 client.15576 populate_metadata read hostname 'smithi084' 2024-05-22T23:02:57.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.892+0000 7f5e45397740 1 --2- 172.21.15.84:0/3521740122 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55a565e769c0 0x55a565e940f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:57.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.892+0000 7f5e45397740 4 client.15576 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:02:57.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.892+0000 7f5e45397740 1 -- 172.21.15.84:0/3521740122 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5e300bd650 con 0x55a565e769c0 2024-05-22T23:02:57.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.892+0000 7f5e3bfff640 1 --2- 172.21.15.84:0/3521740122 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55a565e769c0 0x55a565e940f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:57.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.893+0000 7f5e3bfff640 1 --2- 172.21.15.84:0/3521740122 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55a565e769c0 0x55a565e940f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a358fbcbc0b093c server_cookie=829a44d0b1d0e747 in_seq=0 out_seq=0 2024-05-22T23:02:57.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.893+0000 7f5e1b7fe640 10 client.15576 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:57.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.893+0000 7f5e1b7fe640 1 -- 172.21.15.84:0/3521740122 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f5e300bd650 con 0x55a565e769c0 2024-05-22T23:02:57.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.893+0000 7f5e1b7fe640 10 client.15576 handle_command_reply: tid=0 2024-05-22T23:02:57.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.893+0000 7f5e45397740 1 client.15576 shutdown 2024-05-22T23:02:57.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.894+0000 7f5e45397740 1 -- 172.21.15.84:0/3521740122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e85f40 msgr2=0x55a565e83970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:57.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.894+0000 7f5e45397740 1 --2- 172.21.15.84:0/3521740122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e85f40 0x55a565e83970 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009c70 tx=0x7f5e3400d6b0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.894+0000 7f5e45397740 1 -- 172.21.15.84:0/3521740122 shutdown_connections 2024-05-22T23:02:57.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.894+0000 7f5e45397740 1 --2- 172.21.15.84:0/3521740122 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55a565e769c0 0x55a565e940f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.894+0000 7f5e1b7fe640 0 client.15576 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:57.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.894+0000 7f5e45397740 1 --2- 172.21.15.84:0/3521740122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e85f40 0x55a565e83970 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.894+0000 7f5e45397740 1 --2- 172.21.15.84:0/3521740122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a565e7c8b0 0x55a565e83430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.894+0000 7f5e45397740 1 --2- 172.21.15.84:0/3521740122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a565e7b1c0 0x55a565e82ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.894+0000 7f5e45397740 1 -- 172.21.15.84:0/3521740122 >> 172.21.15.84:0/3521740122 conn(0x55a565e3f8e0 msgr2=0x55a565e72470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:57.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.894+0000 7f5e45397740 1 -- 172.21.15.84:0/3521740122 shutdown_connections 2024-05-22T23:02:57.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.894+0000 7f5e45397740 1 -- 172.21.15.84:0/3521740122 wait complete. 2024-05-22T23:02:57.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.894+0000 7f5e45397740 20 client.15576 trim_cache size 0 max 16384 2024-05-22T23:02:57.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.912+0000 7f5e45397740 1 Processor -- start 2024-05-22T23:02:57.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.912+0000 7f5e45397740 1 -- start start 2024-05-22T23:02:57.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.913+0000 7f5e45397740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a565e769c0 0x55a565e919c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:57.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.913+0000 7f5e45397740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e7b1c0 0x55a565e83790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:57.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.913+0000 7f5e45397740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a565e7c8b0 0x55a565e81ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:57.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.913+0000 7f5e45397740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c1a9410 con 0x55a565e7b1c0 2024-05-22T23:02:57.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.913+0000 7f5e45397740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c1abc90 con 0x55a565e769c0 2024-05-22T23:02:57.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.913+0000 7f5e45397740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c1a9a80 con 0x55a565e7c8b0 2024-05-22T23:02:57.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.913+0000 7f5e3b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e7b1c0 0x55a565e83790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:57.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.913+0000 7f5e3b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e7b1c0 0x55a565e83790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44930/0 (socket says 172.21.15.84:44930) 2024-05-22T23:02:57.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.913+0000 7f5e3b7fe640 1 -- 172.21.15.84:0/3229217895 learned_addr learned my addr 172.21.15.84:0/3229217895 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:57.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.913+0000 7f5e3bfff640 1 --2- 172.21.15.84:0/3229217895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a565e769c0 0x55a565e919c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:57.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.913+0000 7f5e40c29640 1 --2- 172.21.15.84:0/3229217895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a565e7c8b0 0x55a565e81ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:57.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.913+0000 7f5e3b7fe640 1 -- 172.21.15.84:0/3229217895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a565e7c8b0 msgr2=0x55a565e81ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:57.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.913+0000 7f5e3b7fe640 1 --2- 172.21.15.84:0/3229217895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a565e7c8b0 0x55a565e81ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.913+0000 7f5e3b7fe640 1 -- 172.21.15.84:0/3229217895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a565e769c0 msgr2=0x55a565e919c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:57.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.913+0000 7f5e3b7fe640 1 --2- 172.21.15.84:0/3229217895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a565e769c0 0x55a565e919c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.913+0000 7f5e3b7fe640 1 -- 172.21.15.84:0/3229217895 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a565aae0f0 con 0x55a565e7b1c0 2024-05-22T23:02:57.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.913+0000 7f5e40c29640 1 --2- 172.21.15.84:0/3229217895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a565e7c8b0 0x55a565e81ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:02:57.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.913+0000 7f5e3bfff640 1 --2- 172.21.15.84:0/3229217895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a565e769c0 0x55a565e919c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:02:57.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.913+0000 7f5e3b7fe640 1 --2- 172.21.15.84:0/3229217895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e7b1c0 0x55a565e83790 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5e3003b710 tx=0x7f5e300bce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e224bef7ac5b9425 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:57.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.914+0000 7f5e21ffb640 1 -- 172.21.15.84:0/3229217895 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e300bd650 con 0x55a565e7b1c0 2024-05-22T23:02:57.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.914+0000 7f5e21ffb640 1 -- 172.21.15.84:0/3229217895 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e300c1300 con 0x55a565e7b1c0 2024-05-22T23:02:57.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.914+0000 7f5e21ffb640 1 -- 172.21.15.84:0/3229217895 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e300c5990 con 0x55a565e7b1c0 2024-05-22T23:02:57.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.915+0000 7f5e45397740 1 -- 172.21.15.84:0/3229217895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e7b1c0 msgr2=0x55a565e83790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:57.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.915+0000 7f5e45397740 1 --2- 172.21.15.84:0/3229217895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e7b1c0 0x55a565e83790 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5e3003b710 tx=0x7f5e300bce50 comp rx=0 tx=0).stop 2024-05-22T23:02:57.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.915+0000 7f5e45397740 1 -- 172.21.15.84:0/3229217895 shutdown_connections 2024-05-22T23:02:57.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.915+0000 7f5e45397740 1 --2- 172.21.15.84:0/3229217895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a565e7c8b0 0x55a565e81ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.915+0000 7f5e45397740 1 --2- 172.21.15.84:0/3229217895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e7b1c0 0x55a565e83790 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.915+0000 7f5e45397740 1 --2- 172.21.15.84:0/3229217895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a565e769c0 0x55a565e919c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.915+0000 7f5e45397740 1 -- 172.21.15.84:0/3229217895 >> 172.21.15.84:0/3229217895 conn(0x55a565e3f8e0 msgr2=0x55a565e817c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:57.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.915+0000 7f5e45397740 1 -- 172.21.15.84:0/3229217895 shutdown_connections 2024-05-22T23:02:57.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.915+0000 7f5e45397740 1 -- 172.21.15.84:0/3229217895 wait complete. 2024-05-22T23:02:57.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.915+0000 7f5e45397740 1 -- start start 2024-05-22T23:02:57.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.915+0000 7f5e45397740 1 Processor -- start 2024-05-22T23:02:57.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.915+0000 7f5e45397740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a565e769c0 0x55a565e8e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:57.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.915+0000 7f5e45397740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a565e7b1c0 0x55a565e89590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:57.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.916+0000 7f5e45397740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e7c8b0 0x55a565e89ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:57.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.916+0000 7f5e45397740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c1aa150 con 0x55a565e7c8b0 2024-05-22T23:02:57.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.916+0000 7f5e45397740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c1ab5c0 con 0x55a565e769c0 2024-05-22T23:02:57.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.916+0000 7f5e45397740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c1aa820 con 0x55a565e7b1c0 2024-05-22T23:02:57.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.916+0000 7f5e45397740 10 client.0 _ll_register_callbacks cb 0x7f5e34009670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:02:57.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.916+0000 7f5e40c29640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e7c8b0 0x55a565e89ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:57.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.916+0000 7f5e40c29640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e7c8b0 0x55a565e89ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44940/0 (socket says 172.21.15.84:44940) 2024-05-22T23:02:57.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.916+0000 7f5e40c29640 1 -- 172.21.15.84:0/4175995427 learned_addr learned my addr 172.21.15.84:0/4175995427 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:57.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.916+0000 7f5e3bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a565e769c0 0x55a565e8e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:57.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.916+0000 7f5e3b7fe640 1 --2- 172.21.15.84:0/4175995427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a565e7b1c0 0x55a565e89590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:57.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.916+0000 7f5e40c29640 1 -- 172.21.15.84:0/4175995427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a565e7b1c0 msgr2=0x55a565e89590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:57.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.916+0000 7f5e40c29640 1 --2- 172.21.15.84:0/4175995427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a565e7b1c0 0x55a565e89590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.916+0000 7f5e40c29640 1 -- 172.21.15.84:0/4175995427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a565e769c0 msgr2=0x55a565e8e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:57.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.916+0000 7f5e40c29640 1 --2- 172.21.15.84:0/4175995427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a565e769c0 0x55a565e8e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.916+0000 7f5e40c29640 1 -- 172.21.15.84:0/4175995427 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a565d99fe0 con 0x55a565e7c8b0 2024-05-22T23:02:57.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.916+0000 7f5e3b7fe640 1 --2- 172.21.15.84:0/4175995427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a565e7b1c0 0x55a565e89590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:02:57.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.916+0000 7f5e3bfff640 1 --2- 172.21.15.84:0/4175995427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a565e769c0 0x55a565e8e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:02:57.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.916+0000 7f5e40c29640 1 --2- 172.21.15.84:0/4175995427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e7c8b0 0x55a565e89ad0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5e34006010 tx=0x7f5e34019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0dbbe663200507 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:57.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.916+0000 7f5e1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:02:57.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.916+0000 7f5e1b7fe640 1 -- 172.21.15.84:0/4175995427 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e34039060 con 0x55a565e7c8b0 2024-05-22T23:02:57.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.916+0000 7f5e1b7fe640 1 -- 172.21.15.84:0/4175995427 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f5e34003f20 con 0x55a565e7c8b0 2024-05-22T23:02:57.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.917+0000 7f5e45397740 1 -- 172.21.15.84:0/4175995427 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e300c1300 con 0x55a565e7c8b0 2024-05-22T23:02:57.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.917+0000 7f5e1b7fe640 1 -- 172.21.15.84:0/4175995427 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f5e340094d0 con 0x55a565e7c8b0 2024-05-22T23:02:57.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.917+0000 7f5e45397740 10 client.15582 fetch_fsmap learned FSMap version 18 2024-05-22T23:02:57.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.917+0000 7f5e45397740 1 -- 172.21.15.84:0/4175995427 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55a565d6b1c0 con 0x55a565e7c8b0 2024-05-22T23:02:57.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.917+0000 7f5e1b7fe640 1 -- 172.21.15.84:0/4175995427 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e300c1300 con 0x55a565e7c8b0 2024-05-22T23:02:57.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.917+0000 7f5e1b7fe640 1 -- 172.21.15.84:0/4175995427 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f5e34028b70 con 0x55a565e7c8b0 2024-05-22T23:02:57.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.917+0000 7f5e45397740 10 client.15582 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:02:57.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.917+0000 7f5e45397740 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:02:57.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.917+0000 7f5e45397740 20 client.15582 populate_metadata read hostname 'smithi084' 2024-05-22T23:02:57.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.917+0000 7f5e45397740 1 --2- 172.21.15.84:0/4175995427 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55a565e85f40 0x55a565efa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:57.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.917+0000 7f5e45397740 4 client.15582 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:02:57.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.917+0000 7f5e45397740 1 -- 172.21.15.84:0/4175995427 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a565de84e0 con 0x55a565e85f40 2024-05-22T23:02:57.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.918+0000 7f5e3bfff640 1 --2- 172.21.15.84:0/4175995427 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55a565e85f40 0x55a565efa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:57.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.918+0000 7f5e1b7fe640 10 client.15582 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:57.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.918+0000 7f5e3bfff640 1 --2- 172.21.15.84:0/4175995427 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55a565e85f40 0x55a565efa310 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a176856ed28cd040 server_cookie=313de43d67b02816 in_seq=0 out_seq=0 2024-05-22T23:02:57.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.918+0000 7f5e1b7fe640 1 -- 172.21.15.84:0/4175995427 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a565de84e0 con 0x55a565e85f40 2024-05-22T23:02:57.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.918+0000 7f5e1b7fe640 10 client.15582 handle_command_reply: tid=0 2024-05-22T23:02:57.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.918+0000 7f5e45397740 1 client.15582 shutdown 2024-05-22T23:02:57.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.918+0000 7f5e45397740 1 -- 172.21.15.84:0/4175995427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e7c8b0 msgr2=0x55a565e89ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:57.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.918+0000 7f5e45397740 1 --2- 172.21.15.84:0/4175995427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e7c8b0 0x55a565e89ad0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5e34006010 tx=0x7f5e34019040 comp rx=0 tx=0).stop 2024-05-22T23:02:57.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.918+0000 7f5e45397740 1 -- 172.21.15.84:0/4175995427 shutdown_connections 2024-05-22T23:02:57.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.918+0000 7f5e45397740 1 --2- 172.21.15.84:0/4175995427 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55a565e85f40 0x55a565efa310 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.919+0000 7f5e1b7fe640 0 client.15582 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:57.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.919+0000 7f5e45397740 1 --2- 172.21.15.84:0/4175995427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a565e7c8b0 0x55a565e89ad0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.919+0000 7f5e45397740 1 --2- 172.21.15.84:0/4175995427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a565e7b1c0 0x55a565e89590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.919+0000 7f5e45397740 1 --2- 172.21.15.84:0/4175995427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a565e769c0 0x55a565e8e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.919+0000 7f5e45397740 1 -- 172.21.15.84:0/4175995427 >> 172.21.15.84:0/4175995427 conn(0x55a565e3f8e0 msgr2=0x55a565e71be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:57.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.919+0000 7f5e45397740 1 -- 172.21.15.84:0/4175995427 shutdown_connections 2024-05-22T23:02:57.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.919+0000 7f5e45397740 1 -- 172.21.15.84:0/4175995427 wait complete. 2024-05-22T23:02:57.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.919+0000 7f5e45397740 20 client.15582 trim_cache size 0 max 16384 2024-05-22T23:02:57.927 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:02:57.927 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "no active scrubs running", 2024-05-22T23:02:57.927 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": {} 2024-05-22T23:02:57.928 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:02:57.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.919+0000 7f5e4268c640 1 -- 172.21.15.84:0/2942847707 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f5e240a2ea0 msgr2=0x7f5e240a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:57.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.919+0000 7f5e4268c640 1 --2- 172.21.15.84:0/2942847707 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f5e240a2ea0 0x7f5e240a5360 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00bfd0 tx=0x7f5e2c00b390 comp rx=0 tx=0).stop 2024-05-22T23:02:57.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.919+0000 7f5e4268c640 1 -- 172.21.15.84:0/2942847707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e3c103410 msgr2=0x7f5e3c1986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:57.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.919+0000 7f5e4268c640 1 --2- 172.21.15.84:0/2942847707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e3c103410 0x7f5e3c1986f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5e30006010 tx=0x7f5e300042f0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.919+0000 7f5e4268c640 1 -- 172.21.15.84:0/2942847707 shutdown_connections 2024-05-22T23:02:57.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.919+0000 7f5e4268c640 1 --2- 172.21.15.84:0/2942847707 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f5e240a2ea0 0x7f5e240a5360 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.919+0000 7f5e4268c640 1 --2- 172.21.15.84:0/2942847707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e3c10be30 0x7f5e3c19ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.919+0000 7f5e4268c640 1 --2- 172.21.15.84:0/2942847707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e3c103410 0x7f5e3c1986f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.919+0000 7f5e4268c640 1 --2- 172.21.15.84:0/2942847707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e3c102af0 0x7f5e3c1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:57.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.919+0000 7f5e4268c640 1 -- 172.21.15.84:0/2942847707 >> 172.21.15.84:0/2942847707 conn(0x7f5e3c077870 msgr2=0x7f5e3c0fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:57.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.920+0000 7f5e4268c640 1 -- 172.21.15.84:0/2942847707 shutdown_connections 2024-05-22T23:02:57.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:57.920+0000 7f5e4268c640 1 -- 172.21.15.84:0/2942847707 wait complete. 2024-05-22T23:02:57.932 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-22T23:02:57.932 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:02:58.036 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.035+0000 7ff5c1d9b640 1 -- 172.21.15.84:0/1376376773 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b80319b0 con 0x7ff5c4100750 2024-05-22T23:02:58.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.035+0000 7ff5c9015640 1 -- 172.21.15.84:0/1376376773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5c4100750 msgr2=0x7ff5c4100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:58.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.035+0000 7ff5c9015640 1 --2- 172.21.15.84:0/1376376773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5c4100750 0x7ff5c4100bd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009930 tx=0x7ff5b802a420 comp rx=0 tx=0).stop 2024-05-22T23:02:58.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.035+0000 7ff5c9015640 1 -- 172.21.15.84:0/1376376773 shutdown_connections 2024-05-22T23:02:58.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.035+0000 7ff5c9015640 1 --2- 172.21.15.84:0/1376376773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5c4101110 0x7ff5c410d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.036+0000 7ff5c9015640 1 --2- 172.21.15.84:0/1376376773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5c4100750 0x7ff5c4100bd0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.036+0000 7ff5c9015640 1 --2- 172.21.15.84:0/1376376773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5c4106760 0x7ff5c4106b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.036+0000 7ff5c9015640 1 -- 172.21.15.84:0/1376376773 >> 172.21.15.84:0/1376376773 conn(0x7ff5c4077870 msgr2=0x7ff5c4077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:58.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.036+0000 7ff5c9015640 1 -- 172.21.15.84:0/1376376773 shutdown_connections 2024-05-22T23:02:58.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.036+0000 7ff5c9015640 1 -- 172.21.15.84:0/1376376773 wait complete. 2024-05-22T23:02:58.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.036+0000 7ff5c9015640 1 Processor -- start 2024-05-22T23:02:58.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.036+0000 7ff5c9015640 1 -- start start 2024-05-22T23:02:58.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.037+0000 7ff5c9015640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5c4100750 0x7ff5c4195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:58.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.037+0000 7ff5c9015640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5c4101110 0x7ff5c4196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:58.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.037+0000 7ff5c9015640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5c4106760 0x7ff5c419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:58.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.037+0000 7ff5c9015640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c410fb20 con 0x7ff5c4101110 2024-05-22T23:02:58.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.037+0000 7ff5c9015640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c40698d0 con 0x7ff5c4100750 2024-05-22T23:02:58.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.037+0000 7ff5c9015640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c410fca0 con 0x7ff5c4106760 2024-05-22T23:02:58.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.037+0000 7ff5c259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5c4101110 0x7ff5c4196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:58.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.037+0000 7ff5c259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5c4101110 0x7ff5c4196510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44954/0 (socket says 172.21.15.84:44954) 2024-05-22T23:02:58.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.037+0000 7ff5c259c640 1 -- 172.21.15.84:0/136391537 learned_addr learned my addr 172.21.15.84:0/136391537 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:58.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.037+0000 7ff5c2d9d640 1 --2- 172.21.15.84:0/136391537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5c4100750 0x7ff5c4195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:58.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.037+0000 7ff5c359e640 1 --2- 172.21.15.84:0/136391537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5c4106760 0x7ff5c419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:58.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.037+0000 7ff5c259c640 1 -- 172.21.15.84:0/136391537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5c4106760 msgr2=0x7ff5c419a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:58.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.037+0000 7ff5c259c640 1 --2- 172.21.15.84:0/136391537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5c4106760 0x7ff5c419a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.037+0000 7ff5c259c640 1 -- 172.21.15.84:0/136391537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5c4100750 msgr2=0x7ff5c4195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:58.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.037+0000 7ff5c259c640 1 --2- 172.21.15.84:0/136391537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5c4100750 0x7ff5c4195fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.037+0000 7ff5c259c640 1 -- 172.21.15.84:0/136391537 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c419afc0 con 0x7ff5c4101110 2024-05-22T23:02:58.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.037+0000 7ff5c259c640 1 --2- 172.21.15.84:0/136391537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5c4101110 0x7ff5c4196510 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009230 tx=0x7ff5b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:58.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.038+0000 7ff5abfff640 1 -- 172.21.15.84:0/136391537 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b803f030 con 0x7ff5c4101110 2024-05-22T23:02:58.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.038+0000 7ff5abfff640 1 -- 172.21.15.84:0/136391537 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5b8003ad0 con 0x7ff5c4101110 2024-05-22T23:02:58.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.038+0000 7ff5c9015640 1 -- 172.21.15.84:0/136391537 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c419b250 con 0x7ff5c4101110 2024-05-22T23:02:58.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.038+0000 7ff5c9015640 1 -- 172.21.15.84:0/136391537 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c41a2b40 con 0x7ff5c4101110 2024-05-22T23:02:58.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.038+0000 7ff5abfff640 1 -- 172.21.15.84:0/136391537 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8035070 con 0x7ff5c4101110 2024-05-22T23:02:58.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.039+0000 7ff5c9015640 1 -- 172.21.15.84:0/136391537 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5b0005290 con 0x7ff5c4101110 2024-05-22T23:02:58.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.040+0000 7ff5abfff640 1 -- 172.21.15.84:0/136391537 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7ff5b8042030 con 0x7ff5c4101110 2024-05-22T23:02:58.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.041+0000 7ff5abfff640 1 --2- 172.21.15.84:0/136391537 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff5ac0a2e80 0x7ff5ac0a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:58.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.041+0000 7ff5abfff640 1 -- 172.21.15.84:0/136391537 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7ff5b80efad0 con 0x7ff5c4101110 2024-05-22T23:02:58.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.042+0000 7ff5c2d9d640 1 --2- 172.21.15.84:0/136391537 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff5ac0a2e80 0x7ff5ac0a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:58.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.042+0000 7ff5c2d9d640 1 --2- 172.21.15.84:0/136391537 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff5ac0a2e80 0x7ff5ac0a5340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4005d00 tx=0x7ff5b4017040 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:58.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.049+0000 7ff5abfff640 1 -- 172.21.15.84:0/136391537 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff5b80bc510 con 0x7ff5c4101110 2024-05-22T23:02:58.273 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.271+0000 7ff5c9015640 1 -- 172.21.15.84:0/136391537 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5b0005580 con 0x7ff5c4101110 2024-05-22T23:02:58.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.272+0000 7ff5abfff640 1 -- 172.21.15.84:0/136391537 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7ff5b80bbe10 con 0x7ff5c4101110 2024-05-22T23:02:58.274 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:02:58.274 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:02:58.274 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:02:58.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.274+0000 7ff5c9015640 1 -- 172.21.15.84:0/136391537 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff5ac0a2e80 msgr2=0x7ff5ac0a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:58.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.274+0000 7ff5c9015640 1 --2- 172.21.15.84:0/136391537 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff5ac0a2e80 0x7ff5ac0a5340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4005d00 tx=0x7ff5b4017040 comp rx=0 tx=0).stop 2024-05-22T23:02:58.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.274+0000 7ff5c9015640 1 -- 172.21.15.84:0/136391537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5c4101110 msgr2=0x7ff5c4196510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:58.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.274+0000 7ff5c9015640 1 --2- 172.21.15.84:0/136391537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5c4101110 0x7ff5c4196510 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009230 tx=0x7ff5b8004270 comp rx=0 tx=0).stop 2024-05-22T23:02:58.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.274+0000 7ff5c9015640 1 -- 172.21.15.84:0/136391537 shutdown_connections 2024-05-22T23:02:58.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.274+0000 7ff5c9015640 1 --2- 172.21.15.84:0/136391537 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff5ac0a2e80 0x7ff5ac0a5340 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.274+0000 7ff5c9015640 1 --2- 172.21.15.84:0/136391537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5c4106760 0x7ff5c419a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.274+0000 7ff5c9015640 1 --2- 172.21.15.84:0/136391537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5c4101110 0x7ff5c4196510 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.274+0000 7ff5c9015640 1 --2- 172.21.15.84:0/136391537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5c4100750 0x7ff5c4195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.277 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.274+0000 7ff5c9015640 1 -- 172.21.15.84:0/136391537 >> 172.21.15.84:0/136391537 conn(0x7ff5c4077870 msgr2=0x7ff5c4102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:58.277 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.275+0000 7ff5c9015640 1 -- 172.21.15.84:0/136391537 shutdown_connections 2024-05-22T23:02:58.277 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.275+0000 7ff5c9015640 1 -- 172.21.15.84:0/136391537 wait complete. 2024-05-22T23:02:58.288 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-22T23:02:58.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.392+0000 7f88d384d640 1 -- 172.21.15.84:0/1188437395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88cc102af0 msgr2=0x7f88cc102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:58.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.392+0000 7f88d384d640 1 --2- 172.21.15.84:0/1188437395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88cc102af0 0x7f88cc102ed0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009900 tx=0x7f88c802a510 comp rx=0 tx=0).stop 2024-05-22T23:02:58.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.392+0000 7f88d384d640 1 -- 172.21.15.84:0/1188437395 shutdown_connections 2024-05-22T23:02:58.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.392+0000 7f88d384d640 1 --2- 172.21.15.84:0/1188437395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88cc10be30 0x7f88cc10e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.392+0000 7f88d384d640 1 --2- 172.21.15.84:0/1188437395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88cc103410 0x7f88cc10b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.392+0000 7f88d384d640 1 --2- 172.21.15.84:0/1188437395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88cc102af0 0x7f88cc102ed0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.392+0000 7f88d384d640 1 -- 172.21.15.84:0/1188437395 >> 172.21.15.84:0/1188437395 conn(0x7f88cc077870 msgr2=0x7f88cc077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:58.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.392+0000 7f88d384d640 1 -- 172.21.15.84:0/1188437395 shutdown_connections 2024-05-22T23:02:58.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.392+0000 7f88d384d640 1 -- 172.21.15.84:0/1188437395 wait complete. 2024-05-22T23:02:58.396 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.392+0000 7f88d384d640 1 Processor -- start 2024-05-22T23:02:58.396 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.393+0000 7f88d384d640 1 -- start start 2024-05-22T23:02:58.396 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.393+0000 7f88d384d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88cc102af0 0x7f88cc100740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:58.396 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.393+0000 7f88d384d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88cc103410 0x7f88cc100c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:58.396 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.393+0000 7f88d384d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88cc10be30 0x7f88cc1011c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:58.396 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.393+0000 7f88d384d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc111c30 con 0x7f88cc102af0 2024-05-22T23:02:58.396 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.393+0000 7f88d384d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc111ab0 con 0x7f88cc103410 2024-05-22T23:02:58.397 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.393+0000 7f88d384d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc111db0 con 0x7f88cc10be30 2024-05-22T23:02:58.397 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.393+0000 7f88d15e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88cc102af0 0x7f88cc100740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:58.397 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.393+0000 7f88d15e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88cc102af0 0x7f88cc100740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44978/0 (socket says 172.21.15.84:44978) 2024-05-22T23:02:58.397 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.393+0000 7f88d15e9640 1 -- 172.21.15.84:0/1053952776 learned_addr learned my addr 172.21.15.84:0/1053952776 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:58.397 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.393+0000 7f88d0de8640 1 --2- 172.21.15.84:0/1053952776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88cc103410 0x7f88cc100c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:58.397 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.393+0000 7f88d1dea640 1 --2- 172.21.15.84:0/1053952776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88cc10be30 0x7f88cc1011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:58.397 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.393+0000 7f88d15e9640 1 -- 172.21.15.84:0/1053952776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88cc10be30 msgr2=0x7f88cc1011c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:58.397 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.393+0000 7f88d15e9640 1 --2- 172.21.15.84:0/1053952776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88cc10be30 0x7f88cc1011c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.397 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.393+0000 7f88d15e9640 1 -- 172.21.15.84:0/1053952776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88cc103410 msgr2=0x7f88cc100c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:58.397 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.393+0000 7f88d15e9640 1 --2- 172.21.15.84:0/1053952776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88cc103410 0x7f88cc100c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.397 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.393+0000 7f88d15e9640 1 -- 172.21.15.84:0/1053952776 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88cc1a6650 con 0x7f88cc102af0 2024-05-22T23:02:58.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.393+0000 7f88d0de8640 1 --2- 172.21.15.84:0/1053952776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88cc103410 0x7f88cc100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:02:58.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.393+0000 7f88d1dea640 1 --2- 172.21.15.84:0/1053952776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88cc10be30 0x7f88cc1011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:02:58.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.394+0000 7f88d15e9640 1 --2- 172.21.15.84:0/1053952776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88cc102af0 0x7f88cc100740 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f88c8006010 tx=0x7f88c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:58.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.394+0000 7f88c27fc640 1 -- 172.21.15.84:0/1053952776 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c803f030 con 0x7f88cc102af0 2024-05-22T23:02:58.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.394+0000 7f88d384d640 1 -- 172.21.15.84:0/1053952776 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88cc1a68e0 con 0x7f88cc102af0 2024-05-22T23:02:58.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.394+0000 7f88c27fc640 1 -- 172.21.15.84:0/1053952776 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c8003ce0 con 0x7f88cc102af0 2024-05-22T23:02:58.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.394+0000 7f88d384d640 1 -- 172.21.15.84:0/1053952776 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88cc1a6e20 con 0x7f88cc102af0 2024-05-22T23:02:58.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.394+0000 7f88c27fc640 1 -- 172.21.15.84:0/1053952776 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c8035070 con 0x7f88cc102af0 2024-05-22T23:02:58.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.396+0000 7f88c27fc640 1 -- 172.21.15.84:0/1053952776 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f88c8042030 con 0x7f88cc102af0 2024-05-22T23:02:58.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.398+0000 7f88c27fc640 1 --2- 172.21.15.84:0/1053952776 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f88b80a2ea0 0x7f88b80a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:58.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.398+0000 7f88d6638740 1 Processor -- start 2024-05-22T23:02:58.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.398+0000 7f88d6638740 1 -- start start 2024-05-22T23:02:58.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.398+0000 7f88d0de8640 1 --2- 172.21.15.84:0/1053952776 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f88b80a2ea0 0x7f88b80a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:58.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.398+0000 7f88c27fc640 1 -- 172.21.15.84:0/1053952776 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f88c80efca0 con 0x7f88cc102af0 2024-05-22T23:02:58.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.398+0000 7f88d6638740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558671e5dfa0 0x558671e5e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:58.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.398+0000 7f88d6638740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5e8c0 0x558671e68a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:58.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.398+0000 7f88d6638740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558671e68f90 0x558671e6b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:58.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.398+0000 7f88d6638740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558671dc2db0 con 0x558671e5e8c0 2024-05-22T23:02:58.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.398+0000 7f88d6638740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558671d61400 con 0x558671e5dfa0 2024-05-22T23:02:58.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.398+0000 7f88d6638740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558671dbed70 con 0x558671e68f90 2024-05-22T23:02:58.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.398+0000 7f88d0de8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5e8c0 0x558671e68a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:58.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.398+0000 7f88d0de8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5e8c0 0x558671e68a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44984/0 (socket says 172.21.15.84:44984) 2024-05-22T23:02:58.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.398+0000 7f88d0de8640 1 -- 172.21.15.84:0/3195682360 learned_addr learned my addr 172.21.15.84:0/3195682360 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:58.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.398+0000 7f88d15e9640 1 --2- 172.21.15.84:0/3195682360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558671e5dfa0 0x558671e5e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:58.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.398+0000 7f88d1dea640 1 --2- 172.21.15.84:0/3195682360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558671e68f90 0x558671e6b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:58.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.399+0000 7f88d0de8640 1 --2- 172.21.15.84:0/1053952776 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f88b80a2ea0 0x7f88b80a5360 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0135f0 tx=0x7f88bc0139f0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:58.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.399+0000 7f88d15e9640 1 -- 172.21.15.84:0/3195682360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558671e68f90 msgr2=0x558671e6b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:58.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.399+0000 7f88d15e9640 1 --2- 172.21.15.84:0/3195682360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558671e68f90 0x558671e6b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.399+0000 7f88d15e9640 1 -- 172.21.15.84:0/3195682360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5e8c0 msgr2=0x558671e68a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:58.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.399+0000 7f88d15e9640 1 --2- 172.21.15.84:0/3195682360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5e8c0 0x558671e68a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.399+0000 7f88d15e9640 1 -- 172.21.15.84:0/3195682360 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558671d532f0 con 0x558671e5dfa0 2024-05-22T23:02:58.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.399+0000 7f88d0de8640 1 --2- 172.21.15.84:0/3195682360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5e8c0 0x558671e68a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:02:58.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.399+0000 7f88d1dea640 1 --2- 172.21.15.84:0/3195682360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558671e68f90 0x558671e6b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:02:58.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.399+0000 7f88d15e9640 1 --2- 172.21.15.84:0/3195682360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558671e5dfa0 0x558671e5e380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f88c80ed270 tx=0x7f88c80b8420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61dc3bccfc44dce2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:58.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.399+0000 7f88b77fe640 1 -- 172.21.15.84:0/3195682360 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c8049040 con 0x558671e5dfa0 2024-05-22T23:02:58.403 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.399+0000 7f88b77fe640 1 -- 172.21.15.84:0/3195682360 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c803b580 con 0x558671e5dfa0 2024-05-22T23:02:58.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.400+0000 7f88d6638740 1 -- 172.21.15.84:0/3195682360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558671e5dfa0 msgr2=0x558671e5e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:58.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.400+0000 7f88d6638740 1 --2- 172.21.15.84:0/3195682360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558671e5dfa0 0x558671e5e380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f88c80ed270 tx=0x7f88c80b8420 comp rx=0 tx=0).stop 2024-05-22T23:02:58.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.400+0000 7f88d6638740 1 -- 172.21.15.84:0/3195682360 shutdown_connections 2024-05-22T23:02:58.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.400+0000 7f88d6638740 1 --2- 172.21.15.84:0/3195682360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558671e68f90 0x558671e6b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.400+0000 7f88d6638740 1 --2- 172.21.15.84:0/3195682360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5e8c0 0x558671e68a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.400+0000 7f88d6638740 1 --2- 172.21.15.84:0/3195682360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558671e5dfa0 0x558671e5e380 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.400+0000 7f88d6638740 1 -- 172.21.15.84:0/3195682360 >> 172.21.15.84:0/3195682360 conn(0x558671e5a2c0 msgr2=0x558671e5a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:58.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.400+0000 7f88d6638740 1 -- 172.21.15.84:0/3195682360 shutdown_connections 2024-05-22T23:02:58.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.400+0000 7f88d6638740 1 -- 172.21.15.84:0/3195682360 wait complete. 2024-05-22T23:02:58.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.401+0000 7f88d6638740 1 -- start start 2024-05-22T23:02:58.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.401+0000 7f88d6638740 1 Processor -- start 2024-05-22T23:02:58.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.401+0000 7f88d6638740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5dfa0 0x558671e65e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:58.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.401+0000 7f88d6638740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558671e5e8c0 0x558671e663b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:58.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.401+0000 7f88d6638740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558671e68f90 0x558671e668f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:58.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.401+0000 7f88d6638740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558671d61400 con 0x558671e5dfa0 2024-05-22T23:02:58.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.401+0000 7f88d6638740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558671d7d2e0 con 0x558671e5e8c0 2024-05-22T23:02:58.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.401+0000 7f88d6638740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558671e6bb40 con 0x558671e68f90 2024-05-22T23:02:58.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.401+0000 7f88d6638740 10 client.0 _ll_register_callbacks cb 0x558671d86a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:02:58.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.401+0000 7f88d15e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5dfa0 0x558671e65e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:58.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.401+0000 7f88d15e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5dfa0 0x558671e65e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44990/0 (socket says 172.21.15.84:44990) 2024-05-22T23:02:58.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.401+0000 7f88d15e9640 1 -- 172.21.15.84:0/2086381105 learned_addr learned my addr 172.21.15.84:0/2086381105 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:58.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.402+0000 7f88d0de8640 1 --2- 172.21.15.84:0/2086381105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558671e5e8c0 0x558671e663b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:58.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.402+0000 7f88d15e9640 1 -- 172.21.15.84:0/2086381105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558671e68f90 msgr2=0x558671e668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:58.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.402+0000 7f88d15e9640 1 --2- 172.21.15.84:0/2086381105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558671e68f90 0x558671e668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.402+0000 7f88d15e9640 1 -- 172.21.15.84:0/2086381105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558671e5e8c0 msgr2=0x558671e663b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:58.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.402+0000 7f88d15e9640 1 --2- 172.21.15.84:0/2086381105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558671e5e8c0 0x558671e663b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.402+0000 7f88d15e9640 1 -- 172.21.15.84:0/2086381105 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558671ad63f0 con 0x558671e5dfa0 2024-05-22T23:02:58.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.402+0000 7f88d0de8640 1 --2- 172.21.15.84:0/2086381105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558671e5e8c0 0x558671e663b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:02:58.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.402+0000 7f88d15e9640 1 --2- 172.21.15.84:0/2086381105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5dfa0 0x558671e65e70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f88c803d040 tx=0x7f88c80ed1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad47d7c2da2fe64d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:58.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.402+0000 7f88b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:02:58.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.402+0000 7f88b4ff9640 1 -- 172.21.15.84:0/2086381105 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c8049070 con 0x558671e5dfa0 2024-05-22T23:02:58.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.402+0000 7f88b4ff9640 1 -- 172.21.15.84:0/2086381105 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f88c80c4ae0 con 0x558671e5dfa0 2024-05-22T23:02:58.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.402+0000 7f88d6638740 1 -- 172.21.15.84:0/2086381105 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558671dd04e0 con 0x558671e5dfa0 2024-05-22T23:02:58.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.402+0000 7f88b4ff9640 1 -- 172.21.15.84:0/2086381105 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f88c80bba60 con 0x558671e5dfa0 2024-05-22T23:02:58.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.402+0000 7f88d6638740 10 client.15600 fetch_fsmap learned FSMap version 18 2024-05-22T23:02:58.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.402+0000 7f88d6638740 1 -- 172.21.15.84:0/2086381105 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x558671ba07d0 con 0x558671e5dfa0 2024-05-22T23:02:58.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.403+0000 7f88b4ff9640 1 -- 172.21.15.84:0/2086381105 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558671dd04e0 con 0x558671e5dfa0 2024-05-22T23:02:58.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.403+0000 7f88b4ff9640 1 -- 172.21.15.84:0/2086381105 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f88c80ae030 con 0x558671e5dfa0 2024-05-22T23:02:58.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.403+0000 7f88d6638740 10 client.15600 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:02:58.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.403+0000 7f88d6638740 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:02:58.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.403+0000 7f88d6638740 20 client.15600 populate_metadata read hostname 'smithi084' 2024-05-22T23:02:58.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.403+0000 7f88d6638740 1 --2- 172.21.15.84:0/2086381105 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558671e78d30 0x558671e7b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:58.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.403+0000 7f88d6638740 4 client.15600 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:02:58.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.404+0000 7f88d6638740 1 -- 172.21.15.84:0/2086381105 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558671dca830 con 0x558671e78d30 2024-05-22T23:02:58.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.404+0000 7f88d0de8640 1 --2- 172.21.15.84:0/2086381105 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558671e78d30 0x558671e7b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:58.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.404+0000 7f88b4ff9640 10 client.15600 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:58.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.404+0000 7f88d0de8640 1 --2- 172.21.15.84:0/2086381105 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558671e78d30 0x558671e7b120 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bbe7c0bfe8b642c server_cookie=90b30b1ff274f738 in_seq=0 out_seq=0 2024-05-22T23:02:58.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.405+0000 7f88b4ff9640 1 -- 172.21.15.84:0/2086381105 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558671dca830 con 0x558671e78d30 2024-05-22T23:02:58.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.405+0000 7f88b4ff9640 10 client.15600 handle_command_reply: tid=0 2024-05-22T23:02:58.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.405+0000 7f88d6638740 1 client.15600 shutdown 2024-05-22T23:02:58.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.405+0000 7f88d6638740 1 -- 172.21.15.84:0/2086381105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5dfa0 msgr2=0x558671e65e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:58.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.405+0000 7f88d6638740 1 --2- 172.21.15.84:0/2086381105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5dfa0 0x558671e65e70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f88c803d040 tx=0x7f88c80ed1e0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.405+0000 7f88d6638740 1 -- 172.21.15.84:0/2086381105 shutdown_connections 2024-05-22T23:02:58.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.405+0000 7f88d6638740 1 --2- 172.21.15.84:0/2086381105 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558671e78d30 0x558671e7b120 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.405+0000 7f88d6638740 1 --2- 172.21.15.84:0/2086381105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558671e68f90 0x558671e668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.405+0000 7f88d6638740 1 --2- 172.21.15.84:0/2086381105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558671e5e8c0 0x558671e663b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.405+0000 7f88d6638740 1 --2- 172.21.15.84:0/2086381105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5dfa0 0x558671e65e70 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.405+0000 7f88d6638740 1 -- 172.21.15.84:0/2086381105 >> 172.21.15.84:0/2086381105 conn(0x558671e5a2c0 msgr2=0x558671e600c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:58.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.405+0000 7f88b4ff9640 0 client.15600 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:58.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.405+0000 7f88d6638740 1 -- 172.21.15.84:0/2086381105 shutdown_connections 2024-05-22T23:02:58.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.405+0000 7f88d6638740 1 -- 172.21.15.84:0/2086381105 wait complete. 2024-05-22T23:02:58.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.405+0000 7f88d6638740 20 client.15600 trim_cache size 0 max 16384 2024-05-22T23:02:58.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.431+0000 7f88d6638740 1 Processor -- start 2024-05-22T23:02:58.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.431+0000 7f88d6638740 1 -- start start 2024-05-22T23:02:58.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.431+0000 7f88d6638740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558671e5dfa0 0x558671e74920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:58.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.431+0000 7f88d6638740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5e8c0 0x558671e58c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:58.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.431+0000 7f88d6638740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558671e68f90 0x558671e66710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:58.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.431+0000 7f88d6638740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc1ad850 con 0x558671e5e8c0 2024-05-22T23:02:58.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.431+0000 7f88d6638740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc1b00d0 con 0x558671e5dfa0 2024-05-22T23:02:58.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.431+0000 7f88d6638740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc1adec0 con 0x558671e68f90 2024-05-22T23:02:58.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.431+0000 7f88d0de8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5e8c0 0x558671e58c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:58.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.431+0000 7f88d0de8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5e8c0 0x558671e58c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45004/0 (socket says 172.21.15.84:45004) 2024-05-22T23:02:58.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.431+0000 7f88d0de8640 1 -- 172.21.15.84:0/69939619 learned_addr learned my addr 172.21.15.84:0/69939619 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:58.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.431+0000 7f88d15e9640 1 --2- 172.21.15.84:0/69939619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558671e5dfa0 0x558671e74920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:58.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.432+0000 7f88d1dea640 1 --2- 172.21.15.84:0/69939619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558671e68f90 0x558671e66710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:58.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.432+0000 7f88d0de8640 1 -- 172.21.15.84:0/69939619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558671e68f90 msgr2=0x558671e66710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:58.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.432+0000 7f88d0de8640 1 --2- 172.21.15.84:0/69939619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558671e68f90 0x558671e66710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.432+0000 7f88d0de8640 1 -- 172.21.15.84:0/69939619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558671e5dfa0 msgr2=0x558671e74920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:58.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.432+0000 7f88d0de8640 1 --2- 172.21.15.84:0/69939619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558671e5dfa0 0x558671e74920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.432+0000 7f88d0de8640 1 -- 172.21.15.84:0/69939619 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558671e5a0f0 con 0x558671e5e8c0 2024-05-22T23:02:58.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.432+0000 7f88d0de8640 1 --2- 172.21.15.84:0/69939619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5e8c0 0x558671e58c30 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00b5b0 tx=0x7f88bc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6df6bc990c103b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:58.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.432+0000 7f88b6ffd640 1 -- 172.21.15.84:0/69939619 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc028dc0 con 0x558671e5e8c0 2024-05-22T23:02:58.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.432+0000 7f88b6ffd640 1 -- 172.21.15.84:0/69939619 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88bc00fe60 con 0x558671e5e8c0 2024-05-22T23:02:58.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.432+0000 7f88b6ffd640 1 -- 172.21.15.84:0/69939619 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc020e60 con 0x558671e5e8c0 2024-05-22T23:02:58.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.433+0000 7f88d6638740 1 -- 172.21.15.84:0/69939619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5e8c0 msgr2=0x558671e58c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:58.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.433+0000 7f88d6638740 1 --2- 172.21.15.84:0/69939619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5e8c0 0x558671e58c30 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00b5b0 tx=0x7f88bc01c000 comp rx=0 tx=0).stop 2024-05-22T23:02:58.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.433+0000 7f88d6638740 1 -- 172.21.15.84:0/69939619 shutdown_connections 2024-05-22T23:02:58.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.433+0000 7f88d6638740 1 --2- 172.21.15.84:0/69939619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558671e68f90 0x558671e66710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.433+0000 7f88d6638740 1 --2- 172.21.15.84:0/69939619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5e8c0 0x558671e58c30 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.433+0000 7f88d6638740 1 --2- 172.21.15.84:0/69939619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558671e5dfa0 0x558671e74920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.433+0000 7f88d6638740 1 -- 172.21.15.84:0/69939619 >> 172.21.15.84:0/69939619 conn(0x558671e5a2c0 msgr2=0x558671e60000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:58.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.433+0000 7f88d6638740 1 -- 172.21.15.84:0/69939619 shutdown_connections 2024-05-22T23:02:58.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.434+0000 7f88d6638740 1 -- 172.21.15.84:0/69939619 wait complete. 2024-05-22T23:02:58.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.434+0000 7f88d6638740 1 -- start start 2024-05-22T23:02:58.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.434+0000 7f88d6638740 1 Processor -- start 2024-05-22T23:02:58.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.434+0000 7f88d6638740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5dfa0 0x558671e62b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:58.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.434+0000 7f88d6638740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558671e5e8c0 0x558671e630d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:58.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.434+0000 7f88d6638740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558671e68f90 0x558671e63610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:58.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.435+0000 7f88d6638740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc1ae590 con 0x558671e5dfa0 2024-05-22T23:02:58.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.435+0000 7f88d6638740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc1afa00 con 0x558671e5e8c0 2024-05-22T23:02:58.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.435+0000 7f88d6638740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc1aec60 con 0x558671e68f90 2024-05-22T23:02:58.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.435+0000 7f88d6638740 10 client.0 _ll_register_callbacks cb 0x7f88bc00b4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:02:58.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.435+0000 7f88d15e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5dfa0 0x558671e62b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:58.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.435+0000 7f88d15e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5dfa0 0x558671e62b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45008/0 (socket says 172.21.15.84:45008) 2024-05-22T23:02:58.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.435+0000 7f88d15e9640 1 -- 172.21.15.84:0/1129197766 learned_addr learned my addr 172.21.15.84:0/1129197766 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:58.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.435+0000 7f88d1dea640 1 --2- 172.21.15.84:0/1129197766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558671e68f90 0x558671e63610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:58.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.435+0000 7f88d0de8640 1 --2- 172.21.15.84:0/1129197766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558671e5e8c0 0x558671e630d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:58.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.435+0000 7f88d1dea640 1 -- 172.21.15.84:0/1129197766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558671e5e8c0 msgr2=0x558671e630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:58.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.435+0000 7f88d1dea640 1 --2- 172.21.15.84:0/1129197766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558671e5e8c0 0x558671e630d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.435+0000 7f88d1dea640 1 -- 172.21.15.84:0/1129197766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5dfa0 msgr2=0x558671e62b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:58.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.435+0000 7f88d1dea640 1 --2- 172.21.15.84:0/1129197766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5dfa0 0x558671e62b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.435+0000 7f88d1dea640 1 -- 172.21.15.84:0/1129197766 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558671ede0b0 con 0x558671e68f90 2024-05-22T23:02:58.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.435+0000 7f88d15e9640 1 --2- 172.21.15.84:0/1129197766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5dfa0 0x558671e62b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:02:58.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.435+0000 7f88d1dea640 1 --2- 172.21.15.84:0/1129197766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558671e68f90 0x558671e63610 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x558671e5b450 tx=0x7f88c4010eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85949c9f291cf786 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:58.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.435+0000 7f88b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-22T23:02:58.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.436+0000 7f88b4ff9640 1 -- 172.21.15.84:0/1129197766 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c401e060 con 0x558671e68f90 2024-05-22T23:02:58.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.436+0000 7f88b4ff9640 1 -- 172.21.15.84:0/1129197766 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f88c400eb30 con 0x558671e68f90 2024-05-22T23:02:58.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.436+0000 7f88d6638740 1 -- 172.21.15.84:0/1129197766 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88bc00fe60 con 0x558671e68f90 2024-05-22T23:02:58.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.436+0000 7f88b4ff9640 1 -- 172.21.15.84:0/1129197766 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f88c400ed30 con 0x558671e68f90 2024-05-22T23:02:58.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.436+0000 7f88d6638740 10 client.25102 fetch_fsmap learned FSMap version 18 2024-05-22T23:02:58.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.436+0000 7f88d6638740 1 -- 172.21.15.84:0/1129197766 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x558671edf3f0 con 0x558671e68f90 2024-05-22T23:02:58.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.436+0000 7f88b4ff9640 1 -- 172.21.15.84:0/1129197766 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c40198f0 con 0x558671e68f90 2024-05-22T23:02:58.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.437+0000 7f88b4ff9640 1 -- 172.21.15.84:0/1129197766 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f88c4021550 con 0x558671e68f90 2024-05-22T23:02:58.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.437+0000 7f88d6638740 10 client.25102 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:02:58.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.437+0000 7f88d6638740 10 client.25102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:02:58.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.437+0000 7f88d6638740 20 client.25102 populate_metadata read hostname 'smithi084' 2024-05-22T23:02:58.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.437+0000 7f88d6638740 1 --2- 172.21.15.84:0/1129197766 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558671edfba0 0x558671ee1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:58.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.437+0000 7f88d6638740 4 client.25102 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-22T23:02:58.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.437+0000 7f88d6638740 1 -- 172.21.15.84:0/1129197766 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f88bc00fe60 con 0x558671edfba0 2024-05-22T23:02:58.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.437+0000 7f88d15e9640 1 --2- 172.21.15.84:0/1129197766 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558671edfba0 0x558671ee1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:58.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.437+0000 7f88b4ff9640 10 client.25102 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:58.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.437+0000 7f88d15e9640 1 --2- 172.21.15.84:0/1129197766 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558671edfba0 0x558671ee1fc0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c057d6d08c5b8f6 server_cookie=a6f6b6cd82deb85b in_seq=0 out_seq=0 2024-05-22T23:02:58.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.438+0000 7f88b4ff9640 1 -- 172.21.15.84:0/1129197766 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f88bc00fe60 con 0x558671edfba0 2024-05-22T23:02:58.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.438+0000 7f88b4ff9640 10 client.25102 handle_command_reply: tid=0 2024-05-22T23:02:58.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.438+0000 7f88d6638740 1 client.25102 shutdown 2024-05-22T23:02:58.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.438+0000 7f88d6638740 1 -- 172.21.15.84:0/1129197766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558671e68f90 msgr2=0x558671e63610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:58.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.438+0000 7f88d6638740 1 --2- 172.21.15.84:0/1129197766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558671e68f90 0x558671e63610 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x558671e5b450 tx=0x7f88c4010eb0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.438+0000 7f88d6638740 1 -- 172.21.15.84:0/1129197766 shutdown_connections 2024-05-22T23:02:58.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.438+0000 7f88d6638740 1 --2- 172.21.15.84:0/1129197766 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558671edfba0 0x558671ee1fc0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.438+0000 7f88d6638740 1 --2- 172.21.15.84:0/1129197766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558671e68f90 0x558671e63610 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.438+0000 7f88d6638740 1 --2- 172.21.15.84:0/1129197766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558671e5e8c0 0x558671e630d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.438+0000 7f88d6638740 1 --2- 172.21.15.84:0/1129197766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558671e5dfa0 0x558671e62b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.438+0000 7f88d6638740 1 -- 172.21.15.84:0/1129197766 >> 172.21.15.84:0/1129197766 conn(0x558671e5a2c0 msgr2=0x558671e643e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:58.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.438+0000 7f88b4ff9640 0 client.25102 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:58.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.438+0000 7f88d6638740 1 -- 172.21.15.84:0/1129197766 shutdown_connections 2024-05-22T23:02:58.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.438+0000 7f88d6638740 1 -- 172.21.15.84:0/1129197766 wait complete. 2024-05-22T23:02:58.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.438+0000 7f88d6638740 20 client.25102 trim_cache size 0 max 16384 2024-05-22T23:02:58.446 INFO:teuthology.orchestra.run.smithi084.stdout:[] 2024-05-22T23:02:58.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.439+0000 7f88d384d640 1 -- 172.21.15.84:0/1053952776 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f88b80a2ea0 msgr2=0x7f88b80a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:58.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.439+0000 7f88d384d640 1 --2- 172.21.15.84:0/1053952776 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f88b80a2ea0 0x7f88b80a5360 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0135f0 tx=0x7f88bc0139f0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.439+0000 7f88d384d640 1 -- 172.21.15.84:0/1053952776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88cc102af0 msgr2=0x7f88cc100740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:58.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.439+0000 7f88d384d640 1 --2- 172.21.15.84:0/1053952776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88cc102af0 0x7f88cc100740 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f88c8006010 tx=0x7f88c80042f0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.439+0000 7f88d384d640 1 -- 172.21.15.84:0/1053952776 shutdown_connections 2024-05-22T23:02:58.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.439+0000 7f88d384d640 1 --2- 172.21.15.84:0/1053952776 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f88b80a2ea0 0x7f88b80a5360 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.439+0000 7f88d384d640 1 --2- 172.21.15.84:0/1053952776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88cc10be30 0x7f88cc1011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.439+0000 7f88d384d640 1 --2- 172.21.15.84:0/1053952776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88cc103410 0x7f88cc100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.448 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.439+0000 7f88d384d640 1 --2- 172.21.15.84:0/1053952776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88cc102af0 0x7f88cc100740 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:58.448 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.440+0000 7f88d384d640 1 -- 172.21.15.84:0/1053952776 >> 172.21.15.84:0/1053952776 conn(0x7f88cc077870 msgr2=0x7f88cc0fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:58.448 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.440+0000 7f88d384d640 1 -- 172.21.15.84:0/1053952776 shutdown_connections 2024-05-22T23:02:58.448 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:58.440+0000 7f88d384d640 1 -- 172.21.15.84:0/1053952776 wait complete. 2024-05-22T23:02:59.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:59 smithi096 ceph-mon[36392]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:02:59.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:02:59 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/136391537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:02:59.377 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:59 smithi084 ceph-mon[32186]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:02:59.377 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:02:59 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/136391537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:02:59.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:59 smithi204 ceph-mon[36211]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:02:59.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:02:59 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/136391537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:02:59.456 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-22T23:02:59.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:02:59.457 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-22T23:02:59.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.574+0000 7f2e448a5640 1 -- 172.21.15.84:0/2006601176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e40101100 msgr2=0x7f2e4010d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:59.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.574+0000 7f2e448a5640 1 --2- 172.21.15.84:0/2006601176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e40101100 0x7f2e4010d610 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009cd0 tx=0x7f2e3802a8e0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.574+0000 7f2e448a5640 1 -- 172.21.15.84:0/2006601176 shutdown_connections 2024-05-22T23:02:59.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.574+0000 7f2e448a5640 1 --2- 172.21.15.84:0/2006601176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e40101100 0x7f2e4010d610 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.574+0000 7f2e448a5640 1 --2- 172.21.15.84:0/2006601176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e40100740 0x7f2e40100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.574+0000 7f2e448a5640 1 --2- 172.21.15.84:0/2006601176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e40106750 0x7f2e40106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.574+0000 7f2e448a5640 1 -- 172.21.15.84:0/2006601176 >> 172.21.15.84:0/2006601176 conn(0x7f2e40077870 msgr2=0x7f2e40077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:59.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.574+0000 7f2e448a5640 1 -- 172.21.15.84:0/2006601176 shutdown_connections 2024-05-22T23:02:59.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.574+0000 7f2e448a5640 1 -- 172.21.15.84:0/2006601176 wait complete. 2024-05-22T23:02:59.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.575+0000 7f2e448a5640 1 Processor -- start 2024-05-22T23:02:59.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.575+0000 7f2e448a5640 1 -- start start 2024-05-22T23:02:59.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.575+0000 7f2e448a5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e40100740 0x7f2e4010df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.575+0000 7f2e448a5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e40101100 0x7f2e4010e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.575+0000 7f2e448a5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e40106750 0x7f2e4010e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.575+0000 7f2e448a5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2e40111cd0 con 0x7f2e40100740 2024-05-22T23:02:59.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.575+0000 7f2e448a5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2e40111b50 con 0x7f2e40106750 2024-05-22T23:02:59.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.575+0000 7f2e448a5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2e40111e50 con 0x7f2e40101100 2024-05-22T23:02:59.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.575+0000 7f2e3e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e40100740 0x7f2e4010df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.575+0000 7f2e3e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e40100740 0x7f2e4010df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45022/0 (socket says 172.21.15.84:45022) 2024-05-22T23:02:59.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.575+0000 7f2e3e59c640 1 -- 172.21.15.84:0/3858499911 learned_addr learned my addr 172.21.15.84:0/3858499911 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:59.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.575+0000 7f2e3dd9b640 1 --2- 172.21.15.84:0/3858499911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e40101100 0x7f2e4010e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.575+0000 7f2e3ed9d640 1 --2- 172.21.15.84:0/3858499911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e40106750 0x7f2e4010e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.576+0000 7f2e3e59c640 1 -- 172.21.15.84:0/3858499911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e40101100 msgr2=0x7f2e4010e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:59.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.576+0000 7f2e3e59c640 1 --2- 172.21.15.84:0/3858499911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e40101100 0x7f2e4010e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.576+0000 7f2e3e59c640 1 -- 172.21.15.84:0/3858499911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e40106750 msgr2=0x7f2e4010e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:59.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.576+0000 7f2e3e59c640 1 --2- 172.21.15.84:0/3858499911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e40106750 0x7f2e4010e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.576+0000 7f2e3e59c640 1 -- 172.21.15.84:0/3858499911 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e401aaab0 con 0x7f2e40100740 2024-05-22T23:02:59.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.576+0000 7f2e3ed9d640 1 --2- 172.21.15.84:0/3858499911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e40106750 0x7f2e4010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:02:59.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.576+0000 7f2e3e59c640 1 --2- 172.21.15.84:0/3858499911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e40100740 0x7f2e4010df20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000e9f0 tx=0x7f2e3000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:59.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.576+0000 7f2e2f7fe640 1 -- 172.21.15.84:0/3858499911 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3000cd80 con 0x7f2e40100740 2024-05-22T23:02:59.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.576+0000 7f2e448a5640 1 -- 172.21.15.84:0/3858499911 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e401aace0 con 0x7f2e40100740 2024-05-22T23:02:59.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.576+0000 7f2e448a5640 1 -- 172.21.15.84:0/3858499911 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e401ab1d0 con 0x7f2e40100740 2024-05-22T23:02:59.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.576+0000 7f2e2f7fe640 1 -- 172.21.15.84:0/3858499911 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e30005500 con 0x7f2e40100740 2024-05-22T23:02:59.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.576+0000 7f2e2f7fe640 1 -- 172.21.15.84:0/3858499911 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e30010690 con 0x7f2e40100740 2024-05-22T23:02:59.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.579+0000 7f2e2f7fe640 1 -- 172.21.15.84:0/3858499911 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f2e300046c0 con 0x7f2e40100740 2024-05-22T23:02:59.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.580+0000 7f2e475b8740 1 Processor -- start 2024-05-22T23:02:59.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.580+0000 7f2e2f7fe640 1 --2- 172.21.15.84:0/3858499911 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f2e280a2ea0 0x7f2e280a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.580+0000 7f2e2f7fe640 1 -- 172.21.15.84:0/3858499911 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f2e300cf9b0 con 0x7f2e40100740 2024-05-22T23:02:59.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.580+0000 7f2e3ed9d640 1 --2- 172.21.15.84:0/3858499911 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f2e280a2ea0 0x7f2e280a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.580+0000 7f2e475b8740 1 -- start start 2024-05-22T23:02:59.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.580+0000 7f2e475b8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b33825ac50 0x55b33825b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.580+0000 7f2e475b8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b33825b570 0x55b338265700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.580+0000 7f2e475b8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b338265c40 0x55b338268030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.580+0000 7f2e475b8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b3381bea40 con 0x55b33825ac50 2024-05-22T23:02:59.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.580+0000 7f2e475b8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b33815d440 con 0x55b338265c40 2024-05-22T23:02:59.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.580+0000 7f2e475b8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b3381baa00 con 0x55b33825b570 2024-05-22T23:02:59.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.581+0000 7f2e3ed9d640 1 --2- 172.21.15.84:0/3858499911 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f2e280a2ea0 0x7f2e280a5360 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2e38005e30 tx=0x7f2e38005d80 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:59.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.581+0000 7f2e3e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b33825ac50 0x55b33825b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.581+0000 7f2e3e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b33825ac50 0x55b33825b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45034/0 (socket says 172.21.15.84:45034) 2024-05-22T23:02:59.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.581+0000 7f2e3e59c640 1 -- 172.21.15.84:0/218618515 learned_addr learned my addr 172.21.15.84:0/218618515 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:59.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.581+0000 7f2e3dd9b640 1 --2- 172.21.15.84:0/218618515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b33825b570 0x55b338265700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.581+0000 7f2e3ed9d640 1 --2- 172.21.15.84:0/218618515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b338265c40 0x55b338268030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.581+0000 7f2e3e59c640 1 -- 172.21.15.84:0/218618515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b33825b570 msgr2=0x55b338265700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:59.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.581+0000 7f2e3e59c640 1 --2- 172.21.15.84:0/218618515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b33825b570 0x55b338265700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.581+0000 7f2e3e59c640 1 -- 172.21.15.84:0/218618515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b338265c40 msgr2=0x55b338268030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:59.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.581+0000 7f2e3e59c640 1 --2- 172.21.15.84:0/218618515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b338265c40 0x55b338268030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.581+0000 7f2e3e59c640 1 -- 172.21.15.84:0/218618515 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3381672d0 con 0x55b33825ac50 2024-05-22T23:02:59.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.581+0000 7f2e3dd9b640 1 --2- 172.21.15.84:0/218618515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b33825b570 0x55b338265700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:02:59.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.581+0000 7f2e3e59c640 1 --2- 172.21.15.84:0/218618515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b33825ac50 0x55b33825b030 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000e9c0 tx=0x7f2e3009b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5080f8674491743 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:59.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.582+0000 7f2e2cff9640 1 -- 172.21.15.84:0/218618515 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e300a48f0 con 0x55b33825ac50 2024-05-22T23:02:59.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.582+0000 7f2e2cff9640 1 -- 172.21.15.84:0/218618515 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e30099f80 con 0x55b33825ac50 2024-05-22T23:02:59.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.582+0000 7f2e2cff9640 1 -- 172.21.15.84:0/218618515 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e300cd4c0 con 0x55b33825ac50 2024-05-22T23:02:59.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.582+0000 7f2e475b8740 1 -- 172.21.15.84:0/218618515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b33825ac50 msgr2=0x55b33825b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:59.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.582+0000 7f2e475b8740 1 --2- 172.21.15.84:0/218618515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b33825ac50 0x55b33825b030 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000e9c0 tx=0x7f2e3009b6a0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.582+0000 7f2e475b8740 1 -- 172.21.15.84:0/218618515 shutdown_connections 2024-05-22T23:02:59.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.582+0000 7f2e475b8740 1 --2- 172.21.15.84:0/218618515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b338265c40 0x55b338268030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.582+0000 7f2e475b8740 1 --2- 172.21.15.84:0/218618515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b33825b570 0x55b338265700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.582+0000 7f2e475b8740 1 --2- 172.21.15.84:0/218618515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b33825ac50 0x55b33825b030 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.582+0000 7f2e475b8740 1 -- 172.21.15.84:0/218618515 >> 172.21.15.84:0/218618515 conn(0x55b338256f70 msgr2=0x55b338257350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:59.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.583+0000 7f2e475b8740 1 -- 172.21.15.84:0/218618515 shutdown_connections 2024-05-22T23:02:59.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.583+0000 7f2e475b8740 1 -- 172.21.15.84:0/218618515 wait complete. 2024-05-22T23:02:59.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.583+0000 7f2e475b8740 1 -- start start 2024-05-22T23:02:59.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.583+0000 7f2e475b8740 1 Processor -- start 2024-05-22T23:02:59.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.583+0000 7f2e475b8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b33825ac50 0x55b338262be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.583+0000 7f2e475b8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b33825b570 0x55b338263120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.583+0000 7f2e475b8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b338265c40 0x55b338263660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.583+0000 7f2e475b8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b338268a30 con 0x55b338265c40 2024-05-22T23:02:59.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.583+0000 7f2e475b8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b3381485f0 con 0x55b33825ac50 2024-05-22T23:02:59.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.583+0000 7f2e475b8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b338268bb0 con 0x55b33825b570 2024-05-22T23:02:59.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.583+0000 7f2e475b8740 10 client.0 _ll_register_callbacks cb 0x55b338143cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:02:59.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.583+0000 7f2e3ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b338265c40 0x55b338263660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.583+0000 7f2e3e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b33825ac50 0x55b338262be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.583+0000 7f2e3ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b338265c40 0x55b338263660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45038/0 (socket says 172.21.15.84:45038) 2024-05-22T23:02:59.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.583+0000 7f2e3ed9d640 1 -- 172.21.15.84:0/3525106263 learned_addr learned my addr 172.21.15.84:0/3525106263 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:59.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.583+0000 7f2e3dd9b640 1 --2- 172.21.15.84:0/3525106263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b33825b570 0x55b338263120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.584+0000 7f2e3e59c640 1 -- 172.21.15.84:0/3525106263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b33825b570 msgr2=0x55b338263120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:59.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.584+0000 7f2e3e59c640 1 --2- 172.21.15.84:0/3525106263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b33825b570 0x55b338263120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.584+0000 7f2e3e59c640 1 -- 172.21.15.84:0/3525106263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b338265c40 msgr2=0x55b338263660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:59.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.584+0000 7f2e3e59c640 1 --2- 172.21.15.84:0/3525106263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b338265c40 0x55b338263660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.584+0000 7f2e3e59c640 1 -- 172.21.15.84:0/3525106263 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b337f9a7e0 con 0x55b33825ac50 2024-05-22T23:02:59.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.584+0000 7f2e3ed9d640 1 --2- 172.21.15.84:0/3525106263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b338265c40 0x55b338263660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:02:59.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.584+0000 7f2e3e59c640 1 --2- 172.21.15.84:0/3525106263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b33825ac50 0x55b338262be0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55b338268600 tx=0x7f2e300cd610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5522e9ff7a2705 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:59.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.584+0000 7f2e25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:02:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.584+0000 7f2e25ffb640 1 -- 172.21.15.84:0/3525106263 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e300a4d40 con 0x55b33825ac50 2024-05-22T23:02:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.584+0000 7f2e25ffb640 1 -- 172.21.15.84:0/3525106263 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f2e3009b0d0 con 0x55b33825ac50 2024-05-22T23:02:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.584+0000 7f2e475b8740 1 -- 172.21.15.84:0/3525106263 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e30099f80 con 0x55b33825ac50 2024-05-22T23:02:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.584+0000 7f2e25ffb640 1 -- 172.21.15.84:0/3525106263 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f2e300a48f0 con 0x55b33825ac50 2024-05-22T23:02:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.584+0000 7f2e475b8740 10 client.25205 fetch_fsmap learned FSMap version 18 2024-05-22T23:02:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.584+0000 7f2e475b8740 1 -- 172.21.15.84:0/3525106263 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55b337ffaf70 con 0x55b33825ac50 2024-05-22T23:02:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.585+0000 7f2e25ffb640 1 -- 172.21.15.84:0/3525106263 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e30099f80 con 0x55b33825ac50 2024-05-22T23:02:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.585+0000 7f2e25ffb640 1 -- 172.21.15.84:0/3525106263 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f2e300ac030 con 0x55b33825ac50 2024-05-22T23:02:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.585+0000 7f2e475b8740 10 client.25205 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:02:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.585+0000 7f2e475b8740 10 client.25205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:02:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.585+0000 7f2e475b8740 20 client.25205 populate_metadata read hostname 'smithi084' 2024-05-22T23:02:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.585+0000 7f2e475b8740 1 --2- 172.21.15.84:0/3525106263 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b338275ea0 0x55b338278290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.585+0000 7f2e475b8740 4 client.25205 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:02:59.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.585+0000 7f2e475b8740 1 -- 172.21.15.84:0/3525106263 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3381d2460 con 0x55b338275ea0 2024-05-22T23:02:59.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.585+0000 7f2e3dd9b640 1 --2- 172.21.15.84:0/3525106263 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b338275ea0 0x55b338278290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.586+0000 7f2e25ffb640 10 client.25205 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:59.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.586+0000 7f2e3dd9b640 1 --2- 172.21.15.84:0/3525106263 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b338275ea0 0x55b338278290 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=772a013df0c8d866 server_cookie=47e0d8e155fb245d in_seq=0 out_seq=0 2024-05-22T23:02:59.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.586+0000 7f2e25ffb640 1 -- 172.21.15.84:0/3525106263 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b3381d2460 con 0x55b338275ea0 2024-05-22T23:02:59.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.586+0000 7f2e25ffb640 10 client.25205 handle_command_reply: tid=0 2024-05-22T23:02:59.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.586+0000 7f2e475b8740 1 client.25205 shutdown 2024-05-22T23:02:59.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.586+0000 7f2e475b8740 1 -- 172.21.15.84:0/3525106263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b33825ac50 msgr2=0x55b338262be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:59.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.586+0000 7f2e475b8740 1 --2- 172.21.15.84:0/3525106263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b33825ac50 0x55b338262be0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55b338268600 tx=0x7f2e300cd610 comp rx=0 tx=0).stop 2024-05-22T23:02:59.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.586+0000 7f2e475b8740 1 -- 172.21.15.84:0/3525106263 shutdown_connections 2024-05-22T23:02:59.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.586+0000 7f2e475b8740 1 --2- 172.21.15.84:0/3525106263 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b338275ea0 0x55b338278290 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.586+0000 7f2e475b8740 1 --2- 172.21.15.84:0/3525106263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b338265c40 0x55b338263660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.586+0000 7f2e475b8740 1 --2- 172.21.15.84:0/3525106263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b33825b570 0x55b338263120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.586+0000 7f2e475b8740 1 --2- 172.21.15.84:0/3525106263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b33825ac50 0x55b338262be0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.586+0000 7f2e475b8740 1 -- 172.21.15.84:0/3525106263 >> 172.21.15.84:0/3525106263 conn(0x55b338256f70 msgr2=0x55b3382561b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:59.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.586+0000 7f2e25ffb640 0 client.25205 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:59.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.587+0000 7f2e475b8740 1 -- 172.21.15.84:0/3525106263 shutdown_connections 2024-05-22T23:02:59.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.587+0000 7f2e475b8740 1 -- 172.21.15.84:0/3525106263 wait complete. 2024-05-22T23:02:59.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.587+0000 7f2e475b8740 20 client.25205 trim_cache size 0 max 16384 2024-05-22T23:02:59.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.604+0000 7f2e475b8740 1 Processor -- start 2024-05-22T23:02:59.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.604+0000 7f2e475b8740 1 -- start start 2024-05-22T23:02:59.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.604+0000 7f2e475b8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b33825ac50 0x55b338271920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.604+0000 7f2e475b8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b33825b570 0x55b3382634e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.605+0000 7f2e475b8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b338265c40 0x55b33827d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.605+0000 7f2e475b8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2000c740 con 0x55b33825ac50 2024-05-22T23:02:59.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.605+0000 7f2e475b8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2000efc0 con 0x55b33825b570 2024-05-22T23:02:59.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.605+0000 7f2e475b8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2000cdb0 con 0x55b338265c40 2024-05-22T23:02:59.608 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.605+0000 7f2e3e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b33825ac50 0x55b338271920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.608 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.605+0000 7f2e3e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b33825ac50 0x55b338271920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45048/0 (socket says 172.21.15.84:45048) 2024-05-22T23:02:59.608 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.605+0000 7f2e3e59c640 1 -- 172.21.15.84:0/2822192938 learned_addr learned my addr 172.21.15.84:0/2822192938 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:59.608 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.605+0000 7f2e3dd9b640 1 --2- 172.21.15.84:0/2822192938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b33825b570 0x55b3382634e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.608 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.605+0000 7f2e3ed9d640 1 --2- 172.21.15.84:0/2822192938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b338265c40 0x55b33827d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.608 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.605+0000 7f2e3e59c640 1 -- 172.21.15.84:0/2822192938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b338265c40 msgr2=0x55b33827d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:59.608 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.605+0000 7f2e3e59c640 1 --2- 172.21.15.84:0/2822192938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b338265c40 0x55b33827d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.608 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.605+0000 7f2e3e59c640 1 -- 172.21.15.84:0/2822192938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b33825b570 msgr2=0x55b3382634e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:59.608 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.605+0000 7f2e3e59c640 1 --2- 172.21.15.84:0/2822192938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b33825b570 0x55b3382634e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.608 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.605+0000 7f2e3e59c640 1 -- 172.21.15.84:0/2822192938 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3382683a0 con 0x55b33825ac50 2024-05-22T23:02:59.608 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.605+0000 7f2e3e59c640 1 --2- 172.21.15.84:0/2822192938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b33825ac50 0x55b338271920 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2e300cd700 tx=0x7f2e300057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5196a045a9bb9c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:59.608 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.605+0000 7f2e27fff640 1 -- 172.21.15.84:0/2822192938 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3009ae60 con 0x55b33825ac50 2024-05-22T23:02:59.608 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.606+0000 7f2e27fff640 1 -- 172.21.15.84:0/2822192938 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e30099f80 con 0x55b33825ac50 2024-05-22T23:02:59.609 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.606+0000 7f2e27fff640 1 -- 172.21.15.84:0/2822192938 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e300a4da0 con 0x55b33825ac50 2024-05-22T23:02:59.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.606+0000 7f2e475b8740 1 -- 172.21.15.84:0/2822192938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b33825ac50 msgr2=0x55b338271920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:59.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.606+0000 7f2e475b8740 1 --2- 172.21.15.84:0/2822192938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b33825ac50 0x55b338271920 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2e300cd700 tx=0x7f2e300057b0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.606+0000 7f2e475b8740 1 -- 172.21.15.84:0/2822192938 shutdown_connections 2024-05-22T23:02:59.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.606+0000 7f2e475b8740 1 --2- 172.21.15.84:0/2822192938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b338265c40 0x55b33827d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.606+0000 7f2e475b8740 1 --2- 172.21.15.84:0/2822192938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b33825b570 0x55b3382634e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.606+0000 7f2e475b8740 1 --2- 172.21.15.84:0/2822192938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b33825ac50 0x55b338271920 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.606+0000 7f2e475b8740 1 -- 172.21.15.84:0/2822192938 >> 172.21.15.84:0/2822192938 conn(0x55b338256f70 msgr2=0x55b33825cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:59.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.606+0000 7f2e475b8740 1 -- 172.21.15.84:0/2822192938 shutdown_connections 2024-05-22T23:02:59.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.606+0000 7f2e475b8740 1 -- 172.21.15.84:0/2822192938 wait complete. 2024-05-22T23:02:59.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.607+0000 7f2e475b8740 1 -- start start 2024-05-22T23:02:59.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.607+0000 7f2e475b8740 1 Processor -- start 2024-05-22T23:02:59.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.607+0000 7f2e475b8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b33825ac50 0x55b33826e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.607+0000 7f2e475b8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b33825b570 0x55b33825f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.607+0000 7f2e475b8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b338265c40 0x55b33825fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.607+0000 7f2e475b8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2000d480 con 0x55b338265c40 2024-05-22T23:02:59.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.607+0000 7f2e475b8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2000e8f0 con 0x55b33825ac50 2024-05-22T23:02:59.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.607+0000 7f2e475b8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2000db50 con 0x55b33825b570 2024-05-22T23:02:59.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.607+0000 7f2e3ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b338265c40 0x55b33825fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.607+0000 7f2e475b8740 10 client.0 _ll_register_callbacks cb 0x7f2e34009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:02:59.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.607+0000 7f2e3ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b338265c40 0x55b33825fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45060/0 (socket says 172.21.15.84:45060) 2024-05-22T23:02:59.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.607+0000 7f2e3ed9d640 1 -- 172.21.15.84:0/1413010338 learned_addr learned my addr 172.21.15.84:0/1413010338 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:59.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.607+0000 7f2e3dd9b640 1 --2- 172.21.15.84:0/1413010338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b33825b570 0x55b33825f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.607+0000 7f2e3e59c640 1 --2- 172.21.15.84:0/1413010338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b33825ac50 0x55b33826e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.608+0000 7f2e3ed9d640 1 -- 172.21.15.84:0/1413010338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b33825b570 msgr2=0x55b33825f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:59.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.608+0000 7f2e3ed9d640 1 --2- 172.21.15.84:0/1413010338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b33825b570 0x55b33825f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.608+0000 7f2e3ed9d640 1 -- 172.21.15.84:0/1413010338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b33825ac50 msgr2=0x55b33826e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:59.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.608+0000 7f2e3ed9d640 1 --2- 172.21.15.84:0/1413010338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b33825ac50 0x55b33826e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.608+0000 7f2e3ed9d640 1 -- 172.21.15.84:0/1413010338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b337e92170 con 0x55b338265c40 2024-05-22T23:02:59.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.608+0000 7f2e3dd9b640 1 --2- 172.21.15.84:0/1413010338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b33825b570 0x55b33825f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:02:59.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.608+0000 7f2e3e59c640 1 --2- 172.21.15.84:0/1413010338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b33825ac50 0x55b33826e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:02:59.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.608+0000 7f2e3ed9d640 1 --2- 172.21.15.84:0/1413010338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b338265c40 0x55b33825fe30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2e38035000 tx=0x7f2e3802a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8ac2e4d5daf9f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:59.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.608+0000 7f2e25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:02:59.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.608+0000 7f2e25ffb640 1 -- 172.21.15.84:0/1413010338 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38041070 con 0x55b338265c40 2024-05-22T23:02:59.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.608+0000 7f2e25ffb640 1 -- 172.21.15.84:0/1413010338 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f2e380047d0 con 0x55b338265c40 2024-05-22T23:02:59.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.608+0000 7f2e475b8740 1 -- 172.21.15.84:0/1413010338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e30099f80 con 0x55b338265c40 2024-05-22T23:02:59.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.608+0000 7f2e25ffb640 1 -- 172.21.15.84:0/1413010338 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f2e380305a0 con 0x55b338265c40 2024-05-22T23:02:59.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.608+0000 7f2e475b8740 10 client.15624 fetch_fsmap learned FSMap version 18 2024-05-22T23:02:59.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.608+0000 7f2e475b8740 1 -- 172.21.15.84:0/1413010338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55b338187930 con 0x55b338265c40 2024-05-22T23:02:59.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.609+0000 7f2e25ffb640 1 -- 172.21.15.84:0/1413010338 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e30099f80 con 0x55b338265c40 2024-05-22T23:02:59.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.609+0000 7f2e25ffb640 1 -- 172.21.15.84:0/1413010338 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f2e38030070 con 0x55b338265c40 2024-05-22T23:02:59.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.609+0000 7f2e475b8740 10 client.15624 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:02:59.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.609+0000 7f2e475b8740 10 client.15624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:02:59.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.609+0000 7f2e475b8740 20 client.15624 populate_metadata read hostname 'smithi084' 2024-05-22T23:02:59.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.609+0000 7f2e475b8740 1 --2- 172.21.15.84:0/1413010338 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b3382dc650 0x55b3382dea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.609+0000 7f2e475b8740 4 client.15624 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-22T23:02:59.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.609+0000 7f2e475b8740 1 -- 172.21.15.84:0/1413010338 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b3381cc170 con 0x55b3382dc650 2024-05-22T23:02:59.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.609+0000 7f2e3e59c640 1 --2- 172.21.15.84:0/1413010338 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b3382dc650 0x55b3382dea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.610+0000 7f2e25ffb640 10 client.15624 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:59.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.610+0000 7f2e3e59c640 1 --2- 172.21.15.84:0/1413010338 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b3382dc650 0x55b3382dea70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7200558e1fe2223a server_cookie=9fe52396f974fe34 in_seq=0 out_seq=0 2024-05-22T23:02:59.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.611+0000 7f2e25ffb640 1 -- 172.21.15.84:0/1413010338 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b3381cc170 con 0x55b3382dc650 2024-05-22T23:02:59.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.611+0000 7f2e25ffb640 10 client.15624 handle_command_reply: tid=0 2024-05-22T23:02:59.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.611+0000 7f2e475b8740 1 client.15624 shutdown 2024-05-22T23:02:59.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.611+0000 7f2e475b8740 1 -- 172.21.15.84:0/1413010338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b338265c40 msgr2=0x55b33825fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:59.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.611+0000 7f2e475b8740 1 --2- 172.21.15.84:0/1413010338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b338265c40 0x55b33825fe30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2e38035000 tx=0x7f2e3802a8e0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.611+0000 7f2e475b8740 1 -- 172.21.15.84:0/1413010338 shutdown_connections 2024-05-22T23:02:59.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.611+0000 7f2e475b8740 1 --2- 172.21.15.84:0/1413010338 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b3382dc650 0x55b3382dea70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.611+0000 7f2e475b8740 1 --2- 172.21.15.84:0/1413010338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b338265c40 0x55b33825fe30 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.611+0000 7f2e25ffb640 0 client.15624 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:59.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.611+0000 7f2e475b8740 1 --2- 172.21.15.84:0/1413010338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b33825b570 0x55b33825f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.611+0000 7f2e475b8740 1 --2- 172.21.15.84:0/1413010338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b33825ac50 0x55b33826e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.611+0000 7f2e475b8740 1 -- 172.21.15.84:0/1413010338 >> 172.21.15.84:0/1413010338 conn(0x55b338256f70 msgr2=0x55b338264410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:59.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.612+0000 7f2e475b8740 1 -- 172.21.15.84:0/1413010338 shutdown_connections 2024-05-22T23:02:59.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.612+0000 7f2e475b8740 1 -- 172.21.15.84:0/1413010338 wait complete. 2024-05-22T23:02:59.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.612+0000 7f2e475b8740 20 client.15624 trim_cache size 0 max 16384 2024-05-22T23:02:59.618 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:02:59.618 INFO:teuthology.orchestra.run.smithi084.stdout: "return_code": 0, 2024-05-22T23:02:59.618 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_tag": "a5d96970-4cb1-400f-a33c-13e3e40a584b", 2024-05-22T23:02:59.618 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": "asynchronous" 2024-05-22T23:02:59.618 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:02:59.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.612+0000 7f2e448a5640 1 -- 172.21.15.84:0/3858499911 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f2e280a2ea0 msgr2=0x7f2e280a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:59.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.612+0000 7f2e448a5640 1 --2- 172.21.15.84:0/3858499911 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f2e280a2ea0 0x7f2e280a5360 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2e38005e30 tx=0x7f2e38005d80 comp rx=0 tx=0).stop 2024-05-22T23:02:59.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.612+0000 7f2e448a5640 1 -- 172.21.15.84:0/3858499911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e40100740 msgr2=0x7f2e4010df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:59.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.612+0000 7f2e448a5640 1 --2- 172.21.15.84:0/3858499911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e40100740 0x7f2e4010df20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000e9f0 tx=0x7f2e3000edf0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.613+0000 7f2e448a5640 1 -- 172.21.15.84:0/3858499911 shutdown_connections 2024-05-22T23:02:59.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.613+0000 7f2e448a5640 1 --2- 172.21.15.84:0/3858499911 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f2e280a2ea0 0x7f2e280a5360 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.613+0000 7f2e448a5640 1 --2- 172.21.15.84:0/3858499911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e40106750 0x7f2e4010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.613+0000 7f2e448a5640 1 --2- 172.21.15.84:0/3858499911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e40101100 0x7f2e4010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.613+0000 7f2e448a5640 1 --2- 172.21.15.84:0/3858499911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e40100740 0x7f2e4010df20 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.613+0000 7f2e448a5640 1 -- 172.21.15.84:0/3858499911 >> 172.21.15.84:0/3858499911 conn(0x7f2e40077870 msgr2=0x7f2e40102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:59.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.613+0000 7f2e448a5640 1 -- 172.21.15.84:0/3858499911 shutdown_connections 2024-05-22T23:02:59.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.613+0000 7f2e448a5640 1 -- 172.21.15.84:0/3858499911 wait complete. 2024-05-22T23:02:59.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:02:59.626 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:02:59.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.730+0000 7f96c8d9e640 1 -- 172.21.15.84:0/787069460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96c4101100 msgr2=0x7f96c410d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:59.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.730+0000 7f96c8d9e640 1 --2- 172.21.15.84:0/787069460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96c4101100 0x7f96c410d610 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f96bc009be0 tx=0x7f96bc02a6c0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.730+0000 7f96c8d9e640 1 -- 172.21.15.84:0/787069460 shutdown_connections 2024-05-22T23:02:59.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.730+0000 7f96c8d9e640 1 --2- 172.21.15.84:0/787069460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96c4101100 0x7f96c410d610 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.730+0000 7f96c8d9e640 1 --2- 172.21.15.84:0/787069460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96c4100740 0x7f96c4100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.730+0000 7f96c8d9e640 1 --2- 172.21.15.84:0/787069460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96c4106750 0x7f96c4106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.730+0000 7f96c8d9e640 1 -- 172.21.15.84:0/787069460 >> 172.21.15.84:0/787069460 conn(0x7f96c4077870 msgr2=0x7f96c4077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:59.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.730+0000 7f96c8d9e640 1 -- 172.21.15.84:0/787069460 shutdown_connections 2024-05-22T23:02:59.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.730+0000 7f96c8d9e640 1 -- 172.21.15.84:0/787069460 wait complete. 2024-05-22T23:02:59.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.731+0000 7f96c8d9e640 1 Processor -- start 2024-05-22T23:02:59.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.731+0000 7f96c8d9e640 1 -- start start 2024-05-22T23:02:59.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.731+0000 7f96c8d9e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96c4100740 0x7f96c4102cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.731+0000 7f96c8d9e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96c4101100 0x7f96c4103200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.732+0000 7f96c8d9e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96c4106750 0x7f96c4103740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.732+0000 7f96c8d9e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f96c4110b10 con 0x7f96c4106750 2024-05-22T23:02:59.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.732+0000 7f96c8d9e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f96c4110990 con 0x7f96c4100740 2024-05-22T23:02:59.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.732+0000 7f96c8d9e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f96c4110c90 con 0x7f96c4101100 2024-05-22T23:02:59.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.732+0000 7f96c2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96c4106750 0x7f96c4103740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.732+0000 7f96c2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96c4106750 0x7f96c4103740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45088/0 (socket says 172.21.15.84:45088) 2024-05-22T23:02:59.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.732+0000 7f96c2d9d640 1 -- 172.21.15.84:0/3535520125 learned_addr learned my addr 172.21.15.84:0/3535520125 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:59.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.732+0000 7f96c259c640 1 --2- 172.21.15.84:0/3535520125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96c4100740 0x7f96c4102cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.732+0000 7f96c2d9d640 1 -- 172.21.15.84:0/3535520125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96c4101100 msgr2=0x7f96c4103200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:59.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.732+0000 7f96c2d9d640 1 --2- 172.21.15.84:0/3535520125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96c4101100 0x7f96c4103200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.732+0000 7f96c2d9d640 1 -- 172.21.15.84:0/3535520125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96c4100740 msgr2=0x7f96c4102cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:59.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.732+0000 7f96c2d9d640 1 --2- 172.21.15.84:0/3535520125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96c4100740 0x7f96c4102cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.732+0000 7f96c2d9d640 1 -- 172.21.15.84:0/3535520125 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c41aaa80 con 0x7f96c4106750 2024-05-22T23:02:59.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.732+0000 7f96c259c640 1 --2- 172.21.15.84:0/3535520125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96c4100740 0x7f96c4102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:02:59.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.732+0000 7f96c2d9d640 1 --2- 172.21.15.84:0/3535520125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96c4106750 0x7f96c4103740 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f96bc009d00 tx=0x7f96bc030cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:59.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.733+0000 7f96b37fe640 1 -- 172.21.15.84:0/3535520125 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc041020 con 0x7f96c4106750 2024-05-22T23:02:59.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.733+0000 7f96b37fe640 1 -- 172.21.15.84:0/3535520125 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96bc037040 con 0x7f96c4106750 2024-05-22T23:02:59.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.733+0000 7f96b37fe640 1 -- 172.21.15.84:0/3535520125 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc036070 con 0x7f96c4106750 2024-05-22T23:02:59.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.733+0000 7f96c8d9e640 1 -- 172.21.15.84:0/3535520125 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c41aac50 con 0x7f96c4106750 2024-05-22T23:02:59.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.733+0000 7f96c8d9e640 1 -- 172.21.15.84:0/3535520125 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c41ab140 con 0x7f96c4106750 2024-05-22T23:02:59.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.736+0000 7f96b37fe640 1 -- 172.21.15.84:0/3535520125 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f96bc044020 con 0x7f96c4106750 2024-05-22T23:02:59.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.736+0000 7f96cbb87740 1 Processor -- start 2024-05-22T23:02:59.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.736+0000 7f96cbb87740 1 -- start start 2024-05-22T23:02:59.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.736+0000 7f96cbb87740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de9760a4a0 0x55de976087e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.737+0000 7f96cbb87740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de97608db0 0x55de97609230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.737+0000 7f96cbb87740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de97613b30 0x55de97615f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.737+0000 7f96cbb87740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55de9756cdb0 con 0x55de97613b30 2024-05-22T23:02:59.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.737+0000 7f96cbb87740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55de9750b360 con 0x55de9760a4a0 2024-05-22T23:02:59.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.737+0000 7f96cbb87740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55de97568d70 con 0x55de97608db0 2024-05-22T23:02:59.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.737+0000 7f96c1d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de97613b30 0x55de97615f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.737+0000 7f96c259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de97608db0 0x55de97609230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.737+0000 7f96c259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de97608db0 0x55de97609230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:50268/0 (socket says 172.21.15.84:50268) 2024-05-22T23:02:59.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.737+0000 7f96c259c640 1 -- 172.21.15.84:0/1236265004 learned_addr learned my addr 172.21.15.84:0/1236265004 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:59.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.737+0000 7f96b37fe640 1 --2- 172.21.15.84:0/3535520125 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f96ac0a2ea0 0x7f96ac0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.737+0000 7f96c1d9b640 1 --2- 172.21.15.84:0/1236265004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de9760a4a0 0x55de976087e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.737+0000 7f96c1d9b640 1 -- 172.21.15.84:0/1236265004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de97608db0 msgr2=0x55de97609230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:59.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.737+0000 7f96c1d9b640 1 --2- 172.21.15.84:0/1236265004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de97608db0 0x55de97609230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.737+0000 7f96c1d9b640 1 -- 172.21.15.84:0/1236265004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de9760a4a0 msgr2=0x55de976087e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:59.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.737+0000 7f96c1d9b640 1 --2- 172.21.15.84:0/1236265004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de9760a4a0 0x55de976087e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.737+0000 7f96c1d9b640 1 -- 172.21.15.84:0/1236265004 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de9752ea60 con 0x55de97613b30 2024-05-22T23:02:59.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.738+0000 7f96b37fe640 1 -- 172.21.15.84:0/3535520125 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f96bc0efee0 con 0x7f96c4106750 2024-05-22T23:02:59.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.738+0000 7f96c1d9b640 1 --2- 172.21.15.84:0/1236265004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de97613b30 0x55de97615f20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f96c4104510 tx=0x7f96b80097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5a133eb3e443fa server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:59.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.738+0000 7f96c2d9d640 1 --2- 172.21.15.84:0/3535520125 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f96ac0a2ea0 0x7f96ac0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.738+0000 7f96b0ff9640 1 -- 172.21.15.84:0/1236265004 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b8009e90 con 0x55de97613b30 2024-05-22T23:02:59.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.738+0000 7f96b0ff9640 1 -- 172.21.15.84:0/1236265004 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96b8016070 con 0x55de97613b30 2024-05-22T23:02:59.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.738+0000 7f96b0ff9640 1 -- 172.21.15.84:0/1236265004 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b8010430 con 0x55de97613b30 2024-05-22T23:02:59.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.739+0000 7f96cbb87740 1 -- 172.21.15.84:0/1236265004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de97613b30 msgr2=0x55de97615f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:59.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.739+0000 7f96cbb87740 1 --2- 172.21.15.84:0/1236265004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de97613b30 0x55de97615f20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f96c4104510 tx=0x7f96b80097d0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.739+0000 7f96c2d9d640 1 --2- 172.21.15.84:0/3535520125 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f96ac0a2ea0 0x7f96ac0a5360 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f96bc0ed4f0 tx=0x7f96bc030940 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:59.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.739+0000 7f96cbb87740 1 -- 172.21.15.84:0/1236265004 shutdown_connections 2024-05-22T23:02:59.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.739+0000 7f96cbb87740 1 --2- 172.21.15.84:0/1236265004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de97613b30 0x55de97615f20 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.739+0000 7f96cbb87740 1 --2- 172.21.15.84:0/1236265004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de97608db0 0x55de97609230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.739+0000 7f96cbb87740 1 --2- 172.21.15.84:0/1236265004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de9760a4a0 0x55de976087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.739+0000 7f96cbb87740 1 -- 172.21.15.84:0/1236265004 >> 172.21.15.84:0/1236265004 conn(0x55de975d18e0 msgr2=0x55de97605310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:59.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.739+0000 7f96cbb87740 1 -- 172.21.15.84:0/1236265004 shutdown_connections 2024-05-22T23:02:59.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.739+0000 7f96cbb87740 1 -- 172.21.15.84:0/1236265004 wait complete. 2024-05-22T23:02:59.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.740+0000 7f96cbb87740 1 -- start start 2024-05-22T23:02:59.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.740+0000 7f96cbb87740 1 Processor -- start 2024-05-22T23:02:59.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.740+0000 7f96cbb87740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de97608db0 0x55de97610b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.740+0000 7f96cbb87740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de9760a4a0 0x55de97611040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.740+0000 7f96cbb87740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de97613b30 0x55de97611580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.740+0000 7f96cbb87740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55de976169c0 con 0x55de9760a4a0 2024-05-22T23:02:59.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.740+0000 7f96cbb87740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55de975270f0 con 0x55de97608db0 2024-05-22T23:02:59.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.740+0000 7f96cbb87740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55de97616b40 con 0x55de97613b30 2024-05-22T23:02:59.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.740+0000 7f96cbb87740 10 client.0 _ll_register_callbacks cb 0x55de9752bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:02:59.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.740+0000 7f96c259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de9760a4a0 0x55de97611040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.740+0000 7f96c259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de9760a4a0 0x55de97611040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45110/0 (socket says 172.21.15.84:45110) 2024-05-22T23:02:59.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.740+0000 7f96c259c640 1 -- 172.21.15.84:0/187667590 learned_addr learned my addr 172.21.15.84:0/187667590 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:59.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.740+0000 7f96c1d9b640 1 --2- 172.21.15.84:0/187667590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de97608db0 0x55de97610b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.740+0000 7f96c1d9b640 1 --2- 172.21.15.84:0/187667590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de97613b30 0x55de97611580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.741+0000 7f96c259c640 1 -- 172.21.15.84:0/187667590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de97613b30 msgr2=0x55de97611580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:59.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.741+0000 7f96c259c640 1 --2- 172.21.15.84:0/187667590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de97613b30 0x55de97611580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.741+0000 7f96c259c640 1 -- 172.21.15.84:0/187667590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de97608db0 msgr2=0x55de97610b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:59.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.741+0000 7f96c259c640 1 --2- 172.21.15.84:0/187667590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de97608db0 0x55de97610b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.741+0000 7f96c259c640 1 -- 172.21.15.84:0/187667590 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de9734a7d0 con 0x55de9760a4a0 2024-05-22T23:02:59.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.741+0000 7f96c1d9b640 1 --2- 172.21.15.84:0/187667590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de97608db0 0x55de97610b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:02:59.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.741+0000 7f96c259c640 1 --2- 172.21.15.84:0/187667590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de9760a4a0 0x55de97611040 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f96b4013780 tx=0x7f96b4013c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c63b9313c983b95 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:59.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.741+0000 7f96a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:02:59.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.741+0000 7f96a9ffb640 1 -- 172.21.15.84:0/187667590 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4020060 con 0x55de9760a4a0 2024-05-22T23:02:59.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.741+0000 7f96a9ffb640 1 -- 172.21.15.84:0/187667590 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f96b4011dd0 con 0x55de9760a4a0 2024-05-22T23:02:59.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.741+0000 7f96cbb87740 1 -- 172.21.15.84:0/187667590 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96b8016070 con 0x55de9760a4a0 2024-05-22T23:02:59.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.741+0000 7f96a9ffb640 1 -- 172.21.15.84:0/187667590 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f96b4024410 con 0x55de9760a4a0 2024-05-22T23:02:59.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.741+0000 7f96cbb87740 10 client.15636 fetch_fsmap learned FSMap version 18 2024-05-22T23:02:59.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.741+0000 7f96cbb87740 1 -- 172.21.15.84:0/187667590 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55de972803f0 con 0x55de9760a4a0 2024-05-22T23:02:59.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.742+0000 7f96a9ffb640 1 -- 172.21.15.84:0/187667590 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96b8016070 con 0x55de9760a4a0 2024-05-22T23:02:59.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.742+0000 7f96a9ffb640 1 -- 172.21.15.84:0/187667590 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f96b401a030 con 0x55de9760a4a0 2024-05-22T23:02:59.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.742+0000 7f96cbb87740 10 client.15636 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:02:59.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.742+0000 7f96cbb87740 10 client.15636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:02:59.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.742+0000 7f96cbb87740 20 client.15636 populate_metadata read hostname 'smithi084' 2024-05-22T23:02:59.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.742+0000 7f96cbb87740 1 --2- 172.21.15.84:0/187667590 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55de976240d0 0x55de976264c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.742+0000 7f96cbb87740 4 client.15636 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:02:59.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.742+0000 7f96cbb87740 1 -- 172.21.15.84:0/187667590 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de9757a4e0 con 0x55de976240d0 2024-05-22T23:02:59.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.742+0000 7f96c1d9b640 1 --2- 172.21.15.84:0/187667590 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55de976240d0 0x55de976264c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.742+0000 7f96a9ffb640 10 client.15636 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:59.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.742+0000 7f96c1d9b640 1 --2- 172.21.15.84:0/187667590 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55de976240d0 0x55de976264c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c6176f9340f8664 server_cookie=d969c8287a0404b5 in_seq=0 out_seq=0 2024-05-22T23:02:59.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.743+0000 7f96a9ffb640 1 -- 172.21.15.84:0/187667590 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55de9757a4e0 con 0x55de976240d0 2024-05-22T23:02:59.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.743+0000 7f96a9ffb640 10 client.15636 handle_command_reply: tid=0 2024-05-22T23:02:59.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.743+0000 7f96cbb87740 1 client.15636 shutdown 2024-05-22T23:02:59.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.743+0000 7f96cbb87740 1 -- 172.21.15.84:0/187667590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de9760a4a0 msgr2=0x55de97611040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:59.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.743+0000 7f96cbb87740 1 --2- 172.21.15.84:0/187667590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de9760a4a0 0x55de97611040 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f96b4013780 tx=0x7f96b4013c40 comp rx=0 tx=0).stop 2024-05-22T23:02:59.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.743+0000 7f96cbb87740 1 -- 172.21.15.84:0/187667590 shutdown_connections 2024-05-22T23:02:59.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.743+0000 7f96cbb87740 1 --2- 172.21.15.84:0/187667590 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55de976240d0 0x55de976264c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.743+0000 7f96a9ffb640 0 client.15636 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:59.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.743+0000 7f96cbb87740 1 --2- 172.21.15.84:0/187667590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de97613b30 0x55de97611580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.743+0000 7f96cbb87740 1 --2- 172.21.15.84:0/187667590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de9760a4a0 0x55de97611040 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.743+0000 7f96cbb87740 1 --2- 172.21.15.84:0/187667590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de97608db0 0x55de97610b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.743+0000 7f96cbb87740 1 -- 172.21.15.84:0/187667590 >> 172.21.15.84:0/187667590 conn(0x55de975d18e0 msgr2=0x55de9760b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:59.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.743+0000 7f96cbb87740 1 -- 172.21.15.84:0/187667590 shutdown_connections 2024-05-22T23:02:59.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.743+0000 7f96cbb87740 1 -- 172.21.15.84:0/187667590 wait complete. 2024-05-22T23:02:59.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.744+0000 7f96cbb87740 20 client.15636 trim_cache size 0 max 16384 2024-05-22T23:02:59.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.762+0000 7f96cbb87740 1 Processor -- start 2024-05-22T23:02:59.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.762+0000 7f96cbb87740 1 -- start start 2024-05-22T23:02:59.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.762+0000 7f96cbb87740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de97608db0 0x55de976054e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.762+0000 7f96cbb87740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de9760a4a0 0x55de976204a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.762+0000 7f96cbb87740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de97613b30 0x55de9761f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.762+0000 7f96cbb87740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f96c41b1bd0 con 0x55de97608db0 2024-05-22T23:02:59.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.762+0000 7f96cbb87740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f96c41b4450 con 0x55de97613b30 2024-05-22T23:02:59.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.762+0000 7f96cbb87740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f96c41b2240 con 0x55de9760a4a0 2024-05-22T23:02:59.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.762+0000 7f96c1d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de97608db0 0x55de976054e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.762+0000 7f96c1d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de97608db0 0x55de976054e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45122/0 (socket says 172.21.15.84:45122) 2024-05-22T23:02:59.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.762+0000 7f96c1d9b640 1 -- 172.21.15.84:0/2783793261 learned_addr learned my addr 172.21.15.84:0/2783793261 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:59.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.762+0000 7f96c259c640 1 --2- 172.21.15.84:0/2783793261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de9760a4a0 0x55de976204a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.762+0000 7f96c1d9b640 1 -- 172.21.15.84:0/2783793261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de9760a4a0 msgr2=0x55de976204a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:59.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.762+0000 7f96c1d9b640 1 --2- 172.21.15.84:0/2783793261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de9760a4a0 0x55de976204a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.762+0000 7f96c1d9b640 1 -- 172.21.15.84:0/2783793261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de97613b30 msgr2=0x55de9761f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:59.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.762+0000 7f96c1d9b640 1 --2- 172.21.15.84:0/2783793261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de97613b30 0x55de9761f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.762+0000 7f96c1d9b640 1 -- 172.21.15.84:0/2783793261 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de972400f0 con 0x55de97608db0 2024-05-22T23:02:59.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.763+0000 7f96c259c640 1 --2- 172.21.15.84:0/2783793261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de9760a4a0 0x55de976204a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:02:59.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.763+0000 7f96c1d9b640 1 --2- 172.21.15.84:0/2783793261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de97608db0 0x55de976054e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f96b80096c0 tx=0x7f96b8003bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f173f53c22059c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:59.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.763+0000 7f96abfff640 1 -- 172.21.15.84:0/2783793261 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b8009260 con 0x55de97608db0 2024-05-22T23:02:59.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.763+0000 7f96abfff640 1 -- 172.21.15.84:0/2783793261 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96b8016050 con 0x55de97608db0 2024-05-22T23:02:59.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.763+0000 7f96abfff640 1 -- 172.21.15.84:0/2783793261 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b8005020 con 0x55de97608db0 2024-05-22T23:02:59.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.764+0000 7f96cbb87740 1 -- 172.21.15.84:0/2783793261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de97608db0 msgr2=0x55de976054e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:59.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.764+0000 7f96cbb87740 1 --2- 172.21.15.84:0/2783793261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de97608db0 0x55de976054e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f96b80096c0 tx=0x7f96b8003bf0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.764+0000 7f96cbb87740 1 -- 172.21.15.84:0/2783793261 shutdown_connections 2024-05-22T23:02:59.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.764+0000 7f96cbb87740 1 --2- 172.21.15.84:0/2783793261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de97613b30 0x55de9761f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.764+0000 7f96cbb87740 1 --2- 172.21.15.84:0/2783793261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de9760a4a0 0x55de976204a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.764+0000 7f96cbb87740 1 --2- 172.21.15.84:0/2783793261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de97608db0 0x55de976054e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.764+0000 7f96cbb87740 1 -- 172.21.15.84:0/2783793261 >> 172.21.15.84:0/2783793261 conn(0x55de975d18e0 msgr2=0x55de97604dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:59.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.764+0000 7f96cbb87740 1 -- 172.21.15.84:0/2783793261 shutdown_connections 2024-05-22T23:02:59.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.764+0000 7f96cbb87740 1 -- 172.21.15.84:0/2783793261 wait complete. 2024-05-22T23:02:59.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.764+0000 7f96cbb87740 1 -- start start 2024-05-22T23:02:59.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.764+0000 7f96cbb87740 1 Processor -- start 2024-05-22T23:02:59.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.764+0000 7f96cbb87740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de97608db0 0x55de9761bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.765+0000 7f96cbb87740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de9760a4a0 0x55de9761c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.765+0000 7f96c1d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de97608db0 0x55de9761bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.765+0000 7f96cbb87740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de97613b30 0x55de97616f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.765+0000 7f96c1d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de97608db0 0x55de9761bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45132/0 (socket says 172.21.15.84:45132) 2024-05-22T23:02:59.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.765+0000 7f96c1d9b640 1 -- 172.21.15.84:0/791638267 learned_addr learned my addr 172.21.15.84:0/791638267 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:02:59.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.765+0000 7f96cbb87740 1 -- 172.21.15.84:0/791638267 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f96c41b2910 con 0x55de97608db0 2024-05-22T23:02:59.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.765+0000 7f96cbb87740 1 -- 172.21.15.84:0/791638267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f96c41b3d80 con 0x55de97613b30 2024-05-22T23:02:59.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.765+0000 7f96cbb87740 1 -- 172.21.15.84:0/791638267 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f96c41b2fe0 con 0x55de9760a4a0 2024-05-22T23:02:59.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.765+0000 7f96cbb87740 10 client.0 _ll_register_callbacks cb 0x7f96b4009820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:02:59.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.765+0000 7f96c259c640 1 --2- 172.21.15.84:0/791638267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de9760a4a0 0x55de9761c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.765+0000 7f96c1d9b640 1 --2- 172.21.15.84:0/791638267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de97613b30 0x55de97616f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.765+0000 7f96c259c640 1 -- 172.21.15.84:0/791638267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de97613b30 msgr2=0x55de97616f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:59.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.765+0000 7f96c259c640 1 --2- 172.21.15.84:0/791638267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de97613b30 0x55de97616f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.765+0000 7f96c259c640 1 -- 172.21.15.84:0/791638267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de97608db0 msgr2=0x55de9761bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:02:59.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.765+0000 7f96c259c640 1 --2- 172.21.15.84:0/791638267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de97608db0 0x55de9761bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.765+0000 7f96c259c640 1 -- 172.21.15.84:0/791638267 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de9752bfe0 con 0x55de9760a4a0 2024-05-22T23:02:59.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.765+0000 7f96c1d9b640 1 --2- 172.21.15.84:0/791638267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de97608db0 0x55de9761bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:02:59.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.765+0000 7f96c259c640 1 --2- 172.21.15.84:0/791638267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de9760a4a0 0x55de9761c020 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f96b4010720 tx=0x7f96b4011950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75744ea6f7dff1ff server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:02:59.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.765+0000 7f96a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-22T23:02:59.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.766+0000 7f96a9ffb640 1 -- 172.21.15.84:0/791638267 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4011dd0 con 0x55de9760a4a0 2024-05-22T23:02:59.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.766+0000 7f96a9ffb640 1 -- 172.21.15.84:0/791638267 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f96b4013020 con 0x55de9760a4a0 2024-05-22T23:02:59.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.766+0000 7f96cbb87740 1 -- 172.21.15.84:0/791638267 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96b8016050 con 0x55de9760a4a0 2024-05-22T23:02:59.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.766+0000 7f96a9ffb640 1 -- 172.21.15.84:0/791638267 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f96b4013240 con 0x55de9760a4a0 2024-05-22T23:02:59.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.766+0000 7f96cbb87740 10 client.25135 fetch_fsmap learned FSMap version 18 2024-05-22T23:02:59.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.766+0000 7f96cbb87740 1 -- 172.21.15.84:0/791638267 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55de974fd1c0 con 0x55de9760a4a0 2024-05-22T23:02:59.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.766+0000 7f96a9ffb640 1 -- 172.21.15.84:0/791638267 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96b8016050 con 0x55de9760a4a0 2024-05-22T23:02:59.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.766+0000 7f96a9ffb640 1 -- 172.21.15.84:0/791638267 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f96b402b070 con 0x55de9760a4a0 2024-05-22T23:02:59.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.766+0000 7f96cbb87740 10 client.25135 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:02:59.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.766+0000 7f96cbb87740 10 client.25135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:02:59.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.766+0000 7f96cbb87740 20 client.25135 populate_metadata read hostname 'smithi084' 2024-05-22T23:02:59.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.766+0000 7f96cbb87740 1 --2- 172.21.15.84:0/791638267 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55de976240d0 0x55de9768c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:02:59.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.767+0000 7f96cbb87740 4 client.25135 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:02:59.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.767+0000 7f96cbb87740 1 -- 172.21.15.84:0/791638267 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55de97574830 con 0x55de976240d0 2024-05-22T23:02:59.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.767+0000 7f96c1d9b640 1 --2- 172.21.15.84:0/791638267 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55de976240d0 0x55de9768c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:02:59.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.767+0000 7f96a9ffb640 10 client.25135 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:59.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.767+0000 7f96c1d9b640 1 --2- 172.21.15.84:0/791638267 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55de976240d0 0x55de9768c3e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72ad68a5e2aeefdc server_cookie=e9f711808f6392c5 in_seq=0 out_seq=0 2024-05-22T23:02:59.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.767+0000 7f96a9ffb640 1 -- 172.21.15.84:0/791638267 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55de97574830 con 0x55de976240d0 2024-05-22T23:02:59.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.767+0000 7f96a9ffb640 10 client.25135 handle_command_reply: tid=0 2024-05-22T23:02:59.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.767+0000 7f96cbb87740 1 client.25135 shutdown 2024-05-22T23:02:59.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.768+0000 7f96cbb87740 1 -- 172.21.15.84:0/791638267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de9760a4a0 msgr2=0x55de9761c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:59.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.768+0000 7f96cbb87740 1 --2- 172.21.15.84:0/791638267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de9760a4a0 0x55de9761c020 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f96b4010720 tx=0x7f96b4011950 comp rx=0 tx=0).stop 2024-05-22T23:02:59.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.768+0000 7f96cbb87740 1 -- 172.21.15.84:0/791638267 shutdown_connections 2024-05-22T23:02:59.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.768+0000 7f96cbb87740 1 --2- 172.21.15.84:0/791638267 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55de976240d0 0x55de9768c3e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.768+0000 7f96cbb87740 1 --2- 172.21.15.84:0/791638267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de97613b30 0x55de97616f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.768+0000 7f96cbb87740 1 --2- 172.21.15.84:0/791638267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de9760a4a0 0x55de9761c020 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.768+0000 7f96cbb87740 1 --2- 172.21.15.84:0/791638267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de97608db0 0x55de9761bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.768+0000 7f96a9ffb640 0 client.25135 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:02:59.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.768+0000 7f96cbb87740 1 -- 172.21.15.84:0/791638267 >> 172.21.15.84:0/791638267 conn(0x55de975d18e0 msgr2=0x55de97612330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:59.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.768+0000 7f96cbb87740 1 -- 172.21.15.84:0/791638267 shutdown_connections 2024-05-22T23:02:59.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.768+0000 7f96cbb87740 1 -- 172.21.15.84:0/791638267 wait complete. 2024-05-22T23:02:59.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.768+0000 7f96cbb87740 20 client.25135 trim_cache size 0 max 16384 2024-05-22T23:02:59.775 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:02:59.775 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-22T23:02:59.775 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": { 2024-05-22T23:02:59.775 INFO:teuthology.orchestra.run.smithi084.stdout: "a5d96970-4cb1-400f-a33c-13e3e40a584b": { 2024-05-22T23:02:59.775 INFO:teuthology.orchestra.run.smithi084.stdout: "path": "/", 2024-05-22T23:02:59.775 INFO:teuthology.orchestra.run.smithi084.stdout: "tag": "a5d96970-4cb1-400f-a33c-13e3e40a584b", 2024-05-22T23:02:59.775 INFO:teuthology.orchestra.run.smithi084.stdout: "options": "recursive,force" 2024-05-22T23:02:59.775 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:02:59.775 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:02:59.775 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:02:59.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.768+0000 7f96c8d9e640 1 -- 172.21.15.84:0/3535520125 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f96ac0a2ea0 msgr2=0x7f96ac0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:59.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.769+0000 7f96c8d9e640 1 --2- 172.21.15.84:0/3535520125 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f96ac0a2ea0 0x7f96ac0a5360 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f96bc0ed4f0 tx=0x7f96bc030940 comp rx=0 tx=0).stop 2024-05-22T23:02:59.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.769+0000 7f96c8d9e640 1 -- 172.21.15.84:0/3535520125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96c4106750 msgr2=0x7f96c4103740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:02:59.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.769+0000 7f96c8d9e640 1 --2- 172.21.15.84:0/3535520125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96c4106750 0x7f96c4103740 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f96bc009d00 tx=0x7f96bc030cb0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.769+0000 7f96c8d9e640 1 -- 172.21.15.84:0/3535520125 shutdown_connections 2024-05-22T23:02:59.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.769+0000 7f96c8d9e640 1 --2- 172.21.15.84:0/3535520125 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f96ac0a2ea0 0x7f96ac0a5360 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.769+0000 7f96c8d9e640 1 --2- 172.21.15.84:0/3535520125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96c4106750 0x7f96c4103740 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.769+0000 7f96c8d9e640 1 --2- 172.21.15.84:0/3535520125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96c4101100 0x7f96c4103200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.769+0000 7f96c8d9e640 1 --2- 172.21.15.84:0/3535520125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96c4100740 0x7f96c4102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:02:59.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.769+0000 7f96c8d9e640 1 -- 172.21.15.84:0/3535520125 >> 172.21.15.84:0/3535520125 conn(0x7f96c4077870 msgr2=0x7f96c4068d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:02:59.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.769+0000 7f96c8d9e640 1 -- 172.21.15.84:0/3535520125 shutdown_connections 2024-05-22T23:02:59.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:02:59.769+0000 7f96c8d9e640 1 -- 172.21.15.84:0/3535520125 wait complete. 2024-05-22T23:02:59.782 INFO:tasks.cephfs.filesystem:scrub status for tag:a5d96970-4cb1-400f-a33c-13e3e40a584b - {'path': '/', 'tag': 'a5d96970-4cb1-400f-a33c-13e3e40a584b', 'options': 'recursive,force'} 2024-05-22T23:03:00.013 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:00 smithi084 ceph-mon[32186]: scrub queued for path: / 2024-05-22T23:03:00.013 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:00 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:03:00.013 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:00 smithi084 ceph-mon[32186]: scrub summary: active paths [/] 2024-05-22T23:03:00.013 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:00 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:03:00.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:00 smithi096 ceph-mon[36392]: scrub queued for path: / 2024-05-22T23:03:00.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:00 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:03:00.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:00 smithi096 ceph-mon[36392]: scrub summary: active paths [/] 2024-05-22T23:03:00.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:00 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:03:00.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:00 smithi204 ceph-mon[36211]: scrub queued for path: / 2024-05-22T23:03:00.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:00 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:03:00.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:00 smithi204 ceph-mon[36211]: scrub summary: active paths [/] 2024-05-22T23:03:00.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:00 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:03:01.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:01 smithi096 ceph-mon[36392]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:03:01.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:01 smithi096 ceph-mon[36392]: scrub summary: idle 2024-05-22T23:03:01.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:01 smithi084 ceph-mon[32186]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:03:01.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:01 smithi084 ceph-mon[32186]: scrub summary: idle 2024-05-22T23:03:01.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:01 smithi204 ceph-mon[36211]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:03:01.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:01 smithi204 ceph-mon[36211]: scrub summary: idle 2024-05-22T23:03:03.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:03 smithi096 ceph-mon[36392]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-22T23:03:03.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:03 smithi084 ceph-mon[32186]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-22T23:03:03.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:03 smithi204 ceph-mon[36211]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-22T23:03:05.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:05 smithi096 ceph-mon[36392]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-22T23:03:05.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:05 smithi084 ceph-mon[32186]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-22T23:03:05.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:05 smithi204 ceph-mon[36211]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-22T23:03:07.302 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:07 smithi096 ceph-mon[36392]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-22T23:03:07.377 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:07 smithi084 ceph-mon[32186]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-22T23:03:07.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:07 smithi204 ceph-mon[36211]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-22T23:03:09.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:09 smithi096 ceph-mon[36392]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-22T23:03:09.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:09 smithi084 ceph-mon[32186]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-22T23:03:09.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:09 smithi204 ceph-mon[36211]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-22T23:03:11.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:11 smithi096 ceph-mon[36392]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-22T23:03:11.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:11 smithi084 ceph-mon[32186]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-22T23:03:11.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:11 smithi204 ceph-mon[36211]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-22T23:03:13.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:13 smithi096 ceph-mon[36392]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-22T23:03:13.322 INFO:tasks.workunit.client.0.smithi084.stderr:Note: switching to '935217f1cb4dc722d5303f5ad364c883e9133376'. 2024-05-22T23:03:13.322 INFO:tasks.workunit.client.0.smithi084.stderr: 2024-05-22T23:03:13.323 INFO:tasks.workunit.client.0.smithi084.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-22T23:03:13.323 INFO:tasks.workunit.client.0.smithi084.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-22T23:03:13.323 INFO:tasks.workunit.client.0.smithi084.stderr:state without impacting any branches by switching back to a branch. 2024-05-22T23:03:13.323 INFO:tasks.workunit.client.0.smithi084.stderr: 2024-05-22T23:03:13.323 INFO:tasks.workunit.client.0.smithi084.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-22T23:03:13.323 INFO:tasks.workunit.client.0.smithi084.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-22T23:03:13.323 INFO:tasks.workunit.client.0.smithi084.stderr: 2024-05-22T23:03:13.323 INFO:tasks.workunit.client.0.smithi084.stderr: git switch -c 2024-05-22T23:03:13.323 INFO:tasks.workunit.client.0.smithi084.stderr: 2024-05-22T23:03:13.323 INFO:tasks.workunit.client.0.smithi084.stderr:Or undo this operation with: 2024-05-22T23:03:13.323 INFO:tasks.workunit.client.0.smithi084.stderr: 2024-05-22T23:03:13.323 INFO:tasks.workunit.client.0.smithi084.stderr: git switch - 2024-05-22T23:03:13.323 INFO:tasks.workunit.client.0.smithi084.stderr: 2024-05-22T23:03:13.324 INFO:tasks.workunit.client.0.smithi084.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-22T23:03:13.324 INFO:tasks.workunit.client.0.smithi084.stderr: 2024-05-22T23:03:13.324 INFO:tasks.workunit.client.0.smithi084.stderr:HEAD is now at 935217f1cb4 Merge PR #57340 into squid 2024-05-22T23:03:13.327 DEBUG:teuthology.orchestra.run.smithi084:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-22T23:03:13.352 INFO:tasks.workunit.client.0.smithi084.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-22T23:03:13.354 INFO:tasks.workunit.client.0.smithi084.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-22T23:03:13.354 INFO:tasks.workunit.client.0.smithi084.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-22T23:03:13.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:13 smithi084 ceph-mon[32186]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-22T23:03:13.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:13 smithi204 ceph-mon[36211]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-22T23:03:14.008 INFO:tasks.workunit.client.0.smithi084.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-22T23:03:14.067 INFO:tasks.workunit.client.0.smithi084.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-22T23:03:14.107 INFO:tasks.workunit.client.0.smithi084.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-22T23:03:14.109 INFO:tasks.workunit.client.0.smithi084.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-22T23:03:14.109 INFO:tasks.workunit.client.0.smithi084.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-22T23:03:14.148 INFO:tasks.workunit.client.0.smithi084.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-22T23:03:14.152 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-22T23:03:14.152 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-22T23:03:14.208 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-05-22T23:03:14.209 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-05-22T23:03:14.209 DEBUG:teuthology.orchestra.run.smithi084:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=935217f1cb4dc722d5303f5ad364c883e9133376 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-05-22T23:03:14.269 INFO:tasks.workunit.client.0.smithi084.stderr:+ dbench 1 2024-05-22T23:03:14.313 INFO:tasks.workunit.client.0.smithi084.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-05-22T23:03:14.314 INFO:tasks.workunit.client.0.smithi084.stdout: 2024-05-22T23:03:14.314 INFO:tasks.workunit.client.0.smithi084.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-05-22T23:03:14.322 INFO:tasks.workunit.client.0.smithi084.stdout:0 of 1 processes prepared for launch 0 sec 2024-05-22T23:03:14.422 INFO:tasks.workunit.client.0.smithi084.stdout:1 of 1 processes prepared for launch 0 sec 2024-05-22T23:03:14.422 INFO:tasks.workunit.client.0.smithi084.stdout:releasing clients 2024-05-22T23:03:15.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:15 smithi096 ceph-mon[36392]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 170 B/s wr, 1 op/s 2024-05-22T23:03:15.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:15 smithi084 ceph-mon[32186]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 170 B/s wr, 1 op/s 2024-05-22T23:03:15.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:15 smithi204 ceph-mon[36211]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 170 B/s wr, 1 op/s 2024-05-22T23:03:15.422 INFO:tasks.workunit.client.0.smithi084.stdout: 1 3773 39.76 MB/sec warmup 1 sec latency 6.608 ms 2024-05-22T23:03:16.422 INFO:tasks.workunit.client.0.smithi084.stdout: 1 6934 27.29 MB/sec warmup 2 sec latency 19.308 ms 2024-05-22T23:03:17.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:17 smithi096 ceph-mon[36392]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 170 B/s wr, 1 op/s 2024-05-22T23:03:17.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:17 smithi084 ceph-mon[32186]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 170 B/s wr, 1 op/s 2024-05-22T23:03:17.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:17 smithi204 ceph-mon[36211]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 170 B/s wr, 1 op/s 2024-05-22T23:03:17.422 INFO:tasks.workunit.client.0.smithi084.stdout: 1 10111 23.92 MB/sec warmup 3 sec latency 14.657 ms 2024-05-22T23:03:18.422 INFO:tasks.workunit.client.0.smithi084.stdout: 1 13143 21.52 MB/sec warmup 4 sec latency 16.769 ms 2024-05-22T23:03:19.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:19 smithi096 ceph-mon[36392]: pgmap v47: 97 pgs: 97 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 273 KiB/s wr, 32 op/s 2024-05-22T23:03:19.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:19 smithi084 ceph-mon[32186]: pgmap v47: 97 pgs: 97 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 273 KiB/s wr, 32 op/s 2024-05-22T23:03:19.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:19 smithi204 ceph-mon[36211]: pgmap v47: 97 pgs: 97 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 273 KiB/s wr, 32 op/s 2024-05-22T23:03:19.422 INFO:tasks.workunit.client.0.smithi084.stdout: 1 16379 20.65 MB/sec warmup 5 sec latency 20.549 ms 2024-05-22T23:03:20.422 INFO:tasks.workunit.client.0.smithi084.stdout: 1 19509 20.31 MB/sec warmup 6 sec latency 13.680 ms 2024-05-22T23:03:21.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:21 smithi096 ceph-mon[36392]: pgmap v48: 97 pgs: 97 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 273 KiB/s wr, 31 op/s 2024-05-22T23:03:21.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:21 smithi084 ceph-mon[32186]: pgmap v48: 97 pgs: 97 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 273 KiB/s wr, 31 op/s 2024-05-22T23:03:21.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:21 smithi204 ceph-mon[36211]: pgmap v48: 97 pgs: 97 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 273 KiB/s wr, 31 op/s 2024-05-22T23:03:21.422 INFO:tasks.workunit.client.0.smithi084.stdout: 1 22933 20.13 MB/sec warmup 7 sec latency 13.467 ms 2024-05-22T23:03:22.423 INFO:tasks.workunit.client.0.smithi084.stdout: 1 25967 19.60 MB/sec warmup 8 sec latency 21.355 ms 2024-05-22T23:03:23.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:23 smithi096 ceph-mon[36392]: pgmap v49: 97 pgs: 97 active+clean; 12 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-05-22T23:03:23.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:23 smithi096 ceph-mon[36392]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-05-22T23:03:23.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:23 smithi084 ceph-mon[32186]: pgmap v49: 97 pgs: 97 active+clean; 12 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-05-22T23:03:23.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:23 smithi084 ceph-mon[32186]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-05-22T23:03:23.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:23 smithi204 ceph-mon[36211]: pgmap v49: 97 pgs: 97 active+clean; 12 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-05-22T23:03:23.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:23 smithi204 ceph-mon[36211]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-05-22T23:03:23.423 INFO:tasks.workunit.client.0.smithi084.stdout: 1 29300 19.41 MB/sec warmup 9 sec latency 16.165 ms 2024-05-22T23:03:24.423 INFO:tasks.workunit.client.0.smithi084.stdout: 1 32405 19.31 MB/sec warmup 10 sec latency 49.543 ms 2024-05-22T23:03:25.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:25 smithi096 ceph-mon[36392]: pgmap v50: 97 pgs: 97 active+clean; 12 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-05-22T23:03:25.377 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:25 smithi084 ceph-mon[32186]: pgmap v50: 97 pgs: 97 active+clean; 12 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-05-22T23:03:25.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:25 smithi204 ceph-mon[36211]: pgmap v50: 97 pgs: 97 active+clean; 12 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-05-22T23:03:25.423 INFO:tasks.workunit.client.0.smithi084.stdout: 1 35893 19.29 MB/sec warmup 11 sec latency 14.569 ms 2024-05-22T23:03:26.423 INFO:tasks.workunit.client.0.smithi084.stdout: 1 39263 19.25 MB/sec warmup 12 sec latency 24.916 ms 2024-05-22T23:03:27.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:27 smithi096 ceph-mon[36392]: pgmap v51: 97 pgs: 97 active+clean; 12 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-05-22T23:03:27.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:27 smithi084 ceph-mon[32186]: pgmap v51: 97 pgs: 97 active+clean; 12 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-05-22T23:03:27.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:27 smithi204 ceph-mon[36211]: pgmap v51: 97 pgs: 97 active+clean; 12 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-05-22T23:03:27.423 INFO:tasks.workunit.client.0.smithi084.stdout: 1 42392 18.90 MB/sec warmup 13 sec latency 15.249 ms 2024-05-22T23:03:28.423 INFO:tasks.workunit.client.0.smithi084.stdout: 1 45205 18.46 MB/sec warmup 14 sec latency 14.757 ms 2024-05-22T23:03:29.378 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:29 smithi084 ceph-mon[32186]: pgmap v52: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 1.8 MiB/s wr, 218 op/s 2024-05-22T23:03:29.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:29 smithi204 ceph-mon[36211]: pgmap v52: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 1.8 MiB/s wr, 218 op/s 2024-05-22T23:03:29.423 INFO:tasks.workunit.client.0.smithi084.stdout: 1 48524 18.44 MB/sec warmup 15 sec latency 27.636 ms 2024-05-22T23:03:29.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:29 smithi096 ceph-mon[36392]: pgmap v52: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 1.8 MiB/s wr, 218 op/s 2024-05-22T23:03:29.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:03:29.783 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:03:29.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.890+0000 7f98fc302640 1 -- 172.21.15.84:0/3748650597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98f4101100 msgr2=0x7f98f410d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:29.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.890+0000 7f98fc302640 1 --2- 172.21.15.84:0/3748650597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98f4101100 0x7f98f410d610 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009cd0 tx=0x7f98ec02a8e0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.890+0000 7f98fc302640 1 -- 172.21.15.84:0/3748650597 shutdown_connections 2024-05-22T23:03:29.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.890+0000 7f98fc302640 1 --2- 172.21.15.84:0/3748650597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98f4101100 0x7f98f410d610 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.890+0000 7f98fc302640 1 --2- 172.21.15.84:0/3748650597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98f4100740 0x7f98f4100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.890+0000 7f98fc302640 1 --2- 172.21.15.84:0/3748650597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98f4106750 0x7f98f4106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.890+0000 7f98fc302640 1 -- 172.21.15.84:0/3748650597 >> 172.21.15.84:0/3748650597 conn(0x7f98f4077870 msgr2=0x7f98f4077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:29.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.890+0000 7f98fc302640 1 -- 172.21.15.84:0/3748650597 shutdown_connections 2024-05-22T23:03:29.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.890+0000 7f98fc302640 1 -- 172.21.15.84:0/3748650597 wait complete. 2024-05-22T23:03:29.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.890+0000 7f98fc302640 1 Processor -- start 2024-05-22T23:03:29.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.890+0000 7f98fc302640 1 -- start start 2024-05-22T23:03:29.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.891+0000 7f98fc302640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98f4100740 0x7f98f4198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:29.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.891+0000 7f98fc302640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98f4101100 0x7f98f4198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:29.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.891+0000 7f98fc302640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98f4106750 0x7f98f419ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:29.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.891+0000 7f98fc302640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f98f4110b60 con 0x7f98f4100740 2024-05-22T23:03:29.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.891+0000 7f98fc302640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f98f41109e0 con 0x7f98f4106750 2024-05-22T23:03:29.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.891+0000 7f98fc302640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f98f4110ce0 con 0x7f98f4101100 2024-05-22T23:03:29.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.891+0000 7f98fa09e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98f4100740 0x7f98f4198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:29.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.891+0000 7f98fa09e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98f4100740 0x7f98f4198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49384/0 (socket says 172.21.15.84:49384) 2024-05-22T23:03:29.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.891+0000 7f98fa09e640 1 -- 172.21.15.84:0/979467151 learned_addr learned my addr 172.21.15.84:0/979467151 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:03:29.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.891+0000 7f98fa89f640 1 --2- 172.21.15.84:0/979467151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98f4106750 0x7f98f419ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:29.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.891+0000 7f98f989d640 1 --2- 172.21.15.84:0/979467151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98f4101100 0x7f98f4198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:29.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.891+0000 7f98fa89f640 1 -- 172.21.15.84:0/979467151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98f4101100 msgr2=0x7f98f4198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:29.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.891+0000 7f98fa89f640 1 --2- 172.21.15.84:0/979467151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98f4101100 0x7f98f4198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.891+0000 7f98fa89f640 1 -- 172.21.15.84:0/979467151 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98f4100740 msgr2=0x7f98f4198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:29.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.891+0000 7f98fa89f640 1 --2- 172.21.15.84:0/979467151 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98f4100740 0x7f98f4198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.891+0000 7f98fa89f640 1 -- 172.21.15.84:0/979467151 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f419d140 con 0x7f98f4106750 2024-05-22T23:03:29.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.891+0000 7f98fa09e640 1 --2- 172.21.15.84:0/979467151 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98f4100740 0x7f98f4198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:03:29.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.891+0000 7f98fa89f640 1 --2- 172.21.15.84:0/979467151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98f4106750 0x7f98f419ca10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f98ec02ae00 tx=0x7f98ec031e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:29.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.892+0000 7f98eb7fe640 1 -- 172.21.15.84:0/979467151 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec004040 con 0x7f98f4106750 2024-05-22T23:03:29.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.892+0000 7f98eb7fe640 1 -- 172.21.15.84:0/979467151 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98ec0046c0 con 0x7f98f4106750 2024-05-22T23:03:29.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.892+0000 7f98eb7fe640 1 -- 172.21.15.84:0/979467151 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec030e40 con 0x7f98f4106750 2024-05-22T23:03:29.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.892+0000 7f98fc302640 1 -- 172.21.15.84:0/979467151 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98f419d3d0 con 0x7f98f4106750 2024-05-22T23:03:29.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.892+0000 7f98fc302640 1 -- 172.21.15.84:0/979467151 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98f41a4cc0 con 0x7f98f4106750 2024-05-22T23:03:29.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.894+0000 7f98eb7fe640 1 -- 172.21.15.84:0/979467151 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f98ec0305a0 con 0x7f98f4106750 2024-05-22T23:03:29.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.894+0000 7f98ff005740 1 Processor -- start 2024-05-22T23:03:29.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.895+0000 7f98ff005740 1 -- start start 2024-05-22T23:03:29.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.895+0000 7f98ff005740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7f8f674a0 0x55c7f8f657e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:29.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.895+0000 7f98ff005740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f65db0 0x55c7f8f66230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:29.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.895+0000 7f98ff005740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c7f8f74f40 0x55c7f8f77330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:29.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.895+0000 7f98ff005740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c7f8ec9db0 con 0x55c7f8f74f40 2024-05-22T23:03:29.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.895+0000 7f98ff005740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c7f8e68360 con 0x55c7f8f65db0 2024-05-22T23:03:29.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.895+0000 7f98ff005740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c7f8ec5d70 con 0x55c7f8f674a0 2024-05-22T23:03:29.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.895+0000 7f98eb7fe640 1 --2- 172.21.15.84:0/979467151 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f98e00a2d80 0x7f98e00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:29.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.895+0000 7f98fa09e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f65db0 0x55c7f8f66230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:29.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.895+0000 7f98fa09e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f65db0 0x55c7f8f66230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:45998/0 (socket says 172.21.15.84:45998) 2024-05-22T23:03:29.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.895+0000 7f98fa09e640 1 -- 172.21.15.84:0/4229570526 learned_addr learned my addr 172.21.15.84:0/4229570526 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:03:29.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.895+0000 7f98f989d640 1 --2- 172.21.15.84:0/4229570526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c7f8f74f40 0x55c7f8f77330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:29.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.895+0000 7f98fa89f640 1 --2- 172.21.15.84:0/979467151 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f98e00a2d80 0x7f98e00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:29.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.895+0000 7f98eb7fe640 1 -- 172.21.15.84:0/979467151 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f98ec02e030 con 0x7f98f4106750 2024-05-22T23:03:29.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.895+0000 7f98f989d640 1 --2- 172.21.15.84:0/4229570526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7f8f674a0 0x55c7f8f657e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:29.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.895+0000 7f98fa09e640 1 -- 172.21.15.84:0/4229570526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7f8f674a0 msgr2=0x55c7f8f657e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:29.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.895+0000 7f98fa09e640 1 --2- 172.21.15.84:0/4229570526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7f8f674a0 0x55c7f8f657e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.895+0000 7f98fa09e640 1 -- 172.21.15.84:0/4229570526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c7f8f74f40 msgr2=0x55c7f8f77330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:29.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.895+0000 7f98fa09e640 1 --2- 172.21.15.84:0/4229570526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c7f8f74f40 0x55c7f8f77330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.895+0000 7f98fa09e640 1 -- 172.21.15.84:0/4229570526 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7f8e8ba60 con 0x55c7f8f65db0 2024-05-22T23:03:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.895+0000 7f98f989d640 1 --2- 172.21.15.84:0/4229570526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7f8f674a0 0x55c7f8f657e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:03:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.896+0000 7f98fa89f640 1 --2- 172.21.15.84:0/979467151 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f98e00a2d80 0x7f98e00a5240 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0b98f0 tx=0x7f98ec08eee0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.896+0000 7f98fa09e640 1 --2- 172.21.15.84:0/4229570526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f65db0 0x55c7f8f66230 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f98f00115c0 tx=0x7f98f0011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d4bcd91821ca953 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.896+0000 7f98e8bfe640 1 -- 172.21.15.84:0/4229570526 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f000f660 con 0x55c7f8f65db0 2024-05-22T23:03:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.896+0000 7f98e8bfe640 1 -- 172.21.15.84:0/4229570526 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98f000fce0 con 0x55c7f8f65db0 2024-05-22T23:03:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.896+0000 7f98e8bfe640 1 -- 172.21.15.84:0/4229570526 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f0019db0 con 0x55c7f8f65db0 2024-05-22T23:03:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.896+0000 7f98ff005740 1 -- 172.21.15.84:0/4229570526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f65db0 msgr2=0x55c7f8f66230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.896+0000 7f98ff005740 1 --2- 172.21.15.84:0/4229570526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f65db0 0x55c7f8f66230 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f98f00115c0 tx=0x7f98f0011a80 comp rx=0 tx=0).stop 2024-05-22T23:03:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.897+0000 7f98ff005740 1 -- 172.21.15.84:0/4229570526 shutdown_connections 2024-05-22T23:03:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.897+0000 7f98ff005740 1 --2- 172.21.15.84:0/4229570526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c7f8f74f40 0x55c7f8f77330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.897+0000 7f98ff005740 1 --2- 172.21.15.84:0/4229570526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f65db0 0x55c7f8f66230 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.897+0000 7f98ff005740 1 --2- 172.21.15.84:0/4229570526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7f8f674a0 0x55c7f8f657e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.897+0000 7f98ff005740 1 -- 172.21.15.84:0/4229570526 >> 172.21.15.84:0/4229570526 conn(0x55c7f8f2e8e0 msgr2=0x55c7f8f62310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.897+0000 7f98ff005740 1 -- 172.21.15.84:0/4229570526 shutdown_connections 2024-05-22T23:03:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.897+0000 7f98ff005740 1 -- 172.21.15.84:0/4229570526 wait complete. 2024-05-22T23:03:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.897+0000 7f98ff005740 1 -- start start 2024-05-22T23:03:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.897+0000 7f98ff005740 1 Processor -- start 2024-05-22T23:03:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.897+0000 7f98ff005740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7f8f65db0 0x55c7f8f71e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.897+0000 7f98ff005740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c7f8f674a0 0x55c7f8f723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.898+0000 7f98ff005740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f74f40 0x55c7f8f72900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.898+0000 7f98ff005740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c7f8f77dd0 con 0x55c7f8f674a0 2024-05-22T23:03:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.898+0000 7f98ff005740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c7f8e840f0 con 0x55c7f8f74f40 2024-05-22T23:03:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.898+0000 7f98ff005740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c7f8f77f50 con 0x55c7f8f65db0 2024-05-22T23:03:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.898+0000 7f98ff005740 10 client.0 _ll_register_callbacks cb 0x55c7f8e88f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:03:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.898+0000 7f98f989d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7f8f65db0 0x55c7f8f71e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:29.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.898+0000 7f98f989d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7f8f65db0 0x55c7f8f71e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:47042/0 (socket says 172.21.15.84:47042) 2024-05-22T23:03:29.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.898+0000 7f98f989d640 1 -- 172.21.15.84:0/4246825647 learned_addr learned my addr 172.21.15.84:0/4246825647 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:03:29.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.898+0000 7f98f989d640 1 --2- 172.21.15.84:0/4246825647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f74f40 0x55c7f8f72900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:29.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.898+0000 7f98fa09e640 1 --2- 172.21.15.84:0/4246825647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c7f8f674a0 0x55c7f8f723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:29.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.898+0000 7f98f989d640 1 -- 172.21.15.84:0/4246825647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7f8f65db0 msgr2=0x55c7f8f71e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:29.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.898+0000 7f98f989d640 1 --2- 172.21.15.84:0/4246825647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7f8f65db0 0x55c7f8f71e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.898+0000 7f98f989d640 1 -- 172.21.15.84:0/4246825647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c7f8f674a0 msgr2=0x55c7f8f723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:29.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.898+0000 7f98f989d640 1 --2- 172.21.15.84:0/4246825647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c7f8f674a0 0x55c7f8f723c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.898+0000 7f98f989d640 1 -- 172.21.15.84:0/4246825647 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7f8ca77d0 con 0x55c7f8f74f40 2024-05-22T23:03:29.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.898+0000 7f98f989d640 1 --2- 172.21.15.84:0/4246825647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f74f40 0x55c7f8f72900 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f98e4012d90 tx=0x7f98e4005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c11dc7a20fe847c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:29.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.898+0000 7f98ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:03:29.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.898+0000 7f98ddffb640 1 -- 172.21.15.84:0/4246825647 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e401b040 con 0x55c7f8f74f40 2024-05-22T23:03:29.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.898+0000 7f98ddffb640 1 -- 172.21.15.84:0/4246825647 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f98e4005020 con 0x55c7f8f74f40 2024-05-22T23:03:29.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.898+0000 7f98ff005740 1 -- 172.21.15.84:0/4246825647 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98f000fce0 con 0x55c7f8f74f40 2024-05-22T23:03:29.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.899+0000 7f98ddffb640 1 -- 172.21.15.84:0/4246825647 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f98e4012620 con 0x55c7f8f74f40 2024-05-22T23:03:29.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.899+0000 7f98ff005740 10 client.25235 fetch_fsmap learned FSMap version 18 2024-05-22T23:03:29.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.899+0000 7f98ff005740 1 -- 172.21.15.84:0/4246825647 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55c7f8bdd3f0 con 0x55c7f8f74f40 2024-05-22T23:03:29.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.899+0000 7f98ddffb640 1 -- 172.21.15.84:0/4246825647 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98f000fce0 con 0x55c7f8f74f40 2024-05-22T23:03:29.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.899+0000 7f98ddffb640 1 -- 172.21.15.84:0/4246825647 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f98e40047a0 con 0x55c7f8f74f40 2024-05-22T23:03:29.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.899+0000 7f98ff005740 10 client.25235 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:03:29.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.899+0000 7f98ff005740 10 client.25235 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:03:29.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.899+0000 7f98ff005740 20 client.25235 populate_metadata read hostname 'smithi084' 2024-05-22T23:03:29.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.899+0000 7f98ff005740 1 --2- 172.21.15.84:0/4246825647 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c7f8f853d0 0x55c7f8f87800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:29.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.899+0000 7f98ff005740 4 client.25235 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:03:29.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.899+0000 7f98ff005740 1 -- 172.21.15.84:0/4246825647 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7f8ed74e0 con 0x55c7f8f853d0 2024-05-22T23:03:29.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.899+0000 7f98fa09e640 1 --2- 172.21.15.84:0/4246825647 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c7f8f853d0 0x55c7f8f87800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:29.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.900+0000 7f98ddffb640 10 client.25235 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:03:29.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.900+0000 7f98fa09e640 1 --2- 172.21.15.84:0/4246825647 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c7f8f853d0 0x55c7f8f87800 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2de34e5c94ceae9b server_cookie=89c6eae31db4677c in_seq=0 out_seq=0 2024-05-22T23:03:29.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.902+0000 7f98ddffb640 1 -- 172.21.15.84:0/4246825647 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c7f8ed74e0 con 0x55c7f8f853d0 2024-05-22T23:03:29.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.902+0000 7f98ddffb640 10 client.25235 handle_command_reply: tid=0 2024-05-22T23:03:29.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.902+0000 7f98ff005740 1 client.25235 shutdown 2024-05-22T23:03:29.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.902+0000 7f98ff005740 1 -- 172.21.15.84:0/4246825647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f74f40 msgr2=0x55c7f8f72900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:29.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.902+0000 7f98ff005740 1 --2- 172.21.15.84:0/4246825647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f74f40 0x55c7f8f72900 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f98e4012d90 tx=0x7f98e4005c90 comp rx=0 tx=0).stop 2024-05-22T23:03:29.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.902+0000 7f98ff005740 1 -- 172.21.15.84:0/4246825647 shutdown_connections 2024-05-22T23:03:29.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.902+0000 7f98ff005740 1 --2- 172.21.15.84:0/4246825647 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c7f8f853d0 0x55c7f8f87800 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.902+0000 7f98ff005740 1 --2- 172.21.15.84:0/4246825647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f74f40 0x55c7f8f72900 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.902+0000 7f98ff005740 1 --2- 172.21.15.84:0/4246825647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c7f8f674a0 0x55c7f8f723c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.902+0000 7f98ff005740 1 --2- 172.21.15.84:0/4246825647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7f8f65db0 0x55c7f8f71e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.902+0000 7f98ddffb640 0 client.25235 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:03:29.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.902+0000 7f98ff005740 1 -- 172.21.15.84:0/4246825647 >> 172.21.15.84:0/4246825647 conn(0x55c7f8f2e8e0 msgr2=0x55c7f8f68180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:29.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.902+0000 7f98ff005740 1 -- 172.21.15.84:0/4246825647 shutdown_connections 2024-05-22T23:03:29.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.902+0000 7f98ff005740 1 -- 172.21.15.84:0/4246825647 wait complete. 2024-05-22T23:03:29.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.902+0000 7f98ff005740 20 client.25235 trim_cache size 0 max 16384 2024-05-22T23:03:29.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.920+0000 7f98ff005740 1 Processor -- start 2024-05-22T23:03:29.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.920+0000 7f98ff005740 1 -- start start 2024-05-22T23:03:29.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.920+0000 7f98ff005740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f65db0 0x55c7f8f80d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:29.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.920+0000 7f98ff005740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c7f8f674a0 0x55c7f8f73dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:29.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.920+0000 7f98ff005740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7f8f74f40 0x55c7f8f8d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:29.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.920+0000 7f98ff005740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f98f41a9410 con 0x55c7f8f674a0 2024-05-22T23:03:29.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.920+0000 7f98ff005740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f98f41abc90 con 0x55c7f8f65db0 2024-05-22T23:03:29.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.920+0000 7f98ff005740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f98f41a9a80 con 0x55c7f8f74f40 2024-05-22T23:03:29.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.920+0000 7f98fa09e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c7f8f674a0 0x55c7f8f73dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:29.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.920+0000 7f98fa09e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c7f8f674a0 0x55c7f8f73dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49424/0 (socket says 172.21.15.84:49424) 2024-05-22T23:03:29.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.920+0000 7f98f989d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f65db0 0x55c7f8f80d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:29.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.920+0000 7f98fa09e640 1 -- 172.21.15.84:0/990420019 learned_addr learned my addr 172.21.15.84:0/990420019 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:03:29.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.921+0000 7f98f989d640 1 --2- 172.21.15.84:0/990420019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7f8f74f40 0x55c7f8f8d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:29.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.921+0000 7f98fa09e640 1 -- 172.21.15.84:0/990420019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7f8f74f40 msgr2=0x55c7f8f8d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:29.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.921+0000 7f98fa09e640 1 --2- 172.21.15.84:0/990420019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7f8f74f40 0x55c7f8f8d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.921+0000 7f98fa09e640 1 -- 172.21.15.84:0/990420019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f65db0 msgr2=0x55c7f8f80d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:29.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.921+0000 7f98fa09e640 1 --2- 172.21.15.84:0/990420019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f65db0 0x55c7f8f80d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.921+0000 7f98fa09e640 1 -- 172.21.15.84:0/990420019 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7f8b9d0f0 con 0x55c7f8f674a0 2024-05-22T23:03:29.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.921+0000 7f98f989d640 1 --2- 172.21.15.84:0/990420019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f65db0 0x55c7f8f80d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:03:29.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.921+0000 7f98fa09e640 1 --2- 172.21.15.84:0/990420019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c7f8f674a0 0x55c7f8f73dc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f98f0009ee0 tx=0x7f98f0009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27135f2a354cab85 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:29.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.921+0000 7f98dffff640 1 -- 172.21.15.84:0/990420019 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f0019750 con 0x55c7f8f674a0 2024-05-22T23:03:29.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.921+0000 7f98dffff640 1 -- 172.21.15.84:0/990420019 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98f0005040 con 0x55c7f8f674a0 2024-05-22T23:03:29.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.921+0000 7f98dffff640 1 -- 172.21.15.84:0/990420019 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f000f8c0 con 0x55c7f8f674a0 2024-05-22T23:03:29.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.922+0000 7f98ff005740 1 -- 172.21.15.84:0/990420019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c7f8f674a0 msgr2=0x55c7f8f73dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:29.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.922+0000 7f98ff005740 1 --2- 172.21.15.84:0/990420019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c7f8f674a0 0x55c7f8f73dc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f98f0009ee0 tx=0x7f98f0009530 comp rx=0 tx=0).stop 2024-05-22T23:03:29.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.922+0000 7f98ff005740 1 -- 172.21.15.84:0/990420019 shutdown_connections 2024-05-22T23:03:29.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.922+0000 7f98ff005740 1 --2- 172.21.15.84:0/990420019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7f8f74f40 0x55c7f8f8d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.922+0000 7f98ff005740 1 --2- 172.21.15.84:0/990420019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c7f8f674a0 0x55c7f8f73dc0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.922+0000 7f98ff005740 1 --2- 172.21.15.84:0/990420019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f65db0 0x55c7f8f80d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.922+0000 7f98ff005740 1 -- 172.21.15.84:0/990420019 >> 172.21.15.84:0/990420019 conn(0x55c7f8f2e8e0 msgr2=0x55c7f8f736b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:29.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.922+0000 7f98ff005740 1 -- 172.21.15.84:0/990420019 shutdown_connections 2024-05-22T23:03:29.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.922+0000 7f98ff005740 1 -- 172.21.15.84:0/990420019 wait complete. 2024-05-22T23:03:29.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.922+0000 7f98ff005740 1 -- start start 2024-05-22T23:03:29.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.922+0000 7f98ff005740 1 Processor -- start 2024-05-22T23:03:29.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.923+0000 7f98ff005740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7f8f65db0 0x55c7f8f61650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:29.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.923+0000 7f98ff005740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f674a0 0x55c7f8f61b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:29.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.923+0000 7f98ff005740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c7f8f74f40 0x55c7f8f7d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:29.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.923+0000 7f98ff005740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f98f41aa150 con 0x55c7f8f74f40 2024-05-22T23:03:29.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.923+0000 7f98ff005740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f98f41ab5c0 con 0x55c7f8f674a0 2024-05-22T23:03:29.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.923+0000 7f98ff005740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f98f41aa820 con 0x55c7f8f65db0 2024-05-22T23:03:29.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.923+0000 7f98ff005740 10 client.0 _ll_register_callbacks cb 0x7f98e4012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:03:29.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.923+0000 7f98f989d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c7f8f74f40 0x55c7f8f7d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:29.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.923+0000 7f98fa09e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f674a0 0x55c7f8f61b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:29.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.923+0000 7f98fa09e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f674a0 0x55c7f8f61b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:46030/0 (socket says 172.21.15.84:46030) 2024-05-22T23:03:29.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.923+0000 7f98fa09e640 1 -- 172.21.15.84:0/19011053 learned_addr learned my addr 172.21.15.84:0/19011053 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:03:29.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.923+0000 7f98f989d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7f8f65db0 0x55c7f8f61650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:29.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.923+0000 7f98f989d640 1 -- 172.21.15.84:0/19011053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f674a0 msgr2=0x55c7f8f61b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:29.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.923+0000 7f98f989d640 1 --2- 172.21.15.84:0/19011053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f674a0 0x55c7f8f61b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.923+0000 7f98f989d640 1 -- 172.21.15.84:0/19011053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c7f8f74f40 msgr2=0x55c7f8f7d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:29.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.923+0000 7f98f989d640 1 --2- 172.21.15.84:0/19011053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c7f8f74f40 0x55c7f8f7d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.923+0000 7f98f989d640 1 -- 172.21.15.84:0/19011053 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7f8e88fe0 con 0x55c7f8f65db0 2024-05-22T23:03:29.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.923+0000 7f98fa09e640 1 --2- 172.21.15.84:0/19011053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f674a0 0x55c7f8f61b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:03:29.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.923+0000 7f98f989d640 1 --2- 172.21.15.84:0/19011053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7f8f65db0 0x55c7f8f61650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f98e400c010 tx=0x7f98e4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaf0bc0619d02844 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:29.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.923+0000 7f98ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-22T23:03:29.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.923+0000 7f98ddffb640 1 -- 172.21.15.84:0/19011053 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4030030 con 0x55c7f8f65db0 2024-05-22T23:03:29.941 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.924+0000 7f98ff005740 1 -- 172.21.15.84:0/19011053 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98f0005040 con 0x55c7f8f65db0 2024-05-22T23:03:29.941 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.924+0000 7f98ddffb640 1 -- 172.21.15.84:0/19011053 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f98e40046c0 con 0x55c7f8f65db0 2024-05-22T23:03:29.941 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.924+0000 7f98ddffb640 1 -- 172.21.15.84:0/19011053 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f98e40048c0 con 0x55c7f8f65db0 2024-05-22T23:03:29.941 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.924+0000 7f98ff005740 10 client.25153 fetch_fsmap learned FSMap version 18 2024-05-22T23:03:29.941 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.924+0000 7f98ff005740 1 -- 172.21.15.84:0/19011053 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55c7f8e5a1c0 con 0x55c7f8f65db0 2024-05-22T23:03:29.941 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.924+0000 7f98ddffb640 1 -- 172.21.15.84:0/19011053 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98f0005040 con 0x55c7f8f65db0 2024-05-22T23:03:29.941 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.924+0000 7f98ddffb640 1 -- 172.21.15.84:0/19011053 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f98e4041050 con 0x55c7f8f65db0 2024-05-22T23:03:29.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.924+0000 7f98ff005740 10 client.25153 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:03:29.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.924+0000 7f98ff005740 10 client.25153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:03:29.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.924+0000 7f98ff005740 20 client.25153 populate_metadata read hostname 'smithi084' 2024-05-22T23:03:29.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.924+0000 7f98ff005740 1 --2- 172.21.15.84:0/19011053 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c7f8ff02c0 0x55c7f8ff26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:29.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.924+0000 7f98ff005740 4 client.25153 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:03:29.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.924+0000 7f98ff005740 1 -- 172.21.15.84:0/19011053 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c7f8ed1830 con 0x55c7f8ff02c0 2024-05-22T23:03:29.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.924+0000 7f98fa09e640 1 --2- 172.21.15.84:0/19011053 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c7f8ff02c0 0x55c7f8ff26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:29.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.925+0000 7f98ddffb640 10 client.25153 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:03:29.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.925+0000 7f98fa09e640 1 --2- 172.21.15.84:0/19011053 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c7f8ff02c0 0x55c7f8ff26e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdbd4c1436aa51d6 server_cookie=b2505ffbac5c354b in_seq=0 out_seq=0 2024-05-22T23:03:29.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.925+0000 7f98ddffb640 1 -- 172.21.15.84:0/19011053 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c7f8ed1830 con 0x55c7f8ff02c0 2024-05-22T23:03:29.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.925+0000 7f98ddffb640 10 client.25153 handle_command_reply: tid=0 2024-05-22T23:03:29.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.925+0000 7f98ff005740 1 client.25153 shutdown 2024-05-22T23:03:29.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.925+0000 7f98ff005740 1 -- 172.21.15.84:0/19011053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7f8f65db0 msgr2=0x55c7f8f61650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:29.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.925+0000 7f98ff005740 1 --2- 172.21.15.84:0/19011053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7f8f65db0 0x55c7f8f61650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f98e400c010 tx=0x7f98e4000bc0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.925+0000 7f98ff005740 1 -- 172.21.15.84:0/19011053 shutdown_connections 2024-05-22T23:03:29.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.925+0000 7f98ff005740 1 --2- 172.21.15.84:0/19011053 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c7f8ff02c0 0x55c7f8ff26e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.925+0000 7f98ff005740 1 --2- 172.21.15.84:0/19011053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c7f8f74f40 0x55c7f8f7d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.925+0000 7f98ff005740 1 --2- 172.21.15.84:0/19011053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7f8f674a0 0x55c7f8f61b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.925+0000 7f98ff005740 1 --2- 172.21.15.84:0/19011053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7f8f65db0 0x55c7f8f61650 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.925+0000 7f98ddffb640 0 client.25153 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:03:29.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.925+0000 7f98ff005740 1 -- 172.21.15.84:0/19011053 >> 172.21.15.84:0/19011053 conn(0x55c7f8f2e8e0 msgr2=0x55c7f8f726d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:29.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.925+0000 7f98ff005740 1 -- 172.21.15.84:0/19011053 shutdown_connections 2024-05-22T23:03:29.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.925+0000 7f98ff005740 1 -- 172.21.15.84:0/19011053 wait complete. 2024-05-22T23:03:29.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.925+0000 7f98ff005740 20 client.25153 trim_cache size 0 max 16384 2024-05-22T23:03:29.946 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:03:29.946 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "no active scrubs running", 2024-05-22T23:03:29.947 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": {} 2024-05-22T23:03:29.947 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:03:29.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.926+0000 7f98fc302640 1 -- 172.21.15.84:0/979467151 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f98e00a2d80 msgr2=0x7f98e00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:29.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.926+0000 7f98fc302640 1 --2- 172.21.15.84:0/979467151 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f98e00a2d80 0x7f98e00a5240 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0b98f0 tx=0x7f98ec08eee0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.926+0000 7f98fc302640 1 -- 172.21.15.84:0/979467151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98f4106750 msgr2=0x7f98f419ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:29.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.926+0000 7f98fc302640 1 --2- 172.21.15.84:0/979467151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98f4106750 0x7f98f419ca10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f98ec02ae00 tx=0x7f98ec031e60 comp rx=0 tx=0).stop 2024-05-22T23:03:29.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.926+0000 7f98fc302640 1 -- 172.21.15.84:0/979467151 shutdown_connections 2024-05-22T23:03:29.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.926+0000 7f98fc302640 1 --2- 172.21.15.84:0/979467151 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f98e00a2d80 0x7f98e00a5240 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.926+0000 7f98fc302640 1 --2- 172.21.15.84:0/979467151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98f4106750 0x7f98f419ca10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.926+0000 7f98fc302640 1 --2- 172.21.15.84:0/979467151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98f4101100 0x7f98f4198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.926+0000 7f98fc302640 1 --2- 172.21.15.84:0/979467151 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98f4100740 0x7f98f4198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:29.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.926+0000 7f98fc302640 1 -- 172.21.15.84:0/979467151 >> 172.21.15.84:0/979467151 conn(0x7f98f4077870 msgr2=0x7f98f4068d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:29.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.926+0000 7f98fc302640 1 -- 172.21.15.84:0/979467151 shutdown_connections 2024-05-22T23:03:29.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:29.926+0000 7f98fc302640 1 -- 172.21.15.84:0/979467151 wait complete. 2024-05-22T23:03:29.951 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-22T23:03:29.951 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:03:30.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.055+0000 7fa967e4d640 1 -- 172.21.15.84:0/1823417410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa960102b00 msgr2=0x7fa960102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:30.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.055+0000 7fa967e4d640 1 --2- 172.21.15.84:0/1823417410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa960102b00 0x7fa960102ee0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009ca0 tx=0x7fa95c02a9e0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.056+0000 7fa967e4d640 1 -- 172.21.15.84:0/1823417410 shutdown_connections 2024-05-22T23:03:30.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.056+0000 7fa967e4d640 1 --2- 172.21.15.84:0/1823417410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa96010be40 0x7fa96010e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.056+0000 7fa967e4d640 1 --2- 172.21.15.84:0/1823417410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa960103420 0x7fa96010b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.056+0000 7fa967e4d640 1 --2- 172.21.15.84:0/1823417410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa960102b00 0x7fa960102ee0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.056+0000 7fa967e4d640 1 -- 172.21.15.84:0/1823417410 >> 172.21.15.84:0/1823417410 conn(0x7fa960077870 msgr2=0x7fa960077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:30.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.056+0000 7fa967e4d640 1 -- 172.21.15.84:0/1823417410 shutdown_connections 2024-05-22T23:03:30.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.056+0000 7fa967e4d640 1 -- 172.21.15.84:0/1823417410 wait complete. 2024-05-22T23:03:30.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.056+0000 7fa967e4d640 1 Processor -- start 2024-05-22T23:03:30.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.056+0000 7fa967e4d640 1 -- start start 2024-05-22T23:03:30.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.056+0000 7fa967e4d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa960102b00 0x7fa960100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:30.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.056+0000 7fa967e4d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa960103420 0x7fa960100c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:30.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.056+0000 7fa967e4d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa96010be40 0x7fa9601011d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:30.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.056+0000 7fa967e4d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa960111c90 con 0x7fa96010be40 2024-05-22T23:03:30.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.056+0000 7fa967e4d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa960111b10 con 0x7fa960102b00 2024-05-22T23:03:30.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.056+0000 7fa967e4d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa960111e10 con 0x7fa960103420 2024-05-22T23:03:30.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.057+0000 7fa965be9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa960102b00 0x7fa960100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:30.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.057+0000 7fa9663ea640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa96010be40 0x7fa9601011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:30.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.057+0000 7fa965be9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa960102b00 0x7fa960100750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:46038/0 (socket says 172.21.15.84:46038) 2024-05-22T23:03:30.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.057+0000 7fa965be9640 1 -- 172.21.15.84:0/574289652 learned_addr learned my addr 172.21.15.84:0/574289652 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:03:30.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.057+0000 7fa9653e8640 1 --2- 172.21.15.84:0/574289652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa960103420 0x7fa960100c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:30.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.057+0000 7fa9663ea640 1 -- 172.21.15.84:0/574289652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa960103420 msgr2=0x7fa960100c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:30.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.057+0000 7fa9663ea640 1 --2- 172.21.15.84:0/574289652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa960103420 0x7fa960100c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.057+0000 7fa9663ea640 1 -- 172.21.15.84:0/574289652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa960102b00 msgr2=0x7fa960100750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:30.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.057+0000 7fa9663ea640 1 --2- 172.21.15.84:0/574289652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa960102b00 0x7fa960100750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.057+0000 7fa9663ea640 1 -- 172.21.15.84:0/574289652 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9601a6670 con 0x7fa96010be40 2024-05-22T23:03:30.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.057+0000 7fa9663ea640 1 --2- 172.21.15.84:0/574289652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa96010be40 0x7fa9601011d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa95800d6b0 tx=0x7fa95800db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:30.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.057+0000 7fa956ffd640 1 -- 172.21.15.84:0/574289652 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958017070 con 0x7fa96010be40 2024-05-22T23:03:30.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.057+0000 7fa967e4d640 1 -- 172.21.15.84:0/574289652 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9601a6960 con 0x7fa96010be40 2024-05-22T23:03:30.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.057+0000 7fa956ffd640 1 -- 172.21.15.84:0/574289652 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa958009d00 con 0x7fa96010be40 2024-05-22T23:03:30.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.057+0000 7fa956ffd640 1 -- 172.21.15.84:0/574289652 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958003d60 con 0x7fa96010be40 2024-05-22T23:03:30.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.057+0000 7fa967e4d640 1 -- 172.21.15.84:0/574289652 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9601a6eb0 con 0x7fa96010be40 2024-05-22T23:03:30.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.058+0000 7fa967e4d640 1 -- 172.21.15.84:0/574289652 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa960068690 con 0x7fa96010be40 2024-05-22T23:03:30.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.059+0000 7fa956ffd640 1 -- 172.21.15.84:0/574289652 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa958009820 con 0x7fa96010be40 2024-05-22T23:03:30.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.060+0000 7fa956ffd640 1 --2- 172.21.15.84:0/574289652 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fa94c0a2d00 0x7fa94c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:30.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.060+0000 7fa965be9640 1 --2- 172.21.15.84:0/574289652 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fa94c0a2d00 0x7fa94c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:30.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.060+0000 7fa956ffd640 1 -- 172.21.15.84:0/574289652 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fa9580ced00 con 0x7fa96010be40 2024-05-22T23:03:30.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.060+0000 7fa965be9640 1 --2- 172.21.15.84:0/574289652 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fa94c0a2d00 0x7fa94c0a51c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009a60 tx=0x7fa95c03a040 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:30.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.065+0000 7fa956ffd640 1 -- 172.21.15.84:0/574289652 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa95809b920 con 0x7fa96010be40 2024-05-22T23:03:30.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.300+0000 7fa967e4d640 1 -- 172.21.15.84:0/574289652 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa96002d200 con 0x7fa96010be40 2024-05-22T23:03:30.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.301+0000 7fa956ffd640 1 -- 172.21.15.84:0/574289652 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7fa95809b220 con 0x7fa96010be40 2024-05-22T23:03:30.304 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:03:30.304 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:03:30.304 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:03:30.306 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.303+0000 7fa967e4d640 1 -- 172.21.15.84:0/574289652 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fa94c0a2d00 msgr2=0x7fa94c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:30.306 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.303+0000 7fa967e4d640 1 --2- 172.21.15.84:0/574289652 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fa94c0a2d00 0x7fa94c0a51c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009a60 tx=0x7fa95c03a040 comp rx=0 tx=0).stop 2024-05-22T23:03:30.306 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.303+0000 7fa967e4d640 1 -- 172.21.15.84:0/574289652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa96010be40 msgr2=0x7fa9601011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:30.306 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.303+0000 7fa967e4d640 1 --2- 172.21.15.84:0/574289652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa96010be40 0x7fa9601011d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa95800d6b0 tx=0x7fa95800db70 comp rx=0 tx=0).stop 2024-05-22T23:03:30.306 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.303+0000 7fa967e4d640 1 -- 172.21.15.84:0/574289652 shutdown_connections 2024-05-22T23:03:30.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.303+0000 7fa967e4d640 1 --2- 172.21.15.84:0/574289652 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fa94c0a2d00 0x7fa94c0a51c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.303+0000 7fa967e4d640 1 --2- 172.21.15.84:0/574289652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa96010be40 0x7fa9601011d0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.303+0000 7fa967e4d640 1 --2- 172.21.15.84:0/574289652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa960103420 0x7fa960100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.303+0000 7fa967e4d640 1 --2- 172.21.15.84:0/574289652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa960102b00 0x7fa960100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.303+0000 7fa967e4d640 1 -- 172.21.15.84:0/574289652 >> 172.21.15.84:0/574289652 conn(0x7fa960077870 msgr2=0x7fa9600fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:30.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.303+0000 7fa967e4d640 1 -- 172.21.15.84:0/574289652 shutdown_connections 2024-05-22T23:03:30.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.303+0000 7fa967e4d640 1 -- 172.21.15.84:0/574289652 wait complete. 2024-05-22T23:03:30.317 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-22T23:03:30.423 INFO:tasks.workunit.client.0.smithi084.stdout: 1 51466 18.38 MB/sec warmup 16 sec latency 14.406 ms 2024-05-22T23:03:30.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.423+0000 7f72c7658640 1 -- 172.21.15.84:0/2042741959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72c0102280 msgr2=0x7f72c010e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:30.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.423+0000 7f72c7658640 1 --2- 172.21.15.84:0/2042741959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72c0102280 0x7f72c010e790 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009cd0 tx=0x7f72b802a8e0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.423+0000 7f72c7658640 1 -- 172.21.15.84:0/2042741959 shutdown_connections 2024-05-22T23:03:30.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.423+0000 7f72c7658640 1 --2- 172.21.15.84:0/2042741959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72c0102280 0x7f72c010e790 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.423+0000 7f72c7658640 1 --2- 172.21.15.84:0/2042741959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72c01018c0 0x7f72c0101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.423+0000 7f72c7658640 1 --2- 172.21.15.84:0/2042741959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72c01078d0 0x7f72c0107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.423+0000 7f72c7658640 1 -- 172.21.15.84:0/2042741959 >> 172.21.15.84:0/2042741959 conn(0x7f72c00fd3d0 msgr2=0x7f72c00ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:30.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.423+0000 7f72c7658640 1 -- 172.21.15.84:0/2042741959 shutdown_connections 2024-05-22T23:03:30.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.423+0000 7f72c7658640 1 -- 172.21.15.84:0/2042741959 wait complete. 2024-05-22T23:03:30.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.424+0000 7f72c7658640 1 Processor -- start 2024-05-22T23:03:30.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.424+0000 7f72c7658640 1 -- start start 2024-05-22T23:03:30.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.424+0000 7f72c7658640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72c01018c0 0x7f72c01981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:30.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.424+0000 7f72c7658640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72c0102280 0x7f72c01986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:30.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.424+0000 7f72c7658640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72c01078d0 0x7f72c019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:30.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.424+0000 7f72c7658640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f72c0111ce0 con 0x7f72c01078d0 2024-05-22T23:03:30.429 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.424+0000 7f72c7658640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f72c0111b60 con 0x7f72c0102280 2024-05-22T23:03:30.430 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.424+0000 7f72c7658640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f72c0111e60 con 0x7f72c01018c0 2024-05-22T23:03:30.430 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.424+0000 7f72c5bf5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72c01078d0 0x7f72c019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:30.430 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.424+0000 7f72c4bf3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72c0102280 0x7f72c01986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:30.430 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.424+0000 7f72c5bf5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72c01078d0 0x7f72c019ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49474/0 (socket says 172.21.15.84:49474) 2024-05-22T23:03:30.430 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.424+0000 7f72c5bf5640 1 -- 172.21.15.84:0/3249104077 learned_addr learned my addr 172.21.15.84:0/3249104077 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:03:30.430 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.424+0000 7f72c53f4640 1 --2- 172.21.15.84:0/3249104077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72c01018c0 0x7f72c01981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:30.430 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.425+0000 7f72c4bf3640 1 -- 172.21.15.84:0/3249104077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72c01018c0 msgr2=0x7f72c01981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:30.430 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.425+0000 7f72c4bf3640 1 --2- 172.21.15.84:0/3249104077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72c01018c0 0x7f72c01981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.430 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.425+0000 7f72c4bf3640 1 -- 172.21.15.84:0/3249104077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72c01078d0 msgr2=0x7f72c019ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:30.430 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.425+0000 7f72c4bf3640 1 --2- 172.21.15.84:0/3249104077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72c01078d0 0x7f72c019ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.430 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.425+0000 7f72c4bf3640 1 -- 172.21.15.84:0/3249104077 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c019d190 con 0x7f72c0102280 2024-05-22T23:03:30.430 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.425+0000 7f72c53f4640 1 --2- 172.21.15.84:0/3249104077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72c01018c0 0x7f72c01981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:03:30.430 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.425+0000 7f72c4bf3640 1 --2- 172.21.15.84:0/3249104077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72c0102280 0x7f72c01986e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f72b000c970 tx=0x7f72b000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:30.431 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.425+0000 7f72b67fc640 1 -- 172.21.15.84:0/3249104077 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0007bb0 con 0x7f72c0102280 2024-05-22T23:03:30.431 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.425+0000 7f72b67fc640 1 -- 172.21.15.84:0/3249104077 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72b0007d50 con 0x7f72c0102280 2024-05-22T23:03:30.431 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.425+0000 7f72c7658640 1 -- 172.21.15.84:0/3249104077 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c019d480 con 0x7f72c0102280 2024-05-22T23:03:30.431 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.425+0000 7f72b67fc640 1 -- 172.21.15.84:0/3249104077 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0018820 con 0x7f72c0102280 2024-05-22T23:03:30.431 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.425+0000 7f72c7658640 1 -- 172.21.15.84:0/3249104077 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c01a4d10 con 0x7f72c0102280 2024-05-22T23:03:30.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.427+0000 7f72b67fc640 1 -- 172.21.15.84:0/3249104077 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f72b00189c0 con 0x7f72c0102280 2024-05-22T23:03:30.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.428+0000 7f72ca365740 1 Processor -- start 2024-05-22T23:03:30.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.428+0000 7f72ca365740 1 -- start start 2024-05-22T23:03:30.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.428+0000 7f72b67fc640 1 --2- 172.21.15.84:0/3249104077 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f72ac0a2d00 0x7f72ac0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:30.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.428+0000 7f72ca365740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f2ac83f80 0x563f2ac84360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:30.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.428+0000 7f72ca365740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f2ac848a0 0x563f2ac8ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:30.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.428+0000 7f72c5bf5640 1 --2- 172.21.15.84:0/3249104077 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f72ac0a2d00 0x7f72ac0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:30.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.428+0000 7f72ca365740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac8ef70 0x563f2ac91360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:30.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.428+0000 7f72ca365740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563f2abe8db0 con 0x563f2ac8ef70 2024-05-22T23:03:30.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.428+0000 7f72ca365740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563f2ab87400 con 0x563f2ac848a0 2024-05-22T23:03:30.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.428+0000 7f72ca365740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563f2abe4d70 con 0x563f2ac83f80 2024-05-22T23:03:30.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.428+0000 7f72b67fc640 1 -- 172.21.15.84:0/3249104077 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f72b00cf540 con 0x7f72c0102280 2024-05-22T23:03:30.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.428+0000 7f72c5bf5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac8ef70 0x563f2ac91360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:30.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.428+0000 7f72c5bf5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac8ef70 0x563f2ac91360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49476/0 (socket says 172.21.15.84:49476) 2024-05-22T23:03:30.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.428+0000 7f72c5bf5640 1 -- 172.21.15.84:0/2199405270 learned_addr learned my addr 172.21.15.84:0/2199405270 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:03:30.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.429+0000 7f72c4bf3640 1 --2- 172.21.15.84:0/2199405270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f2ac848a0 0x563f2ac8ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:30.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.429+0000 7f72c53f4640 1 --2- 172.21.15.84:0/2199405270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f2ac83f80 0x563f2ac84360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:30.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.429+0000 7f72c4bf3640 1 -- 172.21.15.84:0/2199405270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f2ac83f80 msgr2=0x563f2ac84360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:30.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.429+0000 7f72c4bf3640 1 --2- 172.21.15.84:0/2199405270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f2ac83f80 0x563f2ac84360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.429+0000 7f72c4bf3640 1 -- 172.21.15.84:0/2199405270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac8ef70 msgr2=0x563f2ac91360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:30.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.429+0000 7f72c4bf3640 1 --2- 172.21.15.84:0/2199405270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac8ef70 0x563f2ac91360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.429+0000 7f72c4bf3640 1 -- 172.21.15.84:0/2199405270 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f2ab792f0 con 0x563f2ac848a0 2024-05-22T23:03:30.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.429+0000 7f72c5bf5640 1 --2- 172.21.15.84:0/3249104077 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f72ac0a2d00 0x7f72ac0a51c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009a60 tx=0x7f72b8031430 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:30.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.429+0000 7f72c4bf3640 1 --2- 172.21.15.84:0/2199405270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f2ac848a0 0x563f2ac8ea30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f72b00ccc30 tx=0x7f72b0018d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c09ad7f1e946565c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:30.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.429+0000 7f72ab7fe640 1 -- 172.21.15.84:0/2199405270 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b00b4070 con 0x563f2ac848a0 2024-05-22T23:03:30.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.429+0000 7f72ab7fe640 1 -- 172.21.15.84:0/2199405270 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72b009fdb0 con 0x563f2ac848a0 2024-05-22T23:03:30.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.429+0000 7f72ab7fe640 1 -- 172.21.15.84:0/2199405270 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b00a41e0 con 0x563f2ac848a0 2024-05-22T23:03:30.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.430+0000 7f72ca365740 1 -- 172.21.15.84:0/2199405270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f2ac848a0 msgr2=0x563f2ac8ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:30.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.430+0000 7f72ca365740 1 --2- 172.21.15.84:0/2199405270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f2ac848a0 0x563f2ac8ea30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f72b00ccc30 tx=0x7f72b0018d30 comp rx=0 tx=0).stop 2024-05-22T23:03:30.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.430+0000 7f72ca365740 1 -- 172.21.15.84:0/2199405270 shutdown_connections 2024-05-22T23:03:30.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.430+0000 7f72ca365740 1 --2- 172.21.15.84:0/2199405270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac8ef70 0x563f2ac91360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.430+0000 7f72ca365740 1 --2- 172.21.15.84:0/2199405270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f2ac848a0 0x563f2ac8ea30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.430+0000 7f72ca365740 1 --2- 172.21.15.84:0/2199405270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f2ac83f80 0x563f2ac84360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.430+0000 7f72ca365740 1 -- 172.21.15.84:0/2199405270 >> 172.21.15.84:0/2199405270 conn(0x563f2ac802a0 msgr2=0x563f2ac80680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:30.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.430+0000 7f72ca365740 1 -- 172.21.15.84:0/2199405270 shutdown_connections 2024-05-22T23:03:30.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.430+0000 7f72ca365740 1 -- 172.21.15.84:0/2199405270 wait complete. 2024-05-22T23:03:30.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.431+0000 7f72ca365740 1 -- start start 2024-05-22T23:03:30.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.431+0000 7f72ca365740 1 Processor -- start 2024-05-22T23:03:30.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.431+0000 7f72ca365740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac83f80 0x563f2ac8bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:30.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.431+0000 7f72ca365740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f2ac848a0 0x563f2ac8c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:30.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.431+0000 7f72ca365740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f2ac8ef70 0x563f2ac8c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:30.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.431+0000 7f72ca365740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563f2ac91d60 con 0x563f2ac83f80 2024-05-22T23:03:30.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.431+0000 7f72ca365740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563f2aba32e0 con 0x563f2ac848a0 2024-05-22T23:03:30.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.431+0000 7f72ca365740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563f2ac91ee0 con 0x563f2ac8ef70 2024-05-22T23:03:30.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.431+0000 7f72ca365740 10 client.0 _ll_register_callbacks cb 0x563f2abaca70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:03:30.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.431+0000 7f72c53f4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac83f80 0x563f2ac8bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:30.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.431+0000 7f72c53f4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac83f80 0x563f2ac8bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49484/0 (socket says 172.21.15.84:49484) 2024-05-22T23:03:30.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.431+0000 7f72c53f4640 1 -- 172.21.15.84:0/878295230 learned_addr learned my addr 172.21.15.84:0/878295230 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:03:30.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.431+0000 7f72c4bf3640 1 --2- 172.21.15.84:0/878295230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f2ac848a0 0x563f2ac8c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:30.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.431+0000 7f72c5bf5640 1 --2- 172.21.15.84:0/878295230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f2ac8ef70 0x563f2ac8c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:30.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.431+0000 7f72c53f4640 1 -- 172.21.15.84:0/878295230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f2ac8ef70 msgr2=0x563f2ac8c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:30.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.431+0000 7f72c53f4640 1 --2- 172.21.15.84:0/878295230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f2ac8ef70 0x563f2ac8c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.431+0000 7f72c53f4640 1 -- 172.21.15.84:0/878295230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f2ac848a0 msgr2=0x563f2ac8c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:30.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.431+0000 7f72c53f4640 1 --2- 172.21.15.84:0/878295230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f2ac848a0 0x563f2ac8c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.431+0000 7f72c53f4640 1 -- 172.21.15.84:0/878295230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f2a8fc3f0 con 0x563f2ac83f80 2024-05-22T23:03:30.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.432+0000 7f72c53f4640 1 --2- 172.21.15.84:0/878295230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac83f80 0x563f2ac8bf60 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f72bc009fd0 tx=0x7f72bc0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a036057bf62efd server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:30.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.432+0000 7f72a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:03:30.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.432+0000 7f72a8ff9640 1 -- 172.21.15.84:0/878295230 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc013930 con 0x563f2ac83f80 2024-05-22T23:03:30.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.432+0000 7f72a8ff9640 1 -- 172.21.15.84:0/878295230 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f72bc013ad0 con 0x563f2ac83f80 2024-05-22T23:03:30.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.432+0000 7f72ca365740 1 -- 172.21.15.84:0/878295230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72b009fdb0 con 0x563f2ac83f80 2024-05-22T23:03:30.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.432+0000 7f72a8ff9640 1 -- 172.21.15.84:0/878295230 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f72bc013cd0 con 0x563f2ac83f80 2024-05-22T23:03:30.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.432+0000 7f72ca365740 10 client.15678 fetch_fsmap learned FSMap version 18 2024-05-22T23:03:30.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.432+0000 7f72ca365740 1 -- 172.21.15.84:0/878295230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x563f2a9c67d0 con 0x563f2ac83f80 2024-05-22T23:03:30.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.432+0000 7f72a8ff9640 1 -- 172.21.15.84:0/878295230 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72b009fdb0 con 0x563f2ac83f80 2024-05-22T23:03:30.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.432+0000 7f72a8ff9640 1 -- 172.21.15.84:0/878295230 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f72bc01a070 con 0x563f2ac83f80 2024-05-22T23:03:30.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.432+0000 7f72ca365740 10 client.15678 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:03:30.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.432+0000 7f72ca365740 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:03:30.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.432+0000 7f72ca365740 20 client.15678 populate_metadata read hostname 'smithi084' 2024-05-22T23:03:30.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.432+0000 7f72ca365740 1 --2- 172.21.15.84:0/878295230 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563f2ac9f350 0x563f2aca17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:30.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.432+0000 7f72ca365740 4 client.15678 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:03:30.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.432+0000 7f72ca365740 1 -- 172.21.15.84:0/878295230 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f2abf64e0 con 0x563f2ac9f350 2024-05-22T23:03:30.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.433+0000 7f72c4bf3640 1 --2- 172.21.15.84:0/878295230 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563f2ac9f350 0x563f2aca17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:30.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.433+0000 7f72c4bf3640 1 --2- 172.21.15.84:0/878295230 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563f2ac9f350 0x563f2aca17e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc7b1e369c10017d server_cookie=9d01b142c034f2f6 in_seq=0 out_seq=0 2024-05-22T23:03:30.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.433+0000 7f72a8ff9640 10 client.15678 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:03:30.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.434+0000 7f72a8ff9640 1 -- 172.21.15.84:0/878295230 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f2abf64e0 con 0x563f2ac9f350 2024-05-22T23:03:30.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.434+0000 7f72a8ff9640 10 client.15678 handle_command_reply: tid=0 2024-05-22T23:03:30.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.434+0000 7f72ca365740 1 client.15678 shutdown 2024-05-22T23:03:30.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.434+0000 7f72ca365740 1 -- 172.21.15.84:0/878295230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac83f80 msgr2=0x563f2ac8bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:30.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.434+0000 7f72ca365740 1 --2- 172.21.15.84:0/878295230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac83f80 0x563f2ac8bf60 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f72bc009fd0 tx=0x7f72bc0039c0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.434+0000 7f72ca365740 1 -- 172.21.15.84:0/878295230 shutdown_connections 2024-05-22T23:03:30.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.434+0000 7f72ca365740 1 --2- 172.21.15.84:0/878295230 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563f2ac9f350 0x563f2aca17e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.434+0000 7f72ca365740 1 --2- 172.21.15.84:0/878295230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f2ac8ef70 0x563f2ac8c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.434+0000 7f72a8ff9640 0 client.15678 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:03:30.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.434+0000 7f72ca365740 1 --2- 172.21.15.84:0/878295230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f2ac848a0 0x563f2ac8c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.434+0000 7f72ca365740 1 --2- 172.21.15.84:0/878295230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac83f80 0x563f2ac8bf60 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.434+0000 7f72ca365740 1 -- 172.21.15.84:0/878295230 >> 172.21.15.84:0/878295230 conn(0x563f2ac802a0 msgr2=0x563f2ac860a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:30.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.434+0000 7f72ca365740 1 -- 172.21.15.84:0/878295230 shutdown_connections 2024-05-22T23:03:30.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.434+0000 7f72ca365740 1 -- 172.21.15.84:0/878295230 wait complete. 2024-05-22T23:03:30.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.434+0000 7f72ca365740 20 client.15678 trim_cache size 0 max 16384 2024-05-22T23:03:30.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.449+0000 7f72ca365740 1 Processor -- start 2024-05-22T23:03:30.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.449+0000 7f72ca365740 1 -- start start 2024-05-22T23:03:30.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.450+0000 7f72ca365740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f2ac83f80 0x563f2ac9ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:30.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.450+0000 7f72ca365740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac848a0 0x563f2ac8c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:30.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.450+0000 7f72ca365740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f2ac8ef70 0x563f2aca70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:30.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.450+0000 7f72ca365740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f72c01ab630 con 0x563f2ac848a0 2024-05-22T23:03:30.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.450+0000 7f72ca365740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f72c01adeb0 con 0x563f2ac83f80 2024-05-22T23:03:30.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.450+0000 7f72ca365740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f72c01abca0 con 0x563f2ac8ef70 2024-05-22T23:03:30.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.450+0000 7f72c4bf3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac848a0 0x563f2ac8c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:30.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.450+0000 7f72c4bf3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac848a0 0x563f2ac8c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49486/0 (socket says 172.21.15.84:49486) 2024-05-22T23:03:30.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.450+0000 7f72c4bf3640 1 -- 172.21.15.84:0/3582080697 learned_addr learned my addr 172.21.15.84:0/3582080697 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:03:30.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.450+0000 7f72c53f4640 1 --2- 172.21.15.84:0/3582080697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f2ac83f80 0x563f2ac9ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:30.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.450+0000 7f72c4bf3640 1 -- 172.21.15.84:0/3582080697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f2ac8ef70 msgr2=0x563f2aca70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:30.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.450+0000 7f72c5bf5640 1 --2- 172.21.15.84:0/3582080697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f2ac8ef70 0x563f2aca70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:30.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.450+0000 7f72c4bf3640 1 --2- 172.21.15.84:0/3582080697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f2ac8ef70 0x563f2aca70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.450+0000 7f72c4bf3640 1 -- 172.21.15.84:0/3582080697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f2ac83f80 msgr2=0x563f2ac9ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:30.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.450+0000 7f72c4bf3640 1 --2- 172.21.15.84:0/3582080697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f2ac83f80 0x563f2ac9ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.450+0000 7f72c4bf3640 1 -- 172.21.15.84:0/3582080697 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f2ac846d0 con 0x563f2ac848a0 2024-05-22T23:03:30.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.450+0000 7f72c5bf5640 1 --2- 172.21.15.84:0/3582080697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f2ac8ef70 0x563f2aca70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:03:30.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.450+0000 7f72c4bf3640 1 --2- 172.21.15.84:0/3582080697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac848a0 0x563f2ac8c800 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f72b0099cb0 tx=0x7f72b0086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494233120d464af2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:30.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.450+0000 7f72aaffd640 1 -- 172.21.15.84:0/3582080697 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0065070 con 0x563f2ac848a0 2024-05-22T23:03:30.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.450+0000 7f72aaffd640 1 -- 172.21.15.84:0/3582080697 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72b000fac0 con 0x563f2ac848a0 2024-05-22T23:03:30.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.450+0000 7f72aaffd640 1 -- 172.21.15.84:0/3582080697 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b00b4040 con 0x563f2ac848a0 2024-05-22T23:03:30.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.451+0000 7f72ca365740 1 -- 172.21.15.84:0/3582080697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac848a0 msgr2=0x563f2ac8c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:30.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.451+0000 7f72ca365740 1 --2- 172.21.15.84:0/3582080697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac848a0 0x563f2ac8c800 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f72b0099cb0 tx=0x7f72b0086040 comp rx=0 tx=0).stop 2024-05-22T23:03:30.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.451+0000 7f72ca365740 1 -- 172.21.15.84:0/3582080697 shutdown_connections 2024-05-22T23:03:30.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.451+0000 7f72ca365740 1 --2- 172.21.15.84:0/3582080697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f2ac8ef70 0x563f2aca70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.451+0000 7f72ca365740 1 --2- 172.21.15.84:0/3582080697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac848a0 0x563f2ac8c800 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.451+0000 7f72ca365740 1 --2- 172.21.15.84:0/3582080697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f2ac83f80 0x563f2ac9ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.451+0000 7f72ca365740 1 -- 172.21.15.84:0/3582080697 >> 172.21.15.84:0/3582080697 conn(0x563f2ac802a0 msgr2=0x563f2ac85f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:30.459 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.451+0000 7f72ca365740 1 -- 172.21.15.84:0/3582080697 shutdown_connections 2024-05-22T23:03:30.459 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.451+0000 7f72ca365740 1 -- 172.21.15.84:0/3582080697 wait complete. 2024-05-22T23:03:30.459 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.451+0000 7f72ca365740 1 -- start start 2024-05-22T23:03:30.459 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.451+0000 7f72ca365740 1 Processor -- start 2024-05-22T23:03:30.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.452+0000 7f72ca365740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac83f80 0x563f2ac92c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:30.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.452+0000 7f72ca365740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f2ac848a0 0x563f2ac931d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:30.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.452+0000 7f72ca365740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f2ac8ef70 0x563f2ac93710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:30.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.452+0000 7f72ca365740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f72c01ac370 con 0x563f2ac83f80 2024-05-22T23:03:30.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.452+0000 7f72ca365740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f72c01ad7e0 con 0x563f2ac8ef70 2024-05-22T23:03:30.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.452+0000 7f72ca365740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f72c01aca40 con 0x563f2ac848a0 2024-05-22T23:03:30.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.452+0000 7f72ca365740 10 client.0 _ll_register_callbacks cb 0x7f72bc0041e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:03:30.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.452+0000 7f72c53f4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac83f80 0x563f2ac92c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:30.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.452+0000 7f72c53f4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac83f80 0x563f2ac92c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49492/0 (socket says 172.21.15.84:49492) 2024-05-22T23:03:30.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.452+0000 7f72c53f4640 1 -- 172.21.15.84:0/3975452579 learned_addr learned my addr 172.21.15.84:0/3975452579 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:03:30.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.452+0000 7f72c5bf5640 1 --2- 172.21.15.84:0/3975452579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f2ac8ef70 0x563f2ac93710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:30.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.452+0000 7f72c4bf3640 1 --2- 172.21.15.84:0/3975452579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f2ac848a0 0x563f2ac931d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:30.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.452+0000 7f72c53f4640 1 -- 172.21.15.84:0/3975452579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f2ac848a0 msgr2=0x563f2ac931d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:30.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.452+0000 7f72c53f4640 1 --2- 172.21.15.84:0/3975452579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f2ac848a0 0x563f2ac931d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.452+0000 7f72c53f4640 1 -- 172.21.15.84:0/3975452579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f2ac8ef70 msgr2=0x563f2ac93710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:30.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.452+0000 7f72c53f4640 1 --2- 172.21.15.84:0/3975452579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f2ac8ef70 0x563f2ac93710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.452+0000 7f72c53f4640 1 -- 172.21.15.84:0/3975452579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f2ad04a80 con 0x563f2ac83f80 2024-05-22T23:03:30.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.452+0000 7f72c53f4640 1 --2- 172.21.15.84:0/3975452579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac83f80 0x563f2ac92c90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f72bc001eb0 tx=0x7f72bc005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9100ed2de3a57994 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:30.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.452+0000 7f72a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:03:30.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.453+0000 7f72a8ff9640 1 -- 172.21.15.84:0/3975452579 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc011a90 con 0x563f2ac83f80 2024-05-22T23:03:30.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.453+0000 7f72a8ff9640 1 -- 172.21.15.84:0/3975452579 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f72bc011c30 con 0x563f2ac83f80 2024-05-22T23:03:30.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.453+0000 7f72ca365740 1 -- 172.21.15.84:0/3975452579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72b000fac0 con 0x563f2ac83f80 2024-05-22T23:03:30.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.453+0000 7f72a8ff9640 1 -- 172.21.15.84:0/3975452579 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f72bc009b50 con 0x563f2ac83f80 2024-05-22T23:03:30.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.453+0000 7f72ca365740 10 client.15684 fetch_fsmap learned FSMap version 18 2024-05-22T23:03:30.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.453+0000 7f72ca365740 1 -- 172.21.15.84:0/3975452579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x563f2ad05cd0 con 0x563f2ac83f80 2024-05-22T23:03:30.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.453+0000 7f72a8ff9640 1 -- 172.21.15.84:0/3975452579 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72b000fac0 con 0x563f2ac83f80 2024-05-22T23:03:30.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.453+0000 7f72a8ff9640 1 -- 172.21.15.84:0/3975452579 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f72bc02c030 con 0x563f2ac83f80 2024-05-22T23:03:30.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.453+0000 7f72ca365740 10 client.15684 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:03:30.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.453+0000 7f72ca365740 10 client.15684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:03:30.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.453+0000 7f72ca365740 20 client.15684 populate_metadata read hostname 'smithi084' 2024-05-22T23:03:30.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.453+0000 7f72ca365740 1 --2- 172.21.15.84:0/3975452579 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563f2ad06450 0x563f2ad08870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:30.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.453+0000 7f72ca365740 4 client.15684 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-22T23:03:30.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.453+0000 7f72ca365740 1 -- 172.21.15.84:0/3975452579 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563f2abf0830 con 0x563f2ad06450 2024-05-22T23:03:30.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.453+0000 7f72c4bf3640 1 --2- 172.21.15.84:0/3975452579 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563f2ad06450 0x563f2ad08870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:30.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.454+0000 7f72a8ff9640 10 client.15684 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:03:30.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.454+0000 7f72c4bf3640 1 --2- 172.21.15.84:0/3975452579 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563f2ad06450 0x563f2ad08870 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e19f2507a7f9d4c server_cookie=9992b4237698a5d5 in_seq=0 out_seq=0 2024-05-22T23:03:30.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.454+0000 7f72a8ff9640 1 -- 172.21.15.84:0/3975452579 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563f2abf0830 con 0x563f2ad06450 2024-05-22T23:03:30.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.454+0000 7f72a8ff9640 10 client.15684 handle_command_reply: tid=0 2024-05-22T23:03:30.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.454+0000 7f72ca365740 1 client.15684 shutdown 2024-05-22T23:03:30.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.454+0000 7f72ca365740 1 -- 172.21.15.84:0/3975452579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac83f80 msgr2=0x563f2ac92c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:30.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.454+0000 7f72ca365740 1 --2- 172.21.15.84:0/3975452579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac83f80 0x563f2ac92c90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f72bc001eb0 tx=0x7f72bc005270 comp rx=0 tx=0).stop 2024-05-22T23:03:30.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.454+0000 7f72ca365740 1 -- 172.21.15.84:0/3975452579 shutdown_connections 2024-05-22T23:03:30.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.454+0000 7f72ca365740 1 --2- 172.21.15.84:0/3975452579 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563f2ad06450 0x563f2ad08870 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.454+0000 7f72ca365740 1 --2- 172.21.15.84:0/3975452579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f2ac8ef70 0x563f2ac93710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.454+0000 7f72ca365740 1 --2- 172.21.15.84:0/3975452579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f2ac848a0 0x563f2ac931d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.454+0000 7f72ca365740 1 --2- 172.21.15.84:0/3975452579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563f2ac83f80 0x563f2ac92c90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.454+0000 7f72a8ff9640 0 client.15684 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:03:30.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.454+0000 7f72ca365740 1 -- 172.21.15.84:0/3975452579 >> 172.21.15.84:0/3975452579 conn(0x563f2ac802a0 msgr2=0x563f2ac7ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:30.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.454+0000 7f72ca365740 1 -- 172.21.15.84:0/3975452579 shutdown_connections 2024-05-22T23:03:30.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.454+0000 7f72ca365740 1 -- 172.21.15.84:0/3975452579 wait complete. 2024-05-22T23:03:30.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.454+0000 7f72ca365740 20 client.15684 trim_cache size 0 max 16384 2024-05-22T23:03:30.467 INFO:teuthology.orchestra.run.smithi084.stdout:[] 2024-05-22T23:03:30.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.455+0000 7f72c7658640 1 -- 172.21.15.84:0/3249104077 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f72ac0a2d00 msgr2=0x7f72ac0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:30.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.455+0000 7f72c7658640 1 --2- 172.21.15.84:0/3249104077 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f72ac0a2d00 0x7f72ac0a51c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009a60 tx=0x7f72b8031430 comp rx=0 tx=0).stop 2024-05-22T23:03:30.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.455+0000 7f72c7658640 1 -- 172.21.15.84:0/3249104077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72c0102280 msgr2=0x7f72c01986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:30.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.455+0000 7f72c7658640 1 --2- 172.21.15.84:0/3249104077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72c0102280 0x7f72c01986e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f72b000c970 tx=0x7f72b000ce30 comp rx=0 tx=0).stop 2024-05-22T23:03:30.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.455+0000 7f72c7658640 1 -- 172.21.15.84:0/3249104077 shutdown_connections 2024-05-22T23:03:30.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.455+0000 7f72c7658640 1 --2- 172.21.15.84:0/3249104077 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f72ac0a2d00 0x7f72ac0a51c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.455+0000 7f72c7658640 1 --2- 172.21.15.84:0/3249104077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72c01078d0 0x7f72c019ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.455+0000 7f72c7658640 1 --2- 172.21.15.84:0/3249104077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72c0102280 0x7f72c01986e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.455+0000 7f72c7658640 1 --2- 172.21.15.84:0/3249104077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72c01018c0 0x7f72c01981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:30.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.455+0000 7f72c7658640 1 -- 172.21.15.84:0/3249104077 >> 172.21.15.84:0/3249104077 conn(0x7f72c00fd3d0 msgr2=0x7f72c00fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:30.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.455+0000 7f72c7658640 1 -- 172.21.15.84:0/3249104077 shutdown_connections 2024-05-22T23:03:30.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:30.455+0000 7f72c7658640 1 -- 172.21.15.84:0/3249104077 wait complete. 2024-05-22T23:03:31.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:31 smithi084 ceph-mon[32186]: pgmap v53: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 1.6 MiB/s wr, 187 op/s 2024-05-22T23:03:31.377 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:31 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/574289652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:03:31.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:31 smithi204 ceph-mon[36211]: pgmap v53: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 1.6 MiB/s wr, 187 op/s 2024-05-22T23:03:31.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:31 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/574289652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:03:31.423 INFO:tasks.workunit.client.0.smithi084.stdout: 1 54391 18.23 MB/sec warmup 17 sec latency 15.899 ms 2024-05-22T23:03:31.470 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-22T23:03:31.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:03:31.470 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-22T23:03:31.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:31 smithi096 ceph-mon[36392]: pgmap v53: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 1.6 MiB/s wr, 187 op/s 2024-05-22T23:03:31.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:31 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/574289652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:03:31.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.580+0000 7f1ae9d14640 1 -- 172.21.15.84:0/1580514877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ae4108aa0 msgr2=0x7f1ae410ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:31.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.580+0000 7f1ae9d14640 1 --2- 172.21.15.84:0/1580514877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ae4108aa0 0x7f1ae410ca90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009900 tx=0x7f1ad802a410 comp rx=0 tx=0).stop 2024-05-22T23:03:31.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.580+0000 7f1ae9d14640 1 -- 172.21.15.84:0/1580514877 shutdown_connections 2024-05-22T23:03:31.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.580+0000 7f1ae9d14640 1 --2- 172.21.15.84:0/1580514877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ae410d170 0x7f1ae4114d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.580+0000 7f1ae9d14640 1 --2- 172.21.15.84:0/1580514877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ae4108aa0 0x7f1ae410ca90 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.580+0000 7f1ae9d14640 1 --2- 172.21.15.84:0/1580514877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ae41080f0 0x7f1ae41084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.580+0000 7f1ae9d14640 1 -- 172.21.15.84:0/1580514877 >> 172.21.15.84:0/1580514877 conn(0x7f1ae4075db0 msgr2=0x7f1ae40781d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:31.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.580+0000 7f1ae9d14640 1 -- 172.21.15.84:0/1580514877 shutdown_connections 2024-05-22T23:03:31.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.580+0000 7f1ae9d14640 1 -- 172.21.15.84:0/1580514877 wait complete. 2024-05-22T23:03:31.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.580+0000 7f1ae9d14640 1 Processor -- start 2024-05-22T23:03:31.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.581+0000 7f1ae9d14640 1 -- start start 2024-05-22T23:03:31.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.581+0000 7f1ae9d14640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ae41080f0 0x7f1ae41a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.581+0000 7f1ae9d14640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ae410d170 0x7f1ae41a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.581+0000 7f1ae9d14640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ae419e5f0 0x7f1ae419eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.581+0000 7f1ae9d14640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae4118230 con 0x7f1ae410d170 2024-05-22T23:03:31.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.581+0000 7f1ae9d14640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae41180b0 con 0x7f1ae41080f0 2024-05-22T23:03:31.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.581+0000 7f1ae9d14640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae41183b0 con 0x7f1ae419e5f0 2024-05-22T23:03:31.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.581+0000 7f1ae2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ae410d170 0x7f1ae41a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.581+0000 7f1ae37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ae41080f0 0x7f1ae41a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.581+0000 7f1ae37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ae41080f0 0x7f1ae41a4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:60378/0 (socket says 172.21.15.84:60378) 2024-05-22T23:03:31.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.581+0000 7f1ae37fe640 1 -- 172.21.15.84:0/244862509 learned_addr learned my addr 172.21.15.84:0/244862509 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:03:31.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.581+0000 7f1ae2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ae410d170 0x7f1ae41a4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43646/0 (socket says 172.21.15.84:43646) 2024-05-22T23:03:31.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.581+0000 7f1ae3fff640 1 --2- 172.21.15.84:0/244862509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ae419e5f0 0x7f1ae419eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.581+0000 7f1ae2ffd640 1 -- 172.21.15.84:0/244862509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ae419e5f0 msgr2=0x7f1ae419eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:31.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.582+0000 7f1ae2ffd640 1 --2- 172.21.15.84:0/244862509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ae419e5f0 0x7f1ae419eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.582+0000 7f1ae2ffd640 1 -- 172.21.15.84:0/244862509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ae41080f0 msgr2=0x7f1ae41a4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:31.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.582+0000 7f1ae2ffd640 1 --2- 172.21.15.84:0/244862509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ae41080f0 0x7f1ae41a4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.582+0000 7f1ae2ffd640 1 -- 172.21.15.84:0/244862509 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae419f110 con 0x7f1ae410d170 2024-05-22T23:03:31.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.582+0000 7f1ae37fe640 1 --2- 172.21.15.84:0/244862509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ae41080f0 0x7f1ae41a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:03:31.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.582+0000 7f1ae3fff640 1 --2- 172.21.15.84:0/244862509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ae419e5f0 0x7f1ae419eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:03:31.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.582+0000 7f1ae2ffd640 1 --2- 172.21.15.84:0/244862509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ae410d170 0x7f1ae41a4980 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1ad802a930 tx=0x7f1ad8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:31.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.582+0000 7f1ae0ff9640 1 -- 172.21.15.84:0/244862509 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad803f070 con 0x7f1ae410d170 2024-05-22T23:03:31.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.582+0000 7f1ae9d14640 1 -- 172.21.15.84:0/244862509 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae419f3a0 con 0x7f1ae410d170 2024-05-22T23:03:31.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.582+0000 7f1ae0ff9640 1 -- 172.21.15.84:0/244862509 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ad8031c90 con 0x7f1ae410d170 2024-05-22T23:03:31.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.582+0000 7f1ae0ff9640 1 -- 172.21.15.84:0/244862509 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8035030 con 0x7f1ae410d170 2024-05-22T23:03:31.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.582+0000 7f1ae9d14640 1 -- 172.21.15.84:0/244862509 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae41ab5f0 con 0x7f1ae410d170 2024-05-22T23:03:31.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.584+0000 7f1ae0ff9640 1 -- 172.21.15.84:0/244862509 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1ad8042070 con 0x7f1ae410d170 2024-05-22T23:03:31.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.585+0000 7f1aeca25740 1 Processor -- start 2024-05-22T23:03:31.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.585+0000 7f1aeca25740 1 -- start start 2024-05-22T23:03:31.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.585+0000 7f1aeca25740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd5bc50 0x55932fd5c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.585+0000 7f1aeca25740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932fd5c570 0x55932fd66700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.585+0000 7f1aeca25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55932fd66c40 0x55932fd69030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.585+0000 7f1aeca25740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55932fcbfa40 con 0x55932fd5bc50 2024-05-22T23:03:31.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.585+0000 7f1aeca25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55932fc5e440 con 0x55932fd66c40 2024-05-22T23:03:31.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.585+0000 7f1aeca25740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55932fcbba00 con 0x55932fd5c570 2024-05-22T23:03:31.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.585+0000 7f1ae3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd5bc50 0x55932fd5c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.585+0000 7f1ae3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd5bc50 0x55932fd5c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43660/0 (socket says 172.21.15.84:43660) 2024-05-22T23:03:31.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.585+0000 7f1ae3fff640 1 -- 172.21.15.84:0/2422118872 learned_addr learned my addr 172.21.15.84:0/2422118872 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:03:31.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.585+0000 7f1ae37fe640 1 --2- 172.21.15.84:0/2422118872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932fd5c570 0x55932fd66700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.585+0000 7f1ae2ffd640 1 --2- 172.21.15.84:0/2422118872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55932fd66c40 0x55932fd69030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.585+0000 7f1ae0ff9640 1 --2- 172.21.15.84:0/244862509 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f1acc0a2d00 0x7f1acc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.585+0000 7f1ae3fff640 1 -- 172.21.15.84:0/2422118872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932fd5c570 msgr2=0x55932fd66700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:31.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.585+0000 7f1ae3fff640 1 --2- 172.21.15.84:0/2422118872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932fd5c570 0x55932fd66700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.585+0000 7f1ae3fff640 1 -- 172.21.15.84:0/2422118872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55932fd66c40 msgr2=0x55932fd69030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:31.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.585+0000 7f1ae3fff640 1 --2- 172.21.15.84:0/2422118872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55932fd66c40 0x55932fd69030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.585+0000 7f1ae3fff640 1 -- 172.21.15.84:0/2422118872 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55932fc682d0 con 0x55932fd5bc50 2024-05-22T23:03:31.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.585+0000 7f1ae37fe640 1 --2- 172.21.15.84:0/2422118872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932fd5c570 0x55932fd66700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:03:31.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.585+0000 7f1ae0ff9640 1 -- 172.21.15.84:0/244862509 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f1ad80ef610 con 0x7f1ae410d170 2024-05-22T23:03:31.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.585+0000 7f1ae3fff640 1 --2- 172.21.15.84:0/244862509 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f1acc0a2d00 0x7f1acc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.585+0000 7f1ae3fff640 1 --2- 172.21.15.84:0/2422118872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd5bc50 0x55932fd5c030 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1adc011910 tx=0x7f1adc011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7d760a39adeba0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:31.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.586+0000 7f1ac9ffb640 1 -- 172.21.15.84:0/2422118872 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc017050 con 0x55932fd5bc50 2024-05-22T23:03:31.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.586+0000 7f1ac9ffb640 1 -- 172.21.15.84:0/2422118872 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1adc00e3b0 con 0x55932fd5bc50 2024-05-22T23:03:31.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.586+0000 7f1ac9ffb640 1 -- 172.21.15.84:0/2422118872 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc01a5c0 con 0x55932fd5bc50 2024-05-22T23:03:31.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.586+0000 7f1aeca25740 1 -- 172.21.15.84:0/2422118872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd5bc50 msgr2=0x55932fd5c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:31.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.586+0000 7f1aeca25740 1 --2- 172.21.15.84:0/2422118872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd5bc50 0x55932fd5c030 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1adc011910 tx=0x7f1adc011d10 comp rx=0 tx=0).stop 2024-05-22T23:03:31.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.586+0000 7f1ae3fff640 1 --2- 172.21.15.84:0/244862509 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f1acc0a2d00 0x7f1acc0a51c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1adc023000 tx=0x7f1adc024000 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:31.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.587+0000 7f1aeca25740 1 -- 172.21.15.84:0/2422118872 shutdown_connections 2024-05-22T23:03:31.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.587+0000 7f1aeca25740 1 --2- 172.21.15.84:0/2422118872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55932fd66c40 0x55932fd69030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.587+0000 7f1aeca25740 1 --2- 172.21.15.84:0/2422118872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932fd5c570 0x55932fd66700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.587+0000 7f1aeca25740 1 --2- 172.21.15.84:0/2422118872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd5bc50 0x55932fd5c030 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.587+0000 7f1aeca25740 1 -- 172.21.15.84:0/2422118872 >> 172.21.15.84:0/2422118872 conn(0x55932fd57f70 msgr2=0x55932fd58350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:31.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.587+0000 7f1aeca25740 1 -- 172.21.15.84:0/2422118872 shutdown_connections 2024-05-22T23:03:31.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.587+0000 7f1aeca25740 1 -- 172.21.15.84:0/2422118872 wait complete. 2024-05-22T23:03:31.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.587+0000 7f1aeca25740 1 -- start start 2024-05-22T23:03:31.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.587+0000 7f1aeca25740 1 Processor -- start 2024-05-22T23:03:31.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.587+0000 7f1aeca25740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932fd5bc50 0x55932fd63c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.587+0000 7f1aeca25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55932fd5c570 0x55932fd64140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.587+0000 7f1aeca25740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd66c40 0x55932fd64680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.587+0000 7f1aeca25740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55932fd69a30 con 0x55932fd66c40 2024-05-22T23:03:31.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.587+0000 7f1aeca25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55932fc495f0 con 0x55932fd5c570 2024-05-22T23:03:31.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.587+0000 7f1aeca25740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55932fd69bb0 con 0x55932fd5bc50 2024-05-22T23:03:31.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.587+0000 7f1aeca25740 10 client.0 _ll_register_callbacks cb 0x55932fc44cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:03:31.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.587+0000 7f1ae3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd66c40 0x55932fd64680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.587+0000 7f1ae3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd66c40 0x55932fd64680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43664/0 (socket says 172.21.15.84:43664) 2024-05-22T23:03:31.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.587+0000 7f1ae3fff640 1 -- 172.21.15.84:0/2396972338 learned_addr learned my addr 172.21.15.84:0/2396972338 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:03:31.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.588+0000 7f1ae2ffd640 1 --2- 172.21.15.84:0/2396972338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55932fd5c570 0x55932fd64140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.588+0000 7f1ae37fe640 1 --2- 172.21.15.84:0/2396972338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932fd5bc50 0x55932fd63c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.588+0000 7f1ae3fff640 1 -- 172.21.15.84:0/2396972338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932fd5bc50 msgr2=0x55932fd63c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:31.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.588+0000 7f1ae3fff640 1 --2- 172.21.15.84:0/2396972338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932fd5bc50 0x55932fd63c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.588+0000 7f1ae3fff640 1 -- 172.21.15.84:0/2396972338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55932fd5c570 msgr2=0x55932fd64140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:31.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.588+0000 7f1ae3fff640 1 --2- 172.21.15.84:0/2396972338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55932fd5c570 0x55932fd64140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.588+0000 7f1ae3fff640 1 -- 172.21.15.84:0/2396972338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55932fa9b7e0 con 0x55932fd66c40 2024-05-22T23:03:31.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.588+0000 7f1ae3fff640 1 --2- 172.21.15.84:0/2396972338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd66c40 0x55932fd64680 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00eb30 tx=0x7f1adc005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5c07d2089c10b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:31.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.588+0000 7f1ac2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:03:31.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.588+0000 7f1ac2ffd640 1 -- 172.21.15.84:0/2396972338 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc02b040 con 0x55932fd66c40 2024-05-22T23:03:31.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.588+0000 7f1ac2ffd640 1 -- 172.21.15.84:0/2396972338 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f1adc003b70 con 0x55932fd66c40 2024-05-22T23:03:31.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.588+0000 7f1aeca25740 1 -- 172.21.15.84:0/2396972338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1adc00e3b0 con 0x55932fd66c40 2024-05-22T23:03:31.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.588+0000 7f1ac2ffd640 1 -- 172.21.15.84:0/2396972338 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f1adc017020 con 0x55932fd66c40 2024-05-22T23:03:31.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.588+0000 7f1aeca25740 10 client.15696 fetch_fsmap learned FSMap version 18 2024-05-22T23:03:31.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.588+0000 7f1aeca25740 1 -- 172.21.15.84:0/2396972338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55932fafbf70 con 0x55932fd66c40 2024-05-22T23:03:31.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.588+0000 7f1ac2ffd640 1 -- 172.21.15.84:0/2396972338 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1adc00e3b0 con 0x55932fd66c40 2024-05-22T23:03:31.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.589+0000 7f1ac2ffd640 1 -- 172.21.15.84:0/2396972338 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f1adc037030 con 0x55932fd66c40 2024-05-22T23:03:31.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.589+0000 7f1aeca25740 10 client.15696 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:03:31.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.589+0000 7f1aeca25740 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:03:31.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.589+0000 7f1aeca25740 20 client.15696 populate_metadata read hostname 'smithi084' 2024-05-22T23:03:31.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.589+0000 7f1aeca25740 1 --2- 172.21.15.84:0/2396972338 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55932fd76de0 0x55932fd791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.589+0000 7f1aeca25740 4 client.15696 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:03:31.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.589+0000 7f1aeca25740 1 -- 172.21.15.84:0/2396972338 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55932fcd3460 con 0x55932fd76de0 2024-05-22T23:03:31.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.589+0000 7f1ae37fe640 1 --2- 172.21.15.84:0/2396972338 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55932fd76de0 0x55932fd791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.589+0000 7f1ae37fe640 1 --2- 172.21.15.84:0/2396972338 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55932fd76de0 0x55932fd791d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a4af618f59b5171 server_cookie=da504f4533caeb04 in_seq=0 out_seq=0 2024-05-22T23:03:31.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.589+0000 7f1ac2ffd640 10 client.15696 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:03:31.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.590+0000 7f1ac2ffd640 1 -- 172.21.15.84:0/2396972338 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55932fcd3460 con 0x55932fd76de0 2024-05-22T23:03:31.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.590+0000 7f1ac2ffd640 10 client.15696 handle_command_reply: tid=0 2024-05-22T23:03:31.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.590+0000 7f1aeca25740 1 client.15696 shutdown 2024-05-22T23:03:31.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.590+0000 7f1aeca25740 1 -- 172.21.15.84:0/2396972338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd66c40 msgr2=0x55932fd64680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:31.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.590+0000 7f1aeca25740 1 --2- 172.21.15.84:0/2396972338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd66c40 0x55932fd64680 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00eb30 tx=0x7f1adc005f70 comp rx=0 tx=0).stop 2024-05-22T23:03:31.602 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.590+0000 7f1aeca25740 1 -- 172.21.15.84:0/2396972338 shutdown_connections 2024-05-22T23:03:31.602 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.590+0000 7f1aeca25740 1 --2- 172.21.15.84:0/2396972338 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55932fd76de0 0x55932fd791d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.602 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.590+0000 7f1aeca25740 1 --2- 172.21.15.84:0/2396972338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd66c40 0x55932fd64680 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.603 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.590+0000 7f1aeca25740 1 --2- 172.21.15.84:0/2396972338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55932fd5c570 0x55932fd64140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.603 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.590+0000 7f1ac2ffd640 0 client.15696 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:03:31.603 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.590+0000 7f1aeca25740 1 --2- 172.21.15.84:0/2396972338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932fd5bc50 0x55932fd63c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.603 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.590+0000 7f1aeca25740 1 -- 172.21.15.84:0/2396972338 >> 172.21.15.84:0/2396972338 conn(0x55932fd57f70 msgr2=0x55932fd5dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:31.603 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.590+0000 7f1aeca25740 1 -- 172.21.15.84:0/2396972338 shutdown_connections 2024-05-22T23:03:31.603 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.591+0000 7f1aeca25740 1 -- 172.21.15.84:0/2396972338 wait complete. 2024-05-22T23:03:31.603 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.591+0000 7f1aeca25740 20 client.15696 trim_cache size 0 max 16384 2024-05-22T23:03:31.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.609+0000 7f1aeca25740 1 Processor -- start 2024-05-22T23:03:31.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.609+0000 7f1aeca25740 1 -- start start 2024-05-22T23:03:31.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.609+0000 7f1aeca25740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932fd5bc50 0x55932fd728f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.609+0000 7f1aeca25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55932fd5c570 0x55932fd644a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.609+0000 7f1aeca25740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd66c40 0x55932fd7e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.609+0000 7f1aeca25740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae41b1c30 con 0x55932fd66c40 2024-05-22T23:03:31.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.609+0000 7f1aeca25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae41b4490 con 0x55932fd5c570 2024-05-22T23:03:31.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.609+0000 7f1aeca25740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae41b2280 con 0x55932fd5bc50 2024-05-22T23:03:31.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.609+0000 7f1ae3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd66c40 0x55932fd7e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.609+0000 7f1ae3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd66c40 0x55932fd7e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43672/0 (socket says 172.21.15.84:43672) 2024-05-22T23:03:31.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.609+0000 7f1ae3fff640 1 -- 172.21.15.84:0/2605842897 learned_addr learned my addr 172.21.15.84:0/2605842897 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:03:31.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.609+0000 7f1ae2ffd640 1 --2- 172.21.15.84:0/2605842897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55932fd5c570 0x55932fd644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.609+0000 7f1ae37fe640 1 --2- 172.21.15.84:0/2605842897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932fd5bc50 0x55932fd728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.609+0000 7f1ae3fff640 1 -- 172.21.15.84:0/2605842897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932fd5bc50 msgr2=0x55932fd728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:31.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.609+0000 7f1ae3fff640 1 --2- 172.21.15.84:0/2605842897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932fd5bc50 0x55932fd728f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.609+0000 7f1ae3fff640 1 -- 172.21.15.84:0/2605842897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55932fd5c570 msgr2=0x55932fd644a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:31.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.609+0000 7f1ae3fff640 1 --2- 172.21.15.84:0/2605842897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55932fd5c570 0x55932fd644a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.609+0000 7f1ae3fff640 1 -- 172.21.15.84:0/2605842897 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55932f993170 con 0x55932fd66c40 2024-05-22T23:03:31.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.610+0000 7f1ae3fff640 1 --2- 172.21.15.84:0/2605842897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd66c40 0x55932fd7e9b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1adc005ba0 tx=0x7f1adc009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee907490e17cda4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:31.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.610+0000 7f1ac97fa640 1 -- 172.21.15.84:0/2605842897 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc045070 con 0x55932fd66c40 2024-05-22T23:03:31.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.610+0000 7f1ac97fa640 1 -- 172.21.15.84:0/2605842897 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1adc00e3b0 con 0x55932fd66c40 2024-05-22T23:03:31.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.610+0000 7f1ac97fa640 1 -- 172.21.15.84:0/2605842897 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc02b070 con 0x55932fd66c40 2024-05-22T23:03:31.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.610+0000 7f1aeca25740 1 -- 172.21.15.84:0/2605842897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd66c40 msgr2=0x55932fd7e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:31.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.610+0000 7f1aeca25740 1 --2- 172.21.15.84:0/2605842897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd66c40 0x55932fd7e9b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1adc005ba0 tx=0x7f1adc009690 comp rx=0 tx=0).stop 2024-05-22T23:03:31.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.611+0000 7f1aeca25740 1 -- 172.21.15.84:0/2605842897 shutdown_connections 2024-05-22T23:03:31.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.611+0000 7f1aeca25740 1 --2- 172.21.15.84:0/2605842897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd66c40 0x55932fd7e9b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.611+0000 7f1aeca25740 1 --2- 172.21.15.84:0/2605842897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55932fd5c570 0x55932fd644a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.611+0000 7f1aeca25740 1 --2- 172.21.15.84:0/2605842897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932fd5bc50 0x55932fd728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.611+0000 7f1aeca25740 1 -- 172.21.15.84:0/2605842897 >> 172.21.15.84:0/2605842897 conn(0x55932fd57f70 msgr2=0x55932fd5dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:31.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.611+0000 7f1aeca25740 1 -- 172.21.15.84:0/2605842897 shutdown_connections 2024-05-22T23:03:31.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.611+0000 7f1aeca25740 1 -- 172.21.15.84:0/2605842897 wait complete. 2024-05-22T23:03:31.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.611+0000 7f1aeca25740 1 -- start start 2024-05-22T23:03:31.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.611+0000 7f1aeca25740 1 Processor -- start 2024-05-22T23:03:31.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.611+0000 7f1aeca25740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932fd5bc50 0x55932fd6eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.611+0000 7f1aeca25740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd5c570 0x55932fd6a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.611+0000 7f1aeca25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55932fd66c40 0x55932fd6a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.611+0000 7f1aeca25740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae41b2950 con 0x55932fd5c570 2024-05-22T23:03:31.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.611+0000 7f1aeca25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae41b3dc0 con 0x55932fd66c40 2024-05-22T23:03:31.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.611+0000 7f1aeca25740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae41b3020 con 0x55932fd5bc50 2024-05-22T23:03:31.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.611+0000 7f1aeca25740 10 client.0 _ll_register_callbacks cb 0x7f1ad80b7e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:03:31.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.611+0000 7f1ae2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd5c570 0x55932fd6a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.611+0000 7f1ae3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55932fd66c40 0x55932fd6a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.611+0000 7f1ae2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd5c570 0x55932fd6a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43676/0 (socket says 172.21.15.84:43676) 2024-05-22T23:03:31.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.612+0000 7f1ae37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932fd5bc50 0x55932fd6eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.612+0000 7f1ae2ffd640 1 -- 172.21.15.84:0/3147492137 learned_addr learned my addr 172.21.15.84:0/3147492137 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:03:31.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.612+0000 7f1ae2ffd640 1 -- 172.21.15.84:0/3147492137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932fd5bc50 msgr2=0x55932fd6eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:31.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.612+0000 7f1ae2ffd640 1 --2- 172.21.15.84:0/3147492137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932fd5bc50 0x55932fd6eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.612+0000 7f1ae2ffd640 1 -- 172.21.15.84:0/3147492137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55932fd66c40 msgr2=0x55932fd6a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:31.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.612+0000 7f1ae2ffd640 1 --2- 172.21.15.84:0/3147492137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55932fd66c40 0x55932fd6a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.612+0000 7f1ae2ffd640 1 -- 172.21.15.84:0/3147492137 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55932fc88930 con 0x55932fd5c570 2024-05-22T23:03:31.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.612+0000 7f1ae2ffd640 1 --2- 172.21.15.84:0/3147492137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd5c570 0x55932fd6a3c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1ad803b7d0 tx=0x7f1ad80046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940a9789b69dcf4a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:31.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.612+0000 7f1ac2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:03:31.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.612+0000 7f1ac2ffd640 1 -- 172.21.15.84:0/3147492137 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad80bce60 con 0x55932fd5c570 2024-05-22T23:03:31.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.612+0000 7f1ac2ffd640 1 -- 172.21.15.84:0/3147492137 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f1ad80b81a0 con 0x55932fd5c570 2024-05-22T23:03:31.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.612+0000 7f1aeca25740 1 -- 172.21.15.84:0/3147492137 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1adc00e3b0 con 0x55932fd5c570 2024-05-22T23:03:31.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.612+0000 7f1ac2ffd640 1 -- 172.21.15.84:0/3147492137 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f1ad8049040 con 0x55932fd5c570 2024-05-22T23:03:31.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.612+0000 7f1aeca25740 10 client.15702 fetch_fsmap learned FSMap version 18 2024-05-22T23:03:31.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.612+0000 7f1aeca25740 1 -- 172.21.15.84:0/3147492137 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55932fddcda0 con 0x55932fd5c570 2024-05-22T23:03:31.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.612+0000 7f1ac2ffd640 1 -- 172.21.15.84:0/3147492137 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1adc00e3b0 con 0x55932fd5c570 2024-05-22T23:03:31.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.613+0000 7f1ac2ffd640 1 -- 172.21.15.84:0/3147492137 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f1ad80e7040 con 0x55932fd5c570 2024-05-22T23:03:31.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.613+0000 7f1aeca25740 10 client.15702 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:03:31.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.613+0000 7f1aeca25740 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:03:31.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.613+0000 7f1aeca25740 20 client.15702 populate_metadata read hostname 'smithi084' 2024-05-22T23:03:31.624 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.613+0000 7f1aeca25740 1 --2- 172.21.15.84:0/3147492137 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55932fddd520 0x55932fddf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.624 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.613+0000 7f1aeca25740 4 client.15702 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-22T23:03:31.624 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.613+0000 7f1aeca25740 1 -- 172.21.15.84:0/3147492137 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55932fccd170 con 0x55932fddd520 2024-05-22T23:03:31.624 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.613+0000 7f1ae37fe640 1 --2- 172.21.15.84:0/3147492137 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55932fddd520 0x55932fddf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.624 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.613+0000 7f1ae37fe640 1 --2- 172.21.15.84:0/3147492137 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55932fddd520 0x55932fddf940 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfc4470aff1a8fc9 server_cookie=690453eb92347f3 in_seq=0 out_seq=0 2024-05-22T23:03:31.624 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.613+0000 7f1ac2ffd640 10 client.15702 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:03:31.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.615+0000 7f1ac2ffd640 1 -- 172.21.15.84:0/3147492137 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55932fccd170 con 0x55932fddd520 2024-05-22T23:03:31.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.615+0000 7f1ac2ffd640 10 client.15702 handle_command_reply: tid=0 2024-05-22T23:03:31.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.615+0000 7f1aeca25740 1 client.15702 shutdown 2024-05-22T23:03:31.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.615+0000 7f1aeca25740 1 -- 172.21.15.84:0/3147492137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd5c570 msgr2=0x55932fd6a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:31.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.615+0000 7f1aeca25740 1 --2- 172.21.15.84:0/3147492137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd5c570 0x55932fd6a3c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1ad803b7d0 tx=0x7f1ad80046e0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.616+0000 7f1aeca25740 1 -- 172.21.15.84:0/3147492137 shutdown_connections 2024-05-22T23:03:31.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.616+0000 7f1aeca25740 1 --2- 172.21.15.84:0/3147492137 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55932fddd520 0x55932fddf940 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.616+0000 7f1aeca25740 1 --2- 172.21.15.84:0/3147492137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55932fd66c40 0x55932fd6a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.616+0000 7f1aeca25740 1 --2- 172.21.15.84:0/3147492137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55932fd5c570 0x55932fd6a3c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.616+0000 7f1ac2ffd640 0 client.15702 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:03:31.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.616+0000 7f1aeca25740 1 --2- 172.21.15.84:0/3147492137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932fd5bc50 0x55932fd6eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.616+0000 7f1aeca25740 1 -- 172.21.15.84:0/3147492137 >> 172.21.15.84:0/3147492137 conn(0x55932fd57f70 msgr2=0x55932fd5d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:31.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.616+0000 7f1aeca25740 1 -- 172.21.15.84:0/3147492137 shutdown_connections 2024-05-22T23:03:31.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.616+0000 7f1aeca25740 1 -- 172.21.15.84:0/3147492137 wait complete. 2024-05-22T23:03:31.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.616+0000 7f1aeca25740 20 client.15702 trim_cache size 0 max 16384 2024-05-22T23:03:31.627 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:03:31.627 INFO:teuthology.orchestra.run.smithi084.stdout: "return_code": 0, 2024-05-22T23:03:31.627 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_tag": "49319a0e-ca52-4466-88b8-527e003e4766", 2024-05-22T23:03:31.627 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": "asynchronous" 2024-05-22T23:03:31.627 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:03:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.616+0000 7f1ae9d14640 1 -- 172.21.15.84:0/244862509 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f1acc0a2d00 msgr2=0x7f1acc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.616+0000 7f1ae9d14640 1 --2- 172.21.15.84:0/244862509 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f1acc0a2d00 0x7f1acc0a51c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1adc023000 tx=0x7f1adc024000 comp rx=0 tx=0).stop 2024-05-22T23:03:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.616+0000 7f1ae9d14640 1 -- 172.21.15.84:0/244862509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ae410d170 msgr2=0x7f1ae41a4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.616+0000 7f1ae9d14640 1 --2- 172.21.15.84:0/244862509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ae410d170 0x7f1ae41a4980 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1ad802a930 tx=0x7f1ad8003b90 comp rx=0 tx=0).stop 2024-05-22T23:03:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.616+0000 7f1ae9d14640 1 -- 172.21.15.84:0/244862509 shutdown_connections 2024-05-22T23:03:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.616+0000 7f1ae9d14640 1 --2- 172.21.15.84:0/244862509 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f1acc0a2d00 0x7f1acc0a51c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.616+0000 7f1ae9d14640 1 --2- 172.21.15.84:0/244862509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ae419e5f0 0x7f1ae419eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.616+0000 7f1ae9d14640 1 --2- 172.21.15.84:0/244862509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ae410d170 0x7f1ae41a4980 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.616+0000 7f1ae9d14640 1 --2- 172.21.15.84:0/244862509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ae41080f0 0x7f1ae41a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.616+0000 7f1ae9d14640 1 -- 172.21.15.84:0/244862509 >> 172.21.15.84:0/244862509 conn(0x7f1ae4075db0 msgr2=0x7f1ae40780f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.616+0000 7f1ae9d14640 1 -- 172.21.15.84:0/244862509 shutdown_connections 2024-05-22T23:03:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.616+0000 7f1ae9d14640 1 -- 172.21.15.84:0/244862509 wait complete. 2024-05-22T23:03:31.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:03:31.632 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:03:31.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.737+0000 7f8c9d038640 1 -- 172.21.15.84:0/3547076022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c98102b00 msgr2=0x7f8c98102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:31.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.737+0000 7f8c9d038640 1 --2- 172.21.15.84:0/3547076022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c98102b00 0x7f8c98102ee0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8c88009900 tx=0x7f8c8802a510 comp rx=0 tx=0).stop 2024-05-22T23:03:31.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.737+0000 7f8c9d038640 1 -- 172.21.15.84:0/3547076022 shutdown_connections 2024-05-22T23:03:31.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.737+0000 7f8c9d038640 1 --2- 172.21.15.84:0/3547076022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c9810be40 0x7f8c9810e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.737+0000 7f8c9d038640 1 --2- 172.21.15.84:0/3547076022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c98103420 0x7f8c9810b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.737+0000 7f8c9d038640 1 --2- 172.21.15.84:0/3547076022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c98102b00 0x7f8c98102ee0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.737+0000 7f8c9d038640 1 -- 172.21.15.84:0/3547076022 >> 172.21.15.84:0/3547076022 conn(0x7f8c98076af0 msgr2=0x7f8c98076f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:31.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.737+0000 7f8c9d038640 1 -- 172.21.15.84:0/3547076022 shutdown_connections 2024-05-22T23:03:31.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.737+0000 7f8c9d038640 1 -- 172.21.15.84:0/3547076022 wait complete. 2024-05-22T23:03:31.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.737+0000 7f8c9d038640 1 Processor -- start 2024-05-22T23:03:31.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.738+0000 7f8c9d038640 1 -- start start 2024-05-22T23:03:31.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.738+0000 7f8c9d038640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c98102b00 0x7f8c98198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.738+0000 7f8c9d038640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c98103420 0x7f8c981986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.738+0000 7f8c9d038640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c9810be40 0x7f8c9819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.738+0000 7f8c9d038640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8c98111c40 con 0x7f8c98103420 2024-05-22T23:03:31.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.738+0000 7f8c9d038640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c98111ac0 con 0x7f8c9810be40 2024-05-22T23:03:31.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.738+0000 7f8c9d038640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8c98111dc0 con 0x7f8c98102b00 2024-05-22T23:03:31.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.738+0000 7f8c9659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c98103420 0x7f8c981986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.738+0000 7f8c96d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c98102b00 0x7f8c98198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.738+0000 7f8c9659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c98103420 0x7f8c981986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43698/0 (socket says 172.21.15.84:43698) 2024-05-22T23:03:31.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.738+0000 7f8c9659c640 1 -- 172.21.15.84:0/1975587966 learned_addr learned my addr 172.21.15.84:0/1975587966 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:03:31.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.738+0000 7f8c9759e640 1 --2- 172.21.15.84:0/1975587966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c9810be40 0x7f8c9819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.738+0000 7f8c96d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c98102b00 0x7f8c98198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:48966/0 (socket says 172.21.15.84:48966) 2024-05-22T23:03:31.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.738+0000 7f8c9659c640 1 -- 172.21.15.84:0/1975587966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c98102b00 msgr2=0x7f8c98198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:31.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.738+0000 7f8c9659c640 1 --2- 172.21.15.84:0/1975587966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c98102b00 0x7f8c98198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.738+0000 7f8c9659c640 1 -- 172.21.15.84:0/1975587966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c9810be40 msgr2=0x7f8c9819ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:31.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.738+0000 7f8c9659c640 1 --2- 172.21.15.84:0/1975587966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c9810be40 0x7f8c9819ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.738+0000 7f8c9659c640 1 -- 172.21.15.84:0/1975587966 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c9819d170 con 0x7f8c98103420 2024-05-22T23:03:31.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.738+0000 7f8c9659c640 1 --2- 172.21.15.84:0/1975587966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c98103420 0x7f8c981986c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00e9f0 tx=0x7f8c8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:31.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.739+0000 7f8c7ffff640 1 -- 172.21.15.84:0/1975587966 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c00cd80 con 0x7f8c98103420 2024-05-22T23:03:31.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.739+0000 7f8c9d038640 1 -- 172.21.15.84:0/1975587966 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c9819d460 con 0x7f8c98103420 2024-05-22T23:03:31.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.739+0000 7f8c7ffff640 1 -- 172.21.15.84:0/1975587966 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c8c010910 con 0x7f8c98103420 2024-05-22T23:03:31.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.739+0000 7f8c7ffff640 1 -- 172.21.15.84:0/1975587966 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c0188f0 con 0x7f8c98103420 2024-05-22T23:03:31.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.739+0000 7f8c9d038640 1 -- 172.21.15.84:0/1975587966 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c981a4cf0 con 0x7f8c98103420 2024-05-22T23:03:31.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.741+0000 7f8c7ffff640 1 -- 172.21.15.84:0/1975587966 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8c8c010ac0 con 0x7f8c98103420 2024-05-22T23:03:31.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.741+0000 7f8c9fe25740 1 Processor -- start 2024-05-22T23:03:31.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.741+0000 7f8c9fe25740 1 -- start start 2024-05-22T23:03:31.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.741+0000 7f8c9fe25740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b0920f4a0 0x562b0920d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.741+0000 7f8c7ffff640 1 --2- 172.21.15.84:0/1975587966 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f8c800a2d00 0x7f8c800a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.741+0000 7f8c9fe25740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920ddb0 0x562b0920e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.742+0000 7f8c9fe25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b09218b30 0x562b0921af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.742+0000 7f8c9fe25740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562b09171db0 con 0x562b0920ddb0 2024-05-22T23:03:31.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.742+0000 7f8c9fe25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562b09110360 con 0x562b09218b30 2024-05-22T23:03:31.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.742+0000 7f8c9fe25740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562b0916dd70 con 0x562b0920f4a0 2024-05-22T23:03:31.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.742+0000 7f8c7ffff640 1 -- 172.21.15.84:0/1975587966 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f8c8c0cf830 con 0x7f8c98103420 2024-05-22T23:03:31.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.743+0000 7f8c9759e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b0920f4a0 0x562b0920d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.743+0000 7f8c96d9d640 1 --2- 172.21.15.84:0/1975587966 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f8c800a2d00 0x7f8c800a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.743+0000 7f8c9659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920ddb0 0x562b0920e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.743+0000 7f8c9659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920ddb0 0x562b0920e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43700/0 (socket says 172.21.15.84:43700) 2024-05-22T23:03:31.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.743+0000 7f8c9659c640 1 -- 172.21.15.84:0/3053747872 learned_addr learned my addr 172.21.15.84:0/3053747872 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:03:31.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.744+0000 7f8c9659c640 1 -- 172.21.15.84:0/3053747872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b0920f4a0 msgr2=0x562b0920d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:31.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.744+0000 7f8c9659c640 1 --2- 172.21.15.84:0/3053747872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b0920f4a0 0x562b0920d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.744+0000 7f8c9659c640 1 -- 172.21.15.84:0/3053747872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b09218b30 msgr2=0x562b0921af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:31.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.744+0000 7f8c96d9d640 1 --2- 172.21.15.84:0/1975587966 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f8c800a2d00 0x7f8c800a51c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8c88009230 tx=0x7f8c880058d0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:31.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.744+0000 7f8c9759e640 1 --2- 172.21.15.84:0/3053747872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b09218b30 0x562b0921af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.744+0000 7f8c9659c640 1 --2- 172.21.15.84:0/3053747872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b09218b30 0x562b0921af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.745+0000 7f8c9659c640 1 -- 172.21.15.84:0/3053747872 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b09133a60 con 0x562b0920ddb0 2024-05-22T23:03:31.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.745+0000 7f8c9759e640 1 --2- 172.21.15.84:0/3053747872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b09218b30 0x562b0921af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:03:31.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.745+0000 7f8c9659c640 1 --2- 172.21.15.84:0/3053747872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920ddb0 0x562b0920e230 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c0987d0 tx=0x7f8c8c09bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9610abbfe8b74af9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:31.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.746+0000 7f8c7d7fa640 1 -- 172.21.15.84:0/3053747872 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c0cd200 con 0x562b0920ddb0 2024-05-22T23:03:31.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.746+0000 7f8c7d7fa640 1 -- 172.21.15.84:0/3053747872 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c8c098470 con 0x562b0920ddb0 2024-05-22T23:03:31.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.746+0000 7f8c7d7fa640 1 -- 172.21.15.84:0/3053747872 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c097db0 con 0x562b0920ddb0 2024-05-22T23:03:31.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.747+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3053747872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920ddb0 msgr2=0x562b0920e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:31.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.747+0000 7f8c9fe25740 1 --2- 172.21.15.84:0/3053747872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920ddb0 0x562b0920e230 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c0987d0 tx=0x7f8c8c09bd00 comp rx=0 tx=0).stop 2024-05-22T23:03:31.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.747+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3053747872 shutdown_connections 2024-05-22T23:03:31.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.747+0000 7f8c9fe25740 1 --2- 172.21.15.84:0/3053747872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b09218b30 0x562b0921af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.747+0000 7f8c9fe25740 1 --2- 172.21.15.84:0/3053747872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920ddb0 0x562b0920e230 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.747+0000 7f8c9fe25740 1 --2- 172.21.15.84:0/3053747872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b0920f4a0 0x562b0920d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.747+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3053747872 >> 172.21.15.84:0/3053747872 conn(0x562b091d68e0 msgr2=0x562b0920a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:31.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.748+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3053747872 shutdown_connections 2024-05-22T23:03:31.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.748+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3053747872 wait complete. 2024-05-22T23:03:31.757 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.750+0000 7f8c9fe25740 1 -- start start 2024-05-22T23:03:31.757 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.750+0000 7f8c9fe25740 1 Processor -- start 2024-05-22T23:03:31.757 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.750+0000 7f8c9fe25740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920ddb0 0x562b09212000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.757 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.751+0000 7f8c9fe25740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b0920f4a0 0x562b09212540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.751+0000 7f8c9fe25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b09218b30 0x562b09216620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.751+0000 7f8c9fe25740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562b0921b880 con 0x562b0920ddb0 2024-05-22T23:03:31.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.751+0000 7f8c9fe25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562b0912c0f0 con 0x562b09218b30 2024-05-22T23:03:31.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.751+0000 7f8c96d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920ddb0 0x562b09212000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.751+0000 7f8c9fe25740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562b0921ba00 con 0x562b0920f4a0 2024-05-22T23:03:31.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.751+0000 7f8c96d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920ddb0 0x562b09212000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43716/0 (socket says 172.21.15.84:43716) 2024-05-22T23:03:31.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.751+0000 7f8c96d9d640 1 -- 172.21.15.84:0/3559869089 learned_addr learned my addr 172.21.15.84:0/3559869089 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:03:31.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.751+0000 7f8c9fe25740 10 client.0 _ll_register_callbacks cb 0x562b09130f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:03:31.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.751+0000 7f8c9659c640 1 --2- 172.21.15.84:0/3559869089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b0920f4a0 0x562b09212540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.752+0000 7f8c96d9d640 1 -- 172.21.15.84:0/3559869089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b0920f4a0 msgr2=0x562b09212540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:31.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.752+0000 7f8c96d9d640 1 --2- 172.21.15.84:0/3559869089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b0920f4a0 0x562b09212540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.752+0000 7f8c96d9d640 1 -- 172.21.15.84:0/3559869089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b09218b30 msgr2=0x562b09216620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:31.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.752+0000 7f8c96d9d640 1 --2- 172.21.15.84:0/3559869089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b09218b30 0x562b09216620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.752+0000 7f8c96d9d640 1 -- 172.21.15.84:0/3559869089 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b08f4f7d0 con 0x562b0920ddb0 2024-05-22T23:03:31.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.752+0000 7f8c96d9d640 1 --2- 172.21.15.84:0/3559869089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920ddb0 0x562b09212000 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8c88009900 tx=0x7f8c88009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5300afc158242e2c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:31.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.752+0000 7f8c7aff5640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:03:31.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.753+0000 7f8c7aff5640 1 -- 172.21.15.84:0/3559869089 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8804d020 con 0x562b0920ddb0 2024-05-22T23:03:31.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.753+0000 7f8c7aff5640 1 -- 172.21.15.84:0/3559869089 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f8c880037f0 con 0x562b0920ddb0 2024-05-22T23:03:31.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.754+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3559869089 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c8c098470 con 0x562b0920ddb0 2024-05-22T23:03:31.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.754+0000 7f8c7aff5640 1 -- 172.21.15.84:0/3559869089 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f8c88035040 con 0x562b0920ddb0 2024-05-22T23:03:31.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.754+0000 7f8c9fe25740 10 client.15714 fetch_fsmap learned FSMap version 18 2024-05-22T23:03:31.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.754+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3559869089 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x562b08e853f0 con 0x562b0920ddb0 2024-05-22T23:03:31.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.755+0000 7f8c7aff5640 1 -- 172.21.15.84:0/3559869089 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c8c098470 con 0x562b0920ddb0 2024-05-22T23:03:31.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.755+0000 7f8c7aff5640 1 -- 172.21.15.84:0/3559869089 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f8c8805a020 con 0x562b0920ddb0 2024-05-22T23:03:31.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.755+0000 7f8c9fe25740 10 client.15714 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:03:31.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.755+0000 7f8c9fe25740 10 client.15714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:03:31.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.755+0000 7f8c9fe25740 20 client.15714 populate_metadata read hostname 'smithi084' 2024-05-22T23:03:31.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.755+0000 7f8c9fe25740 1 --2- 172.21.15.84:0/3559869089 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x562b09229030 0x562b0922b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.755+0000 7f8c9fe25740 4 client.15714 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:03:31.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.755+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3559869089 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b0917f4e0 con 0x562b09229030 2024-05-22T23:03:31.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.757+0000 7f8c9659c640 1 --2- 172.21.15.84:0/3559869089 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x562b09229030 0x562b0922b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.758+0000 7f8c9659c640 1 --2- 172.21.15.84:0/3559869089 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x562b09229030 0x562b0922b4c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73558468197b3c9c server_cookie=e3cfa56f1bb15ae6 in_seq=0 out_seq=0 2024-05-22T23:03:31.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.758+0000 7f8c7aff5640 10 client.15714 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:03:31.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.761+0000 7f8c7aff5640 1 -- 172.21.15.84:0/3559869089 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562b0917f4e0 con 0x562b09229030 2024-05-22T23:03:31.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.761+0000 7f8c7aff5640 10 client.15714 handle_command_reply: tid=0 2024-05-22T23:03:31.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.761+0000 7f8c9fe25740 1 client.15714 shutdown 2024-05-22T23:03:31.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.762+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3559869089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920ddb0 msgr2=0x562b09212000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:31.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.762+0000 7f8c9fe25740 1 --2- 172.21.15.84:0/3559869089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920ddb0 0x562b09212000 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8c88009900 tx=0x7f8c88009260 comp rx=0 tx=0).stop 2024-05-22T23:03:31.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.762+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3559869089 shutdown_connections 2024-05-22T23:03:31.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.762+0000 7f8c9fe25740 1 --2- 172.21.15.84:0/3559869089 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x562b09229030 0x562b0922b4c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.762+0000 7f8c9fe25740 1 --2- 172.21.15.84:0/3559869089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b09218b30 0x562b09216620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.762+0000 7f8c9fe25740 1 --2- 172.21.15.84:0/3559869089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b0920f4a0 0x562b09212540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.762+0000 7f8c9fe25740 1 --2- 172.21.15.84:0/3559869089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920ddb0 0x562b09212000 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.762+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3559869089 >> 172.21.15.84:0/3559869089 conn(0x562b091d68e0 msgr2=0x562b09210180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:31.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.762+0000 7f8c7aff5640 0 client.15714 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:03:31.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.762+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3559869089 shutdown_connections 2024-05-22T23:03:31.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.762+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3559869089 wait complete. 2024-05-22T23:03:31.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.762+0000 7f8c9fe25740 20 client.15714 trim_cache size 0 max 16384 2024-05-22T23:03:31.791 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.782+0000 7f8c9fe25740 1 Processor -- start 2024-05-22T23:03:31.791 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.782+0000 7f8c9fe25740 1 -- start start 2024-05-22T23:03:31.791 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.782+0000 7f8c9fe25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b0920ddb0 0x562b09216610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.791 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.782+0000 7f8c9fe25740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920f4a0 0x562b0920a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.791 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.782+0000 7f8c9fe25740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b09218b30 0x562b09230d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.791 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.782+0000 7f8c9fe25740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8c981a9590 con 0x562b0920f4a0 2024-05-22T23:03:31.791 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.782+0000 7f8c9fe25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c981abe10 con 0x562b0920ddb0 2024-05-22T23:03:31.791 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.782+0000 7f8c9fe25740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8c981a9c00 con 0x562b09218b30 2024-05-22T23:03:31.791 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.782+0000 7f8c9659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920f4a0 0x562b0920a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.782+0000 7f8c9659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920f4a0 0x562b0920a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43724/0 (socket says 172.21.15.84:43724) 2024-05-22T23:03:31.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.782+0000 7f8c9659c640 1 -- 172.21.15.84:0/3600787108 learned_addr learned my addr 172.21.15.84:0/3600787108 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:03:31.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.782+0000 7f8c96d9d640 1 --2- 172.21.15.84:0/3600787108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b0920ddb0 0x562b09216610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.782+0000 7f8c9759e640 1 --2- 172.21.15.84:0/3600787108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b09218b30 0x562b09230d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.782+0000 7f8c9659c640 1 -- 172.21.15.84:0/3600787108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b09218b30 msgr2=0x562b09230d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:31.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.782+0000 7f8c9659c640 1 --2- 172.21.15.84:0/3600787108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b09218b30 0x562b09230d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.782+0000 7f8c9659c640 1 -- 172.21.15.84:0/3600787108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b0920ddb0 msgr2=0x562b09216610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:31.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.782+0000 7f8c9659c640 1 --2- 172.21.15.84:0/3600787108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b0920ddb0 0x562b09216610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.782+0000 7f8c9659c640 1 -- 172.21.15.84:0/3600787108 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b08e450f0 con 0x562b0920f4a0 2024-05-22T23:03:31.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.782+0000 7f8c9759e640 1 --2- 172.21.15.84:0/3600787108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b09218b30 0x562b09230d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:03:31.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.782+0000 7f8c96d9d640 1 --2- 172.21.15.84:0/3600787108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b0920ddb0 0x562b09216610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:03:31.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.783+0000 7f8c9659c640 1 --2- 172.21.15.84:0/3600787108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920f4a0 0x562b0920a4c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c09bd90 tx=0x7f8c8c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4752064d16f08903 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:31.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.783+0000 7f8c7cff9640 1 -- 172.21.15.84:0/3600787108 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c09b250 con 0x562b0920f4a0 2024-05-22T23:03:31.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.783+0000 7f8c7cff9640 1 -- 172.21.15.84:0/3600787108 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c8c09b3f0 con 0x562b0920f4a0 2024-05-22T23:03:31.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.783+0000 7f8c7cff9640 1 -- 172.21.15.84:0/3600787108 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c018a90 con 0x562b0920f4a0 2024-05-22T23:03:31.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.783+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3600787108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920f4a0 msgr2=0x562b0920a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:31.794 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.783+0000 7f8c9fe25740 1 --2- 172.21.15.84:0/3600787108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920f4a0 0x562b0920a4c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c09bd90 tx=0x7f8c8c009310 comp rx=0 tx=0).stop 2024-05-22T23:03:31.794 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.783+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3600787108 shutdown_connections 2024-05-22T23:03:31.794 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.783+0000 7f8c9fe25740 1 --2- 172.21.15.84:0/3600787108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b09218b30 0x562b09230d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.794 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.783+0000 7f8c9fe25740 1 --2- 172.21.15.84:0/3600787108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920f4a0 0x562b0920a4c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.794 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.783+0000 7f8c9fe25740 1 --2- 172.21.15.84:0/3600787108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b0920ddb0 0x562b09216610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.794 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.783+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3600787108 >> 172.21.15.84:0/3600787108 conn(0x562b091d68e0 msgr2=0x562b09209db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:31.794 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.784+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3600787108 shutdown_connections 2024-05-22T23:03:31.794 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.784+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3600787108 wait complete. 2024-05-22T23:03:31.794 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.784+0000 7f8c9fe25740 1 -- start start 2024-05-22T23:03:31.795 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.784+0000 7f8c9fe25740 1 Processor -- start 2024-05-22T23:03:31.795 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.784+0000 7f8c9fe25740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b0920ddb0 0x562b09220210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.795 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.784+0000 7f8c9fe25740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920f4a0 0x562b09220750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.795 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.784+0000 7f8c9fe25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b09218b30 0x562b09220c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.795 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.784+0000 7f8c9fe25740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8c981aa2d0 con 0x562b0920f4a0 2024-05-22T23:03:31.795 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.784+0000 7f8c9fe25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c981ab740 con 0x562b09218b30 2024-05-22T23:03:31.795 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.784+0000 7f8c9fe25740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8c981aa9a0 con 0x562b0920ddb0 2024-05-22T23:03:31.795 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.784+0000 7f8c9fe25740 10 client.0 _ll_register_callbacks cb 0x7f8c88040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:03:31.795 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.784+0000 7f8c9659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920f4a0 0x562b09220750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.795 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.784+0000 7f8c96d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b0920ddb0 0x562b09220210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.784+0000 7f8c9659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920f4a0 0x562b09220750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43734/0 (socket says 172.21.15.84:43734) 2024-05-22T23:03:31.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.784+0000 7f8c9659c640 1 -- 172.21.15.84:0/3498779023 learned_addr learned my addr 172.21.15.84:0/3498779023 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:03:31.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.784+0000 7f8c9759e640 1 --2- 172.21.15.84:0/3498779023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b09218b30 0x562b09220c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.785+0000 7f8c9659c640 1 -- 172.21.15.84:0/3498779023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b0920ddb0 msgr2=0x562b09220210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:31.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.785+0000 7f8c9659c640 1 --2- 172.21.15.84:0/3498779023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b0920ddb0 0x562b09220210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.785+0000 7f8c9659c640 1 -- 172.21.15.84:0/3498779023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b09218b30 msgr2=0x562b09220c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:03:31.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.785+0000 7f8c9659c640 1 --2- 172.21.15.84:0/3498779023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b09218b30 0x562b09220c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.785+0000 7f8c9659c640 1 -- 172.21.15.84:0/3498779023 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b09130fe0 con 0x562b0920f4a0 2024-05-22T23:03:31.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.785+0000 7f8c9659c640 1 --2- 172.21.15.84:0/3498779023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920f4a0 0x562b09220750 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c0cd030 tx=0x7f8c8c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b0c3e9cf5c783d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:03:31.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.785+0000 7f8c7aff5640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:03:31.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.785+0000 7f8c7aff5640 1 -- 172.21.15.84:0/3498779023 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c0ba050 con 0x562b0920f4a0 2024-05-22T23:03:31.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.785+0000 7f8c7aff5640 1 -- 172.21.15.84:0/3498779023 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f8c8c010430 con 0x562b0920f4a0 2024-05-22T23:03:31.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.785+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3498779023 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c8c09b3f0 con 0x562b0920f4a0 2024-05-22T23:03:31.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.785+0000 7f8c7aff5640 1 -- 172.21.15.84:0/3498779023 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f8c8c09b250 con 0x562b0920f4a0 2024-05-22T23:03:31.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.785+0000 7f8c9fe25740 10 client.15720 fetch_fsmap learned FSMap version 18 2024-05-22T23:03:31.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.785+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3498779023 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x562b091021c0 con 0x562b0920f4a0 2024-05-22T23:03:31.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.785+0000 7f8c7aff5640 1 -- 172.21.15.84:0/3498779023 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c8c09b3f0 con 0x562b0920f4a0 2024-05-22T23:03:31.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.785+0000 7f8c7aff5640 1 -- 172.21.15.84:0/3498779023 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f8c8c018a90 con 0x562b0920f4a0 2024-05-22T23:03:31.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.785+0000 7f8c9fe25740 10 client.15720 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:03:31.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.785+0000 7f8c9fe25740 10 client.15720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:03:31.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.785+0000 7f8c9fe25740 20 client.15720 populate_metadata read hostname 'smithi084' 2024-05-22T23:03:31.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.785+0000 7f8c9fe25740 1 --2- 172.21.15.84:0/3498779023 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x562b0928e8a0 0x562b09290c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:03:31.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.785+0000 7f8c9fe25740 4 client.15720 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:03:31.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.785+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3498779023 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562b09179830 con 0x562b0928e8a0 2024-05-22T23:03:31.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.786+0000 7f8c96d9d640 1 --2- 172.21.15.84:0/3498779023 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x562b0928e8a0 0x562b09290c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:03:31.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.786+0000 7f8c7aff5640 10 client.15720 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:03:31.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.786+0000 7f8c96d9d640 1 --2- 172.21.15.84:0/3498779023 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x562b0928e8a0 0x562b09290c90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16e65a6dfedcc6f0 server_cookie=b56398a1e5e7cc14 in_seq=0 out_seq=0 2024-05-22T23:03:31.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.787+0000 7f8c7aff5640 1 -- 172.21.15.84:0/3498779023 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562b09179830 con 0x562b0928e8a0 2024-05-22T23:03:31.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.787+0000 7f8c7aff5640 10 client.15720 handle_command_reply: tid=0 2024-05-22T23:03:31.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.787+0000 7f8c9fe25740 1 client.15720 shutdown 2024-05-22T23:03:31.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.787+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3498779023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920f4a0 msgr2=0x562b09220750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:31.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.787+0000 7f8c9fe25740 1 --2- 172.21.15.84:0/3498779023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920f4a0 0x562b09220750 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c0cd030 tx=0x7f8c8c01f000 comp rx=0 tx=0).stop 2024-05-22T23:03:31.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.787+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3498779023 shutdown_connections 2024-05-22T23:03:31.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.787+0000 7f8c9fe25740 1 --2- 172.21.15.84:0/3498779023 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x562b0928e8a0 0x562b09290c90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.787+0000 7f8c7aff5640 0 client.15720 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:03:31.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.787+0000 7f8c9fe25740 1 --2- 172.21.15.84:0/3498779023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b09218b30 0x562b09220c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.787+0000 7f8c9fe25740 1 --2- 172.21.15.84:0/3498779023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562b0920f4a0 0x562b09220750 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.787+0000 7f8c9fe25740 1 --2- 172.21.15.84:0/3498779023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b0920ddb0 0x562b09220210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.787+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3498779023 >> 172.21.15.84:0/3498779023 conn(0x562b091d68e0 msgr2=0x562b09211dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:31.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.787+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3498779023 shutdown_connections 2024-05-22T23:03:31.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.787+0000 7f8c9fe25740 1 -- 172.21.15.84:0/3498779023 wait complete. 2024-05-22T23:03:31.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.787+0000 7f8c9fe25740 20 client.15720 trim_cache size 0 max 16384 2024-05-22T23:03:31.800 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:03:31.800 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-22T23:03:31.800 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": { 2024-05-22T23:03:31.801 INFO:teuthology.orchestra.run.smithi084.stdout: "49319a0e-ca52-4466-88b8-527e003e4766": { 2024-05-22T23:03:31.801 INFO:teuthology.orchestra.run.smithi084.stdout: "path": "/", 2024-05-22T23:03:31.801 INFO:teuthology.orchestra.run.smithi084.stdout: "tag": "49319a0e-ca52-4466-88b8-527e003e4766", 2024-05-22T23:03:31.801 INFO:teuthology.orchestra.run.smithi084.stdout: "options": "recursive,force" 2024-05-22T23:03:31.801 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:03:31.801 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:03:31.802 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:03:31.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.787+0000 7f8c9d038640 1 -- 172.21.15.84:0/1975587966 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f8c800a2d00 msgr2=0x7f8c800a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:31.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.787+0000 7f8c9d038640 1 --2- 172.21.15.84:0/1975587966 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f8c800a2d00 0x7f8c800a51c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8c88009230 tx=0x7f8c880058d0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.787+0000 7f8c9d038640 1 -- 172.21.15.84:0/1975587966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c98103420 msgr2=0x7f8c981986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:03:31.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.787+0000 7f8c9d038640 1 --2- 172.21.15.84:0/1975587966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c98103420 0x7f8c981986c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00e9f0 tx=0x7f8c8c00eeb0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.788+0000 7f8c9d038640 1 -- 172.21.15.84:0/1975587966 shutdown_connections 2024-05-22T23:03:31.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.788+0000 7f8c9d038640 1 --2- 172.21.15.84:0/1975587966 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f8c800a2d00 0x7f8c800a51c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.788+0000 7f8c9d038640 1 --2- 172.21.15.84:0/1975587966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c9810be40 0x7f8c9819ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.788+0000 7f8c9d038640 1 --2- 172.21.15.84:0/1975587966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c98103420 0x7f8c981986c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.788+0000 7f8c9d038640 1 --2- 172.21.15.84:0/1975587966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c98102b00 0x7f8c98198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:03:31.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.788+0000 7f8c9d038640 1 -- 172.21.15.84:0/1975587966 >> 172.21.15.84:0/1975587966 conn(0x7f8c98076af0 msgr2=0x7f8c980fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:03:31.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.788+0000 7f8c9d038640 1 -- 172.21.15.84:0/1975587966 shutdown_connections 2024-05-22T23:03:31.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:03:31.788+0000 7f8c9d038640 1 -- 172.21.15.84:0/1975587966 wait complete. 2024-05-22T23:03:31.806 INFO:tasks.cephfs.filesystem:scrub status for tag:49319a0e-ca52-4466-88b8-527e003e4766 - {'path': '/', 'tag': '49319a0e-ca52-4466-88b8-527e003e4766', 'options': 'recursive,force'} 2024-05-22T23:03:32.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:32 smithi084 ceph-mon[32186]: scrub queued for path: / 2024-05-22T23:03:32.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:32 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:03:32.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:32 smithi084 ceph-mon[32186]: scrub summary: active paths [/] 2024-05-22T23:03:32.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:32 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:03:32.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:32 smithi204 ceph-mon[36211]: scrub queued for path: / 2024-05-22T23:03:32.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:32 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:03:32.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:32 smithi204 ceph-mon[36211]: scrub summary: active paths [/] 2024-05-22T23:03:32.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:32 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:03:32.423 INFO:tasks.workunit.client.0.smithi084.stdout: 1 57664 18.20 MB/sec warmup 18 sec latency 22.136 ms 2024-05-22T23:03:32.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:32 smithi096 ceph-mon[36392]: scrub queued for path: / 2024-05-22T23:03:32.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:32 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:03:32.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:32 smithi096 ceph-mon[36392]: scrub summary: active paths [/] 2024-05-22T23:03:32.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:32 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:03:33.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:33 smithi084 ceph-mon[32186]: pgmap v54: 97 pgs: 97 active+clean; 26 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 280 op/s 2024-05-22T23:03:33.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:33 smithi204 ceph-mon[36211]: pgmap v54: 97 pgs: 97 active+clean; 26 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 280 op/s 2024-05-22T23:03:33.423 INFO:tasks.workunit.client.0.smithi084.stdout: 1 60995 18.22 MB/sec warmup 19 sec latency 10.864 ms 2024-05-22T23:03:33.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:33 smithi096 ceph-mon[36392]: pgmap v54: 97 pgs: 97 active+clean; 26 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 280 op/s 2024-05-22T23:03:34.424 INFO:tasks.workunit.client.0.smithi084.stdout: 1 64422 18.26 MB/sec warmup 20 sec latency 14.128 ms 2024-05-22T23:03:35.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:35 smithi084 ceph-mon[32186]: pgmap v55: 97 pgs: 97 active+clean; 26 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-05-22T23:03:35.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:35 smithi204 ceph-mon[36211]: pgmap v55: 97 pgs: 97 active+clean; 26 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-05-22T23:03:35.423 INFO:tasks.workunit.client.0.smithi084.stdout: 1 67563 18.24 MB/sec warmup 21 sec latency 108.330 ms 2024-05-22T23:03:35.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:35 smithi096 ceph-mon[36392]: pgmap v55: 97 pgs: 97 active+clean; 26 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-05-22T23:03:36.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:36 smithi084 ceph-mon[32186]: scrub summary: idle 2024-05-22T23:03:36.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:36 smithi204 ceph-mon[36211]: scrub summary: idle 2024-05-22T23:03:36.423 INFO:tasks.workunit.client.0.smithi084.stdout: 1 70794 18.13 MB/sec warmup 22 sec latency 15.072 ms 2024-05-22T23:03:36.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:36 smithi096 ceph-mon[36392]: scrub summary: idle 2024-05-22T23:03:37.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:37 smithi084 ceph-mon[32186]: pgmap v56: 97 pgs: 97 active+clean; 26 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-05-22T23:03:37.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:37 smithi204 ceph-mon[36211]: pgmap v56: 97 pgs: 97 active+clean; 26 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-05-22T23:03:37.424 INFO:tasks.workunit.client.0.smithi084.stdout: 1 73895 18.06 MB/sec warmup 23 sec latency 38.151 ms 2024-05-22T23:03:37.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:37 smithi096 ceph-mon[36392]: pgmap v56: 97 pgs: 97 active+clean; 26 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-05-22T23:03:38.424 INFO:tasks.workunit.client.0.smithi084.stdout: 1 77034 17.93 MB/sec warmup 24 sec latency 13.708 ms 2024-05-22T23:03:39.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:39 smithi084 ceph-mon[32186]: pgmap v57: 97 pgs: 97 active+clean; 34 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 283 op/s 2024-05-22T23:03:39.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:39 smithi204 ceph-mon[36211]: pgmap v57: 97 pgs: 97 active+clean; 34 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 283 op/s 2024-05-22T23:03:39.424 INFO:tasks.workunit.client.0.smithi084.stdout: 1 79958 17.88 MB/sec warmup 25 sec latency 16.838 ms 2024-05-22T23:03:39.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:39 smithi096 ceph-mon[36392]: pgmap v57: 97 pgs: 97 active+clean; 34 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 283 op/s 2024-05-22T23:03:40.424 INFO:tasks.workunit.client.0.smithi084.stdout: 1 83045 17.92 MB/sec warmup 26 sec latency 14.811 ms 2024-05-22T23:03:41.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:41 smithi084 ceph-mon[32186]: pgmap v58: 97 pgs: 97 active+clean; 34 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 1.6 MiB/s wr, 187 op/s 2024-05-22T23:03:41.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:41 smithi204 ceph-mon[36211]: pgmap v58: 97 pgs: 97 active+clean; 34 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 1.6 MiB/s wr, 187 op/s 2024-05-22T23:03:41.424 INFO:tasks.workunit.client.0.smithi084.stdout: 1 85967 17.79 MB/sec warmup 27 sec latency 15.519 ms 2024-05-22T23:03:41.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:41 smithi096 ceph-mon[36392]: pgmap v58: 97 pgs: 97 active+clean; 34 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 1.6 MiB/s wr, 187 op/s 2024-05-22T23:03:42.424 INFO:tasks.workunit.client.0.smithi084.stdout: 1 89392 17.83 MB/sec warmup 28 sec latency 10.726 ms 2024-05-22T23:03:43.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:43 smithi084 ceph-mon[32186]: pgmap v59: 97 pgs: 97 active+clean; 41 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 281 op/s 2024-05-22T23:03:43.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:43 smithi204 ceph-mon[36211]: pgmap v59: 97 pgs: 97 active+clean; 41 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 281 op/s 2024-05-22T23:03:43.424 INFO:tasks.workunit.client.0.smithi084.stdout: 1 92799 17.87 MB/sec warmup 29 sec latency 27.340 ms 2024-05-22T23:03:43.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:43 smithi096 ceph-mon[36392]: pgmap v59: 97 pgs: 97 active+clean; 41 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 281 op/s 2024-05-22T23:03:44.424 INFO:tasks.workunit.client.0.smithi084.stdout: 1 96190 17.91 MB/sec warmup 30 sec latency 13.199 ms 2024-05-22T23:03:45.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:45 smithi084 ceph-mon[32186]: pgmap v60: 97 pgs: 97 active+clean; 41 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-05-22T23:03:45.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:45 smithi204 ceph-mon[36211]: pgmap v60: 97 pgs: 97 active+clean; 41 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-05-22T23:03:45.424 INFO:tasks.workunit.client.0.smithi084.stdout: 1 99199 17.80 MB/sec warmup 31 sec latency 133.495 ms 2024-05-22T23:03:45.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:45 smithi096 ceph-mon[36392]: pgmap v60: 97 pgs: 97 active+clean; 41 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-05-22T23:03:46.424 INFO:tasks.workunit.client.0.smithi084.stdout: 1 99610 17.38 MB/sec warmup 32 sec latency 494.250 ms 2024-05-22T23:03:47.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:47 smithi084 ceph-mon[32186]: pgmap v61: 97 pgs: 97 active+clean; 49 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 2.3 MiB/s wr, 203 op/s 2024-05-22T23:03:47.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:47 smithi204 ceph-mon[36211]: pgmap v61: 97 pgs: 97 active+clean; 49 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 2.3 MiB/s wr, 203 op/s 2024-05-22T23:03:47.424 INFO:tasks.workunit.client.0.smithi084.stdout: 1 101816 17.08 MB/sec warmup 33 sec latency 46.852 ms 2024-05-22T23:03:47.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:47 smithi096 ceph-mon[36392]: pgmap v61: 97 pgs: 97 active+clean; 49 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 2.3 MiB/s wr, 203 op/s 2024-05-22T23:03:48.424 INFO:tasks.workunit.client.0.smithi084.stdout: 1 104771 17.09 MB/sec warmup 34 sec latency 19.121 ms 2024-05-22T23:03:49.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:49 smithi084 ceph-mon[32186]: pgmap v62: 97 pgs: 97 active+clean; 142 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 10 MiB/s wr, 398 op/s 2024-05-22T23:03:49.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:49 smithi204 ceph-mon[36211]: pgmap v62: 97 pgs: 97 active+clean; 142 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 10 MiB/s wr, 398 op/s 2024-05-22T23:03:49.425 INFO:tasks.workunit.client.0.smithi084.stdout: 1 107543 17.05 MB/sec warmup 35 sec latency 13.943 ms 2024-05-22T23:03:49.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:49 smithi096 ceph-mon[36392]: pgmap v62: 97 pgs: 97 active+clean; 142 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 10 MiB/s wr, 398 op/s 2024-05-22T23:03:50.425 INFO:tasks.workunit.client.0.smithi084.stdout: 1 110667 17.05 MB/sec warmup 36 sec latency 11.543 ms 2024-05-22T23:03:51.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:51 smithi084 ceph-mon[32186]: pgmap v63: 97 pgs: 97 active+clean; 142 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 9.5 MiB/s wr, 303 op/s 2024-05-22T23:03:51.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:51 smithi204 ceph-mon[36211]: pgmap v63: 97 pgs: 97 active+clean; 142 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 9.5 MiB/s wr, 303 op/s 2024-05-22T23:03:51.424 INFO:tasks.workunit.client.0.smithi084.stdout: 1 113943 17.11 MB/sec warmup 37 sec latency 22.648 ms 2024-05-22T23:03:51.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:51 smithi096 ceph-mon[36392]: pgmap v63: 97 pgs: 97 active+clean; 142 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 9.5 MiB/s wr, 303 op/s 2024-05-22T23:03:52.425 INFO:tasks.workunit.client.0.smithi084.stdout: 1 117277 17.14 MB/sec warmup 38 sec latency 17.685 ms 2024-05-22T23:03:53.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:53 smithi084 ceph-mon[32186]: pgmap v64: 97 pgs: 97 active+clean; 61 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 505 op/s 2024-05-22T23:03:53.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:53 smithi204 ceph-mon[36211]: pgmap v64: 97 pgs: 97 active+clean; 61 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 505 op/s 2024-05-22T23:03:53.425 INFO:tasks.workunit.client.0.smithi084.stdout: 1 120256 17.06 MB/sec warmup 39 sec latency 22.547 ms 2024-05-22T23:03:53.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:53 smithi096 ceph-mon[36392]: pgmap v64: 97 pgs: 97 active+clean; 61 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 505 op/s 2024-05-22T23:03:54.425 INFO:tasks.workunit.client.0.smithi084.stdout: 1 123192 16.98 MB/sec warmup 40 sec latency 13.883 ms 2024-05-22T23:03:55.353 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:55 smithi096 ceph-mon[36392]: pgmap v65: 97 pgs: 97 active+clean; 61 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.7 MiB/s wr, 411 op/s 2024-05-22T23:03:55.377 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:55 smithi084 ceph-mon[32186]: pgmap v65: 97 pgs: 97 active+clean; 61 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.7 MiB/s wr, 411 op/s 2024-05-22T23:03:55.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:55 smithi204 ceph-mon[36211]: pgmap v65: 97 pgs: 97 active+clean; 61 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.7 MiB/s wr, 411 op/s 2024-05-22T23:03:55.425 INFO:tasks.workunit.client.0.smithi084.stdout: 1 126165 16.97 MB/sec warmup 41 sec latency 14.264 ms 2024-05-22T23:03:56.425 INFO:tasks.workunit.client.0.smithi084.stdout: 1 128934 16.94 MB/sec warmup 42 sec latency 15.088 ms 2024-05-22T23:03:57.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:57 smithi084 ceph-mon[32186]: pgmap v66: 97 pgs: 97 active+clean; 61 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.7 MiB/s wr, 412 op/s 2024-05-22T23:03:57.377 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:57 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:03:57.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:57 smithi204 ceph-mon[36211]: pgmap v66: 97 pgs: 97 active+clean; 61 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.7 MiB/s wr, 412 op/s 2024-05-22T23:03:57.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:57 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:03:57.425 INFO:tasks.workunit.client.0.smithi084.stdout: 1 131943 16.95 MB/sec warmup 43 sec latency 16.896 ms 2024-05-22T23:03:57.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:57 smithi096 ceph-mon[36392]: pgmap v66: 97 pgs: 97 active+clean; 61 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.7 MiB/s wr, 412 op/s 2024-05-22T23:03:57.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:57 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:03:58.377 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:58 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:03:58.377 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:58 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:03:58.377 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:58 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:03:58.377 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:58 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:03:58.377 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:58 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:03:58.377 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:58 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:03:58.378 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:58 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:03:58.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:58 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:03:58.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:58 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:03:58.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:58 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:03:58.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:58 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:03:58.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:58 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:03:58.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:58 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:03:58.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:58 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:03:58.425 INFO:tasks.workunit.client.0.smithi084.stdout: 1 135066 16.99 MB/sec warmup 44 sec latency 71.295 ms 2024-05-22T23:03:58.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:58 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:03:58.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:58 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:03:58.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:58 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:03:58.552 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:58 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:03:58.552 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:58 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:03:58.552 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:58 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:03:58.552 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:58 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:03:59.425 INFO:tasks.workunit.client.0.smithi084.stdout: 1 138374 17.00 MB/sec warmup 45 sec latency 14.223 ms 2024-05-22T23:03:59.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:03:59 smithi096 ceph-mon[36392]: pgmap v67: 97 pgs: 97 active+clean; 68 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 9.7 MiB/s wr, 494 op/s 2024-05-22T23:03:59.626 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:03:59 smithi084 ceph-mon[32186]: pgmap v67: 97 pgs: 97 active+clean; 68 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 9.7 MiB/s wr, 494 op/s 2024-05-22T23:03:59.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:03:59 smithi204 ceph-mon[36211]: pgmap v67: 97 pgs: 97 active+clean; 68 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 9.7 MiB/s wr, 494 op/s 2024-05-22T23:04:00.425 INFO:tasks.workunit.client.0.smithi084.stdout: 1 141341 16.94 MB/sec warmup 46 sec latency 14.014 ms 2024-05-22T23:04:01.425 INFO:tasks.workunit.client.0.smithi084.stdout: 1 143066 16.76 MB/sec warmup 47 sec latency 21.964 ms 2024-05-22T23:04:01.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:01 smithi096 ceph-mon[36392]: pgmap v68: 97 pgs: 97 active+clean; 68 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 299 op/s 2024-05-22T23:04:01.626 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:01 smithi084 ceph-mon[32186]: pgmap v68: 97 pgs: 97 active+clean; 68 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 299 op/s 2024-05-22T23:04:01.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:01 smithi204 ceph-mon[36211]: pgmap v68: 97 pgs: 97 active+clean; 68 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 299 op/s 2024-05-22T23:04:01.807 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:04:01.807 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:04:01.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.919+0000 7f0054010640 1 -- 172.21.15.84:0/1116840509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f004c106750 msgr2=0x7f004c106b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:01.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.919+0000 7f0054010640 1 --2- 172.21.15.84:0/1116840509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f004c106750 0x7f004c106b30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0044009930 tx=0x7f004402a420 comp rx=0 tx=0).stop 2024-05-22T23:04:01.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.919+0000 7f0054010640 1 -- 172.21.15.84:0/1116840509 shutdown_connections 2024-05-22T23:04:01.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.919+0000 7f0054010640 1 --2- 172.21.15.84:0/1116840509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f004c101100 0x7f004c10d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.919+0000 7f0054010640 1 --2- 172.21.15.84:0/1116840509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f004c100740 0x7f004c100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.919+0000 7f0054010640 1 --2- 172.21.15.84:0/1116840509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f004c106750 0x7f004c106b30 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.919+0000 7f0054010640 1 -- 172.21.15.84:0/1116840509 >> 172.21.15.84:0/1116840509 conn(0x7f004c077870 msgr2=0x7f004c077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:01.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.920+0000 7f0054010640 1 -- 172.21.15.84:0/1116840509 shutdown_connections 2024-05-22T23:04:01.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.920+0000 7f0054010640 1 -- 172.21.15.84:0/1116840509 wait complete. 2024-05-22T23:04:01.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.920+0000 7f0054010640 1 Processor -- start 2024-05-22T23:04:01.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.920+0000 7f0054010640 1 -- start start 2024-05-22T23:04:01.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.920+0000 7f0054010640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f004c100740 0x7f004c198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:01.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.920+0000 7f0054010640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f004c101100 0x7f004c199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:01.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.920+0000 7f0054010640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f004c106750 0x7f004c1a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:01.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.920+0000 7f0054010640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f004c110b60 con 0x7f004c100740 2024-05-22T23:04:01.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.920+0000 7f0054010640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f004c1109e0 con 0x7f004c101100 2024-05-22T23:04:01.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.920+0000 7f0054010640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f004c110ce0 con 0x7f004c106750 2024-05-22T23:04:01.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.920+0000 7f0051dac640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f004c100740 0x7f004c198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:01.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.921+0000 7f0051dac640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f004c100740 0x7f004c198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43390/0 (socket says 172.21.15.84:43390) 2024-05-22T23:04:01.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.921+0000 7f0051dac640 1 -- 172.21.15.84:0/235746409 learned_addr learned my addr 172.21.15.84:0/235746409 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:01.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.921+0000 7f00525ad640 1 --2- 172.21.15.84:0/235746409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f004c106750 0x7f004c1a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:01.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.921+0000 7f00515ab640 1 --2- 172.21.15.84:0/235746409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f004c101100 0x7f004c199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:01.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.921+0000 7f0051dac640 1 -- 172.21.15.84:0/235746409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f004c106750 msgr2=0x7f004c1a4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:01.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.921+0000 7f0051dac640 1 --2- 172.21.15.84:0/235746409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f004c106750 0x7f004c1a4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.921+0000 7f0051dac640 1 -- 172.21.15.84:0/235746409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f004c101100 msgr2=0x7f004c199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:01.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.921+0000 7f0051dac640 1 --2- 172.21.15.84:0/235746409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f004c101100 0x7f004c199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.921+0000 7f0051dac640 1 -- 172.21.15.84:0/235746409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004c1a45a0 con 0x7f004c100740 2024-05-22T23:04:01.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.921+0000 7f0051dac640 1 --2- 172.21.15.84:0/235746409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f004c100740 0x7f004c198fe0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0044009230 tx=0x7f0044004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:01.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.921+0000 7f004affd640 1 -- 172.21.15.84:0/235746409 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004403f030 con 0x7f004c100740 2024-05-22T23:04:01.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.921+0000 7f004affd640 1 -- 172.21.15.84:0/235746409 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0044003ad0 con 0x7f004c100740 2024-05-22T23:04:01.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.921+0000 7f0054010640 1 -- 172.21.15.84:0/235746409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f004c1a4770 con 0x7f004c100740 2024-05-22T23:04:01.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.921+0000 7f004affd640 1 -- 172.21.15.84:0/235746409 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0044035070 con 0x7f004c100740 2024-05-22T23:04:01.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.921+0000 7f0054010640 1 -- 172.21.15.84:0/235746409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f004c1a4be0 con 0x7f004c100740 2024-05-22T23:04:01.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.924+0000 7f004affd640 1 -- 172.21.15.84:0/235746409 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0044042030 con 0x7f004c100740 2024-05-22T23:04:01.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.924+0000 7f0056df5740 1 Processor -- start 2024-05-22T23:04:01.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.924+0000 7f0056df5740 1 -- start start 2024-05-22T23:04:01.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.925+0000 7f0056df5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597101084a0 0x5597101067e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:01.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.925+0000 7f0056df5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559710106db0 0x559710107230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:01.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.925+0000 7f0056df5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559710111b30 0x559710113f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:01.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.925+0000 7f0056df5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55971006adb0 con 0x559710106db0 2024-05-22T23:04:01.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.925+0000 7f0056df5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559710009360 con 0x559710111b30 2024-05-22T23:04:01.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.925+0000 7f0056df5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559710066d70 con 0x5597101084a0 2024-05-22T23:04:01.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.925+0000 7f0051dac640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559710106db0 0x559710107230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:01.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.925+0000 7f00525ad640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597101084a0 0x5597101067e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:01.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.925+0000 7f0051dac640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559710106db0 0x559710107230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43402/0 (socket says 172.21.15.84:43402) 2024-05-22T23:04:01.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.925+0000 7f0051dac640 1 -- 172.21.15.84:0/3669713724 learned_addr learned my addr 172.21.15.84:0/3669713724 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:01.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.925+0000 7f00515ab640 1 --2- 172.21.15.84:0/3669713724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559710111b30 0x559710113f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:01.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.925+0000 7f004affd640 1 --2- 172.21.15.84:0/235746409 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f00340a2d00 0x7f00340a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:01.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.925+0000 7f0051dac640 1 -- 172.21.15.84:0/3669713724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597101084a0 msgr2=0x5597101067e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:01.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.925+0000 7f0051dac640 1 --2- 172.21.15.84:0/3669713724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597101084a0 0x5597101067e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.925+0000 7f0051dac640 1 -- 172.21.15.84:0/3669713724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559710111b30 msgr2=0x559710113f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:01.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.925+0000 7f0051dac640 1 --2- 172.21.15.84:0/3669713724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559710111b30 0x559710113f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.925+0000 7f0051dac640 1 -- 172.21.15.84:0/3669713724 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55971002ca60 con 0x559710106db0 2024-05-22T23:04:01.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.925+0000 7f00515ab640 1 --2- 172.21.15.84:0/3669713724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559710111b30 0x559710113f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:01.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.925+0000 7f00525ad640 1 --2- 172.21.15.84:0/235746409 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f00340a2d00 0x7f00340a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:01.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.925+0000 7f004affd640 1 -- 172.21.15.84:0/235746409 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f00440ef9e0 con 0x7f004c100740 2024-05-22T23:04:01.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.925+0000 7f0051dac640 1 --2- 172.21.15.84:0/3669713724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559710106db0 0x559710107230 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f00440ed0a0 tx=0x7f0044086b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73fcbf35dce9a6d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:01.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.925+0000 7f0033fff640 1 -- 172.21.15.84:0/3669713724 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00440bb860 con 0x559710106db0 2024-05-22T23:04:01.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.926+0000 7f00525ad640 1 --2- 172.21.15.84:0/235746409 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f00340a2d00 0x7f00340a51c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f003c009fd0 tx=0x7f003c004a40 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:01.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.926+0000 7f0033fff640 1 -- 172.21.15.84:0/3669713724 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00440bc600 con 0x559710106db0 2024-05-22T23:04:01.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.926+0000 7f0033fff640 1 -- 172.21.15.84:0/3669713724 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004408ac40 con 0x559710106db0 2024-05-22T23:04:01.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.926+0000 7f0056df5740 1 -- 172.21.15.84:0/3669713724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559710106db0 msgr2=0x559710107230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:01.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.926+0000 7f0056df5740 1 --2- 172.21.15.84:0/3669713724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559710106db0 0x559710107230 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f00440ed0a0 tx=0x7f0044086b30 comp rx=0 tx=0).stop 2024-05-22T23:04:01.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.926+0000 7f0056df5740 1 -- 172.21.15.84:0/3669713724 shutdown_connections 2024-05-22T23:04:01.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.926+0000 7f0056df5740 1 --2- 172.21.15.84:0/3669713724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559710111b30 0x559710113f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.926+0000 7f0056df5740 1 --2- 172.21.15.84:0/3669713724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559710106db0 0x559710107230 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.926+0000 7f0056df5740 1 --2- 172.21.15.84:0/3669713724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597101084a0 0x5597101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.926+0000 7f0056df5740 1 -- 172.21.15.84:0/3669713724 >> 172.21.15.84:0/3669713724 conn(0x5597100cf8e0 msgr2=0x559710103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:01.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.926+0000 7f0056df5740 1 -- 172.21.15.84:0/3669713724 shutdown_connections 2024-05-22T23:04:01.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.926+0000 7f0056df5740 1 -- 172.21.15.84:0/3669713724 wait complete. 2024-05-22T23:04:01.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.927+0000 7f0056df5740 1 -- start start 2024-05-22T23:04:01.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.927+0000 7f0056df5740 1 Processor -- start 2024-05-22T23:04:01.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.927+0000 7f0056df5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559710106db0 0x55971010ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:01.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.927+0000 7f0056df5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597101084a0 0x55971010ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:01.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.927+0000 7f0056df5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559710111b30 0x55971010f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:01.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.927+0000 7f0056df5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559710009360 con 0x5597101084a0 2024-05-22T23:04:01.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.927+0000 7f0056df5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5597100250f0 con 0x559710106db0 2024-05-22T23:04:01.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.927+0000 7f0056df5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55971006adb0 con 0x559710111b30 2024-05-22T23:04:01.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.927+0000 7f0056df5740 10 client.0 _ll_register_callbacks cb 0x559710029f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:04:01.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.927+0000 7f00515ab640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597101084a0 0x55971010ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:01.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.927+0000 7f00515ab640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597101084a0 0x55971010ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43416/0 (socket says 172.21.15.84:43416) 2024-05-22T23:04:01.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.927+0000 7f00515ab640 1 -- 172.21.15.84:0/1116131601 learned_addr learned my addr 172.21.15.84:0/1116131601 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:01.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.927+0000 7f0051dac640 1 --2- 172.21.15.84:0/1116131601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559710106db0 0x55971010ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:01.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.927+0000 7f00525ad640 1 --2- 172.21.15.84:0/1116131601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559710111b30 0x55971010f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:01.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.928+0000 7f00515ab640 1 -- 172.21.15.84:0/1116131601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559710111b30 msgr2=0x55971010f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:01.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.928+0000 7f00515ab640 1 --2- 172.21.15.84:0/1116131601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559710111b30 0x55971010f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.928+0000 7f00515ab640 1 -- 172.21.15.84:0/1116131601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559710106db0 msgr2=0x55971010ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:01.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.928+0000 7f00515ab640 1 --2- 172.21.15.84:0/1116131601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559710106db0 0x55971010ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.928+0000 7f00515ab640 1 -- 172.21.15.84:0/1116131601 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55970fe487d0 con 0x5597101084a0 2024-05-22T23:04:01.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.928+0000 7f0051dac640 1 --2- 172.21.15.84:0/1116131601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559710106db0 0x55971010ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:01.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.928+0000 7f00515ab640 1 --2- 172.21.15.84:0/1116131601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597101084a0 0x55971010ef80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f00400115a0 tx=0x7f0040011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a526a7c4889033 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:01.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.928+0000 7f00317fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:04:01.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.928+0000 7f00317fa640 1 -- 172.21.15.84:0/1116131601 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0040013670 con 0x5597101084a0 2024-05-22T23:04:01.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.928+0000 7f00317fa640 1 -- 172.21.15.84:0/1116131601 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f0040013810 con 0x5597101084a0 2024-05-22T23:04:01.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.928+0000 7f0056df5740 1 -- 172.21.15.84:0/1116131601 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00440bc600 con 0x5597101084a0 2024-05-22T23:04:01.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.928+0000 7f00317fa640 1 -- 172.21.15.84:0/1116131601 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f0040013a10 con 0x5597101084a0 2024-05-22T23:04:01.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.928+0000 7f0056df5740 10 client.15732 fetch_fsmap learned FSMap version 18 2024-05-22T23:04:01.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.928+0000 7f0056df5740 1 -- 172.21.15.84:0/1116131601 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55970fd7e3f0 con 0x5597101084a0 2024-05-22T23:04:01.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.928+0000 7f00317fa640 1 -- 172.21.15.84:0/1116131601 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00440bc600 con 0x5597101084a0 2024-05-22T23:04:01.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.928+0000 7f00317fa640 1 -- 172.21.15.84:0/1116131601 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f004001d400 con 0x5597101084a0 2024-05-22T23:04:01.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.928+0000 7f0056df5740 10 client.15732 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:04:01.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.929+0000 7f0056df5740 10 client.15732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:04:01.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.929+0000 7f0056df5740 20 client.15732 populate_metadata read hostname 'smithi084' 2024-05-22T23:04:01.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.929+0000 7f0056df5740 1 --2- 172.21.15.84:0/1116131601 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x559710121ad0 0x559710123f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:01.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.929+0000 7f0056df5740 4 client.15732 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:04:01.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.929+0000 7f0056df5740 1 -- 172.21.15.84:0/1116131601 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597100784e0 con 0x559710121ad0 2024-05-22T23:04:01.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.929+0000 7f0051dac640 1 --2- 172.21.15.84:0/1116131601 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x559710121ad0 0x559710123f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:01.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.929+0000 7f0051dac640 1 --2- 172.21.15.84:0/1116131601 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x559710121ad0 0x559710123f00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db04010a5bb41dae server_cookie=93b4e884454966db in_seq=0 out_seq=0 2024-05-22T23:04:01.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.929+0000 7f00317fa640 10 client.15732 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:01.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.930+0000 7f00317fa640 1 -- 172.21.15.84:0/1116131601 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5597100784e0 con 0x559710121ad0 2024-05-22T23:04:01.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.930+0000 7f00317fa640 10 client.15732 handle_command_reply: tid=0 2024-05-22T23:04:01.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.930+0000 7f0056df5740 1 client.15732 shutdown 2024-05-22T23:04:01.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.930+0000 7f0056df5740 1 -- 172.21.15.84:0/1116131601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597101084a0 msgr2=0x55971010ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:01.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.930+0000 7f0056df5740 1 --2- 172.21.15.84:0/1116131601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597101084a0 0x55971010ef80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f00400115a0 tx=0x7f0040011a60 comp rx=0 tx=0).stop 2024-05-22T23:04:01.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.930+0000 7f0056df5740 1 -- 172.21.15.84:0/1116131601 shutdown_connections 2024-05-22T23:04:01.941 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.930+0000 7f0056df5740 1 --2- 172.21.15.84:0/1116131601 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x559710121ad0 0x559710123f00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.941 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.930+0000 7f0056df5740 1 --2- 172.21.15.84:0/1116131601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559710111b30 0x55971010f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.941 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.930+0000 7f0056df5740 1 --2- 172.21.15.84:0/1116131601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597101084a0 0x55971010ef80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.941 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.930+0000 7f0056df5740 1 --2- 172.21.15.84:0/1116131601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559710106db0 0x55971010ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.941 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.930+0000 7f00317fa640 0 client.15732 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:01.941 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.930+0000 7f0056df5740 1 -- 172.21.15.84:0/1116131601 >> 172.21.15.84:0/1116131601 conn(0x5597100cf8e0 msgr2=0x559710109180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:01.941 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.930+0000 7f0056df5740 1 -- 172.21.15.84:0/1116131601 shutdown_connections 2024-05-22T23:04:01.941 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.930+0000 7f0056df5740 1 -- 172.21.15.84:0/1116131601 wait complete. 2024-05-22T23:04:01.941 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.930+0000 7f0056df5740 20 client.15732 trim_cache size 0 max 16384 2024-05-22T23:04:01.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.949+0000 7f0056df5740 1 Processor -- start 2024-05-22T23:04:01.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.949+0000 7f0056df5740 1 -- start start 2024-05-22T23:04:01.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.949+0000 7f0056df5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559710106db0 0x55971011d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:01.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.949+0000 7f0056df5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597101084a0 0x559710129860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:01.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.949+0000 7f0056df5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559710111b30 0x559710129da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:01.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.949+0000 7f0056df5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f004c1ab660 con 0x5597101084a0 2024-05-22T23:04:01.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.949+0000 7f0056df5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f004c1adee0 con 0x559710106db0 2024-05-22T23:04:01.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.949+0000 7f0056df5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f004c1abcd0 con 0x559710111b30 2024-05-22T23:04:01.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.950+0000 7f00515ab640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597101084a0 0x559710129860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:01.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.950+0000 7f00515ab640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597101084a0 0x559710129860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43418/0 (socket says 172.21.15.84:43418) 2024-05-22T23:04:01.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.950+0000 7f00515ab640 1 -- 172.21.15.84:0/3021914856 learned_addr learned my addr 172.21.15.84:0/3021914856 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:01.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.950+0000 7f0051dac640 1 --2- 172.21.15.84:0/3021914856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559710106db0 0x55971011d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:01.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.950+0000 7f00525ad640 1 --2- 172.21.15.84:0/3021914856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559710111b30 0x559710129da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:01.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.950+0000 7f00515ab640 1 -- 172.21.15.84:0/3021914856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559710111b30 msgr2=0x559710129da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:01.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.950+0000 7f00515ab640 1 --2- 172.21.15.84:0/3021914856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559710111b30 0x559710129da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.950+0000 7f00515ab640 1 -- 172.21.15.84:0/3021914856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559710106db0 msgr2=0x55971011d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:01.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.950+0000 7f00515ab640 1 --2- 172.21.15.84:0/3021914856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559710106db0 0x55971011d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.950+0000 7f00515ab640 1 -- 172.21.15.84:0/3021914856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55970fd3e0f0 con 0x5597101084a0 2024-05-22T23:04:01.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.950+0000 7f00515ab640 1 --2- 172.21.15.84:0/3021914856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597101084a0 0x559710129860 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f004001d990 tx=0x7f0040006b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ecb1a84b548cf38 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:01.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.950+0000 7f00337fe640 1 -- 172.21.15.84:0/3021914856 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004002a070 con 0x5597101084a0 2024-05-22T23:04:01.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.950+0000 7f00337fe640 1 -- 172.21.15.84:0/3021914856 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f004001d400 con 0x5597101084a0 2024-05-22T23:04:01.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.950+0000 7f00337fe640 1 -- 172.21.15.84:0/3021914856 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004002f460 con 0x5597101084a0 2024-05-22T23:04:01.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.951+0000 7f0056df5740 1 -- 172.21.15.84:0/3021914856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597101084a0 msgr2=0x559710129860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:01.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.951+0000 7f0056df5740 1 --2- 172.21.15.84:0/3021914856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597101084a0 0x559710129860 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f004001d990 tx=0x7f0040006b60 comp rx=0 tx=0).stop 2024-05-22T23:04:01.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.951+0000 7f0056df5740 1 -- 172.21.15.84:0/3021914856 shutdown_connections 2024-05-22T23:04:01.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.951+0000 7f0056df5740 1 --2- 172.21.15.84:0/3021914856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559710111b30 0x559710129da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.951+0000 7f0056df5740 1 --2- 172.21.15.84:0/3021914856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597101084a0 0x559710129860 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.951+0000 7f0056df5740 1 --2- 172.21.15.84:0/3021914856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559710106db0 0x55971011d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.951+0000 7f0056df5740 1 -- 172.21.15.84:0/3021914856 >> 172.21.15.84:0/3021914856 conn(0x5597100cf8e0 msgr2=0x55971010d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:01.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.951+0000 7f0056df5740 1 -- 172.21.15.84:0/3021914856 shutdown_connections 2024-05-22T23:04:01.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.951+0000 7f0056df5740 1 -- 172.21.15.84:0/3021914856 wait complete. 2024-05-22T23:04:01.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.951+0000 7f0056df5740 1 -- start start 2024-05-22T23:04:01.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.951+0000 7f0056df5740 1 Processor -- start 2024-05-22T23:04:01.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.952+0000 7f0056df5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559710106db0 0x559710119890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:01.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.952+0000 7f0056df5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597101084a0 0x559710119dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:01.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.952+0000 7f0056df5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559710111b30 0x559710116860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:01.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.952+0000 7f0056df5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f004c1ac3a0 con 0x559710111b30 2024-05-22T23:04:01.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.952+0000 7f0056df5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f004c1ad810 con 0x5597101084a0 2024-05-22T23:04:01.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.952+0000 7f0056df5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f004c1aca70 con 0x559710106db0 2024-05-22T23:04:01.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.952+0000 7f0056df5740 10 client.0 _ll_register_callbacks cb 0x7f00400094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:04:01.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.952+0000 7f00525ad640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559710111b30 0x559710116860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:01.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.952+0000 7f00515ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597101084a0 0x559710119dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:01.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.952+0000 7f00525ad640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559710111b30 0x559710116860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43428/0 (socket says 172.21.15.84:43428) 2024-05-22T23:04:01.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.952+0000 7f00525ad640 1 -- 172.21.15.84:0/267576865 learned_addr learned my addr 172.21.15.84:0/267576865 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:01.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.952+0000 7f00515ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597101084a0 0x559710119dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:37336/0 (socket says 172.21.15.84:37336) 2024-05-22T23:04:01.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.952+0000 7f0051dac640 1 --2- 172.21.15.84:0/267576865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559710106db0 0x559710119890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:01.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.952+0000 7f00525ad640 1 -- 172.21.15.84:0/267576865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559710106db0 msgr2=0x559710119890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:01.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.952+0000 7f00525ad640 1 --2- 172.21.15.84:0/267576865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559710106db0 0x559710119890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.952+0000 7f00525ad640 1 -- 172.21.15.84:0/267576865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597101084a0 msgr2=0x559710119dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:01.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.952+0000 7f00525ad640 1 --2- 172.21.15.84:0/267576865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597101084a0 0x559710119dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.952+0000 7f00525ad640 1 -- 172.21.15.84:0/267576865 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559710029fe0 con 0x559710111b30 2024-05-22T23:04:01.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.952+0000 7f0051dac640 1 --2- 172.21.15.84:0/267576865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559710106db0 0x559710119890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:04:01.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.953+0000 7f00525ad640 1 --2- 172.21.15.84:0/267576865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559710111b30 0x559710116860 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f003c00fd50 tx=0x7f003c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c27b718d64148a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:01.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.953+0000 7f00317fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:04:01.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.953+0000 7f00317fa640 1 -- 172.21.15.84:0/267576865 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c02f060 con 0x559710111b30 2024-05-22T23:04:01.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.953+0000 7f00317fa640 1 -- 172.21.15.84:0/267576865 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f003c02a640 con 0x559710111b30 2024-05-22T23:04:01.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.953+0000 7f0056df5740 1 -- 172.21.15.84:0/267576865 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f004001d400 con 0x559710111b30 2024-05-22T23:04:01.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.953+0000 7f00317fa640 1 -- 172.21.15.84:0/267576865 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f003c02a840 con 0x559710111b30 2024-05-22T23:04:01.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.953+0000 7f0056df5740 10 client.15738 fetch_fsmap learned FSMap version 18 2024-05-22T23:04:01.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.953+0000 7f0056df5740 1 -- 172.21.15.84:0/267576865 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55970fffb1c0 con 0x559710111b30 2024-05-22T23:04:01.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.953+0000 7f00317fa640 1 -- 172.21.15.84:0/267576865 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f004001d400 con 0x559710111b30 2024-05-22T23:04:01.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.953+0000 7f00317fa640 1 -- 172.21.15.84:0/267576865 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f003c01bac0 con 0x559710111b30 2024-05-22T23:04:01.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.953+0000 7f0056df5740 10 client.15738 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:04:01.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.953+0000 7f0056df5740 10 client.15738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:04:01.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.953+0000 7f0056df5740 20 client.15738 populate_metadata read hostname 'smithi084' 2024-05-22T23:04:01.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.953+0000 7f0056df5740 1 --2- 172.21.15.84:0/267576865 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x559710188fd0 0x55971018b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:01.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.953+0000 7f0056df5740 4 client.15738 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:04:01.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.953+0000 7f0056df5740 1 -- 172.21.15.84:0/267576865 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559710072830 con 0x559710188fd0 2024-05-22T23:04:01.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.954+0000 7f0051dac640 1 --2- 172.21.15.84:0/267576865 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x559710188fd0 0x55971018b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:01.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.954+0000 7f00317fa640 10 client.15738 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:01.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.954+0000 7f0051dac640 1 --2- 172.21.15.84:0/267576865 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x559710188fd0 0x55971018b3f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac59a1dbafbed9ed server_cookie=c4d0c7963a7ecdbb in_seq=0 out_seq=0 2024-05-22T23:04:01.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.955+0000 7f00317fa640 1 -- 172.21.15.84:0/267576865 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559710072830 con 0x559710188fd0 2024-05-22T23:04:01.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.955+0000 7f00317fa640 10 client.15738 handle_command_reply: tid=0 2024-05-22T23:04:01.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.955+0000 7f0056df5740 1 client.15738 shutdown 2024-05-22T23:04:01.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.955+0000 7f0056df5740 1 -- 172.21.15.84:0/267576865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559710111b30 msgr2=0x559710116860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:01.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.955+0000 7f0056df5740 1 --2- 172.21.15.84:0/267576865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559710111b30 0x559710116860 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f003c00fd50 tx=0x7f003c004ab0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.955+0000 7f0056df5740 1 -- 172.21.15.84:0/267576865 shutdown_connections 2024-05-22T23:04:01.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.955+0000 7f0056df5740 1 --2- 172.21.15.84:0/267576865 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x559710188fd0 0x55971018b3f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.955+0000 7f0056df5740 1 --2- 172.21.15.84:0/267576865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559710111b30 0x559710116860 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.955+0000 7f0056df5740 1 --2- 172.21.15.84:0/267576865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597101084a0 0x559710119dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.955+0000 7f0056df5740 1 --2- 172.21.15.84:0/267576865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559710106db0 0x559710119890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.955+0000 7f00317fa640 0 client.15738 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:01.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.955+0000 7f0056df5740 1 -- 172.21.15.84:0/267576865 >> 172.21.15.84:0/267576865 conn(0x5597100cf8e0 msgr2=0x559710108880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:01.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.955+0000 7f0056df5740 1 -- 172.21.15.84:0/267576865 shutdown_connections 2024-05-22T23:04:01.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.955+0000 7f0056df5740 1 -- 172.21.15.84:0/267576865 wait complete. 2024-05-22T23:04:01.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.955+0000 7f0056df5740 20 client.15738 trim_cache size 0 max 16384 2024-05-22T23:04:01.964 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:04:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "no active scrubs running", 2024-05-22T23:04:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": {} 2024-05-22T23:04:01.964 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:04:01.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.956+0000 7f0054010640 1 -- 172.21.15.84:0/235746409 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f00340a2d00 msgr2=0x7f00340a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:01.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.956+0000 7f0054010640 1 --2- 172.21.15.84:0/235746409 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f00340a2d00 0x7f00340a51c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f003c009fd0 tx=0x7f003c004a40 comp rx=0 tx=0).stop 2024-05-22T23:04:01.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.956+0000 7f0054010640 1 -- 172.21.15.84:0/235746409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f004c100740 msgr2=0x7f004c198fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:01.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.956+0000 7f0054010640 1 --2- 172.21.15.84:0/235746409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f004c100740 0x7f004c198fe0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0044009230 tx=0x7f0044004270 comp rx=0 tx=0).stop 2024-05-22T23:04:01.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.956+0000 7f0054010640 1 -- 172.21.15.84:0/235746409 shutdown_connections 2024-05-22T23:04:01.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.956+0000 7f0054010640 1 --2- 172.21.15.84:0/235746409 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f00340a2d00 0x7f00340a51c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.956+0000 7f0054010640 1 --2- 172.21.15.84:0/235746409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f004c106750 0x7f004c1a4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.956+0000 7f0054010640 1 --2- 172.21.15.84:0/235746409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f004c101100 0x7f004c199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.956+0000 7f0054010640 1 --2- 172.21.15.84:0/235746409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f004c100740 0x7f004c198fe0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:01.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.956+0000 7f0054010640 1 -- 172.21.15.84:0/235746409 >> 172.21.15.84:0/235746409 conn(0x7f004c077870 msgr2=0x7f004c102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:01.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.956+0000 7f0054010640 1 -- 172.21.15.84:0/235746409 shutdown_connections 2024-05-22T23:04:01.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:01.956+0000 7f0054010640 1 -- 172.21.15.84:0/235746409 wait complete. 2024-05-22T23:04:01.969 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-22T23:04:01.969 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:04:02.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.072+0000 7f2ac9aca640 1 -- 172.21.15.84:0/1117517134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac4102b00 msgr2=0x7f2ac4102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:02.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.072+0000 7f2ac9aca640 1 --2- 172.21.15.84:0/1117517134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac4102b00 0x7f2ac4102ee0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4009900 tx=0x7f2ab402a510 comp rx=0 tx=0).stop 2024-05-22T23:04:02.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.072+0000 7f2ac9aca640 1 -- 172.21.15.84:0/1117517134 shutdown_connections 2024-05-22T23:04:02.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.072+0000 7f2ac9aca640 1 --2- 172.21.15.84:0/1117517134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ac410be40 0x7f2ac410e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.072+0000 7f2ac9aca640 1 --2- 172.21.15.84:0/1117517134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ac4103420 0x7f2ac410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.072+0000 7f2ac9aca640 1 --2- 172.21.15.84:0/1117517134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac4102b00 0x7f2ac4102ee0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.072+0000 7f2ac9aca640 1 -- 172.21.15.84:0/1117517134 >> 172.21.15.84:0/1117517134 conn(0x7f2ac4077870 msgr2=0x7f2ac4077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.072+0000 7f2ac9aca640 1 -- 172.21.15.84:0/1117517134 shutdown_connections 2024-05-22T23:04:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.072+0000 7f2ac9aca640 1 -- 172.21.15.84:0/1117517134 wait complete. 2024-05-22T23:04:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.072+0000 7f2ac9aca640 1 Processor -- start 2024-05-22T23:04:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.072+0000 7f2ac9aca640 1 -- start start 2024-05-22T23:04:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.073+0000 7f2ac9aca640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ac4102b00 0x7f2ac4198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.073+0000 7f2ac9aca640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ac4103420 0x7f2ac41987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.073+0000 7f2ac9aca640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac410be40 0x7f2ac419cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.073+0000 7f2ac9aca640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac4111c80 con 0x7f2ac410be40 2024-05-22T23:04:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.073+0000 7f2ac9aca640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac4111b00 con 0x7f2ac4102b00 2024-05-22T23:04:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.073+0000 7f2ac9aca640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac4111e00 con 0x7f2ac4103420 2024-05-22T23:04:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.073+0000 7f2ac3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac410be40 0x7f2ac419cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:02.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.073+0000 7f2ac37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ac4102b00 0x7f2ac4198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:02.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.073+0000 7f2ac3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac410be40 0x7f2ac419cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43446/0 (socket says 172.21.15.84:43446) 2024-05-22T23:04:02.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.073+0000 7f2ac3fff640 1 -- 172.21.15.84:0/3107494287 learned_addr learned my addr 172.21.15.84:0/3107494287 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:02.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.073+0000 7f2ac2ffd640 1 --2- 172.21.15.84:0/3107494287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ac4103420 0x7f2ac41987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:02.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.073+0000 7f2ac3fff640 1 -- 172.21.15.84:0/3107494287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ac4103420 msgr2=0x7f2ac41987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:02.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.073+0000 7f2ac3fff640 1 --2- 172.21.15.84:0/3107494287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ac4103420 0x7f2ac41987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.073+0000 7f2ac3fff640 1 -- 172.21.15.84:0/3107494287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ac4102b00 msgr2=0x7f2ac4198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:02.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.073+0000 7f2ac3fff640 1 --2- 172.21.15.84:0/3107494287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ac4102b00 0x7f2ac4198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.073+0000 7f2ac3fff640 1 -- 172.21.15.84:0/3107494287 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac419d270 con 0x7f2ac410be40 2024-05-22T23:04:02.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.073+0000 7f2ac2ffd640 1 --2- 172.21.15.84:0/3107494287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ac4103420 0x7f2ac41987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:04:02.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.073+0000 7f2ac3fff640 1 --2- 172.21.15.84:0/3107494287 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac410be40 0x7f2ac419cb40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00eb10 tx=0x7f2abc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:02.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.073+0000 7f2ac0ff9640 1 -- 172.21.15.84:0/3107494287 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc004b20 con 0x7f2ac410be40 2024-05-22T23:04:02.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.073+0000 7f2ac0ff9640 1 -- 172.21.15.84:0/3107494287 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2abc004cc0 con 0x7f2ac410be40 2024-05-22T23:04:02.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.073+0000 7f2ac9aca640 1 -- 172.21.15.84:0/3107494287 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac419d560 con 0x7f2ac410be40 2024-05-22T23:04:02.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.073+0000 7f2ac0ff9640 1 -- 172.21.15.84:0/3107494287 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc010760 con 0x7f2ac410be40 2024-05-22T23:04:02.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.073+0000 7f2ac9aca640 1 -- 172.21.15.84:0/3107494287 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac41a4df0 con 0x7f2ac410be40 2024-05-22T23:04:02.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.074+0000 7f2ac9aca640 1 -- 172.21.15.84:0/3107494287 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ac4068690 con 0x7f2ac410be40 2024-05-22T23:04:02.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.076+0000 7f2ac0ff9640 1 -- 172.21.15.84:0/3107494287 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2abc010900 con 0x7f2ac410be40 2024-05-22T23:04:02.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.076+0000 7f2ac0ff9640 1 --2- 172.21.15.84:0/3107494287 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f2aac0a2d00 0x7f2aac0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:02.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.076+0000 7f2ac37fe640 1 --2- 172.21.15.84:0/3107494287 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f2aac0a2d00 0x7f2aac0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:02.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.076+0000 7f2ac0ff9640 1 -- 172.21.15.84:0/3107494287 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f2abc0cfc80 con 0x7f2ac410be40 2024-05-22T23:04:02.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.077+0000 7f2ac37fe640 1 --2- 172.21.15.84:0/3107494287 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f2aac0a2d00 0x7f2aac0a51c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4009230 tx=0x7f2ab40058d0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:02.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.080+0000 7f2ac0ff9640 1 -- 172.21.15.84:0/3107494287 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2abc053850 con 0x7f2ac410be40 2024-05-22T23:04:02.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.309+0000 7f2ac9aca640 1 -- 172.21.15.84:0/3107494287 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ac402d200 con 0x7f2ac410be40 2024-05-22T23:04:02.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.309+0000 7f2ac0ff9640 1 -- 172.21.15.84:0/3107494287 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7f2abc09c8a0 con 0x7f2ac410be40 2024-05-22T23:04:02.312 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:04:02.312 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:04:02.312 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:04:02.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.311+0000 7f2ac9aca640 1 -- 172.21.15.84:0/3107494287 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f2aac0a2d00 msgr2=0x7f2aac0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:02.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.311+0000 7f2ac9aca640 1 --2- 172.21.15.84:0/3107494287 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f2aac0a2d00 0x7f2aac0a51c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4009230 tx=0x7f2ab40058d0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.311+0000 7f2ac9aca640 1 -- 172.21.15.84:0/3107494287 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac410be40 msgr2=0x7f2ac419cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:02.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.311+0000 7f2ac9aca640 1 --2- 172.21.15.84:0/3107494287 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac410be40 0x7f2ac419cb40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00eb10 tx=0x7f2abc00c2d0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.311+0000 7f2ac9aca640 1 -- 172.21.15.84:0/3107494287 shutdown_connections 2024-05-22T23:04:02.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.311+0000 7f2ac9aca640 1 --2- 172.21.15.84:0/3107494287 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f2aac0a2d00 0x7f2aac0a51c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.311+0000 7f2ac9aca640 1 --2- 172.21.15.84:0/3107494287 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac410be40 0x7f2ac419cb40 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.311+0000 7f2ac9aca640 1 --2- 172.21.15.84:0/3107494287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ac4103420 0x7f2ac41987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.311+0000 7f2ac9aca640 1 --2- 172.21.15.84:0/3107494287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ac4102b00 0x7f2ac4198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.311+0000 7f2ac9aca640 1 -- 172.21.15.84:0/3107494287 >> 172.21.15.84:0/3107494287 conn(0x7f2ac4077870 msgr2=0x7f2ac40fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:02.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.312+0000 7f2ac9aca640 1 -- 172.21.15.84:0/3107494287 shutdown_connections 2024-05-22T23:04:02.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.312+0000 7f2ac9aca640 1 -- 172.21.15.84:0/3107494287 wait complete. 2024-05-22T23:04:02.325 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-22T23:04:02.425 INFO:tasks.workunit.client.0.smithi084.stdout: 1 146081 16.77 MB/sec warmup 48 sec latency 14.874 ms 2024-05-22T23:04:02.430 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.428+0000 7f2a9234c640 1 -- 172.21.15.84:0/1212406902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a8c10be30 msgr2=0x7f2a8c10e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:02.431 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.428+0000 7f2a9234c640 1 --2- 172.21.15.84:0/1212406902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a8c10be30 0x7f2a8c10e2d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009ca0 tx=0x7f2a8402a9e0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.431 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.428+0000 7f2a9234c640 1 -- 172.21.15.84:0/1212406902 shutdown_connections 2024-05-22T23:04:02.431 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.428+0000 7f2a9234c640 1 --2- 172.21.15.84:0/1212406902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a8c10be30 0x7f2a8c10e2d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.431 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.428+0000 7f2a9234c640 1 --2- 172.21.15.84:0/1212406902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8c103410 0x7f2a8c10b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.431 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.428+0000 7f2a9234c640 1 --2- 172.21.15.84:0/1212406902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a8c102af0 0x7f2a8c102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.431 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.428+0000 7f2a9234c640 1 -- 172.21.15.84:0/1212406902 >> 172.21.15.84:0/1212406902 conn(0x7f2a8c077870 msgr2=0x7f2a8c077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:02.431 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.428+0000 7f2a9234c640 1 -- 172.21.15.84:0/1212406902 shutdown_connections 2024-05-22T23:04:02.431 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.428+0000 7f2a9234c640 1 -- 172.21.15.84:0/1212406902 wait complete. 2024-05-22T23:04:02.431 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.428+0000 7f2a9234c640 1 Processor -- start 2024-05-22T23:04:02.431 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.429+0000 7f2a9234c640 1 -- start start 2024-05-22T23:04:02.433 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.429+0000 7f2a9234c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a8c102af0 0x7f2a8c198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:02.433 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.429+0000 7f2a9234c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a8c103410 0x7f2a8c198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:02.433 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.429+0000 7f2a9234c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8c10be30 0x7f2a8c19c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:02.433 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.429+0000 7f2a9234c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c111c30 con 0x7f2a8c103410 2024-05-22T23:04:02.433 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.429+0000 7f2a9234c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c111ab0 con 0x7f2a8c10be30 2024-05-22T23:04:02.433 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.429+0000 7f2a9234c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c111db0 con 0x7f2a8c102af0 2024-05-22T23:04:02.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.429+0000 7f2a8b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a8c103410 0x7f2a8c198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:02.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.429+0000 7f2a8b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a8c103410 0x7f2a8c198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43458/0 (socket says 172.21.15.84:43458) 2024-05-22T23:04:02.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.429+0000 7f2a8b7fe640 1 -- 172.21.15.84:0/560791417 learned_addr learned my addr 172.21.15.84:0/560791417 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:02.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.429+0000 7f2a8bfff640 1 --2- 172.21.15.84:0/560791417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a8c102af0 0x7f2a8c198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:02.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.429+0000 7f2a908e9640 1 --2- 172.21.15.84:0/560791417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8c10be30 0x7f2a8c19c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:02.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.429+0000 7f2a8b7fe640 1 -- 172.21.15.84:0/560791417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a8c102af0 msgr2=0x7f2a8c198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:02.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.429+0000 7f2a8b7fe640 1 --2- 172.21.15.84:0/560791417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a8c102af0 0x7f2a8c198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.429+0000 7f2a8b7fe640 1 -- 172.21.15.84:0/560791417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8c10be30 msgr2=0x7f2a8c19c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:02.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.429+0000 7f2a8b7fe640 1 --2- 172.21.15.84:0/560791417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8c10be30 0x7f2a8c19c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.429+0000 7f2a8b7fe640 1 -- 172.21.15.84:0/560791417 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8c19d0f0 con 0x7f2a8c103410 2024-05-22T23:04:02.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.429+0000 7f2a8b7fe640 1 --2- 172.21.15.84:0/560791417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a8c103410 0x7f2a8c198690 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000e9f0 tx=0x7f2a8000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:02.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.430+0000 7f2a897fa640 1 -- 172.21.15.84:0/560791417 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8000cd80 con 0x7f2a8c103410 2024-05-22T23:04:02.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.430+0000 7f2a897fa640 1 -- 172.21.15.84:0/560791417 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a80010910 con 0x7f2a8c103410 2024-05-22T23:04:02.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.430+0000 7f2a9234c640 1 -- 172.21.15.84:0/560791417 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a8c19d3e0 con 0x7f2a8c103410 2024-05-22T23:04:02.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.430+0000 7f2a897fa640 1 -- 172.21.15.84:0/560791417 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80018990 con 0x7f2a8c103410 2024-05-22T23:04:02.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.430+0000 7f2a9234c640 1 -- 172.21.15.84:0/560791417 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a8c1a4c70 con 0x7f2a8c103410 2024-05-22T23:04:02.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.432+0000 7f2a897fa640 1 -- 172.21.15.84:0/560791417 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2a80010ac0 con 0x7f2a8c103410 2024-05-22T23:04:02.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.433+0000 7f2a95058740 1 Processor -- start 2024-05-22T23:04:02.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.433+0000 7f2a95058740 1 -- start start 2024-05-22T23:04:02.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.433+0000 7f2a95058740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557049dc1f80 0x557049dc2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:02.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.433+0000 7f2a95058740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557049dc28a0 0x557049dcca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:02.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.433+0000 7f2a95058740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dccf70 0x557049dcf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:02.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.433+0000 7f2a95058740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557049d26db0 con 0x557049dccf70 2024-05-22T23:04:02.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.433+0000 7f2a95058740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557049cc5400 con 0x557049dc1f80 2024-05-22T23:04:02.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.433+0000 7f2a95058740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557049d22d70 con 0x557049dc28a0 2024-05-22T23:04:02.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.433+0000 7f2a8b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dccf70 0x557049dcf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:02.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.433+0000 7f2a908e9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557049dc1f80 0x557049dc2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:02.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.433+0000 7f2a8b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dccf70 0x557049dcf360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43470/0 (socket says 172.21.15.84:43470) 2024-05-22T23:04:02.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.433+0000 7f2a8b7fe640 1 -- 172.21.15.84:0/3764286916 learned_addr learned my addr 172.21.15.84:0/3764286916 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:02.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.433+0000 7f2a8bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557049dc28a0 0x557049dcca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:02.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.433+0000 7f2a897fa640 1 --2- 172.21.15.84:0/560791417 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f2a740a2d00 0x7f2a740a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:02.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.433+0000 7f2a8b7fe640 1 -- 172.21.15.84:0/3764286916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557049dc28a0 msgr2=0x557049dcca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:02.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.433+0000 7f2a8b7fe640 1 --2- 172.21.15.84:0/3764286916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557049dc28a0 0x557049dcca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.433+0000 7f2a8b7fe640 1 -- 172.21.15.84:0/3764286916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557049dc1f80 msgr2=0x557049dc2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:02.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.433+0000 7f2a8b7fe640 1 --2- 172.21.15.84:0/3764286916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557049dc1f80 0x557049dc2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.433+0000 7f2a8b7fe640 1 -- 172.21.15.84:0/3764286916 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557049cb72f0 con 0x557049dccf70 2024-05-22T23:04:02.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.433+0000 7f2a908e9640 1 --2- 172.21.15.84:0/3764286916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557049dc1f80 0x557049dc2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:02.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.433+0000 7f2a908e9640 1 --2- 172.21.15.84:0/560791417 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f2a740a2d00 0x7f2a740a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:02.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.433+0000 7f2a897fa640 1 -- 172.21.15.84:0/560791417 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f2a800cf6e0 con 0x7f2a8c103410 2024-05-22T23:04:02.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.434+0000 7f2a8b7fe640 1 --2- 172.21.15.84:0/3764286916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dccf70 0x557049dcf360 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2a800ccda0 tx=0x7f2a80024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a579087cc467085 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:02.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.434+0000 7f2a727fc640 1 -- 172.21.15.84:0/3764286916 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8009b7f0 con 0x557049dccf70 2024-05-22T23:04:02.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.434+0000 7f2a727fc640 1 -- 172.21.15.84:0/3764286916 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a80025170 con 0x557049dccf70 2024-05-22T23:04:02.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.434+0000 7f2a727fc640 1 -- 172.21.15.84:0/3764286916 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8009b610 con 0x557049dccf70 2024-05-22T23:04:02.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.434+0000 7f2a908e9640 1 --2- 172.21.15.84:0/560791417 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f2a740a2d00 0x7f2a740a51c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2a84036550 tx=0x7f2a840095c0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:02.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.434+0000 7f2a95058740 1 -- 172.21.15.84:0/3764286916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dccf70 msgr2=0x557049dcf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:02.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.434+0000 7f2a95058740 1 --2- 172.21.15.84:0/3764286916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dccf70 0x557049dcf360 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2a800ccda0 tx=0x7f2a80024000 comp rx=0 tx=0).stop 2024-05-22T23:04:02.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.435+0000 7f2a95058740 1 -- 172.21.15.84:0/3764286916 shutdown_connections 2024-05-22T23:04:02.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.435+0000 7f2a95058740 1 --2- 172.21.15.84:0/3764286916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dccf70 0x557049dcf360 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.435+0000 7f2a95058740 1 --2- 172.21.15.84:0/3764286916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557049dc28a0 0x557049dcca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.435+0000 7f2a95058740 1 --2- 172.21.15.84:0/3764286916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557049dc1f80 0x557049dc2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.435+0000 7f2a95058740 1 -- 172.21.15.84:0/3764286916 >> 172.21.15.84:0/3764286916 conn(0x557049dbe2a0 msgr2=0x557049dbe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:02.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.435+0000 7f2a95058740 1 -- 172.21.15.84:0/3764286916 shutdown_connections 2024-05-22T23:04:02.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.435+0000 7f2a95058740 1 -- 172.21.15.84:0/3764286916 wait complete. 2024-05-22T23:04:02.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.435+0000 7f2a95058740 1 -- start start 2024-05-22T23:04:02.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.435+0000 7f2a95058740 1 Processor -- start 2024-05-22T23:04:02.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.435+0000 7f2a95058740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dc1f80 0x557049dc9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:02.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.435+0000 7f2a95058740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557049dc28a0 0x557049dca430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:02.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.435+0000 7f2a95058740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557049dccf70 0x557049dca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:02.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.435+0000 7f2a95058740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557049dcfd00 con 0x557049dc1f80 2024-05-22T23:04:02.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.435+0000 7f2a95058740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557049ce12e0 con 0x557049dccf70 2024-05-22T23:04:02.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.435+0000 7f2a95058740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557049dcfe80 con 0x557049dc28a0 2024-05-22T23:04:02.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.435+0000 7f2a95058740 10 client.0 _ll_register_callbacks cb 0x557049ceaa70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:04:02.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.435+0000 7f2a8bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dc1f80 0x557049dc9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:02.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.436+0000 7f2a8bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dc1f80 0x557049dc9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43480/0 (socket says 172.21.15.84:43480) 2024-05-22T23:04:02.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.436+0000 7f2a8bfff640 1 -- 172.21.15.84:0/1523619737 learned_addr learned my addr 172.21.15.84:0/1523619737 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:02.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.436+0000 7f2a908e9640 1 --2- 172.21.15.84:0/1523619737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557049dccf70 0x557049dca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:02.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.436+0000 7f2a8b7fe640 1 --2- 172.21.15.84:0/1523619737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557049dc28a0 0x557049dca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:02.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.436+0000 7f2a8bfff640 1 -- 172.21.15.84:0/1523619737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557049dc28a0 msgr2=0x557049dca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:02.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.436+0000 7f2a8bfff640 1 --2- 172.21.15.84:0/1523619737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557049dc28a0 0x557049dca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.436+0000 7f2a8bfff640 1 -- 172.21.15.84:0/1523619737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557049dccf70 msgr2=0x557049dca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:02.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.436+0000 7f2a8bfff640 1 --2- 172.21.15.84:0/1523619737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557049dccf70 0x557049dca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.436+0000 7f2a8bfff640 1 -- 172.21.15.84:0/1523619737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557049a3a3f0 con 0x557049dc1f80 2024-05-22T23:04:02.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.436+0000 7f2a8bfff640 1 --2- 172.21.15.84:0/1523619737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dc1f80 0x557049dc9ef0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00b990 tx=0x7f2a7c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45014656995de5e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:02.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.436+0000 7f2a6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:04:02.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.436+0000 7f2a6b7fe640 1 -- 172.21.15.84:0/1523619737 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c011440 con 0x557049dc1f80 2024-05-22T23:04:02.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.436+0000 7f2a6b7fe640 1 -- 172.21.15.84:0/1523619737 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f2a7c0115e0 con 0x557049dc1f80 2024-05-22T23:04:02.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.436+0000 7f2a95058740 1 -- 172.21.15.84:0/1523619737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a80025170 con 0x557049dc1f80 2024-05-22T23:04:02.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.436+0000 7f2a6b7fe640 1 -- 172.21.15.84:0/1523619737 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f2a7c0117e0 con 0x557049dc1f80 2024-05-22T23:04:02.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.436+0000 7f2a95058740 10 client.15756 fetch_fsmap learned FSMap version 18 2024-05-22T23:04:02.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.436+0000 7f2a95058740 1 -- 172.21.15.84:0/1523619737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x557049b047d0 con 0x557049dc1f80 2024-05-22T23:04:02.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.436+0000 7f2a6b7fe640 1 -- 172.21.15.84:0/1523619737 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a80025170 con 0x557049dc1f80 2024-05-22T23:04:02.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.437+0000 7f2a6b7fe640 1 -- 172.21.15.84:0/1523619737 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f2a7c01b9b0 con 0x557049dc1f80 2024-05-22T23:04:02.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.437+0000 7f2a95058740 10 client.15756 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:04:02.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.437+0000 7f2a95058740 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:04:02.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.437+0000 7f2a95058740 20 client.15756 populate_metadata read hostname 'smithi084' 2024-05-22T23:04:02.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.437+0000 7f2a95058740 1 --2- 172.21.15.84:0/1523619737 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x557049ddd2b0 0x557049ddf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:02.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.437+0000 7f2a95058740 4 client.15756 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:04:02.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.437+0000 7f2a95058740 1 -- 172.21.15.84:0/1523619737 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557049d344e0 con 0x557049ddd2b0 2024-05-22T23:04:02.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.437+0000 7f2a8b7fe640 1 --2- 172.21.15.84:0/1523619737 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x557049ddd2b0 0x557049ddf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:02.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.437+0000 7f2a6b7fe640 10 client.15756 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:02.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.437+0000 7f2a8b7fe640 1 --2- 172.21.15.84:0/1523619737 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x557049ddd2b0 0x557049ddf720 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33ccc087bfb5f7bf server_cookie=fa3b85d04ab23295 in_seq=0 out_seq=0 2024-05-22T23:04:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.438+0000 7f2a6b7fe640 1 -- 172.21.15.84:0/1523619737 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557049d344e0 con 0x557049ddd2b0 2024-05-22T23:04:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.438+0000 7f2a6b7fe640 10 client.15756 handle_command_reply: tid=0 2024-05-22T23:04:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.438+0000 7f2a95058740 1 client.15756 shutdown 2024-05-22T23:04:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.438+0000 7f2a95058740 1 -- 172.21.15.84:0/1523619737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dc1f80 msgr2=0x557049dc9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.438+0000 7f2a95058740 1 --2- 172.21.15.84:0/1523619737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dc1f80 0x557049dc9ef0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00b990 tx=0x7f2a7c00be50 comp rx=0 tx=0).stop 2024-05-22T23:04:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.438+0000 7f2a95058740 1 -- 172.21.15.84:0/1523619737 shutdown_connections 2024-05-22T23:04:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.438+0000 7f2a95058740 1 --2- 172.21.15.84:0/1523619737 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x557049ddd2b0 0x557049ddf720 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.438+0000 7f2a95058740 1 --2- 172.21.15.84:0/1523619737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557049dccf70 0x557049dca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.438+0000 7f2a6b7fe640 0 client.15756 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.438+0000 7f2a95058740 1 --2- 172.21.15.84:0/1523619737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557049dc28a0 0x557049dca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.438+0000 7f2a95058740 1 --2- 172.21.15.84:0/1523619737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dc1f80 0x557049dc9ef0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.438+0000 7f2a95058740 1 -- 172.21.15.84:0/1523619737 >> 172.21.15.84:0/1523619737 conn(0x557049dbe2a0 msgr2=0x557049dc40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.438+0000 7f2a95058740 1 -- 172.21.15.84:0/1523619737 shutdown_connections 2024-05-22T23:04:02.448 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.438+0000 7f2a95058740 1 -- 172.21.15.84:0/1523619737 wait complete. 2024-05-22T23:04:02.448 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.438+0000 7f2a95058740 20 client.15756 trim_cache size 0 max 16384 2024-05-22T23:04:02.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.453+0000 7f2a95058740 1 Processor -- start 2024-05-22T23:04:02.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.454+0000 7f2a95058740 1 -- start start 2024-05-22T23:04:02.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.454+0000 7f2a95058740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557049dc1f80 0x557049dd8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:02.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.454+0000 7f2a95058740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557049dc28a0 0x557049dca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:02.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.454+0000 7f2a95058740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dccf70 0x557049de4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:02.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.454+0000 7f2a95058740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1ab670 con 0x557049dccf70 2024-05-22T23:04:02.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.454+0000 7f2a95058740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1adef0 con 0x557049dc28a0 2024-05-22T23:04:02.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.454+0000 7f2a95058740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1abce0 con 0x557049dc1f80 2024-05-22T23:04:02.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.454+0000 7f2a8bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557049dc1f80 0x557049dd8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:02.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.454+0000 7f2a908e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dccf70 0x557049de4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:02.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.454+0000 7f2a8b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557049dc28a0 0x557049dca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:02.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.454+0000 7f2a8bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557049dc1f80 0x557049dd8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:60556/0 (socket says 172.21.15.84:60556) 2024-05-22T23:04:02.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.454+0000 7f2a8bfff640 1 -- 172.21.15.84:0/3650984805 learned_addr learned my addr 172.21.15.84:0/3650984805 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:02.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.454+0000 7f2a8bfff640 1 -- 172.21.15.84:0/3650984805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557049dc28a0 msgr2=0x557049dca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:02.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.454+0000 7f2a8bfff640 1 --2- 172.21.15.84:0/3650984805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557049dc28a0 0x557049dca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.454+0000 7f2a8bfff640 1 -- 172.21.15.84:0/3650984805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dccf70 msgr2=0x557049de4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:02.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.454+0000 7f2a8bfff640 1 --2- 172.21.15.84:0/3650984805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dccf70 0x557049de4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.454+0000 7f2a8bfff640 1 -- 172.21.15.84:0/3650984805 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557049db9c60 con 0x557049dc1f80 2024-05-22T23:04:02.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.455+0000 7f2a8bfff640 1 --2- 172.21.15.84:0/3650984805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557049dc1f80 0x557049dd8bf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00b960 tx=0x7f2a7c005f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62154b85bfced1f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:02.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.455+0000 7f2a71ffb640 1 -- 172.21.15.84:0/3650984805 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c011440 con 0x557049dc1f80 2024-05-22T23:04:02.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.455+0000 7f2a71ffb640 1 -- 172.21.15.84:0/3650984805 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a7c0115e0 con 0x557049dc1f80 2024-05-22T23:04:02.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.455+0000 7f2a71ffb640 1 -- 172.21.15.84:0/3650984805 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c01b680 con 0x557049dc1f80 2024-05-22T23:04:02.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.456+0000 7f2a95058740 1 -- 172.21.15.84:0/3650984805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557049dc1f80 msgr2=0x557049dd8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:02.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.456+0000 7f2a95058740 1 --2- 172.21.15.84:0/3650984805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557049dc1f80 0x557049dd8bf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00b960 tx=0x7f2a7c005f30 comp rx=0 tx=0).stop 2024-05-22T23:04:02.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.456+0000 7f2a95058740 1 -- 172.21.15.84:0/3650984805 shutdown_connections 2024-05-22T23:04:02.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.456+0000 7f2a95058740 1 --2- 172.21.15.84:0/3650984805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dccf70 0x557049de4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.456+0000 7f2a95058740 1 --2- 172.21.15.84:0/3650984805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557049dc28a0 0x557049dca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.456+0000 7f2a95058740 1 --2- 172.21.15.84:0/3650984805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557049dc1f80 0x557049dd8bf0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.456+0000 7f2a95058740 1 -- 172.21.15.84:0/3650984805 >> 172.21.15.84:0/3650984805 conn(0x557049dbe2a0 msgr2=0x557049dbd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:02.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.456+0000 7f2a95058740 1 -- 172.21.15.84:0/3650984805 shutdown_connections 2024-05-22T23:04:02.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.456+0000 7f2a95058740 1 -- 172.21.15.84:0/3650984805 wait complete. 2024-05-22T23:04:02.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.456+0000 7f2a95058740 1 -- start start 2024-05-22T23:04:02.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.456+0000 7f2a95058740 1 Processor -- start 2024-05-22T23:04:02.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.456+0000 7f2a95058740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557049dc1f80 0x557049dc6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:02.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.457+0000 7f2a95058740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557049dc28a0 0x557049dc71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:02.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.457+0000 7f2a95058740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dccf70 0x557049dc7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:02.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.457+0000 7f2a95058740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1ac3b0 con 0x557049dccf70 2024-05-22T23:04:02.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.457+0000 7f2a95058740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1ad820 con 0x557049dc28a0 2024-05-22T23:04:02.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.457+0000 7f2a95058740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1aca80 con 0x557049dc1f80 2024-05-22T23:04:02.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.457+0000 7f2a95058740 10 client.0 _ll_register_callbacks cb 0x7f2a7c0042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:04:02.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.457+0000 7f2a908e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dccf70 0x557049dc7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:02.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.457+0000 7f2a908e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dccf70 0x557049dc7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43500/0 (socket says 172.21.15.84:43500) 2024-05-22T23:04:02.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.457+0000 7f2a908e9640 1 -- 172.21.15.84:0/3911663819 learned_addr learned my addr 172.21.15.84:0/3911663819 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:02.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.457+0000 7f2a8b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557049dc28a0 0x557049dc71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:02.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.457+0000 7f2a908e9640 1 -- 172.21.15.84:0/3911663819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557049dc1f80 msgr2=0x557049dc6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:02.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.457+0000 7f2a908e9640 1 --2- 172.21.15.84:0/3911663819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557049dc1f80 0x557049dc6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.457+0000 7f2a908e9640 1 -- 172.21.15.84:0/3911663819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557049dc28a0 msgr2=0x557049dc71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:02.463 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.457+0000 7f2a908e9640 1 --2- 172.21.15.84:0/3911663819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557049dc28a0 0x557049dc71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.457+0000 7f2a908e9640 1 -- 172.21.15.84:0/3911663819 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557049e46d40 con 0x557049dccf70 2024-05-22T23:04:02.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.457+0000 7f2a8b7fe640 1 --2- 172.21.15.84:0/3911663819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557049dc28a0 0x557049dc71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:02.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.457+0000 7f2a908e9640 1 --2- 172.21.15.84:0/3911663819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dccf70 0x557049dc7710 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2a84035000 tx=0x7f2a8403a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64cddf46fcc58f2b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:02.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.457+0000 7f2a6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:04:02.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.457+0000 7f2a6b7fe640 1 -- 172.21.15.84:0/3911663819 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8404a040 con 0x557049dccf70 2024-05-22T23:04:02.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.457+0000 7f2a6b7fe640 1 -- 172.21.15.84:0/3911663819 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f2a8402fd90 con 0x557049dccf70 2024-05-22T23:04:02.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.457+0000 7f2a95058740 1 -- 172.21.15.84:0/3911663819 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a7c0115e0 con 0x557049dccf70 2024-05-22T23:04:02.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.457+0000 7f2a6b7fe640 1 -- 172.21.15.84:0/3911663819 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f2a84033040 con 0x557049dccf70 2024-05-22T23:04:02.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.457+0000 7f2a95058740 10 client.15762 fetch_fsmap learned FSMap version 18 2024-05-22T23:04:02.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.457+0000 7f2a95058740 1 -- 172.21.15.84:0/3911663819 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x557049e48040 con 0x557049dccf70 2024-05-22T23:04:02.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.458+0000 7f2a6b7fe640 1 -- 172.21.15.84:0/3911663819 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a7c0115e0 con 0x557049dccf70 2024-05-22T23:04:02.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.458+0000 7f2a6b7fe640 1 -- 172.21.15.84:0/3911663819 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f2a840387b0 con 0x557049dccf70 2024-05-22T23:04:02.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.458+0000 7f2a95058740 10 client.15762 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:04:02.464 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.458+0000 7f2a95058740 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:04:02.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.458+0000 7f2a95058740 20 client.15762 populate_metadata read hostname 'smithi084' 2024-05-22T23:04:02.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.458+0000 7f2a95058740 1 --2- 172.21.15.84:0/3911663819 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x557049e48710 0x557049e4ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:02.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.458+0000 7f2a95058740 4 client.15762 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-22T23:04:02.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.458+0000 7f2a95058740 1 -- 172.21.15.84:0/3911663819 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557049d2e830 con 0x557049e48710 2024-05-22T23:04:02.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.458+0000 7f2a8bfff640 1 --2- 172.21.15.84:0/3911663819 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x557049e48710 0x557049e4ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:02.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.458+0000 7f2a6b7fe640 10 client.15762 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:02.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.458+0000 7f2a8bfff640 1 --2- 172.21.15.84:0/3911663819 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x557049e48710 0x557049e4ab50 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb211dce09d5d4af server_cookie=c36df1c340820451 in_seq=0 out_seq=0 2024-05-22T23:04:02.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.461+0000 7f2a6b7fe640 1 -- 172.21.15.84:0/3911663819 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557049d2e830 con 0x557049e48710 2024-05-22T23:04:02.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.461+0000 7f2a6b7fe640 10 client.15762 handle_command_reply: tid=0 2024-05-22T23:04:02.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.461+0000 7f2a95058740 1 client.15762 shutdown 2024-05-22T23:04:02.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.462+0000 7f2a95058740 1 -- 172.21.15.84:0/3911663819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dccf70 msgr2=0x557049dc7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:02.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.462+0000 7f2a95058740 1 --2- 172.21.15.84:0/3911663819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dccf70 0x557049dc7710 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2a84035000 tx=0x7f2a8403a040 comp rx=0 tx=0).stop 2024-05-22T23:04:02.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.462+0000 7f2a95058740 1 -- 172.21.15.84:0/3911663819 shutdown_connections 2024-05-22T23:04:02.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.462+0000 7f2a95058740 1 --2- 172.21.15.84:0/3911663819 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x557049e48710 0x557049e4ab50 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.462+0000 7f2a95058740 1 --2- 172.21.15.84:0/3911663819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557049dccf70 0x557049dc7710 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.462+0000 7f2a95058740 1 --2- 172.21.15.84:0/3911663819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557049dc28a0 0x557049dc71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.462+0000 7f2a95058740 1 --2- 172.21.15.84:0/3911663819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557049dc1f80 0x557049dc6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.462+0000 7f2a6b7fe640 0 client.15762 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:02.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.462+0000 7f2a95058740 1 -- 172.21.15.84:0/3911663819 >> 172.21.15.84:0/3911663819 conn(0x557049dbe2a0 msgr2=0x557049dd8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:02.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.462+0000 7f2a95058740 1 -- 172.21.15.84:0/3911663819 shutdown_connections 2024-05-22T23:04:02.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.462+0000 7f2a95058740 1 -- 172.21.15.84:0/3911663819 wait complete. 2024-05-22T23:04:02.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.462+0000 7f2a95058740 20 client.15762 trim_cache size 0 max 16384 2024-05-22T23:04:02.467 INFO:teuthology.orchestra.run.smithi084.stdout:[] 2024-05-22T23:04:02.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.462+0000 7f2a9234c640 1 -- 172.21.15.84:0/560791417 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f2a740a2d00 msgr2=0x7f2a740a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:02.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.462+0000 7f2a9234c640 1 --2- 172.21.15.84:0/560791417 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f2a740a2d00 0x7f2a740a51c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2a84036550 tx=0x7f2a840095c0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.462+0000 7f2a9234c640 1 -- 172.21.15.84:0/560791417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a8c103410 msgr2=0x7f2a8c198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:02.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.462+0000 7f2a9234c640 1 --2- 172.21.15.84:0/560791417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a8c103410 0x7f2a8c198690 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000e9f0 tx=0x7f2a8000eeb0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.462+0000 7f2a9234c640 1 -- 172.21.15.84:0/560791417 shutdown_connections 2024-05-22T23:04:02.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.462+0000 7f2a9234c640 1 --2- 172.21.15.84:0/560791417 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f2a740a2d00 0x7f2a740a51c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.462+0000 7f2a9234c640 1 --2- 172.21.15.84:0/560791417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8c10be30 0x7f2a8c19c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.462+0000 7f2a9234c640 1 --2- 172.21.15.84:0/560791417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a8c103410 0x7f2a8c198690 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.462+0000 7f2a9234c640 1 --2- 172.21.15.84:0/560791417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a8c102af0 0x7f2a8c198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:02.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.462+0000 7f2a9234c640 1 -- 172.21.15.84:0/560791417 >> 172.21.15.84:0/560791417 conn(0x7f2a8c077870 msgr2=0x7f2a8c0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:02.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.462+0000 7f2a9234c640 1 -- 172.21.15.84:0/560791417 shutdown_connections 2024-05-22T23:04:02.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:02.462+0000 7f2a9234c640 1 -- 172.21.15.84:0/560791417 wait complete. 2024-05-22T23:04:03.425 INFO:tasks.workunit.client.0.smithi084.stdout: 1 149405 16.82 MB/sec warmup 49 sec latency 14.654 ms 2024-05-22T23:04:03.734 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-22T23:04:03.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:04:03.734 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-22T23:04:03.735 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:03 smithi204 ceph-mon[36211]: pgmap v69: 97 pgs: 97 active+clean; 75 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.5 MiB/s wr, 407 op/s 2024-05-22T23:04:03.735 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:03 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3107494287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:04:03.736 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:03 smithi096 ceph-mon[36392]: pgmap v69: 97 pgs: 97 active+clean; 75 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.5 MiB/s wr, 407 op/s 2024-05-22T23:04:03.736 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:03 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3107494287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:04:03.736 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:03 smithi084 ceph-mon[32186]: pgmap v69: 97 pgs: 97 active+clean; 75 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.5 MiB/s wr, 407 op/s 2024-05-22T23:04:03.736 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:03 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3107494287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:04:03.844 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.842+0000 7f4121e9e640 1 -- 172.21.15.84:0/1361012599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f411c101110 msgr2=0x7f411c10d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:03.845 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.842+0000 7f4121e9e640 1 --2- 172.21.15.84:0/1361012599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f411c101110 0x7f411c10d620 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4114009be0 tx=0x7f411402a6c0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.845 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.842+0000 7f4121e9e640 1 -- 172.21.15.84:0/1361012599 shutdown_connections 2024-05-22T23:04:03.845 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.842+0000 7f4121e9e640 1 --2- 172.21.15.84:0/1361012599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f411c101110 0x7f411c10d620 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.845 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.842+0000 7f4121e9e640 1 --2- 172.21.15.84:0/1361012599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f411c100750 0x7f411c100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.845 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.842+0000 7f4121e9e640 1 --2- 172.21.15.84:0/1361012599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f411c106760 0x7f411c106b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.845 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.842+0000 7f4121e9e640 1 -- 172.21.15.84:0/1361012599 >> 172.21.15.84:0/1361012599 conn(0x7f411c076af0 msgr2=0x7f411c076f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:03.845 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.842+0000 7f4121e9e640 1 -- 172.21.15.84:0/1361012599 shutdown_connections 2024-05-22T23:04:03.845 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.842+0000 7f4121e9e640 1 -- 172.21.15.84:0/1361012599 wait complete. 2024-05-22T23:04:03.845 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.842+0000 7f4121e9e640 1 Processor -- start 2024-05-22T23:04:03.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.843+0000 7f4121e9e640 1 -- start start 2024-05-22T23:04:03.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.843+0000 7f4121e9e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f411c100750 0x7f411c196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:03.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.843+0000 7f4121e9e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f411c101110 0x7f411c197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:03.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.843+0000 7f4121e9e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f411c106760 0x7f411c19b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:03.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.843+0000 7f4121e9e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f411c110b20 con 0x7f411c100750 2024-05-22T23:04:03.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.843+0000 7f4121e9e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f411c1109a0 con 0x7f411c101110 2024-05-22T23:04:03.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.843+0000 7f4121e9e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f411c110ca0 con 0x7f411c106760 2024-05-22T23:04:03.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.843+0000 7f411b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f411c100750 0x7f411c196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:03.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.843+0000 7f411b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f411c100750 0x7f411c196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43510/0 (socket says 172.21.15.84:43510) 2024-05-22T23:04:03.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.843+0000 7f411b7fe640 1 -- 172.21.15.84:0/3437837751 learned_addr learned my addr 172.21.15.84:0/3437837751 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:03.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.843+0000 7f411affd640 1 --2- 172.21.15.84:0/3437837751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f411c101110 0x7f411c197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:03.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.843+0000 7f411bfff640 1 --2- 172.21.15.84:0/3437837751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f411c106760 0x7f411c19b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:03.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.843+0000 7f411b7fe640 1 -- 172.21.15.84:0/3437837751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f411c106760 msgr2=0x7f411c19b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:03.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.843+0000 7f411b7fe640 1 --2- 172.21.15.84:0/3437837751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f411c106760 0x7f411c19b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.843+0000 7f411b7fe640 1 -- 172.21.15.84:0/3437837751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f411c101110 msgr2=0x7f411c197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:03.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.843+0000 7f411b7fe640 1 --2- 172.21.15.84:0/3437837751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f411c101110 0x7f411c197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.843+0000 7f411b7fe640 1 -- 172.21.15.84:0/3437837751 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c19bfb0 con 0x7f411c100750 2024-05-22T23:04:03.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.843+0000 7f411b7fe640 1 --2- 172.21.15.84:0/3437837751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f411c100750 0x7f411c196fc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f410c00e9c0 tx=0x7f410c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:03.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.844+0000 7f4118ff9640 1 -- 172.21.15.84:0/3437837751 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c00ccc0 con 0x7f411c100750 2024-05-22T23:04:03.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.844+0000 7f4118ff9640 1 -- 172.21.15.84:0/3437837751 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f410c005500 con 0x7f411c100750 2024-05-22T23:04:03.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.844+0000 7f4118ff9640 1 -- 172.21.15.84:0/3437837751 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c010430 con 0x7f411c100750 2024-05-22T23:04:03.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.844+0000 7f4121e9e640 1 -- 172.21.15.84:0/3437837751 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f411c19c2a0 con 0x7f411c100750 2024-05-22T23:04:03.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.844+0000 7f4121e9e640 1 -- 172.21.15.84:0/3437837751 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f411c1a3b30 con 0x7f411c100750 2024-05-22T23:04:03.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.847+0000 7f4118ff9640 1 -- 172.21.15.84:0/3437837751 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f410c0046c0 con 0x7f411c100750 2024-05-22T23:04:03.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.847+0000 7f4124bac740 1 Processor -- start 2024-05-22T23:04:03.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.847+0000 7f4124bac740 1 -- start start 2024-05-22T23:04:03.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.847+0000 7f4124bac740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649041bf900 0x5649041bfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:03.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.847+0000 7f4124bac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649041c02b0 0x5649041cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:03.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.847+0000 7f4124bac740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041cce50 0x5649041cf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:03.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.848+0000 7f4124bac740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564904123a40 con 0x5649041cce50 2024-05-22T23:04:03.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.848+0000 7f4124bac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5649040c2440 con 0x5649041c02b0 2024-05-22T23:04:03.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.848+0000 7f4124bac740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56490411fa00 con 0x5649041bf900 2024-05-22T23:04:03.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.848+0000 7f4118ff9640 1 --2- 172.21.15.84:0/3437837751 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f41040a2e50 0x7f41040a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:03.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.848+0000 7f411affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041cce50 0x5649041cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:03.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.848+0000 7f411b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649041c02b0 0x5649041cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:03.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.848+0000 7f411bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649041bf900 0x5649041bfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:03.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.848+0000 7f411bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649041bf900 0x5649041bfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:60594/0 (socket says 172.21.15.84:60594) 2024-05-22T23:04:03.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.848+0000 7f411bfff640 1 -- 172.21.15.84:0/3133702825 learned_addr learned my addr 172.21.15.84:0/3133702825 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:03.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.848+0000 7f411bfff640 1 --2- 172.21.15.84:0/3437837751 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f41040a2e50 0x7f41040a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:03.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.848+0000 7f4118ff9640 1 -- 172.21.15.84:0/3437837751 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f410c0cf5b0 con 0x7f411c100750 2024-05-22T23:04:03.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.848+0000 7f411affd640 1 -- 172.21.15.84:0/3133702825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649041bf900 msgr2=0x5649041bfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:03.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.848+0000 7f411affd640 1 --2- 172.21.15.84:0/3133702825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649041bf900 0x5649041bfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.848+0000 7f411affd640 1 -- 172.21.15.84:0/3133702825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649041c02b0 msgr2=0x5649041cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:03.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.848+0000 7f411affd640 1 --2- 172.21.15.84:0/3133702825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649041c02b0 0x5649041cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.848+0000 7f411affd640 1 -- 172.21.15.84:0/3133702825 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649040cc2d0 con 0x5649041cce50 2024-05-22T23:04:03.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.848+0000 7f411b7fe640 1 --2- 172.21.15.84:0/3133702825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649041c02b0 0x5649041cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:03.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.848+0000 7f411bfff640 1 --2- 172.21.15.84:0/3437837751 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f41040a2e50 0x7f41040a5310 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f41140309c0 tx=0x7f411402a6c0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:03.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.848+0000 7f411affd640 1 --2- 172.21.15.84:0/3133702825 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041cce50 0x5649041cf240 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f411000bbc0 tx=0x7f41100107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254aede61f662009 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:03.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.848+0000 7f4101ffb640 1 -- 172.21.15.84:0/3133702825 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110004030 con 0x5649041cce50 2024-05-22T23:04:03.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.848+0000 7f4101ffb640 1 -- 172.21.15.84:0/3133702825 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41100041d0 con 0x5649041cce50 2024-05-22T23:04:03.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.848+0000 7f4101ffb640 1 -- 172.21.15.84:0/3133702825 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110011640 con 0x5649041cce50 2024-05-22T23:04:03.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.849+0000 7f4124bac740 1 -- 172.21.15.84:0/3133702825 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041cce50 msgr2=0x5649041cf240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:03.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.849+0000 7f4124bac740 1 --2- 172.21.15.84:0/3133702825 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041cce50 0x5649041cf240 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f411000bbc0 tx=0x7f41100107a0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.849+0000 7f4124bac740 1 -- 172.21.15.84:0/3133702825 shutdown_connections 2024-05-22T23:04:03.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.849+0000 7f4124bac740 1 --2- 172.21.15.84:0/3133702825 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041cce50 0x5649041cf240 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.849+0000 7f4124bac740 1 --2- 172.21.15.84:0/3133702825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649041c02b0 0x5649041cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.849+0000 7f4124bac740 1 --2- 172.21.15.84:0/3133702825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649041bf900 0x5649041bfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.849+0000 7f4124bac740 1 -- 172.21.15.84:0/3133702825 >> 172.21.15.84:0/3133702825 conn(0x5649041bbf70 msgr2=0x5649041bc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:03.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.849+0000 7f4124bac740 1 -- 172.21.15.84:0/3133702825 shutdown_connections 2024-05-22T23:04:03.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.849+0000 7f4124bac740 1 -- 172.21.15.84:0/3133702825 wait complete. 2024-05-22T23:04:03.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.849+0000 7f4124bac740 1 -- start start 2024-05-22T23:04:03.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.850+0000 7f4124bac740 1 Processor -- start 2024-05-22T23:04:03.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.850+0000 7f4124bac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649041bf900 0x5649041c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:03.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.850+0000 7f4124bac740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041c02b0 0x5649041ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:03.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.850+0000 7f4124bac740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649041cce50 0x5649041ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:03.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.850+0000 7f4124bac740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5649041cfc40 con 0x5649041c02b0 2024-05-22T23:04:03.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.850+0000 7f4124bac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5649040ad5f0 con 0x5649041bf900 2024-05-22T23:04:03.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.850+0000 7f4124bac740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5649041cfdc0 con 0x5649041cce50 2024-05-22T23:04:03.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.850+0000 7f4124bac740 10 client.0 _ll_register_callbacks cb 0x5649040a8cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:04:03.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.850+0000 7f411affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041c02b0 0x5649041ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:03.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.850+0000 7f411b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649041bf900 0x5649041c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:03.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.850+0000 7f411affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041c02b0 0x5649041ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43530/0 (socket says 172.21.15.84:43530) 2024-05-22T23:04:03.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.850+0000 7f411affd640 1 -- 172.21.15.84:0/2608946112 learned_addr learned my addr 172.21.15.84:0/2608946112 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:03.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.850+0000 7f411bfff640 1 --2- 172.21.15.84:0/2608946112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649041cce50 0x5649041ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:03.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.850+0000 7f411affd640 1 -- 172.21.15.84:0/2608946112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649041cce50 msgr2=0x5649041ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:03.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.850+0000 7f411affd640 1 --2- 172.21.15.84:0/2608946112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649041cce50 0x5649041ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.850+0000 7f411affd640 1 -- 172.21.15.84:0/2608946112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649041bf900 msgr2=0x5649041c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:03.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.850+0000 7f411affd640 1 --2- 172.21.15.84:0/2608946112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649041bf900 0x5649041c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.850+0000 7f411affd640 1 -- 172.21.15.84:0/2608946112 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564903eff7e0 con 0x5649041c02b0 2024-05-22T23:04:03.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.851+0000 7f411affd640 1 --2- 172.21.15.84:0/2608946112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041c02b0 0x5649041ca2f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4110011010 tx=0x7f4110003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a93cfcfd22c8a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:03.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.851+0000 7f40faffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:04:03.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.851+0000 7f40faffd640 1 -- 172.21.15.84:0/2608946112 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411002a060 con 0x5649041c02b0 2024-05-22T23:04:03.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.851+0000 7f40faffd640 1 -- 172.21.15.84:0/2608946112 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f411000bcb0 con 0x5649041c02b0 2024-05-22T23:04:03.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.851+0000 7f4124bac740 1 -- 172.21.15.84:0/2608946112 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41100041d0 con 0x5649041c02b0 2024-05-22T23:04:03.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.851+0000 7f40faffd640 1 -- 172.21.15.84:0/2608946112 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f41100035f0 con 0x5649041c02b0 2024-05-22T23:04:03.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.851+0000 7f4124bac740 10 client.15774 fetch_fsmap learned FSMap version 18 2024-05-22T23:04:03.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.851+0000 7f4124bac740 1 -- 172.21.15.84:0/2608946112 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x564903f5ff70 con 0x5649041c02b0 2024-05-22T23:04:03.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.851+0000 7f40faffd640 1 -- 172.21.15.84:0/2608946112 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41100041d0 con 0x5649041c02b0 2024-05-22T23:04:03.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.851+0000 7f40faffd640 1 -- 172.21.15.84:0/2608946112 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f41100046e0 con 0x5649041c02b0 2024-05-22T23:04:03.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.851+0000 7f4124bac740 10 client.15774 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:04:03.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.851+0000 7f4124bac740 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:04:03.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.851+0000 7f4124bac740 20 client.15774 populate_metadata read hostname 'smithi084' 2024-05-22T23:04:03.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.851+0000 7f4124bac740 1 --2- 172.21.15.84:0/2608946112 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5649041dd060 0x5649041df450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:03.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.851+0000 7f4124bac740 4 client.15774 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:04:03.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.851+0000 7f4124bac740 1 -- 172.21.15.84:0/2608946112 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564904137460 con 0x5649041dd060 2024-05-22T23:04:03.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.851+0000 7f411b7fe640 1 --2- 172.21.15.84:0/2608946112 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5649041dd060 0x5649041df450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:03.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.852+0000 7f40faffd640 10 client.15774 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:03.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.852+0000 7f411b7fe640 1 --2- 172.21.15.84:0/2608946112 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5649041dd060 0x5649041df450 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9018c8a355d8091d server_cookie=b04736f065b5b507 in_seq=0 out_seq=0 2024-05-22T23:04:03.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.853+0000 7f40faffd640 1 -- 172.21.15.84:0/2608946112 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564904137460 con 0x5649041dd060 2024-05-22T23:04:03.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.853+0000 7f40faffd640 10 client.15774 handle_command_reply: tid=0 2024-05-22T23:04:03.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.853+0000 7f4124bac740 1 client.15774 shutdown 2024-05-22T23:04:03.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.853+0000 7f4124bac740 1 -- 172.21.15.84:0/2608946112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041c02b0 msgr2=0x5649041ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:03.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.853+0000 7f4124bac740 1 --2- 172.21.15.84:0/2608946112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041c02b0 0x5649041ca2f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4110011010 tx=0x7f4110003be0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.854+0000 7f4124bac740 1 -- 172.21.15.84:0/2608946112 shutdown_connections 2024-05-22T23:04:03.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.854+0000 7f4124bac740 1 --2- 172.21.15.84:0/2608946112 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5649041dd060 0x5649041df450 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.854+0000 7f4124bac740 1 --2- 172.21.15.84:0/2608946112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649041cce50 0x5649041ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.854+0000 7f40faffd640 0 client.15774 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:03.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.854+0000 7f4124bac740 1 --2- 172.21.15.84:0/2608946112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041c02b0 0x5649041ca2f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.854+0000 7f4124bac740 1 --2- 172.21.15.84:0/2608946112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649041bf900 0x5649041c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.854+0000 7f4124bac740 1 -- 172.21.15.84:0/2608946112 >> 172.21.15.84:0/2608946112 conn(0x5649041bbf70 msgr2=0x5649041c1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:03.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.854+0000 7f4124bac740 1 -- 172.21.15.84:0/2608946112 shutdown_connections 2024-05-22T23:04:03.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.854+0000 7f4124bac740 1 -- 172.21.15.84:0/2608946112 wait complete. 2024-05-22T23:04:03.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.854+0000 7f4124bac740 20 client.15774 trim_cache size 0 max 16384 2024-05-22T23:04:03.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.881+0000 7f4124bac740 1 Processor -- start 2024-05-22T23:04:03.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.881+0000 7f4124bac740 1 -- start start 2024-05-22T23:04:03.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.881+0000 7f4124bac740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649041bf900 0x5649041d8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:03.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.881+0000 7f4124bac740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041c02b0 0x5649041e49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:03.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.881+0000 7f4124bac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649041cce50 0x5649041e4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:03.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.881+0000 7f4124bac740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f411c1a9420 con 0x5649041c02b0 2024-05-22T23:04:03.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.881+0000 7f4124bac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f411c1abca0 con 0x5649041cce50 2024-05-22T23:04:03.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.881+0000 7f4124bac740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f411c1a9a90 con 0x5649041bf900 2024-05-22T23:04:03.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.881+0000 7f411affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041c02b0 0x5649041e49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:03.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.881+0000 7f411bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649041cce50 0x5649041e4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:03.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.881+0000 7f411affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041c02b0 0x5649041e49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43540/0 (socket says 172.21.15.84:43540) 2024-05-22T23:04:03.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.881+0000 7f411affd640 1 -- 172.21.15.84:0/1690113055 learned_addr learned my addr 172.21.15.84:0/1690113055 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:03.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.881+0000 7f411b7fe640 1 --2- 172.21.15.84:0/1690113055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649041bf900 0x5649041d8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:03.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.881+0000 7f411affd640 1 -- 172.21.15.84:0/1690113055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649041bf900 msgr2=0x5649041d8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:03.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.881+0000 7f411affd640 1 --2- 172.21.15.84:0/1690113055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649041bf900 0x5649041d8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.881+0000 7f411affd640 1 -- 172.21.15.84:0/1690113055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649041cce50 msgr2=0x5649041e4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:03.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.881+0000 7f411affd640 1 --2- 172.21.15.84:0/1690113055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649041cce50 0x5649041e4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.881+0000 7f411affd640 1 -- 172.21.15.84:0/1690113055 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649041cf5b0 con 0x5649041c02b0 2024-05-22T23:04:03.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.881+0000 7f411bfff640 1 --2- 172.21.15.84:0/1690113055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649041cce50 0x5649041e4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:03.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.882+0000 7f411affd640 1 --2- 172.21.15.84:0/1690113055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041c02b0 0x5649041e49f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4110000c00 tx=0x7f4110003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8068b1a1b12b4c71 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:03.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.882+0000 7f41017fa640 1 -- 172.21.15.84:0/1690113055 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411002a030 con 0x5649041c02b0 2024-05-22T23:04:03.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.882+0000 7f41017fa640 1 -- 172.21.15.84:0/1690113055 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4110004160 con 0x5649041c02b0 2024-05-22T23:04:03.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.882+0000 7f41017fa640 1 -- 172.21.15.84:0/1690113055 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110004590 con 0x5649041c02b0 2024-05-22T23:04:03.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.883+0000 7f4124bac740 1 -- 172.21.15.84:0/1690113055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041c02b0 msgr2=0x5649041e49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:03.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.883+0000 7f4124bac740 1 --2- 172.21.15.84:0/1690113055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041c02b0 0x5649041e49f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4110000c00 tx=0x7f4110003de0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.883+0000 7f4124bac740 1 -- 172.21.15.84:0/1690113055 shutdown_connections 2024-05-22T23:04:03.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.883+0000 7f4124bac740 1 --2- 172.21.15.84:0/1690113055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649041cce50 0x5649041e4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.883+0000 7f4124bac740 1 --2- 172.21.15.84:0/1690113055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041c02b0 0x5649041e49f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.883+0000 7f4124bac740 1 --2- 172.21.15.84:0/1690113055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649041bf900 0x5649041d8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.883+0000 7f4124bac740 1 -- 172.21.15.84:0/1690113055 >> 172.21.15.84:0/1690113055 conn(0x5649041bbf70 msgr2=0x5649041bb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:03.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.883+0000 7f4124bac740 1 -- 172.21.15.84:0/1690113055 shutdown_connections 2024-05-22T23:04:03.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.883+0000 7f4124bac740 1 -- 172.21.15.84:0/1690113055 wait complete. 2024-05-22T23:04:03.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.883+0000 7f4124bac740 1 -- start start 2024-05-22T23:04:03.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.883+0000 7f4124bac740 1 Processor -- start 2024-05-22T23:04:03.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.883+0000 7f4124bac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649041bf900 0x5649041d5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:03.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.883+0000 7f4124bac740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649041c02b0 0x5649041d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:03.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.884+0000 7f4124bac740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041cce50 0x5649041d2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:03.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.884+0000 7f4124bac740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f411c1aa160 con 0x5649041cce50 2024-05-22T23:04:03.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.884+0000 7f4124bac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f411c1ab5d0 con 0x5649041bf900 2024-05-22T23:04:03.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.884+0000 7f4124bac740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f411c1aa830 con 0x5649041c02b0 2024-05-22T23:04:03.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.884+0000 7f4124bac740 10 client.0 _ll_register_callbacks cb 0x7f410c09b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:04:03.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.884+0000 7f411b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649041bf900 0x5649041d5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:03.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.884+0000 7f411bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041cce50 0x5649041d2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:03.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.884+0000 7f411b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649041bf900 0x5649041d5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:37470/0 (socket says 172.21.15.84:37470) 2024-05-22T23:04:03.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.884+0000 7f411b7fe640 1 -- 172.21.15.84:0/386696656 learned_addr learned my addr 172.21.15.84:0/386696656 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:03.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.884+0000 7f411affd640 1 --2- 172.21.15.84:0/386696656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649041c02b0 0x5649041d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:03.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.884+0000 7f411b7fe640 1 -- 172.21.15.84:0/386696656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649041c02b0 msgr2=0x5649041d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:03.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.884+0000 7f411b7fe640 1 --2- 172.21.15.84:0/386696656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649041c02b0 0x5649041d2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.884+0000 7f411b7fe640 1 -- 172.21.15.84:0/386696656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041cce50 msgr2=0x5649041d2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:03.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.884+0000 7f411b7fe640 1 --2- 172.21.15.84:0/386696656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041cce50 0x5649041d2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.884+0000 7f411b7fe640 1 -- 172.21.15.84:0/386696656 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564903df7170 con 0x5649041bf900 2024-05-22T23:04:03.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.884+0000 7f411bfff640 1 --2- 172.21.15.84:0/386696656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041cce50 0x5649041d2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:03.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.884+0000 7f411b7fe640 1 --2- 172.21.15.84:0/386696656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649041bf900 0x5649041d5240 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f410c097db0 tx=0x7f410c06b9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f27737fc181fedb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:03.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.884+0000 7f40faffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:04:03.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.884+0000 7f40faffd640 1 -- 172.21.15.84:0/386696656 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c09fe80 con 0x5649041bf900 2024-05-22T23:04:03.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.884+0000 7f40faffd640 1 -- 172.21.15.84:0/386696656 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f410c0ba3a0 con 0x5649041bf900 2024-05-22T23:04:03.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.884+0000 7f4124bac740 1 -- 172.21.15.84:0/386696656 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4110004160 con 0x5649041bf900 2024-05-22T23:04:03.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.884+0000 7f40faffd640 1 -- 172.21.15.84:0/386696656 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f410c0a4740 con 0x5649041bf900 2024-05-22T23:04:03.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.884+0000 7f4124bac740 10 client.25355 fetch_fsmap learned FSMap version 18 2024-05-22T23:04:03.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.884+0000 7f4124bac740 1 -- 172.21.15.84:0/386696656 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5649040ec930 con 0x5649041bf900 2024-05-22T23:04:03.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.885+0000 7f40faffd640 1 -- 172.21.15.84:0/386696656 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4110004160 con 0x5649041bf900 2024-05-22T23:04:03.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.885+0000 7f40faffd640 1 -- 172.21.15.84:0/386696656 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f410c0a4340 con 0x5649041bf900 2024-05-22T23:04:03.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.885+0000 7f4124bac740 10 client.25355 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:04:03.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.885+0000 7f4124bac740 10 client.25355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:04:03.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.885+0000 7f4124bac740 20 client.25355 populate_metadata read hostname 'smithi084' 2024-05-22T23:04:03.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.885+0000 7f4124bac740 1 --2- 172.21.15.84:0/386696656 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564904244280 0x564904246670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:03.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.885+0000 7f4124bac740 4 client.25355 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-22T23:04:03.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.885+0000 7f4124bac740 1 -- 172.21.15.84:0/386696656 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564904131170 con 0x564904244280 2024-05-22T23:04:03.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.885+0000 7f411affd640 1 --2- 172.21.15.84:0/386696656 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564904244280 0x564904246670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:03.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.885+0000 7f40faffd640 10 client.25355 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:03.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.885+0000 7f411affd640 1 --2- 172.21.15.84:0/386696656 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564904244280 0x564904246670 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e525a74f01838705 server_cookie=8343b84f98e647ab in_seq=0 out_seq=0 2024-05-22T23:04:03.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.886+0000 7f40faffd640 1 -- 172.21.15.84:0/386696656 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564904131170 con 0x564904244280 2024-05-22T23:04:03.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.886+0000 7f40faffd640 10 client.25355 handle_command_reply: tid=0 2024-05-22T23:04:03.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.886+0000 7f4124bac740 1 client.25355 shutdown 2024-05-22T23:04:03.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.886+0000 7f4124bac740 1 -- 172.21.15.84:0/386696656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649041bf900 msgr2=0x5649041d5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:03.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.886+0000 7f4124bac740 1 --2- 172.21.15.84:0/386696656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649041bf900 0x5649041d5240 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f410c097db0 tx=0x7f410c06b9b0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.886+0000 7f4124bac740 1 -- 172.21.15.84:0/386696656 shutdown_connections 2024-05-22T23:04:03.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.886+0000 7f4124bac740 1 --2- 172.21.15.84:0/386696656 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564904244280 0x564904246670 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.886+0000 7f4124bac740 1 --2- 172.21.15.84:0/386696656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649041cce50 0x5649041d2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.886+0000 7f40faffd640 0 client.25355 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:03.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.886+0000 7f4124bac740 1 --2- 172.21.15.84:0/386696656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649041c02b0 0x5649041d2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.886+0000 7f4124bac740 1 --2- 172.21.15.84:0/386696656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649041bf900 0x5649041d5240 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.886+0000 7f4124bac740 1 -- 172.21.15.84:0/386696656 >> 172.21.15.84:0/386696656 conn(0x5649041bbf70 msgr2=0x5649041c1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:03.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.886+0000 7f4124bac740 1 -- 172.21.15.84:0/386696656 shutdown_connections 2024-05-22T23:04:03.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.886+0000 7f4124bac740 1 -- 172.21.15.84:0/386696656 wait complete. 2024-05-22T23:04:03.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.886+0000 7f4124bac740 20 client.25355 trim_cache size 0 max 16384 2024-05-22T23:04:03.897 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:04:03.897 INFO:teuthology.orchestra.run.smithi084.stdout: "return_code": 0, 2024-05-22T23:04:03.897 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_tag": "94e039ff-0708-44e0-af87-806d248df2ad", 2024-05-22T23:04:03.897 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": "asynchronous" 2024-05-22T23:04:03.897 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:04:03.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.887+0000 7f4121e9e640 1 -- 172.21.15.84:0/3437837751 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f41040a2e50 msgr2=0x7f41040a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:03.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.887+0000 7f4121e9e640 1 --2- 172.21.15.84:0/3437837751 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f41040a2e50 0x7f41040a5310 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f41140309c0 tx=0x7f411402a6c0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.887+0000 7f4121e9e640 1 -- 172.21.15.84:0/3437837751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f411c100750 msgr2=0x7f411c196fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:03.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.887+0000 7f4121e9e640 1 --2- 172.21.15.84:0/3437837751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f411c100750 0x7f411c196fc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f410c00e9c0 tx=0x7f410c00ee80 comp rx=0 tx=0).stop 2024-05-22T23:04:03.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.887+0000 7f4121e9e640 1 -- 172.21.15.84:0/3437837751 shutdown_connections 2024-05-22T23:04:03.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.887+0000 7f4121e9e640 1 --2- 172.21.15.84:0/3437837751 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f41040a2e50 0x7f41040a5310 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.887+0000 7f4121e9e640 1 --2- 172.21.15.84:0/3437837751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f411c106760 0x7f411c19b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.887+0000 7f4121e9e640 1 --2- 172.21.15.84:0/3437837751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f411c101110 0x7f411c197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.887+0000 7f4121e9e640 1 --2- 172.21.15.84:0/3437837751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f411c100750 0x7f411c196fc0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:03.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.887+0000 7f4121e9e640 1 -- 172.21.15.84:0/3437837751 >> 172.21.15.84:0/3437837751 conn(0x7f411c076af0 msgr2=0x7f411c0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:03.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.887+0000 7f4121e9e640 1 -- 172.21.15.84:0/3437837751 shutdown_connections 2024-05-22T23:04:03.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:03.887+0000 7f4121e9e640 1 -- 172.21.15.84:0/3437837751 wait complete. 2024-05-22T23:04:03.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:04:03.901 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:04:04.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.008+0000 7f1e7fec4640 1 -- 172.21.15.84:0/49917151 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e78100740 msgr2=0x7f1e78100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:04.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.008+0000 7f1e7fec4640 1 --2- 172.21.15.84:0/49917151 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e78100740 0x7f1e78100bc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009890 tx=0x7f1e6802a380 comp rx=0 tx=0).stop 2024-05-22T23:04:04.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.008+0000 7f1e7fec4640 1 -- 172.21.15.84:0/49917151 shutdown_connections 2024-05-22T23:04:04.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.008+0000 7f1e7fec4640 1 --2- 172.21.15.84:0/49917151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e78101100 0x7f1e7810d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.008+0000 7f1e7fec4640 1 --2- 172.21.15.84:0/49917151 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e78100740 0x7f1e78100bc0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.008+0000 7f1e7fec4640 1 --2- 172.21.15.84:0/49917151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e78106750 0x7f1e78106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.008+0000 7f1e7fec4640 1 -- 172.21.15.84:0/49917151 >> 172.21.15.84:0/49917151 conn(0x7f1e78077870 msgr2=0x7f1e78077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:04.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.009+0000 7f1e7fec4640 1 -- 172.21.15.84:0/49917151 shutdown_connections 2024-05-22T23:04:04.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.009+0000 7f1e7fec4640 1 -- 172.21.15.84:0/49917151 wait complete. 2024-05-22T23:04:04.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.009+0000 7f1e7fec4640 1 Processor -- start 2024-05-22T23:04:04.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.009+0000 7f1e7fec4640 1 -- start start 2024-05-22T23:04:04.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.009+0000 7f1e7fec4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e78100740 0x7f1e78102d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:04.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.009+0000 7f1e7fec4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e78101100 0x7f1e78103240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:04.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.009+0000 7f1e7fec4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e78106750 0x7f1e78103780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:04.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.009+0000 7f1e7fec4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78110b60 con 0x7f1e78101100 2024-05-22T23:04:04.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.009+0000 7f1e7fec4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1e781109e0 con 0x7f1e78100740 2024-05-22T23:04:04.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.009+0000 7f1e7fec4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78110ce0 con 0x7f1e78106750 2024-05-22T23:04:04.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.009+0000 7f1e7d45f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e78101100 0x7f1e78103240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:04.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.010+0000 7f1e7d45f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e78101100 0x7f1e78103240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43564/0 (socket says 172.21.15.84:43564) 2024-05-22T23:04:04.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.010+0000 7f1e7d45f640 1 -- 172.21.15.84:0/2531914958 learned_addr learned my addr 172.21.15.84:0/2531914958 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:04.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.010+0000 7f1e7dc60640 1 --2- 172.21.15.84:0/2531914958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e78100740 0x7f1e78102d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:04.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.010+0000 7f1e7e461640 1 --2- 172.21.15.84:0/2531914958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e78106750 0x7f1e78103780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:04.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.010+0000 7f1e7d45f640 1 -- 172.21.15.84:0/2531914958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e78106750 msgr2=0x7f1e78103780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:04.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.010+0000 7f1e7d45f640 1 --2- 172.21.15.84:0/2531914958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e78106750 0x7f1e78103780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.010+0000 7f1e7d45f640 1 -- 172.21.15.84:0/2531914958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e78100740 msgr2=0x7f1e78102d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:04.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.010+0000 7f1e7d45f640 1 --2- 172.21.15.84:0/2531914958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e78100740 0x7f1e78102d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.010+0000 7f1e7d45f640 1 -- 172.21.15.84:0/2531914958 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e781a6690 con 0x7f1e78101100 2024-05-22T23:04:04.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.010+0000 7f1e7e461640 1 --2- 172.21.15.84:0/2531914958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e78106750 0x7f1e78103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:04:04.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.010+0000 7f1e7d45f640 1 --2- 172.21.15.84:0/2531914958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e78101100 0x7f1e78103240 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009860 tx=0x7f1e68033d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:04.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.010+0000 7f1e6effd640 1 -- 172.21.15.84:0/2531914958 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6803f020 con 0x7f1e78101100 2024-05-22T23:04:04.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.010+0000 7f1e6effd640 1 -- 172.21.15.84:0/2531914958 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e68032770 con 0x7f1e78101100 2024-05-22T23:04:04.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.010+0000 7f1e7fec4640 1 -- 172.21.15.84:0/2531914958 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e781a6920 con 0x7f1e78101100 2024-05-22T23:04:04.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.010+0000 7f1e6effd640 1 -- 172.21.15.84:0/2531914958 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68037040 con 0x7f1e78101100 2024-05-22T23:04:04.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.010+0000 7f1e7fec4640 1 -- 172.21.15.84:0/2531914958 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e781a6e60 con 0x7f1e78101100 2024-05-22T23:04:04.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.012+0000 7f1e6effd640 1 -- 172.21.15.84:0/2531914958 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1e68042020 con 0x7f1e78101100 2024-05-22T23:04:04.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.013+0000 7f1e82bd8740 1 Processor -- start 2024-05-22T23:04:04.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.013+0000 7f1e82bd8740 1 -- start start 2024-05-22T23:04:04.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.013+0000 7f1e82bd8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586057864a0 0x5586057847e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:04.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.013+0000 7f1e82bd8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605784db0 0x558605785230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:04.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.013+0000 7f1e82bd8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55860578fb30 0x558605791f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:04.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.013+0000 7f1e82bd8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5586056e8db0 con 0x55860578fb30 2024-05-22T23:04:04.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.013+0000 7f1e82bd8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558605687360 con 0x5586057864a0 2024-05-22T23:04:04.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.013+0000 7f1e82bd8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5586056e4d70 con 0x558605784db0 2024-05-22T23:04:04.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.013+0000 7f1e7e461640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586057864a0 0x5586057847e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:04.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.013+0000 7f1e7d45f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55860578fb30 0x558605791f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:04.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.013+0000 7f1e7e461640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586057864a0 0x5586057847e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:37490/0 (socket says 172.21.15.84:37490) 2024-05-22T23:04:04.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.013+0000 7f1e7e461640 1 -- 172.21.15.84:0/664734559 learned_addr learned my addr 172.21.15.84:0/664734559 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:04.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.013+0000 7f1e7dc60640 1 --2- 172.21.15.84:0/664734559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605784db0 0x558605785230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:04.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.013+0000 7f1e6effd640 1 --2- 172.21.15.84:0/2531914958 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f1e640a2d80 0x7f1e640a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:04.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.013+0000 7f1e7e461640 1 -- 172.21.15.84:0/664734559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605784db0 msgr2=0x558605785230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:04.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.013+0000 7f1e7e461640 1 --2- 172.21.15.84:0/664734559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605784db0 0x558605785230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.013+0000 7f1e7e461640 1 -- 172.21.15.84:0/664734559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55860578fb30 msgr2=0x558605791f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:04.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.013+0000 7f1e7e461640 1 --2- 172.21.15.84:0/664734559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55860578fb30 0x558605791f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.013+0000 7f1e7e461640 1 -- 172.21.15.84:0/664734559 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586056aaa60 con 0x5586057864a0 2024-05-22T23:04:04.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.014+0000 7f1e7dc60640 1 --2- 172.21.15.84:0/664734559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605784db0 0x558605785230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:04:04.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.014+0000 7f1e7e461640 1 --2- 172.21.15.84:0/2531914958 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f1e640a2d80 0x7f1e640a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:04.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.014+0000 7f1e6effd640 1 -- 172.21.15.84:0/2531914958 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f1e680ef3f0 con 0x7f1e78101100 2024-05-22T23:04:04.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.014+0000 7f1e7e461640 1 --2- 172.21.15.84:0/664734559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586057864a0 0x5586057847e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000f4b0 tx=0x7f1e7000f970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=824e028d15dd34f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:04.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.014+0000 7f1e63fff640 1 -- 172.21.15.84:0/664734559 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70018070 con 0x5586057864a0 2024-05-22T23:04:04.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.014+0000 7f1e63fff640 1 -- 172.21.15.84:0/664734559 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e70021070 con 0x5586057864a0 2024-05-22T23:04:04.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.014+0000 7f1e63fff640 1 -- 172.21.15.84:0/664734559 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e700109a0 con 0x5586057864a0 2024-05-22T23:04:04.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.015+0000 7f1e82bd8740 1 -- 172.21.15.84:0/664734559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586057864a0 msgr2=0x5586057847e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:04.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.015+0000 7f1e7e461640 1 --2- 172.21.15.84:0/2531914958 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f1e640a2d80 0x7f1e640a5240 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1e70020000 tx=0x7f1e7001e000 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:04.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.015+0000 7f1e82bd8740 1 --2- 172.21.15.84:0/664734559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586057864a0 0x5586057847e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000f4b0 tx=0x7f1e7000f970 comp rx=0 tx=0).stop 2024-05-22T23:04:04.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.015+0000 7f1e82bd8740 1 -- 172.21.15.84:0/664734559 shutdown_connections 2024-05-22T23:04:04.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.015+0000 7f1e82bd8740 1 --2- 172.21.15.84:0/664734559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55860578fb30 0x558605791f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.015+0000 7f1e82bd8740 1 --2- 172.21.15.84:0/664734559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605784db0 0x558605785230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.015+0000 7f1e82bd8740 1 --2- 172.21.15.84:0/664734559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586057864a0 0x5586057847e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.015+0000 7f1e82bd8740 1 -- 172.21.15.84:0/664734559 >> 172.21.15.84:0/664734559 conn(0x55860574d8e0 msgr2=0x558605781310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:04.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.015+0000 7f1e82bd8740 1 -- 172.21.15.84:0/664734559 shutdown_connections 2024-05-22T23:04:04.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.015+0000 7f1e82bd8740 1 -- 172.21.15.84:0/664734559 wait complete. 2024-05-22T23:04:04.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.015+0000 7f1e82bd8740 1 -- start start 2024-05-22T23:04:04.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.015+0000 7f1e82bd8740 1 Processor -- start 2024-05-22T23:04:04.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.016+0000 7f1e82bd8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558605784db0 0x55860578ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:04.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.016+0000 7f1e82bd8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586057864a0 0x55860578cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:04.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.016+0000 7f1e82bd8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55860578fb30 0x55860578d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:04.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.016+0000 7f1e82bd8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5586057929c0 con 0x55860578fb30 2024-05-22T23:04:04.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.016+0000 7f1e82bd8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5586056a30f0 con 0x558605784db0 2024-05-22T23:04:04.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.016+0000 7f1e82bd8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558605792b40 con 0x5586057864a0 2024-05-22T23:04:04.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.016+0000 7f1e82bd8740 10 client.0 _ll_register_callbacks cb 0x5586056a7f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:04:04.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.016+0000 7f1e7dc60640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558605784db0 0x55860578ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:04.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.016+0000 7f1e7e461640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55860578fb30 0x55860578d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:04.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.016+0000 7f1e7dc60640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558605784db0 0x55860578ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:37494/0 (socket says 172.21.15.84:37494) 2024-05-22T23:04:04.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.016+0000 7f1e7dc60640 1 -- 172.21.15.84:0/2300631666 learned_addr learned my addr 172.21.15.84:0/2300631666 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:04.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.016+0000 7f1e7d45f640 1 --2- 172.21.15.84:0/2300631666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586057864a0 0x55860578cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:04.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.016+0000 7f1e7e461640 1 -- 172.21.15.84:0/2300631666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586057864a0 msgr2=0x55860578cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:04.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.016+0000 7f1e7e461640 1 --2- 172.21.15.84:0/2300631666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586057864a0 0x55860578cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.016+0000 7f1e7e461640 1 -- 172.21.15.84:0/2300631666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558605784db0 msgr2=0x55860578ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:04.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.016+0000 7f1e7e461640 1 --2- 172.21.15.84:0/2300631666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558605784db0 0x55860578ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.016+0000 7f1e7e461640 1 -- 172.21.15.84:0/2300631666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586054c67d0 con 0x55860578fb30 2024-05-22T23:04:04.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.016+0000 7f1e7dc60640 1 --2- 172.21.15.84:0/2300631666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558605784db0 0x55860578ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:04.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.017+0000 7f1e7e461640 1 --2- 172.21.15.84:0/2300631666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55860578fb30 0x55860578d4c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1e70000f80 tx=0x7f1e700104e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750c18d10633dfb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:04.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.017+0000 7f1e617fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:04:04.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.017+0000 7f1e617fa640 1 -- 172.21.15.84:0/2300631666 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70018030 con 0x55860578fb30 2024-05-22T23:04:04.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.017+0000 7f1e617fa640 1 -- 172.21.15.84:0/2300631666 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f1e70010920 con 0x55860578fb30 2024-05-22T23:04:04.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.017+0000 7f1e82bd8740 1 -- 172.21.15.84:0/2300631666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e70021070 con 0x55860578fb30 2024-05-22T23:04:04.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.017+0000 7f1e617fa640 1 -- 172.21.15.84:0/2300631666 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f1e700124b0 con 0x55860578fb30 2024-05-22T23:04:04.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.017+0000 7f1e82bd8740 10 client.15792 fetch_fsmap learned FSMap version 18 2024-05-22T23:04:04.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.017+0000 7f1e82bd8740 1 -- 172.21.15.84:0/2300631666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5586053fc3f0 con 0x55860578fb30 2024-05-22T23:04:04.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.017+0000 7f1e617fa640 1 -- 172.21.15.84:0/2300631666 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e70021070 con 0x55860578fb30 2024-05-22T23:04:04.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.017+0000 7f1e617fa640 1 -- 172.21.15.84:0/2300631666 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f1e700050b0 con 0x55860578fb30 2024-05-22T23:04:04.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.017+0000 7f1e82bd8740 10 client.15792 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:04:04.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.017+0000 7f1e82bd8740 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:04:04.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.017+0000 7f1e82bd8740 20 client.15792 populate_metadata read hostname 'smithi084' 2024-05-22T23:04:04.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.017+0000 7f1e82bd8740 1 --2- 172.21.15.84:0/2300631666 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55860579fe80 0x5586057a2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:04.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.017+0000 7f1e82bd8740 4 client.15792 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:04:04.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.017+0000 7f1e82bd8740 1 -- 172.21.15.84:0/2300631666 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586056f64e0 con 0x55860579fe80 2024-05-22T23:04:04.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.017+0000 7f1e7dc60640 1 --2- 172.21.15.84:0/2300631666 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55860579fe80 0x5586057a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:04.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.018+0000 7f1e617fa640 10 client.15792 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:04.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.018+0000 7f1e7dc60640 1 --2- 172.21.15.84:0/2300631666 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55860579fe80 0x5586057a2270 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=192be1a73b284c64 server_cookie=ac328ac7a3a540e in_seq=0 out_seq=0 2024-05-22T23:04:04.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.020+0000 7f1e617fa640 1 -- 172.21.15.84:0/2300631666 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586056f64e0 con 0x55860579fe80 2024-05-22T23:04:04.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.020+0000 7f1e617fa640 10 client.15792 handle_command_reply: tid=0 2024-05-22T23:04:04.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.020+0000 7f1e82bd8740 1 client.15792 shutdown 2024-05-22T23:04:04.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.020+0000 7f1e82bd8740 1 -- 172.21.15.84:0/2300631666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55860578fb30 msgr2=0x55860578d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:04.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.020+0000 7f1e82bd8740 1 --2- 172.21.15.84:0/2300631666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55860578fb30 0x55860578d4c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1e70000f80 tx=0x7f1e700104e0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.020+0000 7f1e82bd8740 1 -- 172.21.15.84:0/2300631666 shutdown_connections 2024-05-22T23:04:04.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.020+0000 7f1e82bd8740 1 --2- 172.21.15.84:0/2300631666 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55860579fe80 0x5586057a2270 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.020+0000 7f1e82bd8740 1 --2- 172.21.15.84:0/2300631666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55860578fb30 0x55860578d4c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.020+0000 7f1e617fa640 0 client.15792 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:04.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.020+0000 7f1e82bd8740 1 --2- 172.21.15.84:0/2300631666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586057864a0 0x55860578cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.020+0000 7f1e82bd8740 1 --2- 172.21.15.84:0/2300631666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558605784db0 0x55860578ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.020+0000 7f1e82bd8740 1 -- 172.21.15.84:0/2300631666 >> 172.21.15.84:0/2300631666 conn(0x55860574d8e0 msgr2=0x558605787110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:04.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.020+0000 7f1e82bd8740 1 -- 172.21.15.84:0/2300631666 shutdown_connections 2024-05-22T23:04:04.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.020+0000 7f1e82bd8740 1 -- 172.21.15.84:0/2300631666 wait complete. 2024-05-22T23:04:04.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.020+0000 7f1e82bd8740 20 client.15792 trim_cache size 0 max 16384 2024-05-22T23:04:04.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.038+0000 7f1e82bd8740 1 Processor -- start 2024-05-22T23:04:04.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.038+0000 7f1e82bd8740 1 -- start start 2024-05-22T23:04:04.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.038+0000 7f1e82bd8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605784db0 0x55860579b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:04.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.038+0000 7f1e82bd8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586057864a0 0x558605786f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:04.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.038+0000 7f1e82bd8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55860578fb30 0x55860578bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:04.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.038+0000 7f1e82bd8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1e781ad8c0 con 0x5586057864a0 2024-05-22T23:04:04.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.038+0000 7f1e82bd8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1e781b0140 con 0x55860578fb30 2024-05-22T23:04:04.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.038+0000 7f1e82bd8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1e781adf30 con 0x558605784db0 2024-05-22T23:04:04.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.038+0000 7f1e7d45f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586057864a0 0x558605786f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:04.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.038+0000 7f1e7d45f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586057864a0 0x558605786f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43604/0 (socket says 172.21.15.84:43604) 2024-05-22T23:04:04.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.038+0000 7f1e7d45f640 1 -- 172.21.15.84:0/2089952793 learned_addr learned my addr 172.21.15.84:0/2089952793 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:04.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.038+0000 7f1e7e461640 1 --2- 172.21.15.84:0/2089952793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55860578fb30 0x55860578bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:04.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.039+0000 7f1e7d45f640 1 -- 172.21.15.84:0/2089952793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605784db0 msgr2=0x55860579b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:04.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.039+0000 7f1e7d45f640 1 --2- 172.21.15.84:0/2089952793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605784db0 0x55860579b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.039+0000 7f1e7d45f640 1 -- 172.21.15.84:0/2089952793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55860578fb30 msgr2=0x55860578bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:04.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.039+0000 7f1e7d45f640 1 --2- 172.21.15.84:0/2089952793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55860578fb30 0x55860578bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.039+0000 7f1e7d45f640 1 -- 172.21.15.84:0/2089952793 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586053bc0f0 con 0x5586057864a0 2024-05-22T23:04:04.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.039+0000 7f1e7d45f640 1 --2- 172.21.15.84:0/2089952793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586057864a0 0x558605786f90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1e680c4600 tx=0x7f1e680f2f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ae5fff1d883bcc server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:04.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.039+0000 7f1e637fe640 1 -- 172.21.15.84:0/2089952793 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e680bb7f0 con 0x5586057864a0 2024-05-22T23:04:04.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.039+0000 7f1e637fe640 1 -- 172.21.15.84:0/2089952793 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e680392f0 con 0x5586057864a0 2024-05-22T23:04:04.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.039+0000 7f1e637fe640 1 -- 172.21.15.84:0/2089952793 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e680bb380 con 0x5586057864a0 2024-05-22T23:04:04.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.040+0000 7f1e82bd8740 1 -- 172.21.15.84:0/2089952793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586057864a0 msgr2=0x558605786f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:04.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.040+0000 7f1e82bd8740 1 --2- 172.21.15.84:0/2089952793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586057864a0 0x558605786f90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1e680c4600 tx=0x7f1e680f2f40 comp rx=0 tx=0).stop 2024-05-22T23:04:04.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.040+0000 7f1e82bd8740 1 -- 172.21.15.84:0/2089952793 shutdown_connections 2024-05-22T23:04:04.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.040+0000 7f1e82bd8740 1 --2- 172.21.15.84:0/2089952793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55860578fb30 0x55860578bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.040+0000 7f1e82bd8740 1 --2- 172.21.15.84:0/2089952793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586057864a0 0x558605786f90 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.040+0000 7f1e82bd8740 1 --2- 172.21.15.84:0/2089952793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605784db0 0x55860579b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.040+0000 7f1e82bd8740 1 -- 172.21.15.84:0/2089952793 >> 172.21.15.84:0/2089952793 conn(0x55860574d8e0 msgr2=0x558605786880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:04.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.040+0000 7f1e82bd8740 1 -- 172.21.15.84:0/2089952793 shutdown_connections 2024-05-22T23:04:04.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.040+0000 7f1e82bd8740 1 -- 172.21.15.84:0/2089952793 wait complete. 2024-05-22T23:04:04.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.040+0000 7f1e82bd8740 1 -- start start 2024-05-22T23:04:04.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.040+0000 7f1e82bd8740 1 Processor -- start 2024-05-22T23:04:04.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.040+0000 7f1e82bd8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558605784db0 0x558605793040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:04.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.040+0000 7f1e82bd8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586057864a0 0x558605793580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:04.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.040+0000 7f1e82bd8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55860578fb30 0x5586057fe310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:04.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.040+0000 7f1e82bd8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1e781ae600 con 0x55860578fb30 2024-05-22T23:04:04.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.040+0000 7f1e82bd8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1e781afa70 con 0x558605784db0 2024-05-22T23:04:04.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.040+0000 7f1e82bd8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1e781aecd0 con 0x5586057864a0 2024-05-22T23:04:04.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.041+0000 7f1e82bd8740 10 client.0 _ll_register_callbacks cb 0x7f1e680f2e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:04:04.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.041+0000 7f1e7dc60640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558605784db0 0x558605793040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:04.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.041+0000 7f1e7e461640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55860578fb30 0x5586057fe310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:04.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.041+0000 7f1e7dc60640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558605784db0 0x558605793040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:37510/0 (socket says 172.21.15.84:37510) 2024-05-22T23:04:04.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.041+0000 7f1e7dc60640 1 -- 172.21.15.84:0/4232950903 learned_addr learned my addr 172.21.15.84:0/4232950903 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:04.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.041+0000 7f1e7d45f640 1 --2- 172.21.15.84:0/4232950903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586057864a0 0x558605793580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:04.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.041+0000 7f1e7dc60640 1 -- 172.21.15.84:0/4232950903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586057864a0 msgr2=0x558605793580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:04.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.041+0000 7f1e7dc60640 1 --2- 172.21.15.84:0/4232950903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586057864a0 0x558605793580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.041+0000 7f1e7dc60640 1 -- 172.21.15.84:0/4232950903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55860578fb30 msgr2=0x5586057fe310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:04.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.041+0000 7f1e7dc60640 1 --2- 172.21.15.84:0/4232950903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55860578fb30 0x5586057fe310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.041+0000 7f1e7dc60640 1 -- 172.21.15.84:0/4232950903 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586056a7fe0 con 0x558605784db0 2024-05-22T23:04:04.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.041+0000 7f1e7d45f640 1 --2- 172.21.15.84:0/4232950903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586057864a0 0x558605793580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:04:04.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.041+0000 7f1e7e461640 1 --2- 172.21.15.84:0/4232950903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55860578fb30 0x5586057fe310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:04.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.041+0000 7f1e7dc60640 1 --2- 172.21.15.84:0/4232950903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558605784db0 0x558605793040 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1e74004390 tx=0x7f1e740039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37e44daffbc04181 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:04.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.041+0000 7f1e617fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:04:04.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.041+0000 7f1e617fa640 1 -- 172.21.15.84:0/4232950903 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e74020570 con 0x558605784db0 2024-05-22T23:04:04.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.041+0000 7f1e82bd8740 1 -- 172.21.15.84:0/4232950903 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e680392f0 con 0x558605784db0 2024-05-22T23:04:04.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.041+0000 7f1e617fa640 1 -- 172.21.15.84:0/4232950903 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f1e74020710 con 0x558605784db0 2024-05-22T23:04:04.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.041+0000 7f1e617fa640 1 -- 172.21.15.84:0/4232950903 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f1e74030050 con 0x558605784db0 2024-05-22T23:04:04.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.041+0000 7f1e82bd8740 10 client.25373 fetch_fsmap learned FSMap version 18 2024-05-22T23:04:04.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.041+0000 7f1e82bd8740 1 -- 172.21.15.84:0/4232950903 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5586056791c0 con 0x558605784db0 2024-05-22T23:04:04.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.042+0000 7f1e617fa640 1 -- 172.21.15.84:0/4232950903 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e680392f0 con 0x558605784db0 2024-05-22T23:04:04.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.042+0000 7f1e617fa640 1 -- 172.21.15.84:0/4232950903 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f1e7401d480 con 0x558605784db0 2024-05-22T23:04:04.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.042+0000 7f1e82bd8740 10 client.25373 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:04:04.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.042+0000 7f1e82bd8740 10 client.25373 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:04:04.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.042+0000 7f1e82bd8740 20 client.25373 populate_metadata read hostname 'smithi084' 2024-05-22T23:04:04.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.042+0000 7f1e82bd8740 1 --2- 172.21.15.84:0/4232950903 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55860579fe80 0x5586058080f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:04.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.042+0000 7f1e82bd8740 4 client.25373 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:04:04.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.042+0000 7f1e82bd8740 1 -- 172.21.15.84:0/4232950903 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5586056f0830 con 0x55860579fe80 2024-05-22T23:04:04.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.042+0000 7f1e7d45f640 1 --2- 172.21.15.84:0/4232950903 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55860579fe80 0x5586058080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:04.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.042+0000 7f1e617fa640 10 client.25373 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:04.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.042+0000 7f1e7d45f640 1 --2- 172.21.15.84:0/4232950903 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55860579fe80 0x5586058080f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef7f543d62ed9a14 server_cookie=fa8e86abdc20c4dd in_seq=0 out_seq=0 2024-05-22T23:04:04.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.043+0000 7f1e617fa640 1 -- 172.21.15.84:0/4232950903 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5586056f0830 con 0x55860579fe80 2024-05-22T23:04:04.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.043+0000 7f1e617fa640 10 client.25373 handle_command_reply: tid=0 2024-05-22T23:04:04.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.043+0000 7f1e82bd8740 1 client.25373 shutdown 2024-05-22T23:04:04.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.043+0000 7f1e82bd8740 1 -- 172.21.15.84:0/4232950903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558605784db0 msgr2=0x558605793040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:04.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.043+0000 7f1e82bd8740 1 --2- 172.21.15.84:0/4232950903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558605784db0 0x558605793040 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1e74004390 tx=0x7f1e740039c0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.043+0000 7f1e82bd8740 1 -- 172.21.15.84:0/4232950903 shutdown_connections 2024-05-22T23:04:04.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.043+0000 7f1e82bd8740 1 --2- 172.21.15.84:0/4232950903 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55860579fe80 0x5586058080f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.043+0000 7f1e82bd8740 1 --2- 172.21.15.84:0/4232950903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55860578fb30 0x5586057fe310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.043+0000 7f1e617fa640 0 client.25373 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:04.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.043+0000 7f1e82bd8740 1 --2- 172.21.15.84:0/4232950903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586057864a0 0x558605793580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.043+0000 7f1e82bd8740 1 --2- 172.21.15.84:0/4232950903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558605784db0 0x558605793040 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.043+0000 7f1e82bd8740 1 -- 172.21.15.84:0/4232950903 >> 172.21.15.84:0/4232950903 conn(0x55860574d8e0 msgr2=0x558605787110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:04.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.043+0000 7f1e82bd8740 1 -- 172.21.15.84:0/4232950903 shutdown_connections 2024-05-22T23:04:04.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.043+0000 7f1e82bd8740 1 -- 172.21.15.84:0/4232950903 wait complete. 2024-05-22T23:04:04.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.044+0000 7f1e82bd8740 20 client.25373 trim_cache size 0 max 16384 2024-05-22T23:04:04.053 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:04:04.053 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-22T23:04:04.053 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": { 2024-05-22T23:04:04.053 INFO:teuthology.orchestra.run.smithi084.stdout: "94e039ff-0708-44e0-af87-806d248df2ad": { 2024-05-22T23:04:04.053 INFO:teuthology.orchestra.run.smithi084.stdout: "path": "/", 2024-05-22T23:04:04.053 INFO:teuthology.orchestra.run.smithi084.stdout: "tag": "94e039ff-0708-44e0-af87-806d248df2ad", 2024-05-22T23:04:04.053 INFO:teuthology.orchestra.run.smithi084.stdout: "options": "recursive,force" 2024-05-22T23:04:04.053 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:04:04.053 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:04:04.053 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:04:04.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.044+0000 7f1e7fec4640 1 -- 172.21.15.84:0/2531914958 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f1e640a2d80 msgr2=0x7f1e640a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:04.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.044+0000 7f1e7fec4640 1 --2- 172.21.15.84:0/2531914958 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f1e640a2d80 0x7f1e640a5240 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1e70020000 tx=0x7f1e7001e000 comp rx=0 tx=0).stop 2024-05-22T23:04:04.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.044+0000 7f1e7fec4640 1 -- 172.21.15.84:0/2531914958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e78101100 msgr2=0x7f1e78103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:04.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.044+0000 7f1e7fec4640 1 --2- 172.21.15.84:0/2531914958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e78101100 0x7f1e78103240 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009860 tx=0x7f1e68033d10 comp rx=0 tx=0).stop 2024-05-22T23:04:04.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.044+0000 7f1e7fec4640 1 -- 172.21.15.84:0/2531914958 shutdown_connections 2024-05-22T23:04:04.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.044+0000 7f1e7fec4640 1 --2- 172.21.15.84:0/2531914958 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f1e640a2d80 0x7f1e640a5240 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.044+0000 7f1e7fec4640 1 --2- 172.21.15.84:0/2531914958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e78106750 0x7f1e78103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.044+0000 7f1e7fec4640 1 --2- 172.21.15.84:0/2531914958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e78101100 0x7f1e78103240 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.044+0000 7f1e7fec4640 1 --2- 172.21.15.84:0/2531914958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e78100740 0x7f1e78102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:04.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.044+0000 7f1e7fec4640 1 -- 172.21.15.84:0/2531914958 >> 172.21.15.84:0/2531914958 conn(0x7f1e78077870 msgr2=0x7f1e78068d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:04.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.044+0000 7f1e7fec4640 1 -- 172.21.15.84:0/2531914958 shutdown_connections 2024-05-22T23:04:04.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:04.044+0000 7f1e7fec4640 1 -- 172.21.15.84:0/2531914958 wait complete. 2024-05-22T23:04:04.058 INFO:tasks.cephfs.filesystem:scrub status for tag:94e039ff-0708-44e0-af87-806d248df2ad - {'path': '/', 'tag': '94e039ff-0708-44e0-af87-806d248df2ad', 'options': 'recursive,force'} 2024-05-22T23:04:04.376 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:04 smithi084 ceph-mon[32186]: scrub queued for path: / 2024-05-22T23:04:04.377 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:04 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:04:04.377 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:04 smithi084 ceph-mon[32186]: scrub summary: active paths [/] 2024-05-22T23:04:04.377 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:04 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:04:04.425 INFO:tasks.workunit.client.0.smithi084.stdout: 1 152706 16.84 MB/sec warmup 50 sec latency 23.921 ms 2024-05-22T23:04:04.552 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:04 smithi096 ceph-mon[36392]: scrub queued for path: / 2024-05-22T23:04:04.552 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:04 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:04:04.552 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:04 smithi096 ceph-mon[36392]: scrub summary: active paths [/] 2024-05-22T23:04:04.552 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:04 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:04:04.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:04 smithi204 ceph-mon[36211]: scrub queued for path: / 2024-05-22T23:04:04.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:04 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:04:04.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:04 smithi204 ceph-mon[36211]: scrub summary: active paths [/] 2024-05-22T23:04:04.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:04 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:04:05.425 INFO:tasks.workunit.client.0.smithi084.stdout: 1 155715 16.79 MB/sec warmup 51 sec latency 15.131 ms 2024-05-22T23:04:05.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:05 smithi096 ceph-mon[36392]: pgmap v70: 97 pgs: 97 active+clean; 75 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-05-22T23:04:05.626 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:05 smithi084 ceph-mon[32186]: pgmap v70: 97 pgs: 97 active+clean; 75 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-05-22T23:04:05.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:05 smithi204 ceph-mon[36211]: pgmap v70: 97 pgs: 97 active+clean; 75 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-05-22T23:04:06.425 INFO:tasks.workunit.client.0.smithi084.stdout: 1 158495 16.72 MB/sec warmup 52 sec latency 13.776 ms 2024-05-22T23:04:06.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:06 smithi096 ceph-mon[36392]: scrub summary: idle 2024-05-22T23:04:06.626 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:06 smithi084 ceph-mon[32186]: scrub summary: idle 2024-05-22T23:04:06.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:06 smithi204 ceph-mon[36211]: scrub summary: idle 2024-05-22T23:04:07.426 INFO:tasks.workunit.client.0.smithi084.stdout: 1 160902 16.67 MB/sec warmup 53 sec latency 71.087 ms 2024-05-22T23:04:07.551 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:07 smithi096 ceph-mon[36392]: pgmap v71: 97 pgs: 97 active+clean; 76 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.6 MiB/s wr, 216 op/s 2024-05-22T23:04:07.628 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:07 smithi084 ceph-mon[32186]: pgmap v71: 97 pgs: 97 active+clean; 76 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.6 MiB/s wr, 216 op/s 2024-05-22T23:04:07.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:07 smithi204 ceph-mon[36211]: pgmap v71: 97 pgs: 97 active+clean; 76 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.6 MiB/s wr, 216 op/s 2024-05-22T23:04:08.426 INFO:tasks.workunit.client.0.smithi084.stdout: 1 164010 16.69 MB/sec warmup 54 sec latency 14.807 ms 2024-05-22T23:04:09.426 INFO:tasks.workunit.client.0.smithi084.stdout: 1 167298 16.72 MB/sec warmup 55 sec latency 24.574 ms 2024-05-22T23:04:09.626 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:09 smithi084 ceph-mon[32186]: pgmap v72: 97 pgs: 97 active+clean; 83 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 315 op/s 2024-05-22T23:04:09.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:09 smithi204 ceph-mon[36211]: pgmap v72: 97 pgs: 97 active+clean; 83 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 315 op/s 2024-05-22T23:04:09.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:09 smithi096 ceph-mon[36392]: pgmap v72: 97 pgs: 97 active+clean; 83 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 315 op/s 2024-05-22T23:04:10.426 INFO:tasks.workunit.client.0.smithi084.stdout: 1 170502 16.76 MB/sec warmup 56 sec latency 24.216 ms 2024-05-22T23:04:11.426 INFO:tasks.workunit.client.0.smithi084.stdout: 1 173291 16.69 MB/sec warmup 57 sec latency 18.644 ms 2024-05-22T23:04:11.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:11 smithi084 ceph-mon[32186]: pgmap v73: 97 pgs: 97 active+clean; 83 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 218 op/s 2024-05-22T23:04:11.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:11 smithi204 ceph-mon[36211]: pgmap v73: 97 pgs: 97 active+clean; 83 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 218 op/s 2024-05-22T23:04:11.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:11 smithi096 ceph-mon[36392]: pgmap v73: 97 pgs: 97 active+clean; 83 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 218 op/s 2024-05-22T23:04:12.426 INFO:tasks.workunit.client.0.smithi084.stdout: 1 176246 16.64 MB/sec warmup 58 sec latency 26.758 ms 2024-05-22T23:04:13.426 INFO:tasks.workunit.client.0.smithi084.stdout: 1 178861 16.65 MB/sec warmup 59 sec latency 26.923 ms 2024-05-22T23:04:13.626 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:13 smithi084 ceph-mon[32186]: pgmap v74: 97 pgs: 97 active+clean; 91 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 312 op/s 2024-05-22T23:04:13.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:13 smithi204 ceph-mon[36211]: pgmap v74: 97 pgs: 97 active+clean; 91 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 312 op/s 2024-05-22T23:04:13.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:13 smithi096 ceph-mon[36392]: pgmap v74: 97 pgs: 97 active+clean; 91 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 312 op/s 2024-05-22T23:04:14.426 INFO:tasks.workunit.client.0.smithi084.stdout: 1 181623 16.61 MB/sec warmup 60 sec latency 25.067 ms 2024-05-22T23:04:15.426 INFO:tasks.workunit.client.0.smithi084.stdout: 1 184707 16.64 MB/sec warmup 61 sec latency 17.260 ms 2024-05-22T23:04:15.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:15 smithi084 ceph-mon[32186]: pgmap v75: 97 pgs: 97 active+clean; 91 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 204 op/s 2024-05-22T23:04:15.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:15 smithi204 ceph-mon[36211]: pgmap v75: 97 pgs: 97 active+clean; 91 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 204 op/s 2024-05-22T23:04:15.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:15 smithi096 ceph-mon[36392]: pgmap v75: 97 pgs: 97 active+clean; 91 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 204 op/s 2024-05-22T23:04:16.426 INFO:tasks.workunit.client.0.smithi084.stdout: 1 187477 16.58 MB/sec warmup 62 sec latency 23.234 ms 2024-05-22T23:04:17.426 INFO:tasks.workunit.client.0.smithi084.stdout: 1 190565 16.55 MB/sec warmup 63 sec latency 16.504 ms 2024-05-22T23:04:17.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:17 smithi084 ceph-mon[32186]: pgmap v76: 97 pgs: 97 active+clean; 93 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2024-05-22T23:04:17.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:17 smithi204 ceph-mon[36211]: pgmap v76: 97 pgs: 97 active+clean; 93 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2024-05-22T23:04:17.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:17 smithi096 ceph-mon[36392]: pgmap v76: 97 pgs: 97 active+clean; 93 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2024-05-22T23:04:18.426 INFO:tasks.workunit.client.0.smithi084.stdout: 1 193311 16.55 MB/sec warmup 64 sec latency 17.651 ms 2024-05-22T23:04:19.426 INFO:tasks.workunit.client.0.smithi084.stdout: 1 196086 16.52 MB/sec warmup 65 sec latency 19.456 ms 2024-05-22T23:04:19.626 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:19 smithi084 ceph-mon[32186]: pgmap v77: 97 pgs: 97 active+clean; 101 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 291 op/s 2024-05-22T23:04:19.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:19 smithi204 ceph-mon[36211]: pgmap v77: 97 pgs: 97 active+clean; 101 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 291 op/s 2024-05-22T23:04:19.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:19 smithi096 ceph-mon[36392]: pgmap v77: 97 pgs: 97 active+clean; 101 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 291 op/s 2024-05-22T23:04:20.426 INFO:tasks.workunit.client.0.smithi084.stdout: 1 199305 16.54 MB/sec warmup 66 sec latency 7.497 ms 2024-05-22T23:04:21.426 INFO:tasks.workunit.client.0.smithi084.stdout: 1 202329 16.56 MB/sec warmup 67 sec latency 49.869 ms 2024-05-22T23:04:21.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:21 smithi084 ceph-mon[32186]: pgmap v78: 97 pgs: 97 active+clean; 101 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 191 op/s 2024-05-22T23:04:21.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:21 smithi204 ceph-mon[36211]: pgmap v78: 97 pgs: 97 active+clean; 101 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 191 op/s 2024-05-22T23:04:21.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:21 smithi096 ceph-mon[36392]: pgmap v78: 97 pgs: 97 active+clean; 101 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 191 op/s 2024-05-22T23:04:22.426 INFO:tasks.workunit.client.0.smithi084.stdout: 1 205102 16.52 MB/sec warmup 68 sec latency 26.822 ms 2024-05-22T23:04:23.426 INFO:tasks.workunit.client.0.smithi084.stdout: 1 207424 16.45 MB/sec warmup 69 sec latency 33.317 ms 2024-05-22T23:04:23.626 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:23 smithi084 ceph-mon[32186]: pgmap v79: 97 pgs: 97 active+clean; 110 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.7 MiB/s wr, 286 op/s 2024-05-22T23:04:23.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:23 smithi204 ceph-mon[36211]: pgmap v79: 97 pgs: 97 active+clean; 110 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.7 MiB/s wr, 286 op/s 2024-05-22T23:04:23.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:23 smithi096 ceph-mon[36392]: pgmap v79: 97 pgs: 97 active+clean; 110 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.7 MiB/s wr, 286 op/s 2024-05-22T23:04:24.427 INFO:tasks.workunit.client.0.smithi084.stdout: 1 210257 16.43 MB/sec warmup 70 sec latency 13.894 ms 2024-05-22T23:04:25.426 INFO:tasks.workunit.client.0.smithi084.stdout: 1 213502 16.46 MB/sec warmup 71 sec latency 13.690 ms 2024-05-22T23:04:25.603 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:25 smithi096 ceph-mon[36392]: pgmap v80: 97 pgs: 97 active+clean; 110 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 192 op/s 2024-05-22T23:04:25.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:25 smithi084 ceph-mon[32186]: pgmap v80: 97 pgs: 97 active+clean; 110 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 192 op/s 2024-05-22T23:04:25.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:25 smithi204 ceph-mon[36211]: pgmap v80: 97 pgs: 97 active+clean; 110 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 192 op/s 2024-05-22T23:04:26.426 INFO:tasks.workunit.client.0.smithi084.stdout: 1 216636 16.49 MB/sec warmup 72 sec latency 172.221 ms 2024-05-22T23:04:27.427 INFO:tasks.workunit.client.0.smithi084.stdout: 1 219416 16.44 MB/sec warmup 73 sec latency 34.676 ms 2024-05-22T23:04:27.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:27 smithi084 ceph-mon[32186]: pgmap v81: 97 pgs: 97 active+clean; 110 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 211 op/s 2024-05-22T23:04:27.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:27 smithi204 ceph-mon[36211]: pgmap v81: 97 pgs: 97 active+clean; 110 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 211 op/s 2024-05-22T23:04:27.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:27 smithi096 ceph-mon[36392]: pgmap v81: 97 pgs: 97 active+clean; 110 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 211 op/s 2024-05-22T23:04:28.429 INFO:tasks.workunit.client.0.smithi084.stdout: 1 221495 16.37 MB/sec warmup 74 sec latency 169.675 ms 2024-05-22T23:04:29.427 INFO:tasks.workunit.client.0.smithi084.stdout: 1 224385 16.35 MB/sec warmup 75 sec latency 17.033 ms 2024-05-22T23:04:29.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:29 smithi084 ceph-mon[32186]: pgmap v82: 97 pgs: 97 active+clean; 117 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 305 op/s 2024-05-22T23:04:29.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:29 smithi204 ceph-mon[36211]: pgmap v82: 97 pgs: 97 active+clean; 117 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 305 op/s 2024-05-22T23:04:29.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:29 smithi096 ceph-mon[36392]: pgmap v82: 97 pgs: 97 active+clean; 117 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 305 op/s 2024-05-22T23:04:30.427 INFO:tasks.workunit.client.0.smithi084.stdout: 1 227539 16.38 MB/sec warmup 76 sec latency 16.249 ms 2024-05-22T23:04:31.427 INFO:tasks.workunit.client.0.smithi084.stdout: 1 230740 16.40 MB/sec warmup 77 sec latency 55.886 ms 2024-05-22T23:04:31.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:31 smithi084 ceph-mon[32186]: pgmap v83: 97 pgs: 97 active+clean; 117 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 222 op/s 2024-05-22T23:04:31.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:31 smithi204 ceph-mon[36211]: pgmap v83: 97 pgs: 97 active+clean; 117 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 222 op/s 2024-05-22T23:04:31.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:31 smithi096 ceph-mon[36392]: pgmap v83: 97 pgs: 97 active+clean; 117 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 222 op/s 2024-05-22T23:04:32.427 INFO:tasks.workunit.client.0.smithi084.stdout: 1 233095 16.30 MB/sec warmup 78 sec latency 92.665 ms 2024-05-22T23:04:33.427 INFO:tasks.workunit.client.0.smithi084.stdout: 1 235670 16.31 MB/sec warmup 79 sec latency 18.195 ms 2024-05-22T23:04:33.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:33 smithi084 ceph-mon[32186]: pgmap v84: 97 pgs: 97 active+clean; 123 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 345 op/s 2024-05-22T23:04:33.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:33 smithi204 ceph-mon[36211]: pgmap v84: 97 pgs: 97 active+clean; 123 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 345 op/s 2024-05-22T23:04:33.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:33 smithi096 ceph-mon[36392]: pgmap v84: 97 pgs: 97 active+clean; 123 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 345 op/s 2024-05-22T23:04:34.059 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:04:34.059 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:04:34.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.163+0000 7f372f031640 1 -- 172.21.15.84:0/3878991636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3728101100 msgr2=0x7f372810d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:34.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.163+0000 7f372f031640 1 --2- 172.21.15.84:0/3878991636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3728101100 0x7f372810d610 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3720009cd0 tx=0x7f372002a8e0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.163+0000 7f372f031640 1 -- 172.21.15.84:0/3878991636 shutdown_connections 2024-05-22T23:04:34.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.163+0000 7f372f031640 1 --2- 172.21.15.84:0/3878991636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3728101100 0x7f372810d610 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.163+0000 7f372f031640 1 --2- 172.21.15.84:0/3878991636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3728100740 0x7f3728100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.163+0000 7f372f031640 1 --2- 172.21.15.84:0/3878991636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3728106750 0x7f3728106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.163+0000 7f372f031640 1 -- 172.21.15.84:0/3878991636 >> 172.21.15.84:0/3878991636 conn(0x7f3728077870 msgr2=0x7f3728077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:34.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.163+0000 7f372f031640 1 -- 172.21.15.84:0/3878991636 shutdown_connections 2024-05-22T23:04:34.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.163+0000 7f372f031640 1 -- 172.21.15.84:0/3878991636 wait complete. 2024-05-22T23:04:34.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.164+0000 7f372f031640 1 Processor -- start 2024-05-22T23:04:34.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.164+0000 7f372f031640 1 -- start start 2024-05-22T23:04:34.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.164+0000 7f372f031640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3728100740 0x7f3728198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.164+0000 7f372f031640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3728101100 0x7f3728198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.164+0000 7f372f031640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3728106750 0x7f372819c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.164+0000 7f372f031640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3728111c60 con 0x7f3728106750 2024-05-22T23:04:34.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.164+0000 7f372f031640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3728111ae0 con 0x7f3728100740 2024-05-22T23:04:34.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.164+0000 7f372f031640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3728111de0 con 0x7f3728101100 2024-05-22T23:04:34.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.164+0000 7f372d5ce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3728106750 0x7f372819c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.164+0000 7f372d5ce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3728106750 0x7f372819c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36742/0 (socket says 172.21.15.84:36742) 2024-05-22T23:04:34.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.164+0000 7f372cdcd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3728100740 0x7f3728198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.164+0000 7f372d5ce640 1 -- 172.21.15.84:0/4008871949 learned_addr learned my addr 172.21.15.84:0/4008871949 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:34.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.164+0000 7f3727fff640 1 --2- 172.21.15.84:0/4008871949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3728101100 0x7f3728198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.164+0000 7f372d5ce640 1 -- 172.21.15.84:0/4008871949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3728101100 msgr2=0x7f3728198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:34.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.164+0000 7f372d5ce640 1 --2- 172.21.15.84:0/4008871949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3728101100 0x7f3728198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.164+0000 7f372d5ce640 1 -- 172.21.15.84:0/4008871949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3728100740 msgr2=0x7f3728198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:34.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.164+0000 7f372d5ce640 1 --2- 172.21.15.84:0/4008871949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3728100740 0x7f3728198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.164+0000 7f372d5ce640 1 -- 172.21.15.84:0/4008871949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372819d110 con 0x7f3728106750 2024-05-22T23:04:34.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.164+0000 7f372cdcd640 1 --2- 172.21.15.84:0/4008871949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3728100740 0x7f3728198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:34.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.164+0000 7f372d5ce640 1 --2- 172.21.15.84:0/4008871949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3728106750 0x7f372819c9e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3720005cc0 tx=0x7f37200093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:34.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.165+0000 7f3725ffb640 1 -- 172.21.15.84:0/4008871949 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720003e00 con 0x7f3728106750 2024-05-22T23:04:34.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.165+0000 7f3725ffb640 1 -- 172.21.15.84:0/4008871949 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3720004520 con 0x7f3728106750 2024-05-22T23:04:34.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.165+0000 7f372f031640 1 -- 172.21.15.84:0/4008871949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f372819d3a0 con 0x7f3728106750 2024-05-22T23:04:34.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.165+0000 7f3725ffb640 1 -- 172.21.15.84:0/4008871949 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720030db0 con 0x7f3728106750 2024-05-22T23:04:34.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.165+0000 7f372f031640 1 -- 172.21.15.84:0/4008871949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37281a4c90 con 0x7f3728106750 2024-05-22T23:04:34.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.167+0000 7f3725ffb640 1 -- 172.21.15.84:0/4008871949 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f37200046d0 con 0x7f3728106750 2024-05-22T23:04:34.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.168+0000 7f3731e19740 1 Processor -- start 2024-05-22T23:04:34.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.168+0000 7f3725ffb640 1 --2- 172.21.15.84:0/4008871949 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f37100a2d80 0x7f37100a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.177 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.168+0000 7f3731e19740 1 -- start start 2024-05-22T23:04:34.177 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.168+0000 7f3727fff640 1 --2- 172.21.15.84:0/4008871949 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f37100a2d80 0x7f37100a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.177 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.168+0000 7f3731e19740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56522d4024a0 0x56522d4007e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.177 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.168+0000 7f3731e19740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56522d400db0 0x56522d401230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.177 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.168+0000 7f3725ffb640 1 -- 172.21.15.84:0/4008871949 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f372002e030 con 0x7f3728106750 2024-05-22T23:04:34.177 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.168+0000 7f3731e19740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d40bb30 0x56522d40df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.177 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.168+0000 7f3731e19740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56522d364db0 con 0x56522d40bb30 2024-05-22T23:04:34.177 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.168+0000 7f3731e19740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56522d303360 con 0x56522d400db0 2024-05-22T23:04:34.177 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.168+0000 7f3731e19740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56522d360d70 con 0x56522d4024a0 2024-05-22T23:04:34.177 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.168+0000 7f372d5ce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d40bb30 0x56522d40df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.177 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.168+0000 7f372d5ce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d40bb30 0x56522d40df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36746/0 (socket says 172.21.15.84:36746) 2024-05-22T23:04:34.177 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.168+0000 7f372d5ce640 1 -- 172.21.15.84:0/2737912598 learned_addr learned my addr 172.21.15.84:0/2737912598 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:34.177 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.168+0000 7f3727fff640 1 --2- 172.21.15.84:0/2737912598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56522d400db0 0x56522d401230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.177 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.168+0000 7f372cdcd640 1 --2- 172.21.15.84:0/2737912598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56522d4024a0 0x56522d4007e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.168+0000 7f372d5ce640 1 -- 172.21.15.84:0/2737912598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56522d4024a0 msgr2=0x56522d4007e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:34.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.168+0000 7f372d5ce640 1 --2- 172.21.15.84:0/2737912598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56522d4024a0 0x56522d4007e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.168+0000 7f372d5ce640 1 -- 172.21.15.84:0/2737912598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56522d400db0 msgr2=0x56522d401230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:34.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.168+0000 7f372d5ce640 1 --2- 172.21.15.84:0/2737912598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56522d400db0 0x56522d401230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.168+0000 7f3727fff640 1 --2- 172.21.15.84:0/4008871949 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f37100a2d80 0x7f37100a5240 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f371c004650 tx=0x7f371c013880 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:34.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.168+0000 7f372d5ce640 1 -- 172.21.15.84:0/2737912598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56522d326a60 con 0x56522d40bb30 2024-05-22T23:04:34.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.168+0000 7f372cdcd640 1 --2- 172.21.15.84:0/2737912598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56522d4024a0 0x56522d4007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:34.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.169+0000 7f372d5ce640 1 --2- 172.21.15.84:0/2737912598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d40bb30 0x56522d40df20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f37200c5670 tx=0x7f37200bce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2245496bdc99e206 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:34.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.169+0000 7f370effd640 1 -- 172.21.15.84:0/2737912598 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37200d5020 con 0x56522d40bb30 2024-05-22T23:04:34.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.169+0000 7f370effd640 1 -- 172.21.15.84:0/2737912598 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3720036040 con 0x56522d40bb30 2024-05-22T23:04:34.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.169+0000 7f370effd640 1 -- 172.21.15.84:0/2737912598 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720030700 con 0x56522d40bb30 2024-05-22T23:04:34.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.169+0000 7f3731e19740 1 -- 172.21.15.84:0/2737912598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d40bb30 msgr2=0x56522d40df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:34.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.169+0000 7f3731e19740 1 --2- 172.21.15.84:0/2737912598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d40bb30 0x56522d40df20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f37200c5670 tx=0x7f37200bce10 comp rx=0 tx=0).stop 2024-05-22T23:04:34.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.169+0000 7f3731e19740 1 -- 172.21.15.84:0/2737912598 shutdown_connections 2024-05-22T23:04:34.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.169+0000 7f3731e19740 1 --2- 172.21.15.84:0/2737912598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d40bb30 0x56522d40df20 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.169+0000 7f3731e19740 1 --2- 172.21.15.84:0/2737912598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56522d400db0 0x56522d401230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.169+0000 7f3731e19740 1 --2- 172.21.15.84:0/2737912598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56522d4024a0 0x56522d4007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.169+0000 7f3731e19740 1 -- 172.21.15.84:0/2737912598 >> 172.21.15.84:0/2737912598 conn(0x56522d3c98e0 msgr2=0x56522d3fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:34.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.170+0000 7f3731e19740 1 -- 172.21.15.84:0/2737912598 shutdown_connections 2024-05-22T23:04:34.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.170+0000 7f3731e19740 1 -- 172.21.15.84:0/2737912598 wait complete. 2024-05-22T23:04:34.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.170+0000 7f3731e19740 1 -- start start 2024-05-22T23:04:34.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.170+0000 7f3731e19740 1 Processor -- start 2024-05-22T23:04:34.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.170+0000 7f3731e19740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d400db0 0x56522d408aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.170+0000 7f3731e19740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56522d4024a0 0x56522d408fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.170+0000 7f3731e19740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56522d40bb30 0x56522d409520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.170+0000 7f3731e19740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56522d40e780 con 0x56522d400db0 2024-05-22T23:04:34.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.170+0000 7f3731e19740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56522d31f0f0 con 0x56522d40bb30 2024-05-22T23:04:34.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.170+0000 7f3731e19740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56522d40e900 con 0x56522d4024a0 2024-05-22T23:04:34.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.170+0000 7f3731e19740 10 client.0 _ll_register_callbacks cb 0x56522d323f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:04:34.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.170+0000 7f372cdcd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d400db0 0x56522d408aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.170+0000 7f372cdcd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d400db0 0x56522d408aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36748/0 (socket says 172.21.15.84:36748) 2024-05-22T23:04:34.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.170+0000 7f372d5ce640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56522d40bb30 0x56522d409520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.170+0000 7f372cdcd640 1 -- 172.21.15.84:0/3973482829 learned_addr learned my addr 172.21.15.84:0/3973482829 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:34.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.171+0000 7f3727fff640 1 --2- 172.21.15.84:0/3973482829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56522d4024a0 0x56522d408fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.171+0000 7f372d5ce640 1 -- 172.21.15.84:0/3973482829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56522d4024a0 msgr2=0x56522d408fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:34.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.171+0000 7f372d5ce640 1 --2- 172.21.15.84:0/3973482829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56522d4024a0 0x56522d408fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.171+0000 7f372d5ce640 1 -- 172.21.15.84:0/3973482829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d400db0 msgr2=0x56522d408aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:34.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.171+0000 7f372d5ce640 1 --2- 172.21.15.84:0/3973482829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d400db0 0x56522d408aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.171+0000 7f372d5ce640 1 -- 172.21.15.84:0/3973482829 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56522d1427d0 con 0x56522d40bb30 2024-05-22T23:04:34.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.171+0000 7f372cdcd640 1 --2- 172.21.15.84:0/3973482829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d400db0 0x56522d408aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:34.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.171+0000 7f372d5ce640 1 --2- 172.21.15.84:0/3973482829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56522d40bb30 0x56522d409520 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f37200bd0f0 tx=0x7f37200095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4b3bfa19707bf51 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:34.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.171+0000 7f3707fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:04:34.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.171+0000 7f3707fff640 1 -- 172.21.15.84:0/3973482829 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37200d5070 con 0x56522d40bb30 2024-05-22T23:04:34.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.171+0000 7f3731e19740 1 -- 172.21.15.84:0/3973482829 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3720036040 con 0x56522d40bb30 2024-05-22T23:04:34.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.171+0000 7f3707fff640 1 -- 172.21.15.84:0/3973482829 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f37200bb230 con 0x56522d40bb30 2024-05-22T23:04:34.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.171+0000 7f3707fff640 1 -- 172.21.15.84:0/3973482829 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f37200c12e0 con 0x56522d40bb30 2024-05-22T23:04:34.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.171+0000 7f3731e19740 10 client.25385 fetch_fsmap learned FSMap version 18 2024-05-22T23:04:34.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.171+0000 7f3731e19740 1 -- 172.21.15.84:0/3973482829 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x56522d0783f0 con 0x56522d40bb30 2024-05-22T23:04:34.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.171+0000 7f3707fff640 1 -- 172.21.15.84:0/3973482829 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3720036040 con 0x56522d40bb30 2024-05-22T23:04:34.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.172+0000 7f3707fff640 1 -- 172.21.15.84:0/3973482829 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f372004e070 con 0x56522d40bb30 2024-05-22T23:04:34.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.172+0000 7f3731e19740 10 client.25385 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:04:34.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.172+0000 7f3731e19740 10 client.25385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:04:34.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.172+0000 7f3731e19740 20 client.25385 populate_metadata read hostname 'smithi084' 2024-05-22T23:04:34.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.172+0000 7f3731e19740 1 --2- 172.21.15.84:0/3973482829 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56522d41bb80 0x56522d41df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.172+0000 7f3731e19740 4 client.25385 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:04:34.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.172+0000 7f3731e19740 1 -- 172.21.15.84:0/3973482829 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56522d3724e0 con 0x56522d41bb80 2024-05-22T23:04:34.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.172+0000 7f372cdcd640 1 --2- 172.21.15.84:0/3973482829 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56522d41bb80 0x56522d41df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.172+0000 7f372cdcd640 1 --2- 172.21.15.84:0/3973482829 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56522d41bb80 0x56522d41df70 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c5daf8203e59682 server_cookie=d21fd1f185a4ac55 in_seq=0 out_seq=0 2024-05-22T23:04:34.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.172+0000 7f3707fff640 10 client.25385 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:34.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.174+0000 7f3707fff640 1 -- 172.21.15.84:0/3973482829 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56522d3724e0 con 0x56522d41bb80 2024-05-22T23:04:34.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.174+0000 7f3707fff640 10 client.25385 handle_command_reply: tid=0 2024-05-22T23:04:34.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.174+0000 7f3731e19740 1 client.25385 shutdown 2024-05-22T23:04:34.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.174+0000 7f3731e19740 1 -- 172.21.15.84:0/3973482829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56522d40bb30 msgr2=0x56522d409520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:34.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.174+0000 7f3731e19740 1 --2- 172.21.15.84:0/3973482829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56522d40bb30 0x56522d409520 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f37200bd0f0 tx=0x7f37200095c0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.174+0000 7f3731e19740 1 -- 172.21.15.84:0/3973482829 shutdown_connections 2024-05-22T23:04:34.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.174+0000 7f3731e19740 1 --2- 172.21.15.84:0/3973482829 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56522d41bb80 0x56522d41df70 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.174+0000 7f3731e19740 1 --2- 172.21.15.84:0/3973482829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56522d40bb30 0x56522d409520 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.174+0000 7f3707fff640 0 client.25385 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:34.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.174+0000 7f3731e19740 1 --2- 172.21.15.84:0/3973482829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56522d4024a0 0x56522d408fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.174+0000 7f3731e19740 1 --2- 172.21.15.84:0/3973482829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d400db0 0x56522d408aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.174+0000 7f3731e19740 1 -- 172.21.15.84:0/3973482829 >> 172.21.15.84:0/3973482829 conn(0x56522d3c98e0 msgr2=0x56522d403180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:34.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.174+0000 7f3731e19740 1 -- 172.21.15.84:0/3973482829 shutdown_connections 2024-05-22T23:04:34.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.174+0000 7f3731e19740 1 -- 172.21.15.84:0/3973482829 wait complete. 2024-05-22T23:04:34.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.174+0000 7f3731e19740 20 client.25385 trim_cache size 0 max 16384 2024-05-22T23:04:34.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.192+0000 7f3731e19740 1 Processor -- start 2024-05-22T23:04:34.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.192+0000 7f3731e19740 1 -- start start 2024-05-22T23:04:34.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.192+0000 7f3731e19740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56522d3c98e0 0x56522d40ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.192+0000 7f3731e19740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d400db0 0x56522d409340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.192+0000 7f3731e19740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56522d4024a0 0x56522d423780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.192+0000 7f3731e19740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f37281ab5e0 con 0x56522d400db0 2024-05-22T23:04:34.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.192+0000 7f3731e19740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f37281ade60 con 0x56522d3c98e0 2024-05-22T23:04:34.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.192+0000 7f3731e19740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f37281abc50 con 0x56522d4024a0 2024-05-22T23:04:34.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.192+0000 7f3727fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d400db0 0x56522d409340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.192+0000 7f3727fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d400db0 0x56522d409340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36758/0 (socket says 172.21.15.84:36758) 2024-05-22T23:04:34.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.192+0000 7f3727fff640 1 -- 172.21.15.84:0/2621123692 learned_addr learned my addr 172.21.15.84:0/2621123692 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:34.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.192+0000 7f372cdcd640 1 --2- 172.21.15.84:0/2621123692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56522d3c98e0 0x56522d40ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.193+0000 7f372d5ce640 1 --2- 172.21.15.84:0/2621123692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56522d4024a0 0x56522d423780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.193+0000 7f3727fff640 1 -- 172.21.15.84:0/2621123692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56522d4024a0 msgr2=0x56522d423780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:34.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.193+0000 7f3727fff640 1 --2- 172.21.15.84:0/2621123692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56522d4024a0 0x56522d423780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.193+0000 7f3727fff640 1 -- 172.21.15.84:0/2621123692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56522d3c98e0 msgr2=0x56522d40ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:34.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.193+0000 7f3727fff640 1 --2- 172.21.15.84:0/2621123692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56522d3c98e0 0x56522d40ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.193+0000 7f3727fff640 1 -- 172.21.15.84:0/2621123692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56522d0380f0 con 0x56522d400db0 2024-05-22T23:04:34.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.193+0000 7f3727fff640 1 --2- 172.21.15.84:0/2621123692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d400db0 0x56522d409340 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3728104e50 tx=0x7f371c012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c32253e70f9e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:34.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.193+0000 7f370e7fc640 1 -- 172.21.15.84:0/2621123692 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c005220 con 0x56522d400db0 2024-05-22T23:04:34.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.193+0000 7f370e7fc640 1 -- 172.21.15.84:0/2621123692 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f371c004c80 con 0x56522d400db0 2024-05-22T23:04:34.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.193+0000 7f370e7fc640 1 -- 172.21.15.84:0/2621123692 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c0107b0 con 0x56522d400db0 2024-05-22T23:04:34.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.194+0000 7f3731e19740 1 -- 172.21.15.84:0/2621123692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d400db0 msgr2=0x56522d409340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:34.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.194+0000 7f3731e19740 1 --2- 172.21.15.84:0/2621123692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d400db0 0x56522d409340 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3728104e50 tx=0x7f371c012040 comp rx=0 tx=0).stop 2024-05-22T23:04:34.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.194+0000 7f3731e19740 1 -- 172.21.15.84:0/2621123692 shutdown_connections 2024-05-22T23:04:34.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.194+0000 7f3731e19740 1 --2- 172.21.15.84:0/2621123692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56522d4024a0 0x56522d423780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.194+0000 7f3731e19740 1 --2- 172.21.15.84:0/2621123692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d400db0 0x56522d409340 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.194+0000 7f3731e19740 1 --2- 172.21.15.84:0/2621123692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56522d3c98e0 0x56522d40ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.194+0000 7f3731e19740 1 -- 172.21.15.84:0/2621123692 >> 172.21.15.84:0/2621123692 conn(0x56522d41e2e0 msgr2=0x56522d407ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:34.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.194+0000 7f3731e19740 1 -- 172.21.15.84:0/2621123692 shutdown_connections 2024-05-22T23:04:34.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.194+0000 7f3731e19740 1 -- 172.21.15.84:0/2621123692 wait complete. 2024-05-22T23:04:34.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.194+0000 7f3731e19740 1 -- start start 2024-05-22T23:04:34.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.194+0000 7f3731e19740 1 Processor -- start 2024-05-22T23:04:34.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.194+0000 7f3731e19740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56522d3c98e0 0x56522d47a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.194+0000 7f3731e19740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d400db0 0x56522d47a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.194+0000 7f3731e19740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56522d4024a0 0x56522d40ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.194+0000 7f3731e19740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f37281ac320 con 0x56522d400db0 2024-05-22T23:04:34.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.194+0000 7f3731e19740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f37281ad790 con 0x56522d3c98e0 2024-05-22T23:04:34.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.194+0000 7f3731e19740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f37281ac9f0 con 0x56522d4024a0 2024-05-22T23:04:34.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.195+0000 7f3731e19740 10 client.0 _ll_register_callbacks cb 0x7f3718009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:04:34.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.195+0000 7f3727fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d400db0 0x56522d47a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.195+0000 7f3727fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d400db0 0x56522d47a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36762/0 (socket says 172.21.15.84:36762) 2024-05-22T23:04:34.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.195+0000 7f3727fff640 1 -- 172.21.15.84:0/538249208 learned_addr learned my addr 172.21.15.84:0/538249208 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:34.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.195+0000 7f372cdcd640 1 --2- 172.21.15.84:0/538249208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56522d3c98e0 0x56522d47a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.195+0000 7f372d5ce640 1 --2- 172.21.15.84:0/538249208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56522d4024a0 0x56522d40ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.195+0000 7f3727fff640 1 -- 172.21.15.84:0/538249208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56522d4024a0 msgr2=0x56522d40ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:34.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.195+0000 7f3727fff640 1 --2- 172.21.15.84:0/538249208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56522d4024a0 0x56522d40ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.195+0000 7f3727fff640 1 -- 172.21.15.84:0/538249208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56522d3c98e0 msgr2=0x56522d47a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:34.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.195+0000 7f3727fff640 1 --2- 172.21.15.84:0/538249208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56522d3c98e0 0x56522d47a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.195+0000 7f3727fff640 1 -- 172.21.15.84:0/538249208 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56522d323fe0 con 0x56522d400db0 2024-05-22T23:04:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.195+0000 7f3727fff640 1 --2- 172.21.15.84:0/538249208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d400db0 0x56522d47a950 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f371c02b040 tx=0x7f371c00f4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2a70794fd61396 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.195+0000 7f3707fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:04:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.195+0000 7f3707fff640 1 -- 172.21.15.84:0/538249208 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c038070 con 0x56522d400db0 2024-05-22T23:04:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.195+0000 7f3707fff640 1 -- 172.21.15.84:0/538249208 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f371c005240 con 0x56522d400db0 2024-05-22T23:04:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.195+0000 7f3731e19740 1 -- 172.21.15.84:0/538249208 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f371c004c80 con 0x56522d400db0 2024-05-22T23:04:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.195+0000 7f3707fff640 1 -- 172.21.15.84:0/538249208 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f371c041050 con 0x56522d400db0 2024-05-22T23:04:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.195+0000 7f3731e19740 10 client.15816 fetch_fsmap learned FSMap version 18 2024-05-22T23:04:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.195+0000 7f3731e19740 1 -- 172.21.15.84:0/538249208 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x56522d2f51c0 con 0x56522d400db0 2024-05-22T23:04:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.196+0000 7f3707fff640 1 -- 172.21.15.84:0/538249208 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f371c004c80 con 0x56522d400db0 2024-05-22T23:04:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.196+0000 7f3707fff640 1 -- 172.21.15.84:0/538249208 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f371c03b020 con 0x56522d400db0 2024-05-22T23:04:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.196+0000 7f3731e19740 10 client.15816 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:04:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.196+0000 7f3731e19740 10 client.15816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:04:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.196+0000 7f3731e19740 20 client.15816 populate_metadata read hostname 'smithi084' 2024-05-22T23:04:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.196+0000 7f3731e19740 1 --2- 172.21.15.84:0/538249208 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56522d40bb30 0x56522d483fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.196+0000 7f3731e19740 4 client.15816 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:04:34.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.196+0000 7f3731e19740 1 -- 172.21.15.84:0/538249208 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56522d36c830 con 0x56522d40bb30 2024-05-22T23:04:34.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.196+0000 7f372cdcd640 1 --2- 172.21.15.84:0/538249208 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56522d40bb30 0x56522d483fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.196+0000 7f3707fff640 10 client.15816 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:34.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.196+0000 7f372cdcd640 1 --2- 172.21.15.84:0/538249208 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56522d40bb30 0x56522d483fc0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95ea3381ae10cee6 server_cookie=64057aa6b75a5678 in_seq=0 out_seq=0 2024-05-22T23:04:34.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.197+0000 7f3707fff640 1 -- 172.21.15.84:0/538249208 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56522d36c830 con 0x56522d40bb30 2024-05-22T23:04:34.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.197+0000 7f3707fff640 10 client.15816 handle_command_reply: tid=0 2024-05-22T23:04:34.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.197+0000 7f3731e19740 1 client.15816 shutdown 2024-05-22T23:04:34.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.198+0000 7f3731e19740 1 -- 172.21.15.84:0/538249208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d400db0 msgr2=0x56522d47a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:34.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.198+0000 7f3731e19740 1 --2- 172.21.15.84:0/538249208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d400db0 0x56522d47a950 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f371c02b040 tx=0x7f371c00f4e0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.198+0000 7f3731e19740 1 -- 172.21.15.84:0/538249208 shutdown_connections 2024-05-22T23:04:34.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.198+0000 7f3731e19740 1 --2- 172.21.15.84:0/538249208 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56522d40bb30 0x56522d483fc0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.198+0000 7f3731e19740 1 --2- 172.21.15.84:0/538249208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56522d4024a0 0x56522d40ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.198+0000 7f3731e19740 1 --2- 172.21.15.84:0/538249208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56522d400db0 0x56522d47a950 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.198+0000 7f3707fff640 0 client.15816 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:34.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.198+0000 7f3731e19740 1 --2- 172.21.15.84:0/538249208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56522d3c98e0 0x56522d47a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.198+0000 7f3731e19740 1 -- 172.21.15.84:0/538249208 >> 172.21.15.84:0/538249208 conn(0x56522d41e2e0 msgr2=0x56522d416630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:34.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.198+0000 7f3731e19740 1 -- 172.21.15.84:0/538249208 shutdown_connections 2024-05-22T23:04:34.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.198+0000 7f3731e19740 1 -- 172.21.15.84:0/538249208 wait complete. 2024-05-22T23:04:34.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.198+0000 7f3731e19740 20 client.15816 trim_cache size 0 max 16384 2024-05-22T23:04:34.207 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:04:34.207 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "no active scrubs running", 2024-05-22T23:04:34.207 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": {} 2024-05-22T23:04:34.207 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:04:34.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.198+0000 7f372f031640 1 -- 172.21.15.84:0/4008871949 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f37100a2d80 msgr2=0x7f37100a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:34.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.198+0000 7f372f031640 1 --2- 172.21.15.84:0/4008871949 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f37100a2d80 0x7f37100a5240 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f371c004650 tx=0x7f371c013880 comp rx=0 tx=0).stop 2024-05-22T23:04:34.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.198+0000 7f372f031640 1 -- 172.21.15.84:0/4008871949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3728106750 msgr2=0x7f372819c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:34.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.198+0000 7f372f031640 1 --2- 172.21.15.84:0/4008871949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3728106750 0x7f372819c9e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3720005cc0 tx=0x7f37200093d0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.198+0000 7f372f031640 1 -- 172.21.15.84:0/4008871949 shutdown_connections 2024-05-22T23:04:34.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.198+0000 7f372f031640 1 --2- 172.21.15.84:0/4008871949 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f37100a2d80 0x7f37100a5240 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.198+0000 7f372f031640 1 --2- 172.21.15.84:0/4008871949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3728106750 0x7f372819c9e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.198+0000 7f372f031640 1 --2- 172.21.15.84:0/4008871949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3728101100 0x7f3728198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.198+0000 7f372f031640 1 --2- 172.21.15.84:0/4008871949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3728100740 0x7f3728198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.198+0000 7f372f031640 1 -- 172.21.15.84:0/4008871949 >> 172.21.15.84:0/4008871949 conn(0x7f3728077870 msgr2=0x7f3728103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:34.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.198+0000 7f372f031640 1 -- 172.21.15.84:0/4008871949 shutdown_connections 2024-05-22T23:04:34.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.198+0000 7f372f031640 1 -- 172.21.15.84:0/4008871949 wait complete. 2024-05-22T23:04:34.213 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-22T23:04:34.213 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:04:34.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.312+0000 7f1fd7b70640 1 -- 172.21.15.84:0/3961645368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fd0107850 msgr2=0x7f1fd0107c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:34.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.312+0000 7f1fd7b70640 1 --2- 172.21.15.84:0/3961645368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fd0107850 0x7f1fd0107c30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc009930 tx=0x7f1fcc02a420 comp rx=0 tx=0).stop 2024-05-22T23:04:34.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.312+0000 7f1fd7b70640 1 -- 172.21.15.84:0/3961645368 shutdown_connections 2024-05-22T23:04:34.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.312+0000 7f1fd7b70640 1 --2- 172.21.15.84:0/3961645368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fd0102200 0x7f1fd010e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.312+0000 7f1fd7b70640 1 --2- 172.21.15.84:0/3961645368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fd0101840 0x7f1fd0101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.312+0000 7f1fd7b70640 1 --2- 172.21.15.84:0/3961645368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fd0107850 0x7f1fd0107c30 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.312+0000 7f1fd7b70640 1 -- 172.21.15.84:0/3961645368 >> 172.21.15.84:0/3961645368 conn(0x7f1fd0078c30 msgr2=0x7f1fd0079040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:34.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.312+0000 7f1fd7b70640 1 -- 172.21.15.84:0/3961645368 shutdown_connections 2024-05-22T23:04:34.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.312+0000 7f1fd7b70640 1 -- 172.21.15.84:0/3961645368 wait complete. 2024-05-22T23:04:34.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.313+0000 7f1fd7b70640 1 Processor -- start 2024-05-22T23:04:34.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.313+0000 7f1fd7b70640 1 -- start start 2024-05-22T23:04:34.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.313+0000 7f1fd7b70640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fd0101840 0x7f1fd0198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.313+0000 7f1fd7b70640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fd0102200 0x7f1fd0198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.313+0000 7f1fd7b70640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fd0107850 0x7f1fd019c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.313+0000 7f1fd7b70640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd0111c20 con 0x7f1fd0107850 2024-05-22T23:04:34.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.313+0000 7f1fd7b70640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd0111aa0 con 0x7f1fd0102200 2024-05-22T23:04:34.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.313+0000 7f1fd7b70640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd0111da0 con 0x7f1fd0101840 2024-05-22T23:04:34.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.313+0000 7f1fd610d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fd0107850 0x7f1fd019c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.313+0000 7f1fd510b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fd0102200 0x7f1fd0198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.313+0000 7f1fd610d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fd0107850 0x7f1fd019c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36792/0 (socket says 172.21.15.84:36792) 2024-05-22T23:04:34.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.313+0000 7f1fd510b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fd0102200 0x7f1fd0198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:43388/0 (socket says 172.21.15.84:43388) 2024-05-22T23:04:34.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.313+0000 7f1fd590c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fd0101840 0x7f1fd0198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.313+0000 7f1fd610d640 1 -- 172.21.15.84:0/855648614 learned_addr learned my addr 172.21.15.84:0/855648614 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:34.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.313+0000 7f1fd610d640 1 -- 172.21.15.84:0/855648614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fd0101840 msgr2=0x7f1fd0198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:34.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.313+0000 7f1fd610d640 1 --2- 172.21.15.84:0/855648614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fd0101840 0x7f1fd0198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.313+0000 7f1fd610d640 1 -- 172.21.15.84:0/855648614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fd0102200 msgr2=0x7f1fd0198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:34.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.313+0000 7f1fd610d640 1 --2- 172.21.15.84:0/855648614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fd0102200 0x7f1fd0198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.313+0000 7f1fd610d640 1 -- 172.21.15.84:0/855648614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd019d0c0 con 0x7f1fd0107850 2024-05-22T23:04:34.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.313+0000 7f1fd510b640 1 --2- 172.21.15.84:0/855648614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fd0102200 0x7f1fd0198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:34.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.313+0000 7f1fd610d640 1 --2- 172.21.15.84:0/855648614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fd0107850 0x7f1fd019c990 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800eb10 tx=0x7f1fc800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:34.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.314+0000 7f1fc6ffd640 1 -- 172.21.15.84:0/855648614 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc800ce20 con 0x7f1fd0107850 2024-05-22T23:04:34.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.314+0000 7f1fc6ffd640 1 -- 172.21.15.84:0/855648614 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fc8005020 con 0x7f1fd0107850 2024-05-22T23:04:34.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.314+0000 7f1fd7b70640 1 -- 172.21.15.84:0/855648614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fd019d3b0 con 0x7f1fd0107850 2024-05-22T23:04:34.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.314+0000 7f1fc6ffd640 1 -- 172.21.15.84:0/855648614 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8010690 con 0x7f1fd0107850 2024-05-22T23:04:34.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.314+0000 7f1fd7b70640 1 -- 172.21.15.84:0/855648614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fd01a4c40 con 0x7f1fd0107850 2024-05-22T23:04:34.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.315+0000 7f1fd7b70640 1 -- 172.21.15.84:0/855648614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fd0068ae0 con 0x7f1fd0107850 2024-05-22T23:04:34.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.316+0000 7f1fc6ffd640 1 -- 172.21.15.84:0/855648614 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1fc8005540 con 0x7f1fd0107850 2024-05-22T23:04:34.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.316+0000 7f1fc6ffd640 1 --2- 172.21.15.84:0/855648614 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f1fbc0a2d00 0x7f1fbc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.322 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.317+0000 7f1fd590c640 1 --2- 172.21.15.84:0/855648614 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f1fbc0a2d00 0x7f1fbc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.322 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.317+0000 7f1fc6ffd640 1 -- 172.21.15.84:0/855648614 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f1fc80cf950 con 0x7f1fd0107850 2024-05-22T23:04:34.322 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.317+0000 7f1fd590c640 1 --2- 172.21.15.84:0/855648614 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f1fbc0a2d00 0x7f1fbc0a51c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc009900 tx=0x7f1fcc0058d0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:34.324 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.321+0000 7f1fc6ffd640 1 -- 172.21.15.84:0/855648614 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1fc809c570 con 0x7f1fd0107850 2024-05-22T23:04:34.427 INFO:tasks.workunit.client.0.smithi084.stdout: 1 238375 16.28 MB/sec warmup 80 sec latency 23.626 ms 2024-05-22T23:04:34.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.545+0000 7f1fd7b70640 1 -- 172.21.15.84:0/855648614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1fd002d200 con 0x7f1fd0107850 2024-05-22T23:04:34.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.546+0000 7f1fc6ffd640 1 -- 172.21.15.84:0/855648614 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7f1fc809be70 con 0x7f1fd0107850 2024-05-22T23:04:34.550 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:04:34.550 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:04:34.550 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:04:34.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.548+0000 7f1fd7b70640 1 -- 172.21.15.84:0/855648614 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f1fbc0a2d00 msgr2=0x7f1fbc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:34.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.548+0000 7f1fd7b70640 1 --2- 172.21.15.84:0/855648614 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f1fbc0a2d00 0x7f1fbc0a51c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc009900 tx=0x7f1fcc0058d0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.548+0000 7f1fd7b70640 1 -- 172.21.15.84:0/855648614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fd0107850 msgr2=0x7f1fd019c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:34.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.548+0000 7f1fd7b70640 1 --2- 172.21.15.84:0/855648614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fd0107850 0x7f1fd019c990 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800eb10 tx=0x7f1fc800ef10 comp rx=0 tx=0).stop 2024-05-22T23:04:34.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.548+0000 7f1fd7b70640 1 -- 172.21.15.84:0/855648614 shutdown_connections 2024-05-22T23:04:34.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.548+0000 7f1fd7b70640 1 --2- 172.21.15.84:0/855648614 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f1fbc0a2d00 0x7f1fbc0a51c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.548+0000 7f1fd7b70640 1 --2- 172.21.15.84:0/855648614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fd0107850 0x7f1fd019c990 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.548+0000 7f1fd7b70640 1 --2- 172.21.15.84:0/855648614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fd0102200 0x7f1fd0198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.548+0000 7f1fd7b70640 1 --2- 172.21.15.84:0/855648614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fd0101840 0x7f1fd0198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.548+0000 7f1fd7b70640 1 -- 172.21.15.84:0/855648614 >> 172.21.15.84:0/855648614 conn(0x7f1fd0078c30 msgr2=0x7f1fd0104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:34.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.549+0000 7f1fd7b70640 1 -- 172.21.15.84:0/855648614 shutdown_connections 2024-05-22T23:04:34.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.549+0000 7f1fd7b70640 1 -- 172.21.15.84:0/855648614 wait complete. 2024-05-22T23:04:34.563 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-22T23:04:34.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.664+0000 7fc8ff053640 1 -- 172.21.15.84:0/357244130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8f810be30 msgr2=0x7fc8f810e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:34.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.664+0000 7fc8ff053640 1 --2- 172.21.15.84:0/357244130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8f810be30 0x7fc8f810e2d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0009ca0 tx=0x7fc8f002a9e0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.664+0000 7fc8ff053640 1 -- 172.21.15.84:0/357244130 shutdown_connections 2024-05-22T23:04:34.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.664+0000 7fc8ff053640 1 --2- 172.21.15.84:0/357244130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8f810be30 0x7fc8f810e2d0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.664+0000 7fc8ff053640 1 --2- 172.21.15.84:0/357244130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8f8103410 0x7fc8f810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.664+0000 7fc8ff053640 1 --2- 172.21.15.84:0/357244130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8f8102af0 0x7fc8f8102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.664+0000 7fc8ff053640 1 -- 172.21.15.84:0/357244130 >> 172.21.15.84:0/357244130 conn(0x7fc8f8077870 msgr2=0x7fc8f8077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:34.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.665+0000 7fc8ff053640 1 -- 172.21.15.84:0/357244130 shutdown_connections 2024-05-22T23:04:34.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.665+0000 7fc8ff053640 1 -- 172.21.15.84:0/357244130 wait complete. 2024-05-22T23:04:34.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.665+0000 7fc8ff053640 1 Processor -- start 2024-05-22T23:04:34.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.665+0000 7fc8ff053640 1 -- start start 2024-05-22T23:04:34.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.665+0000 7fc8ff053640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8f8102af0 0x7fc8f8198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.665+0000 7fc8ff053640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8f8103410 0x7fc8f8198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.665+0000 7fc8ff053640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8f810be30 0x7fc8f819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.665+0000 7fc8ff053640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f8111c30 con 0x7fc8f810be30 2024-05-22T23:04:34.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.665+0000 7fc8ff053640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f8111ab0 con 0x7fc8f8102af0 2024-05-22T23:04:34.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.665+0000 7fc8ff053640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f8111db0 con 0x7fc8f8103410 2024-05-22T23:04:34.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.665+0000 7fc8fd5f0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8f810be30 0x7fc8f819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.665+0000 7fc8fcdef640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8f8102af0 0x7fc8f8198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.665+0000 7fc8fd5f0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8f810be30 0x7fc8f819c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36810/0 (socket says 172.21.15.84:36810) 2024-05-22T23:04:34.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.665+0000 7fc8fd5f0640 1 -- 172.21.15.84:0/4174028274 learned_addr learned my addr 172.21.15.84:0/4174028274 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:34.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.665+0000 7fc8effff640 1 --2- 172.21.15.84:0/4174028274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8f8103410 0x7fc8f8198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.666+0000 7fc8fcdef640 1 -- 172.21.15.84:0/4174028274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8f8103410 msgr2=0x7fc8f8198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:34.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.666+0000 7fc8fcdef640 1 --2- 172.21.15.84:0/4174028274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8f8103410 0x7fc8f8198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.666+0000 7fc8fcdef640 1 -- 172.21.15.84:0/4174028274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8f810be30 msgr2=0x7fc8f819c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:34.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.666+0000 7fc8fcdef640 1 --2- 172.21.15.84:0/4174028274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8f810be30 0x7fc8f819c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.666+0000 7fc8fcdef640 1 -- 172.21.15.84:0/4174028274 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f819d0d0 con 0x7fc8f8102af0 2024-05-22T23:04:34.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.666+0000 7fc8fd5f0640 1 --2- 172.21.15.84:0/4174028274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8f810be30 0x7fc8f819c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:34.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.666+0000 7fc8fcdef640 1 --2- 172.21.15.84:0/4174028274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8f8102af0 0x7fc8f8198130 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400e9f0 tx=0x7fc8f400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:34.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.666+0000 7fc8edffb640 1 -- 172.21.15.84:0/4174028274 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f400ccc0 con 0x7fc8f8102af0 2024-05-22T23:04:34.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.666+0000 7fc8ff053640 1 -- 172.21.15.84:0/4174028274 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f819d3c0 con 0x7fc8f8102af0 2024-05-22T23:04:34.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.666+0000 7fc8edffb640 1 -- 172.21.15.84:0/4174028274 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f4010910 con 0x7fc8f8102af0 2024-05-22T23:04:34.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.666+0000 7fc8edffb640 1 -- 172.21.15.84:0/4174028274 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4018960 con 0x7fc8f8102af0 2024-05-22T23:04:34.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.666+0000 7fc8ff053640 1 -- 172.21.15.84:0/4174028274 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f8100670 con 0x7fc8f8102af0 2024-05-22T23:04:34.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.669+0000 7fc8edffb640 1 -- 172.21.15.84:0/4174028274 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc8f4010ac0 con 0x7fc8f8102af0 2024-05-22T23:04:34.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.669+0000 7fc901e35740 1 Processor -- start 2024-05-22T23:04:34.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.669+0000 7fc901e35740 1 -- start start 2024-05-22T23:04:34.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.669+0000 7fc901e35740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610ffcf7f80 0x5610ffcf8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.669+0000 7fc901e35740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610ffcf88a0 0x5610ffd02a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.669+0000 7fc901e35740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffd02f70 0x5610ffd05360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.669+0000 7fc901e35740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5610ffc5cdb0 con 0x5610ffd02f70 2024-05-22T23:04:34.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.669+0000 7fc901e35740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5610ffbfb400 con 0x5610ffcf88a0 2024-05-22T23:04:34.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.669+0000 7fc901e35740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5610ffc58d70 con 0x5610ffcf7f80 2024-05-22T23:04:34.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.669+0000 7fc8fcdef640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610ffcf88a0 0x5610ffd02a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.669+0000 7fc8effff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffd02f70 0x5610ffd05360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.669+0000 7fc8effff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffd02f70 0x5610ffd05360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36812/0 (socket says 172.21.15.84:36812) 2024-05-22T23:04:34.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.669+0000 7fc8effff640 1 -- 172.21.15.84:0/2684773057 learned_addr learned my addr 172.21.15.84:0/2684773057 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:34.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.669+0000 7fc8fd5f0640 1 --2- 172.21.15.84:0/2684773057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610ffcf7f80 0x5610ffcf8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.669+0000 7fc8effff640 1 -- 172.21.15.84:0/2684773057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610ffcf7f80 msgr2=0x5610ffcf8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:34.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.669+0000 7fc8effff640 1 --2- 172.21.15.84:0/2684773057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610ffcf7f80 0x5610ffcf8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.669+0000 7fc8effff640 1 -- 172.21.15.84:0/2684773057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610ffcf88a0 msgr2=0x5610ffd02a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:34.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.669+0000 7fc8effff640 1 --2- 172.21.15.84:0/2684773057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610ffcf88a0 0x5610ffd02a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.669+0000 7fc8effff640 1 -- 172.21.15.84:0/2684773057 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610ffbed2f0 con 0x5610ffd02f70 2024-05-22T23:04:34.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.669+0000 7fc8fcdef640 1 --2- 172.21.15.84:0/2684773057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610ffcf88a0 0x5610ffd02a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:34.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.669+0000 7fc8edffb640 1 --2- 172.21.15.84:0/4174028274 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fc8e00a2d00 0x7fc8e00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.670+0000 7fc8effff640 1 --2- 172.21.15.84:0/2684773057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffd02f70 0x5610ffd05360 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8009c20 tx=0x7fc8e8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f6513f7ac84831 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:34.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.670+0000 7fc8fd5f0640 1 --2- 172.21.15.84:0/4174028274 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fc8e00a2d00 0x7fc8e00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.670+0000 7fc8edffb640 1 -- 172.21.15.84:0/4174028274 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fc8f40cf840 con 0x7fc8f8102af0 2024-05-22T23:04:34.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.670+0000 7fc8deffd640 1 -- 172.21.15.84:0/2684773057 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8004230 con 0x5610ffd02f70 2024-05-22T23:04:34.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.670+0000 7fc8deffd640 1 -- 172.21.15.84:0/2684773057 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8e80043d0 con 0x5610ffd02f70 2024-05-22T23:04:34.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.670+0000 7fc8deffd640 1 -- 172.21.15.84:0/2684773057 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8004db0 con 0x5610ffd02f70 2024-05-22T23:04:34.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.671+0000 7fc901e35740 1 -- 172.21.15.84:0/2684773057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffd02f70 msgr2=0x5610ffd05360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:34.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.671+0000 7fc901e35740 1 --2- 172.21.15.84:0/2684773057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffd02f70 0x5610ffd05360 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8009c20 tx=0x7fc8e8011810 comp rx=0 tx=0).stop 2024-05-22T23:04:34.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.671+0000 7fc8fd5f0640 1 --2- 172.21.15.84:0/4174028274 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fc8e00a2d00 0x7fc8e00a51c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc8f002f040 tx=0x7fc8f003a040 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:34.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.671+0000 7fc901e35740 1 -- 172.21.15.84:0/2684773057 shutdown_connections 2024-05-22T23:04:34.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.671+0000 7fc901e35740 1 --2- 172.21.15.84:0/2684773057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffd02f70 0x5610ffd05360 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.671+0000 7fc901e35740 1 --2- 172.21.15.84:0/2684773057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610ffcf88a0 0x5610ffd02a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.671+0000 7fc901e35740 1 --2- 172.21.15.84:0/2684773057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610ffcf7f80 0x5610ffcf8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.671+0000 7fc901e35740 1 -- 172.21.15.84:0/2684773057 >> 172.21.15.84:0/2684773057 conn(0x5610ffcf42a0 msgr2=0x5610ffcf4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:34.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.671+0000 7fc901e35740 1 -- 172.21.15.84:0/2684773057 shutdown_connections 2024-05-22T23:04:34.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.671+0000 7fc901e35740 1 -- 172.21.15.84:0/2684773057 wait complete. 2024-05-22T23:04:34.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.671+0000 7fc901e35740 1 -- start start 2024-05-22T23:04:34.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.671+0000 7fc901e35740 1 Processor -- start 2024-05-22T23:04:34.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc901e35740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffcf7f80 0x5610ffcfff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc901e35740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610ffcf88a0 0x5610ffd00470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc901e35740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610ffd02f70 0x5610ffd009b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc901e35740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5610ffd05d60 con 0x5610ffcf7f80 2024-05-22T23:04:34.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc901e35740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5610ffc172e0 con 0x5610ffd02f70 2024-05-22T23:04:34.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc901e35740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5610ffd05ee0 con 0x5610ffcf88a0 2024-05-22T23:04:34.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc8fcdef640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffcf7f80 0x5610ffcfff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc901e35740 10 client.0 _ll_register_callbacks cb 0x5610ffc20a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:04:34.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc8fcdef640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffcf7f80 0x5610ffcfff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36814/0 (socket says 172.21.15.84:36814) 2024-05-22T23:04:34.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc8fcdef640 1 -- 172.21.15.84:0/706182723 learned_addr learned my addr 172.21.15.84:0/706182723 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:34.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc8effff640 1 --2- 172.21.15.84:0/706182723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610ffcf88a0 0x5610ffd00470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc8fd5f0640 1 --2- 172.21.15.84:0/706182723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610ffd02f70 0x5610ffd009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc8fcdef640 1 -- 172.21.15.84:0/706182723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610ffcf88a0 msgr2=0x5610ffd00470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:34.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc8fcdef640 1 --2- 172.21.15.84:0/706182723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610ffcf88a0 0x5610ffd00470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc8fcdef640 1 -- 172.21.15.84:0/706182723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610ffd02f70 msgr2=0x5610ffd009b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:34.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc8fcdef640 1 --2- 172.21.15.84:0/706182723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610ffd02f70 0x5610ffd009b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc8fcdef640 1 -- 172.21.15.84:0/706182723 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610ff9703f0 con 0x5610ffcf7f80 2024-05-22T23:04:34.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc8fd5f0640 1 --2- 172.21.15.84:0/706182723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610ffd02f70 0x5610ffd009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:04:34.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc8fcdef640 1 --2- 172.21.15.84:0/706182723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffcf7f80 0x5610ffcfff30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400c6a0 tx=0x7fc8f4010ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23237e5a51eb3b5b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:34.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc8d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:04:34.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc8d7fff640 1 -- 172.21.15.84:0/706182723 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f40cd3c0 con 0x5610ffcf7f80 2024-05-22T23:04:34.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc8d7fff640 1 -- 172.21.15.84:0/706182723 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fc8f409b400 con 0x5610ffcf7f80 2024-05-22T23:04:34.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc901e35740 1 -- 172.21.15.84:0/706182723 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8e80043d0 con 0x5610ffcf7f80 2024-05-22T23:04:34.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc8d7fff640 1 -- 172.21.15.84:0/706182723 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fc8f409bdb0 con 0x5610ffcf7f80 2024-05-22T23:04:34.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc901e35740 10 client.15834 fetch_fsmap learned FSMap version 18 2024-05-22T23:04:34.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.672+0000 7fc901e35740 1 -- 172.21.15.84:0/706182723 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5610ffa3a7d0 con 0x5610ffcf7f80 2024-05-22T23:04:34.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.673+0000 7fc8d7fff640 1 -- 172.21.15.84:0/706182723 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8e80043d0 con 0x5610ffcf7f80 2024-05-22T23:04:34.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.673+0000 7fc8d7fff640 1 -- 172.21.15.84:0/706182723 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fc8f4086040 con 0x5610ffcf7f80 2024-05-22T23:04:34.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.673+0000 7fc901e35740 10 client.15834 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:04:34.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.673+0000 7fc901e35740 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:04:34.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.673+0000 7fc901e35740 20 client.15834 populate_metadata read hostname 'smithi084' 2024-05-22T23:04:34.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.673+0000 7fc901e35740 1 --2- 172.21.15.84:0/706182723 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5610ffd13280 0x5610ffd15670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.673+0000 7fc901e35740 4 client.15834 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:04:34.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.673+0000 7fc901e35740 1 -- 172.21.15.84:0/706182723 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610ffc6a4e0 con 0x5610ffd13280 2024-05-22T23:04:34.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.673+0000 7fc8effff640 1 --2- 172.21.15.84:0/706182723 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5610ffd13280 0x5610ffd15670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.673+0000 7fc8effff640 1 --2- 172.21.15.84:0/706182723 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5610ffd13280 0x5610ffd15670 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c243d5575dca87e7 server_cookie=51712823f241608b in_seq=0 out_seq=0 2024-05-22T23:04:34.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.673+0000 7fc8d7fff640 10 client.15834 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:34.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.675+0000 7fc8d7fff640 1 -- 172.21.15.84:0/706182723 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5610ffc6a4e0 con 0x5610ffd13280 2024-05-22T23:04:34.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.675+0000 7fc8d7fff640 10 client.15834 handle_command_reply: tid=0 2024-05-22T23:04:34.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.675+0000 7fc901e35740 1 client.15834 shutdown 2024-05-22T23:04:34.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.675+0000 7fc901e35740 1 -- 172.21.15.84:0/706182723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffcf7f80 msgr2=0x5610ffcfff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:34.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.675+0000 7fc901e35740 1 --2- 172.21.15.84:0/706182723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffcf7f80 0x5610ffcfff30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400c6a0 tx=0x7fc8f4010ee0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.675+0000 7fc901e35740 1 -- 172.21.15.84:0/706182723 shutdown_connections 2024-05-22T23:04:34.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.675+0000 7fc901e35740 1 --2- 172.21.15.84:0/706182723 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5610ffd13280 0x5610ffd15670 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.675+0000 7fc901e35740 1 --2- 172.21.15.84:0/706182723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610ffd02f70 0x5610ffd009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.675+0000 7fc901e35740 1 --2- 172.21.15.84:0/706182723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610ffcf88a0 0x5610ffd00470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.675+0000 7fc901e35740 1 --2- 172.21.15.84:0/706182723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffcf7f80 0x5610ffcfff30 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.675+0000 7fc8d7fff640 0 client.15834 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:34.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.675+0000 7fc901e35740 1 -- 172.21.15.84:0/706182723 >> 172.21.15.84:0/706182723 conn(0x5610ffcf42a0 msgr2=0x5610ffcfa0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:34.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.676+0000 7fc901e35740 1 -- 172.21.15.84:0/706182723 shutdown_connections 2024-05-22T23:04:34.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.676+0000 7fc901e35740 1 -- 172.21.15.84:0/706182723 wait complete. 2024-05-22T23:04:34.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.676+0000 7fc901e35740 20 client.15834 trim_cache size 0 max 16384 2024-05-22T23:04:34.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.689+0000 7fc901e35740 1 Processor -- start 2024-05-22T23:04:34.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.689+0000 7fc901e35740 1 -- start start 2024-05-22T23:04:34.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.689+0000 7fc901e35740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610ffcf7f80 0x5610ffd02660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.689+0000 7fc901e35740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffcf88a0 0x5610ffcfa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.689+0000 7fc901e35740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610ffd02f70 0x5610ffd007d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.689+0000 7fc901e35740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f81a9390 con 0x5610ffcf88a0 2024-05-22T23:04:34.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.689+0000 7fc901e35740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f81abc10 con 0x5610ffd02f70 2024-05-22T23:04:34.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.689+0000 7fc901e35740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f81a9a00 con 0x5610ffcf7f80 2024-05-22T23:04:34.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.690+0000 7fc8effff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffcf88a0 0x5610ffcfa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.690+0000 7fc8fcdef640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610ffcf7f80 0x5610ffd02660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.690+0000 7fc8effff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffcf88a0 0x5610ffcfa6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36826/0 (socket says 172.21.15.84:36826) 2024-05-22T23:04:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.690+0000 7fc8fcdef640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610ffcf7f80 0x5610ffd02660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:57470/0 (socket says 172.21.15.84:57470) 2024-05-22T23:04:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.690+0000 7fc8effff640 1 -- 172.21.15.84:0/874775261 learned_addr learned my addr 172.21.15.84:0/874775261 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.690+0000 7fc8fd5f0640 1 --2- 172.21.15.84:0/874775261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610ffd02f70 0x5610ffd007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.690+0000 7fc8effff640 1 -- 172.21.15.84:0/874775261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610ffcf7f80 msgr2=0x5610ffd02660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.690+0000 7fc8effff640 1 --2- 172.21.15.84:0/874775261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610ffcf7f80 0x5610ffd02660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.690+0000 7fc8effff640 1 -- 172.21.15.84:0/874775261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610ffd02f70 msgr2=0x5610ffd007d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.690+0000 7fc8effff640 1 --2- 172.21.15.84:0/874775261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610ffd02f70 0x5610ffd007d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.690+0000 7fc8effff640 1 -- 172.21.15.84:0/874775261 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610ffcfeee0 con 0x5610ffcf88a0 2024-05-22T23:04:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.690+0000 7fc8effff640 1 --2- 172.21.15.84:0/874775261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffcf88a0 0x5610ffcfa6e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8014040 tx=0x7fc8e8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8fb1d23d8825148 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.690+0000 7fc8de7fc640 1 -- 172.21.15.84:0/874775261 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8031070 con 0x5610ffcf88a0 2024-05-22T23:04:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.690+0000 7fc8de7fc640 1 -- 172.21.15.84:0/874775261 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8e801f040 con 0x5610ffcf88a0 2024-05-22T23:04:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.690+0000 7fc8de7fc640 1 -- 172.21.15.84:0/874775261 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e801e040 con 0x5610ffcf88a0 2024-05-22T23:04:34.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.691+0000 7fc901e35740 1 -- 172.21.15.84:0/874775261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffcf88a0 msgr2=0x5610ffcfa6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:34.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.691+0000 7fc901e35740 1 --2- 172.21.15.84:0/874775261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffcf88a0 0x5610ffcfa6e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8014040 tx=0x7fc8e8005780 comp rx=0 tx=0).stop 2024-05-22T23:04:34.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.691+0000 7fc901e35740 1 -- 172.21.15.84:0/874775261 shutdown_connections 2024-05-22T23:04:34.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.691+0000 7fc901e35740 1 --2- 172.21.15.84:0/874775261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610ffd02f70 0x5610ffd007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.691+0000 7fc901e35740 1 --2- 172.21.15.84:0/874775261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffcf88a0 0x5610ffcfa6e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.691+0000 7fc901e35740 1 --2- 172.21.15.84:0/874775261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610ffcf7f80 0x5610ffd02660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.691+0000 7fc901e35740 1 -- 172.21.15.84:0/874775261 >> 172.21.15.84:0/874775261 conn(0x5610ffcf42a0 msgr2=0x5610ffcf34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:34.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.691+0000 7fc901e35740 1 -- 172.21.15.84:0/874775261 shutdown_connections 2024-05-22T23:04:34.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.691+0000 7fc901e35740 1 -- 172.21.15.84:0/874775261 wait complete. 2024-05-22T23:04:34.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.691+0000 7fc901e35740 1 -- start start 2024-05-22T23:04:34.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.691+0000 7fc901e35740 1 Processor -- start 2024-05-22T23:04:34.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.692+0000 7fc901e35740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610ffcf7f80 0x5610ffd06b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.692+0000 7fc901e35740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610ffcf88a0 0x5610ffd070a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.692+0000 7fc901e35740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffd02f70 0x5610ffd075e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.692+0000 7fc901e35740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f81aa0d0 con 0x5610ffd02f70 2024-05-22T23:04:34.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.692+0000 7fc901e35740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f81ab540 con 0x5610ffcf7f80 2024-05-22T23:04:34.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.692+0000 7fc901e35740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f81aa7a0 con 0x5610ffcf88a0 2024-05-22T23:04:34.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.692+0000 7fc901e35740 10 client.0 _ll_register_callbacks cb 0x7fc8f40d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:04:34.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.692+0000 7fc8fd5f0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffd02f70 0x5610ffd075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.692+0000 7fc8fcdef640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610ffcf7f80 0x5610ffd06b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.692+0000 7fc8fcdef640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610ffcf7f80 0x5610ffd06b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:43440/0 (socket says 172.21.15.84:43440) 2024-05-22T23:04:34.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.692+0000 7fc8fd5f0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffd02f70 0x5610ffd075e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36832/0 (socket says 172.21.15.84:36832) 2024-05-22T23:04:34.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.692+0000 7fc8fcdef640 1 -- 172.21.15.84:0/1892429308 learned_addr learned my addr 172.21.15.84:0/1892429308 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:34.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.692+0000 7fc8effff640 1 --2- 172.21.15.84:0/1892429308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610ffcf88a0 0x5610ffd070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.692+0000 7fc8fd5f0640 1 -- 172.21.15.84:0/1892429308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610ffcf88a0 msgr2=0x5610ffd070a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:34.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.692+0000 7fc8fd5f0640 1 --2- 172.21.15.84:0/1892429308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610ffcf88a0 0x5610ffd070a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.692+0000 7fc8fd5f0640 1 -- 172.21.15.84:0/1892429308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610ffcf7f80 msgr2=0x5610ffd06b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:34.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.692+0000 7fc8fd5f0640 1 --2- 172.21.15.84:0/1892429308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610ffcf7f80 0x5610ffd06b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.692+0000 7fc8fd5f0640 1 -- 172.21.15.84:0/1892429308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610ffd787e0 con 0x5610ffd02f70 2024-05-22T23:04:34.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.692+0000 7fc8fcdef640 1 --2- 172.21.15.84:0/1892429308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610ffcf7f80 0x5610ffd06b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:34.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.692+0000 7fc8fd5f0640 1 --2- 172.21.15.84:0/1892429308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffd02f70 0x5610ffd075e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0036550 tx=0x7fc8f002a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1a5c4b50db0982 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:34.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.692+0000 7fc8d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:04:34.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.692+0000 7fc8d7fff640 1 -- 172.21.15.84:0/1892429308 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0055070 con 0x5610ffd02f70 2024-05-22T23:04:34.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.692+0000 7fc8d7fff640 1 -- 172.21.15.84:0/1892429308 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fc8f00045d0 con 0x5610ffd02f70 2024-05-22T23:04:34.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.692+0000 7fc901e35740 1 -- 172.21.15.84:0/1892429308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8e801f040 con 0x5610ffd02f70 2024-05-22T23:04:34.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.693+0000 7fc8d7fff640 1 -- 172.21.15.84:0/1892429308 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fc8f0061050 con 0x5610ffd02f70 2024-05-22T23:04:34.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.693+0000 7fc901e35740 10 client.15840 fetch_fsmap learned FSMap version 18 2024-05-22T23:04:34.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.693+0000 7fc901e35740 1 -- 172.21.15.84:0/1892429308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5610ffd79aa0 con 0x5610ffd02f70 2024-05-22T23:04:34.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.693+0000 7fc8d7fff640 1 -- 172.21.15.84:0/1892429308 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8e801f040 con 0x5610ffd02f70 2024-05-22T23:04:34.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.693+0000 7fc8d7fff640 1 -- 172.21.15.84:0/1892429308 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fc8f0036830 con 0x5610ffd02f70 2024-05-22T23:04:34.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.693+0000 7fc901e35740 10 client.15840 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:04:34.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.693+0000 7fc901e35740 10 client.15840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:04:34.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.693+0000 7fc901e35740 20 client.15840 populate_metadata read hostname 'smithi084' 2024-05-22T23:04:34.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.693+0000 7fc901e35740 1 --2- 172.21.15.84:0/1892429308 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5610ffd7a220 0x5610ffd7c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:34.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.693+0000 7fc901e35740 4 client.15840 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-22T23:04:34.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.693+0000 7fc901e35740 1 -- 172.21.15.84:0/1892429308 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610ffc64830 con 0x5610ffd7a220 2024-05-22T23:04:34.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.693+0000 7fc8fcdef640 1 --2- 172.21.15.84:0/1892429308 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5610ffd7a220 0x5610ffd7c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:34.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.693+0000 7fc8d7fff640 10 client.15840 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:34.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.693+0000 7fc8fcdef640 1 --2- 172.21.15.84:0/1892429308 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5610ffd7a220 0x5610ffd7c640 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8c29f89ffe88699 server_cookie=984c141af47c0b in_seq=0 out_seq=0 2024-05-22T23:04:34.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.695+0000 7fc8d7fff640 1 -- 172.21.15.84:0/1892429308 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610ffc64830 con 0x5610ffd7a220 2024-05-22T23:04:34.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.695+0000 7fc8d7fff640 10 client.15840 handle_command_reply: tid=0 2024-05-22T23:04:34.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.695+0000 7fc901e35740 1 client.15840 shutdown 2024-05-22T23:04:34.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.695+0000 7fc901e35740 1 -- 172.21.15.84:0/1892429308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffd02f70 msgr2=0x5610ffd075e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:34.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.695+0000 7fc901e35740 1 --2- 172.21.15.84:0/1892429308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffd02f70 0x5610ffd075e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0036550 tx=0x7fc8f002a9e0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.695+0000 7fc901e35740 1 -- 172.21.15.84:0/1892429308 shutdown_connections 2024-05-22T23:04:34.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.695+0000 7fc901e35740 1 --2- 172.21.15.84:0/1892429308 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5610ffd7a220 0x5610ffd7c640 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.695+0000 7fc901e35740 1 --2- 172.21.15.84:0/1892429308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5610ffd02f70 0x5610ffd075e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.695+0000 7fc901e35740 1 --2- 172.21.15.84:0/1892429308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610ffcf88a0 0x5610ffd070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.695+0000 7fc8d7fff640 0 client.15840 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:34.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.695+0000 7fc901e35740 1 --2- 172.21.15.84:0/1892429308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610ffcf7f80 0x5610ffd06b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.695+0000 7fc901e35740 1 -- 172.21.15.84:0/1892429308 >> 172.21.15.84:0/1892429308 conn(0x5610ffcf42a0 msgr2=0x5610ffd015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:34.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.695+0000 7fc901e35740 1 -- 172.21.15.84:0/1892429308 shutdown_connections 2024-05-22T23:04:34.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.695+0000 7fc901e35740 1 -- 172.21.15.84:0/1892429308 wait complete. 2024-05-22T23:04:34.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.695+0000 7fc901e35740 20 client.15840 trim_cache size 0 max 16384 2024-05-22T23:04:34.704 INFO:teuthology.orchestra.run.smithi084.stdout:[] 2024-05-22T23:04:34.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.695+0000 7fc8ff053640 1 -- 172.21.15.84:0/4174028274 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fc8e00a2d00 msgr2=0x7fc8e00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:34.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.695+0000 7fc8ff053640 1 --2- 172.21.15.84:0/4174028274 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fc8e00a2d00 0x7fc8e00a51c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc8f002f040 tx=0x7fc8f003a040 comp rx=0 tx=0).stop 2024-05-22T23:04:34.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.695+0000 7fc8ff053640 1 -- 172.21.15.84:0/4174028274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8f8102af0 msgr2=0x7fc8f8198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:34.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.695+0000 7fc8ff053640 1 --2- 172.21.15.84:0/4174028274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8f8102af0 0x7fc8f8198130 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400e9f0 tx=0x7fc8f400eeb0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.696+0000 7fc8ff053640 1 -- 172.21.15.84:0/4174028274 shutdown_connections 2024-05-22T23:04:34.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.696+0000 7fc8ff053640 1 --2- 172.21.15.84:0/4174028274 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fc8e00a2d00 0x7fc8e00a51c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.696+0000 7fc8ff053640 1 --2- 172.21.15.84:0/4174028274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8f810be30 0x7fc8f819c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.696+0000 7fc8ff053640 1 --2- 172.21.15.84:0/4174028274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8f8103410 0x7fc8f8198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.696+0000 7fc8ff053640 1 --2- 172.21.15.84:0/4174028274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8f8102af0 0x7fc8f8198130 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:34.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.696+0000 7fc8ff053640 1 -- 172.21.15.84:0/4174028274 >> 172.21.15.84:0/4174028274 conn(0x7fc8f8077870 msgr2=0x7fc8f80fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:34.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.696+0000 7fc8ff053640 1 -- 172.21.15.84:0/4174028274 shutdown_connections 2024-05-22T23:04:34.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:34.696+0000 7fc8ff053640 1 -- 172.21.15.84:0/4174028274 wait complete. 2024-05-22T23:04:35.427 INFO:tasks.workunit.client.0.smithi084.stdout: 1 241787 16.31 MB/sec warmup 81 sec latency 9.317 ms 2024-05-22T23:04:35.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:35 smithi084 ceph-mon[32186]: pgmap v85: 97 pgs: 97 active+clean; 123 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 250 op/s 2024-05-22T23:04:35.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:35 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/855648614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:04:35.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:35 smithi204 ceph-mon[36211]: pgmap v85: 97 pgs: 97 active+clean; 123 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 250 op/s 2024-05-22T23:04:35.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:35 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/855648614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:04:35.711 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-22T23:04:35.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:04:35.712 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-22T23:04:35.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:35 smithi096 ceph-mon[36392]: pgmap v85: 97 pgs: 97 active+clean; 123 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 250 op/s 2024-05-22T23:04:35.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:35 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/855648614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:04:35.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.818+0000 7fb43db1d640 1 -- 172.21.15.84:0/3756338086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb438107860 msgr2=0x7fb438107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:35.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.818+0000 7fb43db1d640 1 --2- 172.21.15.84:0/3756338086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb438107860 0x7fb438107c40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb428009930 tx=0x7fb42802a420 comp rx=0 tx=0).stop 2024-05-22T23:04:35.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.818+0000 7fb43db1d640 1 -- 172.21.15.84:0/3756338086 shutdown_connections 2024-05-22T23:04:35.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.818+0000 7fb43db1d640 1 --2- 172.21.15.84:0/3756338086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb43806a8b0 0x7fb43810e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.818+0000 7fb43db1d640 1 --2- 172.21.15.84:0/3756338086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb438069ef0 0x7fb43806a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.818+0000 7fb43db1d640 1 --2- 172.21.15.84:0/3756338086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb438107860 0x7fb438107c40 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.818+0000 7fb43db1d640 1 -- 172.21.15.84:0/3756338086 >> 172.21.15.84:0/3756338086 conn(0x7fb4380777d0 msgr2=0x7fb438077be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:35.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.818+0000 7fb43db1d640 1 -- 172.21.15.84:0/3756338086 shutdown_connections 2024-05-22T23:04:35.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.818+0000 7fb43db1d640 1 -- 172.21.15.84:0/3756338086 wait complete. 2024-05-22T23:04:35.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.818+0000 7fb43db1d640 1 Processor -- start 2024-05-22T23:04:35.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.818+0000 7fb43db1d640 1 -- start start 2024-05-22T23:04:35.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.819+0000 7fb43db1d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb438069ef0 0x7fb4381981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.819+0000 7fb43db1d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb43806a8b0 0x7fb438198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.819+0000 7fb43db1d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb438107860 0x7fb43819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.819+0000 7fb43db1d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb438111c70 con 0x7fb438069ef0 2024-05-22T23:04:35.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.819+0000 7fb43db1d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb438111af0 con 0x7fb43806a8b0 2024-05-22T23:04:35.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.819+0000 7fb43db1d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb438111df0 con 0x7fb438107860 2024-05-22T23:04:35.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.819+0000 7fb4377fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb438069ef0 0x7fb4381981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.819+0000 7fb4377fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb438069ef0 0x7fb4381981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36858/0 (socket says 172.21.15.84:36858) 2024-05-22T23:04:35.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.819+0000 7fb4377fe640 1 -- 172.21.15.84:0/1408361181 learned_addr learned my addr 172.21.15.84:0/1408361181 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:35.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.819+0000 7fb436ffd640 1 --2- 172.21.15.84:0/1408361181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb43806a8b0 0x7fb438198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.819+0000 7fb437fff640 1 --2- 172.21.15.84:0/1408361181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb438107860 0x7fb43819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.819+0000 7fb4377fe640 1 -- 172.21.15.84:0/1408361181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb438107860 msgr2=0x7fb43819ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:35.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.819+0000 7fb4377fe640 1 --2- 172.21.15.84:0/1408361181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb438107860 0x7fb43819ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.819+0000 7fb4377fe640 1 -- 172.21.15.84:0/1408361181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb43806a8b0 msgr2=0x7fb438198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:35.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.819+0000 7fb4377fe640 1 --2- 172.21.15.84:0/1408361181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb43806a8b0 0x7fb438198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.819+0000 7fb4377fe640 1 -- 172.21.15.84:0/1408361181 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43819d1c0 con 0x7fb438069ef0 2024-05-22T23:04:35.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.819+0000 7fb436ffd640 1 --2- 172.21.15.84:0/1408361181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb43806a8b0 0x7fb438198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:35.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.819+0000 7fb4377fe640 1 --2- 172.21.15.84:0/1408361181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb438069ef0 0x7fb4381981d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb428009230 tx=0x7fb428004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:35.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.819+0000 7fb434ff9640 1 -- 172.21.15.84:0/1408361181 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42803f030 con 0x7fb438069ef0 2024-05-22T23:04:35.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.819+0000 7fb43db1d640 1 -- 172.21.15.84:0/1408361181 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb43819d450 con 0x7fb438069ef0 2024-05-22T23:04:35.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.820+0000 7fb434ff9640 1 -- 172.21.15.84:0/1408361181 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb428003ad0 con 0x7fb438069ef0 2024-05-22T23:04:35.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.820+0000 7fb434ff9640 1 -- 172.21.15.84:0/1408361181 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428035070 con 0x7fb438069ef0 2024-05-22T23:04:35.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.820+0000 7fb43db1d640 1 -- 172.21.15.84:0/1408361181 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4381a4d40 con 0x7fb438069ef0 2024-05-22T23:04:35.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.822+0000 7fb434ff9640 1 -- 172.21.15.84:0/1408361181 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb428042030 con 0x7fb438069ef0 2024-05-22T23:04:35.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.822+0000 7fb440820740 1 Processor -- start 2024-05-22T23:04:35.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.822+0000 7fb440820740 1 -- start start 2024-05-22T23:04:35.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.822+0000 7fb440820740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903ec50 0x55674903f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.822+0000 7fb440820740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55674903f570 0x556749049700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.822+0000 7fb440820740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 0x55674904c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.822+0000 7fb440820740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556748fa2a40 con 0x556749049c40 2024-05-22T23:04:35.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.822+0000 7fb440820740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556748f41440 con 0x55674903ec50 2024-05-22T23:04:35.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.822+0000 7fb440820740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556748f9ea00 con 0x55674903f570 2024-05-22T23:04:35.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.823+0000 7fb437fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903ec50 0x55674903f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.823+0000 7fb436ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 0x55674904c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.823+0000 7fb434ff9640 1 --2- 172.21.15.84:0/1408361181 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fb4200a2d00 0x7fb4200a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.823+0000 7fb437fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903ec50 0x55674903f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:43464/0 (socket says 172.21.15.84:43464) 2024-05-22T23:04:35.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.823+0000 7fb437fff640 1 -- 172.21.15.84:0/1040448113 learned_addr learned my addr 172.21.15.84:0/1040448113 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:35.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.823+0000 7fb4377fe640 1 --2- 172.21.15.84:0/1040448113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55674903f570 0x556749049700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.823+0000 7fb437fff640 1 --2- 172.21.15.84:0/1408361181 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fb4200a2d00 0x7fb4200a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.823+0000 7fb437fff640 1 -- 172.21.15.84:0/1040448113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55674903f570 msgr2=0x556749049700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:35.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.823+0000 7fb437fff640 1 --2- 172.21.15.84:0/1040448113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55674903f570 0x556749049700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.823+0000 7fb437fff640 1 -- 172.21.15.84:0/1040448113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 msgr2=0x55674904c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:35.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.823+0000 7fb437fff640 1 --2- 172.21.15.84:0/1040448113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 0x55674904c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.823+0000 7fb434ff9640 1 -- 172.21.15.84:0/1408361181 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fb4280ef9e0 con 0x7fb438069ef0 2024-05-22T23:04:35.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.823+0000 7fb437fff640 1 -- 172.21.15.84:0/1040448113 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556748f4b2d0 con 0x55674903ec50 2024-05-22T23:04:35.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.823+0000 7fb436ffd640 1 --2- 172.21.15.84:0/1040448113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 0x55674904c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:35.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.823+0000 7fb437fff640 1 --2- 172.21.15.84:0/1040448113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903ec50 0x55674903f030 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb430013880 tx=0x7fb430013d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53c1fc037a0a0006 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:35.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.823+0000 7fb437fff640 1 --2- 172.21.15.84:0/1408361181 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fb4200a2d00 0x7fb4200a51c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb430024000 tx=0x7fb430022000 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:35.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.823+0000 7fb41dffb640 1 -- 172.21.15.84:0/1040448113 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430023020 con 0x55674903ec50 2024-05-22T23:04:35.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.823+0000 7fb41dffb640 1 -- 172.21.15.84:0/1040448113 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb430009d20 con 0x55674903ec50 2024-05-22T23:04:35.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.823+0000 7fb41dffb640 1 -- 172.21.15.84:0/1040448113 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4300108d0 con 0x55674903ec50 2024-05-22T23:04:35.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.824+0000 7fb440820740 1 -- 172.21.15.84:0/1040448113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903ec50 msgr2=0x55674903f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:35.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.824+0000 7fb440820740 1 --2- 172.21.15.84:0/1040448113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903ec50 0x55674903f030 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb430013880 tx=0x7fb430013d40 comp rx=0 tx=0).stop 2024-05-22T23:04:35.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.824+0000 7fb440820740 1 -- 172.21.15.84:0/1040448113 shutdown_connections 2024-05-22T23:04:35.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.824+0000 7fb440820740 1 --2- 172.21.15.84:0/1040448113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 0x55674904c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.824+0000 7fb440820740 1 --2- 172.21.15.84:0/1040448113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55674903f570 0x556749049700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.824+0000 7fb440820740 1 --2- 172.21.15.84:0/1040448113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903ec50 0x55674903f030 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.824+0000 7fb440820740 1 -- 172.21.15.84:0/1040448113 >> 172.21.15.84:0/1040448113 conn(0x55674903af70 msgr2=0x55674903b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:35.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.824+0000 7fb440820740 1 -- 172.21.15.84:0/1040448113 shutdown_connections 2024-05-22T23:04:35.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.824+0000 7fb440820740 1 -- 172.21.15.84:0/1040448113 wait complete. 2024-05-22T23:04:35.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.824+0000 7fb440820740 1 -- start start 2024-05-22T23:04:35.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.825+0000 7fb440820740 1 Processor -- start 2024-05-22T23:04:35.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.825+0000 7fb440820740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55674903ec50 0x556749046bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.825+0000 7fb440820740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903f570 0x556749047130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.825+0000 7fb440820740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 0x556749047670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.825+0000 7fb440820740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55674904ca30 con 0x556749049c40 2024-05-22T23:04:35.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.825+0000 7fb440820740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556748f2c5f0 con 0x55674903f570 2024-05-22T23:04:35.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.825+0000 7fb440820740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55674904cbb0 con 0x55674903ec50 2024-05-22T23:04:35.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.825+0000 7fb440820740 10 client.0 _ll_register_callbacks cb 0x556748f27cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:04:35.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.825+0000 7fb437fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 0x556749047670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.825+0000 7fb437fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 0x556749047670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36868/0 (socket says 172.21.15.84:36868) 2024-05-22T23:04:35.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.825+0000 7fb436ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903f570 0x556749047130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.825+0000 7fb437fff640 1 -- 172.21.15.84:0/1873792729 learned_addr learned my addr 172.21.15.84:0/1873792729 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:35.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.825+0000 7fb4377fe640 1 --2- 172.21.15.84:0/1873792729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55674903ec50 0x556749046bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.825+0000 7fb437fff640 1 -- 172.21.15.84:0/1873792729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55674903ec50 msgr2=0x556749046bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:35.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.825+0000 7fb437fff640 1 --2- 172.21.15.84:0/1873792729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55674903ec50 0x556749046bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.825+0000 7fb437fff640 1 -- 172.21.15.84:0/1873792729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903f570 msgr2=0x556749047130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:35.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.825+0000 7fb437fff640 1 --2- 172.21.15.84:0/1873792729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903f570 0x556749047130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.825+0000 7fb437fff640 1 -- 172.21.15.84:0/1873792729 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556748d7e7e0 con 0x556749049c40 2024-05-22T23:04:35.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.825+0000 7fb436ffd640 1 --2- 172.21.15.84:0/1873792729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903f570 0x556749047130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:35.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.825+0000 7fb437fff640 1 --2- 172.21.15.84:0/1873792729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 0x556749047670 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb430010470 tx=0x7fb430013460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a17eaada81de4f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:35.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.825+0000 7fb416ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:04:35.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.825+0000 7fb416ffd640 1 -- 172.21.15.84:0/1873792729 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430023030 con 0x556749049c40 2024-05-22T23:04:35.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.825+0000 7fb416ffd640 1 -- 172.21.15.84:0/1873792729 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fb430005050 con 0x556749049c40 2024-05-22T23:04:35.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.825+0000 7fb440820740 1 -- 172.21.15.84:0/1873792729 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556748fb6460 con 0x556749049c40 2024-05-22T23:04:35.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.826+0000 7fb416ffd640 1 -- 172.21.15.84:0/1873792729 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fb43001fc60 con 0x556749049c40 2024-05-22T23:04:35.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.826+0000 7fb440820740 10 client.15852 fetch_fsmap learned FSMap version 18 2024-05-22T23:04:35.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.826+0000 7fb440820740 1 -- 172.21.15.84:0/1873792729 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x556748ddef70 con 0x556749049c40 2024-05-22T23:04:35.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.826+0000 7fb416ffd640 1 -- 172.21.15.84:0/1873792729 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556748fb6460 con 0x556749049c40 2024-05-22T23:04:35.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.826+0000 7fb416ffd640 1 -- 172.21.15.84:0/1873792729 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fb43002f070 con 0x556749049c40 2024-05-22T23:04:35.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.826+0000 7fb440820740 10 client.15852 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:04:35.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.826+0000 7fb440820740 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:04:35.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.826+0000 7fb440820740 20 client.15852 populate_metadata read hostname 'smithi084' 2024-05-22T23:04:35.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.826+0000 7fb440820740 1 --2- 172.21.15.84:0/1873792729 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556749059ea0 0x55674905c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.826+0000 7fb440820740 4 client.15852 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:04:35.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.826+0000 7fb440820740 1 -- 172.21.15.84:0/1873792729 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556748fb0170 con 0x556749059ea0 2024-05-22T23:04:35.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.826+0000 7fb4377fe640 1 --2- 172.21.15.84:0/1873792729 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556749059ea0 0x55674905c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.826+0000 7fb416ffd640 10 client.15852 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:35.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.826+0000 7fb4377fe640 1 --2- 172.21.15.84:0/1873792729 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556749059ea0 0x55674905c290 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32f2edfb8a0134c2 server_cookie=e750d6f33b603aa6 in_seq=0 out_seq=0 2024-05-22T23:04:35.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.827+0000 7fb416ffd640 1 -- 172.21.15.84:0/1873792729 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556748fb0170 con 0x556749059ea0 2024-05-22T23:04:35.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.827+0000 7fb416ffd640 10 client.15852 handle_command_reply: tid=0 2024-05-22T23:04:35.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.827+0000 7fb440820740 1 client.15852 shutdown 2024-05-22T23:04:35.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.828+0000 7fb440820740 1 -- 172.21.15.84:0/1873792729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 msgr2=0x556749047670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:35.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.828+0000 7fb440820740 1 --2- 172.21.15.84:0/1873792729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 0x556749047670 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb430010470 tx=0x7fb430013460 comp rx=0 tx=0).stop 2024-05-22T23:04:35.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.828+0000 7fb440820740 1 -- 172.21.15.84:0/1873792729 shutdown_connections 2024-05-22T23:04:35.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.828+0000 7fb440820740 1 --2- 172.21.15.84:0/1873792729 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556749059ea0 0x55674905c290 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.828+0000 7fb440820740 1 --2- 172.21.15.84:0/1873792729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 0x556749047670 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.828+0000 7fb440820740 1 --2- 172.21.15.84:0/1873792729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903f570 0x556749047130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.828+0000 7fb416ffd640 0 client.15852 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:35.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.828+0000 7fb440820740 1 --2- 172.21.15.84:0/1873792729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55674903ec50 0x556749046bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.828+0000 7fb440820740 1 -- 172.21.15.84:0/1873792729 >> 172.21.15.84:0/1873792729 conn(0x55674903af70 msgr2=0x556749040d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:35.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.828+0000 7fb440820740 1 -- 172.21.15.84:0/1873792729 shutdown_connections 2024-05-22T23:04:35.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.828+0000 7fb440820740 1 -- 172.21.15.84:0/1873792729 wait complete. 2024-05-22T23:04:35.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.828+0000 7fb440820740 20 client.15852 trim_cache size 0 max 16384 2024-05-22T23:04:35.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.846+0000 7fb440820740 1 Processor -- start 2024-05-22T23:04:35.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.846+0000 7fb440820740 1 -- start start 2024-05-22T23:04:35.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.847+0000 7fb440820740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55674903ec50 0x556749047490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.847+0000 7fb440820740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903f570 0x556749061b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.847+0000 7fb440820740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 0x556749062040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.847+0000 7fb440820740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381ab600 con 0x556749049c40 2024-05-22T23:04:35.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.847+0000 7fb440820740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381ade80 con 0x55674903f570 2024-05-22T23:04:35.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.847+0000 7fb440820740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381abc70 con 0x55674903ec50 2024-05-22T23:04:35.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.847+0000 7fb437fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 0x556749062040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.847+0000 7fb437fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 0x556749062040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36876/0 (socket says 172.21.15.84:36876) 2024-05-22T23:04:35.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.847+0000 7fb437fff640 1 -- 172.21.15.84:0/2457324888 learned_addr learned my addr 172.21.15.84:0/2457324888 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:35.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.847+0000 7fb436ffd640 1 --2- 172.21.15.84:0/2457324888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903f570 0x556749061b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.847+0000 7fb437fff640 1 -- 172.21.15.84:0/2457324888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55674903ec50 msgr2=0x556749047490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:35.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.847+0000 7fb437fff640 1 --2- 172.21.15.84:0/2457324888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55674903ec50 0x556749047490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.847+0000 7fb437fff640 1 -- 172.21.15.84:0/2457324888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903f570 msgr2=0x556749061b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:35.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.847+0000 7fb437fff640 1 --2- 172.21.15.84:0/2457324888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903f570 0x556749061b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.847+0000 7fb437fff640 1 -- 172.21.15.84:0/2457324888 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556749041a10 con 0x556749049c40 2024-05-22T23:04:35.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.847+0000 7fb436ffd640 1 --2- 172.21.15.84:0/2457324888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903f570 0x556749061b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:04:35.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.847+0000 7fb437fff640 1 --2- 172.21.15.84:0/2457324888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 0x556749062040 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb430012390 tx=0x7fb4300123c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f107cc8ca9f3e35 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:35.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.847+0000 7fb41d7fa640 1 -- 172.21.15.84:0/2457324888 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43001fc60 con 0x556749049c40 2024-05-22T23:04:35.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.847+0000 7fb41d7fa640 1 -- 172.21.15.84:0/2457324888 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb430010ce0 con 0x556749049c40 2024-05-22T23:04:35.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.847+0000 7fb41d7fa640 1 -- 172.21.15.84:0/2457324888 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4300035f0 con 0x556749049c40 2024-05-22T23:04:35.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.848+0000 7fb440820740 1 -- 172.21.15.84:0/2457324888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 msgr2=0x556749062040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:35.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.848+0000 7fb440820740 1 --2- 172.21.15.84:0/2457324888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 0x556749062040 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb430012390 tx=0x7fb4300123c0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.848+0000 7fb440820740 1 -- 172.21.15.84:0/2457324888 shutdown_connections 2024-05-22T23:04:35.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.848+0000 7fb440820740 1 --2- 172.21.15.84:0/2457324888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 0x556749062040 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.848+0000 7fb440820740 1 --2- 172.21.15.84:0/2457324888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903f570 0x556749061b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.848+0000 7fb440820740 1 --2- 172.21.15.84:0/2457324888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55674903ec50 0x556749047490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.848+0000 7fb440820740 1 -- 172.21.15.84:0/2457324888 >> 172.21.15.84:0/2457324888 conn(0x55674903af70 msgr2=0x5567490403e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:35.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.848+0000 7fb440820740 1 -- 172.21.15.84:0/2457324888 shutdown_connections 2024-05-22T23:04:35.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.848+0000 7fb440820740 1 -- 172.21.15.84:0/2457324888 wait complete. 2024-05-22T23:04:35.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.848+0000 7fb440820740 1 -- start start 2024-05-22T23:04:35.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.849+0000 7fb440820740 1 Processor -- start 2024-05-22T23:04:35.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.849+0000 7fb440820740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903ec50 0x5567490438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.849+0000 7fb440820740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55674903f570 0x556749043e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.849+0000 7fb440820740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 0x556749044360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.849+0000 7fb440820740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381ac340 con 0x556749049c40 2024-05-22T23:04:35.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.849+0000 7fb440820740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381ad7b0 con 0x55674903ec50 2024-05-22T23:04:35.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.849+0000 7fb440820740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381aca10 con 0x55674903f570 2024-05-22T23:04:35.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.849+0000 7fb440820740 10 client.0 _ll_register_callbacks cb 0x7fb42c00ad20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:04:35.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.849+0000 7fb437fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 0x556749044360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.849+0000 7fb4377fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903ec50 0x5567490438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.849+0000 7fb4377fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903ec50 0x5567490438e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:43496/0 (socket says 172.21.15.84:43496) 2024-05-22T23:04:35.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.849+0000 7fb4377fe640 1 -- 172.21.15.84:0/2015185061 learned_addr learned my addr 172.21.15.84:0/2015185061 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:35.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.849+0000 7fb436ffd640 1 --2- 172.21.15.84:0/2015185061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55674903f570 0x556749043e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.849+0000 7fb437fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 0x556749044360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36888/0 (socket says 172.21.15.84:36888) 2024-05-22T23:04:35.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.849+0000 7fb437fff640 1 -- 172.21.15.84:0/2015185061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55674903f570 msgr2=0x556749043e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:35.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.849+0000 7fb437fff640 1 --2- 172.21.15.84:0/2015185061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55674903f570 0x556749043e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.849+0000 7fb437fff640 1 -- 172.21.15.84:0/2015185061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903ec50 msgr2=0x5567490438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:35.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.849+0000 7fb437fff640 1 --2- 172.21.15.84:0/2015185061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903ec50 0x5567490438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.849+0000 7fb437fff640 1 -- 172.21.15.84:0/2015185061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556748c76170 con 0x556749049c40 2024-05-22T23:04:35.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.849+0000 7fb4377fe640 1 --2- 172.21.15.84:0/2015185061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903ec50 0x5567490438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:35.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.849+0000 7fb437fff640 1 --2- 172.21.15.84:0/2015185061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 0x556749044360 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb43000fed0 tx=0x7fb4300099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892a7e38e4e1ffb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:35.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.849+0000 7fb416ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:04:35.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.849+0000 7fb416ffd640 1 -- 172.21.15.84:0/2015185061 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4300109b0 con 0x556749049c40 2024-05-22T23:04:35.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.850+0000 7fb416ffd640 1 -- 172.21.15.84:0/2015185061 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fb4300035f0 con 0x556749049c40 2024-05-22T23:04:35.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.850+0000 7fb440820740 1 -- 172.21.15.84:0/2015185061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb430010ce0 con 0x556749049c40 2024-05-22T23:04:35.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.850+0000 7fb416ffd640 1 -- 172.21.15.84:0/2015185061 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fb43000f790 con 0x556749049c40 2024-05-22T23:04:35.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.850+0000 7fb440820740 10 client.15858 fetch_fsmap learned FSMap version 18 2024-05-22T23:04:35.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.850+0000 7fb440820740 1 -- 172.21.15.84:0/2015185061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x556748f6b930 con 0x556749049c40 2024-05-22T23:04:35.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.850+0000 7fb416ffd640 1 -- 172.21.15.84:0/2015185061 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb430010ce0 con 0x556749049c40 2024-05-22T23:04:35.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.850+0000 7fb416ffd640 1 -- 172.21.15.84:0/2015185061 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fb43002f040 con 0x556749049c40 2024-05-22T23:04:35.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.850+0000 7fb440820740 10 client.15858 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:04:35.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.850+0000 7fb440820740 10 client.15858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:04:35.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.850+0000 7fb440820740 20 client.15858 populate_metadata read hostname 'smithi084' 2024-05-22T23:04:35.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.850+0000 7fb440820740 1 --2- 172.21.15.84:0/2015185061 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5567490c08e0 0x5567490c2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.850+0000 7fb440820740 4 client.15858 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-22T23:04:35.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.850+0000 7fb440820740 1 -- 172.21.15.84:0/2015185061 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556748faa4c0 con 0x5567490c08e0 2024-05-22T23:04:35.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.850+0000 7fb4377fe640 1 --2- 172.21.15.84:0/2015185061 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5567490c08e0 0x5567490c2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.851+0000 7fb416ffd640 10 client.15858 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:35.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.851+0000 7fb4377fe640 1 --2- 172.21.15.84:0/2015185061 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5567490c08e0 0x5567490c2cd0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2985d0ca7a86a7ed server_cookie=ee613dc92abd7dc1 in_seq=0 out_seq=0 2024-05-22T23:04:35.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.852+0000 7fb416ffd640 1 -- 172.21.15.84:0/2015185061 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556748faa4c0 con 0x5567490c08e0 2024-05-22T23:04:35.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.852+0000 7fb416ffd640 10 client.15858 handle_command_reply: tid=0 2024-05-22T23:04:35.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.852+0000 7fb440820740 1 client.15858 shutdown 2024-05-22T23:04:35.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.852+0000 7fb440820740 1 -- 172.21.15.84:0/2015185061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 msgr2=0x556749044360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:35.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.852+0000 7fb440820740 1 --2- 172.21.15.84:0/2015185061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 0x556749044360 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb43000fed0 tx=0x7fb4300099f0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.852+0000 7fb440820740 1 -- 172.21.15.84:0/2015185061 shutdown_connections 2024-05-22T23:04:35.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.852+0000 7fb440820740 1 --2- 172.21.15.84:0/2015185061 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5567490c08e0 0x5567490c2cd0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.852+0000 7fb440820740 1 --2- 172.21.15.84:0/2015185061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556749049c40 0x556749044360 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.852+0000 7fb440820740 1 --2- 172.21.15.84:0/2015185061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55674903f570 0x556749043e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.852+0000 7fb440820740 1 --2- 172.21.15.84:0/2015185061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55674903ec50 0x5567490438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.852+0000 7fb416ffd640 0 client.15858 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:35.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.852+0000 7fb440820740 1 -- 172.21.15.84:0/2015185061 >> 172.21.15.84:0/2015185061 conn(0x55674903af70 msgr2=0x556749040c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:35.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.852+0000 7fb440820740 1 -- 172.21.15.84:0/2015185061 shutdown_connections 2024-05-22T23:04:35.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.852+0000 7fb440820740 1 -- 172.21.15.84:0/2015185061 wait complete. 2024-05-22T23:04:35.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.852+0000 7fb440820740 20 client.15858 trim_cache size 0 max 16384 2024-05-22T23:04:35.863 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:04:35.863 INFO:teuthology.orchestra.run.smithi084.stdout: "return_code": 0, 2024-05-22T23:04:35.863 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_tag": "a06af2d0-11d2-4219-83c7-cd3ddd0f099d", 2024-05-22T23:04:35.863 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": "asynchronous" 2024-05-22T23:04:35.863 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:04:35.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.853+0000 7fb43db1d640 1 -- 172.21.15.84:0/1408361181 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fb4200a2d00 msgr2=0x7fb4200a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:35.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.853+0000 7fb43db1d640 1 --2- 172.21.15.84:0/1408361181 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fb4200a2d00 0x7fb4200a51c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb430024000 tx=0x7fb430022000 comp rx=0 tx=0).stop 2024-05-22T23:04:35.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.853+0000 7fb43db1d640 1 -- 172.21.15.84:0/1408361181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb438069ef0 msgr2=0x7fb4381981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:35.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.853+0000 7fb43db1d640 1 --2- 172.21.15.84:0/1408361181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb438069ef0 0x7fb4381981d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb428009230 tx=0x7fb428004270 comp rx=0 tx=0).stop 2024-05-22T23:04:35.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.853+0000 7fb43db1d640 1 -- 172.21.15.84:0/1408361181 shutdown_connections 2024-05-22T23:04:35.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.853+0000 7fb43db1d640 1 --2- 172.21.15.84:0/1408361181 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fb4200a2d00 0x7fb4200a51c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.853+0000 7fb43db1d640 1 --2- 172.21.15.84:0/1408361181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb438107860 0x7fb43819ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.853+0000 7fb43db1d640 1 --2- 172.21.15.84:0/1408361181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb43806a8b0 0x7fb438198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.853+0000 7fb43db1d640 1 --2- 172.21.15.84:0/1408361181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb438069ef0 0x7fb4381981d0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.853+0000 7fb43db1d640 1 -- 172.21.15.84:0/1408361181 >> 172.21.15.84:0/1408361181 conn(0x7fb4380777d0 msgr2=0x7fb4380fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:35.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.853+0000 7fb43db1d640 1 -- 172.21.15.84:0/1408361181 shutdown_connections 2024-05-22T23:04:35.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.853+0000 7fb43db1d640 1 -- 172.21.15.84:0/1408361181 wait complete. 2024-05-22T23:04:35.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:04:35.868 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:04:35.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.974+0000 7fcae091b640 1 -- 172.21.15.84:0/3302867014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcadc106750 msgr2=0x7fcadc106b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:35.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.974+0000 7fcae091b640 1 --2- 172.21.15.84:0/3302867014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcadc106750 0x7fcadc106b30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009cd0 tx=0x7fcacc02a8e0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.975+0000 7fcae091b640 1 -- 172.21.15.84:0/3302867014 shutdown_connections 2024-05-22T23:04:35.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.975+0000 7fcae091b640 1 --2- 172.21.15.84:0/3302867014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcadc101100 0x7fcadc10d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.975+0000 7fcae091b640 1 --2- 172.21.15.84:0/3302867014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcadc100740 0x7fcadc100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.975+0000 7fcae091b640 1 --2- 172.21.15.84:0/3302867014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcadc106750 0x7fcadc106b30 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.975+0000 7fcae091b640 1 -- 172.21.15.84:0/3302867014 >> 172.21.15.84:0/3302867014 conn(0x7fcadc077870 msgr2=0x7fcadc077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:35.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.975+0000 7fcae091b640 1 -- 172.21.15.84:0/3302867014 shutdown_connections 2024-05-22T23:04:35.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.975+0000 7fcae091b640 1 -- 172.21.15.84:0/3302867014 wait complete. 2024-05-22T23:04:35.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.975+0000 7fcae091b640 1 Processor -- start 2024-05-22T23:04:35.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.975+0000 7fcae091b640 1 -- start start 2024-05-22T23:04:35.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.976+0000 7fcae091b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcadc100740 0x7fcadc198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.976+0000 7fcae091b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcadc101100 0x7fcadc198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.976+0000 7fcae091b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcadc106750 0x7fcadc19ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.976+0000 7fcae091b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc111c60 con 0x7fcadc101100 2024-05-22T23:04:35.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.976+0000 7fcae091b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc111ae0 con 0x7fcadc100740 2024-05-22T23:04:35.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.976+0000 7fcae091b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc111de0 con 0x7fcadc106750 2024-05-22T23:04:35.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.976+0000 7fcad9d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcadc101100 0x7fcadc198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.976+0000 7fcada59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcadc100740 0x7fcadc198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.976+0000 7fcad9d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcadc101100 0x7fcadc198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36914/0 (socket says 172.21.15.84:36914) 2024-05-22T23:04:35.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.976+0000 7fcad9d9b640 1 -- 172.21.15.84:0/1311403012 learned_addr learned my addr 172.21.15.84:0/1311403012 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:35.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.976+0000 7fcadad9d640 1 --2- 172.21.15.84:0/1311403012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcadc106750 0x7fcadc19ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.976+0000 7fcad9d9b640 1 -- 172.21.15.84:0/1311403012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcadc106750 msgr2=0x7fcadc19ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:35.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.976+0000 7fcad9d9b640 1 --2- 172.21.15.84:0/1311403012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcadc106750 0x7fcadc19ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.976+0000 7fcad9d9b640 1 -- 172.21.15.84:0/1311403012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcadc100740 msgr2=0x7fcadc198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:35.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.976+0000 7fcad9d9b640 1 --2- 172.21.15.84:0/1311403012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcadc100740 0x7fcadc198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.976+0000 7fcad9d9b640 1 -- 172.21.15.84:0/1311403012 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcadc19d130 con 0x7fcadc101100 2024-05-22T23:04:35.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.976+0000 7fcada59c640 1 --2- 172.21.15.84:0/1311403012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcadc100740 0x7fcadc198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:35.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.976+0000 7fcad9d9b640 1 --2- 172.21.15.84:0/1311403012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcadc101100 0x7fcadc198680 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcad000e9f0 tx=0x7fcad000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:35.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.976+0000 7fcacb7fe640 1 -- 172.21.15.84:0/1311403012 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad000cd80 con 0x7fcadc101100 2024-05-22T23:04:35.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.976+0000 7fcae091b640 1 -- 172.21.15.84:0/1311403012 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcadc19d420 con 0x7fcadc101100 2024-05-22T23:04:35.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.976+0000 7fcacb7fe640 1 -- 172.21.15.84:0/1311403012 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcad0010910 con 0x7fcadc101100 2024-05-22T23:04:35.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.976+0000 7fcacb7fe640 1 -- 172.21.15.84:0/1311403012 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad00188f0 con 0x7fcadc101100 2024-05-22T23:04:35.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.976+0000 7fcae091b640 1 -- 172.21.15.84:0/1311403012 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcadc1a4cb0 con 0x7fcadc101100 2024-05-22T23:04:35.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.979+0000 7fcacb7fe640 1 -- 172.21.15.84:0/1311403012 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcad0010ac0 con 0x7fcadc101100 2024-05-22T23:04:35.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.979+0000 7fcae362c740 1 Processor -- start 2024-05-22T23:04:35.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.979+0000 7fcae362c740 1 -- start start 2024-05-22T23:04:35.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.979+0000 7fcae362c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bed8b0 0x555ad7bebbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.979+0000 7fcae362c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ad7bec1c0 0x555ad7bec640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.979+0000 7fcae362c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad7bf6f40 0x555ad7bf9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.979+0000 7fcae362c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555ad7b4bdb0 con 0x555ad7bed8b0 2024-05-22T23:04:35.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.979+0000 7fcae362c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555ad7aea360 con 0x555ad7bf6f40 2024-05-22T23:04:35.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.979+0000 7fcae362c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555ad7b47d70 con 0x555ad7bec1c0 2024-05-22T23:04:35.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.979+0000 7fcadad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bed8b0 0x555ad7bebbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.979+0000 7fcadad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bed8b0 0x555ad7bebbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36916/0 (socket says 172.21.15.84:36916) 2024-05-22T23:04:35.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.979+0000 7fcadad9d640 1 -- 172.21.15.84:0/1034695298 learned_addr learned my addr 172.21.15.84:0/1034695298 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:35.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.979+0000 7fcad9d9b640 1 --2- 172.21.15.84:0/1034695298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad7bf6f40 0x555ad7bf9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.979+0000 7fcada59c640 1 --2- 172.21.15.84:0/1034695298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ad7bec1c0 0x555ad7bec640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.980+0000 7fcadad9d640 1 -- 172.21.15.84:0/1034695298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ad7bec1c0 msgr2=0x555ad7bec640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:35.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.980+0000 7fcadad9d640 1 --2- 172.21.15.84:0/1034695298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ad7bec1c0 0x555ad7bec640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.980+0000 7fcacb7fe640 1 --2- 172.21.15.84:0/1311403012 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fcac40a2d00 0x7fcac40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.980+0000 7fcadad9d640 1 -- 172.21.15.84:0/1034695298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad7bf6f40 msgr2=0x555ad7bf9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:35.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.980+0000 7fcadad9d640 1 --2- 172.21.15.84:0/1034695298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad7bf6f40 0x555ad7bf9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.980+0000 7fcadad9d640 1 -- 172.21.15.84:0/1034695298 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ad7b0da60 con 0x555ad7bed8b0 2024-05-22T23:04:35.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.980+0000 7fcada59c640 1 --2- 172.21.15.84:0/1034695298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ad7bec1c0 0x555ad7bec640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:35.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.980+0000 7fcacb7fe640 1 -- 172.21.15.84:0/1311403012 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fcad00cf800 con 0x7fcadc101100 2024-05-22T23:04:35.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.980+0000 7fcadad9d640 1 --2- 172.21.15.84:0/1311403012 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fcac40a2d00 0x7fcac40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.980+0000 7fcadad9d640 1 --2- 172.21.15.84:0/1034695298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bed8b0 0x555ad7bebbf0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fcad4009c20 tx=0x7fcad4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b136145ee611436d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:35.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.980+0000 7fcac8ff9640 1 -- 172.21.15.84:0/1034695298 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad401b850 con 0x555ad7bed8b0 2024-05-22T23:04:35.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.980+0000 7fcac8ff9640 1 -- 172.21.15.84:0/1034695298 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcad4019db0 con 0x555ad7bed8b0 2024-05-22T23:04:35.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.980+0000 7fcac8ff9640 1 -- 172.21.15.84:0/1034695298 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad400eda0 con 0x555ad7bed8b0 2024-05-22T23:04:35.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.981+0000 7fcae362c740 1 -- 172.21.15.84:0/1034695298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bed8b0 msgr2=0x555ad7bebbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:35.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.981+0000 7fcadad9d640 1 --2- 172.21.15.84:0/1311403012 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fcac40a2d00 0x7fcac40a51c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcad4009fd0 tx=0x7fcad4021000 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:35.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.981+0000 7fcae362c740 1 --2- 172.21.15.84:0/1034695298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bed8b0 0x555ad7bebbf0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fcad4009c20 tx=0x7fcad4011810 comp rx=0 tx=0).stop 2024-05-22T23:04:35.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.981+0000 7fcae362c740 1 -- 172.21.15.84:0/1034695298 shutdown_connections 2024-05-22T23:04:35.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.981+0000 7fcae362c740 1 --2- 172.21.15.84:0/1034695298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad7bf6f40 0x555ad7bf9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.981+0000 7fcae362c740 1 --2- 172.21.15.84:0/1034695298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ad7bec1c0 0x555ad7bec640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.981+0000 7fcae362c740 1 --2- 172.21.15.84:0/1034695298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bed8b0 0x555ad7bebbf0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.981+0000 7fcae362c740 1 -- 172.21.15.84:0/1034695298 >> 172.21.15.84:0/1034695298 conn(0x555ad7bb08e0 msgr2=0x555ad7be8720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:35.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.981+0000 7fcae362c740 1 -- 172.21.15.84:0/1034695298 shutdown_connections 2024-05-22T23:04:35.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.981+0000 7fcae362c740 1 -- 172.21.15.84:0/1034695298 wait complete. 2024-05-22T23:04:35.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.981+0000 7fcae362c740 1 -- start start 2024-05-22T23:04:35.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.981+0000 7fcae362c740 1 Processor -- start 2024-05-22T23:04:35.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.982+0000 7fcae362c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bec1c0 0x555ad7bf3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.982+0000 7fcae362c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ad7bed8b0 0x555ad7bf4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.982+0000 7fcae362c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad7bf6f40 0x555ad7bf48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.982+0000 7fcae362c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555ad7bf9dd0 con 0x555ad7bec1c0 2024-05-22T23:04:35.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.982+0000 7fcae362c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555ad7b060f0 con 0x555ad7bf6f40 2024-05-22T23:04:35.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.982+0000 7fcae362c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555ad7bf9f50 con 0x555ad7bed8b0 2024-05-22T23:04:35.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.982+0000 7fcae362c740 10 client.0 _ll_register_callbacks cb 0x555ad7b0af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:04:35.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.982+0000 7fcada59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bec1c0 0x555ad7bf3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.982+0000 7fcada59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bec1c0 0x555ad7bf3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36928/0 (socket says 172.21.15.84:36928) 2024-05-22T23:04:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.982+0000 7fcada59c640 1 -- 172.21.15.84:0/2664918353 learned_addr learned my addr 172.21.15.84:0/2664918353 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.982+0000 7fcadad9d640 1 --2- 172.21.15.84:0/2664918353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad7bf6f40 0x555ad7bf48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.982+0000 7fcad9d9b640 1 --2- 172.21.15.84:0/2664918353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ad7bed8b0 0x555ad7bf4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.982+0000 7fcada59c640 1 -- 172.21.15.84:0/2664918353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ad7bed8b0 msgr2=0x555ad7bf4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.982+0000 7fcada59c640 1 --2- 172.21.15.84:0/2664918353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ad7bed8b0 0x555ad7bf4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.982+0000 7fcada59c640 1 -- 172.21.15.84:0/2664918353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad7bf6f40 msgr2=0x555ad7bf48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.982+0000 7fcada59c640 1 --2- 172.21.15.84:0/2664918353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad7bf6f40 0x555ad7bf48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.982+0000 7fcada59c640 1 -- 172.21.15.84:0/2664918353 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ad79297d0 con 0x555ad7bec1c0 2024-05-22T23:04:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.982+0000 7fcada59c640 1 --2- 172.21.15.84:0/2664918353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bec1c0 0x555ad7bf3e40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fcacc035000 tx=0x7fcacc005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8088d3b4abe4df10 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.982+0000 7fcac1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:04:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.982+0000 7fcac1ffb640 1 -- 172.21.15.84:0/2664918353 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc049060 con 0x555ad7bec1c0 2024-05-22T23:04:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.982+0000 7fcac1ffb640 1 -- 172.21.15.84:0/2664918353 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fcacc003cc0 con 0x555ad7bec1c0 2024-05-22T23:04:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.982+0000 7fcae362c740 1 -- 172.21.15.84:0/2664918353 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcad4019db0 con 0x555ad7bec1c0 2024-05-22T23:04:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.983+0000 7fcac1ffb640 1 -- 172.21.15.84:0/2664918353 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fcacc004040 con 0x555ad7bec1c0 2024-05-22T23:04:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.983+0000 7fcae362c740 10 client.15870 fetch_fsmap learned FSMap version 18 2024-05-22T23:04:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.983+0000 7fcae362c740 1 -- 172.21.15.84:0/2664918353 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x555ad785f3f0 con 0x555ad7bec1c0 2024-05-22T23:04:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.983+0000 7fcac1ffb640 1 -- 172.21.15.84:0/2664918353 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcad4019db0 con 0x555ad7bec1c0 2024-05-22T23:04:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.983+0000 7fcac1ffb640 1 -- 172.21.15.84:0/2664918353 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fcacc043070 con 0x555ad7bec1c0 2024-05-22T23:04:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.983+0000 7fcae362c740 10 client.15870 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:04:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.983+0000 7fcae362c740 10 client.15870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:04:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.983+0000 7fcae362c740 20 client.15870 populate_metadata read hostname 'smithi084' 2024-05-22T23:04:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.983+0000 7fcae362c740 1 --2- 172.21.15.84:0/2664918353 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x555ad7c073f0 0x555ad7c09860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.983+0000 7fcae362c740 4 client.15870 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:04:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.983+0000 7fcae362c740 1 -- 172.21.15.84:0/2664918353 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ad7b594e0 con 0x555ad7c073f0 2024-05-22T23:04:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.983+0000 7fcad9d9b640 1 --2- 172.21.15.84:0/2664918353 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x555ad7c073f0 0x555ad7c09860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.983+0000 7fcac1ffb640 10 client.15870 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:35.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.983+0000 7fcad9d9b640 1 --2- 172.21.15.84:0/2664918353 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x555ad7c073f0 0x555ad7c09860 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdbb6972df25b7c3 server_cookie=b8b0a14e459c96d4 in_seq=0 out_seq=0 2024-05-22T23:04:35.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.984+0000 7fcac1ffb640 1 -- 172.21.15.84:0/2664918353 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555ad7b594e0 con 0x555ad7c073f0 2024-05-22T23:04:35.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.984+0000 7fcac1ffb640 10 client.15870 handle_command_reply: tid=0 2024-05-22T23:04:35.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.985+0000 7fcae362c740 1 client.15870 shutdown 2024-05-22T23:04:35.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.985+0000 7fcae362c740 1 -- 172.21.15.84:0/2664918353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bec1c0 msgr2=0x555ad7bf3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:35.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.985+0000 7fcae362c740 1 --2- 172.21.15.84:0/2664918353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bec1c0 0x555ad7bf3e40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fcacc035000 tx=0x7fcacc005f70 comp rx=0 tx=0).stop 2024-05-22T23:04:35.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.985+0000 7fcae362c740 1 -- 172.21.15.84:0/2664918353 shutdown_connections 2024-05-22T23:04:35.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.985+0000 7fcae362c740 1 --2- 172.21.15.84:0/2664918353 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x555ad7c073f0 0x555ad7c09860 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.985+0000 7fcae362c740 1 --2- 172.21.15.84:0/2664918353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad7bf6f40 0x555ad7bf48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.985+0000 7fcae362c740 1 --2- 172.21.15.84:0/2664918353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ad7bed8b0 0x555ad7bf4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.985+0000 7fcac1ffb640 0 client.15870 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:35.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.985+0000 7fcae362c740 1 --2- 172.21.15.84:0/2664918353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bec1c0 0x555ad7bf3e40 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:35.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.985+0000 7fcae362c740 1 -- 172.21.15.84:0/2664918353 >> 172.21.15.84:0/2664918353 conn(0x555ad7bb08e0 msgr2=0x555ad7be3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:35.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.985+0000 7fcae362c740 1 -- 172.21.15.84:0/2664918353 shutdown_connections 2024-05-22T23:04:35.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.985+0000 7fcae362c740 1 -- 172.21.15.84:0/2664918353 wait complete. 2024-05-22T23:04:35.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:35.985+0000 7fcae362c740 20 client.15870 trim_cache size 0 max 16384 2024-05-22T23:04:36.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.003+0000 7fcae362c740 1 Processor -- start 2024-05-22T23:04:36.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.003+0000 7fcae362c740 1 -- start start 2024-05-22T23:04:36.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.003+0000 7fcae362c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ad7bec1c0 0x555ad7c02cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.003+0000 7fcae362c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bed8b0 0x555ad7bef190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.003+0000 7fcae362c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad7bf6f40 0x555ad7bf2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.003+0000 7fcae362c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc1a9340 con 0x555ad7bed8b0 2024-05-22T23:04:36.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.003+0000 7fcae362c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc1abbc0 con 0x555ad7bf6f40 2024-05-22T23:04:36.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.003+0000 7fcae362c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc1a99b0 con 0x555ad7bec1c0 2024-05-22T23:04:36.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.003+0000 7fcad9d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bed8b0 0x555ad7bef190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.003+0000 7fcad9d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bed8b0 0x555ad7bef190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36934/0 (socket says 172.21.15.84:36934) 2024-05-22T23:04:36.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.003+0000 7fcadad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad7bf6f40 0x555ad7bf2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.003+0000 7fcad9d9b640 1 -- 172.21.15.84:0/3119832124 learned_addr learned my addr 172.21.15.84:0/3119832124 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:36.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.003+0000 7fcada59c640 1 --2- 172.21.15.84:0/3119832124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ad7bec1c0 0x555ad7c02cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.004+0000 7fcad9d9b640 1 -- 172.21.15.84:0/3119832124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ad7bec1c0 msgr2=0x555ad7c02cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:36.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.004+0000 7fcad9d9b640 1 --2- 172.21.15.84:0/3119832124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ad7bec1c0 0x555ad7c02cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.004+0000 7fcad9d9b640 1 -- 172.21.15.84:0/3119832124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad7bf6f40 msgr2=0x555ad7bf2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:36.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.004+0000 7fcad9d9b640 1 --2- 172.21.15.84:0/3119832124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad7bf6f40 0x555ad7bf2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.004+0000 7fcad9d9b640 1 -- 172.21.15.84:0/3119832124 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ad781f0f0 con 0x555ad7bed8b0 2024-05-22T23:04:36.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.004+0000 7fcadad9d640 1 --2- 172.21.15.84:0/3119832124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad7bf6f40 0x555ad7bf2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:36.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.004+0000 7fcad9d9b640 1 --2- 172.21.15.84:0/3119832124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bed8b0 0x555ad7bef190 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fcad00cd550 tx=0x7fcad00d1d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d816993986f42543 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:36.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.004+0000 7fcac3fff640 1 -- 172.21.15.84:0/3119832124 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad00d2e60 con 0x555ad7bed8b0 2024-05-22T23:04:36.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.004+0000 7fcac3fff640 1 -- 172.21.15.84:0/3119832124 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcad009bdd0 con 0x555ad7bed8b0 2024-05-22T23:04:36.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.004+0000 7fcac3fff640 1 -- 172.21.15.84:0/3119832124 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad009b220 con 0x555ad7bed8b0 2024-05-22T23:04:36.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.004+0000 7fcae362c740 1 -- 172.21.15.84:0/3119832124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bed8b0 msgr2=0x555ad7bef190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:36.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.004+0000 7fcae362c740 1 --2- 172.21.15.84:0/3119832124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bed8b0 0x555ad7bef190 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fcad00cd550 tx=0x7fcad00d1d80 comp rx=0 tx=0).stop 2024-05-22T23:04:36.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.005+0000 7fcae362c740 1 -- 172.21.15.84:0/3119832124 shutdown_connections 2024-05-22T23:04:36.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.005+0000 7fcae362c740 1 --2- 172.21.15.84:0/3119832124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad7bf6f40 0x555ad7bf2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.005+0000 7fcae362c740 1 --2- 172.21.15.84:0/3119832124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bed8b0 0x555ad7bef190 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.005+0000 7fcae362c740 1 --2- 172.21.15.84:0/3119832124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ad7bec1c0 0x555ad7c02cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.005+0000 7fcae362c740 1 -- 172.21.15.84:0/3119832124 >> 172.21.15.84:0/3119832124 conn(0x555ad7bb08e0 msgr2=0x555ad7bf5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:36.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.005+0000 7fcae362c740 1 -- 172.21.15.84:0/3119832124 shutdown_connections 2024-05-22T23:04:36.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.005+0000 7fcae362c740 1 -- 172.21.15.84:0/3119832124 wait complete. 2024-05-22T23:04:36.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.005+0000 7fcae362c740 1 -- start start 2024-05-22T23:04:36.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.005+0000 7fcae362c740 1 Processor -- start 2024-05-22T23:04:36.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.005+0000 7fcae362c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bec1c0 0x555ad7bff640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.005+0000 7fcae362c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad7bed8b0 0x555ad7bfa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.005+0000 7fcae362c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ad7bf6f40 0x555ad7bfaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.005+0000 7fcae362c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc1aa080 con 0x555ad7bec1c0 2024-05-22T23:04:36.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.005+0000 7fcae362c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc1ab4f0 con 0x555ad7bed8b0 2024-05-22T23:04:36.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.005+0000 7fcae362c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc1aa750 con 0x555ad7bf6f40 2024-05-22T23:04:36.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.005+0000 7fcae362c740 10 client.0 _ll_register_callbacks cb 0x7fcacc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:04:36.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.005+0000 7fcada59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bec1c0 0x555ad7bff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.005+0000 7fcada59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bec1c0 0x555ad7bff640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36936/0 (socket says 172.21.15.84:36936) 2024-05-22T23:04:36.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.005+0000 7fcada59c640 1 -- 172.21.15.84:0/3588360018 learned_addr learned my addr 172.21.15.84:0/3588360018 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:36.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.005+0000 7fcad9d9b640 1 --2- 172.21.15.84:0/3588360018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad7bed8b0 0x555ad7bfa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.005+0000 7fcadad9d640 1 --2- 172.21.15.84:0/3588360018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ad7bf6f40 0x555ad7bfaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.006+0000 7fcada59c640 1 -- 172.21.15.84:0/3588360018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ad7bf6f40 msgr2=0x555ad7bfaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:36.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.006+0000 7fcada59c640 1 --2- 172.21.15.84:0/3588360018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ad7bf6f40 0x555ad7bfaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.006+0000 7fcada59c640 1 -- 172.21.15.84:0/3588360018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad7bed8b0 msgr2=0x555ad7bfa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:36.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.006+0000 7fcada59c640 1 --2- 172.21.15.84:0/3588360018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad7bed8b0 0x555ad7bfa9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.006+0000 7fcada59c640 1 -- 172.21.15.84:0/3588360018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ad7b0afe0 con 0x555ad7bec1c0 2024-05-22T23:04:36.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.006+0000 7fcada59c640 1 --2- 172.21.15.84:0/3588360018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bec1c0 0x555ad7bff640 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fcacc031040 tx=0x7fcacc0095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0e070555a958a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:36.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.006+0000 7fcac1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:04:36.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.006+0000 7fcac1ffb640 1 -- 172.21.15.84:0/3588360018 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc049070 con 0x555ad7bec1c0 2024-05-22T23:04:36.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.006+0000 7fcac1ffb640 1 -- 172.21.15.84:0/3588360018 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fcacc02e040 con 0x555ad7bec1c0 2024-05-22T23:04:36.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.006+0000 7fcae362c740 1 -- 172.21.15.84:0/3588360018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcad009bdd0 con 0x555ad7bec1c0 2024-05-22T23:04:36.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.006+0000 7fcac1ffb640 1 -- 172.21.15.84:0/3588360018 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fcacc05c050 con 0x555ad7bec1c0 2024-05-22T23:04:36.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.006+0000 7fcae362c740 10 client.15876 fetch_fsmap learned FSMap version 18 2024-05-22T23:04:36.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.006+0000 7fcae362c740 1 -- 172.21.15.84:0/3588360018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x555ad7adc1c0 con 0x555ad7bec1c0 2024-05-22T23:04:36.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.006+0000 7fcac1ffb640 1 -- 172.21.15.84:0/3588360018 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcad009bdd0 con 0x555ad7bec1c0 2024-05-22T23:04:36.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.006+0000 7fcac1ffb640 1 -- 172.21.15.84:0/3588360018 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fcacc043040 con 0x555ad7bec1c0 2024-05-22T23:04:36.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.006+0000 7fcae362c740 10 client.15876 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:04:36.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.006+0000 7fcae362c740 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:04:36.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.006+0000 7fcae362c740 20 client.15876 populate_metadata read hostname 'smithi084' 2024-05-22T23:04:36.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.006+0000 7fcae362c740 1 --2- 172.21.15.84:0/3588360018 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x555ad7c6de50 0x555ad7c70270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.006+0000 7fcae362c740 4 client.15876 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:04:36.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.006+0000 7fcae362c740 1 -- 172.21.15.84:0/3588360018 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555ad7b53830 con 0x555ad7c6de50 2024-05-22T23:04:36.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.007+0000 7fcad9d9b640 1 --2- 172.21.15.84:0/3588360018 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x555ad7c6de50 0x555ad7c70270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.007+0000 7fcac1ffb640 10 client.15876 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:36.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.007+0000 7fcad9d9b640 1 --2- 172.21.15.84:0/3588360018 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x555ad7c6de50 0x555ad7c70270 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cee75c0032ffae3 server_cookie=fa33f777eaabc378 in_seq=0 out_seq=0 2024-05-22T23:04:36.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.008+0000 7fcac1ffb640 1 -- 172.21.15.84:0/3588360018 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555ad7b53830 con 0x555ad7c6de50 2024-05-22T23:04:36.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.008+0000 7fcac1ffb640 10 client.15876 handle_command_reply: tid=0 2024-05-22T23:04:36.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.008+0000 7fcae362c740 1 client.15876 shutdown 2024-05-22T23:04:36.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.008+0000 7fcae362c740 1 -- 172.21.15.84:0/3588360018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bec1c0 msgr2=0x555ad7bff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:36.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.008+0000 7fcae362c740 1 --2- 172.21.15.84:0/3588360018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bec1c0 0x555ad7bff640 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fcacc031040 tx=0x7fcacc0095b0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.008+0000 7fcae362c740 1 -- 172.21.15.84:0/3588360018 shutdown_connections 2024-05-22T23:04:36.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.008+0000 7fcae362c740 1 --2- 172.21.15.84:0/3588360018 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x555ad7c6de50 0x555ad7c70270 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.008+0000 7fcae362c740 1 --2- 172.21.15.84:0/3588360018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ad7bf6f40 0x555ad7bfaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.008+0000 7fcac1ffb640 0 client.15876 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:36.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.008+0000 7fcae362c740 1 --2- 172.21.15.84:0/3588360018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad7bed8b0 0x555ad7bfa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.008+0000 7fcae362c740 1 --2- 172.21.15.84:0/3588360018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ad7bec1c0 0x555ad7bff640 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.008+0000 7fcae362c740 1 -- 172.21.15.84:0/3588360018 >> 172.21.15.84:0/3588360018 conn(0x555ad7bb08e0 msgr2=0x555ad7bedc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:36.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.008+0000 7fcae362c740 1 -- 172.21.15.84:0/3588360018 shutdown_connections 2024-05-22T23:04:36.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.008+0000 7fcae362c740 1 -- 172.21.15.84:0/3588360018 wait complete. 2024-05-22T23:04:36.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.008+0000 7fcae362c740 20 client.15876 trim_cache size 0 max 16384 2024-05-22T23:04:36.019 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:04:36.019 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "no active scrubs running", 2024-05-22T23:04:36.019 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": {} 2024-05-22T23:04:36.019 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:04:36.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.009+0000 7fcae091b640 1 -- 172.21.15.84:0/1311403012 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fcac40a2d00 msgr2=0x7fcac40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:36.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.009+0000 7fcae091b640 1 --2- 172.21.15.84:0/1311403012 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fcac40a2d00 0x7fcac40a51c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcad4009fd0 tx=0x7fcad4021000 comp rx=0 tx=0).stop 2024-05-22T23:04:36.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.009+0000 7fcae091b640 1 -- 172.21.15.84:0/1311403012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcadc101100 msgr2=0x7fcadc198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:36.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.009+0000 7fcae091b640 1 --2- 172.21.15.84:0/1311403012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcadc101100 0x7fcadc198680 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcad000e9f0 tx=0x7fcad000eeb0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.009+0000 7fcae091b640 1 -- 172.21.15.84:0/1311403012 shutdown_connections 2024-05-22T23:04:36.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.009+0000 7fcae091b640 1 --2- 172.21.15.84:0/1311403012 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fcac40a2d00 0x7fcac40a51c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.009+0000 7fcae091b640 1 --2- 172.21.15.84:0/1311403012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcadc106750 0x7fcadc19ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.009+0000 7fcae091b640 1 --2- 172.21.15.84:0/1311403012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcadc101100 0x7fcadc198680 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.009+0000 7fcae091b640 1 --2- 172.21.15.84:0/1311403012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcadc100740 0x7fcadc198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.009+0000 7fcae091b640 1 -- 172.21.15.84:0/1311403012 >> 172.21.15.84:0/1311403012 conn(0x7fcadc077870 msgr2=0x7fcadc102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:36.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.009+0000 7fcae091b640 1 -- 172.21.15.84:0/1311403012 shutdown_connections 2024-05-22T23:04:36.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.009+0000 7fcae091b640 1 -- 172.21.15.84:0/1311403012 wait complete. 2024-05-22T23:04:36.023 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-22T23:04:36.024 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:04:36.129 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.124+0000 7f31e7679640 1 -- 172.21.15.84:0/393485151 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e0103420 msgr2=0x7f31e010b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:36.129 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.124+0000 7f31e7679640 1 --2- 172.21.15.84:0/393485151 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e0103420 0x7f31e010b900 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009900 tx=0x7f31dc02a510 comp rx=0 tx=0).stop 2024-05-22T23:04:36.129 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.124+0000 7f31e7679640 1 -- 172.21.15.84:0/393485151 shutdown_connections 2024-05-22T23:04:36.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.124+0000 7f31e7679640 1 --2- 172.21.15.84:0/393485151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31e010be40 0x7f31e010e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.124+0000 7f31e7679640 1 --2- 172.21.15.84:0/393485151 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e0103420 0x7f31e010b900 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.124+0000 7f31e7679640 1 --2- 172.21.15.84:0/393485151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31e0102b00 0x7f31e0102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.124+0000 7f31e7679640 1 -- 172.21.15.84:0/393485151 >> 172.21.15.84:0/393485151 conn(0x7f31e0077870 msgr2=0x7f31e0077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:36.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.124+0000 7f31e7679640 1 -- 172.21.15.84:0/393485151 shutdown_connections 2024-05-22T23:04:36.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.124+0000 7f31e7679640 1 -- 172.21.15.84:0/393485151 wait complete. 2024-05-22T23:04:36.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.124+0000 7f31e7679640 1 Processor -- start 2024-05-22T23:04:36.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.124+0000 7f31e7679640 1 -- start start 2024-05-22T23:04:36.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.125+0000 7f31e7679640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e0102b00 0x7f31e01980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.125+0000 7f31e7679640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31e0103420 0x7f31e0198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.125+0000 7f31e7679640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31e010be40 0x7f31e019c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.125+0000 7f31e7679640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f31e0111c40 con 0x7f31e0102b00 2024-05-22T23:04:36.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.125+0000 7f31e7679640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f31e0111ac0 con 0x7f31e0103420 2024-05-22T23:04:36.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.125+0000 7f31e7679640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f31e0111dc0 con 0x7f31e010be40 2024-05-22T23:04:36.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.125+0000 7f31e5415640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e0102b00 0x7f31e01980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.125+0000 7f31e5415640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e0102b00 0x7f31e01980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36966/0 (socket says 172.21.15.84:36966) 2024-05-22T23:04:36.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.125+0000 7f31e5415640 1 -- 172.21.15.84:0/1646442153 learned_addr learned my addr 172.21.15.84:0/1646442153 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:36.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.125+0000 7f31e4c14640 1 --2- 172.21.15.84:0/1646442153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31e0103420 0x7f31e0198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.125+0000 7f31e5c16640 1 --2- 172.21.15.84:0/1646442153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31e010be40 0x7f31e019c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.125+0000 7f31e5415640 1 -- 172.21.15.84:0/1646442153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31e010be40 msgr2=0x7f31e019c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:36.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.125+0000 7f31e5415640 1 --2- 172.21.15.84:0/1646442153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31e010be40 0x7f31e019c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.125+0000 7f31e5415640 1 -- 172.21.15.84:0/1646442153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31e0103420 msgr2=0x7f31e0198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:36.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.125+0000 7f31e5415640 1 --2- 172.21.15.84:0/1646442153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31e0103420 0x7f31e0198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.125+0000 7f31e5415640 1 -- 172.21.15.84:0/1646442153 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e019d0d0 con 0x7f31e0102b00 2024-05-22T23:04:36.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.125+0000 7f31e5415640 1 --2- 172.21.15.84:0/1646442153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e0102b00 0x7f31e01980e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f31d400e9f0 tx=0x7f31d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:36.132 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.125+0000 7f31d27fc640 1 -- 172.21.15.84:0/1646442153 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d400cd80 con 0x7f31e0102b00 2024-05-22T23:04:36.132 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.125+0000 7f31d27fc640 1 -- 172.21.15.84:0/1646442153 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31d4010910 con 0x7f31e0102b00 2024-05-22T23:04:36.132 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.125+0000 7f31e7679640 1 -- 172.21.15.84:0/1646442153 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e019d3c0 con 0x7f31e0102b00 2024-05-22T23:04:36.132 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.125+0000 7f31d27fc640 1 -- 172.21.15.84:0/1646442153 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4018990 con 0x7f31e0102b00 2024-05-22T23:04:36.132 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.125+0000 7f31e7679640 1 -- 172.21.15.84:0/1646442153 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e01a4c50 con 0x7f31e0102b00 2024-05-22T23:04:36.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.126+0000 7f31e7679640 1 -- 172.21.15.84:0/1646442153 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e0068690 con 0x7f31e0102b00 2024-05-22T23:04:36.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.128+0000 7f31d27fc640 1 -- 172.21.15.84:0/1646442153 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f31d4010ac0 con 0x7f31e0102b00 2024-05-22T23:04:36.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.129+0000 7f31d27fc640 1 --2- 172.21.15.84:0/1646442153 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f31cc0a2d00 0x7f31cc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.129+0000 7f31e4c14640 1 --2- 172.21.15.84:0/1646442153 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f31cc0a2d00 0x7f31cc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.129+0000 7f31d27fc640 1 -- 172.21.15.84:0/1646442153 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f31d40cfb30 con 0x7f31e0102b00 2024-05-22T23:04:36.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.129+0000 7f31e4c14640 1 --2- 172.21.15.84:0/1646442153 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f31cc0a2d00 0x7f31cc0a51c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009230 tx=0x7f31dc0058d0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:36.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.132+0000 7f31d27fc640 1 -- 172.21.15.84:0/1646442153 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f31d409c100 con 0x7f31e0102b00 2024-05-22T23:04:36.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.352+0000 7f31e7679640 1 -- 172.21.15.84:0/1646442153 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31e0199430 con 0x7f31e0102b00 2024-05-22T23:04:36.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.353+0000 7f31d27fc640 1 -- 172.21.15.84:0/1646442153 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7f31e0199430 con 0x7f31e0102b00 2024-05-22T23:04:36.357 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:04:36.357 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:04:36.357 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:04:36.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.355+0000 7f31e7679640 1 -- 172.21.15.84:0/1646442153 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f31cc0a2d00 msgr2=0x7f31cc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:36.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.355+0000 7f31e7679640 1 --2- 172.21.15.84:0/1646442153 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f31cc0a2d00 0x7f31cc0a51c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009230 tx=0x7f31dc0058d0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.355+0000 7f31e7679640 1 -- 172.21.15.84:0/1646442153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e0102b00 msgr2=0x7f31e01980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:36.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.355+0000 7f31e7679640 1 --2- 172.21.15.84:0/1646442153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e0102b00 0x7f31e01980e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f31d400e9f0 tx=0x7f31d400eeb0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.355+0000 7f31e7679640 1 -- 172.21.15.84:0/1646442153 shutdown_connections 2024-05-22T23:04:36.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.355+0000 7f31e7679640 1 --2- 172.21.15.84:0/1646442153 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f31cc0a2d00 0x7f31cc0a51c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.355+0000 7f31e7679640 1 --2- 172.21.15.84:0/1646442153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31e010be40 0x7f31e019c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.355+0000 7f31e7679640 1 --2- 172.21.15.84:0/1646442153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31e0103420 0x7f31e0198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.355+0000 7f31e7679640 1 --2- 172.21.15.84:0/1646442153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e0102b00 0x7f31e01980e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.355+0000 7f31e7679640 1 -- 172.21.15.84:0/1646442153 >> 172.21.15.84:0/1646442153 conn(0x7f31e0077870 msgr2=0x7f31e00fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:36.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.355+0000 7f31e7679640 1 -- 172.21.15.84:0/1646442153 shutdown_connections 2024-05-22T23:04:36.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.355+0000 7f31e7679640 1 -- 172.21.15.84:0/1646442153 wait complete. 2024-05-22T23:04:36.370 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-22T23:04:36.427 INFO:tasks.workunit.client.0.smithi084.stdout: 1 245048 16.34 MB/sec warmup 82 sec latency 13.514 ms 2024-05-22T23:04:36.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.471+0000 7fa683ac1640 1 -- 172.21.15.84:0/914135985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa67c06a8f0 msgr2=0x7fa67c10e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:36.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.471+0000 7fa683ac1640 1 --2- 172.21.15.84:0/914135985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa67c06a8f0 0x7fa67c10e720 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa674009cd0 tx=0x7fa67402a8e0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.471+0000 7fa683ac1640 1 -- 172.21.15.84:0/914135985 shutdown_connections 2024-05-22T23:04:36.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.471+0000 7fa683ac1640 1 --2- 172.21.15.84:0/914135985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa67c06a8f0 0x7fa67c10e720 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.471+0000 7fa683ac1640 1 --2- 172.21.15.84:0/914135985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa67c069f30 0x7fa67c06a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.471+0000 7fa683ac1640 1 --2- 172.21.15.84:0/914135985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa67c107860 0x7fa67c107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.471+0000 7fa683ac1640 1 -- 172.21.15.84:0/914135985 >> 172.21.15.84:0/914135985 conn(0x7fa67c0777d0 msgr2=0x7fa67c077be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:36.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.471+0000 7fa683ac1640 1 -- 172.21.15.84:0/914135985 shutdown_connections 2024-05-22T23:04:36.478 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.471+0000 7fa683ac1640 1 -- 172.21.15.84:0/914135985 wait complete. 2024-05-22T23:04:36.478 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.472+0000 7fa683ac1640 1 Processor -- start 2024-05-22T23:04:36.478 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.472+0000 7fa683ac1640 1 -- start start 2024-05-22T23:04:36.478 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.472+0000 7fa683ac1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa67c069f30 0x7fa67c198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.478 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.472+0000 7fa683ac1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa67c06a8f0 0x7fa67c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.478 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.472+0000 7fa683ac1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa67c107860 0x7fa67c19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.478 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.472+0000 7fa683ac1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c111c70 con 0x7fa67c06a8f0 2024-05-22T23:04:36.478 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.472+0000 7fa683ac1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c111af0 con 0x7fa67c069f30 2024-05-22T23:04:36.478 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.472+0000 7fa683ac1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c111df0 con 0x7fa67c107860 2024-05-22T23:04:36.478 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.472+0000 7fa68105c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa67c06a8f0 0x7fa67c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.478 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.472+0000 7fa68185d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa67c069f30 0x7fa67c198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.478 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.472+0000 7fa68105c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa67c06a8f0 0x7fa67c198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36970/0 (socket says 172.21.15.84:36970) 2024-05-22T23:04:36.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.472+0000 7fa68105c640 1 -- 172.21.15.84:0/1547033780 learned_addr learned my addr 172.21.15.84:0/1547033780 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:36.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.472+0000 7fa68205e640 1 --2- 172.21.15.84:0/1547033780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa67c107860 0x7fa67c19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.473+0000 7fa68105c640 1 -- 172.21.15.84:0/1547033780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa67c107860 msgr2=0x7fa67c19c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:36.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.473+0000 7fa68105c640 1 --2- 172.21.15.84:0/1547033780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa67c107860 0x7fa67c19c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.473+0000 7fa68105c640 1 -- 172.21.15.84:0/1547033780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa67c069f30 msgr2=0x7fa67c198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:36.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.473+0000 7fa68105c640 1 --2- 172.21.15.84:0/1547033780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa67c069f30 0x7fa67c198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.473+0000 7fa68105c640 1 -- 172.21.15.84:0/1547033780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa67c19d120 con 0x7fa67c06a8f0 2024-05-22T23:04:36.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.473+0000 7fa68105c640 1 --2- 172.21.15.84:0/1547033780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa67c06a8f0 0x7fa67c198670 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00e9f0 tx=0x7fa66c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:36.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.473+0000 7fa672ffd640 1 -- 172.21.15.84:0/1547033780 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c00cd80 con 0x7fa67c06a8f0 2024-05-22T23:04:36.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.473+0000 7fa672ffd640 1 -- 172.21.15.84:0/1547033780 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa66c010910 con 0x7fa67c06a8f0 2024-05-22T23:04:36.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.473+0000 7fa683ac1640 1 -- 172.21.15.84:0/1547033780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa67c19d410 con 0x7fa67c06a8f0 2024-05-22T23:04:36.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.473+0000 7fa672ffd640 1 -- 172.21.15.84:0/1547033780 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c018990 con 0x7fa67c06a8f0 2024-05-22T23:04:36.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.473+0000 7fa683ac1640 1 -- 172.21.15.84:0/1547033780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa67c1a4ca0 con 0x7fa67c06a8f0 2024-05-22T23:04:36.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.475+0000 7fa672ffd640 1 -- 172.21.15.84:0/1547033780 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa66c010ac0 con 0x7fa67c06a8f0 2024-05-22T23:04:36.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.476+0000 7fa6867cd740 1 Processor -- start 2024-05-22T23:04:36.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.476+0000 7fa672ffd640 1 --2- 172.21.15.84:0/1547033780 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fa6680a2d80 0x7fa6680a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.476+0000 7fa672ffd640 1 -- 172.21.15.84:0/1547033780 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fa66c0cf6c0 con 0x7fa67c06a8f0 2024-05-22T23:04:36.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.476+0000 7fa68205e640 1 --2- 172.21.15.84:0/1547033780 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fa6680a2d80 0x7fa6680a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.476+0000 7fa6867cd740 1 -- start start 2024-05-22T23:04:36.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.477+0000 7fa6867cd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575f6d18e80 0x5575f6d19260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.477+0000 7fa6867cd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d197a0 0x5575f6d27e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.477+0000 7fa6867cd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575f6d28380 0x5575f6d2a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.477+0000 7fa6867cd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5575f6c7ddb0 con 0x5575f6d197a0 2024-05-22T23:04:36.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.477+0000 7fa6867cd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5575f6c1c400 con 0x5575f6d18e80 2024-05-22T23:04:36.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.477+0000 7fa6867cd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5575f6c79d70 con 0x5575f6d28380 2024-05-22T23:04:36.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.477+0000 7fa68205e640 1 --2- 172.21.15.84:0/1547033780 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fa6680a2d80 0x7fa6680a5240 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa674004ab0 tx=0x7fa674004a40 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:36.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.477+0000 7fa68185d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575f6d18e80 0x5575f6d19260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.477+0000 7fa68105c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d197a0 0x5575f6d27e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.477+0000 7fa68185d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575f6d18e80 0x5575f6d19260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:43616/0 (socket says 172.21.15.84:43616) 2024-05-22T23:04:36.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.477+0000 7fa68185d640 1 -- 172.21.15.84:0/1522870614 learned_addr learned my addr 172.21.15.84:0/1522870614 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:36.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.477+0000 7fa68105c640 1 -- 172.21.15.84:0/1522870614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575f6d28380 msgr2=0x5575f6d2a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:36.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.477+0000 7fa68105c640 1 --2- 172.21.15.84:0/1522870614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575f6d28380 0x5575f6d2a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.477+0000 7fa68105c640 1 -- 172.21.15.84:0/1522870614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575f6d18e80 msgr2=0x5575f6d19260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:36.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.477+0000 7fa68105c640 1 --2- 172.21.15.84:0/1522870614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575f6d18e80 0x5575f6d19260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.477+0000 7fa68105c640 1 -- 172.21.15.84:0/1522870614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575f6c0e2f0 con 0x5575f6d197a0 2024-05-22T23:04:36.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.477+0000 7fa68185d640 1 --2- 172.21.15.84:0/1522870614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575f6d18e80 0x5575f6d19260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:36.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.477+0000 7fa68105c640 1 --2- 172.21.15.84:0/1522870614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d197a0 0x5575f6d27e40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa66c0cce10 tx=0x7fa66c09bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d7d80d6df5b243 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:36.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.477+0000 7fa667fff640 1 -- 172.21.15.84:0/1522870614 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c099f90 con 0x5575f6d197a0 2024-05-22T23:04:36.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.477+0000 7fa667fff640 1 -- 172.21.15.84:0/1522870614 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa66c0cd090 con 0x5575f6d197a0 2024-05-22T23:04:36.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.478+0000 7fa667fff640 1 -- 172.21.15.84:0/1522870614 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c09b340 con 0x5575f6d197a0 2024-05-22T23:04:36.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.478+0000 7fa6867cd740 1 -- 172.21.15.84:0/1522870614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d197a0 msgr2=0x5575f6d27e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:36.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.478+0000 7fa6867cd740 1 --2- 172.21.15.84:0/1522870614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d197a0 0x5575f6d27e40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa66c0cce10 tx=0x7fa66c09bd50 comp rx=0 tx=0).stop 2024-05-22T23:04:36.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.478+0000 7fa6867cd740 1 -- 172.21.15.84:0/1522870614 shutdown_connections 2024-05-22T23:04:36.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.478+0000 7fa6867cd740 1 --2- 172.21.15.84:0/1522870614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575f6d28380 0x5575f6d2a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.478+0000 7fa6867cd740 1 --2- 172.21.15.84:0/1522870614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d197a0 0x5575f6d27e40 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.478+0000 7fa6867cd740 1 --2- 172.21.15.84:0/1522870614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575f6d18e80 0x5575f6d19260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.478+0000 7fa6867cd740 1 -- 172.21.15.84:0/1522870614 >> 172.21.15.84:0/1522870614 conn(0x5575f6d152a0 msgr2=0x5575f6d15680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:36.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.478+0000 7fa6867cd740 1 -- 172.21.15.84:0/1522870614 shutdown_connections 2024-05-22T23:04:36.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.478+0000 7fa6867cd740 1 -- 172.21.15.84:0/1522870614 wait complete. 2024-05-22T23:04:36.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.479+0000 7fa6867cd740 1 -- start start 2024-05-22T23:04:36.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.479+0000 7fa6867cd740 1 Processor -- start 2024-05-22T23:04:36.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.479+0000 7fa6867cd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d18e80 0x5575f6d25310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.479+0000 7fa6867cd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575f6d197a0 0x5575f6d25850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.479+0000 7fa6867cd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575f6d28380 0x5575f6d25d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.479+0000 7fa6867cd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5575f6d2b0e0 con 0x5575f6d18e80 2024-05-22T23:04:36.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.479+0000 7fa6867cd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5575f6c382e0 con 0x5575f6d28380 2024-05-22T23:04:36.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.479+0000 7fa6867cd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5575f6d2b260 con 0x5575f6d197a0 2024-05-22T23:04:36.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.480+0000 7fa6867cd740 10 client.0 _ll_register_callbacks cb 0x5575f6c41a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:04:36.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.480+0000 7fa68185d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d18e80 0x5575f6d25310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.480+0000 7fa68105c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575f6d197a0 0x5575f6d25850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.480+0000 7fa68105c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575f6d197a0 0x5575f6d25850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:57626/0 (socket says 172.21.15.84:57626) 2024-05-22T23:04:36.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.480+0000 7fa68105c640 1 -- 172.21.15.84:0/3095828405 learned_addr learned my addr 172.21.15.84:0/3095828405 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:36.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.481+0000 7fa68205e640 1 --2- 172.21.15.84:0/3095828405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575f6d28380 0x5575f6d25d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.481+0000 7fa68185d640 1 -- 172.21.15.84:0/3095828405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575f6d197a0 msgr2=0x5575f6d25850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:36.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.481+0000 7fa68185d640 1 --2- 172.21.15.84:0/3095828405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575f6d197a0 0x5575f6d25850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.482+0000 7fa68185d640 1 -- 172.21.15.84:0/3095828405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575f6d28380 msgr2=0x5575f6d25d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:36.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.482+0000 7fa68185d640 1 --2- 172.21.15.84:0/3095828405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575f6d28380 0x5575f6d25d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.482+0000 7fa68185d640 1 -- 172.21.15.84:0/3095828405 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575f69913f0 con 0x5575f6d18e80 2024-05-22T23:04:36.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.482+0000 7fa68185d640 1 --2- 172.21.15.84:0/3095828405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d18e80 0x5575f6d25310 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa678012730 tx=0x7fa678012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47aae131a8a2bceb server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:36.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.486+0000 7fa6657fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:04:36.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.486+0000 7fa6657fa640 1 -- 172.21.15.84:0/3095828405 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa678004050 con 0x5575f6d18e80 2024-05-22T23:04:36.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.486+0000 7fa6657fa640 1 -- 172.21.15.84:0/3095828405 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fa6780041f0 con 0x5575f6d18e80 2024-05-22T23:04:36.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.486+0000 7fa6867cd740 1 -- 172.21.15.84:0/3095828405 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa66c0cd090 con 0x5575f6d18e80 2024-05-22T23:04:36.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.487+0000 7fa6657fa640 1 -- 172.21.15.84:0/3095828405 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fa6780043f0 con 0x5575f6d18e80 2024-05-22T23:04:36.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.487+0000 7fa6867cd740 10 client.15894 fetch_fsmap learned FSMap version 18 2024-05-22T23:04:36.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.487+0000 7fa6867cd740 1 -- 172.21.15.84:0/3095828405 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5575f6a5b7d0 con 0x5575f6d18e80 2024-05-22T23:04:36.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.488+0000 7fa6657fa640 1 -- 172.21.15.84:0/3095828405 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa66c0cd090 con 0x5575f6d18e80 2024-05-22T23:04:36.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.488+0000 7fa6657fa640 1 -- 172.21.15.84:0/3095828405 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fa678003c70 con 0x5575f6d18e80 2024-05-22T23:04:36.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.488+0000 7fa6867cd740 10 client.15894 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:04:36.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.488+0000 7fa6867cd740 10 client.15894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:04:36.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.488+0000 7fa6867cd740 20 client.15894 populate_metadata read hostname 'smithi084' 2024-05-22T23:04:36.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.488+0000 7fa6867cd740 1 --2- 172.21.15.84:0/3095828405 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5575f6d386c0 0x5575f6d3aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.488+0000 7fa6867cd740 4 client.15894 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:04:36.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.488+0000 7fa6867cd740 1 -- 172.21.15.84:0/3095828405 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575f6c8b4e0 con 0x5575f6d386c0 2024-05-22T23:04:36.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.496+0000 7fa68105c640 1 --2- 172.21.15.84:0/3095828405 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5575f6d386c0 0x5575f6d3aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.499+0000 7fa68105c640 1 --2- 172.21.15.84:0/3095828405 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5575f6d386c0 0x5575f6d3aab0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f08bb71d071a8e42 server_cookie=f8199d2352d46763 in_seq=0 out_seq=0 2024-05-22T23:04:36.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.499+0000 7fa6657fa640 10 client.15894 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:36.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.507+0000 7fa6657fa640 1 -- 172.21.15.84:0/3095828405 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5575f6c8b4e0 con 0x5575f6d386c0 2024-05-22T23:04:36.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.507+0000 7fa6657fa640 10 client.15894 handle_command_reply: tid=0 2024-05-22T23:04:36.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.508+0000 7fa6867cd740 1 client.15894 shutdown 2024-05-22T23:04:36.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.508+0000 7fa6867cd740 1 -- 172.21.15.84:0/3095828405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d18e80 msgr2=0x5575f6d25310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:36.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.508+0000 7fa6867cd740 1 --2- 172.21.15.84:0/3095828405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d18e80 0x5575f6d25310 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa678012730 tx=0x7fa678012bf0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.508+0000 7fa6867cd740 1 -- 172.21.15.84:0/3095828405 shutdown_connections 2024-05-22T23:04:36.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.508+0000 7fa6867cd740 1 --2- 172.21.15.84:0/3095828405 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5575f6d386c0 0x5575f6d3aab0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.509+0000 7fa6867cd740 1 --2- 172.21.15.84:0/3095828405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575f6d28380 0x5575f6d25d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.509+0000 7fa6867cd740 1 --2- 172.21.15.84:0/3095828405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575f6d197a0 0x5575f6d25850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.509+0000 7fa6867cd740 1 --2- 172.21.15.84:0/3095828405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d18e80 0x5575f6d25310 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.509+0000 7fa6657fa640 0 client.15894 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:36.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.509+0000 7fa6867cd740 1 -- 172.21.15.84:0/3095828405 >> 172.21.15.84:0/3095828405 conn(0x5575f6d152a0 msgr2=0x5575f6d1b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:36.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.509+0000 7fa6867cd740 1 -- 172.21.15.84:0/3095828405 shutdown_connections 2024-05-22T23:04:36.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.509+0000 7fa6867cd740 1 -- 172.21.15.84:0/3095828405 wait complete. 2024-05-22T23:04:36.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.509+0000 7fa6867cd740 20 client.15894 trim_cache size 0 max 16384 2024-05-22T23:04:36.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.527+0000 7fa6867cd740 1 Processor -- start 2024-05-22T23:04:36.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.527+0000 7fa6867cd740 1 -- start start 2024-05-22T23:04:36.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.528+0000 7fa6867cd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575f6d18e80 0x5575f6d33fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.528+0000 7fa6867cd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d197a0 0x5575f6d24360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.528+0000 7fa6867cd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575f6d28380 0x5575f6d25bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.528+0000 7fa6867cd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1a94c0 con 0x5575f6d197a0 2024-05-22T23:04:36.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.528+0000 7fa6867cd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1abd40 con 0x5575f6d28380 2024-05-22T23:04:36.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.528+0000 7fa6867cd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1a9b30 con 0x5575f6d18e80 2024-05-22T23:04:36.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.528+0000 7fa68105c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d197a0 0x5575f6d24360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.528+0000 7fa68105c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d197a0 0x5575f6d24360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36990/0 (socket says 172.21.15.84:36990) 2024-05-22T23:04:36.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.528+0000 7fa68105c640 1 -- 172.21.15.84:0/185716495 learned_addr learned my addr 172.21.15.84:0/185716495 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:36.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.528+0000 7fa68205e640 1 --2- 172.21.15.84:0/185716495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575f6d28380 0x5575f6d25bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.528+0000 7fa68105c640 1 -- 172.21.15.84:0/185716495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575f6d18e80 msgr2=0x5575f6d33fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:36.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.528+0000 7fa68105c640 1 --2- 172.21.15.84:0/185716495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575f6d18e80 0x5575f6d33fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.528+0000 7fa68105c640 1 -- 172.21.15.84:0/185716495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575f6d28380 msgr2=0x5575f6d25bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:36.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.528+0000 7fa68105c640 1 --2- 172.21.15.84:0/185716495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575f6d28380 0x5575f6d25bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.528+0000 7fa68105c640 1 -- 172.21.15.84:0/185716495 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575f6d278e0 con 0x5575f6d197a0 2024-05-22T23:04:36.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.528+0000 7fa68105c640 1 --2- 172.21.15.84:0/185716495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d197a0 0x5575f6d24360 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa66c099f40 tx=0x7fa66c098060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c552beb5e1855bdb server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:36.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.528+0000 7fa6677fe640 1 -- 172.21.15.84:0/185716495 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c0cd090 con 0x5575f6d197a0 2024-05-22T23:04:36.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.528+0000 7fa6677fe640 1 -- 172.21.15.84:0/185716495 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa66c09a010 con 0x5575f6d197a0 2024-05-22T23:04:36.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.528+0000 7fa6677fe640 1 -- 172.21.15.84:0/185716495 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c0979e0 con 0x5575f6d197a0 2024-05-22T23:04:36.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.529+0000 7fa6867cd740 1 -- 172.21.15.84:0/185716495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d197a0 msgr2=0x5575f6d24360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:36.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.529+0000 7fa6867cd740 1 --2- 172.21.15.84:0/185716495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d197a0 0x5575f6d24360 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa66c099f40 tx=0x7fa66c098060 comp rx=0 tx=0).stop 2024-05-22T23:04:36.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.529+0000 7fa6867cd740 1 -- 172.21.15.84:0/185716495 shutdown_connections 2024-05-22T23:04:36.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.529+0000 7fa6867cd740 1 --2- 172.21.15.84:0/185716495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575f6d28380 0x5575f6d25bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.529+0000 7fa6867cd740 1 --2- 172.21.15.84:0/185716495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d197a0 0x5575f6d24360 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.529+0000 7fa6867cd740 1 --2- 172.21.15.84:0/185716495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575f6d18e80 0x5575f6d33fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.529+0000 7fa6867cd740 1 -- 172.21.15.84:0/185716495 >> 172.21.15.84:0/185716495 conn(0x5575f6d152a0 msgr2=0x5575f6d269c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:36.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.529+0000 7fa6867cd740 1 -- 172.21.15.84:0/185716495 shutdown_connections 2024-05-22T23:04:36.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.529+0000 7fa6867cd740 1 -- 172.21.15.84:0/185716495 wait complete. 2024-05-22T23:04:36.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.529+0000 7fa6867cd740 1 -- start start 2024-05-22T23:04:36.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.530+0000 7fa6867cd740 1 Processor -- start 2024-05-22T23:04:36.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.530+0000 7fa6867cd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575f6d18e80 0x5575f6d2c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.530+0000 7fa6867cd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575f6d197a0 0x5575f6d2c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.530+0000 7fa6867cd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d28380 0x5575f6d9dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.530+0000 7fa6867cd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1aa200 con 0x5575f6d28380 2024-05-22T23:04:36.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.530+0000 7fa6867cd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1ab670 con 0x5575f6d18e80 2024-05-22T23:04:36.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.530+0000 7fa6867cd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1aa8d0 con 0x5575f6d197a0 2024-05-22T23:04:36.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.530+0000 7fa6867cd740 10 client.0 _ll_register_callbacks cb 0x7fa678009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:04:36.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.530+0000 7fa68205e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d28380 0x5575f6d9dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.530+0000 7fa68185d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575f6d18e80 0x5575f6d2c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.530+0000 7fa68205e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d28380 0x5575f6d9dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37002/0 (socket says 172.21.15.84:37002) 2024-05-22T23:04:36.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.530+0000 7fa68205e640 1 -- 172.21.15.84:0/613755026 learned_addr learned my addr 172.21.15.84:0/613755026 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:36.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.530+0000 7fa68105c640 1 --2- 172.21.15.84:0/613755026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575f6d197a0 0x5575f6d2c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.530+0000 7fa68185d640 1 -- 172.21.15.84:0/613755026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575f6d197a0 msgr2=0x5575f6d2c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:36.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.530+0000 7fa68185d640 1 --2- 172.21.15.84:0/613755026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575f6d197a0 0x5575f6d2c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.530+0000 7fa68185d640 1 -- 172.21.15.84:0/613755026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d28380 msgr2=0x5575f6d9dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:36.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.530+0000 7fa68185d640 1 --2- 172.21.15.84:0/613755026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d28380 0x5575f6d9dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.530+0000 7fa68185d640 1 -- 172.21.15.84:0/613755026 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575f6d9e110 con 0x5575f6d18e80 2024-05-22T23:04:36.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.530+0000 7fa68185d640 1 --2- 172.21.15.84:0/613755026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575f6d18e80 0x5575f6d2c2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa678010670 tx=0x7fa6780098a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e2ecd328c8a857c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:36.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.530+0000 7fa6657fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:04:36.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.531+0000 7fa6657fa640 1 -- 172.21.15.84:0/613755026 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa678010ac0 con 0x5575f6d18e80 2024-05-22T23:04:36.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.531+0000 7fa6657fa640 1 -- 172.21.15.84:0/613755026 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fa678010c60 con 0x5575f6d18e80 2024-05-22T23:04:36.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.531+0000 7fa6867cd740 1 -- 172.21.15.84:0/613755026 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa66c09a010 con 0x5575f6d18e80 2024-05-22T23:04:36.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.531+0000 7fa6657fa640 1 -- 172.21.15.84:0/613755026 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fa67800f920 con 0x5575f6d18e80 2024-05-22T23:04:36.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.531+0000 7fa6867cd740 10 client.25475 fetch_fsmap learned FSMap version 18 2024-05-22T23:04:36.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.531+0000 7fa6867cd740 1 -- 172.21.15.84:0/613755026 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5575f6d9f2b0 con 0x5575f6d18e80 2024-05-22T23:04:36.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.531+0000 7fa6657fa640 1 -- 172.21.15.84:0/613755026 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa66c09a010 con 0x5575f6d18e80 2024-05-22T23:04:36.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.531+0000 7fa6657fa640 1 -- 172.21.15.84:0/613755026 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fa67801ca70 con 0x5575f6d18e80 2024-05-22T23:04:36.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.531+0000 7fa6867cd740 10 client.25475 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:04:36.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.531+0000 7fa6867cd740 10 client.25475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:04:36.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.531+0000 7fa6867cd740 20 client.25475 populate_metadata read hostname 'smithi084' 2024-05-22T23:04:36.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.531+0000 7fa6867cd740 1 --2- 172.21.15.84:0/613755026 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5575f6d386c0 0x5575f6da1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:36.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.531+0000 7fa6867cd740 4 client.25475 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-22T23:04:36.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.531+0000 7fa6867cd740 1 -- 172.21.15.84:0/613755026 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5575f6c85830 con 0x5575f6d386c0 2024-05-22T23:04:36.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.531+0000 7fa68105c640 1 --2- 172.21.15.84:0/613755026 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5575f6d386c0 0x5575f6da1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:36.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.532+0000 7fa68105c640 1 --2- 172.21.15.84:0/613755026 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5575f6d386c0 0x5575f6da1a90 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c58952b6df08a75d server_cookie=f713fd3f2662826f in_seq=0 out_seq=0 2024-05-22T23:04:36.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.532+0000 7fa6657fa640 10 client.25475 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:36.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.533+0000 7fa6657fa640 1 -- 172.21.15.84:0/613755026 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5575f6c85830 con 0x5575f6d386c0 2024-05-22T23:04:36.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.533+0000 7fa6657fa640 10 client.25475 handle_command_reply: tid=0 2024-05-22T23:04:36.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.533+0000 7fa6867cd740 1 client.25475 shutdown 2024-05-22T23:04:36.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.533+0000 7fa6867cd740 1 -- 172.21.15.84:0/613755026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575f6d18e80 msgr2=0x5575f6d2c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:36.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.533+0000 7fa6867cd740 1 --2- 172.21.15.84:0/613755026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575f6d18e80 0x5575f6d2c2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa678010670 tx=0x7fa6780098a0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.533+0000 7fa6867cd740 1 -- 172.21.15.84:0/613755026 shutdown_connections 2024-05-22T23:04:36.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.533+0000 7fa6867cd740 1 --2- 172.21.15.84:0/613755026 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5575f6d386c0 0x5575f6da1a90 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.533+0000 7fa6867cd740 1 --2- 172.21.15.84:0/613755026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5575f6d28380 0x5575f6d9dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.533+0000 7fa6657fa640 0 client.25475 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:36.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.533+0000 7fa6867cd740 1 --2- 172.21.15.84:0/613755026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575f6d197a0 0x5575f6d2c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.533+0000 7fa6867cd740 1 --2- 172.21.15.84:0/613755026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575f6d18e80 0x5575f6d2c2f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.533+0000 7fa6867cd740 1 -- 172.21.15.84:0/613755026 >> 172.21.15.84:0/613755026 conn(0x5575f6d152a0 msgr2=0x5575f6d25ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:36.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.533+0000 7fa6867cd740 1 -- 172.21.15.84:0/613755026 shutdown_connections 2024-05-22T23:04:36.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.533+0000 7fa6867cd740 1 -- 172.21.15.84:0/613755026 wait complete. 2024-05-22T23:04:36.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.533+0000 7fa6867cd740 20 client.25475 trim_cache size 0 max 16384 2024-05-22T23:04:36.543 INFO:teuthology.orchestra.run.smithi084.stdout:[] 2024-05-22T23:04:36.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.534+0000 7fa683ac1640 1 -- 172.21.15.84:0/1547033780 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fa6680a2d80 msgr2=0x7fa6680a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:36.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.534+0000 7fa683ac1640 1 --2- 172.21.15.84:0/1547033780 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fa6680a2d80 0x7fa6680a5240 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa674004ab0 tx=0x7fa674004a40 comp rx=0 tx=0).stop 2024-05-22T23:04:36.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.534+0000 7fa683ac1640 1 -- 172.21.15.84:0/1547033780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa67c06a8f0 msgr2=0x7fa67c198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:36.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.534+0000 7fa683ac1640 1 --2- 172.21.15.84:0/1547033780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa67c06a8f0 0x7fa67c198670 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00e9f0 tx=0x7fa66c00eeb0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.534+0000 7fa683ac1640 1 -- 172.21.15.84:0/1547033780 shutdown_connections 2024-05-22T23:04:36.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.534+0000 7fa683ac1640 1 --2- 172.21.15.84:0/1547033780 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fa6680a2d80 0x7fa6680a5240 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.534+0000 7fa683ac1640 1 --2- 172.21.15.84:0/1547033780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa67c107860 0x7fa67c19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.534+0000 7fa683ac1640 1 --2- 172.21.15.84:0/1547033780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa67c06a8f0 0x7fa67c198670 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.534+0000 7fa683ac1640 1 --2- 172.21.15.84:0/1547033780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa67c069f30 0x7fa67c198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:36.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.534+0000 7fa683ac1640 1 -- 172.21.15.84:0/1547033780 >> 172.21.15.84:0/1547033780 conn(0x7fa67c0777d0 msgr2=0x7fa67c0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:36.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.534+0000 7fa683ac1640 1 -- 172.21.15.84:0/1547033780 shutdown_connections 2024-05-22T23:04:36.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:36.534+0000 7fa683ac1640 1 -- 172.21.15.84:0/1547033780 wait complete. 2024-05-22T23:04:36.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:36 smithi084 ceph-mon[32186]: scrub queued for path: / 2024-05-22T23:04:36.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:36 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:04:36.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:36 smithi084 ceph-mon[32186]: scrub summary: active paths [/] 2024-05-22T23:04:36.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:36 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:04:36.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:36 smithi084 ceph-mon[32186]: scrub summary: idle 2024-05-22T23:04:36.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:36 smithi204 ceph-mon[36211]: scrub queued for path: / 2024-05-22T23:04:36.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:36 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:04:36.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:36 smithi204 ceph-mon[36211]: scrub summary: active paths [/] 2024-05-22T23:04:36.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:36 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:04:36.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:36 smithi204 ceph-mon[36211]: scrub summary: idle 2024-05-22T23:04:36.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:36 smithi096 ceph-mon[36392]: scrub queued for path: / 2024-05-22T23:04:36.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:36 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:04:36.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:36 smithi096 ceph-mon[36392]: scrub summary: active paths [/] 2024-05-22T23:04:36.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:36 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:04:36.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:36 smithi096 ceph-mon[36392]: scrub summary: idle 2024-05-22T23:04:37.427 INFO:tasks.workunit.client.0.smithi084.stdout: 1 247357 16.25 MB/sec warmup 83 sec latency 101.206 ms 2024-05-22T23:04:37.548 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-22T23:04:37.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:04:37.549 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-22T23:04:37.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:37 smithi084 ceph-mon[32186]: pgmap v86: 97 pgs: 97 active+clean; 129 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.2 MiB/s wr, 372 op/s 2024-05-22T23:04:37.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:37 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1646442153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:04:37.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.657+0000 7f67f16d2640 1 -- 172.21.15.84:0/480579340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67ec102200 msgr2=0x7f67ec10e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:37.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.657+0000 7f67f16d2640 1 --2- 172.21.15.84:0/480579340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67ec102200 0x7f67ec10e710 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009930 tx=0x7f67e402a420 comp rx=0 tx=0).stop 2024-05-22T23:04:37.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.657+0000 7f67f16d2640 1 -- 172.21.15.84:0/480579340 shutdown_connections 2024-05-22T23:04:37.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.657+0000 7f67f16d2640 1 --2- 172.21.15.84:0/480579340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67ec102200 0x7f67ec10e710 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.657+0000 7f67f16d2640 1 --2- 172.21.15.84:0/480579340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67ec101840 0x7f67ec101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.657+0000 7f67f16d2640 1 --2- 172.21.15.84:0/480579340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67ec107850 0x7f67ec107c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.657+0000 7f67f16d2640 1 -- 172.21.15.84:0/480579340 >> 172.21.15.84:0/480579340 conn(0x7f67ec078cc0 msgr2=0x7f67ec0790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:37.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.657+0000 7f67f16d2640 1 -- 172.21.15.84:0/480579340 shutdown_connections 2024-05-22T23:04:37.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.658+0000 7f67f16d2640 1 -- 172.21.15.84:0/480579340 wait complete. 2024-05-22T23:04:37.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.658+0000 7f67f16d2640 1 Processor -- start 2024-05-22T23:04:37.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.658+0000 7f67f16d2640 1 -- start start 2024-05-22T23:04:37.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.658+0000 7f67f16d2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67ec101840 0x7f67ec1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.658+0000 7f67f16d2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67ec102200 0x7f67ec1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.659+0000 7f67f16d2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67ec107850 0x7f67ec19ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.659+0000 7f67f16d2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec111c60 con 0x7f67ec102200 2024-05-22T23:04:37.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.659+0000 7f67f16d2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec111ae0 con 0x7f67ec101840 2024-05-22T23:04:37.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.659+0000 7f67f16d2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec111de0 con 0x7f67ec107850 2024-05-22T23:04:37.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.659+0000 7f67ea7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67ec102200 0x7f67ec1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.659+0000 7f67ea7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67ec102200 0x7f67ec1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37030/0 (socket says 172.21.15.84:37030) 2024-05-22T23:04:37.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.659+0000 7f67ea7fc640 1 -- 172.21.15.84:0/516531077 learned_addr learned my addr 172.21.15.84:0/516531077 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:37.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.659+0000 7f67eaffd640 1 --2- 172.21.15.84:0/516531077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67ec101840 0x7f67ec1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.659+0000 7f67eb7fe640 1 --2- 172.21.15.84:0/516531077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67ec107850 0x7f67ec19ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.659+0000 7f67ea7fc640 1 -- 172.21.15.84:0/516531077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67ec107850 msgr2=0x7f67ec19ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:37.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.659+0000 7f67ea7fc640 1 --2- 172.21.15.84:0/516531077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67ec107850 0x7f67ec19ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.659+0000 7f67ea7fc640 1 -- 172.21.15.84:0/516531077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67ec101840 msgr2=0x7f67ec1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:37.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.659+0000 7f67ea7fc640 1 --2- 172.21.15.84:0/516531077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67ec101840 0x7f67ec1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.659+0000 7f67ea7fc640 1 -- 172.21.15.84:0/516531077 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ec19d140 con 0x7f67ec102200 2024-05-22T23:04:37.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.659+0000 7f67eb7fe640 1 --2- 172.21.15.84:0/516531077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67ec107850 0x7f67ec19ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:04:37.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.659+0000 7f67eaffd640 1 --2- 172.21.15.84:0/516531077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67ec101840 0x7f67ec1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:37.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.659+0000 7f67ea7fc640 1 --2- 172.21.15.84:0/516531077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67ec102200 0x7f67ec1986e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f67d400e9f0 tx=0x7f67d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:37.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.659+0000 7f67e3fff640 1 -- 172.21.15.84:0/516531077 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d400cd80 con 0x7f67ec102200 2024-05-22T23:04:37.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.659+0000 7f67f16d2640 1 -- 172.21.15.84:0/516531077 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67ec19d430 con 0x7f67ec102200 2024-05-22T23:04:37.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.659+0000 7f67e3fff640 1 -- 172.21.15.84:0/516531077 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67d4010910 con 0x7f67ec102200 2024-05-22T23:04:37.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.659+0000 7f67f16d2640 1 -- 172.21.15.84:0/516531077 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67ec1a4cc0 con 0x7f67ec102200 2024-05-22T23:04:37.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.659+0000 7f67e3fff640 1 -- 172.21.15.84:0/516531077 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d40188f0 con 0x7f67ec102200 2024-05-22T23:04:37.668 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:37 smithi204 ceph-mon[36211]: pgmap v86: 97 pgs: 97 active+clean; 129 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.2 MiB/s wr, 372 op/s 2024-05-22T23:04:37.668 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:37 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1646442153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:04:37.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.662+0000 7f67e3fff640 1 -- 172.21.15.84:0/516531077 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f67d4010ac0 con 0x7f67ec102200 2024-05-22T23:04:37.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.662+0000 7f67e3fff640 1 --2- 172.21.15.84:0/516531077 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f67d80a2d00 0x7f67d80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.662+0000 7f67eaffd640 1 --2- 172.21.15.84:0/516531077 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f67d80a2d00 0x7f67d80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.662+0000 7f67e3fff640 1 -- 172.21.15.84:0/516531077 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f67d40cf870 con 0x7f67ec102200 2024-05-22T23:04:37.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.662+0000 7f67f43de740 1 Processor -- start 2024-05-22T23:04:37.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.662+0000 7f67f43de740 1 -- start start 2024-05-22T23:04:37.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.663+0000 7f67eaffd640 1 --2- 172.21.15.84:0/516531077 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f67d80a2d00 0x7f67d80a51c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f67dc005fd0 tx=0x7f67dc009500 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:37.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.663+0000 7f67f43de740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03efc50 0x55b4a03f0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.663+0000 7f67f43de740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4a03f0570 0x55b4a03fa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.663+0000 7f67f43de740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4a03fac40 0x55b4a03fd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.663+0000 7f67f43de740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b4a0353a40 con 0x55b4a03efc50 2024-05-22T23:04:37.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.663+0000 7f67f43de740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b4a02f2440 con 0x55b4a03f0570 2024-05-22T23:04:37.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.663+0000 7f67f43de740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b4a034fa00 con 0x55b4a03fac40 2024-05-22T23:04:37.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.663+0000 7f67eaffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03efc50 0x55b4a03f0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.663+0000 7f67eaffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03efc50 0x55b4a03f0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37042/0 (socket says 172.21.15.84:37042) 2024-05-22T23:04:37.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.663+0000 7f67eaffd640 1 -- 172.21.15.84:0/4121738757 learned_addr learned my addr 172.21.15.84:0/4121738757 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:37.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.663+0000 7f67ea7fc640 1 --2- 172.21.15.84:0/4121738757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4a03f0570 0x55b4a03fa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.663+0000 7f67eb7fe640 1 --2- 172.21.15.84:0/4121738757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4a03fac40 0x55b4a03fd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.663+0000 7f67eaffd640 1 -- 172.21.15.84:0/4121738757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4a03fac40 msgr2=0x55b4a03fd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:37.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.663+0000 7f67eaffd640 1 --2- 172.21.15.84:0/4121738757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4a03fac40 0x55b4a03fd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.663+0000 7f67eaffd640 1 -- 172.21.15.84:0/4121738757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4a03f0570 msgr2=0x55b4a03fa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:37.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.663+0000 7f67eaffd640 1 --2- 172.21.15.84:0/4121738757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4a03f0570 0x55b4a03fa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.663+0000 7f67eaffd640 1 -- 172.21.15.84:0/4121738757 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4a02fc2d0 con 0x55b4a03efc50 2024-05-22T23:04:37.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.663+0000 7f67eb7fe640 1 --2- 172.21.15.84:0/4121738757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4a03fac40 0x55b4a03fd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:04:37.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.663+0000 7f67eaffd640 1 --2- 172.21.15.84:0/4121738757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03efc50 0x55b4a03f0030 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f67dc012fd0 tx=0x7f67dc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e0eb546b709c05 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:37.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.664+0000 7f67e17fa640 1 -- 172.21.15.84:0/4121738757 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc00fce0 con 0x55b4a03efc50 2024-05-22T23:04:37.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.664+0000 7f67e17fa640 1 -- 172.21.15.84:0/4121738757 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67dc00fe80 con 0x55b4a03efc50 2024-05-22T23:04:37.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.664+0000 7f67e17fa640 1 -- 172.21.15.84:0/4121738757 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc01ca60 con 0x55b4a03efc50 2024-05-22T23:04:37.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.664+0000 7f67f43de740 1 -- 172.21.15.84:0/4121738757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03efc50 msgr2=0x55b4a03f0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:37.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.665+0000 7f67f43de740 1 --2- 172.21.15.84:0/4121738757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03efc50 0x55b4a03f0030 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f67dc012fd0 tx=0x7f67dc014000 comp rx=0 tx=0).stop 2024-05-22T23:04:37.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.665+0000 7f67f43de740 1 -- 172.21.15.84:0/4121738757 shutdown_connections 2024-05-22T23:04:37.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.665+0000 7f67f43de740 1 --2- 172.21.15.84:0/4121738757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4a03fac40 0x55b4a03fd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.665+0000 7f67f43de740 1 --2- 172.21.15.84:0/4121738757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4a03f0570 0x55b4a03fa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.665+0000 7f67f43de740 1 --2- 172.21.15.84:0/4121738757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03efc50 0x55b4a03f0030 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.665+0000 7f67f43de740 1 -- 172.21.15.84:0/4121738757 >> 172.21.15.84:0/4121738757 conn(0x55b4a03ebf70 msgr2=0x55b4a03ec350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:37.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.665+0000 7f67f43de740 1 -- 172.21.15.84:0/4121738757 shutdown_connections 2024-05-22T23:04:37.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.665+0000 7f67f43de740 1 -- 172.21.15.84:0/4121738757 wait complete. 2024-05-22T23:04:37.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.665+0000 7f67f43de740 1 -- start start 2024-05-22T23:04:37.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.665+0000 7f67f43de740 1 Processor -- start 2024-05-22T23:04:37.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.665+0000 7f67f43de740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4a03efc50 0x55b4a03f7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.665+0000 7f67f43de740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4a03f0570 0x55b4a03f8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.665+0000 7f67f43de740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03fac40 0x55b4a03f8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.665+0000 7f67f43de740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b4a03fda30 con 0x55b4a03fac40 2024-05-22T23:04:37.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.665+0000 7f67f43de740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b4a02dd5f0 con 0x55b4a03f0570 2024-05-22T23:04:37.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.665+0000 7f67f43de740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b4a03fdbb0 con 0x55b4a03efc50 2024-05-22T23:04:37.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.665+0000 7f67f43de740 10 client.0 _ll_register_callbacks cb 0x55b4a02d8cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:04:37.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.665+0000 7f67eb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03fac40 0x55b4a03f8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.665+0000 7f67eaffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4a03efc50 0x55b4a03f7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.665+0000 7f67eb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03fac40 0x55b4a03f8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37044/0 (socket says 172.21.15.84:37044) 2024-05-22T23:04:37.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.665+0000 7f67eb7fe640 1 -- 172.21.15.84:0/3520628425 learned_addr learned my addr 172.21.15.84:0/3520628425 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:37.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.665+0000 7f67ea7fc640 1 --2- 172.21.15.84:0/3520628425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4a03f0570 0x55b4a03f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.666+0000 7f67eb7fe640 1 -- 172.21.15.84:0/3520628425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4a03efc50 msgr2=0x55b4a03f7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:37.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.666+0000 7f67eb7fe640 1 --2- 172.21.15.84:0/3520628425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4a03efc50 0x55b4a03f7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.666+0000 7f67eb7fe640 1 -- 172.21.15.84:0/3520628425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4a03f0570 msgr2=0x55b4a03f8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:37.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.666+0000 7f67eb7fe640 1 --2- 172.21.15.84:0/3520628425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4a03f0570 0x55b4a03f8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.666+0000 7f67eb7fe640 1 -- 172.21.15.84:0/3520628425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4a012f7e0 con 0x55b4a03fac40 2024-05-22T23:04:37.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.666+0000 7f67ea7fc640 1 --2- 172.21.15.84:0/3520628425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4a03f0570 0x55b4a03f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:37.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.666+0000 7f67eb7fe640 1 --2- 172.21.15.84:0/3520628425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03fac40 0x55b4a03f8660 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f67e4031550 tx=0x7f67e4031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5b002d1966ca5f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:37.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.666+0000 7f67d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:04:37.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.666+0000 7f67d27fc640 1 -- 172.21.15.84:0/3520628425 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4035070 con 0x55b4a03fac40 2024-05-22T23:04:37.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.666+0000 7f67d27fc640 1 -- 172.21.15.84:0/3520628425 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f67e4004240 con 0x55b4a03fac40 2024-05-22T23:04:37.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.666+0000 7f67f43de740 1 -- 172.21.15.84:0/3520628425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b4a0367460 con 0x55b4a03fac40 2024-05-22T23:04:37.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.666+0000 7f67d27fc640 1 -- 172.21.15.84:0/3520628425 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f67e4004440 con 0x55b4a03fac40 2024-05-22T23:04:37.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.666+0000 7f67f43de740 10 client.15912 fetch_fsmap learned FSMap version 18 2024-05-22T23:04:37.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.666+0000 7f67f43de740 1 -- 172.21.15.84:0/3520628425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55b4a018ff70 con 0x55b4a03fac40 2024-05-22T23:04:37.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.667+0000 7f67d27fc640 1 -- 172.21.15.84:0/3520628425 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b4a0367460 con 0x55b4a03fac40 2024-05-22T23:04:37.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.667+0000 7f67d27fc640 1 -- 172.21.15.84:0/3520628425 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f67e4004b00 con 0x55b4a03fac40 2024-05-22T23:04:37.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.667+0000 7f67f43de740 10 client.15912 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:04:37.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.667+0000 7f67f43de740 10 client.15912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:04:37.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.667+0000 7f67f43de740 20 client.15912 populate_metadata read hostname 'smithi084' 2024-05-22T23:04:37.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.667+0000 7f67f43de740 1 --2- 172.21.15.84:0/3520628425 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b4a040af10 0x55b4a040d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.667+0000 7f67f43de740 4 client.15912 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:04:37.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.667+0000 7f67f43de740 1 -- 172.21.15.84:0/3520628425 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4a0361170 con 0x55b4a040af10 2024-05-22T23:04:37.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.667+0000 7f67eaffd640 1 --2- 172.21.15.84:0/3520628425 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b4a040af10 0x55b4a040d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.667+0000 7f67d27fc640 10 client.15912 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:37.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.667+0000 7f67eaffd640 1 --2- 172.21.15.84:0/3520628425 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b4a040af10 0x55b4a040d300 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74e69bcf4c65f8a3 server_cookie=52095aba8024cfc in_seq=0 out_seq=0 2024-05-22T23:04:37.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.669+0000 7f67d27fc640 1 -- 172.21.15.84:0/3520628425 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b4a0361170 con 0x55b4a040af10 2024-05-22T23:04:37.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.669+0000 7f67d27fc640 10 client.15912 handle_command_reply: tid=0 2024-05-22T23:04:37.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.669+0000 7f67f43de740 1 client.15912 shutdown 2024-05-22T23:04:37.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.669+0000 7f67f43de740 1 -- 172.21.15.84:0/3520628425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03fac40 msgr2=0x55b4a03f8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:37.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.669+0000 7f67f43de740 1 --2- 172.21.15.84:0/3520628425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03fac40 0x55b4a03f8660 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f67e4031550 tx=0x7f67e4031860 comp rx=0 tx=0).stop 2024-05-22T23:04:37.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.669+0000 7f67f43de740 1 -- 172.21.15.84:0/3520628425 shutdown_connections 2024-05-22T23:04:37.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.669+0000 7f67f43de740 1 --2- 172.21.15.84:0/3520628425 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b4a040af10 0x55b4a040d300 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.669+0000 7f67f43de740 1 --2- 172.21.15.84:0/3520628425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03fac40 0x55b4a03f8660 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.669+0000 7f67d27fc640 0 client.15912 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:37.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.669+0000 7f67f43de740 1 --2- 172.21.15.84:0/3520628425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4a03f0570 0x55b4a03f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.669+0000 7f67f43de740 1 --2- 172.21.15.84:0/3520628425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4a03efc50 0x55b4a03f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.669+0000 7f67f43de740 1 -- 172.21.15.84:0/3520628425 >> 172.21.15.84:0/3520628425 conn(0x55b4a03ebf70 msgr2=0x55b4a03eb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:37.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.669+0000 7f67f43de740 1 -- 172.21.15.84:0/3520628425 shutdown_connections 2024-05-22T23:04:37.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.669+0000 7f67f43de740 1 -- 172.21.15.84:0/3520628425 wait complete. 2024-05-22T23:04:37.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.669+0000 7f67f43de740 20 client.15912 trim_cache size 0 max 16384 2024-05-22T23:04:37.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.688+0000 7f67f43de740 1 Processor -- start 2024-05-22T23:04:37.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.688+0000 7f67f43de740 1 -- start start 2024-05-22T23:04:37.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.689+0000 7f67f43de740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4a03efc50 0x55b4a04068f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.689+0000 7f67f43de740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4a03f0570 0x55b4a0412bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.689+0000 7f67f43de740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03fac40 0x55b4a04130f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.689+0000 7f67f43de740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1ab6d0 con 0x55b4a03fac40 2024-05-22T23:04:37.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.689+0000 7f67f43de740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1adf50 con 0x55b4a03f0570 2024-05-22T23:04:37.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.689+0000 7f67f43de740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1abd40 con 0x55b4a03efc50 2024-05-22T23:04:37.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.689+0000 7f67eb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03fac40 0x55b4a04130f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.689+0000 7f67eb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03fac40 0x55b4a04130f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37054/0 (socket says 172.21.15.84:37054) 2024-05-22T23:04:37.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.689+0000 7f67eaffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4a03efc50 0x55b4a04068f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.689+0000 7f67eb7fe640 1 -- 172.21.15.84:0/1814761449 learned_addr learned my addr 172.21.15.84:0/1814761449 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:37.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.689+0000 7f67ea7fc640 1 --2- 172.21.15.84:0/1814761449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4a03f0570 0x55b4a0412bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.689+0000 7f67eb7fe640 1 -- 172.21.15.84:0/1814761449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4a03efc50 msgr2=0x55b4a04068f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:37.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.689+0000 7f67eb7fe640 1 --2- 172.21.15.84:0/1814761449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4a03efc50 0x55b4a04068f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.689+0000 7f67eb7fe640 1 -- 172.21.15.84:0/1814761449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4a03f0570 msgr2=0x55b4a0412bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:37.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.689+0000 7f67eb7fe640 1 --2- 172.21.15.84:0/1814761449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4a03f0570 0x55b4a0412bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.689+0000 7f67eb7fe640 1 -- 172.21.15.84:0/1814761449 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4a040d670 con 0x55b4a03fac40 2024-05-22T23:04:37.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.689+0000 7f67eaffd640 1 --2- 172.21.15.84:0/1814761449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4a03efc50 0x55b4a04068f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:37.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.689+0000 7f67ea7fc640 1 --2- 172.21.15.84:0/1814761449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4a03f0570 0x55b4a0412bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:37.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.689+0000 7f67eb7fe640 1 --2- 172.21.15.84:0/1814761449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03fac40 0x55b4a04130f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f67e402a940 tx=0x7f67e40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7df3fe2443cdd4e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:37.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.689+0000 7f67e0ff9640 1 -- 172.21.15.84:0/1814761449 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4035070 con 0x55b4a03fac40 2024-05-22T23:04:37.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.690+0000 7f67e0ff9640 1 -- 172.21.15.84:0/1814761449 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67e4004ba0 con 0x55b4a03fac40 2024-05-22T23:04:37.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.690+0000 7f67e0ff9640 1 -- 172.21.15.84:0/1814761449 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4003b10 con 0x55b4a03fac40 2024-05-22T23:04:37.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.690+0000 7f67f43de740 1 -- 172.21.15.84:0/1814761449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03fac40 msgr2=0x55b4a04130f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:37.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.690+0000 7f67f43de740 1 --2- 172.21.15.84:0/1814761449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03fac40 0x55b4a04130f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f67e402a940 tx=0x7f67e40045e0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.690+0000 7f67f43de740 1 -- 172.21.15.84:0/1814761449 shutdown_connections 2024-05-22T23:04:37.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.690+0000 7f67f43de740 1 --2- 172.21.15.84:0/1814761449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03fac40 0x55b4a04130f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.690+0000 7f67f43de740 1 --2- 172.21.15.84:0/1814761449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4a03f0570 0x55b4a0412bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.690+0000 7f67f43de740 1 --2- 172.21.15.84:0/1814761449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4a03efc50 0x55b4a04068f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.690+0000 7f67f43de740 1 -- 172.21.15.84:0/1814761449 >> 172.21.15.84:0/1814761449 conn(0x55b4a03ebf70 msgr2=0x55b4a03f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:37.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.690+0000 7f67f43de740 1 -- 172.21.15.84:0/1814761449 shutdown_connections 2024-05-22T23:04:37.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.690+0000 7f67f43de740 1 -- 172.21.15.84:0/1814761449 wait complete. 2024-05-22T23:04:37.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.691+0000 7f67f43de740 1 -- start start 2024-05-22T23:04:37.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.691+0000 7f67f43de740 1 Processor -- start 2024-05-22T23:04:37.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.691+0000 7f67f43de740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03efc50 0x55b4a03f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.691+0000 7f67f43de740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4a03f0570 0x55b4a03f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.691+0000 7f67f43de740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4a03fac40 0x55b4a03f5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.691+0000 7f67f43de740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1ac410 con 0x55b4a03efc50 2024-05-22T23:04:37.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.691+0000 7f67f43de740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1ad880 con 0x55b4a03fac40 2024-05-22T23:04:37.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.691+0000 7f67f43de740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1acae0 con 0x55b4a03f0570 2024-05-22T23:04:37.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.691+0000 7f67f43de740 10 client.0 _ll_register_callbacks cb 0x7f67d4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:04:37.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.691+0000 7f67eaffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03efc50 0x55b4a03f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.691+0000 7f67eb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4a03fac40 0x55b4a03f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.691+0000 7f67eaffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03efc50 0x55b4a03f4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37058/0 (socket says 172.21.15.84:37058) 2024-05-22T23:04:37.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.691+0000 7f67ea7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4a03f0570 0x55b4a03f50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.691+0000 7f67eaffd640 1 -- 172.21.15.84:0/240420234 learned_addr learned my addr 172.21.15.84:0/240420234 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:37.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.691+0000 7f67eb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4a03fac40 0x55b4a03f5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:43686/0 (socket says 172.21.15.84:43686) 2024-05-22T23:04:37.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.691+0000 7f67eb7fe640 1 -- 172.21.15.84:0/240420234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4a03f0570 msgr2=0x55b4a03f50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:37.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.691+0000 7f67eb7fe640 1 --2- 172.21.15.84:0/240420234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4a03f0570 0x55b4a03f50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.691+0000 7f67eb7fe640 1 -- 172.21.15.84:0/240420234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03efc50 msgr2=0x55b4a03f4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:37.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.691+0000 7f67eb7fe640 1 --2- 172.21.15.84:0/240420234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03efc50 0x55b4a03f4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.691+0000 7f67eb7fe640 1 -- 172.21.15.84:0/240420234 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4a0027170 con 0x55b4a03fac40 2024-05-22T23:04:37.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.691+0000 7f67eaffd640 1 --2- 172.21.15.84:0/240420234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03efc50 0x55b4a03f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:37.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.692+0000 7f67eb7fe640 1 --2- 172.21.15.84:0/240420234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4a03fac40 0x55b4a03f5620 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f67e4006010 tx=0x7f67e40036f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70750a0465f771c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:37.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.692+0000 7f67d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:04:37.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.692+0000 7f67d27fc640 1 -- 172.21.15.84:0/240420234 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4035040 con 0x55b4a03fac40 2024-05-22T23:04:37.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.692+0000 7f67f43de740 1 -- 172.21.15.84:0/240420234 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67e4004ba0 con 0x55b4a03fac40 2024-05-22T23:04:37.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.692+0000 7f67d27fc640 1 -- 172.21.15.84:0/240420234 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f67e4005320 con 0x55b4a03fac40 2024-05-22T23:04:37.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.692+0000 7f67d27fc640 1 -- 172.21.15.84:0/240420234 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f67e4067050 con 0x55b4a03fac40 2024-05-22T23:04:37.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.692+0000 7f67f43de740 10 client.25493 fetch_fsmap learned FSMap version 18 2024-05-22T23:04:37.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.692+0000 7f67f43de740 1 -- 172.21.15.84:0/240420234 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55b4a031c930 con 0x55b4a03fac40 2024-05-22T23:04:37.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.692+0000 7f67d27fc640 1 -- 172.21.15.84:0/240420234 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67e4004ba0 con 0x55b4a03fac40 2024-05-22T23:04:37.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.692+0000 7f67d27fc640 1 -- 172.21.15.84:0/240420234 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f67e4060060 con 0x55b4a03fac40 2024-05-22T23:04:37.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.692+0000 7f67f43de740 10 client.25493 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:04:37.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.692+0000 7f67f43de740 10 client.25493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:04:37.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.692+0000 7f67f43de740 20 client.25493 populate_metadata read hostname 'smithi084' 2024-05-22T23:04:37.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.692+0000 7f67f43de740 1 --2- 172.21.15.84:0/240420234 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b4a0471ca0 0x55b4a0474090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.692+0000 7f67f43de740 4 client.25493 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-22T23:04:37.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.692+0000 7f67f43de740 1 -- 172.21.15.84:0/240420234 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b4a035b4c0 con 0x55b4a0471ca0 2024-05-22T23:04:37.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.693+0000 7f67eaffd640 1 --2- 172.21.15.84:0/240420234 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b4a0471ca0 0x55b4a0474090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.693+0000 7f67eaffd640 1 --2- 172.21.15.84:0/240420234 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b4a0471ca0 0x55b4a0474090 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98521f518dc629d8 server_cookie=7f023a8bda9c5afe in_seq=0 out_seq=0 2024-05-22T23:04:37.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.693+0000 7f67d27fc640 10 client.25493 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:37.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.694+0000 7f67d27fc640 1 -- 172.21.15.84:0/240420234 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b4a035b4c0 con 0x55b4a0471ca0 2024-05-22T23:04:37.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.694+0000 7f67d27fc640 10 client.25493 handle_command_reply: tid=0 2024-05-22T23:04:37.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.694+0000 7f67f43de740 1 client.25493 shutdown 2024-05-22T23:04:37.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.695+0000 7f67f43de740 1 -- 172.21.15.84:0/240420234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4a03fac40 msgr2=0x55b4a03f5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:37.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.695+0000 7f67f43de740 1 --2- 172.21.15.84:0/240420234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4a03fac40 0x55b4a03f5620 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f67e4006010 tx=0x7f67e40036f0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.695+0000 7f67f43de740 1 -- 172.21.15.84:0/240420234 shutdown_connections 2024-05-22T23:04:37.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.695+0000 7f67f43de740 1 --2- 172.21.15.84:0/240420234 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b4a0471ca0 0x55b4a0474090 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.695+0000 7f67f43de740 1 --2- 172.21.15.84:0/240420234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4a03fac40 0x55b4a03f5620 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.695+0000 7f67f43de740 1 --2- 172.21.15.84:0/240420234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b4a03f0570 0x55b4a03f50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.695+0000 7f67f43de740 1 --2- 172.21.15.84:0/240420234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4a03efc50 0x55b4a03f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.695+0000 7f67d27fc640 0 client.25493 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:37.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.695+0000 7f67f43de740 1 -- 172.21.15.84:0/240420234 >> 172.21.15.84:0/240420234 conn(0x55b4a03ebf70 msgr2=0x55b4a03f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:37.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.695+0000 7f67f43de740 1 -- 172.21.15.84:0/240420234 shutdown_connections 2024-05-22T23:04:37.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.695+0000 7f67f43de740 1 -- 172.21.15.84:0/240420234 wait complete. 2024-05-22T23:04:37.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.695+0000 7f67f43de740 20 client.25493 trim_cache size 0 max 16384 2024-05-22T23:04:37.706 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:04:37.706 INFO:teuthology.orchestra.run.smithi084.stdout: "return_code": 0, 2024-05-22T23:04:37.706 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_tag": "1cd92003-d315-4466-b084-65f1b44d39b1", 2024-05-22T23:04:37.706 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": "asynchronous" 2024-05-22T23:04:37.706 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:04:37.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.695+0000 7f67f16d2640 1 -- 172.21.15.84:0/516531077 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f67d80a2d00 msgr2=0x7f67d80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:37.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.695+0000 7f67f16d2640 1 --2- 172.21.15.84:0/516531077 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f67d80a2d00 0x7f67d80a51c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f67dc005fd0 tx=0x7f67dc009500 comp rx=0 tx=0).stop 2024-05-22T23:04:37.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.695+0000 7f67f16d2640 1 -- 172.21.15.84:0/516531077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67ec102200 msgr2=0x7f67ec1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:37.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.695+0000 7f67f16d2640 1 --2- 172.21.15.84:0/516531077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67ec102200 0x7f67ec1986e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f67d400e9f0 tx=0x7f67d400eeb0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.695+0000 7f67f16d2640 1 -- 172.21.15.84:0/516531077 shutdown_connections 2024-05-22T23:04:37.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.695+0000 7f67f16d2640 1 --2- 172.21.15.84:0/516531077 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f67d80a2d00 0x7f67d80a51c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.695+0000 7f67f16d2640 1 --2- 172.21.15.84:0/516531077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67ec107850 0x7f67ec19ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.695+0000 7f67f16d2640 1 --2- 172.21.15.84:0/516531077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67ec102200 0x7f67ec1986e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.695+0000 7f67f16d2640 1 --2- 172.21.15.84:0/516531077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67ec101840 0x7f67ec1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.695+0000 7f67f16d2640 1 -- 172.21.15.84:0/516531077 >> 172.21.15.84:0/516531077 conn(0x7f67ec078cc0 msgr2=0x7f67ec0ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:37.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.695+0000 7f67f16d2640 1 -- 172.21.15.84:0/516531077 shutdown_connections 2024-05-22T23:04:37.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.695+0000 7f67f16d2640 1 -- 172.21.15.84:0/516531077 wait complete. 2024-05-22T23:04:37.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:04:37.710 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:04:37.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:37 smithi096 ceph-mon[36392]: pgmap v86: 97 pgs: 97 active+clean; 129 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.2 MiB/s wr, 372 op/s 2024-05-22T23:04:37.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:37 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1646442153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:04:37.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.814+0000 7f7ca49a9640 1 -- 172.21.15.84:0/692625786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca0100740 msgr2=0x7f7ca0100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:37.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.814+0000 7f7ca49a9640 1 --2- 172.21.15.84:0/692625786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca0100740 0x7f7ca0100bc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7c88009930 tx=0x7f7c8802a420 comp rx=0 tx=0).stop 2024-05-22T23:04:37.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.814+0000 7f7ca49a9640 1 -- 172.21.15.84:0/692625786 shutdown_connections 2024-05-22T23:04:37.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.814+0000 7f7ca49a9640 1 --2- 172.21.15.84:0/692625786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ca0101100 0x7f7ca010d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.814+0000 7f7ca49a9640 1 --2- 172.21.15.84:0/692625786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca0100740 0x7f7ca0100bc0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.814+0000 7f7ca49a9640 1 --2- 172.21.15.84:0/692625786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ca0106750 0x7f7ca0106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.814+0000 7f7ca49a9640 1 -- 172.21.15.84:0/692625786 >> 172.21.15.84:0/692625786 conn(0x7f7ca0077870 msgr2=0x7f7ca0077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:37.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.814+0000 7f7ca49a9640 1 -- 172.21.15.84:0/692625786 shutdown_connections 2024-05-22T23:04:37.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.814+0000 7f7ca49a9640 1 -- 172.21.15.84:0/692625786 wait complete. 2024-05-22T23:04:37.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.815+0000 7f7ca49a9640 1 Processor -- start 2024-05-22T23:04:37.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.815+0000 7f7ca49a9640 1 -- start start 2024-05-22T23:04:37.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.815+0000 7f7ca49a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca0100740 0x7f7ca019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.815+0000 7f7ca49a9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ca0101100 0x7f7ca019f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.815+0000 7f7ca49a9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ca0106750 0x7f7ca019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.815+0000 7f7ca49a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca010fa50 con 0x7f7ca0100740 2024-05-22T23:04:37.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.815+0000 7f7ca49a9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca010f8d0 con 0x7f7ca0101100 2024-05-22T23:04:37.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.815+0000 7f7ca49a9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca010fbd0 con 0x7f7ca0106750 2024-05-22T23:04:37.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.816+0000 7f7c9e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca0100740 0x7f7ca019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.816+0000 7f7c9e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca0100740 0x7f7ca019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37070/0 (socket says 172.21.15.84:37070) 2024-05-22T23:04:37.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.816+0000 7f7c9e59c640 1 -- 172.21.15.84:0/3463125818 learned_addr learned my addr 172.21.15.84:0/3463125818 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:37.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.816+0000 7f7c9dd9b640 1 --2- 172.21.15.84:0/3463125818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ca0101100 0x7f7ca019f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.816+0000 7f7c9ed9d640 1 --2- 172.21.15.84:0/3463125818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ca0106750 0x7f7ca019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.816+0000 7f7c9e59c640 1 -- 172.21.15.84:0/3463125818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ca0106750 msgr2=0x7f7ca019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:37.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.816+0000 7f7c9e59c640 1 --2- 172.21.15.84:0/3463125818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ca0106750 0x7f7ca019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.816+0000 7f7c9e59c640 1 -- 172.21.15.84:0/3463125818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ca0101100 msgr2=0x7f7ca019f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:37.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.816+0000 7f7c9e59c640 1 --2- 172.21.15.84:0/3463125818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ca0101100 0x7f7ca019f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.816+0000 7f7c9e59c640 1 -- 172.21.15.84:0/3463125818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca019ad10 con 0x7f7ca0100740 2024-05-22T23:04:37.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.816+0000 7f7c9dd9b640 1 --2- 172.21.15.84:0/3463125818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ca0101100 0x7f7ca019f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:37.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.816+0000 7f7c9e59c640 1 --2- 172.21.15.84:0/3463125818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca0100740 0x7f7ca019ef90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000ebc0 tx=0x7f7c9000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:37.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.816+0000 7f7c977fe640 1 -- 172.21.15.84:0/3463125818 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90010630 con 0x7f7ca0100740 2024-05-22T23:04:37.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.816+0000 7f7ca49a9640 1 -- 172.21.15.84:0/3463125818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ca019b000 con 0x7f7ca0100740 2024-05-22T23:04:37.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.816+0000 7f7c977fe640 1 -- 172.21.15.84:0/3463125818 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c90010cb0 con 0x7f7ca0100740 2024-05-22T23:04:37.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.816+0000 7f7c977fe640 1 -- 172.21.15.84:0/3463125818 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90018bd0 con 0x7f7ca0100740 2024-05-22T23:04:37.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.816+0000 7f7ca49a9640 1 -- 172.21.15.84:0/3463125818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ca01a6f00 con 0x7f7ca0100740 2024-05-22T23:04:37.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.819+0000 7f7c977fe640 1 -- 172.21.15.84:0/3463125818 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7c900107d0 con 0x7f7ca0100740 2024-05-22T23:04:37.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.819+0000 7f7ca7799740 1 Processor -- start 2024-05-22T23:04:37.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.819+0000 7f7ca7799740 1 -- start start 2024-05-22T23:04:37.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.819+0000 7f7ca7799740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d81a4c44a0 0x55d81a4c27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.819+0000 7f7ca7799740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d81a4c2db0 0x55d81a4c3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.819+0000 7f7ca7799740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4cdb30 0x55d81a4cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.819+0000 7f7ca7799740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d81a426db0 con 0x55d81a4cdb30 2024-05-22T23:04:37.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.819+0000 7f7ca7799740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d81a3c5360 con 0x55d81a4c44a0 2024-05-22T23:04:37.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.819+0000 7f7ca7799740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d81a422d70 con 0x55d81a4c2db0 2024-05-22T23:04:37.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.819+0000 7f7c9ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d81a4c44a0 0x55d81a4c27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.819+0000 7f7c9dd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4cdb30 0x55d81a4cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.819+0000 7f7c9ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d81a4c44a0 0x55d81a4c27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:43712/0 (socket says 172.21.15.84:43712) 2024-05-22T23:04:37.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.819+0000 7f7c9ed9d640 1 -- 172.21.15.84:0/4262889358 learned_addr learned my addr 172.21.15.84:0/4262889358 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:37.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.819+0000 7f7c9dd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4cdb30 0x55d81a4cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37074/0 (socket says 172.21.15.84:37074) 2024-05-22T23:04:37.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.819+0000 7f7c9e59c640 1 --2- 172.21.15.84:0/4262889358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d81a4c2db0 0x55d81a4c3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.819+0000 7f7c977fe640 1 --2- 172.21.15.84:0/3463125818 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f7c8c0a2d00 0x7f7c8c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.820+0000 7f7c9ed9d640 1 -- 172.21.15.84:0/4262889358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d81a4c2db0 msgr2=0x55d81a4c3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:37.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.820+0000 7f7c9ed9d640 1 --2- 172.21.15.84:0/4262889358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d81a4c2db0 0x55d81a4c3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.820+0000 7f7c9ed9d640 1 -- 172.21.15.84:0/4262889358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4cdb30 msgr2=0x55d81a4cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:37.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.820+0000 7f7c9ed9d640 1 --2- 172.21.15.84:0/4262889358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4cdb30 0x55d81a4cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.820+0000 7f7c9ed9d640 1 -- 172.21.15.84:0/4262889358 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d81a3e8a60 con 0x55d81a4c44a0 2024-05-22T23:04:37.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.820+0000 7f7c9ed9d640 1 --2- 172.21.15.84:0/3463125818 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f7c8c0a2d00 0x7f7c8c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.820+0000 7f7c977fe640 1 -- 172.21.15.84:0/3463125818 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f7c900cfa60 con 0x7f7ca0100740 2024-05-22T23:04:37.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.820+0000 7f7c9ed9d640 1 --2- 172.21.15.84:0/4262889358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d81a4c44a0 0x55d81a4c27e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800bdb0 tx=0x7f7c9800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29164803c74b48f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:37.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.820+0000 7f7c94ff9640 1 -- 172.21.15.84:0/4262889358 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98012070 con 0x55d81a4c44a0 2024-05-22T23:04:37.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.820+0000 7f7c9ed9d640 1 --2- 172.21.15.84:0/3463125818 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f7c8c0a2d00 0x7f7c8c0a51c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7c98020000 tx=0x7f7c9801d000 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:37.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.820+0000 7f7c94ff9640 1 -- 172.21.15.84:0/4262889358 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c98012210 con 0x55d81a4c44a0 2024-05-22T23:04:37.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.820+0000 7f7c94ff9640 1 -- 172.21.15.84:0/4262889358 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9801b760 con 0x55d81a4c44a0 2024-05-22T23:04:37.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.821+0000 7f7ca7799740 1 -- 172.21.15.84:0/4262889358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d81a4c44a0 msgr2=0x55d81a4c27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:37.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.821+0000 7f7ca7799740 1 --2- 172.21.15.84:0/4262889358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d81a4c44a0 0x55d81a4c27e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800bdb0 tx=0x7f7c9800f730 comp rx=0 tx=0).stop 2024-05-22T23:04:37.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.821+0000 7f7ca7799740 1 -- 172.21.15.84:0/4262889358 shutdown_connections 2024-05-22T23:04:37.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.821+0000 7f7ca7799740 1 --2- 172.21.15.84:0/4262889358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4cdb30 0x55d81a4cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.821+0000 7f7ca7799740 1 --2- 172.21.15.84:0/4262889358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d81a4c2db0 0x55d81a4c3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.821+0000 7f7ca7799740 1 --2- 172.21.15.84:0/4262889358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d81a4c44a0 0x55d81a4c27e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.821+0000 7f7ca7799740 1 -- 172.21.15.84:0/4262889358 >> 172.21.15.84:0/4262889358 conn(0x55d81a48b8e0 msgr2=0x55d81a4bf310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:37.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.821+0000 7f7ca7799740 1 -- 172.21.15.84:0/4262889358 shutdown_connections 2024-05-22T23:04:37.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.821+0000 7f7ca7799740 1 -- 172.21.15.84:0/4262889358 wait complete. 2024-05-22T23:04:37.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.821+0000 7f7ca7799740 1 -- start start 2024-05-22T23:04:37.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.821+0000 7f7ca7799740 1 Processor -- start 2024-05-22T23:04:37.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.822+0000 7f7ca7799740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d81a4c2db0 0x55d81a4caa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.822+0000 7f7ca7799740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4c44a0 0x55d81a4caf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.822+0000 7f7ca7799740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d81a4cdb30 0x55d81a4cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.822+0000 7f7ca7799740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d81a4d09c0 con 0x55d81a4c44a0 2024-05-22T23:04:37.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.822+0000 7f7ca7799740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d81a3e10f0 con 0x55d81a4c2db0 2024-05-22T23:04:37.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.822+0000 7f7ca7799740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d81a4d0b40 con 0x55d81a4cdb30 2024-05-22T23:04:37.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.822+0000 7f7ca7799740 10 client.0 _ll_register_callbacks cb 0x55d81a3e5f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:04:37.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.822+0000 7f7c9dd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4c44a0 0x55d81a4caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.822+0000 7f7c9dd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4c44a0 0x55d81a4caf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37076/0 (socket says 172.21.15.84:37076) 2024-05-22T23:04:37.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.822+0000 7f7c9e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d81a4c2db0 0x55d81a4caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.822+0000 7f7c9dd9b640 1 -- 172.21.15.84:0/3364145616 learned_addr learned my addr 172.21.15.84:0/3364145616 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:37.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.822+0000 7f7c9ed9d640 1 --2- 172.21.15.84:0/3364145616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d81a4cdb30 0x55d81a4cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.822+0000 7f7c9dd9b640 1 -- 172.21.15.84:0/3364145616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d81a4cdb30 msgr2=0x55d81a4cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:37.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.822+0000 7f7c9dd9b640 1 --2- 172.21.15.84:0/3364145616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d81a4cdb30 0x55d81a4cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.822+0000 7f7c9dd9b640 1 -- 172.21.15.84:0/3364145616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d81a4c2db0 msgr2=0x55d81a4caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:37.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.822+0000 7f7c9dd9b640 1 --2- 172.21.15.84:0/3364145616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d81a4c2db0 0x55d81a4caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.822+0000 7f7c9dd9b640 1 -- 172.21.15.84:0/3364145616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d81a2047d0 con 0x55d81a4c44a0 2024-05-22T23:04:37.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.822+0000 7f7c9e59c640 1 --2- 172.21.15.84:0/3364145616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d81a4c2db0 0x55d81a4caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:37.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.822+0000 7f7c9ed9d640 1 --2- 172.21.15.84:0/3364145616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d81a4cdb30 0x55d81a4cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:04:37.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.822+0000 7f7c9dd9b640 1 --2- 172.21.15.84:0/3364145616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4c44a0 0x55d81a4caf90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7c88009930 tx=0x7f7c88031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6449a630ad8b5e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:37.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.822+0000 7f7c85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:04:37.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.822+0000 7f7c85ffb640 1 -- 172.21.15.84:0/3364145616 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c88035040 con 0x55d81a4c44a0 2024-05-22T23:04:37.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.822+0000 7f7c85ffb640 1 -- 172.21.15.84:0/3364145616 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f7c88004250 con 0x55d81a4c44a0 2024-05-22T23:04:37.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.822+0000 7f7ca7799740 1 -- 172.21.15.84:0/3364145616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c98012210 con 0x55d81a4c44a0 2024-05-22T23:04:37.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.823+0000 7f7c85ffb640 1 -- 172.21.15.84:0/3364145616 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f7c88004450 con 0x55d81a4c44a0 2024-05-22T23:04:37.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.823+0000 7f7ca7799740 10 client.15930 fetch_fsmap learned FSMap version 18 2024-05-22T23:04:37.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.823+0000 7f7ca7799740 1 -- 172.21.15.84:0/3364145616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55d81a13a3f0 con 0x55d81a4c44a0 2024-05-22T23:04:37.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.823+0000 7f7c85ffb640 1 -- 172.21.15.84:0/3364145616 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c98012210 con 0x55d81a4c44a0 2024-05-22T23:04:37.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.823+0000 7f7c85ffb640 1 -- 172.21.15.84:0/3364145616 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f7c8803d6f0 con 0x55d81a4c44a0 2024-05-22T23:04:37.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.823+0000 7f7ca7799740 10 client.15930 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:04:37.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.823+0000 7f7ca7799740 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:04:37.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.823+0000 7f7ca7799740 20 client.15930 populate_metadata read hostname 'smithi084' 2024-05-22T23:04:37.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.823+0000 7f7ca7799740 1 --2- 172.21.15.84:0/3364145616 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d81a4ddfb0 0x55d81a4e0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.823+0000 7f7ca7799740 4 client.15930 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:04:37.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.823+0000 7f7ca7799740 1 -- 172.21.15.84:0/3364145616 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d81a4344e0 con 0x55d81a4ddfb0 2024-05-22T23:04:37.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.823+0000 7f7c9e59c640 1 --2- 172.21.15.84:0/3364145616 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d81a4ddfb0 0x55d81a4e0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.823+0000 7f7c85ffb640 10 client.15930 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:37.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.823+0000 7f7c9e59c640 1 --2- 172.21.15.84:0/3364145616 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d81a4ddfb0 0x55d81a4e0400 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9f7c41ceafd4983 server_cookie=a9ee9de4b9c84d0c in_seq=0 out_seq=0 2024-05-22T23:04:37.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.824+0000 7f7c85ffb640 1 -- 172.21.15.84:0/3364145616 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d81a4344e0 con 0x55d81a4ddfb0 2024-05-22T23:04:37.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.824+0000 7f7c85ffb640 10 client.15930 handle_command_reply: tid=0 2024-05-22T23:04:37.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.824+0000 7f7ca7799740 1 client.15930 shutdown 2024-05-22T23:04:37.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.824+0000 7f7ca7799740 1 -- 172.21.15.84:0/3364145616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4c44a0 msgr2=0x55d81a4caf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:37.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.824+0000 7f7ca7799740 1 --2- 172.21.15.84:0/3364145616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4c44a0 0x55d81a4caf90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7c88009930 tx=0x7f7c88031920 comp rx=0 tx=0).stop 2024-05-22T23:04:37.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.824+0000 7f7ca7799740 1 -- 172.21.15.84:0/3364145616 shutdown_connections 2024-05-22T23:04:37.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.824+0000 7f7ca7799740 1 --2- 172.21.15.84:0/3364145616 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d81a4ddfb0 0x55d81a4e0400 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.824+0000 7f7ca7799740 1 --2- 172.21.15.84:0/3364145616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d81a4cdb30 0x55d81a4cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.824+0000 7f7c85ffb640 0 client.15930 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:37.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.824+0000 7f7ca7799740 1 --2- 172.21.15.84:0/3364145616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4c44a0 0x55d81a4caf90 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.824+0000 7f7ca7799740 1 --2- 172.21.15.84:0/3364145616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d81a4c2db0 0x55d81a4caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.824+0000 7f7ca7799740 1 -- 172.21.15.84:0/3364145616 >> 172.21.15.84:0/3364145616 conn(0x55d81a48b8e0 msgr2=0x55d81a4c5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:37.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.824+0000 7f7ca7799740 1 -- 172.21.15.84:0/3364145616 shutdown_connections 2024-05-22T23:04:37.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.824+0000 7f7ca7799740 1 -- 172.21.15.84:0/3364145616 wait complete. 2024-05-22T23:04:37.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.824+0000 7f7ca7799740 20 client.15930 trim_cache size 0 max 16384 2024-05-22T23:04:37.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.843+0000 7f7ca7799740 1 Processor -- start 2024-05-22T23:04:37.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.843+0000 7f7ca7799740 1 -- start start 2024-05-22T23:04:37.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.843+0000 7f7ca7799740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d81a4c2db0 0x55d81a4d98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.843+0000 7f7ca7799740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d81a4c44a0 0x55d81a4c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.843+0000 7f7ca7799740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4cdb30 0x55d81a4e5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.843+0000 7f7ca7799740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca01ad850 con 0x55d81a4cdb30 2024-05-22T23:04:37.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.843+0000 7f7ca7799740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca01b00d0 con 0x55d81a4c2db0 2024-05-22T23:04:37.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.843+0000 7f7ca7799740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca01adec0 con 0x55d81a4c44a0 2024-05-22T23:04:37.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.843+0000 7f7c9ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4cdb30 0x55d81a4e5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.843+0000 7f7c9ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4cdb30 0x55d81a4e5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37090/0 (socket says 172.21.15.84:37090) 2024-05-22T23:04:37.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.843+0000 7f7c9ed9d640 1 -- 172.21.15.84:0/1240124313 learned_addr learned my addr 172.21.15.84:0/1240124313 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:37.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.843+0000 7f7c9e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d81a4c2db0 0x55d81a4d98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.843+0000 7f7c9dd9b640 1 --2- 172.21.15.84:0/1240124313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d81a4c44a0 0x55d81a4c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.843+0000 7f7c9e59c640 1 -- 172.21.15.84:0/1240124313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d81a4c44a0 msgr2=0x55d81a4c4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:37.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.843+0000 7f7c9e59c640 1 --2- 172.21.15.84:0/1240124313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d81a4c44a0 0x55d81a4c4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.843+0000 7f7c9e59c640 1 -- 172.21.15.84:0/1240124313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4cdb30 msgr2=0x55d81a4e5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:37.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.843+0000 7f7c9e59c640 1 --2- 172.21.15.84:0/1240124313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4cdb30 0x55d81a4e5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.843+0000 7f7c9e59c640 1 -- 172.21.15.84:0/1240124313 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d81a0fa0f0 con 0x55d81a4c2db0 2024-05-22T23:04:37.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.843+0000 7f7c9ed9d640 1 --2- 172.21.15.84:0/1240124313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4cdb30 0x55d81a4e5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:04:37.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.843+0000 7f7c9dd9b640 1 --2- 172.21.15.84:0/1240124313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d81a4c44a0 0x55d81a4c4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:04:37.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.844+0000 7f7c9e59c640 1 --2- 172.21.15.84:0/1240124313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d81a4c2db0 0x55d81a4d98c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7c900cd1b0 tx=0x7f7c900bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98a0565baa8c72b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:37.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.844+0000 7f7c87fff640 1 -- 172.21.15.84:0/1240124313 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90010e60 con 0x55d81a4c2db0 2024-05-22T23:04:37.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.844+0000 7f7c87fff640 1 -- 172.21.15.84:0/1240124313 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c900a0330 con 0x55d81a4c2db0 2024-05-22T23:04:37.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.844+0000 7f7c87fff640 1 -- 172.21.15.84:0/1240124313 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c900985f0 con 0x55d81a4c2db0 2024-05-22T23:04:37.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.844+0000 7f7ca7799740 1 -- 172.21.15.84:0/1240124313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d81a4c2db0 msgr2=0x55d81a4d98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:37.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.844+0000 7f7ca7799740 1 --2- 172.21.15.84:0/1240124313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d81a4c2db0 0x55d81a4d98c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7c900cd1b0 tx=0x7f7c900bc040 comp rx=0 tx=0).stop 2024-05-22T23:04:37.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.845+0000 7f7ca7799740 1 -- 172.21.15.84:0/1240124313 shutdown_connections 2024-05-22T23:04:37.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.845+0000 7f7ca7799740 1 --2- 172.21.15.84:0/1240124313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4cdb30 0x55d81a4e5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.845+0000 7f7ca7799740 1 --2- 172.21.15.84:0/1240124313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d81a4c44a0 0x55d81a4c4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.845+0000 7f7ca7799740 1 --2- 172.21.15.84:0/1240124313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d81a4c2db0 0x55d81a4d98c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.845+0000 7f7ca7799740 1 -- 172.21.15.84:0/1240124313 >> 172.21.15.84:0/1240124313 conn(0x55d81a48b8e0 msgr2=0x55d81a4c48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:37.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.845+0000 7f7ca7799740 1 -- 172.21.15.84:0/1240124313 shutdown_connections 2024-05-22T23:04:37.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.845+0000 7f7ca7799740 1 -- 172.21.15.84:0/1240124313 wait complete. 2024-05-22T23:04:37.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.845+0000 7f7ca7799740 1 -- start start 2024-05-22T23:04:37.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.845+0000 7f7ca7799740 1 Processor -- start 2024-05-22T23:04:37.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.845+0000 7f7ca7799740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d81a4c2db0 0x55d81a4d6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.845+0000 7f7ca7799740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d81a4c44a0 0x55d81a4c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.845+0000 7f7ca7799740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4cdb30 0x55d81a4c7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.845+0000 7f7ca7799740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca01ae590 con 0x55d81a4cdb30 2024-05-22T23:04:37.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.845+0000 7f7ca7799740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca01afa00 con 0x55d81a4c44a0 2024-05-22T23:04:37.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.845+0000 7f7ca7799740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca01aec60 con 0x55d81a4c2db0 2024-05-22T23:04:37.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.845+0000 7f7ca7799740 10 client.0 _ll_register_callbacks cb 0x7f7c88009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:04:37.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.845+0000 7f7c9ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4cdb30 0x55d81a4c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.845+0000 7f7c9ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4cdb30 0x55d81a4c7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37106/0 (socket says 172.21.15.84:37106) 2024-05-22T23:04:37.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.845+0000 7f7c9dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d81a4c44a0 0x55d81a4c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.845+0000 7f7c9ed9d640 1 -- 172.21.15.84:0/710073867 learned_addr learned my addr 172.21.15.84:0/710073867 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:04:37.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.845+0000 7f7c9e59c640 1 --2- 172.21.15.84:0/710073867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d81a4c2db0 0x55d81a4d6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.846+0000 7f7c9ed9d640 1 -- 172.21.15.84:0/710073867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d81a4c2db0 msgr2=0x55d81a4d6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:37.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.846+0000 7f7c9ed9d640 1 --2- 172.21.15.84:0/710073867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d81a4c2db0 0x55d81a4d6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.846+0000 7f7c9ed9d640 1 -- 172.21.15.84:0/710073867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d81a4c44a0 msgr2=0x55d81a4c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:04:37.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.846+0000 7f7c9ed9d640 1 --2- 172.21.15.84:0/710073867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d81a4c44a0 0x55d81a4c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.846+0000 7f7c9ed9d640 1 -- 172.21.15.84:0/710073867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d81a3e5fe0 con 0x55d81a4cdb30 2024-05-22T23:04:37.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.846+0000 7f7c9ed9d640 1 --2- 172.21.15.84:0/710073867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4cdb30 0x55d81a4c7800 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7c9801b650 tx=0x7f7c9801b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca2370a090fcefa server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:04:37.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.846+0000 7f7c85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:04:37.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.846+0000 7f7c85ffb640 1 -- 172.21.15.84:0/710073867 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98037060 con 0x55d81a4cdb30 2024-05-22T23:04:37.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.846+0000 7f7ca7799740 1 -- 172.21.15.84:0/710073867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c900a0330 con 0x55d81a4cdb30 2024-05-22T23:04:37.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.846+0000 7f7c85ffb640 1 -- 172.21.15.84:0/710073867 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f7c9801bbe0 con 0x55d81a4cdb30 2024-05-22T23:04:37.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.846+0000 7f7c85ffb640 1 -- 172.21.15.84:0/710073867 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f7c9800b2c0 con 0x55d81a4cdb30 2024-05-22T23:04:37.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.846+0000 7f7ca7799740 10 client.15936 fetch_fsmap learned FSMap version 18 2024-05-22T23:04:37.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.846+0000 7f7ca7799740 1 -- 172.21.15.84:0/710073867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55d81a3b71c0 con 0x55d81a4cdb30 2024-05-22T23:04:37.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.846+0000 7f7c85ffb640 1 -- 172.21.15.84:0/710073867 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c900a0330 con 0x55d81a4cdb30 2024-05-22T23:04:37.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.846+0000 7f7c85ffb640 1 -- 172.21.15.84:0/710073867 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f7c9801cb60 con 0x55d81a4cdb30 2024-05-22T23:04:37.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.846+0000 7f7ca7799740 10 client.15936 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:04:37.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.846+0000 7f7ca7799740 10 client.15936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:04:37.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.846+0000 7f7ca7799740 20 client.15936 populate_metadata read hostname 'smithi084' 2024-05-22T23:04:37.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.846+0000 7f7ca7799740 1 --2- 172.21.15.84:0/710073867 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d81a5444e0 0x55d81a546900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:04:37.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.846+0000 7f7ca7799740 4 client.15936 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:04:37.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.846+0000 7f7ca7799740 1 -- 172.21.15.84:0/710073867 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d81a42e830 con 0x55d81a5444e0 2024-05-22T23:04:37.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.847+0000 7f7c9e59c640 1 --2- 172.21.15.84:0/710073867 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d81a5444e0 0x55d81a546900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:04:37.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.847+0000 7f7c85ffb640 10 client.15936 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:37.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.847+0000 7f7c9e59c640 1 --2- 172.21.15.84:0/710073867 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d81a5444e0 0x55d81a546900 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60790ce1d496c8e2 server_cookie=eef65f1b25e6160e in_seq=0 out_seq=0 2024-05-22T23:04:37.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7c85ffb640 1 -- 172.21.15.84:0/710073867 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d81a42e830 con 0x55d81a5444e0 2024-05-22T23:04:37.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7c85ffb640 10 client.15936 handle_command_reply: tid=0 2024-05-22T23:04:37.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7ca7799740 1 client.15936 shutdown 2024-05-22T23:04:37.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7ca7799740 1 -- 172.21.15.84:0/710073867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4cdb30 msgr2=0x55d81a4c7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:37.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7ca7799740 1 --2- 172.21.15.84:0/710073867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4cdb30 0x55d81a4c7800 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7c9801b650 tx=0x7f7c9801b680 comp rx=0 tx=0).stop 2024-05-22T23:04:37.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7ca7799740 1 -- 172.21.15.84:0/710073867 shutdown_connections 2024-05-22T23:04:37.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7ca7799740 1 --2- 172.21.15.84:0/710073867 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d81a5444e0 0x55d81a546900 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7ca7799740 1 --2- 172.21.15.84:0/710073867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d81a4cdb30 0x55d81a4c7800 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7c85ffb640 0 client.15936 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:04:37.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7ca7799740 1 --2- 172.21.15.84:0/710073867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d81a4c44a0 0x55d81a4c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7ca7799740 1 --2- 172.21.15.84:0/710073867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d81a4c2db0 0x55d81a4d6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7ca7799740 1 -- 172.21.15.84:0/710073867 >> 172.21.15.84:0/710073867 conn(0x55d81a48b8e0 msgr2=0x55d81a4c5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:37.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7ca7799740 1 -- 172.21.15.84:0/710073867 shutdown_connections 2024-05-22T23:04:37.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7ca7799740 1 -- 172.21.15.84:0/710073867 wait complete. 2024-05-22T23:04:37.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7ca7799740 20 client.15936 trim_cache size 0 max 16384 2024-05-22T23:04:37.860 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:04:37.860 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-22T23:04:37.860 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": { 2024-05-22T23:04:37.860 INFO:teuthology.orchestra.run.smithi084.stdout: "1cd92003-d315-4466-b084-65f1b44d39b1": { 2024-05-22T23:04:37.860 INFO:teuthology.orchestra.run.smithi084.stdout: "path": "/", 2024-05-22T23:04:37.860 INFO:teuthology.orchestra.run.smithi084.stdout: "tag": "1cd92003-d315-4466-b084-65f1b44d39b1", 2024-05-22T23:04:37.860 INFO:teuthology.orchestra.run.smithi084.stdout: "options": "recursive,force" 2024-05-22T23:04:37.860 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:04:37.860 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:04:37.860 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:04:37.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7ca49a9640 1 -- 172.21.15.84:0/3463125818 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f7c8c0a2d00 msgr2=0x7f7c8c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:37.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7ca49a9640 1 --2- 172.21.15.84:0/3463125818 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f7c8c0a2d00 0x7f7c8c0a51c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7c98020000 tx=0x7f7c9801d000 comp rx=0 tx=0).stop 2024-05-22T23:04:37.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7ca49a9640 1 -- 172.21.15.84:0/3463125818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca0100740 msgr2=0x7f7ca019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:04:37.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7ca49a9640 1 --2- 172.21.15.84:0/3463125818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca0100740 0x7f7ca019ef90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000ebc0 tx=0x7f7c9000c6a0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7ca49a9640 1 -- 172.21.15.84:0/3463125818 shutdown_connections 2024-05-22T23:04:37.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7ca49a9640 1 --2- 172.21.15.84:0/3463125818 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f7c8c0a2d00 0x7f7c8c0a51c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7ca49a9640 1 --2- 172.21.15.84:0/3463125818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ca0106750 0x7f7ca019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7ca49a9640 1 --2- 172.21.15.84:0/3463125818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ca0101100 0x7f7ca019f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7ca49a9640 1 --2- 172.21.15.84:0/3463125818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca0100740 0x7f7ca019ef90 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:04:37.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.848+0000 7f7ca49a9640 1 -- 172.21.15.84:0/3463125818 >> 172.21.15.84:0/3463125818 conn(0x7f7ca0077870 msgr2=0x7f7ca01033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:04:37.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.849+0000 7f7ca49a9640 1 -- 172.21.15.84:0/3463125818 shutdown_connections 2024-05-22T23:04:37.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:04:37.849+0000 7f7ca49a9640 1 -- 172.21.15.84:0/3463125818 wait complete. 2024-05-22T23:04:37.863 INFO:tasks.cephfs.filesystem:scrub status for tag:1cd92003-d315-4466-b084-65f1b44d39b1 - {'path': '/', 'tag': '1cd92003-d315-4466-b084-65f1b44d39b1', 'options': 'recursive,force'} 2024-05-22T23:04:38.427 INFO:tasks.workunit.client.0.smithi084.stdout: 1 249938 16.25 MB/sec warmup 84 sec latency 24.579 ms 2024-05-22T23:04:38.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:38 smithi084 ceph-mon[32186]: scrub queued for path: / 2024-05-22T23:04:38.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:38 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:04:38.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:38 smithi084 ceph-mon[32186]: scrub summary: active paths [/] 2024-05-22T23:04:38.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:38 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:04:38.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:38 smithi204 ceph-mon[36211]: scrub queued for path: / 2024-05-22T23:04:38.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:38 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:04:38.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:38 smithi204 ceph-mon[36211]: scrub summary: active paths [/] 2024-05-22T23:04:38.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:38 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:04:38.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:38 smithi096 ceph-mon[36392]: scrub queued for path: / 2024-05-22T23:04:38.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:38 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:04:38.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:38 smithi096 ceph-mon[36392]: scrub summary: active paths [/] 2024-05-22T23:04:38.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:38 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:04:39.427 INFO:tasks.workunit.client.0.smithi084.stdout: 1 252746 16.23 MB/sec warmup 85 sec latency 14.431 ms 2024-05-22T23:04:39.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:39 smithi084 ceph-mon[32186]: pgmap v87: 97 pgs: 97 active+clean; 129 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 353 op/s 2024-05-22T23:04:39.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:39 smithi204 ceph-mon[36211]: pgmap v87: 97 pgs: 97 active+clean; 129 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 353 op/s 2024-05-22T23:04:39.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:39 smithi096 ceph-mon[36392]: pgmap v87: 97 pgs: 97 active+clean; 129 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 353 op/s 2024-05-22T23:04:40.428 INFO:tasks.workunit.client.0.smithi084.stdout: 1 255999 16.25 MB/sec warmup 86 sec latency 8.979 ms 2024-05-22T23:04:41.427 INFO:tasks.workunit.client.0.smithi084.stdout: 1 259266 16.28 MB/sec warmup 87 sec latency 23.388 ms 2024-05-22T23:04:41.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:41 smithi084 ceph-mon[32186]: pgmap v88: 97 pgs: 97 active+clean; 129 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 245 op/s 2024-05-22T23:04:41.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:41 smithi084 ceph-mon[32186]: scrub summary: idle 2024-05-22T23:04:41.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:41 smithi204 ceph-mon[36211]: pgmap v88: 97 pgs: 97 active+clean; 129 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 245 op/s 2024-05-22T23:04:41.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:41 smithi204 ceph-mon[36211]: scrub summary: idle 2024-05-22T23:04:41.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:41 smithi096 ceph-mon[36392]: pgmap v88: 97 pgs: 97 active+clean; 129 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 245 op/s 2024-05-22T23:04:41.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:41 smithi096 ceph-mon[36392]: scrub summary: idle 2024-05-22T23:04:42.428 INFO:tasks.workunit.client.0.smithi084.stdout: 1 261715 16.23 MB/sec warmup 88 sec latency 56.410 ms 2024-05-22T23:04:43.428 INFO:tasks.workunit.client.0.smithi084.stdout: 1 264358 16.20 MB/sec warmup 89 sec latency 14.704 ms 2024-05-22T23:04:43.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:43 smithi084 ceph-mon[32186]: pgmap v89: 97 pgs: 97 active+clean; 135 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.2 MiB/s wr, 373 op/s 2024-05-22T23:04:43.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:43 smithi204 ceph-mon[36211]: pgmap v89: 97 pgs: 97 active+clean; 135 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.2 MiB/s wr, 373 op/s 2024-05-22T23:04:43.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:43 smithi096 ceph-mon[36392]: pgmap v89: 97 pgs: 97 active+clean; 135 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.2 MiB/s wr, 373 op/s 2024-05-22T23:04:44.428 INFO:tasks.workunit.client.0.smithi084.stdout: 1 267281 16.19 MB/sec warmup 90 sec latency 13.868 ms 2024-05-22T23:04:45.428 INFO:tasks.workunit.client.0.smithi084.stdout: 1 270340 16.20 MB/sec warmup 91 sec latency 23.917 ms 2024-05-22T23:04:45.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:45 smithi084 ceph-mon[32186]: pgmap v90: 97 pgs: 97 active+clean; 135 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 250 op/s 2024-05-22T23:04:45.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:45 smithi204 ceph-mon[36211]: pgmap v90: 97 pgs: 97 active+clean; 135 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 250 op/s 2024-05-22T23:04:45.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:45 smithi096 ceph-mon[36392]: pgmap v90: 97 pgs: 97 active+clean; 135 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 250 op/s 2024-05-22T23:04:46.428 INFO:tasks.workunit.client.0.smithi084.stdout: 1 273619 16.23 MB/sec warmup 92 sec latency 11.278 ms 2024-05-22T23:04:47.428 INFO:tasks.workunit.client.0.smithi084.stdout: 1 276338 16.20 MB/sec warmup 93 sec latency 116.571 ms 2024-05-22T23:04:47.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:47 smithi084 ceph-mon[32186]: pgmap v91: 97 pgs: 97 active+clean; 141 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.3 MiB/s wr, 378 op/s 2024-05-22T23:04:47.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:47 smithi204 ceph-mon[36211]: pgmap v91: 97 pgs: 97 active+clean; 141 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.3 MiB/s wr, 378 op/s 2024-05-22T23:04:47.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:47 smithi096 ceph-mon[36392]: pgmap v91: 97 pgs: 97 active+clean; 141 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.3 MiB/s wr, 378 op/s 2024-05-22T23:04:48.428 INFO:tasks.workunit.client.0.smithi084.stdout: 1 279134 16.17 MB/sec warmup 94 sec latency 14.513 ms 2024-05-22T23:04:49.428 INFO:tasks.workunit.client.0.smithi084.stdout: 1 281774 16.18 MB/sec warmup 95 sec latency 21.651 ms 2024-05-22T23:04:49.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:49 smithi084 ceph-mon[32186]: pgmap v92: 97 pgs: 97 active+clean; 141 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 256 op/s 2024-05-22T23:04:49.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:49 smithi204 ceph-mon[36211]: pgmap v92: 97 pgs: 97 active+clean; 141 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 256 op/s 2024-05-22T23:04:49.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:49 smithi096 ceph-mon[36392]: pgmap v92: 97 pgs: 97 active+clean; 141 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 256 op/s 2024-05-22T23:04:50.428 INFO:tasks.workunit.client.0.smithi084.stdout: 1 284687 16.16 MB/sec warmup 96 sec latency 17.057 ms 2024-05-22T23:04:51.428 INFO:tasks.workunit.client.0.smithi084.stdout: 1 287866 16.18 MB/sec warmup 97 sec latency 9.829 ms 2024-05-22T23:04:51.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:51 smithi084 ceph-mon[32186]: pgmap v93: 97 pgs: 97 active+clean; 141 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 256 op/s 2024-05-22T23:04:51.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:51 smithi204 ceph-mon[36211]: pgmap v93: 97 pgs: 97 active+clean; 141 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 256 op/s 2024-05-22T23:04:51.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:51 smithi096 ceph-mon[36392]: pgmap v93: 97 pgs: 97 active+clean; 141 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 256 op/s 2024-05-22T23:04:52.428 INFO:tasks.workunit.client.0.smithi084.stdout: 1 290627 16.17 MB/sec warmup 98 sec latency 109.257 ms 2024-05-22T23:04:53.428 INFO:tasks.workunit.client.0.smithi084.stdout: 1 293298 16.12 MB/sec warmup 99 sec latency 72.085 ms 2024-05-22T23:04:53.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:53 smithi084 ceph-mon[32186]: pgmap v94: 97 pgs: 97 active+clean; 147 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.5 MiB/s wr, 377 op/s 2024-05-22T23:04:53.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:53 smithi204 ceph-mon[36211]: pgmap v94: 97 pgs: 97 active+clean; 147 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.5 MiB/s wr, 377 op/s 2024-05-22T23:04:53.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:53 smithi096 ceph-mon[36392]: pgmap v94: 97 pgs: 97 active+clean; 147 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.5 MiB/s wr, 377 op/s 2024-05-22T23:04:54.428 INFO:tasks.workunit.client.0.smithi084.stdout: 1 295893 16.13 MB/sec warmup 100 sec latency 16.615 ms 2024-05-22T23:04:55.428 INFO:tasks.workunit.client.0.smithi084.stdout: 1 298966 16.12 MB/sec warmup 101 sec latency 15.540 ms 2024-05-22T23:04:55.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:55 smithi084 ceph-mon[32186]: pgmap v95: 97 pgs: 97 active+clean; 147 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 249 op/s 2024-05-22T23:04:55.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:55 smithi204 ceph-mon[36211]: pgmap v95: 97 pgs: 97 active+clean; 147 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 249 op/s 2024-05-22T23:04:55.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:55 smithi096 ceph-mon[36392]: pgmap v95: 97 pgs: 97 active+clean; 147 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 249 op/s 2024-05-22T23:04:56.428 INFO:tasks.workunit.client.0.smithi084.stdout: 1 302221 16.14 MB/sec warmup 102 sec latency 16.719 ms 2024-05-22T23:04:57.428 INFO:tasks.workunit.client.0.smithi084.stdout: 1 305296 16.17 MB/sec warmup 103 sec latency 92.367 ms 2024-05-22T23:04:57.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:57 smithi084 ceph-mon[32186]: pgmap v96: 97 pgs: 97 active+clean; 153 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.6 MiB/s wr, 377 op/s 2024-05-22T23:04:57.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:57 smithi204 ceph-mon[36211]: pgmap v96: 97 pgs: 97 active+clean; 153 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.6 MiB/s wr, 377 op/s 2024-05-22T23:04:57.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:57 smithi096 ceph-mon[36392]: pgmap v96: 97 pgs: 97 active+clean; 153 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.6 MiB/s wr, 377 op/s 2024-05-22T23:04:58.428 INFO:tasks.workunit.client.0.smithi084.stdout: 1 308086 16.14 MB/sec warmup 104 sec latency 28.256 ms 2024-05-22T23:04:58.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:58 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:04:58.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:58 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:04:58.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:58 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:04:59.428 INFO:tasks.workunit.client.0.smithi084.stdout: 1 311038 16.11 MB/sec warmup 105 sec latency 14.990 ms 2024-05-22T23:04:59.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:59 smithi084 ceph-mon[32186]: pgmap v97: 97 pgs: 97 active+clean; 153 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 248 op/s 2024-05-22T23:04:59.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:59 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:04:59.628 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:59 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:04:59.628 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:04:59 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:04:59.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:59 smithi204 ceph-mon[36211]: pgmap v97: 97 pgs: 97 active+clean; 153 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 248 op/s 2024-05-22T23:04:59.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:59 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:04:59.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:59 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:04:59.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:04:59 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:04:59.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:59 smithi096 ceph-mon[36392]: pgmap v97: 97 pgs: 97 active+clean; 153 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 248 op/s 2024-05-22T23:04:59.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:59 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:04:59.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:59 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:04:59.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:04:59 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:05:00.429 INFO:tasks.workunit.client.0.smithi084.stdout: 1 313661 16.12 MB/sec warmup 106 sec latency 14.772 ms 2024-05-22T23:05:01.429 INFO:tasks.workunit.client.0.smithi084.stdout: 1 316070 16.10 MB/sec warmup 107 sec latency 25.877 ms 2024-05-22T23:05:01.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:01 smithi084 ceph-mon[32186]: pgmap v98: 97 pgs: 97 active+clean; 153 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 248 op/s 2024-05-22T23:05:01.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:01 smithi204 ceph-mon[36211]: pgmap v98: 97 pgs: 97 active+clean; 153 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 248 op/s 2024-05-22T23:05:01.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:01 smithi096 ceph-mon[36392]: pgmap v98: 97 pgs: 97 active+clean; 153 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 248 op/s 2024-05-22T23:05:02.429 INFO:tasks.workunit.client.0.smithi084.stdout: 1 318481 16.05 MB/sec warmup 108 sec latency 123.349 ms 2024-05-22T23:05:03.429 INFO:tasks.workunit.client.0.smithi084.stdout: 1 320671 16.03 MB/sec warmup 109 sec latency 47.844 ms 2024-05-22T23:05:03.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:03 smithi084 ceph-mon[32186]: pgmap v99: 97 pgs: 97 active+clean; 159 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.4 MiB/s wr, 392 op/s 2024-05-22T23:05:03.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:03 smithi204 ceph-mon[36211]: pgmap v99: 97 pgs: 97 active+clean; 159 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.4 MiB/s wr, 392 op/s 2024-05-22T23:05:03.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:03 smithi096 ceph-mon[36392]: pgmap v99: 97 pgs: 97 active+clean; 159 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.4 MiB/s wr, 392 op/s 2024-05-22T23:05:04.429 INFO:tasks.workunit.client.0.smithi084.stdout: 1 323575 16.01 MB/sec warmup 110 sec latency 18.752 ms 2024-05-22T23:05:05.429 INFO:tasks.workunit.client.0.smithi084.stdout: 1 326732 16.04 MB/sec warmup 111 sec latency 44.172 ms 2024-05-22T23:05:05.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:05 smithi084 ceph-mon[32186]: pgmap v100: 97 pgs: 97 active+clean; 159 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 270 op/s 2024-05-22T23:05:05.665 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:05 smithi204 ceph-mon[36211]: pgmap v100: 97 pgs: 97 active+clean; 159 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 270 op/s 2024-05-22T23:05:05.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:05 smithi096 ceph-mon[36392]: pgmap v100: 97 pgs: 97 active+clean; 159 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 270 op/s 2024-05-22T23:05:06.429 INFO:tasks.workunit.client.0.smithi084.stdout: 1 329990 16.05 MB/sec warmup 112 sec latency 14.963 ms 2024-05-22T23:05:07.429 INFO:tasks.workunit.client.0.smithi084.stdout: 1 332327 15.99 MB/sec warmup 113 sec latency 90.014 ms 2024-05-22T23:05:07.627 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:07 smithi084 ceph-mon[32186]: pgmap v101: 97 pgs: 97 active+clean; 166 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.5 MiB/s wr, 404 op/s 2024-05-22T23:05:07.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:07 smithi204 ceph-mon[36211]: pgmap v101: 97 pgs: 97 active+clean; 166 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.5 MiB/s wr, 404 op/s 2024-05-22T23:05:07.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:07 smithi096 ceph-mon[36392]: pgmap v101: 97 pgs: 97 active+clean; 166 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.5 MiB/s wr, 404 op/s 2024-05-22T23:05:07.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:05:07.864 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:05:07.972 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.968+0000 7f9ccda9a640 1 -- 172.21.15.84:0/2252474317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cc8100740 msgr2=0x7f9cc8100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:07.972 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.968+0000 7f9ccda9a640 1 --2- 172.21.15.84:0/2252474317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cc8100740 0x7f9cc8100bc0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009930 tx=0x7f9cbc02a420 comp rx=0 tx=0).stop 2024-05-22T23:05:07.972 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.968+0000 7f9ccda9a640 1 -- 172.21.15.84:0/2252474317 shutdown_connections 2024-05-22T23:05:07.972 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.968+0000 7f9ccda9a640 1 --2- 172.21.15.84:0/2252474317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9cc8101100 0x7f9cc810d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:07.972 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.968+0000 7f9ccda9a640 1 --2- 172.21.15.84:0/2252474317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cc8100740 0x7f9cc8100bc0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:07.972 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.968+0000 7f9ccda9a640 1 --2- 172.21.15.84:0/2252474317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cc8106750 0x7f9cc8106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:07.972 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.968+0000 7f9ccda9a640 1 -- 172.21.15.84:0/2252474317 >> 172.21.15.84:0/2252474317 conn(0x7f9cc8077870 msgr2=0x7f9cc8077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:07.972 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.968+0000 7f9ccda9a640 1 -- 172.21.15.84:0/2252474317 shutdown_connections 2024-05-22T23:05:07.973 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.968+0000 7f9ccda9a640 1 -- 172.21.15.84:0/2252474317 wait complete. 2024-05-22T23:05:07.973 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.969+0000 7f9ccda9a640 1 Processor -- start 2024-05-22T23:05:07.973 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.969+0000 7f9ccda9a640 1 -- start start 2024-05-22T23:05:07.975 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.969+0000 7f9ccda9a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cc8100740 0x7f9cc8198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:07.975 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.969+0000 7f9ccda9a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cc8101100 0x7f9cc81986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:07.975 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.969+0000 7f9ccda9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9cc8106750 0x7f9cc819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:07.975 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.969+0000 7f9ccda9a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc8110b20 con 0x7f9cc8100740 2024-05-22T23:05:07.975 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.969+0000 7f9ccda9a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc81109a0 con 0x7f9cc8106750 2024-05-22T23:05:07.975 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.969+0000 7f9ccda9a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc8110ca0 con 0x7f9cc8101100 2024-05-22T23:05:07.975 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.969+0000 7f9cc77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cc8100740 0x7f9cc8198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:07.976 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.969+0000 7f9cc77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cc8100740 0x7f9cc8198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57856/0 (socket says 172.21.15.84:57856) 2024-05-22T23:05:07.976 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.970+0000 7f9cc77fe640 1 -- 172.21.15.84:0/1500538266 learned_addr learned my addr 172.21.15.84:0/1500538266 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:07.976 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.970+0000 7f9cc7fff640 1 --2- 172.21.15.84:0/1500538266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9cc8106750 0x7f9cc819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:07.976 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.970+0000 7f9cc6ffd640 1 --2- 172.21.15.84:0/1500538266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cc8101100 0x7f9cc81986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:07.976 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.970+0000 7f9cc77fe640 1 -- 172.21.15.84:0/1500538266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cc8101100 msgr2=0x7f9cc81986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:07.976 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.970+0000 7f9cc77fe640 1 --2- 172.21.15.84:0/1500538266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cc8101100 0x7f9cc81986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:07.976 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.970+0000 7f9cc77fe640 1 -- 172.21.15.84:0/1500538266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9cc8106750 msgr2=0x7f9cc819ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:07.976 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.970+0000 7f9cc77fe640 1 --2- 172.21.15.84:0/1500538266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9cc8106750 0x7f9cc819ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:07.976 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.970+0000 7f9cc77fe640 1 -- 172.21.15.84:0/1500538266 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc819d160 con 0x7f9cc8100740 2024-05-22T23:05:07.976 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.970+0000 7f9cc6ffd640 1 --2- 172.21.15.84:0/1500538266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cc8101100 0x7f9cc81986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:05:07.976 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.970+0000 7f9cc77fe640 1 --2- 172.21.15.84:0/1500538266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cc8100740 0x7f9cc8198170 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800e9f0 tx=0x7f9cb800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:07.976 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.970+0000 7f9cc4ff9640 1 -- 172.21.15.84:0/1500538266 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb800cd80 con 0x7f9cc8100740 2024-05-22T23:05:07.977 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.970+0000 7f9ccda9a640 1 -- 172.21.15.84:0/1500538266 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc819d450 con 0x7f9cc8100740 2024-05-22T23:05:07.977 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.970+0000 7f9cc4ff9640 1 -- 172.21.15.84:0/1500538266 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cb8010910 con 0x7f9cc8100740 2024-05-22T23:05:07.977 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.970+0000 7f9cc4ff9640 1 -- 172.21.15.84:0/1500538266 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb80188f0 con 0x7f9cc8100740 2024-05-22T23:05:07.977 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.970+0000 7f9ccda9a640 1 -- 172.21.15.84:0/1500538266 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc81a4ce0 con 0x7f9cc8100740 2024-05-22T23:05:07.977 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.972+0000 7f9cc4ff9640 1 -- 172.21.15.84:0/1500538266 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9cb8010ac0 con 0x7f9cc8100740 2024-05-22T23:05:07.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.973+0000 7f9cd07a1740 1 Processor -- start 2024-05-22T23:05:07.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.973+0000 7f9cd07a1740 1 -- start start 2024-05-22T23:05:07.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.973+0000 7f9cd07a1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca207eb4a0 0x55ca207e97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:07.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.973+0000 7f9cd07a1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca207e9db0 0x55ca207ea230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:07.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.973+0000 7f9cd07a1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207f8f40 0x55ca207fb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:07.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.973+0000 7f9cd07a1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ca2074ddb0 con 0x55ca207f8f40 2024-05-22T23:05:07.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.973+0000 7f9cd07a1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ca206ec360 con 0x55ca207e9db0 2024-05-22T23:05:07.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.973+0000 7f9cd07a1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ca20749d70 con 0x55ca207eb4a0 2024-05-22T23:05:07.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.973+0000 7f9cc4ff9640 1 --2- 172.21.15.84:0/1500538266 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f9cb00a2d00 0x7f9cb00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:07.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.973+0000 7f9cc6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207f8f40 0x55ca207fb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:07.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.973+0000 7f9cc6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207f8f40 0x55ca207fb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57866/0 (socket says 172.21.15.84:57866) 2024-05-22T23:05:07.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.973+0000 7f9cc6ffd640 1 -- 172.21.15.84:0/2730964249 learned_addr learned my addr 172.21.15.84:0/2730964249 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:07.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.973+0000 7f9cc77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca207e9db0 0x55ca207ea230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:07.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.973+0000 7f9cc7fff640 1 --2- 172.21.15.84:0/2730964249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca207eb4a0 0x55ca207e97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:07.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.973+0000 7f9cc7fff640 1 --2- 172.21.15.84:0/1500538266 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f9cb00a2d00 0x7f9cb00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:07.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.973+0000 7f9cc4ff9640 1 -- 172.21.15.84:0/1500538266 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f9cb80cf860 con 0x7f9cc8100740 2024-05-22T23:05:07.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.973+0000 7f9cc6ffd640 1 -- 172.21.15.84:0/2730964249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca207eb4a0 msgr2=0x55ca207e97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:07.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.973+0000 7f9cc6ffd640 1 --2- 172.21.15.84:0/2730964249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca207eb4a0 0x55ca207e97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:07.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.974+0000 7f9cc6ffd640 1 -- 172.21.15.84:0/2730964249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca207e9db0 msgr2=0x55ca207ea230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:07.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.974+0000 7f9cc6ffd640 1 --2- 172.21.15.84:0/2730964249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca207e9db0 0x55ca207ea230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:07.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.974+0000 7f9cc6ffd640 1 -- 172.21.15.84:0/2730964249 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca2070fa60 con 0x55ca207f8f40 2024-05-22T23:05:07.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.974+0000 7f9cc7fff640 1 --2- 172.21.15.84:0/1500538266 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f9cb00a2d00 0x7f9cb00a51c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0005c40 tx=0x7f9cc0009500 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:07.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.974+0000 7f9cc6ffd640 1 --2- 172.21.15.84:0/2730964249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207f8f40 0x55ca207fb330 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009930 tx=0x7f9cbc005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf325b594987cc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:07.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.974+0000 7f9cadffb640 1 -- 172.21.15.84:0/2730964249 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc03f070 con 0x55ca207f8f40 2024-05-22T23:05:07.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.974+0000 7f9cadffb640 1 -- 172.21.15.84:0/2730964249 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cbc004030 con 0x55ca207f8f40 2024-05-22T23:05:07.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.974+0000 7f9cadffb640 1 -- 172.21.15.84:0/2730964249 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc035070 con 0x55ca207f8f40 2024-05-22T23:05:07.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.975+0000 7f9cd07a1740 1 -- 172.21.15.84:0/2730964249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207f8f40 msgr2=0x55ca207fb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:07.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.975+0000 7f9cd07a1740 1 --2- 172.21.15.84:0/2730964249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207f8f40 0x55ca207fb330 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009930 tx=0x7f9cbc005e60 comp rx=0 tx=0).stop 2024-05-22T23:05:07.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.975+0000 7f9cd07a1740 1 -- 172.21.15.84:0/2730964249 shutdown_connections 2024-05-22T23:05:07.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.975+0000 7f9cd07a1740 1 --2- 172.21.15.84:0/2730964249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207f8f40 0x55ca207fb330 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:07.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.975+0000 7f9cd07a1740 1 --2- 172.21.15.84:0/2730964249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca207e9db0 0x55ca207ea230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:07.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.975+0000 7f9cd07a1740 1 --2- 172.21.15.84:0/2730964249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca207eb4a0 0x55ca207e97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:07.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.975+0000 7f9cd07a1740 1 -- 172.21.15.84:0/2730964249 >> 172.21.15.84:0/2730964249 conn(0x55ca207b28e0 msgr2=0x55ca207e6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:07.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.975+0000 7f9cd07a1740 1 -- 172.21.15.84:0/2730964249 shutdown_connections 2024-05-22T23:05:07.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.975+0000 7f9cd07a1740 1 -- 172.21.15.84:0/2730964249 wait complete. 2024-05-22T23:05:07.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.975+0000 7f9cd07a1740 1 -- start start 2024-05-22T23:05:07.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.975+0000 7f9cd07a1740 1 Processor -- start 2024-05-22T23:05:07.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9cd07a1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca207e9db0 0x55ca207f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:07.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9cd07a1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca207eb4a0 0x55ca207f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:07.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9cd07a1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207f8f40 0x55ca207f6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:07.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9cd07a1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ca207fbd70 con 0x55ca207f8f40 2024-05-22T23:05:07.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9cd07a1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ca207080f0 con 0x55ca207eb4a0 2024-05-22T23:05:07.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9cd07a1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ca207fbef0 con 0x55ca207e9db0 2024-05-22T23:05:07.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9cd07a1740 10 client.0 _ll_register_callbacks cb 0x55ca2070cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:05:07.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9cc7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207f8f40 0x55ca207f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:07.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9cc7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207f8f40 0x55ca207f6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57882/0 (socket says 172.21.15.84:57882) 2024-05-22T23:05:07.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9cc6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca207eb4a0 0x55ca207f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:07.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9cc7fff640 1 -- 172.21.15.84:0/3678013994 learned_addr learned my addr 172.21.15.84:0/3678013994 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:07.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9cc77fe640 1 --2- 172.21.15.84:0/3678013994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca207e9db0 0x55ca207f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:07.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9cc6ffd640 1 -- 172.21.15.84:0/3678013994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca207e9db0 msgr2=0x55ca207f5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:07.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9cc6ffd640 1 --2- 172.21.15.84:0/3678013994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca207e9db0 0x55ca207f5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:07.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9cc6ffd640 1 -- 172.21.15.84:0/3678013994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207f8f40 msgr2=0x55ca207f6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:07.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9cc6ffd640 1 --2- 172.21.15.84:0/3678013994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207f8f40 0x55ca207f6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:07.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9cc6ffd640 1 -- 172.21.15.84:0/3678013994 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca2052b7d0 con 0x55ca207eb4a0 2024-05-22T23:05:07.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9cc7fff640 1 --2- 172.21.15.84:0/3678013994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207f8f40 0x55ca207f6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:05:07.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9cc6ffd640 1 --2- 172.21.15.84:0/3678013994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca207eb4a0 0x55ca207f62c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009900 tx=0x7f9cbc0050e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb11cb162e140515 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:07.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9ca6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:05:07.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9ca6ffd640 1 -- 172.21.15.84:0/3678013994 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc035070 con 0x55ca207eb4a0 2024-05-22T23:05:07.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9cd07a1740 1 -- 172.21.15.84:0/3678013994 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cbc004030 con 0x55ca207eb4a0 2024-05-22T23:05:07.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9ca6ffd640 1 -- 172.21.15.84:0/3678013994 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f9cbc0315b0 con 0x55ca207eb4a0 2024-05-22T23:05:07.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9ca6ffd640 1 -- 172.21.15.84:0/3678013994 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f9cbc055050 con 0x55ca207eb4a0 2024-05-22T23:05:07.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9cd07a1740 10 client.25523 fetch_fsmap learned FSMap version 18 2024-05-22T23:05:07.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.976+0000 7f9cd07a1740 1 -- 172.21.15.84:0/3678013994 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55ca204613f0 con 0x55ca207eb4a0 2024-05-22T23:05:07.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.977+0000 7f9ca6ffd640 1 -- 172.21.15.84:0/3678013994 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cbc004030 con 0x55ca207eb4a0 2024-05-22T23:05:07.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.977+0000 7f9ca6ffd640 1 -- 172.21.15.84:0/3678013994 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f9cbc0035f0 con 0x55ca207eb4a0 2024-05-22T23:05:07.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.977+0000 7f9cd07a1740 10 client.25523 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:05:07.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.977+0000 7f9cd07a1740 10 client.25523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:05:07.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.977+0000 7f9cd07a1740 20 client.25523 populate_metadata read hostname 'smithi084' 2024-05-22T23:05:07.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.977+0000 7f9cd07a1740 1 --2- 172.21.15.84:0/3678013994 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55ca20809130 0x55ca2080b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:07.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.977+0000 7f9cd07a1740 4 client.25523 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:05:07.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.977+0000 7f9cd07a1740 1 -- 172.21.15.84:0/3678013994 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca2075b4e0 con 0x55ca20809130 2024-05-22T23:05:07.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.977+0000 7f9cc77fe640 1 --2- 172.21.15.84:0/3678013994 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55ca20809130 0x55ca2080b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:07.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.977+0000 7f9cc77fe640 1 --2- 172.21.15.84:0/3678013994 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55ca20809130 0x55ca2080b520 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1b56e6595f3937f server_cookie=4da4bd703e90689d in_seq=0 out_seq=0 2024-05-22T23:05:07.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.977+0000 7f9ca6ffd640 10 client.25523 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:07.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.978+0000 7f9ca6ffd640 1 -- 172.21.15.84:0/3678013994 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ca2075b4e0 con 0x55ca20809130 2024-05-22T23:05:07.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.978+0000 7f9ca6ffd640 10 client.25523 handle_command_reply: tid=0 2024-05-22T23:05:07.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.978+0000 7f9cd07a1740 1 client.25523 shutdown 2024-05-22T23:05:07.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.978+0000 7f9cd07a1740 1 -- 172.21.15.84:0/3678013994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca207eb4a0 msgr2=0x55ca207f62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:07.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.978+0000 7f9cd07a1740 1 --2- 172.21.15.84:0/3678013994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca207eb4a0 0x55ca207f62c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009900 tx=0x7f9cbc0050e0 comp rx=0 tx=0).stop 2024-05-22T23:05:07.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.978+0000 7f9cd07a1740 1 -- 172.21.15.84:0/3678013994 shutdown_connections 2024-05-22T23:05:07.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.978+0000 7f9cd07a1740 1 --2- 172.21.15.84:0/3678013994 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55ca20809130 0x55ca2080b520 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:07.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.978+0000 7f9cd07a1740 1 --2- 172.21.15.84:0/3678013994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207f8f40 0x55ca207f6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:07.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.978+0000 7f9cd07a1740 1 --2- 172.21.15.84:0/3678013994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca207eb4a0 0x55ca207f62c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:07.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.978+0000 7f9ca6ffd640 0 client.25523 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:07.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.978+0000 7f9cd07a1740 1 --2- 172.21.15.84:0/3678013994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca207e9db0 0x55ca207f5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:07.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.978+0000 7f9cd07a1740 1 -- 172.21.15.84:0/3678013994 >> 172.21.15.84:0/3678013994 conn(0x55ca207b28e0 msgr2=0x55ca207ec110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:07.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.978+0000 7f9cd07a1740 1 -- 172.21.15.84:0/3678013994 shutdown_connections 2024-05-22T23:05:07.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.978+0000 7f9cd07a1740 1 -- 172.21.15.84:0/3678013994 wait complete. 2024-05-22T23:05:07.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.978+0000 7f9cd07a1740 20 client.25523 trim_cache size 0 max 16384 2024-05-22T23:05:08.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.997+0000 7f9cd07a1740 1 Processor -- start 2024-05-22T23:05:08.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.997+0000 7f9cd07a1740 1 -- start start 2024-05-22T23:05:08.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.997+0000 7f9cd07a1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca207e9db0 0x55ca20804ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.997+0000 7f9cd07a1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207eb4a0 0x55ca20810b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.997+0000 7f9cd07a1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca207f8f40 0x55ca20811060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.997+0000 7f9cd07a1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc81a9480 con 0x55ca207eb4a0 2024-05-22T23:05:08.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.997+0000 7f9cd07a1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc81abd00 con 0x55ca207e9db0 2024-05-22T23:05:08.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.997+0000 7f9cd07a1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc81a9af0 con 0x55ca207f8f40 2024-05-22T23:05:08.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.997+0000 7f9cc6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207eb4a0 0x55ca20810b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.997+0000 7f9cc77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca207e9db0 0x55ca20804ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.997+0000 7f9cc6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207eb4a0 0x55ca20810b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57888/0 (socket says 172.21.15.84:57888) 2024-05-22T23:05:08.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.997+0000 7f9cc6ffd640 1 -- 172.21.15.84:0/3570304233 learned_addr learned my addr 172.21.15.84:0/3570304233 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:08.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.997+0000 7f9cc7fff640 1 --2- 172.21.15.84:0/3570304233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca207f8f40 0x55ca20811060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.997+0000 7f9cc6ffd640 1 -- 172.21.15.84:0/3570304233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca207f8f40 msgr2=0x55ca20811060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:08.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.997+0000 7f9cc6ffd640 1 --2- 172.21.15.84:0/3570304233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca207f8f40 0x55ca20811060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.997+0000 7f9cc6ffd640 1 -- 172.21.15.84:0/3570304233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca207e9db0 msgr2=0x55ca20804ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:08.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.997+0000 7f9cc6ffd640 1 --2- 172.21.15.84:0/3570304233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca207e9db0 0x55ca20804ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.997+0000 7f9cc6ffd640 1 -- 172.21.15.84:0/3570304233 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca204210f0 con 0x55ca207eb4a0 2024-05-22T23:05:08.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.997+0000 7f9cc77fe640 1 --2- 172.21.15.84:0/3570304233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca207e9db0 0x55ca20804ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:05:08.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.997+0000 7f9cc6ffd640 1 --2- 172.21.15.84:0/3570304233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207eb4a0 0x55ca20810b20 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc02a940 tx=0x7f9cbc005150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7026b5303a71d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:08.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.997+0000 7f9cad7fa640 1 -- 172.21.15.84:0/3570304233 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc035070 con 0x55ca207eb4a0 2024-05-22T23:05:08.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.998+0000 7f9cad7fa640 1 -- 172.21.15.84:0/3570304233 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cbc04c070 con 0x55ca207eb4a0 2024-05-22T23:05:08.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.998+0000 7f9cad7fa640 1 -- 172.21.15.84:0/3570304233 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc03f040 con 0x55ca207eb4a0 2024-05-22T23:05:08.006 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.998+0000 7f9cd07a1740 1 -- 172.21.15.84:0/3570304233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207eb4a0 msgr2=0x55ca20810b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:08.007 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.998+0000 7f9cd07a1740 1 --2- 172.21.15.84:0/3570304233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207eb4a0 0x55ca20810b20 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc02a940 tx=0x7f9cbc005150 comp rx=0 tx=0).stop 2024-05-22T23:05:08.007 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.998+0000 7f9cd07a1740 1 -- 172.21.15.84:0/3570304233 shutdown_connections 2024-05-22T23:05:08.007 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.998+0000 7f9cd07a1740 1 --2- 172.21.15.84:0/3570304233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca207f8f40 0x55ca20811060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.007 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.998+0000 7f9cd07a1740 1 --2- 172.21.15.84:0/3570304233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207eb4a0 0x55ca20810b20 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.007 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.998+0000 7f9cd07a1740 1 --2- 172.21.15.84:0/3570304233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca207e9db0 0x55ca20804ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.007 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.998+0000 7f9cd07a1740 1 -- 172.21.15.84:0/3570304233 >> 172.21.15.84:0/3570304233 conn(0x55ca207b28e0 msgr2=0x55ca207f75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:08.007 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.998+0000 7f9cd07a1740 1 -- 172.21.15.84:0/3570304233 shutdown_connections 2024-05-22T23:05:08.007 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.998+0000 7f9cd07a1740 1 -- 172.21.15.84:0/3570304233 wait complete. 2024-05-22T23:05:08.007 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.999+0000 7f9cd07a1740 1 -- start start 2024-05-22T23:05:08.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.999+0000 7f9cd07a1740 1 Processor -- start 2024-05-22T23:05:08.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.999+0000 7f9cd07a1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207e9db0 0x55ca20801600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.999+0000 7f9cd07a1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca207eb4a0 0x55ca207fc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.999+0000 7f9cd07a1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca207f8f40 0x55ca207fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.999+0000 7f9cd07a1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc81aa1c0 con 0x55ca207e9db0 2024-05-22T23:05:08.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.999+0000 7f9cd07a1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc81ab630 con 0x55ca207eb4a0 2024-05-22T23:05:08.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.999+0000 7f9cd07a1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc81aa890 con 0x55ca207f8f40 2024-05-22T23:05:08.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.999+0000 7f9cd07a1740 10 client.0 _ll_register_callbacks cb 0x7f9cb8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:05:08.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.999+0000 7f9cc77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207e9db0 0x55ca20801600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.999+0000 7f9cc77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207e9db0 0x55ca20801600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57894/0 (socket says 172.21.15.84:57894) 2024-05-22T23:05:08.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.999+0000 7f9cc6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca207eb4a0 0x55ca207fc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.999+0000 7f9cc77fe640 1 -- 172.21.15.84:0/45137374 learned_addr learned my addr 172.21.15.84:0/45137374 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:08.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.999+0000 7f9cc7fff640 1 --2- 172.21.15.84:0/45137374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca207f8f40 0x55ca207fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.999+0000 7f9cc77fe640 1 -- 172.21.15.84:0/45137374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca207f8f40 msgr2=0x55ca207fcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:08.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.999+0000 7f9cc77fe640 1 --2- 172.21.15.84:0/45137374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca207f8f40 0x55ca207fcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.999+0000 7f9cc77fe640 1 -- 172.21.15.84:0/45137374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca207eb4a0 msgr2=0x55ca207fc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:08.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.999+0000 7f9cc77fe640 1 --2- 172.21.15.84:0/45137374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca207eb4a0 0x55ca207fc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.999+0000 7f9cc77fe640 1 -- 172.21.15.84:0/45137374 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca2070cfe0 con 0x55ca207e9db0 2024-05-22T23:05:08.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:07.999+0000 7f9cc6ffd640 1 --2- 172.21.15.84:0/45137374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca207eb4a0 0x55ca207fc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:05:08.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.000+0000 7f9cc77fe640 1 --2- 172.21.15.84:0/45137374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207e9db0 0x55ca20801600 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80cd010 tx=0x7f9cb80d1d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca542f2990a81f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:08.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.000+0000 7f9ca6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:05:08.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.000+0000 7f9ca6ffd640 1 -- 172.21.15.84:0/45137374 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb80a49f0 con 0x55ca207e9db0 2024-05-22T23:05:08.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.000+0000 7f9ca6ffd640 1 -- 172.21.15.84:0/45137374 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f9cb80cd230 con 0x55ca207e9db0 2024-05-22T23:05:08.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.000+0000 7f9cd07a1740 1 -- 172.21.15.84:0/45137374 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cbc04c070 con 0x55ca207e9db0 2024-05-22T23:05:08.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.000+0000 7f9ca6ffd640 1 -- 172.21.15.84:0/45137374 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f9cb80983f0 con 0x55ca207e9db0 2024-05-22T23:05:08.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.000+0000 7f9cd07a1740 10 client.15954 fetch_fsmap learned FSMap version 18 2024-05-22T23:05:08.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.000+0000 7f9cd07a1740 1 -- 172.21.15.84:0/45137374 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55ca206de1c0 con 0x55ca207e9db0 2024-05-22T23:05:08.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.000+0000 7f9ca6ffd640 1 -- 172.21.15.84:0/45137374 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cbc04c070 con 0x55ca207e9db0 2024-05-22T23:05:08.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.000+0000 7f9ca6ffd640 1 -- 172.21.15.84:0/45137374 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f9cb80b2030 con 0x55ca207e9db0 2024-05-22T23:05:08.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.000+0000 7f9cd07a1740 10 client.15954 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:05:08.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.000+0000 7f9cd07a1740 10 client.15954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:05:08.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.000+0000 7f9cd07a1740 20 client.15954 populate_metadata read hostname 'smithi084' 2024-05-22T23:05:08.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.000+0000 7f9cd07a1740 1 --2- 172.21.15.84:0/45137374 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55ca2086f770 0x55ca20871ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.000+0000 7f9cd07a1740 4 client.15954 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:05:08.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.000+0000 7f9cd07a1740 1 -- 172.21.15.84:0/45137374 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ca20755830 con 0x55ca2086f770 2024-05-22T23:05:08.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.001+0000 7f9cc6ffd640 1 --2- 172.21.15.84:0/45137374 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55ca2086f770 0x55ca20871ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.001+0000 7f9ca6ffd640 10 client.15954 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:08.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.001+0000 7f9cc6ffd640 1 --2- 172.21.15.84:0/45137374 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55ca2086f770 0x55ca20871ba0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df8363e9581ed3c1 server_cookie=e7162610a1e53171 in_seq=0 out_seq=0 2024-05-22T23:05:08.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.002+0000 7f9ca6ffd640 1 -- 172.21.15.84:0/45137374 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ca20755830 con 0x55ca2086f770 2024-05-22T23:05:08.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.002+0000 7f9ca6ffd640 10 client.15954 handle_command_reply: tid=0 2024-05-22T23:05:08.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.002+0000 7f9cd07a1740 1 client.15954 shutdown 2024-05-22T23:05:08.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.002+0000 7f9cd07a1740 1 -- 172.21.15.84:0/45137374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207e9db0 msgr2=0x55ca20801600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:08.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.002+0000 7f9cd07a1740 1 --2- 172.21.15.84:0/45137374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207e9db0 0x55ca20801600 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80cd010 tx=0x7f9cb80d1d70 comp rx=0 tx=0).stop 2024-05-22T23:05:08.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.002+0000 7f9cd07a1740 1 -- 172.21.15.84:0/45137374 shutdown_connections 2024-05-22T23:05:08.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.002+0000 7f9cd07a1740 1 --2- 172.21.15.84:0/45137374 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55ca2086f770 0x55ca20871ba0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.002+0000 7f9cd07a1740 1 --2- 172.21.15.84:0/45137374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca207f8f40 0x55ca207fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.002+0000 7f9cd07a1740 1 --2- 172.21.15.84:0/45137374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca207eb4a0 0x55ca207fc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.002+0000 7f9ca6ffd640 0 client.15954 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:08.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.002+0000 7f9cd07a1740 1 --2- 172.21.15.84:0/45137374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca207e9db0 0x55ca20801600 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.002+0000 7f9cd07a1740 1 -- 172.21.15.84:0/45137374 >> 172.21.15.84:0/45137374 conn(0x55ca207b28e0 msgr2=0x55ca207f5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:08.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.002+0000 7f9cd07a1740 1 -- 172.21.15.84:0/45137374 shutdown_connections 2024-05-22T23:05:08.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.002+0000 7f9cd07a1740 1 -- 172.21.15.84:0/45137374 wait complete. 2024-05-22T23:05:08.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.002+0000 7f9cd07a1740 20 client.15954 trim_cache size 0 max 16384 2024-05-22T23:05:08.012 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:05:08.012 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "no active scrubs running", 2024-05-22T23:05:08.012 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": {} 2024-05-22T23:05:08.012 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:05:08.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.002+0000 7f9ccda9a640 1 -- 172.21.15.84:0/1500538266 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f9cb00a2d00 msgr2=0x7f9cb00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:08.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.002+0000 7f9ccda9a640 1 --2- 172.21.15.84:0/1500538266 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f9cb00a2d00 0x7f9cb00a51c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0005c40 tx=0x7f9cc0009500 comp rx=0 tx=0).stop 2024-05-22T23:05:08.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.002+0000 7f9ccda9a640 1 -- 172.21.15.84:0/1500538266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cc8100740 msgr2=0x7f9cc8198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:08.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.002+0000 7f9ccda9a640 1 --2- 172.21.15.84:0/1500538266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cc8100740 0x7f9cc8198170 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800e9f0 tx=0x7f9cb800eeb0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.003+0000 7f9ccda9a640 1 -- 172.21.15.84:0/1500538266 shutdown_connections 2024-05-22T23:05:08.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.003+0000 7f9ccda9a640 1 --2- 172.21.15.84:0/1500538266 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f9cb00a2d00 0x7f9cb00a51c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.003+0000 7f9ccda9a640 1 --2- 172.21.15.84:0/1500538266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9cc8106750 0x7f9cc819ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.003+0000 7f9ccda9a640 1 --2- 172.21.15.84:0/1500538266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cc8101100 0x7f9cc81986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.003+0000 7f9ccda9a640 1 --2- 172.21.15.84:0/1500538266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cc8100740 0x7f9cc8198170 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.003+0000 7f9ccda9a640 1 -- 172.21.15.84:0/1500538266 >> 172.21.15.84:0/1500538266 conn(0x7f9cc8077870 msgr2=0x7f9cc80ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:08.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.003+0000 7f9ccda9a640 1 -- 172.21.15.84:0/1500538266 shutdown_connections 2024-05-22T23:05:08.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.003+0000 7f9ccda9a640 1 -- 172.21.15.84:0/1500538266 wait complete. 2024-05-22T23:05:08.016 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-22T23:05:08.016 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:05:08.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.119+0000 7f9268ba5640 1 -- 172.21.15.84:0/2227765922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9264107850 msgr2=0x7f9264107c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:08.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.119+0000 7f9268ba5640 1 --2- 172.21.15.84:0/2227765922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9264107850 0x7f9264107c30 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9254009930 tx=0x7f925402a420 comp rx=0 tx=0).stop 2024-05-22T23:05:08.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.119+0000 7f9268ba5640 1 -- 172.21.15.84:0/2227765922 shutdown_connections 2024-05-22T23:05:08.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.119+0000 7f9268ba5640 1 --2- 172.21.15.84:0/2227765922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9264102200 0x7f926410e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.119+0000 7f9268ba5640 1 --2- 172.21.15.84:0/2227765922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9264101840 0x7f9264101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.119+0000 7f9268ba5640 1 --2- 172.21.15.84:0/2227765922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9264107850 0x7f9264107c30 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.119+0000 7f9268ba5640 1 -- 172.21.15.84:0/2227765922 >> 172.21.15.84:0/2227765922 conn(0x7f9264078c30 msgr2=0x7f9264079040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:08.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.119+0000 7f9268ba5640 1 -- 172.21.15.84:0/2227765922 shutdown_connections 2024-05-22T23:05:08.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.119+0000 7f9268ba5640 1 -- 172.21.15.84:0/2227765922 wait complete. 2024-05-22T23:05:08.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.119+0000 7f9268ba5640 1 Processor -- start 2024-05-22T23:05:08.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.120+0000 7f9268ba5640 1 -- start start 2024-05-22T23:05:08.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.120+0000 7f9268ba5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9264101840 0x7f9264198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.120+0000 7f9268ba5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9264102200 0x7f92641986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.120+0000 7f9268ba5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9264107850 0x7f926419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.120+0000 7f9268ba5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9264111c60 con 0x7f9264101840 2024-05-22T23:05:08.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.120+0000 7f9268ba5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9264111ae0 con 0x7f9264102200 2024-05-22T23:05:08.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.120+0000 7f9268ba5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9264111de0 con 0x7f9264107850 2024-05-22T23:05:08.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.120+0000 7f926259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9264101840 0x7f9264198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.120+0000 7f926259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9264101840 0x7f9264198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57910/0 (socket says 172.21.15.84:57910) 2024-05-22T23:05:08.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.120+0000 7f926259c640 1 -- 172.21.15.84:0/4201050800 learned_addr learned my addr 172.21.15.84:0/4201050800 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:08.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.120+0000 7f9259d9b640 1 --2- 172.21.15.84:0/4201050800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9264102200 0x7f92641986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.120+0000 7f9262d9d640 1 --2- 172.21.15.84:0/4201050800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9264107850 0x7f926419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.120+0000 7f926259c640 1 -- 172.21.15.84:0/4201050800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9264107850 msgr2=0x7f926419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:08.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.120+0000 7f926259c640 1 --2- 172.21.15.84:0/4201050800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9264107850 0x7f926419ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.120+0000 7f926259c640 1 -- 172.21.15.84:0/4201050800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9264102200 msgr2=0x7f92641986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:08.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.120+0000 7f926259c640 1 --2- 172.21.15.84:0/4201050800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9264102200 0x7f92641986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.120+0000 7f926259c640 1 -- 172.21.15.84:0/4201050800 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926419d170 con 0x7f9264101840 2024-05-22T23:05:08.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.120+0000 7f9259d9b640 1 --2- 172.21.15.84:0/4201050800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9264102200 0x7f92641986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:05:08.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.121+0000 7f926259c640 1 --2- 172.21.15.84:0/4201050800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9264101840 0x7f9264198180 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9254009230 tx=0x7f9254004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:08.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.121+0000 7f925bfff640 1 -- 172.21.15.84:0/4201050800 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925403f030 con 0x7f9264101840 2024-05-22T23:05:08.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.121+0000 7f9268ba5640 1 -- 172.21.15.84:0/4201050800 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f926419d400 con 0x7f9264101840 2024-05-22T23:05:08.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.121+0000 7f925bfff640 1 -- 172.21.15.84:0/4201050800 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9254003ad0 con 0x7f9264101840 2024-05-22T23:05:08.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.121+0000 7f925bfff640 1 -- 172.21.15.84:0/4201050800 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9254035070 con 0x7f9264101840 2024-05-22T23:05:08.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.121+0000 7f9268ba5640 1 -- 172.21.15.84:0/4201050800 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92641a4cf0 con 0x7f9264101840 2024-05-22T23:05:08.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.122+0000 7f9268ba5640 1 -- 172.21.15.84:0/4201050800 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9264068ae0 con 0x7f9264101840 2024-05-22T23:05:08.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.125+0000 7f925bfff640 1 -- 172.21.15.84:0/4201050800 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9254042030 con 0x7f9264101840 2024-05-22T23:05:08.129 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.126+0000 7f925bfff640 1 --2- 172.21.15.84:0/4201050800 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f92500a2d80 0x7f92500a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.129 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.127+0000 7f9259d9b640 1 --2- 172.21.15.84:0/4201050800 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f92500a2d80 0x7f92500a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.127+0000 7f925bfff640 1 -- 172.21.15.84:0/4201050800 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f92540efa40 con 0x7f9264101840 2024-05-22T23:05:08.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.129+0000 7f9259d9b640 1 --2- 172.21.15.84:0/4201050800 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f92500a2d80 0x7f92500a5240 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9264103b80 tx=0x7f924c00b390 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:08.137 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.134+0000 7f925bfff640 1 -- 172.21.15.84:0/4201050800 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f92540bbfb0 con 0x7f9264101840 2024-05-22T23:05:08.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.383+0000 7f9268ba5640 1 -- 172.21.15.84:0/4201050800 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9264109db0 con 0x7f9264101840 2024-05-22T23:05:08.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.384+0000 7f925bfff640 1 -- 172.21.15.84:0/4201050800 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7f92540c02b0 con 0x7f9264101840 2024-05-22T23:05:08.388 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:05:08.388 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:05:08.388 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:05:08.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.386+0000 7f9268ba5640 1 -- 172.21.15.84:0/4201050800 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f92500a2d80 msgr2=0x7f92500a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:08.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.386+0000 7f9268ba5640 1 --2- 172.21.15.84:0/4201050800 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f92500a2d80 0x7f92500a5240 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9264103b80 tx=0x7f924c00b390 comp rx=0 tx=0).stop 2024-05-22T23:05:08.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.386+0000 7f9268ba5640 1 -- 172.21.15.84:0/4201050800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9264101840 msgr2=0x7f9264198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:08.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.386+0000 7f9268ba5640 1 --2- 172.21.15.84:0/4201050800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9264101840 0x7f9264198180 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9254009230 tx=0x7f9254004270 comp rx=0 tx=0).stop 2024-05-22T23:05:08.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.386+0000 7f9268ba5640 1 -- 172.21.15.84:0/4201050800 shutdown_connections 2024-05-22T23:05:08.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.386+0000 7f9268ba5640 1 --2- 172.21.15.84:0/4201050800 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f92500a2d80 0x7f92500a5240 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.386+0000 7f9268ba5640 1 --2- 172.21.15.84:0/4201050800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9264107850 0x7f926419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.386+0000 7f9268ba5640 1 --2- 172.21.15.84:0/4201050800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9264102200 0x7f92641986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.386+0000 7f9268ba5640 1 --2- 172.21.15.84:0/4201050800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9264101840 0x7f9264198180 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.386+0000 7f9268ba5640 1 -- 172.21.15.84:0/4201050800 >> 172.21.15.84:0/4201050800 conn(0x7f9264078c30 msgr2=0x7f9264104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:08.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.386+0000 7f9268ba5640 1 -- 172.21.15.84:0/4201050800 shutdown_connections 2024-05-22T23:05:08.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.386+0000 7f9268ba5640 1 -- 172.21.15.84:0/4201050800 wait complete. 2024-05-22T23:05:08.400 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-22T23:05:08.429 INFO:tasks.workunit.client.0.smithi084.stdout: 1 334666 15.97 MB/sec warmup 114 sec latency 37.983 ms 2024-05-22T23:05:08.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.508+0000 7fd8e401b640 1 -- 172.21.15.84:0/1157374721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8dc102250 msgr2=0x7fd8dc10e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:08.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.508+0000 7fd8e401b640 1 --2- 172.21.15.84:0/1157374721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8dc102250 0x7fd8dc10e760 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009cd0 tx=0x7fd8d402a8e0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.508+0000 7fd8e401b640 1 -- 172.21.15.84:0/1157374721 shutdown_connections 2024-05-22T23:05:08.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.508+0000 7fd8e401b640 1 --2- 172.21.15.84:0/1157374721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8dc102250 0x7fd8dc10e760 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.508+0000 7fd8e401b640 1 --2- 172.21.15.84:0/1157374721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8dc101890 0x7fd8dc101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.508+0000 7fd8e401b640 1 --2- 172.21.15.84:0/1157374721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8dc1078a0 0x7fd8dc107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.508+0000 7fd8e401b640 1 -- 172.21.15.84:0/1157374721 >> 172.21.15.84:0/1157374721 conn(0x7fd8dc0fd380 msgr2=0x7fd8dc0ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:08.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.508+0000 7fd8e401b640 1 -- 172.21.15.84:0/1157374721 shutdown_connections 2024-05-22T23:05:08.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.508+0000 7fd8e401b640 1 -- 172.21.15.84:0/1157374721 wait complete. 2024-05-22T23:05:08.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.508+0000 7fd8e401b640 1 Processor -- start 2024-05-22T23:05:08.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.509+0000 7fd8e401b640 1 -- start start 2024-05-22T23:05:08.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.509+0000 7fd8e401b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8dc101890 0x7fd8dc198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.509+0000 7fd8e401b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8dc102250 0x7fd8dc1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.509+0000 7fd8e401b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8dc1078a0 0x7fd8dc19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.509+0000 7fd8e401b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd8dc111cb0 con 0x7fd8dc1078a0 2024-05-22T23:05:08.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.509+0000 7fd8e401b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd8dc111b30 con 0x7fd8dc102250 2024-05-22T23:05:08.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.509+0000 7fd8e401b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd8dc111e30 con 0x7fd8dc101890 2024-05-22T23:05:08.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.509+0000 7fd8e25b8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8dc1078a0 0x7fd8dc19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.509+0000 7fd8e15b6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8dc102250 0x7fd8dc1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.509+0000 7fd8e25b8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8dc1078a0 0x7fd8dc19ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57934/0 (socket says 172.21.15.84:57934) 2024-05-22T23:05:08.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.509+0000 7fd8e1db7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8dc101890 0x7fd8dc198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.509+0000 7fd8e25b8640 1 -- 172.21.15.84:0/442352359 learned_addr learned my addr 172.21.15.84:0/442352359 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:08.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.509+0000 7fd8e15b6640 1 -- 172.21.15.84:0/442352359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8dc101890 msgr2=0x7fd8dc198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:08.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.509+0000 7fd8e15b6640 1 --2- 172.21.15.84:0/442352359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8dc101890 0x7fd8dc198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.509+0000 7fd8e15b6640 1 -- 172.21.15.84:0/442352359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8dc1078a0 msgr2=0x7fd8dc19ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:08.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.509+0000 7fd8e15b6640 1 --2- 172.21.15.84:0/442352359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8dc1078a0 0x7fd8dc19ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.509+0000 7fd8e15b6640 1 -- 172.21.15.84:0/442352359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8dc19d180 con 0x7fd8dc102250 2024-05-22T23:05:08.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.509+0000 7fd8e1db7640 1 --2- 172.21.15.84:0/442352359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8dc101890 0x7fd8dc198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:05:08.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.509+0000 7fd8e15b6640 1 --2- 172.21.15.84:0/442352359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8dc102250 0x7fd8dc1986d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00e9f0 tx=0x7fd8cc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:08.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.509+0000 7fd8d2ffd640 1 -- 172.21.15.84:0/442352359 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc00ccc0 con 0x7fd8dc102250 2024-05-22T23:05:08.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.510+0000 7fd8d2ffd640 1 -- 172.21.15.84:0/442352359 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8cc005020 con 0x7fd8dc102250 2024-05-22T23:05:08.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.510+0000 7fd8e401b640 1 -- 172.21.15.84:0/442352359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8dc19d470 con 0x7fd8dc102250 2024-05-22T23:05:08.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.510+0000 7fd8d2ffd640 1 -- 172.21.15.84:0/442352359 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc0049e0 con 0x7fd8dc102250 2024-05-22T23:05:08.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.510+0000 7fd8e401b640 1 -- 172.21.15.84:0/442352359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8dc1a4d00 con 0x7fd8dc102250 2024-05-22T23:05:08.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.512+0000 7fd8d2ffd640 1 -- 172.21.15.84:0/442352359 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd8cc005560 con 0x7fd8dc102250 2024-05-22T23:05:08.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.512+0000 7fd8e6dfd740 1 Processor -- start 2024-05-22T23:05:08.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.512+0000 7fd8e6dfd740 1 -- start start 2024-05-22T23:05:08.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.512+0000 7fd8e6dfd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 0x55c67a05e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.512+0000 7fd8e6dfd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67a05e780 0x55c67a06ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.512+0000 7fd8e6dfd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 0x55c67a06f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.512+0000 7fd8e6dfd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c679fc2db0 con 0x55c67a06d380 2024-05-22T23:05:08.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.512+0000 7fd8e6dfd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c679f61400 con 0x55c67a05de60 2024-05-22T23:05:08.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.512+0000 7fd8e6dfd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c679fbed70 con 0x55c67a05e780 2024-05-22T23:05:08.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.513+0000 7fd8e25b8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 0x55c67a05e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.513+0000 7fd8e15b6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 0x55c67a06f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.513+0000 7fd8e25b8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 0x55c67a05e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:33850/0 (socket says 172.21.15.84:33850) 2024-05-22T23:05:08.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.513+0000 7fd8e25b8640 1 -- 172.21.15.84:0/2828007424 learned_addr learned my addr 172.21.15.84:0/2828007424 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:08.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.513+0000 7fd8e1db7640 1 --2- 172.21.15.84:0/2828007424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67a05e780 0x55c67a06ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.513+0000 7fd8e25b8640 1 -- 172.21.15.84:0/2828007424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67a05e780 msgr2=0x55c67a06ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:08.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.513+0000 7fd8e25b8640 1 --2- 172.21.15.84:0/2828007424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67a05e780 0x55c67a06ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.513+0000 7fd8e25b8640 1 -- 172.21.15.84:0/2828007424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 msgr2=0x55c67a06f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:08.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.513+0000 7fd8e25b8640 1 --2- 172.21.15.84:0/2828007424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 0x55c67a06f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.513+0000 7fd8e25b8640 1 -- 172.21.15.84:0/2828007424 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c679f532f0 con 0x55c67a05de60 2024-05-22T23:05:08.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.513+0000 7fd8d2ffd640 1 --2- 172.21.15.84:0/442352359 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd8c80a2d00 0x7fd8c80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.513+0000 7fd8e25b8640 1 --2- 172.21.15.84:0/2828007424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 0x55c67a05e240 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009a20 tx=0x7fd8d4005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7f777824fb31d1e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:08.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.513+0000 7fd8e1db7640 1 --2- 172.21.15.84:0/442352359 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd8c80a2d00 0x7fd8c80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.513+0000 7fd8d2ffd640 1 -- 172.21.15.84:0/442352359 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fd8cc0cf3c0 con 0x7fd8dc102250 2024-05-22T23:05:08.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.513+0000 7fd8c7fff640 1 -- 172.21.15.84:0/2828007424 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4048070 con 0x55c67a05de60 2024-05-22T23:05:08.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.513+0000 7fd8c7fff640 1 -- 172.21.15.84:0/2828007424 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8d4036070 con 0x55c67a05de60 2024-05-22T23:05:08.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.513+0000 7fd8c7fff640 1 -- 172.21.15.84:0/2828007424 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4004390 con 0x55c67a05de60 2024-05-22T23:05:08.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.514+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/2828007424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 msgr2=0x55c67a05e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:08.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.514+0000 7fd8e1db7640 1 --2- 172.21.15.84:0/442352359 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd8c80a2d00 0x7fd8c80a51c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8012560 tx=0x7fd8d8005f90 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:08.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.514+0000 7fd8e6dfd740 1 --2- 172.21.15.84:0/2828007424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 0x55c67a05e240 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009a20 tx=0x7fd8d4005ef0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.514+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/2828007424 shutdown_connections 2024-05-22T23:05:08.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.514+0000 7fd8e6dfd740 1 --2- 172.21.15.84:0/2828007424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 0x55c67a06f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.514+0000 7fd8e6dfd740 1 --2- 172.21.15.84:0/2828007424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67a05e780 0x55c67a06ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.514+0000 7fd8e6dfd740 1 --2- 172.21.15.84:0/2828007424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 0x55c67a05e240 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.514+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/2828007424 >> 172.21.15.84:0/2828007424 conn(0x55c67a05a2a0 msgr2=0x55c67a05a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:08.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.514+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/2828007424 shutdown_connections 2024-05-22T23:05:08.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.514+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/2828007424 wait complete. 2024-05-22T23:05:08.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.515+0000 7fd8e6dfd740 1 -- start start 2024-05-22T23:05:08.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.515+0000 7fd8e6dfd740 1 Processor -- start 2024-05-22T23:05:08.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.515+0000 7fd8e6dfd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 0x55c67a06a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.515+0000 7fd8e6dfd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67a05e780 0x55c67a06a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.515+0000 7fd8e6dfd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 0x55c67a06ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.515+0000 7fd8e6dfd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c67a070160 con 0x55c67a06d380 2024-05-22T23:05:08.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.515+0000 7fd8e6dfd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c679f7d2e0 con 0x55c67a05de60 2024-05-22T23:05:08.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.515+0000 7fd8e6dfd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c67a0702e0 con 0x55c67a05e780 2024-05-22T23:05:08.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.515+0000 7fd8e6dfd740 10 client.0 _ll_register_callbacks cb 0x55c679f86a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:05:08.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.515+0000 7fd8e25b8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 0x55c67a06ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.515+0000 7fd8e1db7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 0x55c67a06a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.515+0000 7fd8e15b6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67a05e780 0x55c67a06a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.515+0000 7fd8e1db7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 0x55c67a06a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:33854/0 (socket says 172.21.15.84:33854) 2024-05-22T23:05:08.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.515+0000 7fd8e25b8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 0x55c67a06ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57950/0 (socket says 172.21.15.84:57950) 2024-05-22T23:05:08.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.515+0000 7fd8e1db7640 1 -- 172.21.15.84:0/1659282649 learned_addr learned my addr 172.21.15.84:0/1659282649 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:08.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.515+0000 7fd8e1db7640 1 -- 172.21.15.84:0/1659282649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67a05e780 msgr2=0x55c67a06a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:08.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.515+0000 7fd8e1db7640 1 --2- 172.21.15.84:0/1659282649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67a05e780 0x55c67a06a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.515+0000 7fd8e1db7640 1 -- 172.21.15.84:0/1659282649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 msgr2=0x55c67a06ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:08.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.515+0000 7fd8e1db7640 1 --2- 172.21.15.84:0/1659282649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 0x55c67a06ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.515+0000 7fd8e1db7640 1 -- 172.21.15.84:0/1659282649 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c679cd63f0 con 0x55c67a05de60 2024-05-22T23:05:08.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.516+0000 7fd8e1db7640 1 --2- 172.21.15.84:0/1659282649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 0x55c67a06a2e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd8d800fbd0 tx=0x7fd8d8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0021bebb4e8fa6d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:08.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.516+0000 7fd8c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:05:08.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.516+0000 7fd8c57fa640 1 -- 172.21.15.84:0/1659282649 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d800fe40 con 0x55c67a05de60 2024-05-22T23:05:08.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.516+0000 7fd8c57fa640 1 -- 172.21.15.84:0/1659282649 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fd8d8018030 con 0x55c67a05de60 2024-05-22T23:05:08.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.516+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/1659282649 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8d4036070 con 0x55c67a05de60 2024-05-22T23:05:08.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.516+0000 7fd8c57fa640 1 -- 172.21.15.84:0/1659282649 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fd8d801e8a0 con 0x55c67a05de60 2024-05-22T23:05:08.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.516+0000 7fd8e6dfd740 10 client.25547 fetch_fsmap learned FSMap version 18 2024-05-22T23:05:08.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.516+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/1659282649 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55c679da07d0 con 0x55c67a05de60 2024-05-22T23:05:08.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.516+0000 7fd8c57fa640 1 -- 172.21.15.84:0/1659282649 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8d4036070 con 0x55c67a05de60 2024-05-22T23:05:08.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.516+0000 7fd8c57fa640 1 -- 172.21.15.84:0/1659282649 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fd8d8032c30 con 0x55c67a05de60 2024-05-22T23:05:08.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.516+0000 7fd8e6dfd740 10 client.25547 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:05:08.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.516+0000 7fd8e6dfd740 10 client.25547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:05:08.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.516+0000 7fd8e6dfd740 20 client.25547 populate_metadata read hostname 'smithi084' 2024-05-22T23:05:08.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.516+0000 7fd8e6dfd740 1 --2- 172.21.15.84:0/1659282649 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c67a07d740 0x55c67a07fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.516+0000 7fd8e6dfd740 4 client.25547 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:05:08.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.516+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/1659282649 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c679fd04e0 con 0x55c67a07d740 2024-05-22T23:05:08.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.517+0000 7fd8e15b6640 1 --2- 172.21.15.84:0/1659282649 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c67a07d740 0x55c67a07fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.517+0000 7fd8c57fa640 10 client.25547 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:08.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.517+0000 7fd8e15b6640 1 --2- 172.21.15.84:0/1659282649 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c67a07d740 0x55c67a07fb90 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc95b6419f8d43b6 server_cookie=abb346ea5f1f3c99 in_seq=0 out_seq=0 2024-05-22T23:05:08.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.519+0000 7fd8c57fa640 1 -- 172.21.15.84:0/1659282649 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c679fd04e0 con 0x55c67a07d740 2024-05-22T23:05:08.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.519+0000 7fd8c57fa640 10 client.25547 handle_command_reply: tid=0 2024-05-22T23:05:08.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.519+0000 7fd8e6dfd740 1 client.25547 shutdown 2024-05-22T23:05:08.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.519+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/1659282649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 msgr2=0x55c67a06a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:08.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.519+0000 7fd8e6dfd740 1 --2- 172.21.15.84:0/1659282649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 0x55c67a06a2e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd8d800fbd0 tx=0x7fd8d8004730 comp rx=0 tx=0).stop 2024-05-22T23:05:08.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.520+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/1659282649 shutdown_connections 2024-05-22T23:05:08.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.520+0000 7fd8e6dfd740 1 --2- 172.21.15.84:0/1659282649 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c67a07d740 0x55c67a07fb90 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.520+0000 7fd8e6dfd740 1 --2- 172.21.15.84:0/1659282649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 0x55c67a06ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.520+0000 7fd8c57fa640 0 client.25547 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:08.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.520+0000 7fd8e6dfd740 1 --2- 172.21.15.84:0/1659282649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67a05e780 0x55c67a06a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.520+0000 7fd8e6dfd740 1 --2- 172.21.15.84:0/1659282649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 0x55c67a06a2e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.520+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/1659282649 >> 172.21.15.84:0/1659282649 conn(0x55c67a05a2a0 msgr2=0x55c67a060050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:08.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.520+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/1659282649 shutdown_connections 2024-05-22T23:05:08.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.520+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/1659282649 wait complete. 2024-05-22T23:05:08.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.520+0000 7fd8e6dfd740 20 client.25547 trim_cache size 0 max 16384 2024-05-22T23:05:08.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.533+0000 7fd8e6dfd740 1 Processor -- start 2024-05-22T23:05:08.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.534+0000 7fd8e6dfd740 1 -- start start 2024-05-22T23:05:08.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.534+0000 7fd8e6dfd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 0x55c67a079090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.534+0000 7fd8e6dfd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67a05e780 0x55c67a059c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.534+0000 7fd8e6dfd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 0x55c67a06a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.534+0000 7fd8e6dfd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd8dc1a9390 con 0x55c67a06d380 2024-05-22T23:05:08.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.534+0000 7fd8e6dfd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd8dc1abc10 con 0x55c67a05de60 2024-05-22T23:05:08.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.534+0000 7fd8e6dfd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd8dc1a9a00 con 0x55c67a05e780 2024-05-22T23:05:08.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.534+0000 7fd8e25b8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 0x55c67a06a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.534+0000 7fd8e25b8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 0x55c67a06a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57964/0 (socket says 172.21.15.84:57964) 2024-05-22T23:05:08.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.534+0000 7fd8e1db7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 0x55c67a079090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.534+0000 7fd8e25b8640 1 -- 172.21.15.84:0/1711700437 learned_addr learned my addr 172.21.15.84:0/1711700437 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:08.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.534+0000 7fd8e15b6640 1 --2- 172.21.15.84:0/1711700437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67a05e780 0x55c67a059c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.534+0000 7fd8e1db7640 1 -- 172.21.15.84:0/1711700437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67a05e780 msgr2=0x55c67a059c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:08.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.534+0000 7fd8e1db7640 1 --2- 172.21.15.84:0/1711700437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67a05e780 0x55c67a059c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.534+0000 7fd8e1db7640 1 -- 172.21.15.84:0/1711700437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 msgr2=0x55c67a06a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:08.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.534+0000 7fd8e1db7640 1 --2- 172.21.15.84:0/1711700437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 0x55c67a06a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.534+0000 7fd8e1db7640 1 -- 172.21.15.84:0/1711700437 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c67a05eb60 con 0x55c67a05de60 2024-05-22T23:05:08.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.534+0000 7fd8e25b8640 1 --2- 172.21.15.84:0/1711700437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 0x55c67a06a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:05:08.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.534+0000 7fd8e1db7640 1 --2- 172.21.15.84:0/1711700437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 0x55c67a079090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8029660 tx=0x7fd8d8012e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=709f3a9f89084c5f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:08.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.535+0000 7fd8c77fe640 1 -- 172.21.15.84:0/1711700437 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d801ea40 con 0x55c67a05de60 2024-05-22T23:05:08.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.535+0000 7fd8c77fe640 1 -- 172.21.15.84:0/1711700437 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8d8004520 con 0x55c67a05de60 2024-05-22T23:05:08.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.535+0000 7fd8c77fe640 1 -- 172.21.15.84:0/1711700437 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8005550 con 0x55c67a05de60 2024-05-22T23:05:08.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.535+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/1711700437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 msgr2=0x55c67a079090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:08.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.535+0000 7fd8e6dfd740 1 --2- 172.21.15.84:0/1711700437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 0x55c67a079090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8029660 tx=0x7fd8d8012e70 comp rx=0 tx=0).stop 2024-05-22T23:05:08.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.535+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/1711700437 shutdown_connections 2024-05-22T23:05:08.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.535+0000 7fd8e6dfd740 1 --2- 172.21.15.84:0/1711700437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 0x55c67a06a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.535+0000 7fd8e6dfd740 1 --2- 172.21.15.84:0/1711700437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67a05e780 0x55c67a059c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.535+0000 7fd8e6dfd740 1 --2- 172.21.15.84:0/1711700437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 0x55c67a079090 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.535+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/1711700437 >> 172.21.15.84:0/1711700437 conn(0x55c67a05a2a0 msgr2=0x55c67a06a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:08.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.535+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/1711700437 shutdown_connections 2024-05-22T23:05:08.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.535+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/1711700437 wait complete. 2024-05-22T23:05:08.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.536+0000 7fd8e6dfd740 1 -- start start 2024-05-22T23:05:08.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.536+0000 7fd8e6dfd740 1 Processor -- start 2024-05-22T23:05:08.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.536+0000 7fd8e6dfd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 0x55c67a070ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.536+0000 7fd8e6dfd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67a05e780 0x55c67a071400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.536+0000 7fd8e6dfd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 0x55c67a071940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.536+0000 7fd8e6dfd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd8dc1aa0d0 con 0x55c67a06d380 2024-05-22T23:05:08.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.536+0000 7fd8e6dfd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd8dc1ab540 con 0x55c67a05de60 2024-05-22T23:05:08.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.536+0000 7fd8e6dfd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd8dc1aa7a0 con 0x55c67a05e780 2024-05-22T23:05:08.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.536+0000 7fd8e6dfd740 10 client.0 _ll_register_callbacks cb 0x7fd8d800f540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:05:08.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.536+0000 7fd8e25b8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 0x55c67a071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.536+0000 7fd8e1db7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 0x55c67a070ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.536+0000 7fd8e25b8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 0x55c67a071940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57966/0 (socket says 172.21.15.84:57966) 2024-05-22T23:05:08.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.536+0000 7fd8e1db7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 0x55c67a070ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:33884/0 (socket says 172.21.15.84:33884) 2024-05-22T23:05:08.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.536+0000 7fd8e25b8640 1 -- 172.21.15.84:0/3531989877 learned_addr learned my addr 172.21.15.84:0/3531989877 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:08.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.536+0000 7fd8e15b6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67a05e780 0x55c67a071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.537+0000 7fd8e25b8640 1 -- 172.21.15.84:0/3531989877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67a05e780 msgr2=0x55c67a071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:08.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.537+0000 7fd8e25b8640 1 --2- 172.21.15.84:0/3531989877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67a05e780 0x55c67a071400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.537+0000 7fd8e25b8640 1 -- 172.21.15.84:0/3531989877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 msgr2=0x55c67a070ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:08.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.537+0000 7fd8e25b8640 1 --2- 172.21.15.84:0/3531989877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 0x55c67a070ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.537+0000 7fd8e25b8640 1 -- 172.21.15.84:0/3531989877 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c67a0e2cd0 con 0x55c67a06d380 2024-05-22T23:05:08.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.537+0000 7fd8e1db7640 1 --2- 172.21.15.84:0/3531989877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 0x55c67a070ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:05:08.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.537+0000 7fd8e15b6640 1 --2- 172.21.15.84:0/3531989877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67a05e780 0x55c67a071400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:05:08.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.537+0000 7fd8e25b8640 1 --2- 172.21.15.84:0/3531989877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 0x55c67a071940 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4005f60 tx=0x7fd8d4003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf70620bd8d7689 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:08.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.537+0000 7fd8c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:05:08.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.537+0000 7fd8c57fa640 1 -- 172.21.15.84:0/3531989877 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4054020 con 0x55c67a06d380 2024-05-22T23:05:08.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.537+0000 7fd8c57fa640 1 -- 172.21.15.84:0/3531989877 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fd8d4031ac0 con 0x55c67a06d380 2024-05-22T23:05:08.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.537+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/3531989877 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8d8004520 con 0x55c67a06d380 2024-05-22T23:05:08.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.537+0000 7fd8c57fa640 1 -- 172.21.15.84:0/3531989877 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fd8d4048040 con 0x55c67a06d380 2024-05-22T23:05:08.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.537+0000 7fd8e6dfd740 10 client.15978 fetch_fsmap learned FSMap version 18 2024-05-22T23:05:08.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.537+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/3531989877 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55c67a0e3ed0 con 0x55c67a06d380 2024-05-22T23:05:08.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.537+0000 7fd8c57fa640 1 -- 172.21.15.84:0/3531989877 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8d8004520 con 0x55c67a06d380 2024-05-22T23:05:08.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.538+0000 7fd8c57fa640 1 -- 172.21.15.84:0/3531989877 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fd8d4042030 con 0x55c67a06d380 2024-05-22T23:05:08.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.538+0000 7fd8e6dfd740 10 client.15978 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:05:08.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.538+0000 7fd8e6dfd740 10 client.15978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:05:08.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.538+0000 7fd8e6dfd740 20 client.15978 populate_metadata read hostname 'smithi084' 2024-05-22T23:05:08.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.538+0000 7fd8e6dfd740 1 --2- 172.21.15.84:0/3531989877 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c67a0e4280 0x55c67a0e66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:08.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.538+0000 7fd8e6dfd740 4 client.15978 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-22T23:05:08.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.538+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/3531989877 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c679fca830 con 0x55c67a0e4280 2024-05-22T23:05:08.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.538+0000 7fd8e1db7640 1 --2- 172.21.15.84:0/3531989877 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c67a0e4280 0x55c67a0e66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:08.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.538+0000 7fd8c57fa640 10 client.15978 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:08.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.538+0000 7fd8e1db7640 1 --2- 172.21.15.84:0/3531989877 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c67a0e4280 0x55c67a0e66a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c89f590b88e149 server_cookie=24d9b5888ed3fc0e in_seq=0 out_seq=0 2024-05-22T23:05:08.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.538+0000 7fd8c57fa640 1 -- 172.21.15.84:0/3531989877 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c679fca830 con 0x55c67a0e4280 2024-05-22T23:05:08.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.538+0000 7fd8c57fa640 10 client.15978 handle_command_reply: tid=0 2024-05-22T23:05:08.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.538+0000 7fd8e6dfd740 1 client.15978 shutdown 2024-05-22T23:05:08.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.538+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/3531989877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 msgr2=0x55c67a071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:08.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.538+0000 7fd8e6dfd740 1 --2- 172.21.15.84:0/3531989877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 0x55c67a071940 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4005f60 tx=0x7fd8d4003e00 comp rx=0 tx=0).stop 2024-05-22T23:05:08.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.539+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/3531989877 shutdown_connections 2024-05-22T23:05:08.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.539+0000 7fd8e6dfd740 1 --2- 172.21.15.84:0/3531989877 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c67a0e4280 0x55c67a0e66a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.539+0000 7fd8e6dfd740 1 --2- 172.21.15.84:0/3531989877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c67a06d380 0x55c67a071940 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.539+0000 7fd8c57fa640 0 client.15978 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:08.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.539+0000 7fd8e6dfd740 1 --2- 172.21.15.84:0/3531989877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c67a05e780 0x55c67a071400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.539+0000 7fd8e6dfd740 1 --2- 172.21.15.84:0/3531989877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c67a05de60 0x55c67a070ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.539+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/3531989877 >> 172.21.15.84:0/3531989877 conn(0x55c67a05a2a0 msgr2=0x55c67a06ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:08.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.539+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/3531989877 shutdown_connections 2024-05-22T23:05:08.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.539+0000 7fd8e6dfd740 1 -- 172.21.15.84:0/3531989877 wait complete. 2024-05-22T23:05:08.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.539+0000 7fd8e6dfd740 20 client.15978 trim_cache size 0 max 16384 2024-05-22T23:05:08.549 INFO:teuthology.orchestra.run.smithi084.stdout:[] 2024-05-22T23:05:08.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.539+0000 7fd8e401b640 1 -- 172.21.15.84:0/442352359 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd8c80a2d00 msgr2=0x7fd8c80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:08.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.539+0000 7fd8e401b640 1 --2- 172.21.15.84:0/442352359 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd8c80a2d00 0x7fd8c80a51c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8012560 tx=0x7fd8d8005f90 comp rx=0 tx=0).stop 2024-05-22T23:05:08.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.539+0000 7fd8e401b640 1 -- 172.21.15.84:0/442352359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8dc102250 msgr2=0x7fd8dc1986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:08.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.539+0000 7fd8e401b640 1 --2- 172.21.15.84:0/442352359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8dc102250 0x7fd8dc1986d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00e9f0 tx=0x7fd8cc00eeb0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.539+0000 7fd8e401b640 1 -- 172.21.15.84:0/442352359 shutdown_connections 2024-05-22T23:05:08.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.539+0000 7fd8e401b640 1 --2- 172.21.15.84:0/442352359 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd8c80a2d00 0x7fd8c80a51c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.539+0000 7fd8e401b640 1 --2- 172.21.15.84:0/442352359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8dc1078a0 0x7fd8dc19ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.539+0000 7fd8e401b640 1 --2- 172.21.15.84:0/442352359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8dc102250 0x7fd8dc1986d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.539+0000 7fd8e401b640 1 --2- 172.21.15.84:0/442352359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8dc101890 0x7fd8dc198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:08.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.539+0000 7fd8e401b640 1 -- 172.21.15.84:0/442352359 >> 172.21.15.84:0/442352359 conn(0x7fd8dc0fd380 msgr2=0x7fd8dc0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:08.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.539+0000 7fd8e401b640 1 -- 172.21.15.84:0/442352359 shutdown_connections 2024-05-22T23:05:08.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:08.539+0000 7fd8e401b640 1 -- 172.21.15.84:0/442352359 wait complete. 2024-05-22T23:05:09.429 INFO:tasks.workunit.client.0.smithi084.stdout: 1 337643 15.98 MB/sec warmup 115 sec latency 21.971 ms 2024-05-22T23:05:09.555 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-22T23:05:09.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:05:09.556 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-22T23:05:09.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:09 smithi204 ceph-mon[36211]: pgmap v102: 97 pgs: 97 active+clean; 166 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 276 op/s 2024-05-22T23:05:09.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:09 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/4201050800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:05:09.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.662+0000 7fa3eb2a5640 1 -- 172.21.15.84:0/1874250797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3e4101100 msgr2=0x7fa3e410d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:09.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.662+0000 7fa3eb2a5640 1 --2- 172.21.15.84:0/1874250797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3e4101100 0x7fa3e410d610 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc009cd0 tx=0x7fa3dc02a8e0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.662+0000 7fa3eb2a5640 1 -- 172.21.15.84:0/1874250797 shutdown_connections 2024-05-22T23:05:09.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.662+0000 7fa3eb2a5640 1 --2- 172.21.15.84:0/1874250797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3e4101100 0x7fa3e410d610 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.662+0000 7fa3eb2a5640 1 --2- 172.21.15.84:0/1874250797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3e4100740 0x7fa3e4100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.662+0000 7fa3eb2a5640 1 --2- 172.21.15.84:0/1874250797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3e4106750 0x7fa3e4106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.662+0000 7fa3eb2a5640 1 -- 172.21.15.84:0/1874250797 >> 172.21.15.84:0/1874250797 conn(0x7fa3e4077870 msgr2=0x7fa3e4077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:09.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.663+0000 7fa3eb2a5640 1 -- 172.21.15.84:0/1874250797 shutdown_connections 2024-05-22T23:05:09.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.663+0000 7fa3eb2a5640 1 -- 172.21.15.84:0/1874250797 wait complete. 2024-05-22T23:05:09.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.663+0000 7fa3eb2a5640 1 Processor -- start 2024-05-22T23:05:09.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.663+0000 7fa3eb2a5640 1 -- start start 2024-05-22T23:05:09.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.663+0000 7fa3eb2a5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3e4100740 0x7fa3e4199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.663+0000 7fa3eb2a5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3e4101100 0x7fa3e4199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.663+0000 7fa3eb2a5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3e4106750 0x7fa3e41a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.663+0000 7fa3eb2a5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e4110b60 con 0x7fa3e4100740 2024-05-22T23:05:09.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.663+0000 7fa3eb2a5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e41109e0 con 0x7fa3e4101100 2024-05-22T23:05:09.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.663+0000 7fa3eb2a5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e4110ce0 con 0x7fa3e4106750 2024-05-22T23:05:09.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.663+0000 7fa3e9041640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3e4100740 0x7fa3e4199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.663+0000 7fa3e9041640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3e4100740 0x7fa3e4199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57980/0 (socket says 172.21.15.84:57980) 2024-05-22T23:05:09.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.663+0000 7fa3e9041640 1 -- 172.21.15.84:0/1686697214 learned_addr learned my addr 172.21.15.84:0/1686697214 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:09.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.663+0000 7fa3e9842640 1 --2- 172.21.15.84:0/1686697214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3e4106750 0x7fa3e41a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.664+0000 7fa3e8840640 1 --2- 172.21.15.84:0/1686697214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3e4101100 0x7fa3e4199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.664+0000 7fa3e9041640 1 -- 172.21.15.84:0/1686697214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3e4106750 msgr2=0x7fa3e41a40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:09.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.664+0000 7fa3e9041640 1 --2- 172.21.15.84:0/1686697214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3e4106750 0x7fa3e41a40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.664+0000 7fa3e9041640 1 -- 172.21.15.84:0/1686697214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3e4101100 msgr2=0x7fa3e4199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:09.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.664+0000 7fa3e9041640 1 --2- 172.21.15.84:0/1686697214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3e4101100 0x7fa3e4199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.664+0000 7fa3e9041640 1 -- 172.21.15.84:0/1686697214 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e41a4600 con 0x7fa3e4100740 2024-05-22T23:05:09.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.664+0000 7fa3e9041640 1 --2- 172.21.15.84:0/1686697214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3e4100740 0x7fa3e4199040 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400e9f0 tx=0x7fa3d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:09.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.664+0000 7fa3e27fc640 1 -- 172.21.15.84:0/1686697214 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d400cd80 con 0x7fa3e4100740 2024-05-22T23:05:09.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.664+0000 7fa3e27fc640 1 -- 172.21.15.84:0/1686697214 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d4005500 con 0x7fa3e4100740 2024-05-22T23:05:09.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.664+0000 7fa3eb2a5640 1 -- 172.21.15.84:0/1686697214 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e41a47d0 con 0x7fa3e4100740 2024-05-22T23:05:09.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.664+0000 7fa3e27fc640 1 -- 172.21.15.84:0/1686697214 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4010690 con 0x7fa3e4100740 2024-05-22T23:05:09.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.664+0000 7fa3eb2a5640 1 -- 172.21.15.84:0/1686697214 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e41a4ca0 con 0x7fa3e4100740 2024-05-22T23:05:09.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.666+0000 7fa3e27fc640 1 -- 172.21.15.84:0/1686697214 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa3d40046c0 con 0x7fa3e4100740 2024-05-22T23:05:09.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.667+0000 7fa3edfb1740 1 Processor -- start 2024-05-22T23:05:09.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.667+0000 7fa3edfb1740 1 -- start start 2024-05-22T23:05:09.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.667+0000 7fa3edfb1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564dcba33c50 0x564dcba34030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.667+0000 7fa3edfb1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba34570 0x564dcba3e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.667+0000 7fa3edfb1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564dcba3ec40 0x564dcba41030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.667+0000 7fa3edfb1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564dcb997a40 con 0x564dcba34570 2024-05-22T23:05:09.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.667+0000 7fa3edfb1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564dcb936440 con 0x564dcba3ec40 2024-05-22T23:05:09.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.667+0000 7fa3edfb1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564dcb993a00 con 0x564dcba33c50 2024-05-22T23:05:09.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.667+0000 7fa3e27fc640 1 --2- 172.21.15.84:0/1686697214 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fa3cc0a2d00 0x7fa3cc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.667+0000 7fa3e9041640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba34570 0x564dcba3e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.667+0000 7fa3e8840640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564dcba3ec40 0x564dcba41030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.667+0000 7fa3e8840640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564dcba3ec40 0x564dcba41030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:33898/0 (socket says 172.21.15.84:33898) 2024-05-22T23:05:09.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.667+0000 7fa3e9041640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba34570 0x564dcba3e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57986/0 (socket says 172.21.15.84:57986) 2024-05-22T23:05:09.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.667+0000 7fa3e8840640 1 -- 172.21.15.84:0/3893649439 learned_addr learned my addr 172.21.15.84:0/3893649439 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:09.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.667+0000 7fa3e9842640 1 --2- 172.21.15.84:0/3893649439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564dcba33c50 0x564dcba34030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.667+0000 7fa3e9842640 1 --2- 172.21.15.84:0/1686697214 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fa3cc0a2d00 0x7fa3cc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.667+0000 7fa3e8840640 1 -- 172.21.15.84:0/3893649439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564dcba33c50 msgr2=0x564dcba34030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:09.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.667+0000 7fa3e8840640 1 --2- 172.21.15.84:0/3893649439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564dcba33c50 0x564dcba34030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.667+0000 7fa3e8840640 1 -- 172.21.15.84:0/3893649439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba34570 msgr2=0x564dcba3e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:09.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.668+0000 7fa3e8840640 1 --2- 172.21.15.84:0/3893649439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba34570 0x564dcba3e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.668+0000 7fa3e8840640 1 -- 172.21.15.84:0/3893649439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dcb9402d0 con 0x564dcba3ec40 2024-05-22T23:05:09.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.668+0000 7fa3e27fc640 1 -- 172.21.15.84:0/1686697214 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fa3d40cf800 con 0x7fa3e4100740 2024-05-22T23:05:09.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.668+0000 7fa3e8840640 1 --2- 172.21.15.84:0/3893649439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564dcba3ec40 0x564dcba41030 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8009c90 tx=0x7fa3d8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=221a49864f397903 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:09.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.668+0000 7fa3e9842640 1 --2- 172.21.15.84:0/1686697214 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fa3cc0a2d00 0x7fa3cc0a51c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc004ab0 tx=0x7fa3dc004a40 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:09.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.668+0000 7fa3cb7fe640 1 -- 172.21.15.84:0/3893649439 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d801e070 con 0x564dcba3ec40 2024-05-22T23:05:09.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.668+0000 7fa3cb7fe640 1 -- 172.21.15.84:0/3893649439 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d8019b10 con 0x564dcba3ec40 2024-05-22T23:05:09.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.668+0000 7fa3cb7fe640 1 -- 172.21.15.84:0/3893649439 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d800ea90 con 0x564dcba3ec40 2024-05-22T23:05:09.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.669+0000 7fa3edfb1740 1 -- 172.21.15.84:0/3893649439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564dcba3ec40 msgr2=0x564dcba41030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:09.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.669+0000 7fa3edfb1740 1 --2- 172.21.15.84:0/3893649439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564dcba3ec40 0x564dcba41030 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8009c90 tx=0x7fa3d8011810 comp rx=0 tx=0).stop 2024-05-22T23:05:09.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.669+0000 7fa3edfb1740 1 -- 172.21.15.84:0/3893649439 shutdown_connections 2024-05-22T23:05:09.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.669+0000 7fa3edfb1740 1 --2- 172.21.15.84:0/3893649439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564dcba3ec40 0x564dcba41030 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.669+0000 7fa3edfb1740 1 --2- 172.21.15.84:0/3893649439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba34570 0x564dcba3e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.669+0000 7fa3edfb1740 1 --2- 172.21.15.84:0/3893649439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564dcba33c50 0x564dcba34030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.669+0000 7fa3edfb1740 1 -- 172.21.15.84:0/3893649439 >> 172.21.15.84:0/3893649439 conn(0x564dcba2ff70 msgr2=0x564dcba30350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:09.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.669+0000 7fa3edfb1740 1 -- 172.21.15.84:0/3893649439 shutdown_connections 2024-05-22T23:05:09.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.669+0000 7fa3edfb1740 1 -- 172.21.15.84:0/3893649439 wait complete. 2024-05-22T23:05:09.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.669+0000 7fa3edfb1740 1 -- start start 2024-05-22T23:05:09.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.669+0000 7fa3edfb1740 1 Processor -- start 2024-05-22T23:05:09.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.670+0000 7fa3edfb1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564dcba33c50 0x564dcba3bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.670+0000 7fa3edfb1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba34570 0x564dcba3c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.670+0000 7fa3edfb1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564dcba3ec40 0x564dcba3c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.670+0000 7fa3edfb1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564dcba41a30 con 0x564dcba34570 2024-05-22T23:05:09.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.670+0000 7fa3edfb1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564dcb9215f0 con 0x564dcba33c50 2024-05-22T23:05:09.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.670+0000 7fa3edfb1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564dcba41bb0 con 0x564dcba3ec40 2024-05-22T23:05:09.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.670+0000 7fa3edfb1740 10 client.0 _ll_register_callbacks cb 0x564dcb91ccf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:05:09.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.670+0000 7fa3e8840640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba34570 0x564dcba3c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.670+0000 7fa3e9041640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564dcba33c50 0x564dcba3bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.670+0000 7fa3e9041640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564dcba33c50 0x564dcba3bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:33906/0 (socket says 172.21.15.84:33906) 2024-05-22T23:05:09.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.670+0000 7fa3e9041640 1 -- 172.21.15.84:0/1199255437 learned_addr learned my addr 172.21.15.84:0/1199255437 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:09.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.670+0000 7fa3e8840640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba34570 0x564dcba3c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58000/0 (socket says 172.21.15.84:58000) 2024-05-22T23:05:09.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.670+0000 7fa3e9842640 1 --2- 172.21.15.84:0/1199255437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564dcba3ec40 0x564dcba3c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.670+0000 7fa3e8840640 1 -- 172.21.15.84:0/1199255437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564dcba3ec40 msgr2=0x564dcba3c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:09.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.670+0000 7fa3e8840640 1 --2- 172.21.15.84:0/1199255437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564dcba3ec40 0x564dcba3c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.670+0000 7fa3e8840640 1 -- 172.21.15.84:0/1199255437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564dcba33c50 msgr2=0x564dcba3bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:09.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.670+0000 7fa3e8840640 1 --2- 172.21.15.84:0/1199255437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564dcba33c50 0x564dcba3bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.670+0000 7fa3e8840640 1 -- 172.21.15.84:0/1199255437 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dcb7737e0 con 0x564dcba34570 2024-05-22T23:05:09.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.670+0000 7fa3e8840640 1 --2- 172.21.15.84:0/1199255437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba34570 0x564dcba3c080 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa3d801c040 tx=0x7fa3d800e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7910b5438b97b470 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:09.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.670+0000 7fa3c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:05:09.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.670+0000 7fa3c8ff9640 1 -- 172.21.15.84:0/1199255437 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d801e050 con 0x564dcba34570 2024-05-22T23:05:09.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.670+0000 7fa3c8ff9640 1 -- 172.21.15.84:0/1199255437 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fa3d8005250 con 0x564dcba34570 2024-05-22T23:05:09.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.670+0000 7fa3edfb1740 1 -- 172.21.15.84:0/1199255437 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3d8019b10 con 0x564dcba34570 2024-05-22T23:05:09.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.671+0000 7fa3c8ff9640 1 -- 172.21.15.84:0/1199255437 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fa3d8005450 con 0x564dcba34570 2024-05-22T23:05:09.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.671+0000 7fa3edfb1740 10 client.15990 fetch_fsmap learned FSMap version 18 2024-05-22T23:05:09.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.671+0000 7fa3edfb1740 1 -- 172.21.15.84:0/1199255437 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x564dcb7d3f70 con 0x564dcba34570 2024-05-22T23:05:09.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.671+0000 7fa3c8ff9640 1 -- 172.21.15.84:0/1199255437 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d8019b10 con 0x564dcba34570 2024-05-22T23:05:09.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.671+0000 7fa3c8ff9640 1 -- 172.21.15.84:0/1199255437 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fa3d800eb50 con 0x564dcba34570 2024-05-22T23:05:09.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.671+0000 7fa3edfb1740 10 client.15990 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:05:09.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.671+0000 7fa3edfb1740 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:05:09.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.671+0000 7fa3edfb1740 20 client.15990 populate_metadata read hostname 'smithi084' 2024-05-22T23:05:09.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.671+0000 7fa3edfb1740 1 --2- 172.21.15.84:0/1199255437 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564dcba4ee10 0x564dcba51200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.671+0000 7fa3edfb1740 4 client.15990 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:05:09.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.671+0000 7fa3edfb1740 1 -- 172.21.15.84:0/1199255437 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564dcb9ab460 con 0x564dcba4ee10 2024-05-22T23:05:09.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.671+0000 7fa3e9041640 1 --2- 172.21.15.84:0/1199255437 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564dcba4ee10 0x564dcba51200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.671+0000 7fa3e9041640 1 --2- 172.21.15.84:0/1199255437 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564dcba4ee10 0x564dcba51200 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=620e4ee1aa83ebc server_cookie=b3a5ecb3d6194d7f in_seq=0 out_seq=0 2024-05-22T23:05:09.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.671+0000 7fa3c8ff9640 10 client.15990 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:09.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.674+0000 7fa3c8ff9640 1 -- 172.21.15.84:0/1199255437 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564dcb9ab460 con 0x564dcba4ee10 2024-05-22T23:05:09.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.674+0000 7fa3c8ff9640 10 client.15990 handle_command_reply: tid=0 2024-05-22T23:05:09.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.674+0000 7fa3edfb1740 1 client.15990 shutdown 2024-05-22T23:05:09.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.674+0000 7fa3edfb1740 1 -- 172.21.15.84:0/1199255437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba34570 msgr2=0x564dcba3c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:09.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.674+0000 7fa3edfb1740 1 --2- 172.21.15.84:0/1199255437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba34570 0x564dcba3c080 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa3d801c040 tx=0x7fa3d800e670 comp rx=0 tx=0).stop 2024-05-22T23:05:09.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.674+0000 7fa3edfb1740 1 -- 172.21.15.84:0/1199255437 shutdown_connections 2024-05-22T23:05:09.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.674+0000 7fa3edfb1740 1 --2- 172.21.15.84:0/1199255437 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564dcba4ee10 0x564dcba51200 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.674+0000 7fa3edfb1740 1 --2- 172.21.15.84:0/1199255437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564dcba3ec40 0x564dcba3c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.674+0000 7fa3edfb1740 1 --2- 172.21.15.84:0/1199255437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba34570 0x564dcba3c080 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.674+0000 7fa3c8ff9640 0 client.15990 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:09.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.674+0000 7fa3edfb1740 1 --2- 172.21.15.84:0/1199255437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564dcba33c50 0x564dcba3bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.674+0000 7fa3edfb1740 1 -- 172.21.15.84:0/1199255437 >> 172.21.15.84:0/1199255437 conn(0x564dcba2ff70 msgr2=0x564dcba35d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:09.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.675+0000 7fa3edfb1740 1 -- 172.21.15.84:0/1199255437 shutdown_connections 2024-05-22T23:05:09.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.675+0000 7fa3edfb1740 1 -- 172.21.15.84:0/1199255437 wait complete. 2024-05-22T23:05:09.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.675+0000 7fa3edfb1740 20 client.15990 trim_cache size 0 max 16384 2024-05-22T23:05:09.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.693+0000 7fa3edfb1740 1 Processor -- start 2024-05-22T23:05:09.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.693+0000 7fa3edfb1740 1 -- start start 2024-05-22T23:05:09.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.694+0000 7fa3edfb1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564dcba33c50 0x564dcba4a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.694+0000 7fa3edfb1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba34570 0x564dcba56970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.694+0000 7fa3edfb1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564dcba3ec40 0x564dcba56eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.694+0000 7fa3edfb1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e41ab620 con 0x564dcba34570 2024-05-22T23:05:09.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.694+0000 7fa3edfb1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e41adea0 con 0x564dcba33c50 2024-05-22T23:05:09.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.694+0000 7fa3edfb1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e41abc90 con 0x564dcba3ec40 2024-05-22T23:05:09.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.694+0000 7fa3e8840640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba34570 0x564dcba56970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.694+0000 7fa3e8840640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba34570 0x564dcba56970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58012/0 (socket says 172.21.15.84:58012) 2024-05-22T23:05:09.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.694+0000 7fa3e9842640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564dcba3ec40 0x564dcba56eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.694+0000 7fa3e8840640 1 -- 172.21.15.84:0/1292171853 learned_addr learned my addr 172.21.15.84:0/1292171853 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:09.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.694+0000 7fa3e9041640 1 --2- 172.21.15.84:0/1292171853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564dcba33c50 0x564dcba4a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.694+0000 7fa3e8840640 1 -- 172.21.15.84:0/1292171853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564dcba3ec40 msgr2=0x564dcba56eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:09.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.694+0000 7fa3e8840640 1 --2- 172.21.15.84:0/1292171853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564dcba3ec40 0x564dcba56eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.694+0000 7fa3e8840640 1 -- 172.21.15.84:0/1292171853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564dcba33c50 msgr2=0x564dcba4a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:09.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.694+0000 7fa3e8840640 1 --2- 172.21.15.84:0/1292171853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564dcba33c50 0x564dcba4a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.694+0000 7fa3e8840640 1 -- 172.21.15.84:0/1292171853 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dcb66b170 con 0x564dcba34570 2024-05-22T23:05:09.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.694+0000 7fa3e8840640 1 --2- 172.21.15.84:0/1292171853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba34570 0x564dcba56970 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa3d800eef0 tx=0x7fa3d8009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c12c9205a85cc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:09.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.694+0000 7fa3caffd640 1 -- 172.21.15.84:0/1292171853 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d801e040 con 0x564dcba34570 2024-05-22T23:05:09.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.695+0000 7fa3caffd640 1 -- 172.21.15.84:0/1292171853 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d801f040 con 0x564dcba34570 2024-05-22T23:05:09.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.695+0000 7fa3caffd640 1 -- 172.21.15.84:0/1292171853 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d800e3b0 con 0x564dcba34570 2024-05-22T23:05:09.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.695+0000 7fa3edfb1740 1 -- 172.21.15.84:0/1292171853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba34570 msgr2=0x564dcba56970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:09.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.695+0000 7fa3edfb1740 1 --2- 172.21.15.84:0/1292171853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba34570 0x564dcba56970 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa3d800eef0 tx=0x7fa3d8009f00 comp rx=0 tx=0).stop 2024-05-22T23:05:09.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.695+0000 7fa3edfb1740 1 -- 172.21.15.84:0/1292171853 shutdown_connections 2024-05-22T23:05:09.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.695+0000 7fa3edfb1740 1 --2- 172.21.15.84:0/1292171853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564dcba3ec40 0x564dcba56eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.695+0000 7fa3edfb1740 1 --2- 172.21.15.84:0/1292171853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba34570 0x564dcba56970 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.695+0000 7fa3edfb1740 1 --2- 172.21.15.84:0/1292171853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564dcba33c50 0x564dcba4a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.695+0000 7fa3edfb1740 1 -- 172.21.15.84:0/1292171853 >> 172.21.15.84:0/1292171853 conn(0x564dcba2ff70 msgr2=0x564dcba2f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:09.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.695+0000 7fa3edfb1740 1 -- 172.21.15.84:0/1292171853 shutdown_connections 2024-05-22T23:05:09.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.695+0000 7fa3edfb1740 1 -- 172.21.15.84:0/1292171853 wait complete. 2024-05-22T23:05:09.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.696+0000 7fa3edfb1740 1 -- start start 2024-05-22T23:05:09.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.696+0000 7fa3edfb1740 1 Processor -- start 2024-05-22T23:05:09.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.696+0000 7fa3edfb1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba33c50 0x564dcba42720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.696+0000 7fa3edfb1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564dcba34570 0x564dcba42c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.696+0000 7fa3edfb1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564dcba3ec40 0x564dcba431a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.696+0000 7fa3edfb1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e41ac360 con 0x564dcba33c50 2024-05-22T23:05:09.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.696+0000 7fa3edfb1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e41ad7d0 con 0x564dcba34570 2024-05-22T23:05:09.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.696+0000 7fa3edfb1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e41aca30 con 0x564dcba3ec40 2024-05-22T23:05:09.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.696+0000 7fa3edfb1740 10 client.0 _ll_register_callbacks cb 0x7fa3d801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:05:09.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.696+0000 7fa3e9041640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba33c50 0x564dcba42720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.696+0000 7fa3e9041640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba33c50 0x564dcba42720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58016/0 (socket says 172.21.15.84:58016) 2024-05-22T23:05:09.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.696+0000 7fa3e9041640 1 -- 172.21.15.84:0/2208001591 learned_addr learned my addr 172.21.15.84:0/2208001591 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:09.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.696+0000 7fa3e8840640 1 --2- 172.21.15.84:0/2208001591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564dcba34570 0x564dcba42c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.696+0000 7fa3e9842640 1 --2- 172.21.15.84:0/2208001591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564dcba3ec40 0x564dcba431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.697+0000 7fa3e9041640 1 -- 172.21.15.84:0/2208001591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564dcba3ec40 msgr2=0x564dcba431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:09.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.697+0000 7fa3e9041640 1 --2- 172.21.15.84:0/2208001591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564dcba3ec40 0x564dcba431a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.697+0000 7fa3e9041640 1 -- 172.21.15.84:0/2208001591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564dcba34570 msgr2=0x564dcba42c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:09.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.697+0000 7fa3e9041640 1 --2- 172.21.15.84:0/2208001591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564dcba34570 0x564dcba42c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.697+0000 7fa3e9041640 1 -- 172.21.15.84:0/2208001591 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dcb960930 con 0x564dcba33c50 2024-05-22T23:05:09.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.697+0000 7fa3e8840640 1 --2- 172.21.15.84:0/2208001591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564dcba34570 0x564dcba42c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:05:09.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.697+0000 7fa3e9041640 1 --2- 172.21.15.84:0/2208001591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba33c50 0x564dcba42720 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa3d40cd000 tx=0x7fa3d400c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d630f26af6364020 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:09.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.697+0000 7fa3c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:05:09.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.697+0000 7fa3c8ff9640 1 -- 172.21.15.84:0/2208001591 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d40b8070 con 0x564dcba33c50 2024-05-22T23:05:09.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.697+0000 7fa3c8ff9640 1 -- 172.21.15.84:0/2208001591 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fa3d4098390 con 0x564dcba33c50 2024-05-22T23:05:09.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.697+0000 7fa3edfb1740 1 -- 172.21.15.84:0/2208001591 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3d801f040 con 0x564dcba33c50 2024-05-22T23:05:09.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.697+0000 7fa3c8ff9640 1 -- 172.21.15.84:0/2208001591 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fa3d409bd20 con 0x564dcba33c50 2024-05-22T23:05:09.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.697+0000 7fa3edfb1740 10 client.15996 fetch_fsmap learned FSMap version 18 2024-05-22T23:05:09.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.697+0000 7fa3edfb1740 1 -- 172.21.15.84:0/2208001591 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x564dcbab5250 con 0x564dcba33c50 2024-05-22T23:05:09.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.697+0000 7fa3c8ff9640 1 -- 172.21.15.84:0/2208001591 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d801f040 con 0x564dcba33c50 2024-05-22T23:05:09.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.697+0000 7fa3c8ff9640 1 -- 172.21.15.84:0/2208001591 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fa3d409bf80 con 0x564dcba33c50 2024-05-22T23:05:09.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.697+0000 7fa3edfb1740 10 client.15996 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:05:09.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.697+0000 7fa3edfb1740 10 client.15996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:05:09.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.697+0000 7fa3edfb1740 20 client.15996 populate_metadata read hostname 'smithi084' 2024-05-22T23:05:09.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.697+0000 7fa3edfb1740 1 --2- 172.21.15.84:0/2208001591 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564dcbab5970 0x564dcbab7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.697+0000 7fa3edfb1740 4 client.15996 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-22T23:05:09.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.697+0000 7fa3edfb1740 1 -- 172.21.15.84:0/2208001591 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564dcb9a5170 con 0x564dcbab5970 2024-05-22T23:05:09.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.698+0000 7fa3e8840640 1 --2- 172.21.15.84:0/2208001591 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564dcbab5970 0x564dcbab7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.698+0000 7fa3c8ff9640 10 client.15996 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:09.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.698+0000 7fa3e8840640 1 --2- 172.21.15.84:0/2208001591 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564dcbab5970 0x564dcbab7d90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=818a658343f5b149 server_cookie=c1f7834cb9bf7110 in_seq=0 out_seq=0 2024-05-22T23:05:09.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.699+0000 7fa3c8ff9640 1 -- 172.21.15.84:0/2208001591 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564dcb9a5170 con 0x564dcbab5970 2024-05-22T23:05:09.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.699+0000 7fa3c8ff9640 10 client.15996 handle_command_reply: tid=0 2024-05-22T23:05:09.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.699+0000 7fa3edfb1740 1 client.15996 shutdown 2024-05-22T23:05:09.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.699+0000 7fa3edfb1740 1 -- 172.21.15.84:0/2208001591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba33c50 msgr2=0x564dcba42720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:09.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.699+0000 7fa3edfb1740 1 --2- 172.21.15.84:0/2208001591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba33c50 0x564dcba42720 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa3d40cd000 tx=0x7fa3d400c7d0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.699+0000 7fa3edfb1740 1 -- 172.21.15.84:0/2208001591 shutdown_connections 2024-05-22T23:05:09.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.699+0000 7fa3edfb1740 1 --2- 172.21.15.84:0/2208001591 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564dcbab5970 0x564dcbab7d90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.699+0000 7fa3edfb1740 1 --2- 172.21.15.84:0/2208001591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564dcba3ec40 0x564dcba431a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.699+0000 7fa3edfb1740 1 --2- 172.21.15.84:0/2208001591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564dcba34570 0x564dcba42c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.699+0000 7fa3c8ff9640 0 client.15996 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:09.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.699+0000 7fa3edfb1740 1 --2- 172.21.15.84:0/2208001591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564dcba33c50 0x564dcba42720 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.699+0000 7fa3edfb1740 1 -- 172.21.15.84:0/2208001591 >> 172.21.15.84:0/2208001591 conn(0x564dcba2ff70 msgr2=0x564dcba2e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:09.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.700+0000 7fa3edfb1740 1 -- 172.21.15.84:0/2208001591 shutdown_connections 2024-05-22T23:05:09.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.700+0000 7fa3edfb1740 1 -- 172.21.15.84:0/2208001591 wait complete. 2024-05-22T23:05:09.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.700+0000 7fa3edfb1740 20 client.15996 trim_cache size 0 max 16384 2024-05-22T23:05:09.710 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:05:09.710 INFO:teuthology.orchestra.run.smithi084.stdout: "return_code": 0, 2024-05-22T23:05:09.710 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_tag": "b848b380-d005-474e-851b-a4faed322900", 2024-05-22T23:05:09.710 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": "asynchronous" 2024-05-22T23:05:09.710 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:05:09.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.700+0000 7fa3eb2a5640 1 -- 172.21.15.84:0/1686697214 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fa3cc0a2d00 msgr2=0x7fa3cc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:09.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.700+0000 7fa3eb2a5640 1 --2- 172.21.15.84:0/1686697214 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fa3cc0a2d00 0x7fa3cc0a51c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc004ab0 tx=0x7fa3dc004a40 comp rx=0 tx=0).stop 2024-05-22T23:05:09.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.700+0000 7fa3eb2a5640 1 -- 172.21.15.84:0/1686697214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3e4100740 msgr2=0x7fa3e4199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:09.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.700+0000 7fa3eb2a5640 1 --2- 172.21.15.84:0/1686697214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3e4100740 0x7fa3e4199040 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400e9f0 tx=0x7fa3d400edf0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.700+0000 7fa3eb2a5640 1 -- 172.21.15.84:0/1686697214 shutdown_connections 2024-05-22T23:05:09.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.700+0000 7fa3eb2a5640 1 --2- 172.21.15.84:0/1686697214 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fa3cc0a2d00 0x7fa3cc0a51c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.700+0000 7fa3eb2a5640 1 --2- 172.21.15.84:0/1686697214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3e4106750 0x7fa3e41a40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.700+0000 7fa3eb2a5640 1 --2- 172.21.15.84:0/1686697214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3e4101100 0x7fa3e4199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.700+0000 7fa3eb2a5640 1 --2- 172.21.15.84:0/1686697214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3e4100740 0x7fa3e4199040 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.700+0000 7fa3eb2a5640 1 -- 172.21.15.84:0/1686697214 >> 172.21.15.84:0/1686697214 conn(0x7fa3e4077870 msgr2=0x7fa3e4102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:09.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.700+0000 7fa3eb2a5640 1 -- 172.21.15.84:0/1686697214 shutdown_connections 2024-05-22T23:05:09.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.700+0000 7fa3eb2a5640 1 -- 172.21.15.84:0/1686697214 wait complete. 2024-05-22T23:05:09.714 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:09 smithi084 ceph-mon[32186]: pgmap v102: 97 pgs: 97 active+clean; 166 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 276 op/s 2024-05-22T23:05:09.715 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:09 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/4201050800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:05:09.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:05:09.716 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:05:09.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:09 smithi096 ceph-mon[36392]: pgmap v102: 97 pgs: 97 active+clean; 166 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 276 op/s 2024-05-22T23:05:09.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:09 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/4201050800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:05:09.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.820+0000 7fece9d5a640 1 -- 172.21.15.84:0/2087241256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fece4102200 msgr2=0x7fece410e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:09.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.820+0000 7fece9d5a640 1 --2- 172.21.15.84:0/2087241256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fece4102200 0x7fece410e710 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009cd0 tx=0x7fecdc02a8e0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.820+0000 7fece9d5a640 1 -- 172.21.15.84:0/2087241256 shutdown_connections 2024-05-22T23:05:09.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.820+0000 7fece9d5a640 1 --2- 172.21.15.84:0/2087241256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fece4102200 0x7fece410e710 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.820+0000 7fece9d5a640 1 --2- 172.21.15.84:0/2087241256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fece4101840 0x7fece4101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.821+0000 7fece9d5a640 1 --2- 172.21.15.84:0/2087241256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fece4107850 0x7fece4107c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.821+0000 7fece9d5a640 1 -- 172.21.15.84:0/2087241256 >> 172.21.15.84:0/2087241256 conn(0x7fece4078cc0 msgr2=0x7fece40790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:09.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.821+0000 7fece9d5a640 1 -- 172.21.15.84:0/2087241256 shutdown_connections 2024-05-22T23:05:09.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.821+0000 7fece9d5a640 1 -- 172.21.15.84:0/2087241256 wait complete. 2024-05-22T23:05:09.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.821+0000 7fece9d5a640 1 Processor -- start 2024-05-22T23:05:09.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.821+0000 7fece9d5a640 1 -- start start 2024-05-22T23:05:09.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.821+0000 7fece9d5a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fece4101840 0x7fece4198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.821+0000 7fece9d5a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fece4102200 0x7fece41986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.821+0000 7fece9d5a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fece4107850 0x7fece419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.821+0000 7fece9d5a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fece4111c60 con 0x7fece4107850 2024-05-22T23:05:09.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.822+0000 7fece9d5a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fece4111ae0 con 0x7fece4101840 2024-05-22T23:05:09.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.822+0000 7fece9d5a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fece4111de0 con 0x7fece4102200 2024-05-22T23:05:09.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.822+0000 7fece3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fece4107850 0x7fece419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.822+0000 7fece37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fece4101840 0x7fece4198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.822+0000 7fece3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fece4107850 0x7fece419ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58022/0 (socket says 172.21.15.84:58022) 2024-05-22T23:05:09.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.822+0000 7fece37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fece4101840 0x7fece4198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:33934/0 (socket says 172.21.15.84:33934) 2024-05-22T23:05:09.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.822+0000 7fece3fff640 1 -- 172.21.15.84:0/1013246927 learned_addr learned my addr 172.21.15.84:0/1013246927 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:09.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.822+0000 7fece2ffd640 1 --2- 172.21.15.84:0/1013246927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fece4102200 0x7fece41986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.822+0000 7fece3fff640 1 -- 172.21.15.84:0/1013246927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fece4102200 msgr2=0x7fece41986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:09.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.822+0000 7fece3fff640 1 --2- 172.21.15.84:0/1013246927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fece4102200 0x7fece41986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.822+0000 7fece3fff640 1 -- 172.21.15.84:0/1013246927 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fece4101840 msgr2=0x7fece4198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:09.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.822+0000 7fece3fff640 1 --2- 172.21.15.84:0/1013246927 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fece4101840 0x7fece4198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.822+0000 7fece3fff640 1 -- 172.21.15.84:0/1013246927 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece419d170 con 0x7fece4107850 2024-05-22T23:05:09.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.822+0000 7fece37fe640 1 --2- 172.21.15.84:0/1013246927 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fece4101840 0x7fece4198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:05:09.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.822+0000 7fece3fff640 1 --2- 172.21.15.84:0/1013246927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fece4107850 0x7fece419ca40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fecdc005cc0 tx=0x7fecdc0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:09.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.822+0000 7fece0ff9640 1 -- 172.21.15.84:0/1013246927 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc003e00 con 0x7fece4107850 2024-05-22T23:05:09.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.822+0000 7fece0ff9640 1 -- 172.21.15.84:0/1013246927 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecdc004520 con 0x7fece4107850 2024-05-22T23:05:09.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.822+0000 7fece9d5a640 1 -- 172.21.15.84:0/1013246927 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece419d400 con 0x7fece4107850 2024-05-22T23:05:09.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.822+0000 7fece0ff9640 1 -- 172.21.15.84:0/1013246927 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc030db0 con 0x7fece4107850 2024-05-22T23:05:09.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.822+0000 7fece9d5a640 1 -- 172.21.15.84:0/1013246927 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece41a4cf0 con 0x7fece4107850 2024-05-22T23:05:09.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.825+0000 7fece0ff9640 1 -- 172.21.15.84:0/1013246927 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fecdc0046d0 con 0x7fece4107850 2024-05-22T23:05:09.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.825+0000 7fececa65740 1 Processor -- start 2024-05-22T23:05:09.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.825+0000 7fececa65740 1 -- start start 2024-05-22T23:05:09.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.825+0000 7fececa65740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d00d0c54a0 0x55d00d0c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.825+0000 7fececa65740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d00d0c3db0 0x55d00d0c4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.825+0000 7fececa65740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0ceb30 0x55d00d0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.825+0000 7fececa65740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d00d027db0 con 0x55d00d0ceb30 2024-05-22T23:05:09.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.825+0000 7fececa65740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d00cfc6360 con 0x55d00d0c3db0 2024-05-22T23:05:09.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.825+0000 7fececa65740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d00d023d70 con 0x55d00d0c54a0 2024-05-22T23:05:09.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.825+0000 7fece2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0ceb30 0x55d00d0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.825+0000 7fece2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0ceb30 0x55d00d0d0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58038/0 (socket says 172.21.15.84:58038) 2024-05-22T23:05:09.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.825+0000 7fece2ffd640 1 -- 172.21.15.84:0/2588817334 learned_addr learned my addr 172.21.15.84:0/2588817334 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:09.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.826+0000 7fece37fe640 1 --2- 172.21.15.84:0/2588817334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d00d0c3db0 0x55d00d0c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.826+0000 7fece2ffd640 1 --2- 172.21.15.84:0/2588817334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d00d0c54a0 0x55d00d0c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.826+0000 7fece0ff9640 1 --2- 172.21.15.84:0/1013246927 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7feccc0a2d00 0x7feccc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.826+0000 7fece3fff640 1 --2- 172.21.15.84:0/1013246927 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7feccc0a2d00 0x7feccc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.826+0000 7fece2ffd640 1 -- 172.21.15.84:0/2588817334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d00d0c54a0 msgr2=0x55d00d0c37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:09.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.826+0000 7fece2ffd640 1 --2- 172.21.15.84:0/2588817334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d00d0c54a0 0x55d00d0c37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.826+0000 7fece2ffd640 1 -- 172.21.15.84:0/2588817334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d00d0c3db0 msgr2=0x55d00d0c4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:09.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.826+0000 7fece2ffd640 1 --2- 172.21.15.84:0/2588817334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d00d0c3db0 0x55d00d0c4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.826+0000 7fece2ffd640 1 -- 172.21.15.84:0/2588817334 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d00cfe9a60 con 0x55d00d0ceb30 2024-05-22T23:05:09.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.826+0000 7fece37fe640 1 --2- 172.21.15.84:0/2588817334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d00d0c3db0 0x55d00d0c4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:05:09.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.826+0000 7fece0ff9640 1 -- 172.21.15.84:0/1013246927 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fecdc02e030 con 0x7fece4107850 2024-05-22T23:05:09.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.826+0000 7fece2ffd640 1 --2- 172.21.15.84:0/2588817334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0ceb30 0x55d00d0d0f20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fecd8009f90 tx=0x7fecd8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526cf3e5944e8db server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:09.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.826+0000 7fece3fff640 1 --2- 172.21.15.84:0/1013246927 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7feccc0a2d00 0x7feccc0a51c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0ee280 tx=0x7fecdc090010 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:09.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.826+0000 7fecc9ffb640 1 -- 172.21.15.84:0/2588817334 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8004030 con 0x55d00d0ceb30 2024-05-22T23:05:09.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.826+0000 7fecc9ffb640 1 -- 172.21.15.84:0/2588817334 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecd80041d0 con 0x55d00d0ceb30 2024-05-22T23:05:09.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.826+0000 7fecc9ffb640 1 -- 172.21.15.84:0/2588817334 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd800f740 con 0x55d00d0ceb30 2024-05-22T23:05:09.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.827+0000 7fececa65740 1 -- 172.21.15.84:0/2588817334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0ceb30 msgr2=0x55d00d0d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:09.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.827+0000 7fececa65740 1 --2- 172.21.15.84:0/2588817334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0ceb30 0x55d00d0d0f20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fecd8009f90 tx=0x7fecd8005850 comp rx=0 tx=0).stop 2024-05-22T23:05:09.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.827+0000 7fececa65740 1 -- 172.21.15.84:0/2588817334 shutdown_connections 2024-05-22T23:05:09.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.827+0000 7fececa65740 1 --2- 172.21.15.84:0/2588817334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0ceb30 0x55d00d0d0f20 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.827+0000 7fececa65740 1 --2- 172.21.15.84:0/2588817334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d00d0c3db0 0x55d00d0c4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.827+0000 7fececa65740 1 --2- 172.21.15.84:0/2588817334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d00d0c54a0 0x55d00d0c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.827+0000 7fececa65740 1 -- 172.21.15.84:0/2588817334 >> 172.21.15.84:0/2588817334 conn(0x55d00d08c8e0 msgr2=0x55d00d0c0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:09.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.827+0000 7fececa65740 1 -- 172.21.15.84:0/2588817334 shutdown_connections 2024-05-22T23:05:09.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.827+0000 7fececa65740 1 -- 172.21.15.84:0/2588817334 wait complete. 2024-05-22T23:05:09.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.827+0000 7fececa65740 1 -- start start 2024-05-22T23:05:09.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.827+0000 7fececa65740 1 Processor -- start 2024-05-22T23:05:09.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.828+0000 7fececa65740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0c3db0 0x55d00d0cba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.828+0000 7fececa65740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d00d0c54a0 0x55d00d0cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.828+0000 7fececa65740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d00d0ceb30 0x55d00d0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.828+0000 7fececa65740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d00d0d19c0 con 0x55d00d0c3db0 2024-05-22T23:05:09.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.828+0000 7fececa65740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d00cfe20f0 con 0x55d00d0ceb30 2024-05-22T23:05:09.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.828+0000 7fececa65740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d00d0d1b40 con 0x55d00d0c54a0 2024-05-22T23:05:09.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.828+0000 7fececa65740 10 client.0 _ll_register_callbacks cb 0x55d00cfe6f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:05:09.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.828+0000 7fece2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0c3db0 0x55d00d0cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.828+0000 7fece2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0c3db0 0x55d00d0cba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58050/0 (socket says 172.21.15.84:58050) 2024-05-22T23:05:09.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.828+0000 7fece2ffd640 1 -- 172.21.15.84:0/2587940554 learned_addr learned my addr 172.21.15.84:0/2587940554 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:09.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.828+0000 7fece2ffd640 1 -- 172.21.15.84:0/2587940554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d00d0c54a0 msgr2=0x55d00d0cbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:09.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.828+0000 7fece2ffd640 1 --2- 172.21.15.84:0/2587940554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d00d0c54a0 0x55d00d0cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.828+0000 7fece2ffd640 1 -- 172.21.15.84:0/2587940554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d00d0ceb30 msgr2=0x55d00d0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:09.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.828+0000 7fece2ffd640 1 --2- 172.21.15.84:0/2587940554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d00d0ceb30 0x55d00d0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.828+0000 7fece2ffd640 1 -- 172.21.15.84:0/2587940554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d00ce057d0 con 0x55d00d0c3db0 2024-05-22T23:05:09.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.828+0000 7fece2ffd640 1 --2- 172.21.15.84:0/2587940554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0c3db0 0x55d00d0cba80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fecd800f420 tx=0x7fecd80056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8564e5c530c08e51 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:09.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.828+0000 7fecc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:05:09.842 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.828+0000 7fecc2ffd640 1 -- 172.21.15.84:0/2587940554 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8005c50 con 0x55d00d0c3db0 2024-05-22T23:05:09.842 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.828+0000 7fecc2ffd640 1 -- 172.21.15.84:0/2587940554 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fecd800f4c0 con 0x55d00d0c3db0 2024-05-22T23:05:09.842 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.828+0000 7fececa65740 1 -- 172.21.15.84:0/2587940554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecd80041d0 con 0x55d00d0c3db0 2024-05-22T23:05:09.842 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.829+0000 7fecc2ffd640 1 -- 172.21.15.84:0/2587940554 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fecd800f6c0 con 0x55d00d0c3db0 2024-05-22T23:05:09.842 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.829+0000 7fececa65740 10 client.16008 fetch_fsmap learned FSMap version 18 2024-05-22T23:05:09.842 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.829+0000 7fececa65740 1 -- 172.21.15.84:0/2587940554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55d00cd3b3f0 con 0x55d00d0c3db0 2024-05-22T23:05:09.842 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.829+0000 7fecc2ffd640 1 -- 172.21.15.84:0/2587940554 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecd80041d0 con 0x55d00d0c3db0 2024-05-22T23:05:09.842 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.829+0000 7fecc2ffd640 1 -- 172.21.15.84:0/2587940554 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fecd8030860 con 0x55d00d0c3db0 2024-05-22T23:05:09.842 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.829+0000 7fececa65740 10 client.16008 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:05:09.842 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.829+0000 7fececa65740 10 client.16008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:05:09.842 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.829+0000 7fececa65740 20 client.16008 populate_metadata read hostname 'smithi084' 2024-05-22T23:05:09.842 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.829+0000 7fececa65740 1 --2- 172.21.15.84:0/2587940554 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d00d0deea0 0x55d00d0e1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.843 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.829+0000 7fececa65740 4 client.16008 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:05:09.843 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.829+0000 7fececa65740 1 -- 172.21.15.84:0/2587940554 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d00d0354e0 con 0x55d00d0deea0 2024-05-22T23:05:09.843 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.829+0000 7fece37fe640 1 --2- 172.21.15.84:0/2587940554 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d00d0deea0 0x55d00d0e1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.843 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.830+0000 7fece37fe640 1 --2- 172.21.15.84:0/2587940554 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d00d0deea0 0x55d00d0e1290 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebe8b1b6c437865f server_cookie=b8db8a594744952e in_seq=0 out_seq=0 2024-05-22T23:05:09.843 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.830+0000 7fecc2ffd640 10 client.16008 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:09.843 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.830+0000 7fecc2ffd640 1 -- 172.21.15.84:0/2587940554 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d00d0354e0 con 0x55d00d0deea0 2024-05-22T23:05:09.843 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.830+0000 7fecc2ffd640 10 client.16008 handle_command_reply: tid=0 2024-05-22T23:05:09.843 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.830+0000 7fececa65740 1 client.16008 shutdown 2024-05-22T23:05:09.843 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.830+0000 7fececa65740 1 -- 172.21.15.84:0/2587940554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0c3db0 msgr2=0x55d00d0cba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:09.843 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.830+0000 7fececa65740 1 --2- 172.21.15.84:0/2587940554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0c3db0 0x55d00d0cba80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fecd800f420 tx=0x7fecd80056d0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.843 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.831+0000 7fececa65740 1 -- 172.21.15.84:0/2587940554 shutdown_connections 2024-05-22T23:05:09.843 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.831+0000 7fececa65740 1 --2- 172.21.15.84:0/2587940554 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d00d0deea0 0x55d00d0e1290 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.844 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.831+0000 7fececa65740 1 --2- 172.21.15.84:0/2587940554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d00d0ceb30 0x55d00d0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.844 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.831+0000 7fececa65740 1 --2- 172.21.15.84:0/2587940554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d00d0c54a0 0x55d00d0cbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.844 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.831+0000 7fececa65740 1 --2- 172.21.15.84:0/2587940554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0c3db0 0x55d00d0cba80 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.844 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.831+0000 7fececa65740 1 -- 172.21.15.84:0/2587940554 >> 172.21.15.84:0/2587940554 conn(0x55d00d08c8e0 msgr2=0x55d00d0c6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:09.844 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.831+0000 7fecc2ffd640 0 client.16008 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:09.844 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.831+0000 7fececa65740 1 -- 172.21.15.84:0/2587940554 shutdown_connections 2024-05-22T23:05:09.844 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.831+0000 7fececa65740 1 -- 172.21.15.84:0/2587940554 wait complete. 2024-05-22T23:05:09.844 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.831+0000 7fececa65740 20 client.16008 trim_cache size 0 max 16384 2024-05-22T23:05:09.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.849+0000 7fececa65740 1 Processor -- start 2024-05-22T23:05:09.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.849+0000 7fececa65740 1 -- start start 2024-05-22T23:05:09.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.849+0000 7fececa65740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0c3db0 0x55d00d0da8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.849+0000 7fececa65740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d00d0c54a0 0x55d00d0bf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.849+0000 7fececa65740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d00d0ceb30 0x55d00d0e6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.849+0000 7fececa65740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fece41ab5d0 con 0x55d00d0c3db0 2024-05-22T23:05:09.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.849+0000 7fececa65740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fece41ade50 con 0x55d00d0c54a0 2024-05-22T23:05:09.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.849+0000 7fececa65740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fece41abc40 con 0x55d00d0ceb30 2024-05-22T23:05:09.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.850+0000 7fece2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0c3db0 0x55d00d0da8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.850+0000 7fece2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0c3db0 0x55d00d0da8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58064/0 (socket says 172.21.15.84:58064) 2024-05-22T23:05:09.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.850+0000 7fece2ffd640 1 -- 172.21.15.84:0/1750423036 learned_addr learned my addr 172.21.15.84:0/1750423036 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:09.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.850+0000 7fece37fe640 1 --2- 172.21.15.84:0/1750423036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d00d0c54a0 0x55d00d0bf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.850+0000 7fece2ffd640 1 -- 172.21.15.84:0/1750423036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d00d0ceb30 msgr2=0x55d00d0e6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:09.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.850+0000 7fece2ffd640 1 --2- 172.21.15.84:0/1750423036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d00d0ceb30 0x55d00d0e6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.850+0000 7fece2ffd640 1 -- 172.21.15.84:0/1750423036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d00d0c54a0 msgr2=0x55d00d0bf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:09.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.850+0000 7fece2ffd640 1 --2- 172.21.15.84:0/1750423036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d00d0c54a0 0x55d00d0bf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.850+0000 7fece2ffd640 1 -- 172.21.15.84:0/1750423036 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d00ccfb0f0 con 0x55d00d0c3db0 2024-05-22T23:05:09.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.850+0000 7fece2ffd640 1 --2- 172.21.15.84:0/1750423036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0c3db0 0x55d00d0da8f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fecd80097c0 tx=0x7fecd8009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9068789eb3ee58fc server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:09.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.850+0000 7fecc97fa640 1 -- 172.21.15.84:0/1750423036 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8005c50 con 0x55d00d0c3db0 2024-05-22T23:05:09.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.850+0000 7fecc97fa640 1 -- 172.21.15.84:0/1750423036 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecd8022070 con 0x55d00d0c3db0 2024-05-22T23:05:09.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.850+0000 7fecc97fa640 1 -- 172.21.15.84:0/1750423036 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8003bf0 con 0x55d00d0c3db0 2024-05-22T23:05:09.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.851+0000 7fececa65740 1 -- 172.21.15.84:0/1750423036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0c3db0 msgr2=0x55d00d0da8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:09.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.851+0000 7fececa65740 1 --2- 172.21.15.84:0/1750423036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0c3db0 0x55d00d0da8f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fecd80097c0 tx=0x7fecd8009d20 comp rx=0 tx=0).stop 2024-05-22T23:05:09.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.851+0000 7fececa65740 1 -- 172.21.15.84:0/1750423036 shutdown_connections 2024-05-22T23:05:09.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.851+0000 7fececa65740 1 --2- 172.21.15.84:0/1750423036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d00d0ceb30 0x55d00d0e6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.851+0000 7fececa65740 1 --2- 172.21.15.84:0/1750423036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d00d0c54a0 0x55d00d0bf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.851+0000 7fececa65740 1 --2- 172.21.15.84:0/1750423036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0c3db0 0x55d00d0da8f0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.851+0000 7fececa65740 1 -- 172.21.15.84:0/1750423036 >> 172.21.15.84:0/1750423036 conn(0x55d00d08c8e0 msgr2=0x55d00d0bebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:09.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.851+0000 7fececa65740 1 -- 172.21.15.84:0/1750423036 shutdown_connections 2024-05-22T23:05:09.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.851+0000 7fececa65740 1 -- 172.21.15.84:0/1750423036 wait complete. 2024-05-22T23:05:09.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.851+0000 7fececa65740 1 -- start start 2024-05-22T23:05:09.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.852+0000 7fececa65740 1 Processor -- start 2024-05-22T23:05:09.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.852+0000 7fececa65740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d00d0c3db0 0x55d00d0d70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.852+0000 7fececa65740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d00d0c54a0 0x55d00d0c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.852+0000 7fececa65740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0ceb30 0x55d00d0c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.852+0000 7fececa65740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fece41ac310 con 0x55d00d0ceb30 2024-05-22T23:05:09.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.852+0000 7fececa65740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fece41ad780 con 0x55d00d0c54a0 2024-05-22T23:05:09.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.852+0000 7fececa65740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fece41ac9e0 con 0x55d00d0c3db0 2024-05-22T23:05:09.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.852+0000 7fececa65740 10 client.0 _ll_register_callbacks cb 0x7fecd4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:05:09.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.852+0000 7fece2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0ceb30 0x55d00d0c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.852+0000 7fece37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d00d0c54a0 0x55d00d0c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.852+0000 7fece2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0ceb30 0x55d00d0c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58074/0 (socket says 172.21.15.84:58074) 2024-05-22T23:05:09.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.852+0000 7fece2ffd640 1 -- 172.21.15.84:0/1204215014 learned_addr learned my addr 172.21.15.84:0/1204215014 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:09.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.852+0000 7fece2ffd640 1 --2- 172.21.15.84:0/1204215014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d00d0c3db0 0x55d00d0d70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.852+0000 7fece37fe640 1 -- 172.21.15.84:0/1204215014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d00d0c3db0 msgr2=0x55d00d0d70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:09.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.852+0000 7fece37fe640 1 --2- 172.21.15.84:0/1204215014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d00d0c3db0 0x55d00d0d70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.852+0000 7fece37fe640 1 -- 172.21.15.84:0/1204215014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0ceb30 msgr2=0x55d00d0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:09.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.852+0000 7fece37fe640 1 --2- 172.21.15.84:0/1204215014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0ceb30 0x55d00d0c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.852+0000 7fece37fe640 1 -- 172.21.15.84:0/1204215014 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d00cfe6fe0 con 0x55d00d0c54a0 2024-05-22T23:05:09.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.852+0000 7fece2ffd640 1 --2- 172.21.15.84:0/1204215014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0ceb30 0x55d00d0c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:05:09.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.852+0000 7fece37fe640 1 --2- 172.21.15.84:0/1204215014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d00d0c54a0 0x55d00d0c83f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55d00d0d1c40 tx=0x7fecd4013980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=343c927a9877bfca server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:09.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.852+0000 7fecc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:05:09.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.853+0000 7fecc2ffd640 1 -- 172.21.15.84:0/1204215014 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd401c3a0 con 0x55d00d0c54a0 2024-05-22T23:05:09.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.853+0000 7fececa65740 1 -- 172.21.15.84:0/1204215014 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecd8022070 con 0x55d00d0c54a0 2024-05-22T23:05:09.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.853+0000 7fecc2ffd640 1 -- 172.21.15.84:0/1204215014 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fecd401c540 con 0x55d00d0c54a0 2024-05-22T23:05:09.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.853+0000 7fecc2ffd640 1 -- 172.21.15.84:0/1204215014 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fecd401c740 con 0x55d00d0c54a0 2024-05-22T23:05:09.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.853+0000 7fececa65740 10 client.25586 fetch_fsmap learned FSMap version 18 2024-05-22T23:05:09.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.853+0000 7fececa65740 1 -- 172.21.15.84:0/1204215014 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55d00cfb81c0 con 0x55d00d0c54a0 2024-05-22T23:05:09.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.853+0000 7fecc2ffd640 1 -- 172.21.15.84:0/1204215014 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecd8022070 con 0x55d00d0c54a0 2024-05-22T23:05:09.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.853+0000 7fecc2ffd640 1 -- 172.21.15.84:0/1204215014 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fecd402e510 con 0x55d00d0c54a0 2024-05-22T23:05:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.853+0000 7fececa65740 10 client.25586 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:05:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.853+0000 7fececa65740 10 client.25586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:05:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.853+0000 7fececa65740 20 client.25586 populate_metadata read hostname 'smithi084' 2024-05-22T23:05:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.853+0000 7fececa65740 1 --2- 172.21.15.84:0/1204215014 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d00d145450 0x55d00d147870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.853+0000 7fececa65740 4 client.25586 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:05:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.853+0000 7fececa65740 1 -- 172.21.15.84:0/1204215014 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d00d02f830 con 0x55d00d145450 2024-05-22T23:05:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.853+0000 7fece2ffd640 1 --2- 172.21.15.84:0/1204215014 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d00d145450 0x55d00d147870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.854+0000 7fecc2ffd640 10 client.25586 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.854+0000 7fece2ffd640 1 --2- 172.21.15.84:0/1204215014 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d00d145450 0x55d00d147870 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c95bdd1436b282d0 server_cookie=b79386838e7cf601 in_seq=0 out_seq=0 2024-05-22T23:05:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.854+0000 7fecc2ffd640 1 -- 172.21.15.84:0/1204215014 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d00d02f830 con 0x55d00d145450 2024-05-22T23:05:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.854+0000 7fecc2ffd640 10 client.25586 handle_command_reply: tid=0 2024-05-22T23:05:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.854+0000 7fececa65740 1 client.25586 shutdown 2024-05-22T23:05:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.854+0000 7fececa65740 1 -- 172.21.15.84:0/1204215014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d00d0c54a0 msgr2=0x55d00d0c83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.854+0000 7fececa65740 1 --2- 172.21.15.84:0/1204215014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d00d0c54a0 0x55d00d0c83f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55d00d0d1c40 tx=0x7fecd4013980 comp rx=0 tx=0).stop 2024-05-22T23:05:09.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.854+0000 7fececa65740 1 -- 172.21.15.84:0/1204215014 shutdown_connections 2024-05-22T23:05:09.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.854+0000 7fececa65740 1 --2- 172.21.15.84:0/1204215014 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d00d145450 0x55d00d147870 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.854+0000 7fececa65740 1 --2- 172.21.15.84:0/1204215014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d00d0ceb30 0x55d00d0c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.854+0000 7fececa65740 1 --2- 172.21.15.84:0/1204215014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d00d0c54a0 0x55d00d0c83f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.854+0000 7fececa65740 1 --2- 172.21.15.84:0/1204215014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d00d0c3db0 0x55d00d0d70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.854+0000 7fecc2ffd640 0 client.25586 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:09.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.854+0000 7fececa65740 1 -- 172.21.15.84:0/1204215014 >> 172.21.15.84:0/1204215014 conn(0x55d00d08c8e0 msgr2=0x55d00d0cba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:09.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.854+0000 7fececa65740 1 -- 172.21.15.84:0/1204215014 shutdown_connections 2024-05-22T23:05:09.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.854+0000 7fececa65740 1 -- 172.21.15.84:0/1204215014 wait complete. 2024-05-22T23:05:09.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.854+0000 7fececa65740 20 client.25586 trim_cache size 0 max 16384 2024-05-22T23:05:09.865 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:05:09.866 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-22T23:05:09.866 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": { 2024-05-22T23:05:09.866 INFO:teuthology.orchestra.run.smithi084.stdout: "b848b380-d005-474e-851b-a4faed322900": { 2024-05-22T23:05:09.866 INFO:teuthology.orchestra.run.smithi084.stdout: "path": "/", 2024-05-22T23:05:09.866 INFO:teuthology.orchestra.run.smithi084.stdout: "tag": "b848b380-d005-474e-851b-a4faed322900", 2024-05-22T23:05:09.866 INFO:teuthology.orchestra.run.smithi084.stdout: "options": "recursive,force" 2024-05-22T23:05:09.866 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:05:09.866 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:05:09.866 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:05:09.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.855+0000 7fece9d5a640 1 -- 172.21.15.84:0/1013246927 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7feccc0a2d00 msgr2=0x7feccc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:09.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.855+0000 7fece9d5a640 1 --2- 172.21.15.84:0/1013246927 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7feccc0a2d00 0x7feccc0a51c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0ee280 tx=0x7fecdc090010 comp rx=0 tx=0).stop 2024-05-22T23:05:09.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.855+0000 7fece9d5a640 1 -- 172.21.15.84:0/1013246927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fece4107850 msgr2=0x7fece419ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:09.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.855+0000 7fece9d5a640 1 --2- 172.21.15.84:0/1013246927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fece4107850 0x7fece419ca40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fecdc005cc0 tx=0x7fecdc0093d0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.855+0000 7fece9d5a640 1 -- 172.21.15.84:0/1013246927 shutdown_connections 2024-05-22T23:05:09.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.855+0000 7fece9d5a640 1 --2- 172.21.15.84:0/1013246927 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7feccc0a2d00 0x7feccc0a51c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.855+0000 7fece9d5a640 1 --2- 172.21.15.84:0/1013246927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fece4107850 0x7fece419ca40 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.855+0000 7fece9d5a640 1 --2- 172.21.15.84:0/1013246927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fece4102200 0x7fece41986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.855+0000 7fece9d5a640 1 --2- 172.21.15.84:0/1013246927 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fece4101840 0x7fece4198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:09.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.855+0000 7fece9d5a640 1 -- 172.21.15.84:0/1013246927 >> 172.21.15.84:0/1013246927 conn(0x7fece4078cc0 msgr2=0x7fece40ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:09.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.855+0000 7fece9d5a640 1 -- 172.21.15.84:0/1013246927 shutdown_connections 2024-05-22T23:05:09.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:09.855+0000 7fece9d5a640 1 -- 172.21.15.84:0/1013246927 wait complete. 2024-05-22T23:05:09.870 INFO:tasks.cephfs.filesystem:scrub status for tag:b848b380-d005-474e-851b-a4faed322900 - {'path': '/', 'tag': 'b848b380-d005-474e-851b-a4faed322900', 'options': 'recursive,force'} 2024-05-22T23:05:10.429 INFO:tasks.workunit.client.0.smithi084.stdout: 1 340905 16.01 MB/sec warmup 116 sec latency 24.570 ms 2024-05-22T23:05:10.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:10 smithi204 ceph-mon[36211]: scrub queued for path: / 2024-05-22T23:05:10.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:10 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:05:10.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:10 smithi204 ceph-mon[36211]: scrub summary: active paths [/] 2024-05-22T23:05:10.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:10 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:05:10.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:10 smithi096 ceph-mon[36392]: scrub queued for path: / 2024-05-22T23:05:10.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:10 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:05:10.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:10 smithi096 ceph-mon[36392]: scrub summary: active paths [/] 2024-05-22T23:05:10.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:10 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:05:10.877 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:10 smithi084 ceph-mon[32186]: scrub queued for path: / 2024-05-22T23:05:10.877 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:10 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:05:10.877 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:10 smithi084 ceph-mon[32186]: scrub summary: active paths [/] 2024-05-22T23:05:10.877 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:10 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:05:11.429 INFO:tasks.workunit.client.0.smithi084.stdout: 1 344072 16.02 MB/sec warmup 117 sec latency 18.232 ms 2024-05-22T23:05:11.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:11 smithi204 ceph-mon[36211]: pgmap v103: 97 pgs: 97 active+clean; 166 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 276 op/s 2024-05-22T23:05:11.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:11 smithi204 ceph-mon[36211]: scrub summary: idle 2024-05-22T23:05:11.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:11 smithi096 ceph-mon[36392]: pgmap v103: 97 pgs: 97 active+clean; 166 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 276 op/s 2024-05-22T23:05:11.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:11 smithi096 ceph-mon[36392]: scrub summary: idle 2024-05-22T23:05:11.877 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:11 smithi084 ceph-mon[32186]: pgmap v103: 97 pgs: 97 active+clean; 166 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 276 op/s 2024-05-22T23:05:11.877 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:11 smithi084 ceph-mon[32186]: scrub summary: idle 2024-05-22T23:05:12.429 INFO:tasks.workunit.client.0.smithi084.stdout: 1 346628 15.96 MB/sec warmup 118 sec latency 68.682 ms 2024-05-22T23:05:13.429 INFO:tasks.workunit.client.0.smithi084.stdout: 1 349035 15.97 MB/sec warmup 119 sec latency 75.585 ms 2024-05-22T23:05:13.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:13 smithi204 ceph-mon[36211]: pgmap v104: 97 pgs: 97 active+clean; 174 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.6 MiB/s wr, 401 op/s 2024-05-22T23:05:13.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:13 smithi096 ceph-mon[36392]: pgmap v104: 97 pgs: 97 active+clean; 174 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.6 MiB/s wr, 401 op/s 2024-05-22T23:05:13.877 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:13 smithi084 ceph-mon[32186]: pgmap v104: 97 pgs: 97 active+clean; 174 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.6 MiB/s wr, 401 op/s 2024-05-22T23:05:15.429 INFO:tasks.workunit.client.0.smithi084.stdout: 1 355239 18.61 MB/sec execute 1 sec latency 24.280 ms 2024-05-22T23:05:15.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:15 smithi204 ceph-mon[36211]: pgmap v105: 97 pgs: 97 active+clean; 174 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 258 op/s 2024-05-22T23:05:15.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:15 smithi096 ceph-mon[36392]: pgmap v105: 97 pgs: 97 active+clean; 174 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 258 op/s 2024-05-22T23:05:15.877 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:15 smithi084 ceph-mon[32186]: pgmap v105: 97 pgs: 97 active+clean; 174 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 258 op/s 2024-05-22T23:05:16.430 INFO:tasks.workunit.client.0.smithi084.stdout: 1 358486 18.70 MB/sec execute 2 sec latency 18.198 ms 2024-05-22T23:05:17.430 INFO:tasks.workunit.client.0.smithi084.stdout: 1 361790 18.39 MB/sec execute 3 sec latency 14.072 ms 2024-05-22T23:05:17.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:17 smithi204 ceph-mon[36211]: pgmap v106: 97 pgs: 97 active+clean; 181 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.6 MiB/s wr, 388 op/s 2024-05-22T23:05:17.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:17 smithi096 ceph-mon[36392]: pgmap v106: 97 pgs: 97 active+clean; 181 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.6 MiB/s wr, 388 op/s 2024-05-22T23:05:17.877 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:17 smithi084 ceph-mon[32186]: pgmap v106: 97 pgs: 97 active+clean; 181 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.6 MiB/s wr, 388 op/s 2024-05-22T23:05:18.430 INFO:tasks.workunit.client.0.smithi084.stdout: 1 363992 15.66 MB/sec execute 4 sec latency 87.353 ms 2024-05-22T23:05:19.430 INFO:tasks.workunit.client.0.smithi084.stdout: 1 366601 15.62 MB/sec execute 5 sec latency 20.516 ms 2024-05-22T23:05:19.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:19 smithi204 ceph-mon[36211]: pgmap v107: 97 pgs: 97 active+clean; 181 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 254 op/s 2024-05-22T23:05:19.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:19 smithi096 ceph-mon[36392]: pgmap v107: 97 pgs: 97 active+clean; 181 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 254 op/s 2024-05-22T23:05:19.877 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:19 smithi084 ceph-mon[32186]: pgmap v107: 97 pgs: 97 active+clean; 181 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 254 op/s 2024-05-22T23:05:20.430 INFO:tasks.workunit.client.0.smithi084.stdout: 1 369673 15.91 MB/sec execute 6 sec latency 14.279 ms 2024-05-22T23:05:21.430 INFO:tasks.workunit.client.0.smithi084.stdout: 1 372847 16.31 MB/sec execute 7 sec latency 10.376 ms 2024-05-22T23:05:21.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:21 smithi204 ceph-mon[36211]: pgmap v108: 97 pgs: 97 active+clean; 181 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 254 op/s 2024-05-22T23:05:21.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:21 smithi096 ceph-mon[36392]: pgmap v108: 97 pgs: 97 active+clean; 181 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 254 op/s 2024-05-22T23:05:21.877 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:21 smithi084 ceph-mon[32186]: pgmap v108: 97 pgs: 97 active+clean; 181 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 254 op/s 2024-05-22T23:05:22.430 INFO:tasks.workunit.client.0.smithi084.stdout: 1 376198 16.63 MB/sec execute 8 sec latency 24.126 ms 2024-05-22T23:05:23.430 INFO:tasks.workunit.client.0.smithi084.stdout: 1 378382 15.59 MB/sec execute 9 sec latency 75.073 ms 2024-05-22T23:05:23.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:23 smithi204 ceph-mon[36211]: pgmap v109: 97 pgs: 97 active+clean; 189 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.9 MiB/s wr, 384 op/s 2024-05-22T23:05:23.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:23 smithi096 ceph-mon[36392]: pgmap v109: 97 pgs: 97 active+clean; 189 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.9 MiB/s wr, 384 op/s 2024-05-22T23:05:23.877 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:23 smithi084 ceph-mon[32186]: pgmap v109: 97 pgs: 97 active+clean; 189 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.9 MiB/s wr, 384 op/s 2024-05-22T23:05:24.430 INFO:tasks.workunit.client.0.smithi084.stdout: 1 381032 15.76 MB/sec execute 10 sec latency 13.736 ms 2024-05-22T23:05:25.430 INFO:tasks.workunit.client.0.smithi084.stdout: 1 383895 15.64 MB/sec execute 11 sec latency 23.147 ms 2024-05-22T23:05:25.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:25 smithi204 ceph-mon[36211]: pgmap v110: 97 pgs: 97 active+clean; 189 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 259 op/s 2024-05-22T23:05:25.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:25 smithi096 ceph-mon[36392]: pgmap v110: 97 pgs: 97 active+clean; 189 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 259 op/s 2024-05-22T23:05:25.877 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:25 smithi084 ceph-mon[32186]: pgmap v110: 97 pgs: 97 active+clean; 189 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 259 op/s 2024-05-22T23:05:26.430 INFO:tasks.workunit.client.0.smithi084.stdout: 1 387104 15.90 MB/sec execute 12 sec latency 14.250 ms 2024-05-22T23:05:27.430 INFO:tasks.workunit.client.0.smithi084.stdout: 1 390365 16.05 MB/sec execute 13 sec latency 16.269 ms 2024-05-22T23:05:27.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:27 smithi204 ceph-mon[36211]: pgmap v111: 97 pgs: 97 active+clean; 198 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.0 MiB/s wr, 385 op/s 2024-05-22T23:05:27.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:27 smithi096 ceph-mon[36392]: pgmap v111: 97 pgs: 97 active+clean; 198 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.0 MiB/s wr, 385 op/s 2024-05-22T23:05:27.877 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:27 smithi084 ceph-mon[32186]: pgmap v111: 97 pgs: 97 active+clean; 198 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.0 MiB/s wr, 385 op/s 2024-05-22T23:05:28.430 INFO:tasks.workunit.client.0.smithi084.stdout: 1 392698 15.55 MB/sec execute 14 sec latency 51.514 ms 2024-05-22T23:05:29.430 INFO:tasks.workunit.client.0.smithi084.stdout: 1 395657 15.63 MB/sec execute 15 sec latency 13.800 ms 2024-05-22T23:05:29.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:29 smithi204 ceph-mon[36211]: pgmap v112: 97 pgs: 97 active+clean; 198 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 255 op/s 2024-05-22T23:05:29.679 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:29 smithi084 ceph-mon[32186]: pgmap v112: 97 pgs: 97 active+clean; 198 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 255 op/s 2024-05-22T23:05:29.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:29 smithi096 ceph-mon[36392]: pgmap v112: 97 pgs: 97 active+clean; 198 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 255 op/s 2024-05-22T23:05:30.430 INFO:tasks.workunit.client.0.smithi084.stdout: 1 398290 15.56 MB/sec execute 16 sec latency 15.224 ms 2024-05-22T23:05:31.430 INFO:tasks.workunit.client.0.smithi084.stdout: 1 401337 15.72 MB/sec execute 17 sec latency 18.317 ms 2024-05-22T23:05:31.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:31 smithi204 ceph-mon[36211]: pgmap v113: 97 pgs: 97 active+clean; 198 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 255 op/s 2024-05-22T23:05:31.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:31 smithi096 ceph-mon[36392]: pgmap v113: 97 pgs: 97 active+clean; 198 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 255 op/s 2024-05-22T23:05:31.877 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:31 smithi084 ceph-mon[32186]: pgmap v113: 97 pgs: 97 active+clean; 198 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 255 op/s 2024-05-22T23:05:32.430 INFO:tasks.workunit.client.0.smithi084.stdout: 1 404534 15.84 MB/sec execute 18 sec latency 23.725 ms 2024-05-22T23:05:33.430 INFO:tasks.workunit.client.0.smithi084.stdout: 1 406698 15.43 MB/sec execute 19 sec latency 112.030 ms 2024-05-22T23:05:33.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:33 smithi204 ceph-mon[36211]: pgmap v114: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.0 MiB/s wr, 382 op/s 2024-05-22T23:05:33.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:33 smithi096 ceph-mon[36392]: pgmap v114: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.0 MiB/s wr, 382 op/s 2024-05-22T23:05:33.877 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:33 smithi084 ceph-mon[32186]: pgmap v114: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.0 MiB/s wr, 382 op/s 2024-05-22T23:05:34.430 INFO:tasks.workunit.client.0.smithi084.stdout: 1 408885 15.32 MB/sec execute 20 sec latency 13.929 ms 2024-05-22T23:05:35.430 INFO:tasks.workunit.client.0.smithi084.stdout: 1 411975 15.46 MB/sec execute 21 sec latency 11.722 ms 2024-05-22T23:05:35.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:35 smithi204 ceph-mon[36211]: pgmap v115: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 252 op/s 2024-05-22T23:05:35.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:35 smithi096 ceph-mon[36392]: pgmap v115: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 252 op/s 2024-05-22T23:05:36.081 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:35 smithi084 ceph-mon[32186]: pgmap v115: 97 pgs: 97 active+clean; 205 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 252 op/s 2024-05-22T23:05:36.430 INFO:tasks.workunit.client.0.smithi084.stdout: 1 415278 15.61 MB/sec execute 22 sec latency 14.944 ms 2024-05-22T23:05:37.431 INFO:tasks.workunit.client.0.smithi084.stdout: 1 418235 15.52 MB/sec execute 23 sec latency 16.850 ms 2024-05-22T23:05:37.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:37 smithi204 ceph-mon[36211]: pgmap v116: 97 pgs: 97 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.6 MiB/s wr, 371 op/s 2024-05-22T23:05:37.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:37 smithi096 ceph-mon[36392]: pgmap v116: 97 pgs: 97 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.6 MiB/s wr, 371 op/s 2024-05-22T23:05:37.877 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:37 smithi084 ceph-mon[32186]: pgmap v116: 97 pgs: 97 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.6 MiB/s wr, 371 op/s 2024-05-22T23:05:38.431 INFO:tasks.workunit.client.0.smithi084.stdout: 1 420052 15.33 MB/sec execute 24 sec latency 72.162 ms 2024-05-22T23:05:39.431 INFO:tasks.workunit.client.0.smithi084.stdout: 1 423065 15.31 MB/sec execute 25 sec latency 14.702 ms 2024-05-22T23:05:39.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:39 smithi096 ceph-mon[36392]: pgmap v117: 97 pgs: 97 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 246 op/s 2024-05-22T23:05:39.871 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:05:39.871 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:05:39.877 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:39 smithi084 ceph-mon[32186]: pgmap v117: 97 pgs: 97 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 246 op/s 2024-05-22T23:05:39.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:39 smithi204 ceph-mon[36211]: pgmap v117: 97 pgs: 97 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 246 op/s 2024-05-22T23:05:39.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.974+0000 7fbea3cba640 1 -- 172.21.15.84:0/3916571606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe9c107850 msgr2=0x7fbe9c107c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:39.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.974+0000 7fbea3cba640 1 --2- 172.21.15.84:0/3916571606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe9c107850 0x7fbe9c107c30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009930 tx=0x7fbe9802a420 comp rx=0 tx=0).stop 2024-05-22T23:05:39.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.974+0000 7fbea3cba640 1 -- 172.21.15.84:0/3916571606 shutdown_connections 2024-05-22T23:05:39.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.974+0000 7fbea3cba640 1 --2- 172.21.15.84:0/3916571606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe9c102200 0x7fbe9c10e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:39.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.974+0000 7fbea3cba640 1 --2- 172.21.15.84:0/3916571606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe9c101840 0x7fbe9c101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:39.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.974+0000 7fbea3cba640 1 --2- 172.21.15.84:0/3916571606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe9c107850 0x7fbe9c107c30 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:39.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.974+0000 7fbea3cba640 1 -- 172.21.15.84:0/3916571606 >> 172.21.15.84:0/3916571606 conn(0x7fbe9c078cc0 msgr2=0x7fbe9c0790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:39.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.974+0000 7fbea3cba640 1 -- 172.21.15.84:0/3916571606 shutdown_connections 2024-05-22T23:05:39.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.974+0000 7fbea3cba640 1 -- 172.21.15.84:0/3916571606 wait complete. 2024-05-22T23:05:39.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.975+0000 7fbea3cba640 1 Processor -- start 2024-05-22T23:05:39.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.975+0000 7fbea3cba640 1 -- start start 2024-05-22T23:05:39.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.975+0000 7fbea3cba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe9c101840 0x7fbe9c1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:39.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.975+0000 7fbea3cba640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe9c102200 0x7fbe9c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:39.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.975+0000 7fbea3cba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe9c107850 0x7fbe9c19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:39.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.975+0000 7fbea3cba640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c111c60 con 0x7fbe9c107850 2024-05-22T23:05:39.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.975+0000 7fbea3cba640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c111ae0 con 0x7fbe9c101840 2024-05-22T23:05:39.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.975+0000 7fbea3cba640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c111de0 con 0x7fbe9c102200 2024-05-22T23:05:39.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.975+0000 7fbea2257640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe9c107850 0x7fbe9c19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:39.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.975+0000 7fbea1a56640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe9c101840 0x7fbe9c1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:39.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.975+0000 7fbea2257640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe9c107850 0x7fbe9c19ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39614/0 (socket says 172.21.15.84:39614) 2024-05-22T23:05:39.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.975+0000 7fbea2257640 1 -- 172.21.15.84:0/2018398818 learned_addr learned my addr 172.21.15.84:0/2018398818 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:39.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.975+0000 7fbea1255640 1 --2- 172.21.15.84:0/2018398818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe9c102200 0x7fbe9c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:39.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.975+0000 7fbea2257640 1 -- 172.21.15.84:0/2018398818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe9c102200 msgr2=0x7fbe9c198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:39.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.975+0000 7fbea2257640 1 --2- 172.21.15.84:0/2018398818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe9c102200 0x7fbe9c198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:39.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.975+0000 7fbea2257640 1 -- 172.21.15.84:0/2018398818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe9c101840 msgr2=0x7fbe9c1981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:39.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.975+0000 7fbea2257640 1 --2- 172.21.15.84:0/2018398818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe9c101840 0x7fbe9c1981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:39.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.975+0000 7fbea2257640 1 -- 172.21.15.84:0/2018398818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9c19d1b0 con 0x7fbe9c107850 2024-05-22T23:05:39.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.976+0000 7fbea1a56640 1 --2- 172.21.15.84:0/2018398818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe9c101840 0x7fbe9c1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:05:39.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.976+0000 7fbea2257640 1 --2- 172.21.15.84:0/2018398818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe9c107850 0x7fbe9c19ca80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400eb10 tx=0x7fbe9400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:39.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.976+0000 7fbe92ffd640 1 -- 172.21.15.84:0/2018398818 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94004b20 con 0x7fbe9c107850 2024-05-22T23:05:39.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.976+0000 7fbe92ffd640 1 -- 172.21.15.84:0/2018398818 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe94004cc0 con 0x7fbe9c107850 2024-05-22T23:05:39.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.976+0000 7fbea3cba640 1 -- 172.21.15.84:0/2018398818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe9c19d4a0 con 0x7fbe9c107850 2024-05-22T23:05:39.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.976+0000 7fbe92ffd640 1 -- 172.21.15.84:0/2018398818 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94010760 con 0x7fbe9c107850 2024-05-22T23:05:39.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.976+0000 7fbea3cba640 1 -- 172.21.15.84:0/2018398818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe9c1a4d30 con 0x7fbe9c107850 2024-05-22T23:05:39.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.978+0000 7fbe92ffd640 1 -- 172.21.15.84:0/2018398818 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbe94005020 con 0x7fbe9c107850 2024-05-22T23:05:39.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.978+0000 7fbea69c8740 1 Processor -- start 2024-05-22T23:05:39.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.979+0000 7fbea69c8740 1 -- start start 2024-05-22T23:05:39.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.979+0000 7fbe92ffd640 1 --2- 172.21.15.84:0/2018398818 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fbe880a2d00 0x7fbe880a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:39.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.979+0000 7fbea69c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8d3e1a4a0 0x55c8d3e187e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:39.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.979+0000 7fbea69c8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e18db0 0x55c8d3e19230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:39.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.979+0000 7fbea69c8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8d3e23b30 0x55c8d3e25f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:39.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.979+0000 7fbea69c8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c8d3d7cdb0 con 0x55c8d3e18db0 2024-05-22T23:05:39.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.979+0000 7fbea69c8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c8d3d1b360 con 0x55c8d3e23b30 2024-05-22T23:05:39.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.979+0000 7fbea69c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c8d3d78d70 con 0x55c8d3e1a4a0 2024-05-22T23:05:39.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.979+0000 7fbea1255640 1 --2- 172.21.15.84:0/2018398818 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fbe880a2d00 0x7fbe880a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:39.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.979+0000 7fbea1a56640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8d3e1a4a0 0x55c8d3e187e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:39.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.979+0000 7fbea1a56640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8d3e1a4a0 0x55c8d3e187e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:37640/0 (socket says 172.21.15.84:37640) 2024-05-22T23:05:39.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.979+0000 7fbea1a56640 1 -- 172.21.15.84:0/1035562941 learned_addr learned my addr 172.21.15.84:0/1035562941 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:39.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.979+0000 7fbea2257640 1 --2- 172.21.15.84:0/1035562941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8d3e23b30 0x55c8d3e25f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:39.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.979+0000 7fbe92ffd640 1 -- 172.21.15.84:0/2018398818 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fbe940cf940 con 0x7fbe9c107850 2024-05-22T23:05:39.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.979+0000 7fbea1255640 1 --2- 172.21.15.84:0/1035562941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e18db0 0x55c8d3e19230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:39.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.979+0000 7fbea1a56640 1 -- 172.21.15.84:0/1035562941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8d3e23b30 msgr2=0x55c8d3e25f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:39.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.979+0000 7fbea1a56640 1 --2- 172.21.15.84:0/1035562941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8d3e23b30 0x55c8d3e25f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:39.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.979+0000 7fbea1a56640 1 -- 172.21.15.84:0/1035562941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e18db0 msgr2=0x55c8d3e19230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:39.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.979+0000 7fbea1a56640 1 --2- 172.21.15.84:0/1035562941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e18db0 0x55c8d3e19230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:39.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.979+0000 7fbea1a56640 1 -- 172.21.15.84:0/1035562941 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8d3d3ea60 con 0x55c8d3e1a4a0 2024-05-22T23:05:39.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.979+0000 7fbea2257640 1 --2- 172.21.15.84:0/1035562941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8d3e23b30 0x55c8d3e25f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:05:39.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.980+0000 7fbea1a56640 1 --2- 172.21.15.84:0/1035562941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8d3e1a4a0 0x55c8d3e187e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbe98000c00 tx=0x7fbe98005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15f9eadcbd6d5314 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:39.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.980+0000 7fbea1255640 1 --2- 172.21.15.84:0/2018398818 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fbe880a2d00 0x7fbe880a51c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009fd0 tx=0x7fbe8c009b80 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:39.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.980+0000 7fbe87fff640 1 -- 172.21.15.84:0/1035562941 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98035040 con 0x55c8d3e1a4a0 2024-05-22T23:05:39.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.980+0000 7fbe87fff640 1 -- 172.21.15.84:0/1035562941 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe98031830 con 0x55c8d3e1a4a0 2024-05-22T23:05:39.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.980+0000 7fbe87fff640 1 -- 172.21.15.84:0/1035562941 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98003d40 con 0x55c8d3e1a4a0 2024-05-22T23:05:39.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.981+0000 7fbea69c8740 1 -- 172.21.15.84:0/1035562941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8d3e1a4a0 msgr2=0x55c8d3e187e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:39.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.981+0000 7fbea69c8740 1 --2- 172.21.15.84:0/1035562941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8d3e1a4a0 0x55c8d3e187e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbe98000c00 tx=0x7fbe98005d50 comp rx=0 tx=0).stop 2024-05-22T23:05:39.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.981+0000 7fbea69c8740 1 -- 172.21.15.84:0/1035562941 shutdown_connections 2024-05-22T23:05:39.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.981+0000 7fbea69c8740 1 --2- 172.21.15.84:0/1035562941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8d3e23b30 0x55c8d3e25f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:39.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.981+0000 7fbea69c8740 1 --2- 172.21.15.84:0/1035562941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e18db0 0x55c8d3e19230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:39.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.981+0000 7fbea69c8740 1 --2- 172.21.15.84:0/1035562941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8d3e1a4a0 0x55c8d3e187e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:39.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.981+0000 7fbea69c8740 1 -- 172.21.15.84:0/1035562941 >> 172.21.15.84:0/1035562941 conn(0x55c8d3de18e0 msgr2=0x55c8d3e15310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:39.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.981+0000 7fbea69c8740 1 -- 172.21.15.84:0/1035562941 shutdown_connections 2024-05-22T23:05:39.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.981+0000 7fbea69c8740 1 -- 172.21.15.84:0/1035562941 wait complete. 2024-05-22T23:05:39.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.981+0000 7fbea69c8740 1 -- start start 2024-05-22T23:05:39.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.981+0000 7fbea69c8740 1 Processor -- start 2024-05-22T23:05:39.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.981+0000 7fbea69c8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8d3e18db0 0x55c8d3e209e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:39.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.981+0000 7fbea69c8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e1a4a0 0x55c8d3e20f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:39.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.981+0000 7fbea69c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8d3e23b30 0x55c8d3e21460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:39.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.981+0000 7fbea69c8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c8d3e26960 con 0x55c8d3e1a4a0 2024-05-22T23:05:39.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.981+0000 7fbea69c8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c8d3d370f0 con 0x55c8d3e18db0 2024-05-22T23:05:39.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.982+0000 7fbea69c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c8d3e26ae0 con 0x55c8d3e23b30 2024-05-22T23:05:39.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.982+0000 7fbea69c8740 10 client.0 _ll_register_callbacks cb 0x55c8d3d3bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:05:39.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.982+0000 7fbea1255640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e1a4a0 0x55c8d3e20f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:39.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.982+0000 7fbea1a56640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8d3e18db0 0x55c8d3e209e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:39.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.982+0000 7fbea1a56640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8d3e18db0 0x55c8d3e209e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:56800/0 (socket says 172.21.15.84:56800) 2024-05-22T23:05:39.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.982+0000 7fbea1255640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e1a4a0 0x55c8d3e20f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39644/0 (socket says 172.21.15.84:39644) 2024-05-22T23:05:39.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.982+0000 7fbea1a56640 1 -- 172.21.15.84:0/759052487 learned_addr learned my addr 172.21.15.84:0/759052487 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:39.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.982+0000 7fbea2257640 1 --2- 172.21.15.84:0/759052487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8d3e23b30 0x55c8d3e21460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:39.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.982+0000 7fbea1255640 1 -- 172.21.15.84:0/759052487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8d3e23b30 msgr2=0x55c8d3e21460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:39.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.982+0000 7fbea1255640 1 --2- 172.21.15.84:0/759052487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8d3e23b30 0x55c8d3e21460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:39.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.982+0000 7fbea1255640 1 -- 172.21.15.84:0/759052487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8d3e18db0 msgr2=0x55c8d3e209e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:39.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.982+0000 7fbea1255640 1 --2- 172.21.15.84:0/759052487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8d3e18db0 0x55c8d3e209e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:39.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.982+0000 7fbea1255640 1 -- 172.21.15.84:0/759052487 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8d3b5a7d0 con 0x55c8d3e1a4a0 2024-05-22T23:05:39.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.982+0000 7fbea1255640 1 --2- 172.21.15.84:0/759052487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e1a4a0 0x55c8d3e20f20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009710 tx=0x7fbe8c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d823d7a0d1f9d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:39.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.982+0000 7fbe857fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:05:39.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.982+0000 7fbe857fa640 1 -- 172.21.15.84:0/759052487 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c009780 con 0x55c8d3e1a4a0 2024-05-22T23:05:39.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.982+0000 7fbe857fa640 1 -- 172.21.15.84:0/759052487 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fbe8c003800 con 0x55c8d3e1a4a0 2024-05-22T23:05:39.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.982+0000 7fbea69c8740 1 -- 172.21.15.84:0/759052487 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe98031830 con 0x55c8d3e1a4a0 2024-05-22T23:05:39.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.982+0000 7fbe857fa640 1 -- 172.21.15.84:0/759052487 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fbe8c0044c0 con 0x55c8d3e1a4a0 2024-05-22T23:05:39.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.982+0000 7fbea69c8740 10 client.16026 fetch_fsmap learned FSMap version 18 2024-05-22T23:05:39.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.982+0000 7fbea69c8740 1 -- 172.21.15.84:0/759052487 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55c8d3a903f0 con 0x55c8d3e1a4a0 2024-05-22T23:05:39.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.983+0000 7fbe857fa640 1 -- 172.21.15.84:0/759052487 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe98031830 con 0x55c8d3e1a4a0 2024-05-22T23:05:39.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.983+0000 7fbe857fa640 1 -- 172.21.15.84:0/759052487 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fbe8c01faf0 con 0x55c8d3e1a4a0 2024-05-22T23:05:39.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.983+0000 7fbea69c8740 10 client.16026 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:05:39.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.983+0000 7fbea69c8740 10 client.16026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:05:39.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.983+0000 7fbea69c8740 20 client.16026 populate_metadata read hostname 'smithi084' 2024-05-22T23:05:39.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.983+0000 7fbea69c8740 1 --2- 172.21.15.84:0/759052487 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c8d3e33ee0 0x55c8d3e36380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:39.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.983+0000 7fbea69c8740 4 client.16026 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:05:39.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.983+0000 7fbea69c8740 1 -- 172.21.15.84:0/759052487 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8d3d8a4e0 con 0x55c8d3e33ee0 2024-05-22T23:05:39.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.983+0000 7fbea1a56640 1 --2- 172.21.15.84:0/759052487 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c8d3e33ee0 0x55c8d3e36380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:39.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.983+0000 7fbe857fa640 10 client.16026 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:39.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.983+0000 7fbea1a56640 1 --2- 172.21.15.84:0/759052487 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c8d3e33ee0 0x55c8d3e36380 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c79beb067c997c server_cookie=2a491dafa4838 in_seq=0 out_seq=0 2024-05-22T23:05:39.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.984+0000 7fbe857fa640 1 -- 172.21.15.84:0/759052487 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c8d3d8a4e0 con 0x55c8d3e33ee0 2024-05-22T23:05:39.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.984+0000 7fbe857fa640 10 client.16026 handle_command_reply: tid=0 2024-05-22T23:05:39.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.985+0000 7fbea69c8740 1 client.16026 shutdown 2024-05-22T23:05:39.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.985+0000 7fbea69c8740 1 -- 172.21.15.84:0/759052487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e1a4a0 msgr2=0x55c8d3e20f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:39.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.985+0000 7fbea69c8740 1 --2- 172.21.15.84:0/759052487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e1a4a0 0x55c8d3e20f20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009710 tx=0x7fbe8c01d000 comp rx=0 tx=0).stop 2024-05-22T23:05:39.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.985+0000 7fbea69c8740 1 -- 172.21.15.84:0/759052487 shutdown_connections 2024-05-22T23:05:39.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.985+0000 7fbea69c8740 1 --2- 172.21.15.84:0/759052487 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c8d3e33ee0 0x55c8d3e36380 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:39.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.985+0000 7fbea69c8740 1 --2- 172.21.15.84:0/759052487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8d3e23b30 0x55c8d3e21460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:39.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.985+0000 7fbea69c8740 1 --2- 172.21.15.84:0/759052487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e1a4a0 0x55c8d3e20f20 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:39.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.985+0000 7fbe857fa640 0 client.16026 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:39.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.985+0000 7fbea69c8740 1 --2- 172.21.15.84:0/759052487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8d3e18db0 0x55c8d3e209e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:39.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.985+0000 7fbea69c8740 1 -- 172.21.15.84:0/759052487 >> 172.21.15.84:0/759052487 conn(0x55c8d3de18e0 msgr2=0x55c8d3e1b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:39.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.985+0000 7fbea69c8740 1 -- 172.21.15.84:0/759052487 shutdown_connections 2024-05-22T23:05:39.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.985+0000 7fbea69c8740 1 -- 172.21.15.84:0/759052487 wait complete. 2024-05-22T23:05:39.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:39.985+0000 7fbea69c8740 20 client.16026 trim_cache size 0 max 16384 2024-05-22T23:05:40.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.003+0000 7fbea69c8740 1 Processor -- start 2024-05-22T23:05:40.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.003+0000 7fbea69c8740 1 -- start start 2024-05-22T23:05:40.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.003+0000 7fbea69c8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8d3e18db0 0x55c8d3e210b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.003+0000 7fbea69c8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e1a4a0 0x55c8d3e3bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.003+0000 7fbea69c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8d3e23b30 0x55c8d3e3c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.003+0000 7fbea69c8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c1ab6e0 con 0x55c8d3e1a4a0 2024-05-22T23:05:40.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.003+0000 7fbea69c8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c1adf60 con 0x55c8d3e18db0 2024-05-22T23:05:40.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.003+0000 7fbea69c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c1abd50 con 0x55c8d3e23b30 2024-05-22T23:05:40.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.003+0000 7fbea1255640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e1a4a0 0x55c8d3e3bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.004+0000 7fbea1255640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e1a4a0 0x55c8d3e3bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39650/0 (socket says 172.21.15.84:39650) 2024-05-22T23:05:40.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.004+0000 7fbea1255640 1 -- 172.21.15.84:0/1435585174 learned_addr learned my addr 172.21.15.84:0/1435585174 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:40.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.004+0000 7fbea1a56640 1 --2- 172.21.15.84:0/1435585174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8d3e18db0 0x55c8d3e210b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.004+0000 7fbea2257640 1 --2- 172.21.15.84:0/1435585174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8d3e23b30 0x55c8d3e3c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.004+0000 7fbea1255640 1 -- 172.21.15.84:0/1435585174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8d3e23b30 msgr2=0x55c8d3e3c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:40.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.004+0000 7fbea1255640 1 --2- 172.21.15.84:0/1435585174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8d3e23b30 0x55c8d3e3c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.004+0000 7fbea1255640 1 -- 172.21.15.84:0/1435585174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8d3e18db0 msgr2=0x55c8d3e210b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:40.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.004+0000 7fbea1255640 1 --2- 172.21.15.84:0/1435585174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8d3e18db0 0x55c8d3e210b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.004+0000 7fbea1255640 1 -- 172.21.15.84:0/1435585174 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8d3a500f0 con 0x55c8d3e1a4a0 2024-05-22T23:05:40.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.004+0000 7fbea1255640 1 --2- 172.21.15.84:0/1435585174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e1a4a0 0x55c8d3e3bc50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c01fe60 tx=0x7fbe8c0117c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9302565530dcd5e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:40.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.004+0000 7fbe877fe640 1 -- 172.21.15.84:0/1435585174 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c009780 con 0x55c8d3e1a4a0 2024-05-22T23:05:40.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.004+0000 7fbe877fe640 1 -- 172.21.15.84:0/1435585174 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe8c003ce0 con 0x55c8d3e1a4a0 2024-05-22T23:05:40.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.004+0000 7fbe877fe640 1 -- 172.21.15.84:0/1435585174 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c005480 con 0x55c8d3e1a4a0 2024-05-22T23:05:40.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.005+0000 7fbea69c8740 1 -- 172.21.15.84:0/1435585174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e1a4a0 msgr2=0x55c8d3e3bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:40.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.005+0000 7fbea69c8740 1 --2- 172.21.15.84:0/1435585174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e1a4a0 0x55c8d3e3bc50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c01fe60 tx=0x7fbe8c0117c0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.005+0000 7fbea69c8740 1 -- 172.21.15.84:0/1435585174 shutdown_connections 2024-05-22T23:05:40.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.005+0000 7fbea69c8740 1 --2- 172.21.15.84:0/1435585174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8d3e23b30 0x55c8d3e3c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.005+0000 7fbea69c8740 1 --2- 172.21.15.84:0/1435585174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e1a4a0 0x55c8d3e3bc50 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.005+0000 7fbea69c8740 1 --2- 172.21.15.84:0/1435585174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8d3e18db0 0x55c8d3e210b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.005+0000 7fbea69c8740 1 -- 172.21.15.84:0/1435585174 >> 172.21.15.84:0/1435585174 conn(0x55c8d3de18e0 msgr2=0x55c8d3e209a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:40.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.005+0000 7fbea69c8740 1 -- 172.21.15.84:0/1435585174 shutdown_connections 2024-05-22T23:05:40.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.005+0000 7fbea69c8740 1 -- 172.21.15.84:0/1435585174 wait complete. 2024-05-22T23:05:40.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.005+0000 7fbea69c8740 1 -- start start 2024-05-22T23:05:40.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.005+0000 7fbea69c8740 1 Processor -- start 2024-05-22T23:05:40.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.005+0000 7fbea69c8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e18db0 0x55c8d3e1d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.005+0000 7fbea69c8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8d3e1a4a0 0x55c8d3e1dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.005+0000 7fbea69c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8d3e23b30 0x55c8d3e1e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.005+0000 7fbea69c8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c1ac420 con 0x55c8d3e18db0 2024-05-22T23:05:40.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.005+0000 7fbea69c8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c1ad890 con 0x55c8d3e1a4a0 2024-05-22T23:05:40.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.005+0000 7fbea69c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c1acaf0 con 0x55c8d3e23b30 2024-05-22T23:05:40.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.006+0000 7fbea69c8740 10 client.0 _ll_register_callbacks cb 0x7fbe8c009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:05:40.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.006+0000 7fbea1a56640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e18db0 0x55c8d3e1d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.006+0000 7fbea1a56640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e18db0 0x55c8d3e1d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39660/0 (socket says 172.21.15.84:39660) 2024-05-22T23:05:40.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.006+0000 7fbea1a56640 1 -- 172.21.15.84:0/3542157425 learned_addr learned my addr 172.21.15.84:0/3542157425 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:40.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.006+0000 7fbea1255640 1 --2- 172.21.15.84:0/3542157425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8d3e1a4a0 0x55c8d3e1dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.006+0000 7fbea2257640 1 --2- 172.21.15.84:0/3542157425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8d3e23b30 0x55c8d3e1e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.006+0000 7fbea1a56640 1 -- 172.21.15.84:0/3542157425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8d3e23b30 msgr2=0x55c8d3e1e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:40.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.006+0000 7fbea1a56640 1 --2- 172.21.15.84:0/3542157425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8d3e23b30 0x55c8d3e1e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.006+0000 7fbea1a56640 1 -- 172.21.15.84:0/3542157425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8d3e1a4a0 msgr2=0x55c8d3e1dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:40.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.006+0000 7fbea1a56640 1 --2- 172.21.15.84:0/3542157425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8d3e1a4a0 0x55c8d3e1dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.006+0000 7fbea1a56640 1 -- 172.21.15.84:0/3542157425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8d3d3bfe0 con 0x55c8d3e18db0 2024-05-22T23:05:40.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.006+0000 7fbea1a56640 1 --2- 172.21.15.84:0/3542157425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e18db0 0x55c8d3e1d830 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbe98000c00 tx=0x7fbe98031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a7bc9641ff9b06 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:40.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.006+0000 7fbe857fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:05:40.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.006+0000 7fbe857fa640 1 -- 172.21.15.84:0/3542157425 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98035040 con 0x55c8d3e18db0 2024-05-22T23:05:40.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.006+0000 7fbe857fa640 1 -- 172.21.15.84:0/3542157425 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fbe98031d00 con 0x55c8d3e18db0 2024-05-22T23:05:40.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.006+0000 7fbea69c8740 1 -- 172.21.15.84:0/3542157425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe8c003ce0 con 0x55c8d3e18db0 2024-05-22T23:05:40.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.006+0000 7fbe857fa640 1 -- 172.21.15.84:0/3542157425 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fbe9805e050 con 0x55c8d3e18db0 2024-05-22T23:05:40.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.006+0000 7fbea69c8740 10 client.16032 fetch_fsmap learned FSMap version 18 2024-05-22T23:05:40.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.006+0000 7fbea69c8740 1 -- 172.21.15.84:0/3542157425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55c8d3d0d1c0 con 0x55c8d3e18db0 2024-05-22T23:05:40.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.006+0000 7fbe857fa640 1 -- 172.21.15.84:0/3542157425 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe8c003ce0 con 0x55c8d3e18db0 2024-05-22T23:05:40.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.007+0000 7fbe857fa640 1 -- 172.21.15.84:0/3542157425 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fbe98059030 con 0x55c8d3e18db0 2024-05-22T23:05:40.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.007+0000 7fbea69c8740 10 client.16032 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:05:40.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.007+0000 7fbea69c8740 10 client.16032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:05:40.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.007+0000 7fbea69c8740 20 client.16032 populate_metadata read hostname 'smithi084' 2024-05-22T23:05:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.007+0000 7fbea69c8740 1 --2- 172.21.15.84:0/3542157425 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c8d3e9a8b0 0x55c8d3e9cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.007+0000 7fbea69c8740 4 client.16032 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:05:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.007+0000 7fbea69c8740 1 -- 172.21.15.84:0/3542157425 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c8d3d84830 con 0x55c8d3e9a8b0 2024-05-22T23:05:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.007+0000 7fbea1255640 1 --2- 172.21.15.84:0/3542157425 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c8d3e9a8b0 0x55c8d3e9cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.007+0000 7fbe857fa640 10 client.16032 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.007+0000 7fbea1255640 1 --2- 172.21.15.84:0/3542157425 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c8d3e9a8b0 0x55c8d3e9cca0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d51e7f74362cee6 server_cookie=6d2f96f6ea03c0a4 in_seq=0 out_seq=0 2024-05-22T23:05:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.008+0000 7fbe857fa640 1 -- 172.21.15.84:0/3542157425 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c8d3d84830 con 0x55c8d3e9a8b0 2024-05-22T23:05:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.008+0000 7fbe857fa640 10 client.16032 handle_command_reply: tid=0 2024-05-22T23:05:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.008+0000 7fbea69c8740 1 client.16032 shutdown 2024-05-22T23:05:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.008+0000 7fbea69c8740 1 -- 172.21.15.84:0/3542157425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e18db0 msgr2=0x55c8d3e1d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.008+0000 7fbea69c8740 1 --2- 172.21.15.84:0/3542157425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e18db0 0x55c8d3e1d830 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbe98000c00 tx=0x7fbe98031400 comp rx=0 tx=0).stop 2024-05-22T23:05:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.008+0000 7fbea69c8740 1 -- 172.21.15.84:0/3542157425 shutdown_connections 2024-05-22T23:05:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.008+0000 7fbea69c8740 1 --2- 172.21.15.84:0/3542157425 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c8d3e9a8b0 0x55c8d3e9cca0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.008+0000 7fbea69c8740 1 --2- 172.21.15.84:0/3542157425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8d3e23b30 0x55c8d3e1e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.008+0000 7fbe857fa640 0 client.16032 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:40.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.008+0000 7fbea69c8740 1 --2- 172.21.15.84:0/3542157425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8d3e1a4a0 0x55c8d3e1dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.008+0000 7fbea69c8740 1 --2- 172.21.15.84:0/3542157425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c8d3e18db0 0x55c8d3e1d830 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.008+0000 7fbea69c8740 1 -- 172.21.15.84:0/3542157425 >> 172.21.15.84:0/3542157425 conn(0x55c8d3de18e0 msgr2=0x55c8d3e21230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:40.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.008+0000 7fbea69c8740 1 -- 172.21.15.84:0/3542157425 shutdown_connections 2024-05-22T23:05:40.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.008+0000 7fbea69c8740 1 -- 172.21.15.84:0/3542157425 wait complete. 2024-05-22T23:05:40.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.008+0000 7fbea69c8740 20 client.16032 trim_cache size 0 max 16384 2024-05-22T23:05:40.019 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:05:40.019 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "no active scrubs running", 2024-05-22T23:05:40.020 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": {} 2024-05-22T23:05:40.020 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:05:40.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.009+0000 7fbea3cba640 1 -- 172.21.15.84:0/2018398818 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fbe880a2d00 msgr2=0x7fbe880a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:40.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.009+0000 7fbea3cba640 1 --2- 172.21.15.84:0/2018398818 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fbe880a2d00 0x7fbe880a51c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009fd0 tx=0x7fbe8c009b80 comp rx=0 tx=0).stop 2024-05-22T23:05:40.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.009+0000 7fbea3cba640 1 -- 172.21.15.84:0/2018398818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe9c107850 msgr2=0x7fbe9c19ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:40.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.009+0000 7fbea3cba640 1 --2- 172.21.15.84:0/2018398818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe9c107850 0x7fbe9c19ca80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400eb10 tx=0x7fbe9400c2d0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.009+0000 7fbea3cba640 1 -- 172.21.15.84:0/2018398818 shutdown_connections 2024-05-22T23:05:40.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.009+0000 7fbea3cba640 1 --2- 172.21.15.84:0/2018398818 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fbe880a2d00 0x7fbe880a51c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.009+0000 7fbea3cba640 1 --2- 172.21.15.84:0/2018398818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe9c107850 0x7fbe9c19ca80 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.009+0000 7fbea3cba640 1 --2- 172.21.15.84:0/2018398818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe9c102200 0x7fbe9c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.009+0000 7fbea3cba640 1 --2- 172.21.15.84:0/2018398818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe9c101840 0x7fbe9c1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.009+0000 7fbea3cba640 1 -- 172.21.15.84:0/2018398818 >> 172.21.15.84:0/2018398818 conn(0x7fbe9c078cc0 msgr2=0x7fbe9c0ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:40.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.009+0000 7fbea3cba640 1 -- 172.21.15.84:0/2018398818 shutdown_connections 2024-05-22T23:05:40.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.009+0000 7fbea3cba640 1 -- 172.21.15.84:0/2018398818 wait complete. 2024-05-22T23:05:40.024 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-22T23:05:40.024 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:05:40.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.164+0000 7fdf69bc1640 1 -- 172.21.15.84:0/1601965959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf64106730 msgr2=0x7fdf64106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:40.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.164+0000 7fdf69bc1640 1 --2- 172.21.15.84:0/1601965959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf64106730 0x7fdf64106b10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009930 tx=0x7fdf5402a420 comp rx=0 tx=0).stop 2024-05-22T23:05:40.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.164+0000 7fdf69bc1640 1 -- 172.21.15.84:0/1601965959 shutdown_connections 2024-05-22T23:05:40.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.164+0000 7fdf69bc1640 1 --2- 172.21.15.84:0/1601965959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf64101160 0x7fdf6410d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.164+0000 7fdf69bc1640 1 --2- 172.21.15.84:0/1601965959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf641007a0 0x7fdf64100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.164+0000 7fdf69bc1640 1 --2- 172.21.15.84:0/1601965959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf64106730 0x7fdf64106b10 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.164+0000 7fdf69bc1640 1 -- 172.21.15.84:0/1601965959 >> 172.21.15.84:0/1601965959 conn(0x7fdf64076880 msgr2=0x7fdf64076c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:40.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.164+0000 7fdf69bc1640 1 -- 172.21.15.84:0/1601965959 shutdown_connections 2024-05-22T23:05:40.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.164+0000 7fdf69bc1640 1 -- 172.21.15.84:0/1601965959 wait complete. 2024-05-22T23:05:40.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.164+0000 7fdf69bc1640 1 Processor -- start 2024-05-22T23:05:40.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.164+0000 7fdf69bc1640 1 -- start start 2024-05-22T23:05:40.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.164+0000 7fdf69bc1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf641007a0 0x7fdf64198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.164+0000 7fdf69bc1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf64101160 0x7fdf641994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.164+0000 7fdf69bc1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf64106730 0x7fdf641a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.165+0000 7fdf69bc1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdf64110b40 con 0x7fdf641007a0 2024-05-22T23:05:40.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.165+0000 7fdf69bc1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdf641109c0 con 0x7fdf64106730 2024-05-22T23:05:40.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.165+0000 7fdf69bc1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdf64110cc0 con 0x7fdf64101160 2024-05-22T23:05:40.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.165+0000 7fdf637fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf641007a0 0x7fdf64198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.165+0000 7fdf637fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf641007a0 0x7fdf64198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39680/0 (socket says 172.21.15.84:39680) 2024-05-22T23:05:40.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.165+0000 7fdf637fe640 1 -- 172.21.15.84:0/152904259 learned_addr learned my addr 172.21.15.84:0/152904259 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:40.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.165+0000 7fdf63fff640 1 --2- 172.21.15.84:0/152904259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf64106730 0x7fdf641a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.165+0000 7fdf5bfff640 1 --2- 172.21.15.84:0/152904259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf64101160 0x7fdf641994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.165+0000 7fdf637fe640 1 -- 172.21.15.84:0/152904259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf64101160 msgr2=0x7fdf641994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:40.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.165+0000 7fdf637fe640 1 --2- 172.21.15.84:0/152904259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf64101160 0x7fdf641994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.165+0000 7fdf637fe640 1 -- 172.21.15.84:0/152904259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf64106730 msgr2=0x7fdf641a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:40.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.165+0000 7fdf637fe640 1 --2- 172.21.15.84:0/152904259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf64106730 0x7fdf641a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.165+0000 7fdf637fe640 1 -- 172.21.15.84:0/152904259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf641a4560 con 0x7fdf641007a0 2024-05-22T23:05:40.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.165+0000 7fdf63fff640 1 --2- 172.21.15.84:0/152904259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf64106730 0x7fdf641a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:05:40.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.165+0000 7fdf637fe640 1 --2- 172.21.15.84:0/152904259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf641007a0 0x7fdf64198fa0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009230 tx=0x7fdf54004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:40.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.165+0000 7fdf617fa640 1 -- 172.21.15.84:0/152904259 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5403f030 con 0x7fdf641007a0 2024-05-22T23:05:40.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.165+0000 7fdf617fa640 1 -- 172.21.15.84:0/152904259 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf54004440 con 0x7fdf641007a0 2024-05-22T23:05:40.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.165+0000 7fdf69bc1640 1 -- 172.21.15.84:0/152904259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf641a4730 con 0x7fdf641007a0 2024-05-22T23:05:40.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.165+0000 7fdf617fa640 1 -- 172.21.15.84:0/152904259 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf54035070 con 0x7fdf641007a0 2024-05-22T23:05:40.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.165+0000 7fdf69bc1640 1 -- 172.21.15.84:0/152904259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf641a4bf0 con 0x7fdf641007a0 2024-05-22T23:05:40.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.166+0000 7fdf69bc1640 1 -- 172.21.15.84:0/152904259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf64101e60 con 0x7fdf641007a0 2024-05-22T23:05:40.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.167+0000 7fdf617fa640 1 -- 172.21.15.84:0/152904259 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdf54042030 con 0x7fdf641007a0 2024-05-22T23:05:40.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.168+0000 7fdf617fa640 1 --2- 172.21.15.84:0/152904259 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fdf500a2d00 0x7fdf500a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.168+0000 7fdf5bfff640 1 --2- 172.21.15.84:0/152904259 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fdf500a2d00 0x7fdf500a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.168+0000 7fdf617fa640 1 -- 172.21.15.84:0/152904259 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fdf540ef730 con 0x7fdf641007a0 2024-05-22T23:05:40.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.168+0000 7fdf5bfff640 1 --2- 172.21.15.84:0/152904259 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fdf500a2d00 0x7fdf500a51c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdf6406aa80 tx=0x7fdf4c005f90 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:40.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.172+0000 7fdf617fa640 1 -- 172.21.15.84:0/152904259 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdf540bbd00 con 0x7fdf641007a0 2024-05-22T23:05:40.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.402+0000 7fdf69bc1640 1 -- 172.21.15.84:0/152904259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf64108c90 con 0x7fdf641007a0 2024-05-22T23:05:40.407 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.403+0000 7fdf617fa640 1 -- 172.21.15.84:0/152904259 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7fdf540c0000 con 0x7fdf641007a0 2024-05-22T23:05:40.407 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:05:40.408 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:05:40.408 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:05:40.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.405+0000 7fdf69bc1640 1 -- 172.21.15.84:0/152904259 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fdf500a2d00 msgr2=0x7fdf500a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:40.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.405+0000 7fdf69bc1640 1 --2- 172.21.15.84:0/152904259 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fdf500a2d00 0x7fdf500a51c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdf6406aa80 tx=0x7fdf4c005f90 comp rx=0 tx=0).stop 2024-05-22T23:05:40.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.405+0000 7fdf69bc1640 1 -- 172.21.15.84:0/152904259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf641007a0 msgr2=0x7fdf64198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:40.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.405+0000 7fdf69bc1640 1 --2- 172.21.15.84:0/152904259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf641007a0 0x7fdf64198fa0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009230 tx=0x7fdf54004040 comp rx=0 tx=0).stop 2024-05-22T23:05:40.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.405+0000 7fdf69bc1640 1 -- 172.21.15.84:0/152904259 shutdown_connections 2024-05-22T23:05:40.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.405+0000 7fdf69bc1640 1 --2- 172.21.15.84:0/152904259 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fdf500a2d00 0x7fdf500a51c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.405+0000 7fdf69bc1640 1 --2- 172.21.15.84:0/152904259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf64106730 0x7fdf641a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.405+0000 7fdf69bc1640 1 --2- 172.21.15.84:0/152904259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf64101160 0x7fdf641994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.405+0000 7fdf69bc1640 1 --2- 172.21.15.84:0/152904259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf641007a0 0x7fdf64198fa0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.405+0000 7fdf69bc1640 1 -- 172.21.15.84:0/152904259 >> 172.21.15.84:0/152904259 conn(0x7fdf64076880 msgr2=0x7fdf640fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:40.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.405+0000 7fdf69bc1640 1 -- 172.21.15.84:0/152904259 shutdown_connections 2024-05-22T23:05:40.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.405+0000 7fdf69bc1640 1 -- 172.21.15.84:0/152904259 wait complete. 2024-05-22T23:05:40.421 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-22T23:05:40.431 INFO:tasks.workunit.client.0.smithi084.stdout: 1 426258 15.43 MB/sec execute 26 sec latency 11.365 ms 2024-05-22T23:05:40.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.522+0000 7f83ea038640 1 -- 172.21.15.84:0/348784389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83e4100740 msgr2=0x7f83e4100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:40.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.522+0000 7f83ea038640 1 --2- 172.21.15.84:0/348784389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83e4100740 0x7f83e4100bc0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009930 tx=0x7f83d802a420 comp rx=0 tx=0).stop 2024-05-22T23:05:40.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.522+0000 7f83ea038640 1 -- 172.21.15.84:0/348784389 shutdown_connections 2024-05-22T23:05:40.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.522+0000 7f83ea038640 1 --2- 172.21.15.84:0/348784389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e4101100 0x7f83e410d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.522+0000 7f83ea038640 1 --2- 172.21.15.84:0/348784389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83e4100740 0x7f83e4100bc0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.522+0000 7f83ea038640 1 --2- 172.21.15.84:0/348784389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e4106750 0x7f83e4106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.522+0000 7f83ea038640 1 -- 172.21.15.84:0/348784389 >> 172.21.15.84:0/348784389 conn(0x7f83e4077870 msgr2=0x7f83e4077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:40.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.522+0000 7f83ea038640 1 -- 172.21.15.84:0/348784389 shutdown_connections 2024-05-22T23:05:40.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.522+0000 7f83ea038640 1 -- 172.21.15.84:0/348784389 wait complete. 2024-05-22T23:05:40.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.523+0000 7f83ea038640 1 Processor -- start 2024-05-22T23:05:40.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.523+0000 7f83ea038640 1 -- start start 2024-05-22T23:05:40.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.523+0000 7f83ea038640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e4100740 0x7f83e4199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.523+0000 7f83ea038640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e4101100 0x7f83e4199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.523+0000 7f83ea038640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83e4106750 0x7f83e41a40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.523+0000 7f83ea038640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f83e4110b60 con 0x7f83e4106750 2024-05-22T23:05:40.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.523+0000 7f83ea038640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f83e41109e0 con 0x7f83e4101100 2024-05-22T23:05:40.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.523+0000 7f83ea038640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f83e4110ce0 con 0x7f83e4100740 2024-05-22T23:05:40.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.524+0000 7f83e3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83e4106750 0x7f83e41a40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.524+0000 7f83e37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e4100740 0x7f83e4199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.524+0000 7f83e3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83e4106750 0x7f83e41a40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39706/0 (socket says 172.21.15.84:39706) 2024-05-22T23:05:40.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.524+0000 7f83e37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e4100740 0x7f83e4199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:37720/0 (socket says 172.21.15.84:37720) 2024-05-22T23:05:40.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.524+0000 7f83e3fff640 1 -- 172.21.15.84:0/2822839142 learned_addr learned my addr 172.21.15.84:0/2822839142 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:40.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.524+0000 7f83e2ffd640 1 --2- 172.21.15.84:0/2822839142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e4101100 0x7f83e4199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.524+0000 7f83e3fff640 1 -- 172.21.15.84:0/2822839142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e4100740 msgr2=0x7f83e4199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:40.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.524+0000 7f83e3fff640 1 --2- 172.21.15.84:0/2822839142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e4100740 0x7f83e4199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.524+0000 7f83e3fff640 1 -- 172.21.15.84:0/2822839142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e4101100 msgr2=0x7f83e4199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:40.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.524+0000 7f83e3fff640 1 --2- 172.21.15.84:0/2822839142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e4101100 0x7f83e4199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.524+0000 7f83e3fff640 1 -- 172.21.15.84:0/2822839142 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e41a45f0 con 0x7f83e4106750 2024-05-22T23:05:40.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.524+0000 7f83e3fff640 1 --2- 172.21.15.84:0/2822839142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83e4106750 0x7f83e41a40b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f83dc00eb10 tx=0x7f83dc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:40.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.525+0000 7f83e0ff9640 1 -- 172.21.15.84:0/2822839142 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc004b20 con 0x7f83e4106750 2024-05-22T23:05:40.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.525+0000 7f83e0ff9640 1 -- 172.21.15.84:0/2822839142 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83dc004cc0 con 0x7f83e4106750 2024-05-22T23:05:40.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.525+0000 7f83e0ff9640 1 -- 172.21.15.84:0/2822839142 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc0107b0 con 0x7f83e4106750 2024-05-22T23:05:40.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.525+0000 7f83ea038640 1 -- 172.21.15.84:0/2822839142 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e4196e20 con 0x7f83e4106750 2024-05-22T23:05:40.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.525+0000 7f83ea038640 1 -- 172.21.15.84:0/2822839142 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e4197340 con 0x7f83e4106750 2024-05-22T23:05:40.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.528+0000 7f83ece29740 1 Processor -- start 2024-05-22T23:05:40.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.528+0000 7f83e0ff9640 1 -- 172.21.15.84:0/2822839142 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f83dc005000 con 0x7f83e4106750 2024-05-22T23:05:40.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.528+0000 7f83ece29740 1 -- start start 2024-05-22T23:05:40.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.528+0000 7f83ece29740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3c135f80 0x55cf3c136360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.528+0000 7f83ece29740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c1368a0 0x55cf3c140a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.528+0000 7f83ece29740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3c140f70 0x55cf3c143360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.528+0000 7f83ece29740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55cf3c09adb0 con 0x55cf3c1368a0 2024-05-22T23:05:40.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.528+0000 7f83ece29740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cf3c039400 con 0x55cf3c135f80 2024-05-22T23:05:40.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.528+0000 7f83ece29740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cf3c096d70 con 0x55cf3c140f70 2024-05-22T23:05:40.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.529+0000 7f83e2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3c135f80 0x55cf3c136360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.529+0000 7f83e37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c1368a0 0x55cf3c140a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.529+0000 7f83e2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3c135f80 0x55cf3c136360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:56856/0 (socket says 172.21.15.84:56856) 2024-05-22T23:05:40.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.529+0000 7f83e2ffd640 1 -- 172.21.15.84:0/3313501618 learned_addr learned my addr 172.21.15.84:0/3313501618 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:40.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.529+0000 7f83e2ffd640 1 --2- 172.21.15.84:0/3313501618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3c140f70 0x55cf3c143360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.529+0000 7f83e37fe640 1 -- 172.21.15.84:0/3313501618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3c140f70 msgr2=0x55cf3c143360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:40.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.529+0000 7f83e0ff9640 1 --2- 172.21.15.84:0/2822839142 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f83cc0a2d80 0x7f83cc0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.529+0000 7f83e37fe640 1 --2- 172.21.15.84:0/3313501618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3c140f70 0x55cf3c143360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.529+0000 7f83e37fe640 1 -- 172.21.15.84:0/3313501618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3c135f80 msgr2=0x55cf3c136360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:40.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.529+0000 7f83e37fe640 1 --2- 172.21.15.84:0/3313501618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3c135f80 0x55cf3c136360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.529+0000 7f83e37fe640 1 -- 172.21.15.84:0/3313501618 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf3c02b2f0 con 0x55cf3c1368a0 2024-05-22T23:05:40.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.529+0000 7f83e3fff640 1 --2- 172.21.15.84:0/2822839142 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f83cc0a2d80 0x7f83cc0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.529+0000 7f83e0ff9640 1 -- 172.21.15.84:0/2822839142 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f83dc0cf9a0 con 0x7f83e4106750 2024-05-22T23:05:40.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.529+0000 7f83e37fe640 1 --2- 172.21.15.84:0/3313501618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c1368a0 0x55cf3c140a30 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f83d4010d30 tx=0x7f83d4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e7aa816f3af4b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:40.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.529+0000 7f83c9ffb640 1 -- 172.21.15.84:0/3313501618 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4004250 con 0x55cf3c1368a0 2024-05-22T23:05:40.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.530+0000 7f83c9ffb640 1 -- 172.21.15.84:0/3313501618 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d40043f0 con 0x55cf3c1368a0 2024-05-22T23:05:40.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.530+0000 7f83c9ffb640 1 -- 172.21.15.84:0/3313501618 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d401a960 con 0x55cf3c1368a0 2024-05-22T23:05:40.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.530+0000 7f83ece29740 1 -- 172.21.15.84:0/3313501618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c1368a0 msgr2=0x55cf3c140a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:40.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.530+0000 7f83e3fff640 1 --2- 172.21.15.84:0/2822839142 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f83cc0a2d80 0x7f83cc0a5240 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0cd1a0 tx=0x7f83dc005990 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:40.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.530+0000 7f83ece29740 1 --2- 172.21.15.84:0/3313501618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c1368a0 0x55cf3c140a30 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f83d4010d30 tx=0x7f83d4005f90 comp rx=0 tx=0).stop 2024-05-22T23:05:40.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.530+0000 7f83ece29740 1 -- 172.21.15.84:0/3313501618 shutdown_connections 2024-05-22T23:05:40.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.530+0000 7f83ece29740 1 --2- 172.21.15.84:0/3313501618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3c140f70 0x55cf3c143360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.530+0000 7f83ece29740 1 --2- 172.21.15.84:0/3313501618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c1368a0 0x55cf3c140a30 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.530+0000 7f83ece29740 1 --2- 172.21.15.84:0/3313501618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3c135f80 0x55cf3c136360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.530+0000 7f83ece29740 1 -- 172.21.15.84:0/3313501618 >> 172.21.15.84:0/3313501618 conn(0x55cf3c1322a0 msgr2=0x55cf3c132680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:40.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.530+0000 7f83ece29740 1 -- 172.21.15.84:0/3313501618 shutdown_connections 2024-05-22T23:05:40.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.531+0000 7f83ece29740 1 -- 172.21.15.84:0/3313501618 wait complete. 2024-05-22T23:05:40.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.531+0000 7f83ece29740 1 -- start start 2024-05-22T23:05:40.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.531+0000 7f83ece29740 1 Processor -- start 2024-05-22T23:05:40.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.531+0000 7f83ece29740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3c135f80 0x55cf3c13de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.531+0000 7f83ece29740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c1368a0 0x55cf3c13e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.531+0000 7f83ece29740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3c140f70 0x55cf3c13e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.531+0000 7f83ece29740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55cf3c143db0 con 0x55cf3c1368a0 2024-05-22T23:05:40.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.531+0000 7f83ece29740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cf3c0552e0 con 0x55cf3c135f80 2024-05-22T23:05:40.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.531+0000 7f83ece29740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cf3c143f30 con 0x55cf3c140f70 2024-05-22T23:05:40.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.531+0000 7f83e37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c1368a0 0x55cf3c13e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.531+0000 7f83ece29740 10 client.0 _ll_register_callbacks cb 0x55cf3c05ea70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:05:40.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.531+0000 7f83e2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3c135f80 0x55cf3c13de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.531+0000 7f83e37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c1368a0 0x55cf3c13e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39714/0 (socket says 172.21.15.84:39714) 2024-05-22T23:05:40.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.532+0000 7f83e37fe640 1 -- 172.21.15.84:0/691861765 learned_addr learned my addr 172.21.15.84:0/691861765 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:40.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.532+0000 7f83e2ffd640 1 --2- 172.21.15.84:0/691861765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3c140f70 0x55cf3c13e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.532+0000 7f83e37fe640 1 -- 172.21.15.84:0/691861765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3c140f70 msgr2=0x55cf3c13e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:40.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.532+0000 7f83e37fe640 1 --2- 172.21.15.84:0/691861765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3c140f70 0x55cf3c13e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.532+0000 7f83e37fe640 1 -- 172.21.15.84:0/691861765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3c135f80 msgr2=0x55cf3c13de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:40.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.532+0000 7f83e37fe640 1 --2- 172.21.15.84:0/691861765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3c135f80 0x55cf3c13de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.532+0000 7f83e37fe640 1 -- 172.21.15.84:0/691861765 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf3bdae3f0 con 0x55cf3c1368a0 2024-05-22T23:05:40.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.532+0000 7f83e2ffd640 1 --2- 172.21.15.84:0/691861765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3c135f80 0x55cf3c13de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:05:40.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.532+0000 7f83e37fe640 1 --2- 172.21.15.84:0/691861765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c1368a0 0x55cf3c13e370 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f83d4010d00 tx=0x7f83d40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31fbe14ad7193740 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:40.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.532+0000 7f83c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:05:40.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.532+0000 7f83c2ffd640 1 -- 172.21.15.84:0/691861765 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d402a070 con 0x55cf3c1368a0 2024-05-22T23:05:40.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.532+0000 7f83c2ffd640 1 -- 172.21.15.84:0/691861765 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f83d4005050 con 0x55cf3c1368a0 2024-05-22T23:05:40.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.532+0000 7f83ece29740 1 -- 172.21.15.84:0/691861765 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83d40043f0 con 0x55cf3c1368a0 2024-05-22T23:05:40.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.532+0000 7f83c2ffd640 1 -- 172.21.15.84:0/691861765 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f83d4036050 con 0x55cf3c1368a0 2024-05-22T23:05:40.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.532+0000 7f83ece29740 10 client.16050 fetch_fsmap learned FSMap version 18 2024-05-22T23:05:40.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.532+0000 7f83ece29740 1 -- 172.21.15.84:0/691861765 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55cf3be787d0 con 0x55cf3c1368a0 2024-05-22T23:05:40.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.532+0000 7f83c2ffd640 1 -- 172.21.15.84:0/691861765 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d40043f0 con 0x55cf3c1368a0 2024-05-22T23:05:40.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.533+0000 7f83c2ffd640 1 -- 172.21.15.84:0/691861765 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f83d401ab30 con 0x55cf3c1368a0 2024-05-22T23:05:40.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.533+0000 7f83ece29740 10 client.16050 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:05:40.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.533+0000 7f83ece29740 10 client.16050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:05:40.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.533+0000 7f83ece29740 20 client.16050 populate_metadata read hostname 'smithi084' 2024-05-22T23:05:40.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.533+0000 7f83ece29740 1 --2- 172.21.15.84:0/691861765 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55cf3c1510c0 0x55cf3c1534b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.533+0000 7f83ece29740 4 client.16050 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:05:40.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.533+0000 7f83ece29740 1 -- 172.21.15.84:0/691861765 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf3c0a84e0 con 0x55cf3c1510c0 2024-05-22T23:05:40.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.533+0000 7f83e2ffd640 1 --2- 172.21.15.84:0/691861765 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55cf3c1510c0 0x55cf3c1534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.533+0000 7f83c2ffd640 10 client.16050 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:40.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.533+0000 7f83e2ffd640 1 --2- 172.21.15.84:0/691861765 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55cf3c1510c0 0x55cf3c1534b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c647f92a2557845 server_cookie=d5a17a2d62360546 in_seq=0 out_seq=0 2024-05-22T23:05:40.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.534+0000 7f83c2ffd640 1 -- 172.21.15.84:0/691861765 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf3c0a84e0 con 0x55cf3c1510c0 2024-05-22T23:05:40.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.534+0000 7f83c2ffd640 10 client.16050 handle_command_reply: tid=0 2024-05-22T23:05:40.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.534+0000 7f83ece29740 1 client.16050 shutdown 2024-05-22T23:05:40.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.534+0000 7f83ece29740 1 -- 172.21.15.84:0/691861765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c1368a0 msgr2=0x55cf3c13e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:40.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.534+0000 7f83ece29740 1 --2- 172.21.15.84:0/691861765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c1368a0 0x55cf3c13e370 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f83d4010d00 tx=0x7f83d40038a0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.534+0000 7f83ece29740 1 -- 172.21.15.84:0/691861765 shutdown_connections 2024-05-22T23:05:40.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.534+0000 7f83ece29740 1 --2- 172.21.15.84:0/691861765 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55cf3c1510c0 0x55cf3c1534b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.535+0000 7f83ece29740 1 --2- 172.21.15.84:0/691861765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3c140f70 0x55cf3c13e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.535+0000 7f83ece29740 1 --2- 172.21.15.84:0/691861765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c1368a0 0x55cf3c13e370 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.535+0000 7f83c2ffd640 0 client.16050 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:40.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.535+0000 7f83ece29740 1 --2- 172.21.15.84:0/691861765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3c135f80 0x55cf3c13de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.535+0000 7f83ece29740 1 -- 172.21.15.84:0/691861765 >> 172.21.15.84:0/691861765 conn(0x55cf3c1322a0 msgr2=0x55cf3c1314c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:40.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.535+0000 7f83ece29740 1 -- 172.21.15.84:0/691861765 shutdown_connections 2024-05-22T23:05:40.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.535+0000 7f83ece29740 1 -- 172.21.15.84:0/691861765 wait complete. 2024-05-22T23:05:40.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.535+0000 7f83ece29740 20 client.16050 trim_cache size 0 max 16384 2024-05-22T23:05:40.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.549+0000 7f83ece29740 1 Processor -- start 2024-05-22T23:05:40.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.549+0000 7f83ece29740 1 -- start start 2024-05-22T23:05:40.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.549+0000 7f83ece29740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3c135f80 0x55cf3c140560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.549+0000 7f83ece29740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3c1368a0 0x55cf3c14cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.549+0000 7f83ece29740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c140f70 0x55cf3c13e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.549+0000 7f83ece29740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f83e41a9450 con 0x55cf3c140f70 2024-05-22T23:05:40.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.549+0000 7f83ece29740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f83e41abcd0 con 0x55cf3c135f80 2024-05-22T23:05:40.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.549+0000 7f83ece29740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f83e41a9ac0 con 0x55cf3c1368a0 2024-05-22T23:05:40.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.549+0000 7f83e2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c140f70 0x55cf3c13e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.549+0000 7f83e2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c140f70 0x55cf3c13e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39718/0 (socket says 172.21.15.84:39718) 2024-05-22T23:05:40.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.549+0000 7f83e2ffd640 1 -- 172.21.15.84:0/598664443 learned_addr learned my addr 172.21.15.84:0/598664443 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:40.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.549+0000 7f83e2ffd640 1 --2- 172.21.15.84:0/598664443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3c135f80 0x55cf3c140560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.549+0000 7f83e37fe640 1 --2- 172.21.15.84:0/598664443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3c1368a0 0x55cf3c14cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.549+0000 7f83e2ffd640 1 -- 172.21.15.84:0/598664443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3c1368a0 msgr2=0x55cf3c14cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:40.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.549+0000 7f83e2ffd640 1 --2- 172.21.15.84:0/598664443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3c1368a0 0x55cf3c14cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.549+0000 7f83e2ffd640 1 -- 172.21.15.84:0/598664443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3c135f80 msgr2=0x55cf3c140560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:40.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.549+0000 7f83e2ffd640 1 --2- 172.21.15.84:0/598664443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3c135f80 0x55cf3c140560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.549+0000 7f83e2ffd640 1 -- 172.21.15.84:0/598664443 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf3c12c8d0 con 0x55cf3c140f70 2024-05-22T23:05:40.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.549+0000 7f83e2ffd640 1 --2- 172.21.15.84:0/598664443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c140f70 0x55cf3c13e500 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009850 tx=0x7f83d8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47bcf8926fd5eda server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:40.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.549+0000 7f83c97fa640 1 -- 172.21.15.84:0/598664443 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8035040 con 0x55cf3c140f70 2024-05-22T23:05:40.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.550+0000 7f83c97fa640 1 -- 172.21.15.84:0/598664443 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d8005500 con 0x55cf3c140f70 2024-05-22T23:05:40.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.550+0000 7f83c97fa640 1 -- 172.21.15.84:0/598664443 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8004af0 con 0x55cf3c140f70 2024-05-22T23:05:40.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.550+0000 7f83ece29740 1 -- 172.21.15.84:0/598664443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c140f70 msgr2=0x55cf3c13e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:40.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.550+0000 7f83ece29740 1 --2- 172.21.15.84:0/598664443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c140f70 0x55cf3c13e500 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009850 tx=0x7f83d8009260 comp rx=0 tx=0).stop 2024-05-22T23:05:40.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.550+0000 7f83ece29740 1 -- 172.21.15.84:0/598664443 shutdown_connections 2024-05-22T23:05:40.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.550+0000 7f83ece29740 1 --2- 172.21.15.84:0/598664443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c140f70 0x55cf3c13e500 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.550+0000 7f83ece29740 1 --2- 172.21.15.84:0/598664443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3c1368a0 0x55cf3c14cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.550+0000 7f83ece29740 1 --2- 172.21.15.84:0/598664443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3c135f80 0x55cf3c140560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.550+0000 7f83ece29740 1 -- 172.21.15.84:0/598664443 >> 172.21.15.84:0/598664443 conn(0x55cf3c1322a0 msgr2=0x55cf3c13ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:40.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.550+0000 7f83ece29740 1 -- 172.21.15.84:0/598664443 shutdown_connections 2024-05-22T23:05:40.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.550+0000 7f83ece29740 1 -- 172.21.15.84:0/598664443 wait complete. 2024-05-22T23:05:40.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.551+0000 7f83ece29740 1 -- start start 2024-05-22T23:05:40.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.551+0000 7f83ece29740 1 Processor -- start 2024-05-22T23:05:40.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.551+0000 7f83ece29740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3c135f80 0x55cf3c149250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.551+0000 7f83ece29740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3c1368a0 0x55cf3c144700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.551+0000 7f83ece29740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c140f70 0x55cf3c144c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.551+0000 7f83ece29740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f83e41aa190 con 0x55cf3c140f70 2024-05-22T23:05:40.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.551+0000 7f83ece29740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f83e41ab600 con 0x55cf3c1368a0 2024-05-22T23:05:40.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.551+0000 7f83ece29740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f83e41aa860 con 0x55cf3c135f80 2024-05-22T23:05:40.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.551+0000 7f83ece29740 10 client.0 _ll_register_callbacks cb 0x7f83d4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:05:40.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.551+0000 7f83e2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c140f70 0x55cf3c144c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.551+0000 7f83e2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c140f70 0x55cf3c144c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39724/0 (socket says 172.21.15.84:39724) 2024-05-22T23:05:40.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.551+0000 7f83e2ffd640 1 -- 172.21.15.84:0/108421843 learned_addr learned my addr 172.21.15.84:0/108421843 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:40.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.551+0000 7f83e37fe640 1 --2- 172.21.15.84:0/108421843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3c1368a0 0x55cf3c144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.551+0000 7f83e2ffd640 1 --2- 172.21.15.84:0/108421843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3c135f80 0x55cf3c149250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.551+0000 7f83e2ffd640 1 -- 172.21.15.84:0/108421843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3c135f80 msgr2=0x55cf3c149250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:40.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.551+0000 7f83e2ffd640 1 --2- 172.21.15.84:0/108421843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3c135f80 0x55cf3c149250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.551+0000 7f83e2ffd640 1 -- 172.21.15.84:0/108421843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3c1368a0 msgr2=0x55cf3c144700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:40.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.551+0000 7f83e2ffd640 1 --2- 172.21.15.84:0/108421843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3c1368a0 0x55cf3c144700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.551+0000 7f83e2ffd640 1 -- 172.21.15.84:0/108421843 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf3c145180 con 0x55cf3c140f70 2024-05-22T23:05:40.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.552+0000 7f83e2ffd640 1 --2- 172.21.15.84:0/108421843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c140f70 0x55cf3c144c40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f83d802a940 tx=0x7f83d804ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db11c399ea3dc9b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:40.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.552+0000 7f83c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:05:40.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.552+0000 7f83c2ffd640 1 -- 172.21.15.84:0/108421843 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8035070 con 0x55cf3c140f70 2024-05-22T23:05:40.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.552+0000 7f83c2ffd640 1 -- 172.21.15.84:0/108421843 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f83d8004230 con 0x55cf3c140f70 2024-05-22T23:05:40.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.552+0000 7f83ece29740 1 -- 172.21.15.84:0/108421843 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83d8005500 con 0x55cf3c140f70 2024-05-22T23:05:40.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.552+0000 7f83c2ffd640 1 -- 172.21.15.84:0/108421843 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f83d8004450 con 0x55cf3c140f70 2024-05-22T23:05:40.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.552+0000 7f83ece29740 10 client.16056 fetch_fsmap learned FSMap version 18 2024-05-22T23:05:40.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.552+0000 7f83ece29740 1 -- 172.21.15.84:0/108421843 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55cf3c1b70e0 con 0x55cf3c140f70 2024-05-22T23:05:40.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.552+0000 7f83c2ffd640 1 -- 172.21.15.84:0/108421843 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d8005500 con 0x55cf3c140f70 2024-05-22T23:05:40.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.552+0000 7f83c2ffd640 1 -- 172.21.15.84:0/108421843 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f83d8056070 con 0x55cf3c140f70 2024-05-22T23:05:40.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.552+0000 7f83ece29740 10 client.16056 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:05:40.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.552+0000 7f83ece29740 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:05:40.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.552+0000 7f83ece29740 20 client.16056 populate_metadata read hostname 'smithi084' 2024-05-22T23:05:40.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.552+0000 7f83ece29740 1 --2- 172.21.15.84:0/108421843 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55cf3c1b7730 0x55cf3c1b9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:40.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.552+0000 7f83ece29740 4 client.16056 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-22T23:05:40.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.552+0000 7f83ece29740 1 -- 172.21.15.84:0/108421843 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf3c0a2830 con 0x55cf3c1b7730 2024-05-22T23:05:40.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.552+0000 7f83e37fe640 1 --2- 172.21.15.84:0/108421843 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55cf3c1b7730 0x55cf3c1b9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:40.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.553+0000 7f83c2ffd640 10 client.16056 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:40.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.553+0000 7f83e37fe640 1 --2- 172.21.15.84:0/108421843 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55cf3c1b7730 0x55cf3c1b9b50 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65bfe14c19c2b495 server_cookie=bf7fabc7afd1d16f in_seq=0 out_seq=0 2024-05-22T23:05:40.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83c2ffd640 1 -- 172.21.15.84:0/108421843 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf3c0a2830 con 0x55cf3c1b7730 2024-05-22T23:05:40.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83c2ffd640 10 client.16056 handle_command_reply: tid=0 2024-05-22T23:05:40.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83ece29740 1 client.16056 shutdown 2024-05-22T23:05:40.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83ece29740 1 -- 172.21.15.84:0/108421843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c140f70 msgr2=0x55cf3c144c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:40.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83ece29740 1 --2- 172.21.15.84:0/108421843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c140f70 0x55cf3c144c40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f83d802a940 tx=0x7f83d804ea10 comp rx=0 tx=0).stop 2024-05-22T23:05:40.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83ece29740 1 -- 172.21.15.84:0/108421843 shutdown_connections 2024-05-22T23:05:40.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83ece29740 1 --2- 172.21.15.84:0/108421843 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55cf3c1b7730 0x55cf3c1b9b50 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83ece29740 1 --2- 172.21.15.84:0/108421843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3c140f70 0x55cf3c144c40 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83ece29740 1 --2- 172.21.15.84:0/108421843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3c1368a0 0x55cf3c144700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83c2ffd640 0 client.16056 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:40.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83ece29740 1 --2- 172.21.15.84:0/108421843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3c135f80 0x55cf3c149250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83ece29740 1 -- 172.21.15.84:0/108421843 >> 172.21.15.84:0/108421843 conn(0x55cf3c1322a0 msgr2=0x55cf3c1314c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:40.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83ece29740 1 -- 172.21.15.84:0/108421843 shutdown_connections 2024-05-22T23:05:40.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83ece29740 1 -- 172.21.15.84:0/108421843 wait complete. 2024-05-22T23:05:40.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83ece29740 20 client.16056 trim_cache size 0 max 16384 2024-05-22T23:05:40.565 INFO:teuthology.orchestra.run.smithi084.stdout:[] 2024-05-22T23:05:40.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83ea038640 1 -- 172.21.15.84:0/2822839142 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f83cc0a2d80 msgr2=0x7f83cc0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:40.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83ea038640 1 --2- 172.21.15.84:0/2822839142 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f83cc0a2d80 0x7f83cc0a5240 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0cd1a0 tx=0x7f83dc005990 comp rx=0 tx=0).stop 2024-05-22T23:05:40.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83ea038640 1 -- 172.21.15.84:0/2822839142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83e4106750 msgr2=0x7f83e41a40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:40.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83ea038640 1 --2- 172.21.15.84:0/2822839142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83e4106750 0x7f83e41a40b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f83dc00eb10 tx=0x7f83dc00c2d0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83ea038640 1 -- 172.21.15.84:0/2822839142 shutdown_connections 2024-05-22T23:05:40.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83ea038640 1 --2- 172.21.15.84:0/2822839142 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f83cc0a2d80 0x7f83cc0a5240 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83ea038640 1 --2- 172.21.15.84:0/2822839142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83e4106750 0x7f83e41a40b0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83ea038640 1 --2- 172.21.15.84:0/2822839142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e4101100 0x7f83e4199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83ea038640 1 --2- 172.21.15.84:0/2822839142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e4100740 0x7f83e4199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:40.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.554+0000 7f83ea038640 1 -- 172.21.15.84:0/2822839142 >> 172.21.15.84:0/2822839142 conn(0x7f83e4077870 msgr2=0x7f83e41030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:40.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.555+0000 7f83ea038640 1 -- 172.21.15.84:0/2822839142 shutdown_connections 2024-05-22T23:05:40.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:40.555+0000 7f83ea038640 1 -- 172.21.15.84:0/2822839142 wait complete. 2024-05-22T23:05:40.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:40 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/152904259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:05:40.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:40 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/152904259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:05:40.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:40 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/152904259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:05:41.431 INFO:tasks.workunit.client.0.smithi084.stdout: 1 429491 15.55 MB/sec execute 27 sec latency 22.276 ms 2024-05-22T23:05:41.571 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-22T23:05:41.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:05:41.571 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-22T23:05:41.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.676+0000 7f6b36e10640 1 -- 172.21.15.84:0/2140559627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b30103410 msgr2=0x7f6b3010b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:41.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.676+0000 7f6b36e10640 1 --2- 172.21.15.84:0/2140559627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b30103410 0x7f6b3010b8f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009900 tx=0x7f6b2402a510 comp rx=0 tx=0).stop 2024-05-22T23:05:41.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.676+0000 7f6b36e10640 1 -- 172.21.15.84:0/2140559627 shutdown_connections 2024-05-22T23:05:41.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.676+0000 7f6b36e10640 1 --2- 172.21.15.84:0/2140559627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b3010be30 0x7f6b3010e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.676+0000 7f6b36e10640 1 --2- 172.21.15.84:0/2140559627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b30103410 0x7f6b3010b8f0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.676+0000 7f6b36e10640 1 --2- 172.21.15.84:0/2140559627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b30102af0 0x7f6b30102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.676+0000 7f6b36e10640 1 -- 172.21.15.84:0/2140559627 >> 172.21.15.84:0/2140559627 conn(0x7f6b30077870 msgr2=0x7f6b30077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:41.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.676+0000 7f6b36e10640 1 -- 172.21.15.84:0/2140559627 shutdown_connections 2024-05-22T23:05:41.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.676+0000 7f6b36e10640 1 -- 172.21.15.84:0/2140559627 wait complete. 2024-05-22T23:05:41.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.677+0000 7f6b36e10640 1 Processor -- start 2024-05-22T23:05:41.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.677+0000 7f6b36e10640 1 -- start start 2024-05-22T23:05:41.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.677+0000 7f6b36e10640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b30102af0 0x7f6b30198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.677+0000 7f6b36e10640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b30103410 0x7f6b301986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.677+0000 7f6b36e10640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b3010be30 0x7f6b3019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.677+0000 7f6b36e10640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6b30111c70 con 0x7f6b30102af0 2024-05-22T23:05:41.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.677+0000 7f6b36e10640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6b30111af0 con 0x7f6b30103410 2024-05-22T23:05:41.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.677+0000 7f6b36e10640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6b30111df0 con 0x7f6b3010be30 2024-05-22T23:05:41.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.677+0000 7f6b34bac640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b30102af0 0x7f6b30198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.677+0000 7f6b34bac640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b30102af0 0x7f6b30198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39926/0 (socket says 172.21.15.84:39926) 2024-05-22T23:05:41.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.677+0000 7f6b34bac640 1 -- 172.21.15.84:0/2798032285 learned_addr learned my addr 172.21.15.84:0/2798032285 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:41.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.677+0000 7f6b2ffff640 1 --2- 172.21.15.84:0/2798032285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b30103410 0x7f6b301986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.677+0000 7f6b353ad640 1 --2- 172.21.15.84:0/2798032285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b3010be30 0x7f6b3019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.678+0000 7f6b34bac640 1 -- 172.21.15.84:0/2798032285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b3010be30 msgr2=0x7f6b3019ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:41.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.678+0000 7f6b34bac640 1 --2- 172.21.15.84:0/2798032285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b3010be30 0x7f6b3019ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.678+0000 7f6b34bac640 1 -- 172.21.15.84:0/2798032285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b30103410 msgr2=0x7f6b301986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:41.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.678+0000 7f6b34bac640 1 --2- 172.21.15.84:0/2798032285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b30103410 0x7f6b301986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.678+0000 7f6b34bac640 1 -- 172.21.15.84:0/2798032285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3019d160 con 0x7f6b30102af0 2024-05-22T23:05:41.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.678+0000 7f6b34bac640 1 --2- 172.21.15.84:0/2798032285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b30102af0 0x7f6b30198170 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000e9f0 tx=0x7f6b2000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:41.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.678+0000 7f6b2dffb640 1 -- 172.21.15.84:0/2798032285 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2000cd80 con 0x7f6b30102af0 2024-05-22T23:05:41.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.678+0000 7f6b36e10640 1 -- 172.21.15.84:0/2798032285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b3019d450 con 0x7f6b30102af0 2024-05-22T23:05:41.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.678+0000 7f6b2dffb640 1 -- 172.21.15.84:0/2798032285 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b20010910 con 0x7f6b30102af0 2024-05-22T23:05:41.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.678+0000 7f6b2dffb640 1 -- 172.21.15.84:0/2798032285 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b200188f0 con 0x7f6b30102af0 2024-05-22T23:05:41.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.678+0000 7f6b36e10640 1 -- 172.21.15.84:0/2798032285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b301a4ce0 con 0x7f6b30102af0 2024-05-22T23:05:41.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.680+0000 7f6b2dffb640 1 -- 172.21.15.84:0/2798032285 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6b20010ac0 con 0x7f6b30102af0 2024-05-22T23:05:41.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.681+0000 7f6b39bf4740 1 Processor -- start 2024-05-22T23:05:41.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.681+0000 7f6b39bf4740 1 -- start start 2024-05-22T23:05:41.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.681+0000 7f6b39bf4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce14b30 0x55aabce14f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.681+0000 7f6b39bf4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabce15450 0x55aabce23b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.681+0000 7f6b39bf4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aabce24050 0x55aabce26440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.681+0000 7f6b39bf4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55aabcd78a40 con 0x55aabce14b30 2024-05-22T23:05:41.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.681+0000 7f6b39bf4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55aabcd17440 con 0x55aabce15450 2024-05-22T23:05:41.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.681+0000 7f6b39bf4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55aabcd74a00 con 0x55aabce24050 2024-05-22T23:05:41.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.681+0000 7f6b353ad640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce14b30 0x55aabce14f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.681+0000 7f6b34bac640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabce15450 0x55aabce23b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.681+0000 7f6b34bac640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabce15450 0x55aabce23b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:53136/0 (socket says 172.21.15.84:53136) 2024-05-22T23:05:41.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.681+0000 7f6b353ad640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce14b30 0x55aabce14f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39942/0 (socket says 172.21.15.84:39942) 2024-05-22T23:05:41.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.681+0000 7f6b34bac640 1 -- 172.21.15.84:0/144134690 learned_addr learned my addr 172.21.15.84:0/144134690 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:41.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.681+0000 7f6b2dffb640 1 --2- 172.21.15.84:0/2798032285 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6b180a2d00 0x7f6b180a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.681+0000 7f6b2ffff640 1 --2- 172.21.15.84:0/144134690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aabce24050 0x55aabce26440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.681+0000 7f6b353ad640 1 -- 172.21.15.84:0/144134690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aabce24050 msgr2=0x55aabce26440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:41.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.681+0000 7f6b353ad640 1 --2- 172.21.15.84:0/144134690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aabce24050 0x55aabce26440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.681+0000 7f6b353ad640 1 -- 172.21.15.84:0/144134690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabce15450 msgr2=0x55aabce23b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:41.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.681+0000 7f6b353ad640 1 --2- 172.21.15.84:0/144134690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabce15450 0x55aabce23b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.681+0000 7f6b353ad640 1 -- 172.21.15.84:0/144134690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aabcd212d0 con 0x55aabce14b30 2024-05-22T23:05:41.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.681+0000 7f6b353ad640 1 --2- 172.21.15.84:0/2798032285 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6b180a2d00 0x7f6b180a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.681+0000 7f6b2dffb640 1 -- 172.21.15.84:0/2798032285 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f6b200cf800 con 0x7f6b30102af0 2024-05-22T23:05:41.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.682+0000 7f6b353ad640 1 --2- 172.21.15.84:0/144134690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce14b30 0x55aabce14f10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800a400 tx=0x7f6b2800a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1467ff3f1dad5bcb server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:41.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.682+0000 7f6b16ffd640 1 -- 172.21.15.84:0/144134690 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b280177a0 con 0x55aabce14b30 2024-05-22T23:05:41.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.682+0000 7f6b16ffd640 1 -- 172.21.15.84:0/144134690 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b28017e20 con 0x55aabce14b30 2024-05-22T23:05:41.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.682+0000 7f6b353ad640 1 --2- 172.21.15.84:0/2798032285 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6b180a2d00 0x7f6b180a51c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6b28020000 tx=0x7f6b28021000 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:41.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.682+0000 7f6b16ffd640 1 -- 172.21.15.84:0/144134690 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b28023e50 con 0x55aabce14b30 2024-05-22T23:05:41.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.682+0000 7f6b39bf4740 1 -- 172.21.15.84:0/144134690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce14b30 msgr2=0x55aabce14f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:41.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.682+0000 7f6b39bf4740 1 --2- 172.21.15.84:0/144134690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce14b30 0x55aabce14f10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800a400 tx=0x7f6b2800a8c0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.682+0000 7f6b39bf4740 1 -- 172.21.15.84:0/144134690 shutdown_connections 2024-05-22T23:05:41.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.682+0000 7f6b39bf4740 1 --2- 172.21.15.84:0/144134690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aabce24050 0x55aabce26440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.682+0000 7f6b39bf4740 1 --2- 172.21.15.84:0/144134690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabce15450 0x55aabce23b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.682+0000 7f6b39bf4740 1 --2- 172.21.15.84:0/144134690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce14b30 0x55aabce14f10 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.682+0000 7f6b39bf4740 1 -- 172.21.15.84:0/144134690 >> 172.21.15.84:0/144134690 conn(0x55aabce10f70 msgr2=0x55aabce11350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:41.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.682+0000 7f6b39bf4740 1 -- 172.21.15.84:0/144134690 shutdown_connections 2024-05-22T23:05:41.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.683+0000 7f6b39bf4740 1 -- 172.21.15.84:0/144134690 wait complete. 2024-05-22T23:05:41.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.683+0000 7f6b39bf4740 1 -- start start 2024-05-22T23:05:41.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.683+0000 7f6b39bf4740 1 Processor -- start 2024-05-22T23:05:41.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.683+0000 7f6b39bf4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce14b30 0x55aabce20f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.683+0000 7f6b39bf4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aabce15450 0x55aabce214d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.683+0000 7f6b39bf4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabce24050 0x55aabce21a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.683+0000 7f6b39bf4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55aabce26e90 con 0x55aabce14b30 2024-05-22T23:05:41.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.683+0000 7f6b39bf4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55aabcd025f0 con 0x55aabce24050 2024-05-22T23:05:41.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.683+0000 7f6b39bf4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55aabce27010 con 0x55aabce15450 2024-05-22T23:05:41.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.683+0000 7f6b39bf4740 10 client.0 _ll_register_callbacks cb 0x55aabccfdcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:05:41.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.683+0000 7f6b34bac640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce14b30 0x55aabce20f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.683+0000 7f6b34bac640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce14b30 0x55aabce20f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39956/0 (socket says 172.21.15.84:39956) 2024-05-22T23:05:41.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.683+0000 7f6b34bac640 1 -- 172.21.15.84:0/164246715 learned_addr learned my addr 172.21.15.84:0/164246715 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:41.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.683+0000 7f6b353ad640 1 --2- 172.21.15.84:0/164246715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabce24050 0x55aabce21a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.684+0000 7f6b2ffff640 1 --2- 172.21.15.84:0/164246715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aabce15450 0x55aabce214d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.684+0000 7f6b34bac640 1 -- 172.21.15.84:0/164246715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aabce15450 msgr2=0x55aabce214d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:41.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.684+0000 7f6b34bac640 1 --2- 172.21.15.84:0/164246715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aabce15450 0x55aabce214d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.684+0000 7f6b34bac640 1 -- 172.21.15.84:0/164246715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabce24050 msgr2=0x55aabce21a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:41.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.684+0000 7f6b34bac640 1 --2- 172.21.15.84:0/164246715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabce24050 0x55aabce21a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.684+0000 7f6b34bac640 1 -- 172.21.15.84:0/164246715 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aabcb547e0 con 0x55aabce14b30 2024-05-22T23:05:41.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.684+0000 7f6b2ffff640 1 --2- 172.21.15.84:0/164246715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aabce15450 0x55aabce214d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:05:41.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.684+0000 7f6b34bac640 1 --2- 172.21.15.84:0/164246715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce14b30 0x55aabce20f90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000c6a0 tx=0x7f6b2009bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5adb6c168be9ffb server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:41.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.684+0000 7f6b0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:05:41.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.684+0000 7f6b0ffff640 1 -- 172.21.15.84:0/164246715 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b200cd3e0 con 0x55aabce14b30 2024-05-22T23:05:41.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.684+0000 7f6b0ffff640 1 -- 172.21.15.84:0/164246715 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f6b200a4700 con 0x55aabce14b30 2024-05-22T23:05:41.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.684+0000 7f6b39bf4740 1 -- 172.21.15.84:0/164246715 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b28017e20 con 0x55aabce14b30 2024-05-22T23:05:41.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.684+0000 7f6b0ffff640 1 -- 172.21.15.84:0/164246715 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f6b200a00d0 con 0x55aabce14b30 2024-05-22T23:05:41.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.684+0000 7f6b39bf4740 10 client.16068 fetch_fsmap learned FSMap version 18 2024-05-22T23:05:41.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.684+0000 7f6b39bf4740 1 -- 172.21.15.84:0/164246715 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55aabcbb4f70 con 0x55aabce14b30 2024-05-22T23:05:41.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.684+0000 7f6b0ffff640 1 -- 172.21.15.84:0/164246715 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b28017e20 con 0x55aabce14b30 2024-05-22T23:05:41.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.684+0000 7f6b0ffff640 1 -- 172.21.15.84:0/164246715 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f6b20018c10 con 0x55aabce14b30 2024-05-22T23:05:41.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.684+0000 7f6b39bf4740 10 client.16068 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:05:41.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.684+0000 7f6b39bf4740 10 client.16068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:05:41.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.684+0000 7f6b39bf4740 20 client.16068 populate_metadata read hostname 'smithi084' 2024-05-22T23:05:41.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.685+0000 7f6b39bf4740 1 --2- 172.21.15.84:0/164246715 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55aabce343f0 0x55aabce36800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.685+0000 7f6b39bf4740 4 client.16068 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:05:41.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.685+0000 7f6b39bf4740 1 -- 172.21.15.84:0/164246715 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aabcd8c460 con 0x55aabce343f0 2024-05-22T23:05:41.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.685+0000 7f6b2ffff640 1 --2- 172.21.15.84:0/164246715 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55aabce343f0 0x55aabce36800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.685+0000 7f6b0ffff640 10 client.16068 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:41.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.685+0000 7f6b2ffff640 1 --2- 172.21.15.84:0/164246715 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55aabce343f0 0x55aabce36800 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ad1c54b589a708d server_cookie=fa2c3bde736ffc91 in_seq=0 out_seq=0 2024-05-22T23:05:41.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.686+0000 7f6b0ffff640 1 -- 172.21.15.84:0/164246715 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aabcd8c460 con 0x55aabce343f0 2024-05-22T23:05:41.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.686+0000 7f6b0ffff640 10 client.16068 handle_command_reply: tid=0 2024-05-22T23:05:41.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.686+0000 7f6b39bf4740 1 client.16068 shutdown 2024-05-22T23:05:41.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.686+0000 7f6b39bf4740 1 -- 172.21.15.84:0/164246715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce14b30 msgr2=0x55aabce20f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:41.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.686+0000 7f6b39bf4740 1 --2- 172.21.15.84:0/164246715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce14b30 0x55aabce20f90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000c6a0 tx=0x7f6b2009bbc0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.687+0000 7f6b39bf4740 1 -- 172.21.15.84:0/164246715 shutdown_connections 2024-05-22T23:05:41.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.687+0000 7f6b39bf4740 1 --2- 172.21.15.84:0/164246715 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55aabce343f0 0x55aabce36800 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.687+0000 7f6b39bf4740 1 --2- 172.21.15.84:0/164246715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabce24050 0x55aabce21a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.687+0000 7f6b39bf4740 1 --2- 172.21.15.84:0/164246715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aabce15450 0x55aabce214d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.687+0000 7f6b39bf4740 1 --2- 172.21.15.84:0/164246715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce14b30 0x55aabce20f90 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.687+0000 7f6b0ffff640 0 client.16068 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:41.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.687+0000 7f6b39bf4740 1 -- 172.21.15.84:0/164246715 >> 172.21.15.84:0/164246715 conn(0x55aabce10f70 msgr2=0x55aabce16d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:41.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.687+0000 7f6b39bf4740 1 -- 172.21.15.84:0/164246715 shutdown_connections 2024-05-22T23:05:41.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.687+0000 7f6b39bf4740 1 -- 172.21.15.84:0/164246715 wait complete. 2024-05-22T23:05:41.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.687+0000 7f6b39bf4740 20 client.16068 trim_cache size 0 max 16384 2024-05-22T23:05:41.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.707+0000 7f6b39bf4740 1 Processor -- start 2024-05-22T23:05:41.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.708+0000 7f6b39bf4740 1 -- start start 2024-05-22T23:05:41.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.708+0000 7f6b39bf4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabce14b30 0x55aabce2fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.708+0000 7f6b39bf4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aabce15450 0x55aabce3c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.708+0000 7f6b39bf4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce24050 0x55aabce3c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.708+0000 7f6b39bf4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6b301a94f0 con 0x55aabce24050 2024-05-22T23:05:41.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.708+0000 7f6b39bf4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6b301abd70 con 0x55aabce14b30 2024-05-22T23:05:41.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.708+0000 7f6b39bf4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6b301a9b60 con 0x55aabce15450 2024-05-22T23:05:41.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.708+0000 7f6b353ad640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce24050 0x55aabce3c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.708+0000 7f6b353ad640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce24050 0x55aabce3c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39958/0 (socket says 172.21.15.84:39958) 2024-05-22T23:05:41.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.708+0000 7f6b353ad640 1 -- 172.21.15.84:0/1016659274 learned_addr learned my addr 172.21.15.84:0/1016659274 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:41.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.708+0000 7f6b34bac640 1 --2- 172.21.15.84:0/1016659274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabce14b30 0x55aabce2fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.708+0000 7f6b2ffff640 1 --2- 172.21.15.84:0/1016659274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aabce15450 0x55aabce3c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.708+0000 7f6b353ad640 1 -- 172.21.15.84:0/1016659274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aabce15450 msgr2=0x55aabce3c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:41.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.708+0000 7f6b353ad640 1 --2- 172.21.15.84:0/1016659274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aabce15450 0x55aabce3c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.708+0000 7f6b353ad640 1 -- 172.21.15.84:0/1016659274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabce14b30 msgr2=0x55aabce2fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:41.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.708+0000 7f6b353ad640 1 --2- 172.21.15.84:0/1016659274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabce14b30 0x55aabce2fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.708+0000 7f6b353ad640 1 -- 172.21.15.84:0/1016659274 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aabca4c170 con 0x55aabce24050 2024-05-22T23:05:41.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.708+0000 7f6b353ad640 1 --2- 172.21.15.84:0/1016659274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce24050 0x55aabce3c610 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6b28009b70 tx=0x7f6b28009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc15ab98751571b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:41.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.709+0000 7f6b167fc640 1 -- 172.21.15.84:0/1016659274 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b28033020 con 0x55aabce24050 2024-05-22T23:05:41.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.709+0000 7f6b167fc640 1 -- 172.21.15.84:0/1016659274 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b28029040 con 0x55aabce24050 2024-05-22T23:05:41.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.709+0000 7f6b167fc640 1 -- 172.21.15.84:0/1016659274 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b28028040 con 0x55aabce24050 2024-05-22T23:05:41.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.709+0000 7f6b39bf4740 1 -- 172.21.15.84:0/1016659274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce24050 msgr2=0x55aabce3c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:41.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.709+0000 7f6b39bf4740 1 --2- 172.21.15.84:0/1016659274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce24050 0x55aabce3c610 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6b28009b70 tx=0x7f6b28009b00 comp rx=0 tx=0).stop 2024-05-22T23:05:41.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.709+0000 7f6b39bf4740 1 -- 172.21.15.84:0/1016659274 shutdown_connections 2024-05-22T23:05:41.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.709+0000 7f6b39bf4740 1 --2- 172.21.15.84:0/1016659274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce24050 0x55aabce3c610 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.709+0000 7f6b39bf4740 1 --2- 172.21.15.84:0/1016659274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aabce15450 0x55aabce3c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.709+0000 7f6b39bf4740 1 --2- 172.21.15.84:0/1016659274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabce14b30 0x55aabce2fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.709+0000 7f6b39bf4740 1 -- 172.21.15.84:0/1016659274 >> 172.21.15.84:0/1016659274 conn(0x55aabce10f70 msgr2=0x55aabce101b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:41.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.709+0000 7f6b39bf4740 1 -- 172.21.15.84:0/1016659274 shutdown_connections 2024-05-22T23:05:41.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.710+0000 7f6b39bf4740 1 -- 172.21.15.84:0/1016659274 wait complete. 2024-05-22T23:05:41.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.710+0000 7f6b39bf4740 1 -- start start 2024-05-22T23:05:41.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.710+0000 7f6b39bf4740 1 Processor -- start 2024-05-22T23:05:41.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.710+0000 7f6b39bf4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aabce14b30 0x55aabce2c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.710+0000 7f6b39bf4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce15450 0x55aabce292a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.710+0000 7f6b39bf4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabce24050 0x55aabce297e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.710+0000 7f6b39bf4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6b301aa230 con 0x55aabce15450 2024-05-22T23:05:41.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.710+0000 7f6b39bf4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6b301ab6a0 con 0x55aabce24050 2024-05-22T23:05:41.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.710+0000 7f6b39bf4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6b301aa900 con 0x55aabce14b30 2024-05-22T23:05:41.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.710+0000 7f6b39bf4740 10 client.0 _ll_register_callbacks cb 0x7f6b20009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:05:41.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.710+0000 7f6b2ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce15450 0x55aabce292a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.710+0000 7f6b2ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce15450 0x55aabce292a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39972/0 (socket says 172.21.15.84:39972) 2024-05-22T23:05:41.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.710+0000 7f6b2ffff640 1 -- 172.21.15.84:0/3778941226 learned_addr learned my addr 172.21.15.84:0/3778941226 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:41.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.710+0000 7f6b353ad640 1 --2- 172.21.15.84:0/3778941226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabce24050 0x55aabce297e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.710+0000 7f6b34bac640 1 --2- 172.21.15.84:0/3778941226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aabce14b30 0x55aabce2c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.711+0000 7f6b2ffff640 1 -- 172.21.15.84:0/3778941226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aabce14b30 msgr2=0x55aabce2c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:41.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.711+0000 7f6b2ffff640 1 --2- 172.21.15.84:0/3778941226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aabce14b30 0x55aabce2c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.711+0000 7f6b2ffff640 1 -- 172.21.15.84:0/3778941226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabce24050 msgr2=0x55aabce297e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:41.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.711+0000 7f6b2ffff640 1 --2- 172.21.15.84:0/3778941226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabce24050 0x55aabce297e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.711+0000 7f6b2ffff640 1 -- 172.21.15.84:0/3778941226 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aabcd41930 con 0x55aabce15450 2024-05-22T23:05:41.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.711+0000 7f6b2ffff640 1 --2- 172.21.15.84:0/3778941226 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce15450 0x55aabce292a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009230 tx=0x7f6b24031950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589f07d8fefdb13a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:41.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.711+0000 7f6b0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:05:41.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.711+0000 7f6b0ffff640 1 -- 172.21.15.84:0/3778941226 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b24049070 con 0x55aabce15450 2024-05-22T23:05:41.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.711+0000 7f6b0ffff640 1 -- 172.21.15.84:0/3778941226 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f6b24004040 con 0x55aabce15450 2024-05-22T23:05:41.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.711+0000 7f6b39bf4740 1 -- 172.21.15.84:0/3778941226 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b28029040 con 0x55aabce15450 2024-05-22T23:05:41.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.711+0000 7f6b0ffff640 1 -- 172.21.15.84:0/3778941226 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f6b24035040 con 0x55aabce15450 2024-05-22T23:05:41.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.711+0000 7f6b39bf4740 10 client.16074 fetch_fsmap learned FSMap version 18 2024-05-22T23:05:41.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.711+0000 7f6b39bf4740 1 -- 172.21.15.84:0/3778941226 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55aabce9b6c0 con 0x55aabce15450 2024-05-22T23:05:41.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.711+0000 7f6b0ffff640 1 -- 172.21.15.84:0/3778941226 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b28029040 con 0x55aabce15450 2024-05-22T23:05:41.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.711+0000 7f6b0ffff640 1 -- 172.21.15.84:0/3778941226 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f6b24003710 con 0x55aabce15450 2024-05-22T23:05:41.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.711+0000 7f6b39bf4740 10 client.16074 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:05:41.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.711+0000 7f6b39bf4740 10 client.16074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:05:41.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.711+0000 7f6b39bf4740 20 client.16074 populate_metadata read hostname 'smithi084' 2024-05-22T23:05:41.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.712+0000 7f6b39bf4740 1 --2- 172.21.15.84:0/3778941226 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55aabce9bdd0 0x55aabce9e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.712+0000 7f6b39bf4740 4 client.16074 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-22T23:05:41.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.712+0000 7f6b39bf4740 1 -- 172.21.15.84:0/3778941226 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55aabcd86170 con 0x55aabce9bdd0 2024-05-22T23:05:41.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.712+0000 7f6b34bac640 1 --2- 172.21.15.84:0/3778941226 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55aabce9bdd0 0x55aabce9e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.712+0000 7f6b0ffff640 10 client.16074 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:41.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.712+0000 7f6b34bac640 1 --2- 172.21.15.84:0/3778941226 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55aabce9bdd0 0x55aabce9e1f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2e779d5ca44d0a0 server_cookie=7533112f028fc50c in_seq=0 out_seq=0 2024-05-22T23:05:41.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.713+0000 7f6b0ffff640 1 -- 172.21.15.84:0/3778941226 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55aabcd86170 con 0x55aabce9bdd0 2024-05-22T23:05:41.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.713+0000 7f6b0ffff640 10 client.16074 handle_command_reply: tid=0 2024-05-22T23:05:41.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.713+0000 7f6b39bf4740 1 client.16074 shutdown 2024-05-22T23:05:41.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.714+0000 7f6b39bf4740 1 -- 172.21.15.84:0/3778941226 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce15450 msgr2=0x55aabce292a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:41.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.714+0000 7f6b39bf4740 1 --2- 172.21.15.84:0/3778941226 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce15450 0x55aabce292a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009230 tx=0x7f6b24031950 comp rx=0 tx=0).stop 2024-05-22T23:05:41.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.714+0000 7f6b39bf4740 1 -- 172.21.15.84:0/3778941226 shutdown_connections 2024-05-22T23:05:41.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.714+0000 7f6b39bf4740 1 --2- 172.21.15.84:0/3778941226 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55aabce9bdd0 0x55aabce9e1f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.714+0000 7f6b39bf4740 1 --2- 172.21.15.84:0/3778941226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabce24050 0x55aabce297e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.714+0000 7f6b39bf4740 1 --2- 172.21.15.84:0/3778941226 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aabce15450 0x55aabce292a0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.714+0000 7f6b0ffff640 0 client.16074 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:41.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.714+0000 7f6b39bf4740 1 --2- 172.21.15.84:0/3778941226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aabce14b30 0x55aabce2c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.714+0000 7f6b39bf4740 1 -- 172.21.15.84:0/3778941226 >> 172.21.15.84:0/3778941226 conn(0x55aabce10f70 msgr2=0x55aabce20f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:41.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.714+0000 7f6b39bf4740 1 -- 172.21.15.84:0/3778941226 shutdown_connections 2024-05-22T23:05:41.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.714+0000 7f6b39bf4740 1 -- 172.21.15.84:0/3778941226 wait complete. 2024-05-22T23:05:41.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.714+0000 7f6b39bf4740 20 client.16074 trim_cache size 0 max 16384 2024-05-22T23:05:41.724 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:05:41.724 INFO:teuthology.orchestra.run.smithi084.stdout: "return_code": 0, 2024-05-22T23:05:41.724 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_tag": "a1d1c1fe-612a-498a-878f-7a7b268310cd", 2024-05-22T23:05:41.724 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": "asynchronous" 2024-05-22T23:05:41.724 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:05:41.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.715+0000 7f6b36e10640 1 -- 172.21.15.84:0/2798032285 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6b180a2d00 msgr2=0x7f6b180a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:41.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.715+0000 7f6b36e10640 1 --2- 172.21.15.84:0/2798032285 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6b180a2d00 0x7f6b180a51c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6b28020000 tx=0x7f6b28021000 comp rx=0 tx=0).stop 2024-05-22T23:05:41.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.715+0000 7f6b36e10640 1 -- 172.21.15.84:0/2798032285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b30102af0 msgr2=0x7f6b30198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:41.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.715+0000 7f6b36e10640 1 --2- 172.21.15.84:0/2798032285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b30102af0 0x7f6b30198170 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000e9f0 tx=0x7f6b2000eeb0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.715+0000 7f6b36e10640 1 -- 172.21.15.84:0/2798032285 shutdown_connections 2024-05-22T23:05:41.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.715+0000 7f6b36e10640 1 --2- 172.21.15.84:0/2798032285 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6b180a2d00 0x7f6b180a51c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.715+0000 7f6b36e10640 1 --2- 172.21.15.84:0/2798032285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b3010be30 0x7f6b3019ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.715+0000 7f6b36e10640 1 --2- 172.21.15.84:0/2798032285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b30103410 0x7f6b301986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.715+0000 7f6b36e10640 1 --2- 172.21.15.84:0/2798032285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b30102af0 0x7f6b30198170 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.715+0000 7f6b36e10640 1 -- 172.21.15.84:0/2798032285 >> 172.21.15.84:0/2798032285 conn(0x7f6b30077870 msgr2=0x7f6b300fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:41.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.715+0000 7f6b36e10640 1 -- 172.21.15.84:0/2798032285 shutdown_connections 2024-05-22T23:05:41.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.715+0000 7f6b36e10640 1 -- 172.21.15.84:0/2798032285 wait complete. 2024-05-22T23:05:41.729 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:41 smithi084 ceph-mon[32186]: pgmap v118: 97 pgs: 97 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 246 op/s 2024-05-22T23:05:41.731 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:05:41.731 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:05:41.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:41 smithi096 ceph-mon[36392]: pgmap v118: 97 pgs: 97 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 246 op/s 2024-05-22T23:05:41.846 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.840+0000 7f7bd6e10640 1 -- 172.21.15.84:0/2873691614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bd0101180 msgr2=0x7f7bd010d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:41.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.840+0000 7f7bd6e10640 1 --2- 172.21.15.84:0/2873691614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bd0101180 0x7f7bd010d690 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8009930 tx=0x7f7bc802a420 comp rx=0 tx=0).stop 2024-05-22T23:05:41.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.840+0000 7f7bd6e10640 1 -- 172.21.15.84:0/2873691614 shutdown_connections 2024-05-22T23:05:41.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.840+0000 7f7bd6e10640 1 --2- 172.21.15.84:0/2873691614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bd0101180 0x7f7bd010d690 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.840+0000 7f7bd6e10640 1 --2- 172.21.15.84:0/2873691614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bd01007c0 0x7f7bd0100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.840+0000 7f7bd6e10640 1 --2- 172.21.15.84:0/2873691614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bd01067d0 0x7f7bd0106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.840+0000 7f7bd6e10640 1 -- 172.21.15.84:0/2873691614 >> 172.21.15.84:0/2873691614 conn(0x7f7bd00778a0 msgr2=0x7f7bd0077cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:41.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.840+0000 7f7bd6e10640 1 -- 172.21.15.84:0/2873691614 shutdown_connections 2024-05-22T23:05:41.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.840+0000 7f7bd6e10640 1 -- 172.21.15.84:0/2873691614 wait complete. 2024-05-22T23:05:41.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.841+0000 7f7bd6e10640 1 Processor -- start 2024-05-22T23:05:41.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.841+0000 7f7bd6e10640 1 -- start start 2024-05-22T23:05:41.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.841+0000 7f7bd6e10640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bd01007c0 0x7f7bd0198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.841+0000 7f7bd6e10640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bd0101180 0x7f7bd01987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.841+0000 7f7bd6e10640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bd01067d0 0x7f7bd019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.841+0000 7f7bd6e10640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd0110be0 con 0x7f7bd0101180 2024-05-22T23:05:41.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.841+0000 7f7bd6e10640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd0110a60 con 0x7f7bd01067d0 2024-05-22T23:05:41.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.841+0000 7f7bd6e10640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd0110d60 con 0x7f7bd01007c0 2024-05-22T23:05:41.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.841+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bd0101180 0x7f7bd01987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.841+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bd0101180 0x7f7bd01987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40000/0 (socket says 172.21.15.84:40000) 2024-05-22T23:05:41.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.841+0000 7f7bc7fff640 1 -- 172.21.15.84:0/804070309 learned_addr learned my addr 172.21.15.84:0/804070309 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:41.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.841+0000 7f7bd53ad640 1 --2- 172.21.15.84:0/804070309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bd01067d0 0x7f7bd019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.841+0000 7f7bd4bac640 1 --2- 172.21.15.84:0/804070309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bd01007c0 0x7f7bd0198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.841+0000 7f7bc7fff640 1 -- 172.21.15.84:0/804070309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bd01007c0 msgr2=0x7f7bd0198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:41.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.841+0000 7f7bc7fff640 1 --2- 172.21.15.84:0/804070309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bd01007c0 0x7f7bd0198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.841+0000 7f7bc7fff640 1 -- 172.21.15.84:0/804070309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bd01067d0 msgr2=0x7f7bd019cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:41.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.841+0000 7f7bc7fff640 1 --2- 172.21.15.84:0/804070309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bd01067d0 0x7f7bd019cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.841+0000 7f7bc7fff640 1 -- 172.21.15.84:0/804070309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd019d280 con 0x7f7bd0101180 2024-05-22T23:05:41.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.842+0000 7f7bd53ad640 1 --2- 172.21.15.84:0/804070309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bd01067d0 0x7f7bd019cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:05:41.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.842+0000 7f7bc7fff640 1 --2- 172.21.15.84:0/804070309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bd0101180 0x7f7bd01987d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000e9f0 tx=0x7f7bc000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:41.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.842+0000 7f7bc5ffb640 1 -- 172.21.15.84:0/804070309 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc000cd80 con 0x7f7bd0101180 2024-05-22T23:05:41.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.842+0000 7f7bc5ffb640 1 -- 172.21.15.84:0/804070309 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bc0005500 con 0x7f7bd0101180 2024-05-22T23:05:41.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.842+0000 7f7bd6e10640 1 -- 172.21.15.84:0/804070309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd019d570 con 0x7f7bd0101180 2024-05-22T23:05:41.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.842+0000 7f7bc5ffb640 1 -- 172.21.15.84:0/804070309 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0010690 con 0x7f7bd0101180 2024-05-22T23:05:41.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.842+0000 7f7bd6e10640 1 -- 172.21.15.84:0/804070309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd01a4e00 con 0x7f7bd0101180 2024-05-22T23:05:41.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.844+0000 7f7bc5ffb640 1 -- 172.21.15.84:0/804070309 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7bc00046c0 con 0x7f7bd0101180 2024-05-22T23:05:41.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.845+0000 7f7bc5ffb640 1 --2- 172.21.15.84:0/804070309 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f7bb80a2d00 0x7f7bb80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.845+0000 7f7bd9bf3740 1 Processor -- start 2024-05-22T23:05:41.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.845+0000 7f7bd9bf3740 1 -- start start 2024-05-22T23:05:41.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.845+0000 7f7bc5ffb640 1 -- 172.21.15.84:0/804070309 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f7bc00cf870 con 0x7f7bd0101180 2024-05-22T23:05:41.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.845+0000 7f7bd4bac640 1 --2- 172.21.15.84:0/804070309 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f7bb80a2d00 0x7f7bb80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.845+0000 7f7bd9bf3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d1b39e4a0 0x560d1b39c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.845+0000 7f7bd9bf3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d1b39cdb0 0x560d1b39d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.845+0000 7f7bd9bf3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b3a7b30 0x560d1b3a9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.845+0000 7f7bd9bf3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560d1b300db0 con 0x560d1b3a7b30 2024-05-22T23:05:41.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.845+0000 7f7bd9bf3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560d1b29f360 con 0x560d1b39cdb0 2024-05-22T23:05:41.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.845+0000 7f7bd9bf3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560d1b2fcd70 con 0x560d1b39e4a0 2024-05-22T23:05:41.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.845+0000 7f7bd4bac640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d1b39e4a0 0x560d1b39c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.845+0000 7f7bd53ad640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b3a7b30 0x560d1b3a9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.845+0000 7f7bd4bac640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d1b39e4a0 0x560d1b39c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:47712/0 (socket says 172.21.15.84:47712) 2024-05-22T23:05:41.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.845+0000 7f7bd4bac640 1 -- 172.21.15.84:0/1086035580 learned_addr learned my addr 172.21.15.84:0/1086035580 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:41.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.845+0000 7f7bc7fff640 1 --2- 172.21.15.84:0/1086035580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d1b39cdb0 0x560d1b39d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.846+0000 7f7bd4bac640 1 --2- 172.21.15.84:0/804070309 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f7bb80a2d00 0x7f7bb80a51c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc009fd0 tx=0x7f7bcc011c60 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:41.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.846+0000 7f7bc7fff640 1 -- 172.21.15.84:0/1086035580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d1b39e4a0 msgr2=0x560d1b39c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:41.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.846+0000 7f7bc7fff640 1 --2- 172.21.15.84:0/1086035580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d1b39e4a0 0x560d1b39c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.846+0000 7f7bc7fff640 1 -- 172.21.15.84:0/1086035580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b3a7b30 msgr2=0x560d1b3a9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:41.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.846+0000 7f7bc7fff640 1 --2- 172.21.15.84:0/1086035580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b3a7b30 0x560d1b3a9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.846+0000 7f7bc7fff640 1 -- 172.21.15.84:0/1086035580 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d1b2c2a60 con 0x560d1b39cdb0 2024-05-22T23:05:41.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.846+0000 7f7bd53ad640 1 --2- 172.21.15.84:0/1086035580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b3a7b30 0x560d1b3a9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:05:41.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.846+0000 7f7bd4bac640 1 --2- 172.21.15.84:0/1086035580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d1b39e4a0 0x560d1b39c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:05:41.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.846+0000 7f7bc7fff640 1 --2- 172.21.15.84:0/1086035580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d1b39cdb0 0x560d1b39d230 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00ccfc0 tx=0x7f7bc009b6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c7e5ddee6909e57 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:41.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.846+0000 7f7bb6ffd640 1 -- 172.21.15.84:0/1086035580 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc00a0140 con 0x560d1b39cdb0 2024-05-22T23:05:41.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.846+0000 7f7bb6ffd640 1 -- 172.21.15.84:0/1086035580 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bc00a4a00 con 0x560d1b39cdb0 2024-05-22T23:05:41.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.846+0000 7f7bb6ffd640 1 -- 172.21.15.84:0/1086035580 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc009b1e0 con 0x560d1b39cdb0 2024-05-22T23:05:41.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.847+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/1086035580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d1b39cdb0 msgr2=0x560d1b39d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:41.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.847+0000 7f7bd9bf3740 1 --2- 172.21.15.84:0/1086035580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d1b39cdb0 0x560d1b39d230 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00ccfc0 tx=0x7f7bc009b6f0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.847+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/1086035580 shutdown_connections 2024-05-22T23:05:41.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.847+0000 7f7bd9bf3740 1 --2- 172.21.15.84:0/1086035580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b3a7b30 0x560d1b3a9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.847+0000 7f7bd9bf3740 1 --2- 172.21.15.84:0/1086035580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d1b39cdb0 0x560d1b39d230 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.847+0000 7f7bd9bf3740 1 --2- 172.21.15.84:0/1086035580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d1b39e4a0 0x560d1b39c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.847+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/1086035580 >> 172.21.15.84:0/1086035580 conn(0x560d1b3658e0 msgr2=0x560d1b399310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:41.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.847+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/1086035580 shutdown_connections 2024-05-22T23:05:41.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.847+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/1086035580 wait complete. 2024-05-22T23:05:41.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.847+0000 7f7bd9bf3740 1 -- start start 2024-05-22T23:05:41.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.847+0000 7f7bd9bf3740 1 Processor -- start 2024-05-22T23:05:41.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.848+0000 7f7bd9bf3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b39cdb0 0x560d1b3a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.848+0000 7f7bd9bf3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d1b39e4a0 0x560d1b3a5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.848+0000 7f7bd9bf3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d1b3a7b30 0x560d1b3a5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.848+0000 7f7bd9bf3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560d1b3aa9c0 con 0x560d1b39cdb0 2024-05-22T23:05:41.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.848+0000 7f7bd9bf3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560d1b2bb0f0 con 0x560d1b39e4a0 2024-05-22T23:05:41.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.848+0000 7f7bd9bf3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560d1b3aab40 con 0x560d1b3a7b30 2024-05-22T23:05:41.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.848+0000 7f7bd4bac640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b39cdb0 0x560d1b3a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.848+0000 7f7bd4bac640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b39cdb0 0x560d1b3a4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40014/0 (socket says 172.21.15.84:40014) 2024-05-22T23:05:41.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.848+0000 7f7bd4bac640 1 -- 172.21.15.84:0/3929807339 learned_addr learned my addr 172.21.15.84:0/3929807339 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:41.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.848+0000 7f7bc7fff640 1 --2- 172.21.15.84:0/3929807339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d1b39e4a0 0x560d1b3a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.848+0000 7f7bd53ad640 1 --2- 172.21.15.84:0/3929807339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d1b3a7b30 0x560d1b3a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.848+0000 7f7bd9bf3740 10 client.0 _ll_register_callbacks cb 0x560d1b2bff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:05:41.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.848+0000 7f7bd4bac640 1 -- 172.21.15.84:0/3929807339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d1b3a7b30 msgr2=0x560d1b3a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:41.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.848+0000 7f7bd4bac640 1 --2- 172.21.15.84:0/3929807339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d1b3a7b30 0x560d1b3a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.848+0000 7f7bd4bac640 1 -- 172.21.15.84:0/3929807339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d1b39e4a0 msgr2=0x560d1b3a5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:41.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.848+0000 7f7bd4bac640 1 --2- 172.21.15.84:0/3929807339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d1b39e4a0 0x560d1b3a5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.848+0000 7f7bd4bac640 1 -- 172.21.15.84:0/3929807339 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d1b0de7d0 con 0x560d1b39cdb0 2024-05-22T23:05:41.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.848+0000 7f7bc7fff640 1 --2- 172.21.15.84:0/3929807339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d1b39e4a0 0x560d1b3a5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:05:41.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.848+0000 7f7bd4bac640 1 --2- 172.21.15.84:0/3929807339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b39cdb0 0x560d1b3a4ac0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc011580 tx=0x7f7bcc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252f884b927cb1c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:41.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.848+0000 7f7baffff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:05:41.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.848+0000 7f7baffff640 1 -- 172.21.15.84:0/3929807339 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc01a040 con 0x560d1b39cdb0 2024-05-22T23:05:41.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.848+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/3929807339 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bc00a4a00 con 0x560d1b39cdb0 2024-05-22T23:05:41.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.848+0000 7f7baffff640 1 -- 172.21.15.84:0/3929807339 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f7bcc01f3b0 con 0x560d1b39cdb0 2024-05-22T23:05:41.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.849+0000 7f7baffff640 1 -- 172.21.15.84:0/3929807339 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f7bcc028600 con 0x560d1b39cdb0 2024-05-22T23:05:41.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.849+0000 7f7bd9bf3740 10 client.16086 fetch_fsmap learned FSMap version 18 2024-05-22T23:05:41.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.849+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/3929807339 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x560d1b0143f0 con 0x560d1b39cdb0 2024-05-22T23:05:41.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.849+0000 7f7baffff640 1 -- 172.21.15.84:0/3929807339 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bc00a4a00 con 0x560d1b39cdb0 2024-05-22T23:05:41.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.849+0000 7f7baffff640 1 -- 172.21.15.84:0/3929807339 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f7bcc01d030 con 0x560d1b39cdb0 2024-05-22T23:05:41.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.849+0000 7f7bd9bf3740 10 client.16086 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:05:41.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.849+0000 7f7bd9bf3740 10 client.16086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:05:41.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.849+0000 7f7bd9bf3740 20 client.16086 populate_metadata read hostname 'smithi084' 2024-05-22T23:05:41.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.849+0000 7f7bd9bf3740 1 --2- 172.21.15.84:0/3929807339 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x560d1b3b7f80 0x560d1b3ba3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.849+0000 7f7bd9bf3740 4 client.16086 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:05:41.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.849+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/3929807339 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d1b30e4e0 con 0x560d1b3b7f80 2024-05-22T23:05:41.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.849+0000 7f7bc7fff640 1 --2- 172.21.15.84:0/3929807339 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x560d1b3b7f80 0x560d1b3ba3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.850+0000 7f7baffff640 10 client.16086 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:41.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.850+0000 7f7bc7fff640 1 --2- 172.21.15.84:0/3929807339 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x560d1b3b7f80 0x560d1b3ba3d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca23d4bec43c5a66 server_cookie=dac7cf1a7cbfde10 in_seq=0 out_seq=0 2024-05-22T23:05:41.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.850+0000 7f7baffff640 1 -- 172.21.15.84:0/3929807339 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d1b30e4e0 con 0x560d1b3b7f80 2024-05-22T23:05:41.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.850+0000 7f7baffff640 10 client.16086 handle_command_reply: tid=0 2024-05-22T23:05:41.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.850+0000 7f7bd9bf3740 1 client.16086 shutdown 2024-05-22T23:05:41.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.850+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/3929807339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b39cdb0 msgr2=0x560d1b3a4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:41.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.850+0000 7f7bd9bf3740 1 --2- 172.21.15.84:0/3929807339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b39cdb0 0x560d1b3a4ac0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc011580 tx=0x7f7bcc004ab0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.850+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/3929807339 shutdown_connections 2024-05-22T23:05:41.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.850+0000 7f7bd9bf3740 1 --2- 172.21.15.84:0/3929807339 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x560d1b3b7f80 0x560d1b3ba3d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.850+0000 7f7bd9bf3740 1 --2- 172.21.15.84:0/3929807339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d1b3a7b30 0x560d1b3a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.850+0000 7f7baffff640 0 client.16086 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:41.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.850+0000 7f7bd9bf3740 1 --2- 172.21.15.84:0/3929807339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d1b39e4a0 0x560d1b3a5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.850+0000 7f7bd9bf3740 1 --2- 172.21.15.84:0/3929807339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b39cdb0 0x560d1b3a4ac0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.850+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/3929807339 >> 172.21.15.84:0/3929807339 conn(0x560d1b3658e0 msgr2=0x560d1b39f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:41.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.850+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/3929807339 shutdown_connections 2024-05-22T23:05:41.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.850+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/3929807339 wait complete. 2024-05-22T23:05:41.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.851+0000 7f7bd9bf3740 20 client.16086 trim_cache size 0 max 16384 2024-05-22T23:05:41.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.868+0000 7f7bd9bf3740 1 Processor -- start 2024-05-22T23:05:41.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.868+0000 7f7bd9bf3740 1 -- start start 2024-05-22T23:05:41.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.869+0000 7f7bd9bf3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d1b39cdb0 0x560d1b3994c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.869+0000 7f7bd9bf3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d1b39e4a0 0x560d1b3b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.869+0000 7f7bd9bf3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b3a7b30 0x560d1b3bfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.869+0000 7f7bd9bf3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01ab660 con 0x560d1b3a7b30 2024-05-22T23:05:41.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.869+0000 7f7bd9bf3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01adee0 con 0x560d1b39e4a0 2024-05-22T23:05:41.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.869+0000 7f7bd9bf3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01abcd0 con 0x560d1b39cdb0 2024-05-22T23:05:41.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.869+0000 7f7bd53ad640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b3a7b30 0x560d1b3bfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.869+0000 7f7bd53ad640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b3a7b30 0x560d1b3bfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40030/0 (socket says 172.21.15.84:40030) 2024-05-22T23:05:41.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.869+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d1b39e4a0 0x560d1b3b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.869+0000 7f7bd53ad640 1 -- 172.21.15.84:0/1035596729 learned_addr learned my addr 172.21.15.84:0/1035596729 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:41.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.869+0000 7f7bd4bac640 1 --2- 172.21.15.84:0/1035596729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d1b39cdb0 0x560d1b3994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.869+0000 7f7bc7fff640 1 -- 172.21.15.84:0/1035596729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d1b39cdb0 msgr2=0x560d1b3994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:41.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.869+0000 7f7bc7fff640 1 --2- 172.21.15.84:0/1035596729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d1b39cdb0 0x560d1b3994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.869+0000 7f7bc7fff640 1 -- 172.21.15.84:0/1035596729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b3a7b30 msgr2=0x560d1b3bfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:41.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.869+0000 7f7bc7fff640 1 --2- 172.21.15.84:0/1035596729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b3a7b30 0x560d1b3bfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.869+0000 7f7bc7fff640 1 -- 172.21.15.84:0/1035596729 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d1afd40f0 con 0x560d1b39e4a0 2024-05-22T23:05:41.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.869+0000 7f7bd53ad640 1 --2- 172.21.15.84:0/1035596729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b3a7b30 0x560d1b3bfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:05:41.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.869+0000 7f7bc7fff640 1 --2- 172.21.15.84:0/1035596729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d1b39e4a0 0x560d1b3b38c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00cd5c0 tx=0x7f7bc00b4f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a045d7281166000 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:41.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.869+0000 7f7bb67fc640 1 -- 172.21.15.84:0/1035596729 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc00a9020 con 0x560d1b39e4a0 2024-05-22T23:05:41.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.869+0000 7f7bb67fc640 1 -- 172.21.15.84:0/1035596729 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bc00a4a00 con 0x560d1b39e4a0 2024-05-22T23:05:41.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.869+0000 7f7bb67fc640 1 -- 172.21.15.84:0/1035596729 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc00cd3d0 con 0x560d1b39e4a0 2024-05-22T23:05:41.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.870+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/1035596729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d1b39e4a0 msgr2=0x560d1b3b38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:41.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.870+0000 7f7bd9bf3740 1 --2- 172.21.15.84:0/1035596729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d1b39e4a0 0x560d1b3b38c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00cd5c0 tx=0x7f7bc00b4f90 comp rx=0 tx=0).stop 2024-05-22T23:05:41.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.870+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/1035596729 shutdown_connections 2024-05-22T23:05:41.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.870+0000 7f7bd9bf3740 1 --2- 172.21.15.84:0/1035596729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b3a7b30 0x560d1b3bfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.870+0000 7f7bd9bf3740 1 --2- 172.21.15.84:0/1035596729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d1b39e4a0 0x560d1b3b38c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.870+0000 7f7bd9bf3740 1 --2- 172.21.15.84:0/1035596729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d1b39cdb0 0x560d1b3994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.870+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/1035596729 >> 172.21.15.84:0/1035596729 conn(0x560d1b3658e0 msgr2=0x560d1b398db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:41.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.870+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/1035596729 shutdown_connections 2024-05-22T23:05:41.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.870+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/1035596729 wait complete. 2024-05-22T23:05:41.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.870+0000 7f7bd9bf3740 1 -- start start 2024-05-22T23:05:41.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.870+0000 7f7bd9bf3740 1 Processor -- start 2024-05-22T23:05:41.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.871+0000 7f7bd9bf3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d1b39cdb0 0x560d1b3affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.871+0000 7f7bd9bf3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b39e4a0 0x560d1b3a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.871+0000 7f7bd9bf3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d1b3a7b30 0x560d1b3a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.871+0000 7f7bd9bf3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01ac3a0 con 0x560d1b39e4a0 2024-05-22T23:05:41.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.871+0000 7f7bd9bf3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01ad810 con 0x560d1b39cdb0 2024-05-22T23:05:41.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.871+0000 7f7bd9bf3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01aca70 con 0x560d1b3a7b30 2024-05-22T23:05:41.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.871+0000 7f7bd9bf3740 10 client.0 _ll_register_callbacks cb 0x7f7bcc009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:05:41.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.871+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b39e4a0 0x560d1b3a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.871+0000 7f7bd4bac640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d1b39cdb0 0x560d1b3affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.871+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b39e4a0 0x560d1b3a13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40038/0 (socket says 172.21.15.84:40038) 2024-05-22T23:05:41.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.871+0000 7f7bc7fff640 1 -- 172.21.15.84:0/1602211093 learned_addr learned my addr 172.21.15.84:0/1602211093 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:05:41.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.871+0000 7f7bd53ad640 1 --2- 172.21.15.84:0/1602211093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d1b3a7b30 0x560d1b3a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.871+0000 7f7bc7fff640 1 -- 172.21.15.84:0/1602211093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d1b3a7b30 msgr2=0x560d1b3a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:41.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.871+0000 7f7bc7fff640 1 --2- 172.21.15.84:0/1602211093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d1b3a7b30 0x560d1b3a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.871+0000 7f7bc7fff640 1 -- 172.21.15.84:0/1602211093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d1b39cdb0 msgr2=0x560d1b3affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:05:41.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.871+0000 7f7bc7fff640 1 --2- 172.21.15.84:0/1602211093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d1b39cdb0 0x560d1b3affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.871+0000 7f7bc7fff640 1 -- 172.21.15.84:0/1602211093 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d1b2bffe0 con 0x560d1b39e4a0 2024-05-22T23:05:41.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.871+0000 7f7bd4bac640 1 --2- 172.21.15.84:0/1602211093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d1b39cdb0 0x560d1b3affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:05:41.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.871+0000 7f7bc7fff640 1 --2- 172.21.15.84:0/1602211093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b39e4a0 0x560d1b3a13b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000e9c0 tx=0x7f7bc009b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627422996347f875 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:05:41.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.871+0000 7f7baffff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:05:41.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.872+0000 7f7baffff640 1 -- 172.21.15.84:0/1602211093 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc008b060 con 0x560d1b39e4a0 2024-05-22T23:05:41.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.872+0000 7f7baffff640 1 -- 172.21.15.84:0/1602211093 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f7bc009b3e0 con 0x560d1b39e4a0 2024-05-22T23:05:41.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.872+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/1602211093 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bc00a4a00 con 0x560d1b39e4a0 2024-05-22T23:05:41.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.872+0000 7f7baffff640 1 -- 172.21.15.84:0/1602211093 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f7bc00a9050 con 0x560d1b39e4a0 2024-05-22T23:05:41.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.872+0000 7f7bd9bf3740 10 client.16092 fetch_fsmap learned FSMap version 18 2024-05-22T23:05:41.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.872+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/1602211093 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x560d1b2911c0 con 0x560d1b39e4a0 2024-05-22T23:05:41.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.872+0000 7f7baffff640 1 -- 172.21.15.84:0/1602211093 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bc00a4a00 con 0x560d1b39e4a0 2024-05-22T23:05:41.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.872+0000 7f7baffff640 1 -- 172.21.15.84:0/1602211093 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f7bc00a12d0 con 0x560d1b39e4a0 2024-05-22T23:05:41.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.872+0000 7f7bd9bf3740 10 client.16092 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:05:41.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.872+0000 7f7bd9bf3740 10 client.16092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:05:41.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.872+0000 7f7bd9bf3740 20 client.16092 populate_metadata read hostname 'smithi084' 2024-05-22T23:05:41.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.872+0000 7f7bd9bf3740 1 --2- 172.21.15.84:0/1602211093 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x560d1b41e070 0x560d1b420460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:05:41.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.872+0000 7f7bd9bf3740 4 client.16092 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:05:41.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.872+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/1602211093 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560d1b308830 con 0x560d1b41e070 2024-05-22T23:05:41.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.872+0000 7f7bd4bac640 1 --2- 172.21.15.84:0/1602211093 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x560d1b41e070 0x560d1b420460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:05:41.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.873+0000 7f7baffff640 10 client.16092 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:41.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.873+0000 7f7bd4bac640 1 --2- 172.21.15.84:0/1602211093 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x560d1b41e070 0x560d1b420460 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2256cd0fd806db80 server_cookie=4062993c10eb47fc in_seq=0 out_seq=0 2024-05-22T23:05:41.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.873+0000 7f7baffff640 1 -- 172.21.15.84:0/1602211093 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560d1b308830 con 0x560d1b41e070 2024-05-22T23:05:41.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.873+0000 7f7baffff640 10 client.16092 handle_command_reply: tid=0 2024-05-22T23:05:41.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.873+0000 7f7bd9bf3740 1 client.16092 shutdown 2024-05-22T23:05:41.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.873+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/1602211093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b39e4a0 msgr2=0x560d1b3a13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:41.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.873+0000 7f7bd9bf3740 1 --2- 172.21.15.84:0/1602211093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b39e4a0 0x560d1b3a13b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000e9c0 tx=0x7f7bc009b820 comp rx=0 tx=0).stop 2024-05-22T23:05:41.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.873+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/1602211093 shutdown_connections 2024-05-22T23:05:41.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.873+0000 7f7bd9bf3740 1 --2- 172.21.15.84:0/1602211093 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x560d1b41e070 0x560d1b420460 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.873+0000 7f7bd9bf3740 1 --2- 172.21.15.84:0/1602211093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d1b3a7b30 0x560d1b3a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.873+0000 7f7baffff640 0 client.16092 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:05:41.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.873+0000 7f7bd9bf3740 1 --2- 172.21.15.84:0/1602211093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560d1b39e4a0 0x560d1b3a13b0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.873+0000 7f7bd9bf3740 1 --2- 172.21.15.84:0/1602211093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d1b39cdb0 0x560d1b3affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.873+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/1602211093 >> 172.21.15.84:0/1602211093 conn(0x560d1b3658e0 msgr2=0x560d1b39e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:41.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.873+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/1602211093 shutdown_connections 2024-05-22T23:05:41.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.873+0000 7f7bd9bf3740 1 -- 172.21.15.84:0/1602211093 wait complete. 2024-05-22T23:05:41.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.873+0000 7f7bd9bf3740 20 client.16092 trim_cache size 0 max 16384 2024-05-22T23:05:41.885 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:05:41.886 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-22T23:05:41.886 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": { 2024-05-22T23:05:41.886 INFO:teuthology.orchestra.run.smithi084.stdout: "a1d1c1fe-612a-498a-878f-7a7b268310cd": { 2024-05-22T23:05:41.886 INFO:teuthology.orchestra.run.smithi084.stdout: "path": "/", 2024-05-22T23:05:41.886 INFO:teuthology.orchestra.run.smithi084.stdout: "tag": "a1d1c1fe-612a-498a-878f-7a7b268310cd", 2024-05-22T23:05:41.886 INFO:teuthology.orchestra.run.smithi084.stdout: "options": "recursive,force" 2024-05-22T23:05:41.886 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:05:41.886 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:05:41.886 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:05:41.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.874+0000 7f7bd6e10640 1 -- 172.21.15.84:0/804070309 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f7bb80a2d00 msgr2=0x7f7bb80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:41.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.874+0000 7f7bd6e10640 1 --2- 172.21.15.84:0/804070309 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f7bb80a2d00 0x7f7bb80a51c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc009fd0 tx=0x7f7bcc011c60 comp rx=0 tx=0).stop 2024-05-22T23:05:41.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.874+0000 7f7bd6e10640 1 -- 172.21.15.84:0/804070309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bd0101180 msgr2=0x7f7bd01987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:05:41.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.874+0000 7f7bd6e10640 1 --2- 172.21.15.84:0/804070309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bd0101180 0x7f7bd01987d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000e9f0 tx=0x7f7bc000edf0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.874+0000 7f7bd6e10640 1 -- 172.21.15.84:0/804070309 shutdown_connections 2024-05-22T23:05:41.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.874+0000 7f7bd6e10640 1 --2- 172.21.15.84:0/804070309 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f7bb80a2d00 0x7f7bb80a51c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.874+0000 7f7bd6e10640 1 --2- 172.21.15.84:0/804070309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bd01067d0 0x7f7bd019cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.874+0000 7f7bd6e10640 1 --2- 172.21.15.84:0/804070309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bd0101180 0x7f7bd01987d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.874+0000 7f7bd6e10640 1 --2- 172.21.15.84:0/804070309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bd01007c0 0x7f7bd0198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:05:41.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.874+0000 7f7bd6e10640 1 -- 172.21.15.84:0/804070309 >> 172.21.15.84:0/804070309 conn(0x7f7bd00778a0 msgr2=0x7f7bd006a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:05:41.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.874+0000 7f7bd6e10640 1 -- 172.21.15.84:0/804070309 shutdown_connections 2024-05-22T23:05:41.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:05:41.874+0000 7f7bd6e10640 1 -- 172.21.15.84:0/804070309 wait complete. 2024-05-22T23:05:41.889 INFO:tasks.cephfs.filesystem:scrub status for tag:a1d1c1fe-612a-498a-878f-7a7b268310cd - {'path': '/', 'tag': 'a1d1c1fe-612a-498a-878f-7a7b268310cd', 'options': 'recursive,force'} 2024-05-22T23:05:41.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:41 smithi204 ceph-mon[36211]: pgmap v118: 97 pgs: 97 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 246 op/s 2024-05-22T23:05:42.431 INFO:tasks.workunit.client.0.smithi084.stdout: 1 432531 15.52 MB/sec execute 28 sec latency 14.797 ms 2024-05-22T23:05:42.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:42 smithi096 ceph-mon[36392]: scrub queued for path: / 2024-05-22T23:05:42.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:42 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:05:42.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:42 smithi096 ceph-mon[36392]: scrub summary: active paths [/] 2024-05-22T23:05:42.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:42 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:05:42.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:42 smithi084 ceph-mon[32186]: scrub queued for path: / 2024-05-22T23:05:42.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:42 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:05:42.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:42 smithi084 ceph-mon[32186]: scrub summary: active paths [/] 2024-05-22T23:05:42.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:42 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:05:42.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:42 smithi204 ceph-mon[36211]: scrub queued for path: / 2024-05-22T23:05:42.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:42 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:05:42.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:42 smithi204 ceph-mon[36211]: scrub summary: active paths [/] 2024-05-22T23:05:42.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:42 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:05:43.431 INFO:tasks.workunit.client.0.smithi084.stdout: 1 434470 15.33 MB/sec execute 29 sec latency 86.468 ms 2024-05-22T23:05:43.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:43 smithi096 ceph-mon[36392]: pgmap v119: 97 pgs: 97 active+clean; 219 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.5 MiB/s wr, 370 op/s 2024-05-22T23:05:43.877 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:43 smithi084 ceph-mon[32186]: pgmap v119: 97 pgs: 97 active+clean; 219 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.5 MiB/s wr, 370 op/s 2024-05-22T23:05:43.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:43 smithi204 ceph-mon[36211]: pgmap v119: 97 pgs: 97 active+clean; 219 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.5 MiB/s wr, 370 op/s 2024-05-22T23:05:44.431 INFO:tasks.workunit.client.0.smithi084.stdout: 1 437421 15.35 MB/sec execute 30 sec latency 16.751 ms 2024-05-22T23:05:45.431 INFO:tasks.workunit.client.0.smithi084.stdout: 1 440399 15.41 MB/sec execute 31 sec latency 15.436 ms 2024-05-22T23:05:45.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:45 smithi096 ceph-mon[36392]: pgmap v120: 97 pgs: 97 active+clean; 219 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 243 op/s 2024-05-22T23:05:45.877 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:45 smithi084 ceph-mon[32186]: pgmap v120: 97 pgs: 97 active+clean; 219 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 243 op/s 2024-05-22T23:05:45.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:45 smithi204 ceph-mon[36211]: pgmap v120: 97 pgs: 97 active+clean; 219 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 243 op/s 2024-05-22T23:05:46.431 INFO:tasks.workunit.client.0.smithi084.stdout: 1 443649 15.51 MB/sec execute 32 sec latency 13.889 ms 2024-05-22T23:05:46.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:46 smithi096 ceph-mon[36392]: scrub summary: idle 2024-05-22T23:05:46.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:46 smithi084 ceph-mon[32186]: scrub summary: idle 2024-05-22T23:05:46.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:46 smithi204 ceph-mon[36211]: scrub summary: idle 2024-05-22T23:05:47.431 INFO:tasks.workunit.client.0.smithi084.stdout: 1 446780 15.49 MB/sec execute 33 sec latency 14.584 ms 2024-05-22T23:05:47.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:47 smithi096 ceph-mon[36392]: pgmap v121: 97 pgs: 97 active+clean; 227 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.5 MiB/s wr, 368 op/s 2024-05-22T23:05:47.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:47 smithi084 ceph-mon[32186]: pgmap v121: 97 pgs: 97 active+clean; 227 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.5 MiB/s wr, 368 op/s 2024-05-22T23:05:47.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:47 smithi204 ceph-mon[36211]: pgmap v121: 97 pgs: 97 active+clean; 227 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.5 MiB/s wr, 368 op/s 2024-05-22T23:05:48.431 INFO:tasks.workunit.client.0.smithi084.stdout: 1 448651 15.33 MB/sec execute 34 sec latency 102.491 ms 2024-05-22T23:05:49.431 INFO:tasks.workunit.client.0.smithi084.stdout: 1 451295 15.30 MB/sec execute 35 sec latency 30.611 ms 2024-05-22T23:05:49.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:49 smithi096 ceph-mon[36392]: pgmap v122: 97 pgs: 97 active+clean; 227 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 249 op/s 2024-05-22T23:05:49.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:49 smithi084 ceph-mon[32186]: pgmap v122: 97 pgs: 97 active+clean; 227 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 249 op/s 2024-05-22T23:05:49.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:49 smithi204 ceph-mon[36211]: pgmap v122: 97 pgs: 97 active+clean; 227 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 249 op/s 2024-05-22T23:05:50.431 INFO:tasks.workunit.client.0.smithi084.stdout: 1 454506 15.39 MB/sec execute 36 sec latency 18.054 ms 2024-05-22T23:05:51.431 INFO:tasks.workunit.client.0.smithi084.stdout: 1 457818 15.48 MB/sec execute 37 sec latency 17.099 ms 2024-05-22T23:05:51.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:51 smithi096 ceph-mon[36392]: pgmap v123: 97 pgs: 97 active+clean; 227 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 249 op/s 2024-05-22T23:05:51.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:51 smithi084 ceph-mon[32186]: pgmap v123: 97 pgs: 97 active+clean; 227 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 249 op/s 2024-05-22T23:05:51.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:51 smithi204 ceph-mon[36211]: pgmap v123: 97 pgs: 97 active+clean; 227 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 249 op/s 2024-05-22T23:05:52.431 INFO:tasks.workunit.client.0.smithi084.stdout: 1 459806 15.77 MB/sec execute 38 sec latency 50.318 ms 2024-05-22T23:05:53.431 INFO:tasks.workunit.client.0.smithi084.stdout: 1 462614 15.74 MB/sec execute 39 sec latency 110.525 ms 2024-05-22T23:05:53.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:53 smithi096 ceph-mon[36392]: pgmap v124: 97 pgs: 97 active+clean; 234 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.6 MiB/s wr, 371 op/s 2024-05-22T23:05:53.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:53 smithi084 ceph-mon[32186]: pgmap v124: 97 pgs: 97 active+clean; 234 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.6 MiB/s wr, 371 op/s 2024-05-22T23:05:53.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:53 smithi204 ceph-mon[36211]: pgmap v124: 97 pgs: 97 active+clean; 234 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.6 MiB/s wr, 371 op/s 2024-05-22T23:05:54.432 INFO:tasks.workunit.client.0.smithi084.stdout: 1 465685 15.81 MB/sec execute 40 sec latency 212.424 ms 2024-05-22T23:05:55.432 INFO:tasks.workunit.client.0.smithi084.stdout: 1 468758 15.78 MB/sec execute 41 sec latency 14.511 ms 2024-05-22T23:05:55.715 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:55 smithi204 ceph-mon[36211]: pgmap v125: 97 pgs: 97 active+clean; 234 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 247 op/s 2024-05-22T23:05:55.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:55 smithi096 ceph-mon[36392]: pgmap v125: 97 pgs: 97 active+clean; 234 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 247 op/s 2024-05-22T23:05:55.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:55 smithi084 ceph-mon[32186]: pgmap v125: 97 pgs: 97 active+clean; 234 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 247 op/s 2024-05-22T23:05:56.432 INFO:tasks.workunit.client.0.smithi084.stdout: 1 471874 15.78 MB/sec execute 42 sec latency 14.196 ms 2024-05-22T23:05:57.432 INFO:tasks.workunit.client.0.smithi084.stdout: 1 474885 15.76 MB/sec execute 43 sec latency 23.789 ms 2024-05-22T23:05:57.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:57 smithi096 ceph-mon[36392]: pgmap v126: 97 pgs: 97 active+clean; 241 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.7 MiB/s wr, 375 op/s 2024-05-22T23:05:57.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:57 smithi084 ceph-mon[32186]: pgmap v126: 97 pgs: 97 active+clean; 241 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.7 MiB/s wr, 375 op/s 2024-05-22T23:05:57.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:57 smithi204 ceph-mon[36211]: pgmap v126: 97 pgs: 97 active+clean; 241 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.7 MiB/s wr, 375 op/s 2024-05-22T23:05:58.432 INFO:tasks.workunit.client.0.smithi084.stdout: 1 477090 15.70 MB/sec execute 44 sec latency 29.834 ms 2024-05-22T23:05:59.432 INFO:tasks.workunit.client.0.smithi084.stdout: 1 479991 15.75 MB/sec execute 45 sec latency 75.027 ms 2024-05-22T23:05:59.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:59 smithi096 ceph-mon[36392]: pgmap v127: 97 pgs: 97 active+clean; 241 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 250 op/s 2024-05-22T23:05:59.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:05:59 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:05:59.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:59 smithi084 ceph-mon[32186]: pgmap v127: 97 pgs: 97 active+clean; 241 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 250 op/s 2024-05-22T23:05:59.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:05:59 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:05:59.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:59 smithi204 ceph-mon[36211]: pgmap v127: 97 pgs: 97 active+clean; 241 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 250 op/s 2024-05-22T23:05:59.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:05:59 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:06:00.432 INFO:tasks.workunit.client.0.smithi084.stdout: 1 483216 15.80 MB/sec execute 46 sec latency 15.315 ms 2024-05-22T23:06:00.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:00 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:06:00.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:00 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:06:00.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:00 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:06:00.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:00 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:06:00.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:00 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:06:00.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:00 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:06:00.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:00 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:06:00.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:00 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:06:00.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:00 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:06:01.432 INFO:tasks.workunit.client.0.smithi084.stdout: 1 484749 15.60 MB/sec execute 47 sec latency 21.782 ms 2024-05-22T23:06:01.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:01 smithi096 ceph-mon[36392]: pgmap v128: 97 pgs: 97 active+clean; 241 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 250 op/s 2024-05-22T23:06:01.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:01 smithi084 ceph-mon[32186]: pgmap v128: 97 pgs: 97 active+clean; 241 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 250 op/s 2024-05-22T23:06:01.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:01 smithi204 ceph-mon[36211]: pgmap v128: 97 pgs: 97 active+clean; 241 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 250 op/s 2024-05-22T23:06:02.432 INFO:tasks.workunit.client.0.smithi084.stdout: 1 487561 15.58 MB/sec execute 48 sec latency 54.064 ms 2024-05-22T23:06:03.432 INFO:tasks.workunit.client.0.smithi084.stdout: 1 490901 15.64 MB/sec execute 49 sec latency 6.891 ms 2024-05-22T23:06:03.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:03 smithi096 ceph-mon[36392]: pgmap v129: 97 pgs: 97 active+clean; 248 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.7 MiB/s wr, 385 op/s 2024-05-22T23:06:03.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:03 smithi084 ceph-mon[32186]: pgmap v129: 97 pgs: 97 active+clean; 248 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.7 MiB/s wr, 385 op/s 2024-05-22T23:06:03.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:03 smithi204 ceph-mon[36211]: pgmap v129: 97 pgs: 97 active+clean; 248 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.7 MiB/s wr, 385 op/s 2024-05-22T23:06:04.432 INFO:tasks.workunit.client.0.smithi084.stdout: 1 493656 15.62 MB/sec execute 50 sec latency 124.977 ms 2024-05-22T23:06:05.432 INFO:tasks.workunit.client.0.smithi084.stdout: 1 496459 15.56 MB/sec execute 51 sec latency 13.969 ms 2024-05-22T23:06:05.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:05 smithi096 ceph-mon[36392]: pgmap v130: 97 pgs: 97 active+clean; 248 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 262 op/s 2024-05-22T23:06:05.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:05 smithi084 ceph-mon[32186]: pgmap v130: 97 pgs: 97 active+clean; 248 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 262 op/s 2024-05-22T23:06:05.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:05 smithi204 ceph-mon[36211]: pgmap v130: 97 pgs: 97 active+clean; 248 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 262 op/s 2024-05-22T23:06:06.432 INFO:tasks.workunit.client.0.smithi084.stdout: 1 499317 15.58 MB/sec execute 52 sec latency 14.283 ms 2024-05-22T23:06:07.432 INFO:tasks.workunit.client.0.smithi084.stdout: 1 501920 15.56 MB/sec execute 53 sec latency 21.898 ms 2024-05-22T23:06:07.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:07 smithi096 ceph-mon[36392]: pgmap v131: 97 pgs: 97 active+clean; 257 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 398 op/s 2024-05-22T23:06:07.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:07 smithi084 ceph-mon[32186]: pgmap v131: 97 pgs: 97 active+clean; 257 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 398 op/s 2024-05-22T23:06:07.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:07 smithi204 ceph-mon[36211]: pgmap v131: 97 pgs: 97 active+clean; 257 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 398 op/s 2024-05-22T23:06:08.432 INFO:tasks.workunit.client.0.smithi084.stdout: 1 505105 15.61 MB/sec execute 54 sec latency 9.712 ms 2024-05-22T23:06:09.432 INFO:tasks.workunit.client.0.smithi084.stdout: 1 508007 15.64 MB/sec execute 55 sec latency 73.901 ms 2024-05-22T23:06:09.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:09 smithi096 ceph-mon[36392]: pgmap v132: 97 pgs: 97 active+clean; 257 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 270 op/s 2024-05-22T23:06:09.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:09 smithi084 ceph-mon[32186]: pgmap v132: 97 pgs: 97 active+clean; 257 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 270 op/s 2024-05-22T23:06:09.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:09 smithi204 ceph-mon[36211]: pgmap v132: 97 pgs: 97 active+clean; 257 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 270 op/s 2024-05-22T23:06:10.432 INFO:tasks.workunit.client.0.smithi084.stdout: 1 510911 15.60 MB/sec execute 56 sec latency 14.433 ms 2024-05-22T23:06:11.432 INFO:tasks.workunit.client.0.smithi084.stdout: 1 513073 15.55 MB/sec execute 57 sec latency 70.795 ms 2024-05-22T23:06:11.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:11 smithi096 ceph-mon[36392]: pgmap v133: 97 pgs: 97 active+clean; 257 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 270 op/s 2024-05-22T23:06:11.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:11 smithi084 ceph-mon[32186]: pgmap v133: 97 pgs: 97 active+clean; 257 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 270 op/s 2024-05-22T23:06:11.890 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:06:11.891 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:06:11.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:11 smithi204 ceph-mon[36211]: pgmap v133: 97 pgs: 97 active+clean; 257 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 270 op/s 2024-05-22T23:06:12.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.996+0000 7fcdf8dd5640 1 -- 172.21.15.84:0/3056038158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdf4106940 msgr2=0x7fcdf4106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:12.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.996+0000 7fcdf8dd5640 1 --2- 172.21.15.84:0/3056038158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdf4106940 0x7fcdf4106d20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009930 tx=0x7fcde402a420 comp rx=0 tx=0).stop 2024-05-22T23:06:12.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.996+0000 7fcdf8dd5640 1 -- 172.21.15.84:0/3056038158 shutdown_connections 2024-05-22T23:06:12.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.996+0000 7fcdf8dd5640 1 --2- 172.21.15.84:0/3056038158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdf4101410 0x7fcdf410d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.996+0000 7fcdf8dd5640 1 --2- 172.21.15.84:0/3056038158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdf4100a50 0x7fcdf4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.996+0000 7fcdf8dd5640 1 --2- 172.21.15.84:0/3056038158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdf4106940 0x7fcdf4106d20 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.996+0000 7fcdf8dd5640 1 -- 172.21.15.84:0/3056038158 >> 172.21.15.84:0/3056038158 conn(0x7fcdf4077750 msgr2=0x7fcdf4077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:12.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.996+0000 7fcdf8dd5640 1 -- 172.21.15.84:0/3056038158 shutdown_connections 2024-05-22T23:06:12.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.996+0000 7fcdf8dd5640 1 -- 172.21.15.84:0/3056038158 wait complete. 2024-05-22T23:06:12.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.997+0000 7fcdf8dd5640 1 Processor -- start 2024-05-22T23:06:12.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.997+0000 7fcdf8dd5640 1 -- start start 2024-05-22T23:06:12.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.997+0000 7fcdf8dd5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdf4100a50 0x7fcdf4198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.997+0000 7fcdf8dd5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdf4101410 0x7fcdf41994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.997+0000 7fcdf8dd5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdf4106940 0x7fcdf41a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.997+0000 7fcdf8dd5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf4110c10 con 0x7fcdf4106940 2024-05-22T23:06:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.997+0000 7fcdf8dd5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf4110a90 con 0x7fcdf4101410 2024-05-22T23:06:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.997+0000 7fcdf8dd5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf4110d90 con 0x7fcdf4100a50 2024-05-22T23:06:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.997+0000 7fcdf2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdf4106940 0x7fcdf41a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.997+0000 7fcdf2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdf4106940 0x7fcdf41a4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56022/0 (socket says 172.21.15.84:56022) 2024-05-22T23:06:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.997+0000 7fcdf2d9d640 1 -- 172.21.15.84:0/3183159029 learned_addr learned my addr 172.21.15.84:0/3183159029 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.997+0000 7fcdf1d9b640 1 --2- 172.21.15.84:0/3183159029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdf4101410 0x7fcdf41994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.998+0000 7fcdf259c640 1 --2- 172.21.15.84:0/3183159029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdf4100a50 0x7fcdf4198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.998+0000 7fcdf2d9d640 1 -- 172.21.15.84:0/3183159029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdf4100a50 msgr2=0x7fcdf4198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.998+0000 7fcdf2d9d640 1 --2- 172.21.15.84:0/3183159029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdf4100a50 0x7fcdf4198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.998+0000 7fcdf2d9d640 1 -- 172.21.15.84:0/3183159029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdf4101410 msgr2=0x7fcdf41994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.998+0000 7fcdf2d9d640 1 --2- 172.21.15.84:0/3183159029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdf4101410 0x7fcdf41994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.998+0000 7fcdf2d9d640 1 -- 172.21.15.84:0/3183159029 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf41a4550 con 0x7fcdf4106940 2024-05-22T23:06:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.998+0000 7fcdf2d9d640 1 --2- 172.21.15.84:0/3183159029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdf4106940 0x7fcdf41a4010 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00eaa0 tx=0x7fcdec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:12.006 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.998+0000 7fcde37fe640 1 -- 172.21.15.84:0/3183159029 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec00cdb0 con 0x7fcdf4106940 2024-05-22T23:06:12.006 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.998+0000 7fcdf8dd5640 1 -- 172.21.15.84:0/3183159029 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf41a4780 con 0x7fcdf4106940 2024-05-22T23:06:12.006 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.998+0000 7fcde37fe640 1 -- 172.21.15.84:0/3183159029 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdec005020 con 0x7fcdf4106940 2024-05-22T23:06:12.006 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.998+0000 7fcde37fe640 1 -- 172.21.15.84:0/3183159029 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec0116d0 con 0x7fcdf4106940 2024-05-22T23:06:12.006 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:11.998+0000 7fcdf8dd5640 1 -- 172.21.15.84:0/3183159029 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf41a4ca0 con 0x7fcdf4106940 2024-05-22T23:06:12.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.001+0000 7fcde37fe640 1 -- 172.21.15.84:0/3183159029 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcdec0054d0 con 0x7fcdf4106940 2024-05-22T23:06:12.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.001+0000 7fcdfbbc2740 1 Processor -- start 2024-05-22T23:06:12.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.001+0000 7fcdfbbc2740 1 -- start start 2024-05-22T23:06:12.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.001+0000 7fcdfbbc2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fce4a0 0x55c779fcc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.001+0000 7fcdfbbc2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c779fccdb0 0x55c779fcd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.002+0000 7fcdfbbc2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c779fd7b30 0x55c779fd9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.002+0000 7fcdfbbc2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c779f30db0 con 0x55c779fce4a0 2024-05-22T23:06:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.002+0000 7fcdfbbc2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c779ecf360 con 0x55c779fd7b30 2024-05-22T23:06:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.002+0000 7fcdfbbc2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c779f2cd70 con 0x55c779fccdb0 2024-05-22T23:06:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.002+0000 7fcde37fe640 1 --2- 172.21.15.84:0/3183159029 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fcddc0a2d00 0x7fcddc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.002+0000 7fcdf1d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fce4a0 0x55c779fcc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.002+0000 7fcdf259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c779fccdb0 0x55c779fcd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.002+0000 7fcdf259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c779fccdb0 0x55c779fcd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:34158/0 (socket says 172.21.15.84:34158) 2024-05-22T23:06:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.002+0000 7fcdf1d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fce4a0 0x55c779fcc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56030/0 (socket says 172.21.15.84:56030) 2024-05-22T23:06:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.002+0000 7fcdf259c640 1 -- 172.21.15.84:0/3425319079 learned_addr learned my addr 172.21.15.84:0/3425319079 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.002+0000 7fcdf1d9b640 1 --2- 172.21.15.84:0/3425319079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c779fd7b30 0x55c779fd9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.002+0000 7fcdf2d9d640 1 --2- 172.21.15.84:0/3183159029 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fcddc0a2d00 0x7fcddc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.002+0000 7fcde37fe640 1 -- 172.21.15.84:0/3183159029 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fcdec0cf630 con 0x7fcdf4106940 2024-05-22T23:06:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.002+0000 7fcdf1d9b640 1 -- 172.21.15.84:0/3425319079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c779fccdb0 msgr2=0x55c779fcd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.002+0000 7fcdf1d9b640 1 --2- 172.21.15.84:0/3425319079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c779fccdb0 0x55c779fcd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.002+0000 7fcdf1d9b640 1 -- 172.21.15.84:0/3425319079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c779fd7b30 msgr2=0x55c779fd9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:12.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.002+0000 7fcdf1d9b640 1 --2- 172.21.15.84:0/3425319079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c779fd7b30 0x55c779fd9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.002+0000 7fcdf1d9b640 1 -- 172.21.15.84:0/3425319079 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c779ef2a60 con 0x55c779fce4a0 2024-05-22T23:06:12.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.002+0000 7fcdf2d9d640 1 --2- 172.21.15.84:0/3183159029 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fcddc0a2d00 0x7fcddc0a51c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcdec0cce30 tx=0x7fcdec01eb40 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:12.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.002+0000 7fcdf1d9b640 1 --2- 172.21.15.84:0/3425319079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fce4a0 0x55c779fcc7e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fcde80058c0 tx=0x7fcde8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca296cf3a2c87fa server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:12.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.002+0000 7fcde0ff9640 1 -- 172.21.15.84:0/3425319079 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde8015070 con 0x55c779fce4a0 2024-05-22T23:06:12.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.002+0000 7fcde0ff9640 1 -- 172.21.15.84:0/3425319079 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcde800e840 con 0x55c779fce4a0 2024-05-22T23:06:12.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.003+0000 7fcde0ff9640 1 -- 172.21.15.84:0/3425319079 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde8005440 con 0x55c779fce4a0 2024-05-22T23:06:12.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.003+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/3425319079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fce4a0 msgr2=0x55c779fcc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:12.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.003+0000 7fcdfbbc2740 1 --2- 172.21.15.84:0/3425319079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fce4a0 0x55c779fcc7e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fcde80058c0 tx=0x7fcde8005850 comp rx=0 tx=0).stop 2024-05-22T23:06:12.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.003+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/3425319079 shutdown_connections 2024-05-22T23:06:12.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.003+0000 7fcdfbbc2740 1 --2- 172.21.15.84:0/3425319079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c779fd7b30 0x55c779fd9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.003+0000 7fcdfbbc2740 1 --2- 172.21.15.84:0/3425319079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c779fccdb0 0x55c779fcd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.003+0000 7fcdfbbc2740 1 --2- 172.21.15.84:0/3425319079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fce4a0 0x55c779fcc7e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.003+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/3425319079 >> 172.21.15.84:0/3425319079 conn(0x55c779f958e0 msgr2=0x55c779fc9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:12.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.003+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/3425319079 shutdown_connections 2024-05-22T23:06:12.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.003+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/3425319079 wait complete. 2024-05-22T23:06:12.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.004+0000 7fcdfbbc2740 1 -- start start 2024-05-22T23:06:12.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.004+0000 7fcdfbbc2740 1 Processor -- start 2024-05-22T23:06:12.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.004+0000 7fcdfbbc2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c779fccdb0 0x55c779fd4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.004+0000 7fcdfbbc2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fce4a0 0x55c779fd4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.004+0000 7fcdfbbc2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c779fd7b30 0x55c779fd54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.004+0000 7fcdfbbc2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c779fda960 con 0x55c779fce4a0 2024-05-22T23:06:12.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.004+0000 7fcdfbbc2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c779eeb0f0 con 0x55c779fd7b30 2024-05-22T23:06:12.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.004+0000 7fcdfbbc2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c779fdaae0 con 0x55c779fccdb0 2024-05-22T23:06:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.004+0000 7fcdf259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fce4a0 0x55c779fd4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.004+0000 7fcdfbbc2740 10 client.0 _ll_register_callbacks cb 0x55c779eeff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:06:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.004+0000 7fcdf259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fce4a0 0x55c779fd4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56046/0 (socket says 172.21.15.84:56046) 2024-05-22T23:06:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.004+0000 7fcdf259c640 1 -- 172.21.15.84:0/470093443 learned_addr learned my addr 172.21.15.84:0/470093443 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.004+0000 7fcdf1d9b640 1 --2- 172.21.15.84:0/470093443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c779fccdb0 0x55c779fd4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.004+0000 7fcdf1d9b640 1 --2- 172.21.15.84:0/470093443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c779fd7b30 0x55c779fd54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.004+0000 7fcdf259c640 1 -- 172.21.15.84:0/470093443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c779fccdb0 msgr2=0x55c779fd4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.004+0000 7fcdf259c640 1 --2- 172.21.15.84:0/470093443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c779fccdb0 0x55c779fd4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.004+0000 7fcdf259c640 1 -- 172.21.15.84:0/470093443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c779fd7b30 msgr2=0x55c779fd54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.004+0000 7fcdf259c640 1 --2- 172.21.15.84:0/470093443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c779fd7b30 0x55c779fd54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.004+0000 7fcdf259c640 1 -- 172.21.15.84:0/470093443 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c779d0e7d0 con 0x55c779fce4a0 2024-05-22T23:06:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.005+0000 7fcdf259c640 1 --2- 172.21.15.84:0/470093443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fce4a0 0x55c779fd4f80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fcde4000c00 tx=0x7fcde40317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466400b785476a58 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.005+0000 7fcdd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:06:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.005+0000 7fcdd9ffb640 1 -- 172.21.15.84:0/470093443 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4035040 con 0x55c779fce4a0 2024-05-22T23:06:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.005+0000 7fcdd9ffb640 1 -- 172.21.15.84:0/470093443 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fcde4004250 con 0x55c779fce4a0 2024-05-22T23:06:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.005+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/470093443 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcde800e840 con 0x55c779fce4a0 2024-05-22T23:06:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.005+0000 7fcdd9ffb640 1 -- 172.21.15.84:0/470093443 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fcde4004490 con 0x55c779fce4a0 2024-05-22T23:06:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.005+0000 7fcdfbbc2740 10 client.16104 fetch_fsmap learned FSMap version 18 2024-05-22T23:06:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.005+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/470093443 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55c779c443f0 con 0x55c779fce4a0 2024-05-22T23:06:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.005+0000 7fcdd9ffb640 1 -- 172.21.15.84:0/470093443 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcde800e840 con 0x55c779fce4a0 2024-05-22T23:06:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.005+0000 7fcdd9ffb640 1 -- 172.21.15.84:0/470093443 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fcde4004c20 con 0x55c779fce4a0 2024-05-22T23:06:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.005+0000 7fcdfbbc2740 10 client.16104 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:06:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.005+0000 7fcdfbbc2740 10 client.16104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:06:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.005+0000 7fcdfbbc2740 20 client.16104 populate_metadata read hostname 'smithi084' 2024-05-22T23:06:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.005+0000 7fcdfbbc2740 1 --2- 172.21.15.84:0/470093443 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c779fe7ef0 0x55c779fea310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.005+0000 7fcdfbbc2740 4 client.16104 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:06:12.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.005+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/470093443 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c779f3e4e0 con 0x55c779fe7ef0 2024-05-22T23:06:12.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.005+0000 7fcdf1d9b640 1 --2- 172.21.15.84:0/470093443 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c779fe7ef0 0x55c779fea310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.006+0000 7fcdd9ffb640 10 client.16104 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:12.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.006+0000 7fcdf1d9b640 1 --2- 172.21.15.84:0/470093443 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c779fe7ef0 0x55c779fea310 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fc149c1569e1b21 server_cookie=dcc886eb6f04f588 in_seq=0 out_seq=0 2024-05-22T23:06:12.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.006+0000 7fcdd9ffb640 1 -- 172.21.15.84:0/470093443 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c779f3e4e0 con 0x55c779fe7ef0 2024-05-22T23:06:12.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.007+0000 7fcdd9ffb640 10 client.16104 handle_command_reply: tid=0 2024-05-22T23:06:12.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.007+0000 7fcdfbbc2740 1 client.16104 shutdown 2024-05-22T23:06:12.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.007+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/470093443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fce4a0 msgr2=0x55c779fd4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:12.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.007+0000 7fcdfbbc2740 1 --2- 172.21.15.84:0/470093443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fce4a0 0x55c779fd4f80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fcde4000c00 tx=0x7fcde40317e0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.007+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/470093443 shutdown_connections 2024-05-22T23:06:12.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.007+0000 7fcdfbbc2740 1 --2- 172.21.15.84:0/470093443 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c779fe7ef0 0x55c779fea310 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.007+0000 7fcdfbbc2740 1 --2- 172.21.15.84:0/470093443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c779fd7b30 0x55c779fd54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.007+0000 7fcdfbbc2740 1 --2- 172.21.15.84:0/470093443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fce4a0 0x55c779fd4f80 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.007+0000 7fcdd9ffb640 0 client.16104 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:12.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.007+0000 7fcdfbbc2740 1 --2- 172.21.15.84:0/470093443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c779fccdb0 0x55c779fd4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.007+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/470093443 >> 172.21.15.84:0/470093443 conn(0x55c779f958e0 msgr2=0x55c779fcf110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:12.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.007+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/470093443 shutdown_connections 2024-05-22T23:06:12.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.007+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/470093443 wait complete. 2024-05-22T23:06:12.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.007+0000 7fcdfbbc2740 20 client.16104 trim_cache size 0 max 16384 2024-05-22T23:06:12.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.024+0000 7fcdfbbc2740 1 Processor -- start 2024-05-22T23:06:12.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.024+0000 7fcdfbbc2740 1 -- start start 2024-05-22T23:06:12.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.024+0000 7fcdfbbc2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c779fccdb0 0x55c779fe3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.024+0000 7fcdfbbc2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fce4a0 0x55c779fc4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.024+0000 7fcdfbbc2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c779fd7b30 0x55c779fd52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.024+0000 7fcdfbbc2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf41a93d0 con 0x55c779fce4a0 2024-05-22T23:06:12.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.024+0000 7fcdfbbc2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf41abc50 con 0x55c779fccdb0 2024-05-22T23:06:12.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.024+0000 7fcdfbbc2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf41a9a40 con 0x55c779fd7b30 2024-05-22T23:06:12.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.025+0000 7fcdf259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fce4a0 0x55c779fc4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.025+0000 7fcdf259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fce4a0 0x55c779fc4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56056/0 (socket says 172.21.15.84:56056) 2024-05-22T23:06:12.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.025+0000 7fcdf1d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c779fccdb0 0x55c779fe3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.025+0000 7fcdf1d9b640 1 --2- 172.21.15.84:0/4184536916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c779fd7b30 0x55c779fd52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.025+0000 7fcdf259c640 1 -- 172.21.15.84:0/4184536916 learned_addr learned my addr 172.21.15.84:0/4184536916 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:12.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.025+0000 7fcdf259c640 1 -- 172.21.15.84:0/4184536916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c779fd7b30 msgr2=0x55c779fd52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:12.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.025+0000 7fcdf259c640 1 --2- 172.21.15.84:0/4184536916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c779fd7b30 0x55c779fd52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.025+0000 7fcdf259c640 1 -- 172.21.15.84:0/4184536916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c779fccdb0 msgr2=0x55c779fe3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:12.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.025+0000 7fcdf259c640 1 --2- 172.21.15.84:0/4184536916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c779fccdb0 0x55c779fe3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.032 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.025+0000 7fcdf259c640 1 -- 172.21.15.84:0/4184536916 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c779fd3a30 con 0x55c779fce4a0 2024-05-22T23:06:12.032 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.025+0000 7fcdf259c640 1 --2- 172.21.15.84:0/4184536916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fce4a0 0x55c779fc4160 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fcde4006010 tx=0x7fcde4031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4812d14e768cbb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:12.032 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.025+0000 7fcddbfff640 1 -- 172.21.15.84:0/4184536916 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4052070 con 0x55c779fce4a0 2024-05-22T23:06:12.032 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.025+0000 7fcddbfff640 1 -- 172.21.15.84:0/4184536916 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcde4040070 con 0x55c779fce4a0 2024-05-22T23:06:12.032 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.025+0000 7fcddbfff640 1 -- 172.21.15.84:0/4184536916 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4035070 con 0x55c779fce4a0 2024-05-22T23:06:12.036 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.026+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/4184536916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fce4a0 msgr2=0x55c779fc4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:12.036 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.026+0000 7fcdfbbc2740 1 --2- 172.21.15.84:0/4184536916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fce4a0 0x55c779fc4160 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fcde4006010 tx=0x7fcde4031b60 comp rx=0 tx=0).stop 2024-05-22T23:06:12.036 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.026+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/4184536916 shutdown_connections 2024-05-22T23:06:12.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.026+0000 7fcdfbbc2740 1 --2- 172.21.15.84:0/4184536916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c779fd7b30 0x55c779fd52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.026+0000 7fcdfbbc2740 1 --2- 172.21.15.84:0/4184536916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fce4a0 0x55c779fc4160 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.026+0000 7fcdfbbc2740 1 --2- 172.21.15.84:0/4184536916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c779fccdb0 0x55c779fe3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.026+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/4184536916 >> 172.21.15.84:0/4184536916 conn(0x55c779f958e0 msgr2=0x55c779fd6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:12.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.026+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/4184536916 shutdown_connections 2024-05-22T23:06:12.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.026+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/4184536916 wait complete. 2024-05-22T23:06:12.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.026+0000 7fcdfbbc2740 1 -- start start 2024-05-22T23:06:12.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.026+0000 7fcdfbbc2740 1 Processor -- start 2024-05-22T23:06:12.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.027+0000 7fcdfbbc2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fccdb0 0x55c779fdff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.027+0000 7fcdfbbc2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c779fce4a0 0x55c779fd12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.027+0000 7fcdfbbc2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c779fd7b30 0x55c779fd1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.027+0000 7fcdfbbc2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf41aa110 con 0x55c779fccdb0 2024-05-22T23:06:12.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.027+0000 7fcdfbbc2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf41ab580 con 0x55c779fce4a0 2024-05-22T23:06:12.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.027+0000 7fcdfbbc2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf41aa7e0 con 0x55c779fd7b30 2024-05-22T23:06:12.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.027+0000 7fcdfbbc2740 10 client.0 _ll_register_callbacks cb 0x7fcde4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:06:12.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.027+0000 7fcdf1d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fccdb0 0x55c779fdff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.027+0000 7fcdf1d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fccdb0 0x55c779fdff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56062/0 (socket says 172.21.15.84:56062) 2024-05-22T23:06:12.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.027+0000 7fcdf1d9b640 1 -- 172.21.15.84:0/2876623386 learned_addr learned my addr 172.21.15.84:0/2876623386 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:12.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.027+0000 7fcdf259c640 1 --2- 172.21.15.84:0/2876623386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c779fce4a0 0x55c779fd12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.027+0000 7fcdf1d9b640 1 --2- 172.21.15.84:0/2876623386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c779fd7b30 0x55c779fd1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.027+0000 7fcdf1d9b640 1 -- 172.21.15.84:0/2876623386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c779fd7b30 msgr2=0x55c779fd1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:12.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.027+0000 7fcdf1d9b640 1 --2- 172.21.15.84:0/2876623386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c779fd7b30 0x55c779fd1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.027+0000 7fcdf1d9b640 1 -- 172.21.15.84:0/2876623386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c779fce4a0 msgr2=0x55c779fd12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:12.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.027+0000 7fcdf1d9b640 1 --2- 172.21.15.84:0/2876623386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c779fce4a0 0x55c779fd12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.027+0000 7fcdf1d9b640 1 -- 172.21.15.84:0/2876623386 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c779c040f0 con 0x55c779fccdb0 2024-05-22T23:06:12.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.027+0000 7fcdf259c640 1 --2- 172.21.15.84:0/2876623386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c779fce4a0 0x55c779fd12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:06:12.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.027+0000 7fcdf1d9b640 1 --2- 172.21.15.84:0/2876623386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fccdb0 0x55c779fdff60 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fcde8017fd0 tx=0x7fcde8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7a221f78c760ea server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:12.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.027+0000 7fcdd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:06:12.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.027+0000 7fcdd9ffb640 1 -- 172.21.15.84:0/2876623386 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde8005310 con 0x55c779fccdb0 2024-05-22T23:06:12.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.027+0000 7fcdd9ffb640 1 -- 172.21.15.84:0/2876623386 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fcde801d040 con 0x55c779fccdb0 2024-05-22T23:06:12.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.027+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/2876623386 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcde4040070 con 0x55c779fccdb0 2024-05-22T23:06:12.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.028+0000 7fcdd9ffb640 1 -- 172.21.15.84:0/2876623386 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fcde80054b0 con 0x55c779fccdb0 2024-05-22T23:06:12.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.028+0000 7fcdfbbc2740 10 client.16110 fetch_fsmap learned FSMap version 18 2024-05-22T23:06:12.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.028+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/2876623386 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55c779eeffe0 con 0x55c779fccdb0 2024-05-22T23:06:12.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.028+0000 7fcdd9ffb640 1 -- 172.21.15.84:0/2876623386 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcde4040070 con 0x55c779fccdb0 2024-05-22T23:06:12.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.028+0000 7fcdd9ffb640 1 -- 172.21.15.84:0/2876623386 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fcde8021030 con 0x55c779fccdb0 2024-05-22T23:06:12.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.028+0000 7fcdfbbc2740 10 client.16110 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:06:12.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.028+0000 7fcdfbbc2740 10 client.16110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:06:12.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.028+0000 7fcdfbbc2740 20 client.16110 populate_metadata read hostname 'smithi084' 2024-05-22T23:06:12.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.028+0000 7fcdfbbc2740 1 --2- 172.21.15.84:0/2876623386 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c77a04e4f0 0x55c77a050910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.028+0000 7fcdfbbc2740 4 client.16110 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:06:12.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.028+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/2876623386 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c779f38830 con 0x55c77a04e4f0 2024-05-22T23:06:12.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.028+0000 7fcdf259c640 1 --2- 172.21.15.84:0/2876623386 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c77a04e4f0 0x55c77a050910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.028+0000 7fcdd9ffb640 10 client.16110 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:12.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.028+0000 7fcdf259c640 1 --2- 172.21.15.84:0/2876623386 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c77a04e4f0 0x55c77a050910 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60c99a02a41777bf server_cookie=e1f33a95474ce7f9 in_seq=0 out_seq=0 2024-05-22T23:06:12.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.029+0000 7fcdd9ffb640 1 -- 172.21.15.84:0/2876623386 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c779f38830 con 0x55c77a04e4f0 2024-05-22T23:06:12.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.029+0000 7fcdd9ffb640 10 client.16110 handle_command_reply: tid=0 2024-05-22T23:06:12.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.029+0000 7fcdfbbc2740 1 client.16110 shutdown 2024-05-22T23:06:12.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.029+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/2876623386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fccdb0 msgr2=0x55c779fdff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:12.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.029+0000 7fcdfbbc2740 1 --2- 172.21.15.84:0/2876623386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fccdb0 0x55c779fdff60 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fcde8017fd0 tx=0x7fcde8005850 comp rx=0 tx=0).stop 2024-05-22T23:06:12.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.029+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/2876623386 shutdown_connections 2024-05-22T23:06:12.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.029+0000 7fcdfbbc2740 1 --2- 172.21.15.84:0/2876623386 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c77a04e4f0 0x55c77a050910 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.029+0000 7fcdfbbc2740 1 --2- 172.21.15.84:0/2876623386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c779fd7b30 0x55c779fd1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.029+0000 7fcdfbbc2740 1 --2- 172.21.15.84:0/2876623386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c779fce4a0 0x55c779fd12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.029+0000 7fcdfbbc2740 1 --2- 172.21.15.84:0/2876623386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c779fccdb0 0x55c779fdff60 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.029+0000 7fcdd9ffb640 0 client.16110 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:12.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.029+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/2876623386 >> 172.21.15.84:0/2876623386 conn(0x55c779f958e0 msgr2=0x55c779fd6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:12.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.029+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/2876623386 shutdown_connections 2024-05-22T23:06:12.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.029+0000 7fcdfbbc2740 1 -- 172.21.15.84:0/2876623386 wait complete. 2024-05-22T23:06:12.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.029+0000 7fcdfbbc2740 20 client.16110 trim_cache size 0 max 16384 2024-05-22T23:06:12.042 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:06:12.042 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "no active scrubs running", 2024-05-22T23:06:12.042 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": {} 2024-05-22T23:06:12.042 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:06:12.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.029+0000 7fcdf8dd5640 1 -- 172.21.15.84:0/3183159029 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fcddc0a2d00 msgr2=0x7fcddc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:12.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.029+0000 7fcdf8dd5640 1 --2- 172.21.15.84:0/3183159029 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fcddc0a2d00 0x7fcddc0a51c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcdec0cce30 tx=0x7fcdec01eb40 comp rx=0 tx=0).stop 2024-05-22T23:06:12.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.029+0000 7fcdf8dd5640 1 -- 172.21.15.84:0/3183159029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdf4106940 msgr2=0x7fcdf41a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:12.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.029+0000 7fcdf8dd5640 1 --2- 172.21.15.84:0/3183159029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdf4106940 0x7fcdf41a4010 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00eaa0 tx=0x7fcdec00ef60 comp rx=0 tx=0).stop 2024-05-22T23:06:12.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.030+0000 7fcdf8dd5640 1 -- 172.21.15.84:0/3183159029 shutdown_connections 2024-05-22T23:06:12.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.030+0000 7fcdf8dd5640 1 --2- 172.21.15.84:0/3183159029 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fcddc0a2d00 0x7fcddc0a51c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.030+0000 7fcdf8dd5640 1 --2- 172.21.15.84:0/3183159029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdf4106940 0x7fcdf41a4010 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.030+0000 7fcdf8dd5640 1 --2- 172.21.15.84:0/3183159029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdf4101410 0x7fcdf41994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.030+0000 7fcdf8dd5640 1 --2- 172.21.15.84:0/3183159029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdf4100a50 0x7fcdf4198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.030+0000 7fcdf8dd5640 1 -- 172.21.15.84:0/3183159029 >> 172.21.15.84:0/3183159029 conn(0x7fcdf4077750 msgr2=0x7fcdf40ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:12.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.030+0000 7fcdf8dd5640 1 -- 172.21.15.84:0/3183159029 shutdown_connections 2024-05-22T23:06:12.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.030+0000 7fcdf8dd5640 1 -- 172.21.15.84:0/3183159029 wait complete. 2024-05-22T23:06:12.045 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-22T23:06:12.046 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:06:12.155 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.147+0000 7f0bb637e640 1 -- 172.21.15.84:0/1149718455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bb0107880 msgr2=0x7f0bb0107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:12.155 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.147+0000 7f0bb637e640 1 --2- 172.21.15.84:0/1149718455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bb0107880 0x7f0bb0107c60 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009930 tx=0x7f0ba002a420 comp rx=0 tx=0).stop 2024-05-22T23:06:12.155 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.147+0000 7f0bb637e640 1 -- 172.21.15.84:0/1149718455 shutdown_connections 2024-05-22T23:06:12.155 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.147+0000 7f0bb637e640 1 --2- 172.21.15.84:0/1149718455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb0102230 0x7f0bb010e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.155 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.147+0000 7f0bb637e640 1 --2- 172.21.15.84:0/1149718455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb0101870 0x7f0bb0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.155 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.147+0000 7f0bb637e640 1 --2- 172.21.15.84:0/1149718455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bb0107880 0x7f0bb0107c60 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.155 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.147+0000 7f0bb637e640 1 -- 172.21.15.84:0/1149718455 >> 172.21.15.84:0/1149718455 conn(0x7f0bb0078d00 msgr2=0x7f0bb0079110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:12.155 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.147+0000 7f0bb637e640 1 -- 172.21.15.84:0/1149718455 shutdown_connections 2024-05-22T23:06:12.155 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.147+0000 7f0bb637e640 1 -- 172.21.15.84:0/1149718455 wait complete. 2024-05-22T23:06:12.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.148+0000 7f0bb637e640 1 Processor -- start 2024-05-22T23:06:12.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.148+0000 7f0bb637e640 1 -- start start 2024-05-22T23:06:12.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.148+0000 7f0bb637e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bb0101870 0x7f0bb01981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.148+0000 7f0bb637e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb0102230 0x7f0bb0198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.148+0000 7f0bb637e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb0107880 0x7f0bb019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.148+0000 7f0bb637e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb0111c90 con 0x7f0bb0101870 2024-05-22T23:06:12.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.148+0000 7f0bb637e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb0111b10 con 0x7f0bb0102230 2024-05-22T23:06:12.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.148+0000 7f0bb637e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb0111e10 con 0x7f0bb0107880 2024-05-22T23:06:12.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.148+0000 7f0baffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bb0101870 0x7f0bb01981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.148+0000 7f0baffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bb0101870 0x7f0bb01981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56082/0 (socket says 172.21.15.84:56082) 2024-05-22T23:06:12.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.148+0000 7f0baffff640 1 -- 172.21.15.84:0/2551037144 learned_addr learned my addr 172.21.15.84:0/2551037144 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:12.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.148+0000 7f0baf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb0102230 0x7f0bb0198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.148+0000 7f0bb491b640 1 --2- 172.21.15.84:0/2551037144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb0107880 0x7f0bb019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.149+0000 7f0baffff640 1 -- 172.21.15.84:0/2551037144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb0107880 msgr2=0x7f0bb019ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:12.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.149+0000 7f0baffff640 1 --2- 172.21.15.84:0/2551037144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb0107880 0x7f0bb019ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.149+0000 7f0baffff640 1 -- 172.21.15.84:0/2551037144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb0102230 msgr2=0x7f0bb0198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:12.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.149+0000 7f0baffff640 1 --2- 172.21.15.84:0/2551037144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb0102230 0x7f0bb0198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.149+0000 7f0baffff640 1 -- 172.21.15.84:0/2551037144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb019d1c0 con 0x7f0bb0101870 2024-05-22T23:06:12.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.149+0000 7f0baffff640 1 --2- 172.21.15.84:0/2551037144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bb0101870 0x7f0bb01981d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009230 tx=0x7f0ba0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:12.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.149+0000 7f0bad7fa640 1 -- 172.21.15.84:0/2551037144 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba003f030 con 0x7f0bb0101870 2024-05-22T23:06:12.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.149+0000 7f0bb637e640 1 -- 172.21.15.84:0/2551037144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb019d450 con 0x7f0bb0101870 2024-05-22T23:06:12.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.149+0000 7f0bb637e640 1 -- 172.21.15.84:0/2551037144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb01a4d40 con 0x7f0bb0101870 2024-05-22T23:06:12.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.149+0000 7f0bad7fa640 1 -- 172.21.15.84:0/2551037144 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ba0004440 con 0x7f0bb0101870 2024-05-22T23:06:12.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.149+0000 7f0bad7fa640 1 -- 172.21.15.84:0/2551037144 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0049060 con 0x7f0bb0101870 2024-05-22T23:06:12.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.150+0000 7f0bb637e640 1 -- 172.21.15.84:0/2551037144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bb0068ae0 con 0x7f0bb0101870 2024-05-22T23:06:12.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.151+0000 7f0bad7fa640 1 -- 172.21.15.84:0/2551037144 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0ba0042030 con 0x7f0bb0101870 2024-05-22T23:06:12.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.152+0000 7f0bad7fa640 1 --2- 172.21.15.84:0/2551037144 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f0b9c0a2d00 0x7f0b9c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.152+0000 7f0baf7fe640 1 --2- 172.21.15.84:0/2551037144 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f0b9c0a2d00 0x7f0b9c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.152+0000 7f0bad7fa640 1 -- 172.21.15.84:0/2551037144 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f0ba00ef790 con 0x7f0bb0101870 2024-05-22T23:06:12.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.152+0000 7f0baf7fe640 1 --2- 172.21.15.84:0/2551037144 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f0b9c0a2d00 0x7f0b9c0a51c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0101190 tx=0x7f0b98005f90 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:12.160 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.155+0000 7f0bad7fa640 1 -- 172.21.15.84:0/2551037144 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0ba00bbd60 con 0x7f0bb0101870 2024-05-22T23:06:12.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.396+0000 7f0bb637e640 1 -- 172.21.15.84:0/2551037144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0bb0109de0 con 0x7f0bb0101870 2024-05-22T23:06:12.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.396+0000 7f0bad7fa640 1 -- 172.21.15.84:0/2551037144 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7f0ba00c0060 con 0x7f0bb0101870 2024-05-22T23:06:12.401 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:06:12.401 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:06:12.401 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:06:12.403 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.399+0000 7f0bb637e640 1 -- 172.21.15.84:0/2551037144 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f0b9c0a2d00 msgr2=0x7f0b9c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:12.403 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.399+0000 7f0bb637e640 1 --2- 172.21.15.84:0/2551037144 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f0b9c0a2d00 0x7f0b9c0a51c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0101190 tx=0x7f0b98005f90 comp rx=0 tx=0).stop 2024-05-22T23:06:12.403 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.399+0000 7f0bb637e640 1 -- 172.21.15.84:0/2551037144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bb0101870 msgr2=0x7f0bb01981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:12.403 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.399+0000 7f0bb637e640 1 --2- 172.21.15.84:0/2551037144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bb0101870 0x7f0bb01981d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009230 tx=0x7f0ba0004040 comp rx=0 tx=0).stop 2024-05-22T23:06:12.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.399+0000 7f0bb637e640 1 -- 172.21.15.84:0/2551037144 shutdown_connections 2024-05-22T23:06:12.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.399+0000 7f0bb637e640 1 --2- 172.21.15.84:0/2551037144 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f0b9c0a2d00 0x7f0b9c0a51c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.399+0000 7f0bb637e640 1 --2- 172.21.15.84:0/2551037144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb0107880 0x7f0bb019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.399+0000 7f0bb637e640 1 --2- 172.21.15.84:0/2551037144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb0102230 0x7f0bb0198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.399+0000 7f0bb637e640 1 --2- 172.21.15.84:0/2551037144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bb0101870 0x7f0bb01981d0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.399+0000 7f0bb637e640 1 -- 172.21.15.84:0/2551037144 >> 172.21.15.84:0/2551037144 conn(0x7f0bb0078d00 msgr2=0x7f0bb00ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:12.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.399+0000 7f0bb637e640 1 -- 172.21.15.84:0/2551037144 shutdown_connections 2024-05-22T23:06:12.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.399+0000 7f0bb637e640 1 -- 172.21.15.84:0/2551037144 wait complete. 2024-05-22T23:06:12.416 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-22T23:06:12.432 INFO:tasks.workunit.client.0.smithi084.stdout: 1 515915 15.53 MB/sec execute 58 sec latency 43.386 ms 2024-05-22T23:06:12.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.521+0000 7f7784142640 1 -- 172.21.15.84:0/3570555582 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f777c100740 msgr2=0x7f777c100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:12.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.521+0000 7f7784142640 1 --2- 172.21.15.84:0/3570555582 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f777c100740 0x7f777c100bc0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7778009930 tx=0x7f777802a420 comp rx=0 tx=0).stop 2024-05-22T23:06:12.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.521+0000 7f7784142640 1 -- 172.21.15.84:0/3570555582 shutdown_connections 2024-05-22T23:06:12.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.521+0000 7f7784142640 1 --2- 172.21.15.84:0/3570555582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f777c101100 0x7f777c10d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.521+0000 7f7784142640 1 --2- 172.21.15.84:0/3570555582 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f777c100740 0x7f777c100bc0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.521+0000 7f7784142640 1 --2- 172.21.15.84:0/3570555582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f777c106750 0x7f777c106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.521+0000 7f7784142640 1 -- 172.21.15.84:0/3570555582 >> 172.21.15.84:0/3570555582 conn(0x7f777c077870 msgr2=0x7f777c077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:12.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.522+0000 7f7784142640 1 -- 172.21.15.84:0/3570555582 shutdown_connections 2024-05-22T23:06:12.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.522+0000 7f7784142640 1 -- 172.21.15.84:0/3570555582 wait complete. 2024-05-22T23:06:12.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.522+0000 7f7784142640 1 Processor -- start 2024-05-22T23:06:12.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.522+0000 7f7784142640 1 -- start start 2024-05-22T23:06:12.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.522+0000 7f7784142640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f777c100740 0x7f777c197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.522+0000 7f7784142640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f777c101100 0x7f777c1975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.522+0000 7f7784142640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f777c106750 0x7f777c19b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.522+0000 7f7784142640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f777c110b60 con 0x7f777c106750 2024-05-22T23:06:12.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.522+0000 7f7784142640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f777c1109e0 con 0x7f777c101100 2024-05-22T23:06:12.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.522+0000 7f7784142640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f777c110ce0 con 0x7f777c100740 2024-05-22T23:06:12.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.523+0000 7f77826df640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f777c106750 0x7f777c19b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.523+0000 7f77816dd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f777c101100 0x7f777c1975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.523+0000 7f77826df640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f777c106750 0x7f777c19b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56096/0 (socket says 172.21.15.84:56096) 2024-05-22T23:06:12.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.523+0000 7f77826df640 1 -- 172.21.15.84:0/3094897040 learned_addr learned my addr 172.21.15.84:0/3094897040 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:12.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.523+0000 7f7781ede640 1 --2- 172.21.15.84:0/3094897040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f777c100740 0x7f777c197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.523+0000 7f77826df640 1 -- 172.21.15.84:0/3094897040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f777c100740 msgr2=0x7f777c197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:12.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.523+0000 7f77826df640 1 --2- 172.21.15.84:0/3094897040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f777c100740 0x7f777c197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.523+0000 7f77826df640 1 -- 172.21.15.84:0/3094897040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f777c101100 msgr2=0x7f777c1975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:12.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.523+0000 7f77826df640 1 --2- 172.21.15.84:0/3094897040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f777c101100 0x7f777c1975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.523+0000 7f77826df640 1 -- 172.21.15.84:0/3094897040 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777c19c070 con 0x7f777c106750 2024-05-22T23:06:12.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.523+0000 7f77816dd640 1 --2- 172.21.15.84:0/3094897040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f777c101100 0x7f777c1975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:06:12.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.523+0000 7f77826df640 1 --2- 172.21.15.84:0/3094897040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f777c106750 0x7f777c19b940 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f777400eca0 tx=0x7f777400b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:12.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.523+0000 7f7772ffd640 1 -- 172.21.15.84:0/3094897040 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7774004b20 con 0x7f777c106750 2024-05-22T23:06:12.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.523+0000 7f7784142640 1 -- 172.21.15.84:0/3094897040 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f777c19c360 con 0x7f777c106750 2024-05-22T23:06:12.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.523+0000 7f7772ffd640 1 -- 172.21.15.84:0/3094897040 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7774004cc0 con 0x7f777c106750 2024-05-22T23:06:12.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.523+0000 7f7772ffd640 1 -- 172.21.15.84:0/3094897040 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7774017650 con 0x7f777c106750 2024-05-22T23:06:12.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.523+0000 7f7784142640 1 -- 172.21.15.84:0/3094897040 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f777c068a00 con 0x7f777c106750 2024-05-22T23:06:12.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.525+0000 7f7772ffd640 1 -- 172.21.15.84:0/3094897040 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7774005020 con 0x7f777c106750 2024-05-22T23:06:12.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.526+0000 7f7786e51740 1 Processor -- start 2024-05-22T23:06:12.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.526+0000 7f7786e51740 1 -- start start 2024-05-22T23:06:12.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.526+0000 7f7786e51740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4e1e60 0x55c9ce4e2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.526+0000 7f7786e51740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ce4e2780 0x55c9ce4f0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.526+0000 7f7786e51740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9ce4f1380 0x55c9ce4f3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.526+0000 7f7786e51740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c9ce446db0 con 0x55c9ce4e1e60 2024-05-22T23:06:12.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.526+0000 7f7786e51740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c9ce3e5400 con 0x55c9ce4e2780 2024-05-22T23:06:12.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.526+0000 7f7786e51740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c9ce442d70 con 0x55c9ce4f1380 2024-05-22T23:06:12.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.526+0000 7f77816dd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4e1e60 0x55c9ce4e2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.526+0000 7f77816dd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4e1e60 0x55c9ce4e2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56100/0 (socket says 172.21.15.84:56100) 2024-05-22T23:06:12.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.526+0000 7f77816dd640 1 -- 172.21.15.84:0/2273898804 learned_addr learned my addr 172.21.15.84:0/2273898804 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:12.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.526+0000 7f7781ede640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ce4e2780 0x55c9ce4f0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.526+0000 7f77816dd640 1 --2- 172.21.15.84:0/2273898804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9ce4f1380 0x55c9ce4f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.526+0000 7f7772ffd640 1 --2- 172.21.15.84:0/3094897040 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f77680a2d00 0x7f77680a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.526+0000 7f77816dd640 1 -- 172.21.15.84:0/2273898804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9ce4f1380 msgr2=0x55c9ce4f3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:12.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.526+0000 7f77816dd640 1 --2- 172.21.15.84:0/2273898804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9ce4f1380 0x55c9ce4f3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.526+0000 7f77816dd640 1 -- 172.21.15.84:0/2273898804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ce4e2780 msgr2=0x55c9ce4f0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:12.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.527+0000 7f77816dd640 1 --2- 172.21.15.84:0/2273898804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ce4e2780 0x55c9ce4f0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.527+0000 7f77816dd640 1 -- 172.21.15.84:0/2273898804 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9ce3d72f0 con 0x55c9ce4e1e60 2024-05-22T23:06:12.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.527+0000 7f7781ede640 1 --2- 172.21.15.84:0/2273898804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ce4e2780 0x55c9ce4f0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:06:12.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.527+0000 7f77826df640 1 --2- 172.21.15.84:0/3094897040 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f77680a2d00 0x7f77680a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.527+0000 7f77816dd640 1 --2- 172.21.15.84:0/2273898804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4e1e60 0x55c9ce4e2240 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7778006010 tx=0x7f7778031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5837d951945ae725 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:12.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.527+0000 7f7772ffd640 1 -- 172.21.15.84:0/3094897040 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f77740d0e80 con 0x7f777c106750 2024-05-22T23:06:12.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.527+0000 7f7767fff640 1 -- 172.21.15.84:0/2273898804 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7778044070 con 0x55c9ce4e1e60 2024-05-22T23:06:12.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.527+0000 7f7767fff640 1 -- 172.21.15.84:0/2273898804 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7778005020 con 0x55c9ce4e1e60 2024-05-22T23:06:12.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.527+0000 7f7767fff640 1 -- 172.21.15.84:0/2273898804 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7778035040 con 0x55c9ce4e1e60 2024-05-22T23:06:12.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.528+0000 7f7786e51740 1 -- 172.21.15.84:0/2273898804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4e1e60 msgr2=0x55c9ce4e2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:12.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.528+0000 7f77826df640 1 --2- 172.21.15.84:0/3094897040 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f77680a2d00 0x7f77680a51c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f77740ce680 tx=0x7f77740c7f30 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:12.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.528+0000 7f7786e51740 1 --2- 172.21.15.84:0/2273898804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4e1e60 0x55c9ce4e2240 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7778006010 tx=0x7f7778031400 comp rx=0 tx=0).stop 2024-05-22T23:06:12.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.528+0000 7f7786e51740 1 -- 172.21.15.84:0/2273898804 shutdown_connections 2024-05-22T23:06:12.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.528+0000 7f7786e51740 1 --2- 172.21.15.84:0/2273898804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9ce4f1380 0x55c9ce4f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.528+0000 7f7786e51740 1 --2- 172.21.15.84:0/2273898804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ce4e2780 0x55c9ce4f0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.528+0000 7f7786e51740 1 --2- 172.21.15.84:0/2273898804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4e1e60 0x55c9ce4e2240 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.528+0000 7f7786e51740 1 -- 172.21.15.84:0/2273898804 >> 172.21.15.84:0/2273898804 conn(0x55c9ce4de2a0 msgr2=0x55c9ce4de680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:12.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.528+0000 7f7786e51740 1 -- 172.21.15.84:0/2273898804 shutdown_connections 2024-05-22T23:06:12.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.528+0000 7f7786e51740 1 -- 172.21.15.84:0/2273898804 wait complete. 2024-05-22T23:06:12.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.528+0000 7f7786e51740 1 -- start start 2024-05-22T23:06:12.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.528+0000 7f7786e51740 1 Processor -- start 2024-05-22T23:06:12.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.529+0000 7f7786e51740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9ce4e1e60 0x55c9ce4ee2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.529+0000 7f7786e51740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ce4e2780 0x55c9ce4ee830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.529+0000 7f7786e51740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4f1380 0x55c9ce4eed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.529+0000 7f7786e51740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c9ce4f4160 con 0x55c9ce4f1380 2024-05-22T23:06:12.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.529+0000 7f7786e51740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c9ce4012e0 con 0x55c9ce4e2780 2024-05-22T23:06:12.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.529+0000 7f7786e51740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c9ce4f42e0 con 0x55c9ce4e1e60 2024-05-22T23:06:12.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.529+0000 7f7786e51740 10 client.0 _ll_register_callbacks cb 0x55c9ce40aa70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:06:12.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.529+0000 7f77816dd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4f1380 0x55c9ce4eed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.529+0000 7f7781ede640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ce4e2780 0x55c9ce4ee830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.529+0000 7f77816dd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4f1380 0x55c9ce4eed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56106/0 (socket says 172.21.15.84:56106) 2024-05-22T23:06:12.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.529+0000 7f77816dd640 1 -- 172.21.15.84:0/3509510713 learned_addr learned my addr 172.21.15.84:0/3509510713 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:12.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.529+0000 7f77816dd640 1 --2- 172.21.15.84:0/3509510713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9ce4e1e60 0x55c9ce4ee2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.529+0000 7f77816dd640 1 -- 172.21.15.84:0/3509510713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9ce4e1e60 msgr2=0x55c9ce4ee2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:12.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.529+0000 7f77816dd640 1 --2- 172.21.15.84:0/3509510713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9ce4e1e60 0x55c9ce4ee2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.529+0000 7f77816dd640 1 -- 172.21.15.84:0/3509510713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ce4e2780 msgr2=0x55c9ce4ee830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:12.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.529+0000 7f77816dd640 1 --2- 172.21.15.84:0/3509510713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ce4e2780 0x55c9ce4ee830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.529+0000 7f77816dd640 1 -- 172.21.15.84:0/3509510713 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9ce15a3f0 con 0x55c9ce4f1380 2024-05-22T23:06:12.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.529+0000 7f77816dd640 1 --2- 172.21.15.84:0/3509510713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4f1380 0x55c9ce4eed70 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7778031470 tx=0x7f77780045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f8d5272303efe2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:12.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.529+0000 7f77657fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:06:12.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.529+0000 7f77657fa640 1 -- 172.21.15.84:0/3509510713 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7778053060 con 0x55c9ce4f1380 2024-05-22T23:06:12.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.529+0000 7f77657fa640 1 -- 172.21.15.84:0/3509510713 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f77780053d0 con 0x55c9ce4f1380 2024-05-22T23:06:12.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.529+0000 7f7786e51740 1 -- 172.21.15.84:0/3509510713 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7778005020 con 0x55c9ce4f1380 2024-05-22T23:06:12.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.530+0000 7f77657fa640 1 -- 172.21.15.84:0/3509510713 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f7778044070 con 0x55c9ce4f1380 2024-05-22T23:06:12.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.530+0000 7f7786e51740 10 client.16128 fetch_fsmap learned FSMap version 18 2024-05-22T23:06:12.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.530+0000 7f7786e51740 1 -- 172.21.15.84:0/3509510713 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55c9ce2247d0 con 0x55c9ce4f1380 2024-05-22T23:06:12.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.530+0000 7f77657fa640 1 -- 172.21.15.84:0/3509510713 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7778005020 con 0x55c9ce4f1380 2024-05-22T23:06:12.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.530+0000 7f77657fa640 1 -- 172.21.15.84:0/3509510713 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f7778003860 con 0x55c9ce4f1380 2024-05-22T23:06:12.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.530+0000 7f7786e51740 10 client.16128 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:06:12.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.530+0000 7f7786e51740 10 client.16128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:06:12.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.530+0000 7f7786e51740 20 client.16128 populate_metadata read hostname 'smithi084' 2024-05-22T23:06:12.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.530+0000 7f7786e51740 1 --2- 172.21.15.84:0/3509510713 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c9ce501640 0x55c9ce503a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.530+0000 7f7786e51740 4 client.16128 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:06:12.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.530+0000 7f7786e51740 1 -- 172.21.15.84:0/3509510713 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9ce4544e0 con 0x55c9ce501640 2024-05-22T23:06:12.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.530+0000 7f7781ede640 1 --2- 172.21.15.84:0/3509510713 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c9ce501640 0x55c9ce503a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.531+0000 7f7781ede640 1 --2- 172.21.15.84:0/3509510713 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c9ce501640 0x55c9ce503a30 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=915df5159995395e server_cookie=71b8a1c3da27cb8d in_seq=0 out_seq=0 2024-05-22T23:06:12.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.531+0000 7f77657fa640 10 client.16128 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:12.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.531+0000 7f77657fa640 1 -- 172.21.15.84:0/3509510713 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c9ce4544e0 con 0x55c9ce501640 2024-05-22T23:06:12.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.531+0000 7f77657fa640 10 client.16128 handle_command_reply: tid=0 2024-05-22T23:06:12.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.531+0000 7f7786e51740 1 client.16128 shutdown 2024-05-22T23:06:12.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.532+0000 7f7786e51740 1 -- 172.21.15.84:0/3509510713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4f1380 msgr2=0x55c9ce4eed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:12.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.532+0000 7f7786e51740 1 --2- 172.21.15.84:0/3509510713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4f1380 0x55c9ce4eed70 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7778031470 tx=0x7f77780045b0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.532+0000 7f7786e51740 1 -- 172.21.15.84:0/3509510713 shutdown_connections 2024-05-22T23:06:12.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.532+0000 7f7786e51740 1 --2- 172.21.15.84:0/3509510713 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c9ce501640 0x55c9ce503a30 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.532+0000 7f7786e51740 1 --2- 172.21.15.84:0/3509510713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4f1380 0x55c9ce4eed70 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.532+0000 7f7786e51740 1 --2- 172.21.15.84:0/3509510713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ce4e2780 0x55c9ce4ee830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.532+0000 7f7786e51740 1 --2- 172.21.15.84:0/3509510713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9ce4e1e60 0x55c9ce4ee2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.532+0000 7f77657fa640 0 client.16128 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:12.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.532+0000 7f7786e51740 1 -- 172.21.15.84:0/3509510713 >> 172.21.15.84:0/3509510713 conn(0x55c9ce4de2a0 msgr2=0x55c9ce4e4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:12.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.532+0000 7f7786e51740 1 -- 172.21.15.84:0/3509510713 shutdown_connections 2024-05-22T23:06:12.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.532+0000 7f7786e51740 1 -- 172.21.15.84:0/3509510713 wait complete. 2024-05-22T23:06:12.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.532+0000 7f7786e51740 20 client.16128 trim_cache size 0 max 16384 2024-05-22T23:06:12.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.547+0000 7f7786e51740 1 Processor -- start 2024-05-22T23:06:12.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.547+0000 7f7786e51740 1 -- start start 2024-05-22T23:06:12.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.547+0000 7f7786e51740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ce4e1e60 0x55c9ce4f0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.547+0000 7f7786e51740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4e2780 0x55c9ce4fcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.547+0000 7f7786e51740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9ce4f1380 0x55c9ce506a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.547+0000 7f7786e51740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f777c1a9290 con 0x55c9ce4e2780 2024-05-22T23:06:12.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.547+0000 7f7786e51740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f777c1abb10 con 0x55c9ce4e1e60 2024-05-22T23:06:12.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.547+0000 7f7786e51740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f777c1a9900 con 0x55c9ce4f1380 2024-05-22T23:06:12.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.547+0000 7f7781ede640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4e2780 0x55c9ce4fcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.547+0000 7f7781ede640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4e2780 0x55c9ce4fcce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56122/0 (socket says 172.21.15.84:56122) 2024-05-22T23:06:12.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.547+0000 7f7781ede640 1 -- 172.21.15.84:0/2095991127 learned_addr learned my addr 172.21.15.84:0/2095991127 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:12.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.547+0000 7f77816dd640 1 --2- 172.21.15.84:0/2095991127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ce4e1e60 0x55c9ce4f0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.547+0000 7f7781ede640 1 -- 172.21.15.84:0/2095991127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9ce4f1380 msgr2=0x55c9ce506a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:12.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.547+0000 7f7781ede640 1 --2- 172.21.15.84:0/2095991127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9ce4f1380 0x55c9ce506a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.547+0000 7f7781ede640 1 -- 172.21.15.84:0/2095991127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ce4e1e60 msgr2=0x55c9ce4f0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:12.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.547+0000 7f7781ede640 1 --2- 172.21.15.84:0/2095991127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ce4e1e60 0x55c9ce4f0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.547+0000 7f7781ede640 1 -- 172.21.15.84:0/2095991127 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9ce4e25b0 con 0x55c9ce4e2780 2024-05-22T23:06:12.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.547+0000 7f7781ede640 1 --2- 172.21.15.84:0/2095991127 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4e2780 0x55c9ce4fcce0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f776c009dd0 tx=0x7f776c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e891c011cf854ea server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:12.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.547+0000 7f77677fe640 1 -- 172.21.15.84:0/2095991127 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c004250 con 0x55c9ce4e2780 2024-05-22T23:06:12.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.548+0000 7f77677fe640 1 -- 172.21.15.84:0/2095991127 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f776c0043f0 con 0x55c9ce4e2780 2024-05-22T23:06:12.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.548+0000 7f77677fe640 1 -- 172.21.15.84:0/2095991127 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c0205d0 con 0x55c9ce4e2780 2024-05-22T23:06:12.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.548+0000 7f7786e51740 1 -- 172.21.15.84:0/2095991127 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4e2780 msgr2=0x55c9ce4fcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:12.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.548+0000 7f7786e51740 1 --2- 172.21.15.84:0/2095991127 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4e2780 0x55c9ce4fcce0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f776c009dd0 tx=0x7f776c005f90 comp rx=0 tx=0).stop 2024-05-22T23:06:12.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.548+0000 7f7786e51740 1 -- 172.21.15.84:0/2095991127 shutdown_connections 2024-05-22T23:06:12.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.548+0000 7f7786e51740 1 --2- 172.21.15.84:0/2095991127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9ce4f1380 0x55c9ce506a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.548+0000 7f7786e51740 1 --2- 172.21.15.84:0/2095991127 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4e2780 0x55c9ce4fcce0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.548+0000 7f7786e51740 1 --2- 172.21.15.84:0/2095991127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ce4e1e60 0x55c9ce4f0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.548+0000 7f7786e51740 1 -- 172.21.15.84:0/2095991127 >> 172.21.15.84:0/2095991127 conn(0x55c9ce4de2a0 msgr2=0x55c9ce4fc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:12.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.548+0000 7f7786e51740 1 -- 172.21.15.84:0/2095991127 shutdown_connections 2024-05-22T23:06:12.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.548+0000 7f7786e51740 1 -- 172.21.15.84:0/2095991127 wait complete. 2024-05-22T23:06:12.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.548+0000 7f7786e51740 1 -- start start 2024-05-22T23:06:12.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.549+0000 7f7786e51740 1 Processor -- start 2024-05-22T23:06:12.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.549+0000 7f7786e51740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ce4e1e60 0x55c9ce506fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.549+0000 7f7786e51740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9ce4e2780 0x55c9ce504360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.549+0000 7f7786e51740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4f1380 0x55c9ce504a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.549+0000 7f7786e51740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f777c1a9fd0 con 0x55c9ce4f1380 2024-05-22T23:06:12.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.549+0000 7f7786e51740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f777c1ab440 con 0x55c9ce4e1e60 2024-05-22T23:06:12.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.549+0000 7f7786e51740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f777c1aa6a0 con 0x55c9ce4e2780 2024-05-22T23:06:12.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.549+0000 7f7786e51740 10 client.0 _ll_register_callbacks cb 0x7f776c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:06:12.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.549+0000 7f77816dd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4f1380 0x55c9ce504a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.549+0000 7f77816dd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4f1380 0x55c9ce504a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56138/0 (socket says 172.21.15.84:56138) 2024-05-22T23:06:12.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.549+0000 7f77816dd640 1 -- 172.21.15.84:0/3222698600 learned_addr learned my addr 172.21.15.84:0/3222698600 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:12.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.549+0000 7f7781ede640 1 --2- 172.21.15.84:0/3222698600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9ce4e2780 0x55c9ce504360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.549+0000 7f77816dd640 1 --2- 172.21.15.84:0/3222698600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ce4e1e60 0x55c9ce506fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.549+0000 7f77816dd640 1 -- 172.21.15.84:0/3222698600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9ce4e2780 msgr2=0x55c9ce504360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:12.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.549+0000 7f77816dd640 1 --2- 172.21.15.84:0/3222698600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9ce4e2780 0x55c9ce504360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.549+0000 7f77816dd640 1 -- 172.21.15.84:0/3222698600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ce4e1e60 msgr2=0x55c9ce506fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:12.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.549+0000 7f77816dd640 1 --2- 172.21.15.84:0/3222698600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ce4e1e60 0x55c9ce506fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.549+0000 7f77816dd640 1 -- 172.21.15.84:0/3222698600 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9ce566820 con 0x55c9ce4f1380 2024-05-22T23:06:12.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.550+0000 7f77816dd640 1 --2- 172.21.15.84:0/3222698600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4f1380 0x55c9ce504a50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f777802abd0 tx=0x7f7778031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c37cd5f1645562 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:12.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.550+0000 7f77657fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:06:12.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.550+0000 7f77657fa640 1 -- 172.21.15.84:0/3222698600 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7778035040 con 0x55c9ce4f1380 2024-05-22T23:06:12.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.550+0000 7f77657fa640 1 -- 172.21.15.84:0/3222698600 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f77780035f0 con 0x55c9ce4f1380 2024-05-22T23:06:12.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.550+0000 7f7786e51740 1 -- 172.21.15.84:0/3222698600 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f776c0043f0 con 0x55c9ce4f1380 2024-05-22T23:06:12.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.550+0000 7f77657fa640 1 -- 172.21.15.84:0/3222698600 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f7778053020 con 0x55c9ce4f1380 2024-05-22T23:06:12.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.550+0000 7f7786e51740 10 client.16134 fetch_fsmap learned FSMap version 18 2024-05-22T23:06:12.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.550+0000 7f7786e51740 1 -- 172.21.15.84:0/3222698600 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55c9ce5679c0 con 0x55c9ce4f1380 2024-05-22T23:06:12.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.550+0000 7f77657fa640 1 -- 172.21.15.84:0/3222698600 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f776c0043f0 con 0x55c9ce4f1380 2024-05-22T23:06:12.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.550+0000 7f77657fa640 1 -- 172.21.15.84:0/3222698600 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f7778032070 con 0x55c9ce4f1380 2024-05-22T23:06:12.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.550+0000 7f7786e51740 10 client.16134 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:06:12.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.550+0000 7f7786e51740 10 client.16134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:06:12.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.550+0000 7f7786e51740 20 client.16134 populate_metadata read hostname 'smithi084' 2024-05-22T23:06:12.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.550+0000 7f7786e51740 1 --2- 172.21.15.84:0/3222698600 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c9ce568020 0x55c9ce56a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:12.579 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.550+0000 7f7786e51740 4 client.16134 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-22T23:06:12.579 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.550+0000 7f7786e51740 1 -- 172.21.15.84:0/3222698600 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c9ce44e830 con 0x55c9ce568020 2024-05-22T23:06:12.579 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.550+0000 7f7781ede640 1 --2- 172.21.15.84:0/3222698600 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c9ce568020 0x55c9ce56a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:12.579 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.551+0000 7f77657fa640 10 client.16134 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:12.579 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.551+0000 7f7781ede640 1 --2- 172.21.15.84:0/3222698600 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c9ce568020 0x55c9ce56a440 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff41c730f1f57f21 server_cookie=5d05a81d48ab170b in_seq=0 out_seq=0 2024-05-22T23:06:12.579 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.552+0000 7f77657fa640 1 -- 172.21.15.84:0/3222698600 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c9ce44e830 con 0x55c9ce568020 2024-05-22T23:06:12.580 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.552+0000 7f77657fa640 10 client.16134 handle_command_reply: tid=0 2024-05-22T23:06:12.580 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.552+0000 7f7786e51740 1 client.16134 shutdown 2024-05-22T23:06:12.580 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.552+0000 7f7786e51740 1 -- 172.21.15.84:0/3222698600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4f1380 msgr2=0x55c9ce504a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:12.580 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.552+0000 7f7786e51740 1 --2- 172.21.15.84:0/3222698600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4f1380 0x55c9ce504a50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f777802abd0 tx=0x7f7778031400 comp rx=0 tx=0).stop 2024-05-22T23:06:12.580 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.552+0000 7f7786e51740 1 -- 172.21.15.84:0/3222698600 shutdown_connections 2024-05-22T23:06:12.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.552+0000 7f7786e51740 1 --2- 172.21.15.84:0/3222698600 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c9ce568020 0x55c9ce56a440 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.552+0000 7f7786e51740 1 --2- 172.21.15.84:0/3222698600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9ce4f1380 0x55c9ce504a50 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.552+0000 7f77657fa640 0 client.16134 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:12.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.552+0000 7f7786e51740 1 --2- 172.21.15.84:0/3222698600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9ce4e2780 0x55c9ce504360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.552+0000 7f7786e51740 1 --2- 172.21.15.84:0/3222698600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ce4e1e60 0x55c9ce506fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.552+0000 7f7786e51740 1 -- 172.21.15.84:0/3222698600 >> 172.21.15.84:0/3222698600 conn(0x55c9ce4de2a0 msgr2=0x55c9ce4fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:12.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.553+0000 7f7786e51740 1 -- 172.21.15.84:0/3222698600 shutdown_connections 2024-05-22T23:06:12.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.553+0000 7f7786e51740 1 -- 172.21.15.84:0/3222698600 wait complete. 2024-05-22T23:06:12.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.553+0000 7f7786e51740 20 client.16134 trim_cache size 0 max 16384 2024-05-22T23:06:12.582 INFO:teuthology.orchestra.run.smithi084.stdout:[] 2024-05-22T23:06:12.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.553+0000 7f7784142640 1 -- 172.21.15.84:0/3094897040 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f77680a2d00 msgr2=0x7f77680a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:12.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.553+0000 7f7784142640 1 --2- 172.21.15.84:0/3094897040 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f77680a2d00 0x7f77680a51c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f77740ce680 tx=0x7f77740c7f30 comp rx=0 tx=0).stop 2024-05-22T23:06:12.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.553+0000 7f7784142640 1 -- 172.21.15.84:0/3094897040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f777c106750 msgr2=0x7f777c19b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:12.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.553+0000 7f7784142640 1 --2- 172.21.15.84:0/3094897040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f777c106750 0x7f777c19b940 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f777400eca0 tx=0x7f777400b690 comp rx=0 tx=0).stop 2024-05-22T23:06:12.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.553+0000 7f7784142640 1 -- 172.21.15.84:0/3094897040 shutdown_connections 2024-05-22T23:06:12.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.553+0000 7f7784142640 1 --2- 172.21.15.84:0/3094897040 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f77680a2d00 0x7f77680a51c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.553+0000 7f7784142640 1 --2- 172.21.15.84:0/3094897040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f777c106750 0x7f777c19b940 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.553+0000 7f7784142640 1 --2- 172.21.15.84:0/3094897040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f777c101100 0x7f777c1975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.553+0000 7f7784142640 1 --2- 172.21.15.84:0/3094897040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f777c100740 0x7f777c197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:12.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.553+0000 7f7784142640 1 -- 172.21.15.84:0/3094897040 >> 172.21.15.84:0/3094897040 conn(0x7f777c077870 msgr2=0x7f777c102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:12.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.553+0000 7f7784142640 1 -- 172.21.15.84:0/3094897040 shutdown_connections 2024-05-22T23:06:12.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:12.553+0000 7f7784142640 1 -- 172.21.15.84:0/3094897040 wait complete. 2024-05-22T23:06:12.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:12 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2551037144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:06:12.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:12 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2551037144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:06:12.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:12 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2551037144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:06:13.433 INFO:tasks.workunit.client.0.smithi084.stdout: 1 519225 15.58 MB/sec execute 59 sec latency 6.697 ms 2024-05-22T23:06:13.585 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-22T23:06:13.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:06:13.585 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-22T23:06:13.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.692+0000 7f033c7db640 1 -- 172.21.15.84:0/3950712385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0334106760 msgr2=0x7f0334106b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:13.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.692+0000 7f033c7db640 1 --2- 172.21.15.84:0/3950712385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0334106760 0x7f0334106b40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0330009930 tx=0x7f033002a420 comp rx=0 tx=0).stop 2024-05-22T23:06:13.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.692+0000 7f033c7db640 1 -- 172.21.15.84:0/3950712385 shutdown_connections 2024-05-22T23:06:13.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.692+0000 7f033c7db640 1 --2- 172.21.15.84:0/3950712385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0334101110 0x7f033410d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.692+0000 7f033c7db640 1 --2- 172.21.15.84:0/3950712385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0334100750 0x7f0334100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.692+0000 7f033c7db640 1 --2- 172.21.15.84:0/3950712385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0334106760 0x7f0334106b40 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.692+0000 7f033c7db640 1 -- 172.21.15.84:0/3950712385 >> 172.21.15.84:0/3950712385 conn(0x7f03340777d0 msgr2=0x7f0334077be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:13.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.692+0000 7f033c7db640 1 -- 172.21.15.84:0/3950712385 shutdown_connections 2024-05-22T23:06:13.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.692+0000 7f033c7db640 1 -- 172.21.15.84:0/3950712385 wait complete. 2024-05-22T23:06:13.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.693+0000 7f033c7db640 1 Processor -- start 2024-05-22T23:06:13.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.693+0000 7f033c7db640 1 -- start start 2024-05-22T23:06:13.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.693+0000 7f033c7db640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0334100750 0x7f03340fd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.693+0000 7f033c7db640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0334101110 0x7f03340fdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.693+0000 7f033c7db640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0334106760 0x7f03340fe200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.693+0000 7f033c7db640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0334110b30 con 0x7f0334100750 2024-05-22T23:06:13.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.693+0000 7f033c7db640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f03341109b0 con 0x7f0334106760 2024-05-22T23:06:13.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.693+0000 7f033c7db640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0334110cb0 con 0x7f0334101110 2024-05-22T23:06:13.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.693+0000 7f033a577640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0334100750 0x7f03340fd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.693+0000 7f033a577640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0334100750 0x7f03340fd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56160/0 (socket says 172.21.15.84:56160) 2024-05-22T23:06:13.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.693+0000 7f033a577640 1 -- 172.21.15.84:0/3222448013 learned_addr learned my addr 172.21.15.84:0/3222448013 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:13.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.693+0000 7f0339d76640 1 --2- 172.21.15.84:0/3222448013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0334101110 0x7f03340fdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.693+0000 7f033ad78640 1 --2- 172.21.15.84:0/3222448013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0334106760 0x7f03340fe200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.693+0000 7f033a577640 1 -- 172.21.15.84:0/3222448013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0334101110 msgr2=0x7f03340fdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:13.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.693+0000 7f033a577640 1 --2- 172.21.15.84:0/3222448013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0334101110 0x7f03340fdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.693+0000 7f033a577640 1 -- 172.21.15.84:0/3222448013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0334106760 msgr2=0x7f03340fe200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:13.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.693+0000 7f033a577640 1 --2- 172.21.15.84:0/3222448013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0334106760 0x7f03340fe200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.694+0000 7f033a577640 1 -- 172.21.15.84:0/3222448013 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03341a6530 con 0x7f0334100750 2024-05-22T23:06:13.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.694+0000 7f033a577640 1 --2- 172.21.15.84:0/3222448013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0334100750 0x7f03340fd780 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0330009230 tx=0x7f0330004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:13.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.694+0000 7f032b7fe640 1 -- 172.21.15.84:0/3222448013 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033003f030 con 0x7f0334100750 2024-05-22T23:06:13.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.694+0000 7f032b7fe640 1 -- 172.21.15.84:0/3222448013 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0330003ad0 con 0x7f0334100750 2024-05-22T23:06:13.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.694+0000 7f033c7db640 1 -- 172.21.15.84:0/3222448013 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03341a67c0 con 0x7f0334100750 2024-05-22T23:06:13.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.694+0000 7f032b7fe640 1 -- 172.21.15.84:0/3222448013 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330035070 con 0x7f0334100750 2024-05-22T23:06:13.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.694+0000 7f033c7db640 1 -- 172.21.15.84:0/3222448013 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03341a6cb0 con 0x7f0334100750 2024-05-22T23:06:13.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.696+0000 7f032b7fe640 1 -- 172.21.15.84:0/3222448013 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0330042030 con 0x7f0334100750 2024-05-22T23:06:13.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.696+0000 7f033f5bd740 1 Processor -- start 2024-05-22T23:06:13.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.697+0000 7f033f5bd740 1 -- start start 2024-05-22T23:06:13.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.697+0000 7f032b7fe640 1 --2- 172.21.15.84:0/3222448013 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f03200a2d30 0x7f03200a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.697+0000 7f033f5bd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945eea5c50 0x55945eea6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.697+0000 7f033f5bd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55945eea6570 0x55945eeb0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.697+0000 7f033f5bd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eeb0c40 0x55945eeb3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.697+0000 7f033f5bd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55945ee09a40 con 0x55945eeb0c40 2024-05-22T23:06:13.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.697+0000 7f033f5bd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55945eda8440 con 0x55945eea6570 2024-05-22T23:06:13.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.697+0000 7f033f5bd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55945ee05a00 con 0x55945eea5c50 2024-05-22T23:06:13.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.697+0000 7f033ad78640 1 --2- 172.21.15.84:0/3222448013 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f03200a2d30 0x7f03200a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.697+0000 7f0339d76640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55945eea6570 0x55945eeb0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.697+0000 7f032b7fe640 1 -- 172.21.15.84:0/3222448013 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f03300ef990 con 0x7f0334100750 2024-05-22T23:06:13.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.697+0000 7f033a577640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945eea5c50 0x55945eea6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.697+0000 7f0339d76640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55945eea6570 0x55945eeb0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:48750/0 (socket says 172.21.15.84:48750) 2024-05-22T23:06:13.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.697+0000 7f0339d76640 1 -- 172.21.15.84:0/4276511580 learned_addr learned my addr 172.21.15.84:0/4276511580 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:13.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.697+0000 7f033ad78640 1 --2- 172.21.15.84:0/4276511580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eeb0c40 0x55945eeb3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.697+0000 7f0339d76640 1 -- 172.21.15.84:0/4276511580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945eea5c50 msgr2=0x55945eea6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:13.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.697+0000 7f0339d76640 1 --2- 172.21.15.84:0/4276511580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945eea5c50 0x55945eea6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.697+0000 7f0339d76640 1 -- 172.21.15.84:0/4276511580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eeb0c40 msgr2=0x55945eeb3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:13.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.697+0000 7f0339d76640 1 --2- 172.21.15.84:0/4276511580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eeb0c40 0x55945eeb3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.697+0000 7f0339d76640 1 -- 172.21.15.84:0/4276511580 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55945edb22d0 con 0x55945eea6570 2024-05-22T23:06:13.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.698+0000 7f033ad78640 1 --2- 172.21.15.84:0/3222448013 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f03200a2d30 0x7f03200a51f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f032c0077f0 tx=0x7f032c013ac0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:13.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.698+0000 7f0339d76640 1 --2- 172.21.15.84:0/4276511580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55945eea6570 0x55945eeb0700 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0324009c90 tx=0x7f0324011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=801aaff22fc200e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:13.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.698+0000 7f0328ff9640 1 -- 172.21.15.84:0/4276511580 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032401e070 con 0x55945eea6570 2024-05-22T23:06:13.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.698+0000 7f0328ff9640 1 -- 172.21.15.84:0/4276511580 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0324019b10 con 0x55945eea6570 2024-05-22T23:06:13.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.698+0000 7f0328ff9640 1 -- 172.21.15.84:0/4276511580 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032400e9f0 con 0x55945eea6570 2024-05-22T23:06:13.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.698+0000 7f033f5bd740 1 -- 172.21.15.84:0/4276511580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55945eea6570 msgr2=0x55945eeb0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:13.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.698+0000 7f033f5bd740 1 --2- 172.21.15.84:0/4276511580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55945eea6570 0x55945eeb0700 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0324009c90 tx=0x7f0324011810 comp rx=0 tx=0).stop 2024-05-22T23:06:13.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.699+0000 7f033f5bd740 1 -- 172.21.15.84:0/4276511580 shutdown_connections 2024-05-22T23:06:13.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.699+0000 7f033f5bd740 1 --2- 172.21.15.84:0/4276511580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eeb0c40 0x55945eeb3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.699+0000 7f033f5bd740 1 --2- 172.21.15.84:0/4276511580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55945eea6570 0x55945eeb0700 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.699+0000 7f033f5bd740 1 --2- 172.21.15.84:0/4276511580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945eea5c50 0x55945eea6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.699+0000 7f033f5bd740 1 -- 172.21.15.84:0/4276511580 >> 172.21.15.84:0/4276511580 conn(0x55945eea1f70 msgr2=0x55945eea2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:13.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.699+0000 7f033f5bd740 1 -- 172.21.15.84:0/4276511580 shutdown_connections 2024-05-22T23:06:13.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.699+0000 7f033f5bd740 1 -- 172.21.15.84:0/4276511580 wait complete. 2024-05-22T23:06:13.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.699+0000 7f033f5bd740 1 -- start start 2024-05-22T23:06:13.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.699+0000 7f033f5bd740 1 Processor -- start 2024-05-22T23:06:13.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.699+0000 7f033f5bd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55945eea5c50 0x55945eeadc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.699+0000 7f033f5bd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945eea6570 0x55945eeae140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.699+0000 7f033f5bd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eeb0c40 0x55945eeae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.699+0000 7f033f5bd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55945eeb3a30 con 0x55945eeb0c40 2024-05-22T23:06:13.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.699+0000 7f033f5bd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55945ed935f0 con 0x55945eea5c50 2024-05-22T23:06:13.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.699+0000 7f033f5bd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55945eeb3bb0 con 0x55945eea6570 2024-05-22T23:06:13.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.699+0000 7f033f5bd740 10 client.0 _ll_register_callbacks cb 0x55945ed8ecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:06:13.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.699+0000 7f033ad78640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eeb0c40 0x55945eeae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.699+0000 7f033ad78640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eeb0c40 0x55945eeae680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56178/0 (socket says 172.21.15.84:56178) 2024-05-22T23:06:13.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.699+0000 7f033a577640 1 --2- 172.21.15.84:0/21205650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55945eea5c50 0x55945eeadc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.699+0000 7f033ad78640 1 -- 172.21.15.84:0/21205650 learned_addr learned my addr 172.21.15.84:0/21205650 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:13.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.700+0000 7f0339d76640 1 --2- 172.21.15.84:0/21205650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945eea6570 0x55945eeae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.700+0000 7f033a577640 1 -- 172.21.15.84:0/21205650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945eea6570 msgr2=0x55945eeae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:13.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.700+0000 7f033a577640 1 --2- 172.21.15.84:0/21205650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945eea6570 0x55945eeae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.700+0000 7f033a577640 1 -- 172.21.15.84:0/21205650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eeb0c40 msgr2=0x55945eeae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:13.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.700+0000 7f033a577640 1 --2- 172.21.15.84:0/21205650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eeb0c40 0x55945eeae680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.700+0000 7f033a577640 1 -- 172.21.15.84:0/21205650 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55945ebe57e0 con 0x55945eea5c50 2024-05-22T23:06:13.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.700+0000 7f0339d76640 1 --2- 172.21.15.84:0/21205650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945eea6570 0x55945eeae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:06:13.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.700+0000 7f033ad78640 1 --2- 172.21.15.84:0/21205650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eeb0c40 0x55945eeae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:06:13.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.700+0000 7f033a577640 1 --2- 172.21.15.84:0/21205650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55945eea5c50 0x55945eeadc00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f03300ed0b0 tx=0x7f03300bb9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc437f32f59cac0b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:13.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.700+0000 7f031dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:06:13.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.700+0000 7f031dffb640 1 -- 172.21.15.84:0/21205650 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330049040 con 0x55945eea5c50 2024-05-22T23:06:13.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.700+0000 7f031dffb640 1 -- 172.21.15.84:0/21205650 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f0330003c80 con 0x55945eea5c50 2024-05-22T23:06:13.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.700+0000 7f033f5bd740 1 -- 172.21.15.84:0/21205650 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0324019b10 con 0x55945eea5c50 2024-05-22T23:06:13.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.700+0000 7f031dffb640 1 -- 172.21.15.84:0/21205650 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f03300c4b20 con 0x55945eea5c50 2024-05-22T23:06:13.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.700+0000 7f033f5bd740 10 client.25718 fetch_fsmap learned FSMap version 18 2024-05-22T23:06:13.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.700+0000 7f033f5bd740 1 -- 172.21.15.84:0/21205650 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55945ec45f70 con 0x55945eea5c50 2024-05-22T23:06:13.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.700+0000 7f031dffb640 1 -- 172.21.15.84:0/21205650 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03300d8080 con 0x55945eea5c50 2024-05-22T23:06:13.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.701+0000 7f031dffb640 1 -- 172.21.15.84:0/21205650 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f0330031bd0 con 0x55945eea5c50 2024-05-22T23:06:13.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.701+0000 7f033f5bd740 10 client.25718 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:06:13.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.701+0000 7f033f5bd740 10 client.25718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:06:13.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.701+0000 7f033f5bd740 20 client.25718 populate_metadata read hostname 'smithi084' 2024-05-22T23:06:13.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.701+0000 7f033f5bd740 1 --2- 172.21.15.84:0/21205650 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55945eec0fe0 0x55945eec3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.701+0000 7f033f5bd740 4 client.25718 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:06:13.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.701+0000 7f033f5bd740 1 -- 172.21.15.84:0/21205650 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55945ee1d460 con 0x55945eec0fe0 2024-05-22T23:06:13.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.701+0000 7f0339d76640 1 --2- 172.21.15.84:0/21205650 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55945eec0fe0 0x55945eec3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.701+0000 7f031dffb640 10 client.25718 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:13.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.701+0000 7f0339d76640 1 --2- 172.21.15.84:0/21205650 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55945eec0fe0 0x55945eec3450 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be22703115609308 server_cookie=eb8ee6783741ca72 in_seq=0 out_seq=0 2024-05-22T23:06:13.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.702+0000 7f031dffb640 1 -- 172.21.15.84:0/21205650 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55945ee1d460 con 0x55945eec0fe0 2024-05-22T23:06:13.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.702+0000 7f031dffb640 10 client.25718 handle_command_reply: tid=0 2024-05-22T23:06:13.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.702+0000 7f033f5bd740 1 client.25718 shutdown 2024-05-22T23:06:13.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.702+0000 7f033f5bd740 1 -- 172.21.15.84:0/21205650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55945eea5c50 msgr2=0x55945eeadc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:13.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.702+0000 7f033f5bd740 1 --2- 172.21.15.84:0/21205650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55945eea5c50 0x55945eeadc00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f03300ed0b0 tx=0x7f03300bb9f0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.702+0000 7f033f5bd740 1 -- 172.21.15.84:0/21205650 shutdown_connections 2024-05-22T23:06:13.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.702+0000 7f033f5bd740 1 --2- 172.21.15.84:0/21205650 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55945eec0fe0 0x55945eec3450 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.702+0000 7f033f5bd740 1 --2- 172.21.15.84:0/21205650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eeb0c40 0x55945eeae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.702+0000 7f031dffb640 0 client.25718 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:13.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.702+0000 7f033f5bd740 1 --2- 172.21.15.84:0/21205650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945eea6570 0x55945eeae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.702+0000 7f033f5bd740 1 --2- 172.21.15.84:0/21205650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55945eea5c50 0x55945eeadc00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.702+0000 7f033f5bd740 1 -- 172.21.15.84:0/21205650 >> 172.21.15.84:0/21205650 conn(0x55945eea1f70 msgr2=0x55945eea7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:13.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.702+0000 7f033f5bd740 1 -- 172.21.15.84:0/21205650 shutdown_connections 2024-05-22T23:06:13.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.702+0000 7f033f5bd740 1 -- 172.21.15.84:0/21205650 wait complete. 2024-05-22T23:06:13.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.702+0000 7f033f5bd740 20 client.25718 trim_cache size 0 max 16384 2024-05-22T23:06:13.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.721+0000 7f033f5bd740 1 Processor -- start 2024-05-22T23:06:13.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.721+0000 7f033f5bd740 1 -- start start 2024-05-22T23:06:13.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.721+0000 7f033f5bd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945eea5c50 0x55945eeae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.721+0000 7f033f5bd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eea6570 0x55945eec8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.721+0000 7f033f5bd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55945eeb0c40 0x55945eec9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.721+0000 7f033f5bd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f03341ad6b0 con 0x55945eea6570 2024-05-22T23:06:13.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.721+0000 7f033f5bd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f03341aff30 con 0x55945eeb0c40 2024-05-22T23:06:13.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.721+0000 7f033f5bd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f03341add20 con 0x55945eea5c50 2024-05-22T23:06:13.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.722+0000 7f0339d76640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eea6570 0x55945eec8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.722+0000 7f0339d76640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eea6570 0x55945eec8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56188/0 (socket says 172.21.15.84:56188) 2024-05-22T23:06:13.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.722+0000 7f0339d76640 1 -- 172.21.15.84:0/477983694 learned_addr learned my addr 172.21.15.84:0/477983694 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:13.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.722+0000 7f033ad78640 1 --2- 172.21.15.84:0/477983694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55945eeb0c40 0x55945eec9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.722+0000 7f033a577640 1 --2- 172.21.15.84:0/477983694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945eea5c50 0x55945eeae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.722+0000 7f0339d76640 1 -- 172.21.15.84:0/477983694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945eea5c50 msgr2=0x55945eeae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:13.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.722+0000 7f0339d76640 1 --2- 172.21.15.84:0/477983694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945eea5c50 0x55945eeae4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.722+0000 7f0339d76640 1 -- 172.21.15.84:0/477983694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55945eeb0c40 msgr2=0x55945eec9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:13.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.722+0000 7f0339d76640 1 --2- 172.21.15.84:0/477983694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55945eeb0c40 0x55945eec9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.722+0000 7f0339d76640 1 -- 172.21.15.84:0/477983694 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55945eadd170 con 0x55945eea6570 2024-05-22T23:06:13.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.722+0000 7f0339d76640 1 --2- 172.21.15.84:0/477983694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eea6570 0x55945eec8d20 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0324009230 tx=0x7f032400e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357eff07733b21a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:13.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.722+0000 7f031ffff640 1 -- 172.21.15.84:0/477983694 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032402d070 con 0x55945eea6570 2024-05-22T23:06:13.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.722+0000 7f031ffff640 1 -- 172.21.15.84:0/477983694 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03240198c0 con 0x55945eea6570 2024-05-22T23:06:13.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.722+0000 7f031ffff640 1 -- 172.21.15.84:0/477983694 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324029e60 con 0x55945eea6570 2024-05-22T23:06:13.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.723+0000 7f033f5bd740 1 -- 172.21.15.84:0/477983694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eea6570 msgr2=0x55945eec8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:13.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.723+0000 7f033f5bd740 1 --2- 172.21.15.84:0/477983694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eea6570 0x55945eec8d20 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0324009230 tx=0x7f032400e670 comp rx=0 tx=0).stop 2024-05-22T23:06:13.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.723+0000 7f033f5bd740 1 -- 172.21.15.84:0/477983694 shutdown_connections 2024-05-22T23:06:13.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.723+0000 7f033f5bd740 1 --2- 172.21.15.84:0/477983694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55945eeb0c40 0x55945eec9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.723+0000 7f033f5bd740 1 --2- 172.21.15.84:0/477983694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eea6570 0x55945eec8d20 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.723+0000 7f033f5bd740 1 --2- 172.21.15.84:0/477983694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945eea5c50 0x55945eeae4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.723+0000 7f033f5bd740 1 -- 172.21.15.84:0/477983694 >> 172.21.15.84:0/477983694 conn(0x55945eea1f70 msgr2=0x55945eea0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:13.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.723+0000 7f033f5bd740 1 -- 172.21.15.84:0/477983694 shutdown_connections 2024-05-22T23:06:13.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.723+0000 7f033f5bd740 1 -- 172.21.15.84:0/477983694 wait complete. 2024-05-22T23:06:13.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.723+0000 7f033f5bd740 1 -- start start 2024-05-22T23:06:13.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.723+0000 7f033f5bd740 1 Processor -- start 2024-05-22T23:06:13.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.724+0000 7f033f5bd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eea5c50 0x55945eeb8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.724+0000 7f033f5bd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945eea6570 0x55945eeb9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.724+0000 7f033f5bd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55945eeb0c40 0x55945eeb5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.724+0000 7f033f5bd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f03341ae3f0 con 0x55945eea5c50 2024-05-22T23:06:13.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.724+0000 7f033f5bd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f03341af860 con 0x55945eeb0c40 2024-05-22T23:06:13.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.724+0000 7f033f5bd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f03341aeac0 con 0x55945eea6570 2024-05-22T23:06:13.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.724+0000 7f033f5bd740 10 client.0 _ll_register_callbacks cb 0x7f0330009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:06:13.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.724+0000 7f033a577640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eea5c50 0x55945eeb8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.724+0000 7f033a577640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eea5c50 0x55945eeb8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56194/0 (socket says 172.21.15.84:56194) 2024-05-22T23:06:13.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.724+0000 7f033a577640 1 -- 172.21.15.84:0/1663789451 learned_addr learned my addr 172.21.15.84:0/1663789451 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:13.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.724+0000 7f033ad78640 1 --2- 172.21.15.84:0/1663789451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55945eeb0c40 0x55945eeb5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.724+0000 7f0339d76640 1 --2- 172.21.15.84:0/1663789451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945eea6570 0x55945eeb9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.724+0000 7f033a577640 1 -- 172.21.15.84:0/1663789451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945eea6570 msgr2=0x55945eeb9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:13.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.724+0000 7f033a577640 1 --2- 172.21.15.84:0/1663789451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945eea6570 0x55945eeb9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.724+0000 7f033a577640 1 -- 172.21.15.84:0/1663789451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55945eeb0c40 msgr2=0x55945eeb5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:13.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.724+0000 7f033a577640 1 --2- 172.21.15.84:0/1663789451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55945eeb0c40 0x55945eeb5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.724+0000 7f033a577640 1 -- 172.21.15.84:0/1663789451 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55945edd2930 con 0x55945eea5c50 2024-05-22T23:06:13.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.724+0000 7f033a577640 1 --2- 172.21.15.84:0/1663789451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eea5c50 0x55945eeb8e00 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f03300bb900 tx=0x7f0330009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d989435e1f2a4472 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:13.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.724+0000 7f031dffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:06:13.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.724+0000 7f031dffb640 1 -- 172.21.15.84:0/1663789451 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330049070 con 0x55945eea5c50 2024-05-22T23:06:13.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.724+0000 7f031dffb640 1 -- 172.21.15.84:0/1663789451 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f03300ed360 con 0x55945eea5c50 2024-05-22T23:06:13.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.724+0000 7f033f5bd740 1 -- 172.21.15.84:0/1663789451 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03240198c0 con 0x55945eea5c50 2024-05-22T23:06:13.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.724+0000 7f031dffb640 1 -- 172.21.15.84:0/1663789451 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f03300a5120 con 0x55945eea5c50 2024-05-22T23:06:13.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.725+0000 7f033f5bd740 10 client.16152 fetch_fsmap learned FSMap version 18 2024-05-22T23:06:13.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.725+0000 7f033f5bd740 1 -- 172.21.15.84:0/1663789451 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55945ef280c0 con 0x55945eea5c50 2024-05-22T23:06:13.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.725+0000 7f031dffb640 1 -- 172.21.15.84:0/1663789451 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03240198c0 con 0x55945eea5c50 2024-05-22T23:06:13.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.725+0000 7f031dffb640 1 -- 172.21.15.84:0/1663789451 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f03300c4720 con 0x55945eea5c50 2024-05-22T23:06:13.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.725+0000 7f033f5bd740 10 client.16152 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:06:13.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.725+0000 7f033f5bd740 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:06:13.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.725+0000 7f033f5bd740 20 client.16152 populate_metadata read hostname 'smithi084' 2024-05-22T23:06:13.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.725+0000 7f033f5bd740 1 --2- 172.21.15.84:0/1663789451 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55945ef286a0 0x55945ef2aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.725+0000 7f033f5bd740 4 client.16152 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-22T23:06:13.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.725+0000 7f033f5bd740 1 -- 172.21.15.84:0/1663789451 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55945ee17170 con 0x55945ef286a0 2024-05-22T23:06:13.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.725+0000 7f0339d76640 1 --2- 172.21.15.84:0/1663789451 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55945ef286a0 0x55945ef2aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.725+0000 7f031dffb640 10 client.16152 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:13.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.725+0000 7f0339d76640 1 --2- 172.21.15.84:0/1663789451 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55945ef286a0 0x55945ef2aa90 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3899ae3f00c8952 server_cookie=34b163a48144b466 in_seq=0 out_seq=0 2024-05-22T23:06:13.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.728+0000 7f031dffb640 1 -- 172.21.15.84:0/1663789451 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55945ee17170 con 0x55945ef286a0 2024-05-22T23:06:13.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.728+0000 7f031dffb640 10 client.16152 handle_command_reply: tid=0 2024-05-22T23:06:13.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.728+0000 7f033f5bd740 1 client.16152 shutdown 2024-05-22T23:06:13.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.728+0000 7f033f5bd740 1 -- 172.21.15.84:0/1663789451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eea5c50 msgr2=0x55945eeb8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:13.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.728+0000 7f033f5bd740 1 --2- 172.21.15.84:0/1663789451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eea5c50 0x55945eeb8e00 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f03300bb900 tx=0x7f0330009680 comp rx=0 tx=0).stop 2024-05-22T23:06:13.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.728+0000 7f033f5bd740 1 -- 172.21.15.84:0/1663789451 shutdown_connections 2024-05-22T23:06:13.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.728+0000 7f033f5bd740 1 --2- 172.21.15.84:0/1663789451 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55945ef286a0 0x55945ef2aa90 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.728+0000 7f033f5bd740 1 --2- 172.21.15.84:0/1663789451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55945eeb0c40 0x55945eeb5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.728+0000 7f033f5bd740 1 --2- 172.21.15.84:0/1663789451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55945eea6570 0x55945eeb9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.728+0000 7f031dffb640 0 client.16152 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:13.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.728+0000 7f033f5bd740 1 --2- 172.21.15.84:0/1663789451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55945eea5c50 0x55945eeb8e00 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.728+0000 7f033f5bd740 1 -- 172.21.15.84:0/1663789451 >> 172.21.15.84:0/1663789451 conn(0x55945eea1f70 msgr2=0x55945eea11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:13.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.728+0000 7f033f5bd740 1 -- 172.21.15.84:0/1663789451 shutdown_connections 2024-05-22T23:06:13.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.728+0000 7f033f5bd740 1 -- 172.21.15.84:0/1663789451 wait complete. 2024-05-22T23:06:13.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.728+0000 7f033f5bd740 20 client.16152 trim_cache size 0 max 16384 2024-05-22T23:06:13.752 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:06:13.753 INFO:teuthology.orchestra.run.smithi084.stdout: "return_code": 0, 2024-05-22T23:06:13.753 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_tag": "48d441e5-8481-4097-a09d-6a29d8e6f2a1", 2024-05-22T23:06:13.753 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": "asynchronous" 2024-05-22T23:06:13.753 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:06:13.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.729+0000 7f033c7db640 1 -- 172.21.15.84:0/3222448013 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f03200a2d30 msgr2=0x7f03200a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:13.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.729+0000 7f033c7db640 1 --2- 172.21.15.84:0/3222448013 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f03200a2d30 0x7f03200a51f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f032c0077f0 tx=0x7f032c013ac0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.729+0000 7f033c7db640 1 -- 172.21.15.84:0/3222448013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0334100750 msgr2=0x7f03340fd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:13.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.729+0000 7f033c7db640 1 --2- 172.21.15.84:0/3222448013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0334100750 0x7f03340fd780 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0330009230 tx=0x7f0330004270 comp rx=0 tx=0).stop 2024-05-22T23:06:13.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.729+0000 7f033c7db640 1 -- 172.21.15.84:0/3222448013 shutdown_connections 2024-05-22T23:06:13.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.729+0000 7f033c7db640 1 --2- 172.21.15.84:0/3222448013 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f03200a2d30 0x7f03200a51f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.729+0000 7f033c7db640 1 --2- 172.21.15.84:0/3222448013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0334106760 0x7f03340fe200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.729+0000 7f033c7db640 1 --2- 172.21.15.84:0/3222448013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0334101110 0x7f03340fdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.729+0000 7f033c7db640 1 --2- 172.21.15.84:0/3222448013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0334100750 0x7f03340fd780 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.729+0000 7f033c7db640 1 -- 172.21.15.84:0/3222448013 >> 172.21.15.84:0/3222448013 conn(0x7f03340777d0 msgr2=0x7f03340ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:13.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.729+0000 7f033c7db640 1 -- 172.21.15.84:0/3222448013 shutdown_connections 2024-05-22T23:06:13.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.729+0000 7f033c7db640 1 -- 172.21.15.84:0/3222448013 wait complete. 2024-05-22T23:06:13.755 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:13 smithi084 ceph-mon[32186]: pgmap v134: 97 pgs: 97 active+clean; 265 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.8 MiB/s wr, 387 op/s 2024-05-22T23:06:13.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:06:13.756 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:06:13.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:13 smithi096 ceph-mon[36392]: pgmap v134: 97 pgs: 97 active+clean; 265 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.8 MiB/s wr, 387 op/s 2024-05-22T23:06:13.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.879+0000 7f91beda2640 1 -- 172.21.15.84:0/1766710787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91b8106760 msgr2=0x7f91b8106b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:13.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.879+0000 7f91beda2640 1 --2- 172.21.15.84:0/1766710787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91b8106760 0x7f91b8106b40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f91b4009930 tx=0x7f91b402a420 comp rx=0 tx=0).stop 2024-05-22T23:06:13.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.879+0000 7f91beda2640 1 -- 172.21.15.84:0/1766710787 shutdown_connections 2024-05-22T23:06:13.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.879+0000 7f91beda2640 1 --2- 172.21.15.84:0/1766710787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91b8101110 0x7f91b810d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.879+0000 7f91beda2640 1 --2- 172.21.15.84:0/1766710787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b8100750 0x7f91b8100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.879+0000 7f91beda2640 1 --2- 172.21.15.84:0/1766710787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91b8106760 0x7f91b8106b40 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.879+0000 7f91beda2640 1 -- 172.21.15.84:0/1766710787 >> 172.21.15.84:0/1766710787 conn(0x7f91b8076af0 msgr2=0x7f91b8076f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:13.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.879+0000 7f91beda2640 1 -- 172.21.15.84:0/1766710787 shutdown_connections 2024-05-22T23:06:13.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.879+0000 7f91beda2640 1 -- 172.21.15.84:0/1766710787 wait complete. 2024-05-22T23:06:13.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.880+0000 7f91beda2640 1 Processor -- start 2024-05-22T23:06:13.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.880+0000 7f91beda2640 1 -- start start 2024-05-22T23:06:13.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.880+0000 7f91beda2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91b8100750 0x7f91b8198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.880+0000 7f91beda2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b8101110 0x7f91b81986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.880+0000 7f91beda2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91b8106760 0x7f91b819ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.880+0000 7f91beda2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f91b8110bc0 con 0x7f91b8106760 2024-05-22T23:06:13.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.880+0000 7f91beda2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f91b8110a40 con 0x7f91b8100750 2024-05-22T23:06:13.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.880+0000 7f91beda2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f91b8110d40 con 0x7f91b8101110 2024-05-22T23:06:13.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.880+0000 7f91bd33f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91b8106760 0x7f91b819ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.880+0000 7f91bcb3e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91b8100750 0x7f91b8198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.880+0000 7f91bcb3e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91b8100750 0x7f91b8198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:48782/0 (socket says 172.21.15.84:48782) 2024-05-22T23:06:13.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.880+0000 7f91bd33f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91b8106760 0x7f91b819ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56218/0 (socket says 172.21.15.84:56218) 2024-05-22T23:06:13.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.880+0000 7f91bcb3e640 1 -- 172.21.15.84:0/316406946 learned_addr learned my addr 172.21.15.84:0/316406946 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:13.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.881+0000 7f91affff640 1 --2- 172.21.15.84:0/316406946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b8101110 0x7f91b81986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.881+0000 7f91bd33f640 1 -- 172.21.15.84:0/316406946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b8101110 msgr2=0x7f91b81986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:13.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.881+0000 7f91bd33f640 1 --2- 172.21.15.84:0/316406946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b8101110 0x7f91b81986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.881+0000 7f91bd33f640 1 -- 172.21.15.84:0/316406946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91b8100750 msgr2=0x7f91b8198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:13.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.881+0000 7f91bd33f640 1 --2- 172.21.15.84:0/316406946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91b8100750 0x7f91b8198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.881+0000 7f91bd33f640 1 -- 172.21.15.84:0/316406946 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b819d150 con 0x7f91b8106760 2024-05-22T23:06:13.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.881+0000 7f91bcb3e640 1 --2- 172.21.15.84:0/316406946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91b8100750 0x7f91b8198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:06:13.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.881+0000 7f91bd33f640 1 --2- 172.21.15.84:0/316406946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91b8106760 0x7f91b819ca20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009fd0 tx=0x7f91b000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:13.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.881+0000 7f91adffb640 1 -- 172.21.15.84:0/316406946 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b000ce20 con 0x7f91b8106760 2024-05-22T23:06:13.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.881+0000 7f91beda2640 1 -- 172.21.15.84:0/316406946 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b819d440 con 0x7f91b8106760 2024-05-22T23:06:13.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.881+0000 7f91adffb640 1 -- 172.21.15.84:0/316406946 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91b0005020 con 0x7f91b8106760 2024-05-22T23:06:13.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.881+0000 7f91adffb640 1 -- 172.21.15.84:0/316406946 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b0010720 con 0x7f91b8106760 2024-05-22T23:06:13.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.881+0000 7f91beda2640 1 -- 172.21.15.84:0/316406946 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b81a4cd0 con 0x7f91b8106760 2024-05-22T23:06:13.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.884+0000 7f91adffb640 1 -- 172.21.15.84:0/316406946 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f91b0005540 con 0x7f91b8106760 2024-05-22T23:06:13.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.885+0000 7f91c1b93740 1 Processor -- start 2024-05-22T23:06:13.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.885+0000 7f91c1b93740 1 -- start start 2024-05-22T23:06:13.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.885+0000 7f91adffb640 1 --2- 172.21.15.84:0/316406946 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f91a00a2d00 0x7f91a00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.885+0000 7f91c1b93740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55febfc094a0 0x55febfc077e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.885+0000 7f91c1b93740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc07db0 0x55febfc08230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.885+0000 7f91adffb640 1 -- 172.21.15.84:0/316406946 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f91b00cfb20 con 0x7f91b8106760 2024-05-22T23:06:13.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.885+0000 7f91c1b93740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55febfc12b30 0x55febfc14f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.885+0000 7f91c1b93740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55febfb6bdb0 con 0x55febfc07db0 2024-05-22T23:06:13.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.885+0000 7f91c1b93740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55febfb0a360 con 0x55febfc12b30 2024-05-22T23:06:13.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.885+0000 7f91c1b93740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55febfb67d70 con 0x55febfc094a0 2024-05-22T23:06:13.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.885+0000 7f91affff640 1 --2- 172.21.15.84:0/316406946 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f91a00a2d00 0x7f91a00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.885+0000 7f91bcb3e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55febfc094a0 0x55febfc077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.885+0000 7f91bcb3e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55febfc094a0 0x55febfc077e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:34346/0 (socket says 172.21.15.84:34346) 2024-05-22T23:06:13.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.885+0000 7f91bcb3e640 1 -- 172.21.15.84:0/1682394250 learned_addr learned my addr 172.21.15.84:0/1682394250 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:13.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.885+0000 7f91bd33f640 1 --2- 172.21.15.84:0/1682394250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55febfc12b30 0x55febfc14f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.885+0000 7f91affff640 1 --2- 172.21.15.84:0/1682394250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc07db0 0x55febfc08230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.885+0000 7f91bcb3e640 1 -- 172.21.15.84:0/1682394250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55febfc12b30 msgr2=0x55febfc14f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:13.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.885+0000 7f91bcb3e640 1 --2- 172.21.15.84:0/1682394250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55febfc12b30 0x55febfc14f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.885+0000 7f91bcb3e640 1 -- 172.21.15.84:0/1682394250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc07db0 msgr2=0x55febfc08230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:13.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.885+0000 7f91bcb3e640 1 --2- 172.21.15.84:0/1682394250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc07db0 0x55febfc08230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.885+0000 7f91bcb3e640 1 -- 172.21.15.84:0/1682394250 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55febfb2da60 con 0x55febfc094a0 2024-05-22T23:06:13.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.885+0000 7f91affff640 1 --2- 172.21.15.84:0/316406946 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f91a00a2d00 0x7f91a00a51c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f91b8104e40 tx=0x7f91a8013830 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:13.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.885+0000 7f91bd33f640 1 --2- 172.21.15.84:0/1682394250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55febfc12b30 0x55febfc14f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:06:13.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.886+0000 7f91bcb3e640 1 --2- 172.21.15.84:0/1682394250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55febfc094a0 0x55febfc077e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f91b4005e10 tx=0x7f91b4005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c20f198bd28e8cc server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:13.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.886+0000 7f919effd640 1 -- 172.21.15.84:0/1682394250 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b4040070 con 0x55febfc094a0 2024-05-22T23:06:13.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.886+0000 7f919effd640 1 -- 172.21.15.84:0/1682394250 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91b4004760 con 0x55febfc094a0 2024-05-22T23:06:13.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.886+0000 7f919effd640 1 -- 172.21.15.84:0/1682394250 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b4035040 con 0x55febfc094a0 2024-05-22T23:06:13.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.886+0000 7f91c1b93740 1 -- 172.21.15.84:0/1682394250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55febfc094a0 msgr2=0x55febfc077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:13.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.886+0000 7f91c1b93740 1 --2- 172.21.15.84:0/1682394250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55febfc094a0 0x55febfc077e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f91b4005e10 tx=0x7f91b4005e40 comp rx=0 tx=0).stop 2024-05-22T23:06:13.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.887+0000 7f91c1b93740 1 -- 172.21.15.84:0/1682394250 shutdown_connections 2024-05-22T23:06:13.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.887+0000 7f91c1b93740 1 --2- 172.21.15.84:0/1682394250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55febfc12b30 0x55febfc14f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.887+0000 7f91c1b93740 1 --2- 172.21.15.84:0/1682394250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc07db0 0x55febfc08230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.887+0000 7f91c1b93740 1 --2- 172.21.15.84:0/1682394250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55febfc094a0 0x55febfc077e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.887+0000 7f91c1b93740 1 -- 172.21.15.84:0/1682394250 >> 172.21.15.84:0/1682394250 conn(0x55febfbd08e0 msgr2=0x55febfc04310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:13.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.887+0000 7f91c1b93740 1 -- 172.21.15.84:0/1682394250 shutdown_connections 2024-05-22T23:06:13.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.887+0000 7f91c1b93740 1 -- 172.21.15.84:0/1682394250 wait complete. 2024-05-22T23:06:13.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.887+0000 7f91c1b93740 1 -- start start 2024-05-22T23:06:13.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.887+0000 7f91c1b93740 1 Processor -- start 2024-05-22T23:06:13.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.887+0000 7f91c1b93740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55febfc07db0 0x55febfc0fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.887+0000 7f91c1b93740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc094a0 0x55febfc0ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.887+0000 7f91c1b93740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55febfc12b30 0x55febfc104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.887+0000 7f91c1b93740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55febfc15960 con 0x55febfc094a0 2024-05-22T23:06:13.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.887+0000 7f91c1b93740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55febfb260f0 con 0x55febfc12b30 2024-05-22T23:06:13.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.887+0000 7f91c1b93740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55febfc15ae0 con 0x55febfc07db0 2024-05-22T23:06:13.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.887+0000 7f91c1b93740 10 client.0 _ll_register_callbacks cb 0x55febfb2af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:06:13.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.887+0000 7f91affff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc094a0 0x55febfc0ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.888+0000 7f91affff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc094a0 0x55febfc0ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56244/0 (socket says 172.21.15.84:56244) 2024-05-22T23:06:13.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.888+0000 7f91affff640 1 -- 172.21.15.84:0/1662728000 learned_addr learned my addr 172.21.15.84:0/1662728000 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:13.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.888+0000 7f91bd33f640 1 --2- 172.21.15.84:0/1662728000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55febfc12b30 0x55febfc104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.888+0000 7f91bcb3e640 1 --2- 172.21.15.84:0/1662728000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55febfc07db0 0x55febfc0fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.888+0000 7f91affff640 1 -- 172.21.15.84:0/1662728000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55febfc07db0 msgr2=0x55febfc0fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:13.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.888+0000 7f91affff640 1 --2- 172.21.15.84:0/1662728000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55febfc07db0 0x55febfc0fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.888+0000 7f91affff640 1 -- 172.21.15.84:0/1662728000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55febfc12b30 msgr2=0x55febfc104a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:13.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.888+0000 7f91affff640 1 --2- 172.21.15.84:0/1662728000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55febfc12b30 0x55febfc104a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.888+0000 7f91affff640 1 -- 172.21.15.84:0/1662728000 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55febf9497d0 con 0x55febfc094a0 2024-05-22T23:06:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.888+0000 7f91affff640 1 --2- 172.21.15.84:0/1662728000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc094a0 0x55febfc0ff60 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f91a800f8a0 tx=0x7f91a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4cb9be1b3c4767 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.888+0000 7f9197fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:06:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.888+0000 7f9197fff640 1 -- 172.21.15.84:0/1662728000 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a8024070 con 0x55febfc094a0 2024-05-22T23:06:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.888+0000 7f9197fff640 1 -- 172.21.15.84:0/1662728000 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f91a8010cc0 con 0x55febfc094a0 2024-05-22T23:06:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.888+0000 7f91c1b93740 1 -- 172.21.15.84:0/1662728000 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91b4004760 con 0x55febfc094a0 2024-05-22T23:06:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.888+0000 7f9197fff640 1 -- 172.21.15.84:0/1662728000 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f91a8005020 con 0x55febfc094a0 2024-05-22T23:06:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.888+0000 7f91c1b93740 10 client.16164 fetch_fsmap learned FSMap version 18 2024-05-22T23:06:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.888+0000 7f91c1b93740 1 -- 172.21.15.84:0/1662728000 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55febf87f3f0 con 0x55febfc094a0 2024-05-22T23:06:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.888+0000 7f9197fff640 1 -- 172.21.15.84:0/1662728000 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91b4004760 con 0x55febfc094a0 2024-05-22T23:06:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.889+0000 7f9197fff640 1 -- 172.21.15.84:0/1662728000 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f91a801f070 con 0x55febfc094a0 2024-05-22T23:06:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.889+0000 7f91c1b93740 10 client.16164 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:06:13.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.889+0000 7f91c1b93740 10 client.16164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:06:13.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.889+0000 7f91c1b93740 20 client.16164 populate_metadata read hostname 'smithi084' 2024-05-22T23:06:13.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.889+0000 7f91c1b93740 1 --2- 172.21.15.84:0/1662728000 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55febfc27370 0x55febfc29790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.889+0000 7f91c1b93740 4 client.16164 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:06:13.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.889+0000 7f91c1b93740 1 -- 172.21.15.84:0/1662728000 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55febfb794e0 con 0x55febfc27370 2024-05-22T23:06:13.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.889+0000 7f91bcb3e640 1 --2- 172.21.15.84:0/1662728000 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55febfc27370 0x55febfc29790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.889+0000 7f91bcb3e640 1 --2- 172.21.15.84:0/1662728000 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55febfc27370 0x55febfc29790 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e349ab966636fd2 server_cookie=35101ced15765d3c in_seq=0 out_seq=0 2024-05-22T23:06:13.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.889+0000 7f9197fff640 10 client.16164 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:13.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.890+0000 7f9197fff640 1 -- 172.21.15.84:0/1662728000 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55febfb794e0 con 0x55febfc27370 2024-05-22T23:06:13.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.890+0000 7f9197fff640 10 client.16164 handle_command_reply: tid=0 2024-05-22T23:06:13.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.890+0000 7f91c1b93740 1 client.16164 shutdown 2024-05-22T23:06:13.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.890+0000 7f91c1b93740 1 -- 172.21.15.84:0/1662728000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc094a0 msgr2=0x55febfc0ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:13.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.890+0000 7f91c1b93740 1 --2- 172.21.15.84:0/1662728000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc094a0 0x55febfc0ff60 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f91a800f8a0 tx=0x7f91a8004730 comp rx=0 tx=0).stop 2024-05-22T23:06:13.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.891+0000 7f91c1b93740 1 -- 172.21.15.84:0/1662728000 shutdown_connections 2024-05-22T23:06:13.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.891+0000 7f91c1b93740 1 --2- 172.21.15.84:0/1662728000 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55febfc27370 0x55febfc29790 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.891+0000 7f91c1b93740 1 --2- 172.21.15.84:0/1662728000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55febfc12b30 0x55febfc104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.891+0000 7f91c1b93740 1 --2- 172.21.15.84:0/1662728000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc094a0 0x55febfc0ff60 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.891+0000 7f91c1b93740 1 --2- 172.21.15.84:0/1662728000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55febfc07db0 0x55febfc0fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.891+0000 7f9197fff640 0 client.16164 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:13.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.891+0000 7f91c1b93740 1 -- 172.21.15.84:0/1662728000 >> 172.21.15.84:0/1662728000 conn(0x55febfbd08e0 msgr2=0x55febfc0a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:13.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.891+0000 7f91c1b93740 1 -- 172.21.15.84:0/1662728000 shutdown_connections 2024-05-22T23:06:13.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.891+0000 7f91c1b93740 1 -- 172.21.15.84:0/1662728000 wait complete. 2024-05-22T23:06:13.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.891+0000 7f91c1b93740 20 client.16164 trim_cache size 0 max 16384 2024-05-22T23:06:13.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.908+0000 7f91c1b93740 1 Processor -- start 2024-05-22T23:06:13.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.908+0000 7f91c1b93740 1 -- start start 2024-05-22T23:06:13.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.908+0000 7f91c1b93740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55febfc07db0 0x55febfc100f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.908+0000 7f91c1b93740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc094a0 0x55febfc2f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.908+0000 7f91c1b93740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55febfc12b30 0x55febfc2f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.908+0000 7f91c1b93740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f91b81a9420 con 0x55febfc094a0 2024-05-22T23:06:13.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.908+0000 7f91c1b93740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f91b81abca0 con 0x55febfc12b30 2024-05-22T23:06:13.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.908+0000 7f91c1b93740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f91b81a9a90 con 0x55febfc07db0 2024-05-22T23:06:13.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.909+0000 7f91affff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc094a0 0x55febfc2f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.909+0000 7f91affff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc094a0 0x55febfc2f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56260/0 (socket says 172.21.15.84:56260) 2024-05-22T23:06:13.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.909+0000 7f91affff640 1 -- 172.21.15.84:0/1875753856 learned_addr learned my addr 172.21.15.84:0/1875753856 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:13.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.909+0000 7f91bd33f640 1 --2- 172.21.15.84:0/1875753856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55febfc12b30 0x55febfc2f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.909+0000 7f91affff640 1 -- 172.21.15.84:0/1875753856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55febfc07db0 msgr2=0x55febfc100f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:13.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.909+0000 7f91bcb3e640 1 --2- 172.21.15.84:0/1875753856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55febfc07db0 0x55febfc100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.909+0000 7f91affff640 1 --2- 172.21.15.84:0/1875753856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55febfc07db0 0x55febfc100f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.909+0000 7f91affff640 1 -- 172.21.15.84:0/1875753856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55febfc12b30 msgr2=0x55febfc2f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:13.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.909+0000 7f91affff640 1 --2- 172.21.15.84:0/1875753856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55febfc12b30 0x55febfc2f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.909+0000 7f91affff640 1 -- 172.21.15.84:0/1875753856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55febf83f0f0 con 0x55febfc094a0 2024-05-22T23:06:13.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.909+0000 7f91affff640 1 --2- 172.21.15.84:0/1875753856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc094a0 0x55febfc2f060 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f91a8005b80 tx=0x7f91a80046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d993fe7fa0197ab server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:13.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.909+0000 7f919e7fc640 1 -- 172.21.15.84:0/1875753856 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a8024040 con 0x55febfc094a0 2024-05-22T23:06:13.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.909+0000 7f919e7fc640 1 -- 172.21.15.84:0/1875753856 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91a8009cf0 con 0x55febfc094a0 2024-05-22T23:06:13.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.909+0000 7f919e7fc640 1 -- 172.21.15.84:0/1875753856 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a8005420 con 0x55febfc094a0 2024-05-22T23:06:13.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.910+0000 7f91c1b93740 1 -- 172.21.15.84:0/1875753856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc094a0 msgr2=0x55febfc2f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:13.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.910+0000 7f91c1b93740 1 --2- 172.21.15.84:0/1875753856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc094a0 0x55febfc2f060 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f91a8005b80 tx=0x7f91a80046a0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.910+0000 7f91c1b93740 1 -- 172.21.15.84:0/1875753856 shutdown_connections 2024-05-22T23:06:13.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.910+0000 7f91c1b93740 1 --2- 172.21.15.84:0/1875753856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55febfc12b30 0x55febfc2f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.910+0000 7f91c1b93740 1 --2- 172.21.15.84:0/1875753856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc094a0 0x55febfc2f060 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.910+0000 7f91c1b93740 1 --2- 172.21.15.84:0/1875753856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55febfc07db0 0x55febfc100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.910+0000 7f91c1b93740 1 -- 172.21.15.84:0/1875753856 >> 172.21.15.84:0/1875753856 conn(0x55febfbd08e0 msgr2=0x55febfc0f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:13.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.910+0000 7f91c1b93740 1 -- 172.21.15.84:0/1875753856 shutdown_connections 2024-05-22T23:06:13.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.910+0000 7f91c1b93740 1 -- 172.21.15.84:0/1875753856 wait complete. 2024-05-22T23:06:13.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.910+0000 7f91c1b93740 1 -- start start 2024-05-22T23:06:13.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.910+0000 7f91c1b93740 1 Processor -- start 2024-05-22T23:06:13.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.911+0000 7f91c1b93740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55febfc07db0 0x55febfc0c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.911+0000 7f91c1b93740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55febfc094a0 0x55febfc0cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.911+0000 7f91c1b93740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc12b30 0x55febfc0d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.911+0000 7f91c1b93740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f91b81aa160 con 0x55febfc12b30 2024-05-22T23:06:13.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.911+0000 7f91c1b93740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f91b81ab5d0 con 0x55febfc094a0 2024-05-22T23:06:13.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.911+0000 7f91c1b93740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f91b81aa830 con 0x55febfc07db0 2024-05-22T23:06:13.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.911+0000 7f91c1b93740 10 client.0 _ll_register_callbacks cb 0x7f91a8005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:06:13.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.911+0000 7f91bd33f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc12b30 0x55febfc0d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.911+0000 7f91bcb3e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55febfc07db0 0x55febfc0c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.911+0000 7f91affff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55febfc094a0 0x55febfc0cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.911+0000 7f91bd33f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc12b30 0x55febfc0d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56268/0 (socket says 172.21.15.84:56268) 2024-05-22T23:06:13.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.911+0000 7f91bcb3e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55febfc07db0 0x55febfc0c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:34380/0 (socket says 172.21.15.84:34380) 2024-05-22T23:06:13.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.911+0000 7f91bd33f640 1 -- 172.21.15.84:0/3075666263 learned_addr learned my addr 172.21.15.84:0/3075666263 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:13.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.911+0000 7f91bcb3e640 1 -- 172.21.15.84:0/3075666263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55febfc094a0 msgr2=0x55febfc0cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:13.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.911+0000 7f91bcb3e640 1 --2- 172.21.15.84:0/3075666263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55febfc094a0 0x55febfc0cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.911+0000 7f91bcb3e640 1 -- 172.21.15.84:0/3075666263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc12b30 msgr2=0x55febfc0d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:13.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.911+0000 7f91bcb3e640 1 --2- 172.21.15.84:0/3075666263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc12b30 0x55febfc0d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.911+0000 7f91bcb3e640 1 -- 172.21.15.84:0/3075666263 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55febfb2afe0 con 0x55febfc07db0 2024-05-22T23:06:13.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.911+0000 7f91bd33f640 1 --2- 172.21.15.84:0/3075666263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc12b30 0x55febfc0d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:06:13.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.911+0000 7f91affff640 1 --2- 172.21.15.84:0/3075666263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55febfc094a0 0x55febfc0cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:06:13.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.911+0000 7f91bcb3e640 1 --2- 172.21.15.84:0/3075666263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55febfc07db0 0x55febfc0c860 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f91b4009900 tx=0x7f91b4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30e73ee5f80895f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:13.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.912+0000 7f9197fff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-22T23:06:13.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.912+0000 7f9197fff640 1 -- 172.21.15.84:0/3075666263 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b4040040 con 0x55febfc07db0 2024-05-22T23:06:13.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.912+0000 7f91c1b93740 1 -- 172.21.15.84:0/3075666263 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91a8009cf0 con 0x55febfc07db0 2024-05-22T23:06:13.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.912+0000 7f9197fff640 1 -- 172.21.15.84:0/3075666263 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f91b4003860 con 0x55febfc07db0 2024-05-22T23:06:13.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.912+0000 7f9197fff640 1 -- 172.21.15.84:0/3075666263 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f91b405f050 con 0x55febfc07db0 2024-05-22T23:06:13.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.912+0000 7f91c1b93740 10 client.25591 fetch_fsmap learned FSMap version 18 2024-05-22T23:06:13.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.912+0000 7f91c1b93740 1 -- 172.21.15.84:0/3075666263 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55febfafc1c0 con 0x55febfc07db0 2024-05-22T23:06:13.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.912+0000 7f9197fff640 1 -- 172.21.15.84:0/3075666263 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91a8009cf0 con 0x55febfc07db0 2024-05-22T23:06:13.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.912+0000 7f9197fff640 1 -- 172.21.15.84:0/3075666263 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f91b4031b90 con 0x55febfc07db0 2024-05-22T23:06:13.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.912+0000 7f91c1b93740 10 client.25591 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:06:13.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.912+0000 7f91c1b93740 10 client.25591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:06:13.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.912+0000 7f91c1b93740 20 client.25591 populate_metadata read hostname 'smithi084' 2024-05-22T23:06:13.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.912+0000 7f91c1b93740 1 --2- 172.21.15.84:0/3075666263 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55febfc8dd60 0x55febfc90150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:13.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.912+0000 7f91c1b93740 4 client.25591 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:06:13.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.912+0000 7f91c1b93740 1 -- 172.21.15.84:0/3075666263 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55febfb73830 con 0x55febfc8dd60 2024-05-22T23:06:13.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.913+0000 7f91affff640 1 --2- 172.21.15.84:0/3075666263 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55febfc8dd60 0x55febfc90150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:13.924 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:13 smithi204 ceph-mon[36211]: pgmap v134: 97 pgs: 97 active+clean; 265 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.8 MiB/s wr, 387 op/s 2024-05-22T23:06:13.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.913+0000 7f9197fff640 10 client.25591 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:13.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.913+0000 7f91affff640 1 --2- 172.21.15.84:0/3075666263 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55febfc8dd60 0x55febfc90150 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc21bbef44ef01f6 server_cookie=e452a81acbede37f in_seq=0 out_seq=0 2024-05-22T23:06:13.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.913+0000 7f9197fff640 1 -- 172.21.15.84:0/3075666263 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55febfb73830 con 0x55febfc8dd60 2024-05-22T23:06:13.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.913+0000 7f9197fff640 10 client.25591 handle_command_reply: tid=0 2024-05-22T23:06:13.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.913+0000 7f91c1b93740 1 client.25591 shutdown 2024-05-22T23:06:13.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.913+0000 7f91c1b93740 1 -- 172.21.15.84:0/3075666263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55febfc07db0 msgr2=0x55febfc0c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:13.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.913+0000 7f91c1b93740 1 --2- 172.21.15.84:0/3075666263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55febfc07db0 0x55febfc0c860 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f91b4009900 tx=0x7f91b4009260 comp rx=0 tx=0).stop 2024-05-22T23:06:13.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.913+0000 7f91c1b93740 1 -- 172.21.15.84:0/3075666263 shutdown_connections 2024-05-22T23:06:13.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.913+0000 7f91c1b93740 1 --2- 172.21.15.84:0/3075666263 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55febfc8dd60 0x55febfc90150 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.913+0000 7f91c1b93740 1 --2- 172.21.15.84:0/3075666263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55febfc12b30 0x55febfc0d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.913+0000 7f91c1b93740 1 --2- 172.21.15.84:0/3075666263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55febfc094a0 0x55febfc0cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.913+0000 7f9197fff640 0 client.25591 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:13.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.913+0000 7f91c1b93740 1 --2- 172.21.15.84:0/3075666263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55febfc07db0 0x55febfc0c860 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.913+0000 7f91c1b93740 1 -- 172.21.15.84:0/3075666263 >> 172.21.15.84:0/3075666263 conn(0x55febfbd08e0 msgr2=0x55febfc0a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:13.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.913+0000 7f91c1b93740 1 -- 172.21.15.84:0/3075666263 shutdown_connections 2024-05-22T23:06:13.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.913+0000 7f91c1b93740 1 -- 172.21.15.84:0/3075666263 wait complete. 2024-05-22T23:06:13.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.914+0000 7f91c1b93740 20 client.25591 trim_cache size 0 max 16384 2024-05-22T23:06:13.927 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:06:13.927 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-22T23:06:13.927 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": { 2024-05-22T23:06:13.927 INFO:teuthology.orchestra.run.smithi084.stdout: "48d441e5-8481-4097-a09d-6a29d8e6f2a1": { 2024-05-22T23:06:13.927 INFO:teuthology.orchestra.run.smithi084.stdout: "path": "/", 2024-05-22T23:06:13.927 INFO:teuthology.orchestra.run.smithi084.stdout: "tag": "48d441e5-8481-4097-a09d-6a29d8e6f2a1", 2024-05-22T23:06:13.927 INFO:teuthology.orchestra.run.smithi084.stdout: "options": "recursive,force" 2024-05-22T23:06:13.927 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:06:13.927 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:06:13.928 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:06:13.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.914+0000 7f91beda2640 1 -- 172.21.15.84:0/316406946 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f91a00a2d00 msgr2=0x7f91a00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:13.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.914+0000 7f91beda2640 1 --2- 172.21.15.84:0/316406946 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f91a00a2d00 0x7f91a00a51c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f91b8104e40 tx=0x7f91a8013830 comp rx=0 tx=0).stop 2024-05-22T23:06:13.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.914+0000 7f91beda2640 1 -- 172.21.15.84:0/316406946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91b8106760 msgr2=0x7f91b819ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:13.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.914+0000 7f91beda2640 1 --2- 172.21.15.84:0/316406946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91b8106760 0x7f91b819ca20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009fd0 tx=0x7f91b000ef10 comp rx=0 tx=0).stop 2024-05-22T23:06:13.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.914+0000 7f91beda2640 1 -- 172.21.15.84:0/316406946 shutdown_connections 2024-05-22T23:06:13.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.914+0000 7f91beda2640 1 --2- 172.21.15.84:0/316406946 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f91a00a2d00 0x7f91a00a51c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.914+0000 7f91beda2640 1 --2- 172.21.15.84:0/316406946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91b8106760 0x7f91b819ca20 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.914+0000 7f91beda2640 1 --2- 172.21.15.84:0/316406946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b8101110 0x7f91b81986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.914+0000 7f91beda2640 1 --2- 172.21.15.84:0/316406946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91b8100750 0x7f91b8198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:13.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.914+0000 7f91beda2640 1 -- 172.21.15.84:0/316406946 >> 172.21.15.84:0/316406946 conn(0x7f91b8076af0 msgr2=0x7f91b806a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:13.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.914+0000 7f91beda2640 1 -- 172.21.15.84:0/316406946 shutdown_connections 2024-05-22T23:06:13.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:13.914+0000 7f91beda2640 1 -- 172.21.15.84:0/316406946 wait complete. 2024-05-22T23:06:13.930 INFO:tasks.cephfs.filesystem:scrub status for tag:48d441e5-8481-4097-a09d-6a29d8e6f2a1 - {'path': '/', 'tag': '48d441e5-8481-4097-a09d-6a29d8e6f2a1', 'options': 'recursive,force'} 2024-05-22T23:06:14.433 INFO:tasks.workunit.client.0.smithi084.stdout: 1 521695 15.53 MB/sec execute 60 sec latency 64.014 ms 2024-05-22T23:06:14.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:14 smithi096 ceph-mon[36392]: scrub queued for path: / 2024-05-22T23:06:14.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:14 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:06:14.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:14 smithi096 ceph-mon[36392]: scrub summary: active paths [/] 2024-05-22T23:06:14.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:14 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:06:14.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:14 smithi084 ceph-mon[32186]: scrub queued for path: / 2024-05-22T23:06:14.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:14 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:06:14.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:14 smithi084 ceph-mon[32186]: scrub summary: active paths [/] 2024-05-22T23:06:14.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:14 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:06:14.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:14 smithi204 ceph-mon[36211]: scrub queued for path: / 2024-05-22T23:06:14.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:14 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:06:14.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:14 smithi204 ceph-mon[36211]: scrub summary: active paths [/] 2024-05-22T23:06:14.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:14 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:06:15.433 INFO:tasks.workunit.client.0.smithi084.stdout: 1 524616 15.50 MB/sec execute 61 sec latency 15.020 ms 2024-05-22T23:06:15.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:15 smithi096 ceph-mon[36392]: pgmap v135: 97 pgs: 97 active+clean; 265 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 252 op/s 2024-05-22T23:06:15.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:15 smithi084 ceph-mon[32186]: pgmap v135: 97 pgs: 97 active+clean; 265 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 252 op/s 2024-05-22T23:06:15.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:15 smithi204 ceph-mon[36211]: pgmap v135: 97 pgs: 97 active+clean; 265 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 252 op/s 2024-05-22T23:06:16.433 INFO:tasks.workunit.client.0.smithi084.stdout: 1 527252 15.53 MB/sec execute 62 sec latency 16.256 ms 2024-05-22T23:06:16.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:16 smithi096 ceph-mon[36392]: scrub summary: idle 2024-05-22T23:06:16.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:16 smithi084 ceph-mon[32186]: scrub summary: idle 2024-05-22T23:06:16.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:16 smithi204 ceph-mon[36211]: scrub summary: idle 2024-05-22T23:06:17.433 INFO:tasks.workunit.client.0.smithi084.stdout: 1 530070 15.51 MB/sec execute 63 sec latency 28.041 ms 2024-05-22T23:06:17.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:17 smithi096 ceph-mon[36392]: pgmap v136: 97 pgs: 97 active+clean; 273 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 374 op/s 2024-05-22T23:06:17.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:17 smithi084 ceph-mon[32186]: pgmap v136: 97 pgs: 97 active+clean; 273 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 374 op/s 2024-05-22T23:06:17.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:17 smithi204 ceph-mon[36211]: pgmap v136: 97 pgs: 97 active+clean; 273 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 374 op/s 2024-05-22T23:06:18.433 INFO:tasks.workunit.client.0.smithi084.stdout: 1 533272 15.56 MB/sec execute 64 sec latency 6.938 ms 2024-05-22T23:06:19.433 INFO:tasks.workunit.client.0.smithi084.stdout: 1 535787 15.51 MB/sec execute 65 sec latency 120.791 ms 2024-05-22T23:06:19.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:19 smithi096 ceph-mon[36392]: pgmap v137: 97 pgs: 97 active+clean; 273 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-05-22T23:06:19.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:19 smithi084 ceph-mon[32186]: pgmap v137: 97 pgs: 97 active+clean; 273 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-05-22T23:06:19.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:19 smithi204 ceph-mon[36211]: pgmap v137: 97 pgs: 97 active+clean; 273 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-05-22T23:06:20.433 INFO:tasks.workunit.client.0.smithi084.stdout: 1 538769 15.48 MB/sec execute 66 sec latency 14.114 ms 2024-05-22T23:06:21.433 INFO:tasks.workunit.client.0.smithi084.stdout: 1 541414 15.51 MB/sec execute 67 sec latency 14.281 ms 2024-05-22T23:06:21.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:21 smithi096 ceph-mon[36392]: pgmap v138: 97 pgs: 97 active+clean; 273 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-05-22T23:06:21.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:21 smithi084 ceph-mon[32186]: pgmap v138: 97 pgs: 97 active+clean; 273 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-05-22T23:06:21.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:21 smithi204 ceph-mon[36211]: pgmap v138: 97 pgs: 97 active+clean; 273 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-05-22T23:06:22.433 INFO:tasks.workunit.client.0.smithi084.stdout: 1 544253 15.50 MB/sec execute 68 sec latency 29.393 ms 2024-05-22T23:06:23.433 INFO:tasks.workunit.client.0.smithi084.stdout: 1 547442 15.54 MB/sec execute 69 sec latency 46.535 ms 2024-05-22T23:06:23.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:23 smithi096 ceph-mon[36392]: pgmap v139: 97 pgs: 97 active+clean; 282 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.0 MiB/s wr, 356 op/s 2024-05-22T23:06:23.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:23 smithi084 ceph-mon[32186]: pgmap v139: 97 pgs: 97 active+clean; 282 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.0 MiB/s wr, 356 op/s 2024-05-22T23:06:23.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:23 smithi204 ceph-mon[36211]: pgmap v139: 97 pgs: 97 active+clean; 282 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.0 MiB/s wr, 356 op/s 2024-05-22T23:06:24.433 INFO:tasks.workunit.client.0.smithi084.stdout: 1 549641 15.44 MB/sec execute 70 sec latency 80.408 ms 2024-05-22T23:06:25.433 INFO:tasks.workunit.client.0.smithi084.stdout: 1 552143 15.45 MB/sec execute 71 sec latency 22.728 ms 2024-05-22T23:06:25.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:25 smithi096 ceph-mon[36392]: pgmap v140: 97 pgs: 97 active+clean; 282 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 239 op/s 2024-05-22T23:06:25.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:25 smithi084 ceph-mon[32186]: pgmap v140: 97 pgs: 97 active+clean; 282 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 239 op/s 2024-05-22T23:06:25.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:25 smithi204 ceph-mon[36211]: pgmap v140: 97 pgs: 97 active+clean; 282 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 239 op/s 2024-05-22T23:06:26.433 INFO:tasks.workunit.client.0.smithi084.stdout: 1 555034 15.44 MB/sec execute 72 sec latency 14.063 ms 2024-05-22T23:06:27.433 INFO:tasks.workunit.client.0.smithi084.stdout: 1 558273 15.48 MB/sec execute 73 sec latency 7.405 ms 2024-05-22T23:06:27.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:27 smithi096 ceph-mon[36392]: pgmap v141: 97 pgs: 97 active+clean; 309 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.5 MiB/s wr, 362 op/s 2024-05-22T23:06:27.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:27 smithi084 ceph-mon[32186]: pgmap v141: 97 pgs: 97 active+clean; 309 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.5 MiB/s wr, 362 op/s 2024-05-22T23:06:27.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:27 smithi204 ceph-mon[36211]: pgmap v141: 97 pgs: 97 active+clean; 309 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.5 MiB/s wr, 362 op/s 2024-05-22T23:06:28.433 INFO:tasks.workunit.client.0.smithi084.stdout: 1 561511 15.52 MB/sec execute 74 sec latency 15.264 ms 2024-05-22T23:06:29.434 INFO:tasks.workunit.client.0.smithi084.stdout: 1 563690 15.42 MB/sec execute 75 sec latency 81.821 ms 2024-05-22T23:06:29.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:29 smithi096 ceph-mon[36392]: pgmap v142: 97 pgs: 97 active+clean; 309 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.3 MiB/s wr, 240 op/s 2024-05-22T23:06:29.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:29 smithi084 ceph-mon[32186]: pgmap v142: 97 pgs: 97 active+clean; 309 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.3 MiB/s wr, 240 op/s 2024-05-22T23:06:29.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:29 smithi204 ceph-mon[36211]: pgmap v142: 97 pgs: 97 active+clean; 309 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.3 MiB/s wr, 240 op/s 2024-05-22T23:06:30.434 INFO:tasks.workunit.client.0.smithi084.stdout: 1 565973 15.40 MB/sec execute 76 sec latency 40.755 ms 2024-05-22T23:06:31.434 INFO:tasks.workunit.client.0.smithi084.stdout: 1 568940 15.42 MB/sec execute 77 sec latency 8.291 ms 2024-05-22T23:06:31.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:31 smithi096 ceph-mon[36392]: pgmap v143: 97 pgs: 97 active+clean; 309 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.3 MiB/s wr, 240 op/s 2024-05-22T23:06:31.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:31 smithi084 ceph-mon[32186]: pgmap v143: 97 pgs: 97 active+clean; 309 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.3 MiB/s wr, 240 op/s 2024-05-22T23:06:31.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:31 smithi204 ceph-mon[36211]: pgmap v143: 97 pgs: 97 active+clean; 309 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.3 MiB/s wr, 240 op/s 2024-05-22T23:06:32.434 INFO:tasks.workunit.client.0.smithi084.stdout: 1 572199 15.46 MB/sec execute 78 sec latency 20.492 ms 2024-05-22T23:06:33.434 INFO:tasks.workunit.client.0.smithi084.stdout: 1 575464 15.49 MB/sec execute 79 sec latency 13.341 ms 2024-05-22T23:06:33.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:33 smithi096 ceph-mon[36392]: pgmap v144: 97 pgs: 97 active+clean; 317 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.4 MiB/s wr, 363 op/s 2024-05-22T23:06:33.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:33 smithi084 ceph-mon[32186]: pgmap v144: 97 pgs: 97 active+clean; 317 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.4 MiB/s wr, 363 op/s 2024-05-22T23:06:33.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:33 smithi204 ceph-mon[36211]: pgmap v144: 97 pgs: 97 active+clean; 317 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.4 MiB/s wr, 363 op/s 2024-05-22T23:06:34.434 INFO:tasks.workunit.client.0.smithi084.stdout: 1 577781 15.40 MB/sec execute 80 sec latency 88.253 ms 2024-05-22T23:06:35.434 INFO:tasks.workunit.client.0.smithi084.stdout: 1 579845 15.37 MB/sec execute 81 sec latency 56.789 ms 2024-05-22T23:06:35.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:35 smithi096 ceph-mon[36392]: pgmap v145: 97 pgs: 97 active+clean; 317 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.5 MiB/s wr, 245 op/s 2024-05-22T23:06:35.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:35 smithi084 ceph-mon[32186]: pgmap v145: 97 pgs: 97 active+clean; 317 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.5 MiB/s wr, 245 op/s 2024-05-22T23:06:35.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:35 smithi204 ceph-mon[36211]: pgmap v145: 97 pgs: 97 active+clean; 317 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.5 MiB/s wr, 245 op/s 2024-05-22T23:06:36.434 INFO:tasks.workunit.client.0.smithi084.stdout: 1 582978 15.41 MB/sec execute 82 sec latency 8.034 ms 2024-05-22T23:06:37.434 INFO:tasks.workunit.client.0.smithi084.stdout: 1 586283 15.45 MB/sec execute 83 sec latency 17.636 ms 2024-05-22T23:06:37.802 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:37 smithi096 ceph-mon[36392]: pgmap v146: 97 pgs: 97 active+clean; 325 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.3 MiB/s wr, 368 op/s 2024-05-22T23:06:37.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:37 smithi084 ceph-mon[32186]: pgmap v146: 97 pgs: 97 active+clean; 325 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.3 MiB/s wr, 368 op/s 2024-05-22T23:06:37.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:37 smithi204 ceph-mon[36211]: pgmap v146: 97 pgs: 97 active+clean; 325 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.3 MiB/s wr, 368 op/s 2024-05-22T23:06:38.434 INFO:tasks.workunit.client.0.smithi084.stdout: 1 589094 15.42 MB/sec execute 84 sec latency 13.746 ms 2024-05-22T23:06:39.434 INFO:tasks.workunit.client.0.smithi084.stdout: 1 591428 15.38 MB/sec execute 85 sec latency 49.017 ms 2024-05-22T23:06:39.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:39 smithi096 ceph-mon[36392]: pgmap v147: 97 pgs: 97 active+clean; 325 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 245 op/s 2024-05-22T23:06:39.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:39 smithi084 ceph-mon[32186]: pgmap v147: 97 pgs: 97 active+clean; 325 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 245 op/s 2024-05-22T23:06:39.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:39 smithi204 ceph-mon[36211]: pgmap v147: 97 pgs: 97 active+clean; 325 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 245 op/s 2024-05-22T23:06:40.434 INFO:tasks.workunit.client.0.smithi084.stdout: 1 593991 15.37 MB/sec execute 86 sec latency 103.598 ms 2024-05-22T23:06:41.434 INFO:tasks.workunit.client.0.smithi084.stdout: 1 597137 15.40 MB/sec execute 87 sec latency 22.319 ms 2024-05-22T23:06:41.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:41 smithi096 ceph-mon[36392]: pgmap v148: 97 pgs: 97 active+clean; 325 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 245 op/s 2024-05-22T23:06:41.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:41 smithi084 ceph-mon[32186]: pgmap v148: 97 pgs: 97 active+clean; 325 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 245 op/s 2024-05-22T23:06:41.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:41 smithi204 ceph-mon[36211]: pgmap v148: 97 pgs: 97 active+clean; 325 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 245 op/s 2024-05-22T23:06:42.434 INFO:tasks.workunit.client.0.smithi084.stdout: 1 600388 15.43 MB/sec execute 88 sec latency 18.904 ms 2024-05-22T23:06:43.434 INFO:tasks.workunit.client.0.smithi084.stdout: 1 603233 15.41 MB/sec execute 89 sec latency 16.962 ms 2024-05-22T23:06:43.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:43 smithi096 ceph-mon[36392]: pgmap v149: 97 pgs: 97 active+clean; 334 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.9 MiB/s wr, 376 op/s 2024-05-22T23:06:43.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:43 smithi084 ceph-mon[32186]: pgmap v149: 97 pgs: 97 active+clean; 334 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.9 MiB/s wr, 376 op/s 2024-05-22T23:06:43.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:43 smithi204 ceph-mon[36211]: pgmap v149: 97 pgs: 97 active+clean; 334 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.9 MiB/s wr, 376 op/s 2024-05-22T23:06:43.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:06:43.931 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:06:44.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.038+0000 7fea2fc76640 1 -- 172.21.15.84:0/3935795553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea28100740 msgr2=0x7fea28100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:44.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.038+0000 7fea2fc76640 1 --2- 172.21.15.84:0/3935795553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea28100740 0x7fea28100bc0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fea18009930 tx=0x7fea1802a420 comp rx=0 tx=0).stop 2024-05-22T23:06:44.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.038+0000 7fea2fc76640 1 -- 172.21.15.84:0/3935795553 shutdown_connections 2024-05-22T23:06:44.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.038+0000 7fea2fc76640 1 --2- 172.21.15.84:0/3935795553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea28101100 0x7fea2810d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.038+0000 7fea2fc76640 1 --2- 172.21.15.84:0/3935795553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea28100740 0x7fea28100bc0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.038+0000 7fea2fc76640 1 --2- 172.21.15.84:0/3935795553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea28106750 0x7fea28106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.038+0000 7fea2fc76640 1 -- 172.21.15.84:0/3935795553 >> 172.21.15.84:0/3935795553 conn(0x7fea28077870 msgr2=0x7fea28077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:44.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.038+0000 7fea2fc76640 1 -- 172.21.15.84:0/3935795553 shutdown_connections 2024-05-22T23:06:44.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.038+0000 7fea2fc76640 1 -- 172.21.15.84:0/3935795553 wait complete. 2024-05-22T23:06:44.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.039+0000 7fea2fc76640 1 Processor -- start 2024-05-22T23:06:44.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.039+0000 7fea2fc76640 1 -- start start 2024-05-22T23:06:44.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.039+0000 7fea2fc76640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea28100740 0x7fea281970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.039+0000 7fea2fc76640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea28101100 0x7fea28197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.039+0000 7fea2fc76640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea28106750 0x7fea2819b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.039+0000 7fea2fc76640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fea28110b60 con 0x7fea28106750 2024-05-22T23:06:44.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.039+0000 7fea2fc76640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fea281109e0 con 0x7fea28100740 2024-05-22T23:06:44.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.039+0000 7fea2fc76640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fea28110ce0 con 0x7fea28101100 2024-05-22T23:06:44.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.039+0000 7fea2e213640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea28106750 0x7fea2819b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.039+0000 7fea2e213640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea28106750 0x7fea2819b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37748/0 (socket says 172.21.15.84:37748) 2024-05-22T23:06:44.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.039+0000 7fea2e213640 1 -- 172.21.15.84:0/2860812275 learned_addr learned my addr 172.21.15.84:0/2860812275 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:44.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.039+0000 7fea2d211640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea28101100 0x7fea28197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.039+0000 7fea2da12640 1 --2- 172.21.15.84:0/2860812275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea28100740 0x7fea281970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.040+0000 7fea2e213640 1 -- 172.21.15.84:0/2860812275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea28101100 msgr2=0x7fea28197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:44.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.040+0000 7fea2e213640 1 --2- 172.21.15.84:0/2860812275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea28101100 0x7fea28197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.040+0000 7fea2e213640 1 -- 172.21.15.84:0/2860812275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea28100740 msgr2=0x7fea281970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:44.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.040+0000 7fea2e213640 1 --2- 172.21.15.84:0/2860812275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea28100740 0x7fea281970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.040+0000 7fea2e213640 1 -- 172.21.15.84:0/2860812275 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2819c0b0 con 0x7fea28106750 2024-05-22T23:06:44.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.040+0000 7fea2da12640 1 --2- 172.21.15.84:0/2860812275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea28100740 0x7fea281970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:06:44.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.040+0000 7fea2e213640 1 --2- 172.21.15.84:0/2860812275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea28106750 0x7fea2819b980 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fea20009fd0 tx=0x7fea20007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:44.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.040+0000 7fea1effd640 1 -- 172.21.15.84:0/2860812275 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2000de10 con 0x7fea28106750 2024-05-22T23:06:44.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.040+0000 7fea2fc76640 1 -- 172.21.15.84:0/2860812275 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea2819c3a0 con 0x7fea28106750 2024-05-22T23:06:44.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.040+0000 7fea1effd640 1 -- 172.21.15.84:0/2860812275 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea20003e50 con 0x7fea28106750 2024-05-22T23:06:44.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.040+0000 7fea1effd640 1 -- 172.21.15.84:0/2860812275 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20018420 con 0x7fea28106750 2024-05-22T23:06:44.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.040+0000 7fea2fc76640 1 -- 172.21.15.84:0/2860812275 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea28068a00 con 0x7fea28106750 2024-05-22T23:06:44.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.042+0000 7fea1effd640 1 -- 172.21.15.84:0/2860812275 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fea20003970 con 0x7fea28106750 2024-05-22T23:06:44.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.043+0000 7fea32988740 1 Processor -- start 2024-05-22T23:06:44.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.043+0000 7fea32988740 1 -- start start 2024-05-22T23:06:44.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.043+0000 7fea32988740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd634a0 0x55b08dd617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.043+0000 7fea32988740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b08dd61db0 0x55b08dd62230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.043+0000 7fea32988740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b08dd70f40 0x55b08dd73330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.043+0000 7fea32988740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b08dcc5db0 con 0x55b08dd634a0 2024-05-22T23:06:44.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.043+0000 7fea32988740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b08dc64360 con 0x55b08dd70f40 2024-05-22T23:06:44.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.043+0000 7fea32988740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b08dcc1d70 con 0x55b08dd61db0 2024-05-22T23:06:44.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.043+0000 7fea2d211640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd634a0 0x55b08dd617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.043+0000 7fea2da12640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b08dd61db0 0x55b08dd62230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.043+0000 7fea2d211640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd634a0 0x55b08dd617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37758/0 (socket says 172.21.15.84:37758) 2024-05-22T23:06:44.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.044+0000 7fea2d211640 1 -- 172.21.15.84:0/4166667979 learned_addr learned my addr 172.21.15.84:0/4166667979 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:44.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.044+0000 7fea2d211640 1 --2- 172.21.15.84:0/4166667979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b08dd70f40 0x55b08dd73330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.044+0000 7fea1effd640 1 --2- 172.21.15.84:0/2860812275 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fea140a2d00 0x7fea140a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.044+0000 7fea2da12640 1 -- 172.21.15.84:0/4166667979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b08dd70f40 msgr2=0x55b08dd73330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:44.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.044+0000 7fea2e213640 1 --2- 172.21.15.84:0/2860812275 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fea140a2d00 0x7fea140a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.044+0000 7fea2da12640 1 --2- 172.21.15.84:0/4166667979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b08dd70f40 0x55b08dd73330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.044+0000 7fea2da12640 1 -- 172.21.15.84:0/4166667979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd634a0 msgr2=0x55b08dd617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:44.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.044+0000 7fea2da12640 1 --2- 172.21.15.84:0/4166667979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd634a0 0x55b08dd617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.044+0000 7fea2da12640 1 -- 172.21.15.84:0/4166667979 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b08dc87a60 con 0x55b08dd61db0 2024-05-22T23:06:44.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.044+0000 7fea2d211640 1 --2- 172.21.15.84:0/4166667979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd634a0 0x55b08dd617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:06:44.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.044+0000 7fea1effd640 1 -- 172.21.15.84:0/2860812275 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fea200d0b80 con 0x7fea28106750 2024-05-22T23:06:44.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.044+0000 7fea2da12640 1 --2- 172.21.15.84:0/4166667979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b08dd61db0 0x55b08dd62230 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fea24009f50 tx=0x7fea240118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ed38376056ab990 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:44.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.044+0000 7fea2e213640 1 --2- 172.21.15.84:0/2860812275 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fea140a2d00 0x7fea140a51c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fea2009ce50 tx=0x7fea20004340 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:44.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.044+0000 7fea13fff640 1 -- 172.21.15.84:0/4166667979 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea24004030 con 0x55b08dd61db0 2024-05-22T23:06:44.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.044+0000 7fea13fff640 1 -- 172.21.15.84:0/4166667979 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea24005500 con 0x55b08dd61db0 2024-05-22T23:06:44.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.044+0000 7fea13fff640 1 -- 172.21.15.84:0/4166667979 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea24004db0 con 0x55b08dd61db0 2024-05-22T23:06:44.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.045+0000 7fea32988740 1 -- 172.21.15.84:0/4166667979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b08dd61db0 msgr2=0x55b08dd62230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:44.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.045+0000 7fea32988740 1 --2- 172.21.15.84:0/4166667979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b08dd61db0 0x55b08dd62230 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fea24009f50 tx=0x7fea240118d0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.045+0000 7fea32988740 1 -- 172.21.15.84:0/4166667979 shutdown_connections 2024-05-22T23:06:44.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.045+0000 7fea32988740 1 --2- 172.21.15.84:0/4166667979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b08dd70f40 0x55b08dd73330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.045+0000 7fea32988740 1 --2- 172.21.15.84:0/4166667979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b08dd61db0 0x55b08dd62230 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.045+0000 7fea32988740 1 --2- 172.21.15.84:0/4166667979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd634a0 0x55b08dd617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.045+0000 7fea32988740 1 -- 172.21.15.84:0/4166667979 >> 172.21.15.84:0/4166667979 conn(0x55b08dd2a8e0 msgr2=0x55b08dd5e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:44.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.045+0000 7fea32988740 1 -- 172.21.15.84:0/4166667979 shutdown_connections 2024-05-22T23:06:44.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.045+0000 7fea32988740 1 -- 172.21.15.84:0/4166667979 wait complete. 2024-05-22T23:06:44.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.045+0000 7fea32988740 1 -- start start 2024-05-22T23:06:44.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.046+0000 7fea32988740 1 Processor -- start 2024-05-22T23:06:44.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.046+0000 7fea32988740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b08dd61db0 0x55b08dd6de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.046+0000 7fea32988740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd634a0 0x55b08dd6e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.046+0000 7fea32988740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b08dd70f40 0x55b08dd6e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.046+0000 7fea2d211640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b08dd61db0 0x55b08dd6de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.046+0000 7fea32988740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b08dd73dd0 con 0x55b08dd634a0 2024-05-22T23:06:44.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.046+0000 7fea2d211640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b08dd61db0 0x55b08dd6de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:54212/0 (socket says 172.21.15.84:54212) 2024-05-22T23:06:44.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.046+0000 7fea2d211640 1 -- 172.21.15.84:0/1409190913 learned_addr learned my addr 172.21.15.84:0/1409190913 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:44.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.046+0000 7fea2da12640 1 --2- 172.21.15.84:0/1409190913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd634a0 0x55b08dd6e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.046+0000 7fea32988740 1 -- 172.21.15.84:0/1409190913 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b08dc800f0 con 0x55b08dd70f40 2024-05-22T23:06:44.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.046+0000 7fea32988740 1 -- 172.21.15.84:0/1409190913 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b08dd73f50 con 0x55b08dd61db0 2024-05-22T23:06:44.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.046+0000 7fea2d211640 1 --2- 172.21.15.84:0/1409190913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b08dd70f40 0x55b08dd6e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.046+0000 7fea32988740 10 client.0 _ll_register_callbacks cb 0x55b08dc84f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:06:44.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.047+0000 7fea2da12640 1 -- 172.21.15.84:0/1409190913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b08dd61db0 msgr2=0x55b08dd6de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:44.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.047+0000 7fea2da12640 1 --2- 172.21.15.84:0/1409190913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b08dd61db0 0x55b08dd6de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.047+0000 7fea2da12640 1 -- 172.21.15.84:0/1409190913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b08dd70f40 msgr2=0x55b08dd6e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:44.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.047+0000 7fea2da12640 1 --2- 172.21.15.84:0/1409190913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b08dd70f40 0x55b08dd6e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.047+0000 7fea2da12640 1 -- 172.21.15.84:0/1409190913 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b08daa37d0 con 0x55b08dd634a0 2024-05-22T23:06:44.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.047+0000 7fea2d211640 1 --2- 172.21.15.84:0/1409190913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b08dd61db0 0x55b08dd6de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:06:44.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.047+0000 7fea2da12640 1 --2- 172.21.15.84:0/1409190913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd634a0 0x55b08dd6e380 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fea2401b040 tx=0x7fea24005290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94563208fcc1ab8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:44.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.047+0000 7fea117fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:06:44.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.047+0000 7fea117fa640 1 -- 172.21.15.84:0/1409190913 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea24029020 con 0x55b08dd634a0 2024-05-22T23:06:44.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.047+0000 7fea32988740 1 -- 172.21.15.84:0/1409190913 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea24005500 con 0x55b08dd634a0 2024-05-22T23:06:44.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.047+0000 7fea117fa640 1 -- 172.21.15.84:0/1409190913 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fea2400f750 con 0x55b08dd634a0 2024-05-22T23:06:44.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.047+0000 7fea117fa640 1 -- 172.21.15.84:0/1409190913 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fea240046c0 con 0x55b08dd634a0 2024-05-22T23:06:44.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.047+0000 7fea32988740 10 client.16182 fetch_fsmap learned FSMap version 18 2024-05-22T23:06:44.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.047+0000 7fea32988740 1 -- 172.21.15.84:0/1409190913 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55b08d9d93f0 con 0x55b08dd634a0 2024-05-22T23:06:44.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.047+0000 7fea117fa640 1 -- 172.21.15.84:0/1409190913 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea24005500 con 0x55b08dd634a0 2024-05-22T23:06:44.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.048+0000 7fea117fa640 1 -- 172.21.15.84:0/1409190913 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fea24025500 con 0x55b08dd634a0 2024-05-22T23:06:44.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.048+0000 7fea32988740 10 client.16182 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:06:44.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.048+0000 7fea32988740 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:06:44.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.048+0000 7fea32988740 20 client.16182 populate_metadata read hostname 'smithi084' 2024-05-22T23:06:44.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.048+0000 7fea32988740 1 --2- 172.21.15.84:0/1409190913 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b08dd81230 0x55b08dd83620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.048+0000 7fea32988740 4 client.16182 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:06:44.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.048+0000 7fea32988740 1 -- 172.21.15.84:0/1409190913 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b08dcd34e0 con 0x55b08dd81230 2024-05-22T23:06:44.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.048+0000 7fea2d211640 1 --2- 172.21.15.84:0/1409190913 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b08dd81230 0x55b08dd83620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.048+0000 7fea117fa640 10 client.16182 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:44.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.048+0000 7fea2d211640 1 --2- 172.21.15.84:0/1409190913 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b08dd81230 0x55b08dd83620 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a615a4ac958896ea server_cookie=6e6ba3b47fe4692f in_seq=0 out_seq=0 2024-05-22T23:06:44.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.049+0000 7fea117fa640 1 -- 172.21.15.84:0/1409190913 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b08dcd34e0 con 0x55b08dd81230 2024-05-22T23:06:44.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.049+0000 7fea117fa640 10 client.16182 handle_command_reply: tid=0 2024-05-22T23:06:44.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.049+0000 7fea32988740 1 client.16182 shutdown 2024-05-22T23:06:44.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.049+0000 7fea32988740 1 -- 172.21.15.84:0/1409190913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd634a0 msgr2=0x55b08dd6e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:44.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.049+0000 7fea32988740 1 --2- 172.21.15.84:0/1409190913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd634a0 0x55b08dd6e380 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fea2401b040 tx=0x7fea24005290 comp rx=0 tx=0).stop 2024-05-22T23:06:44.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.049+0000 7fea32988740 1 -- 172.21.15.84:0/1409190913 shutdown_connections 2024-05-22T23:06:44.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.049+0000 7fea32988740 1 --2- 172.21.15.84:0/1409190913 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b08dd81230 0x55b08dd83620 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.049+0000 7fea32988740 1 --2- 172.21.15.84:0/1409190913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b08dd70f40 0x55b08dd6e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.049+0000 7fea32988740 1 --2- 172.21.15.84:0/1409190913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd634a0 0x55b08dd6e380 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.049+0000 7fea117fa640 0 client.16182 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:44.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.049+0000 7fea32988740 1 --2- 172.21.15.84:0/1409190913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b08dd61db0 0x55b08dd6de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.049+0000 7fea32988740 1 -- 172.21.15.84:0/1409190913 >> 172.21.15.84:0/1409190913 conn(0x55b08dd2a8e0 msgr2=0x55b08dd64110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:44.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.050+0000 7fea32988740 1 -- 172.21.15.84:0/1409190913 shutdown_connections 2024-05-22T23:06:44.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.050+0000 7fea32988740 1 -- 172.21.15.84:0/1409190913 wait complete. 2024-05-22T23:06:44.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.050+0000 7fea32988740 20 client.16182 trim_cache size 0 max 16384 2024-05-22T23:06:44.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.070+0000 7fea32988740 1 Processor -- start 2024-05-22T23:06:44.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.070+0000 7fea32988740 1 -- start start 2024-05-22T23:06:44.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.070+0000 7fea32988740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd61db0 0x55b08dd7ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.070+0000 7fea32988740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b08dd634a0 0x55b08dd88e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.070+0000 7fea32988740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b08dd70f40 0x55b08dd89390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.070+0000 7fea32988740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fea281a9490 con 0x55b08dd61db0 2024-05-22T23:06:44.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.070+0000 7fea32988740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fea281abd10 con 0x55b08dd70f40 2024-05-22T23:06:44.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.070+0000 7fea32988740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fea281a9b00 con 0x55b08dd634a0 2024-05-22T23:06:44.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.070+0000 7fea2d211640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd61db0 0x55b08dd7ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.070+0000 7fea2d211640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd61db0 0x55b08dd7ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37788/0 (socket says 172.21.15.84:37788) 2024-05-22T23:06:44.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.070+0000 7fea2d211640 1 -- 172.21.15.84:0/206314595 learned_addr learned my addr 172.21.15.84:0/206314595 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:44.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.070+0000 7fea2da12640 1 --2- 172.21.15.84:0/206314595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b08dd634a0 0x55b08dd88e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.070+0000 7fea2d211640 1 --2- 172.21.15.84:0/206314595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b08dd70f40 0x55b08dd89390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.071+0000 7fea2d211640 1 -- 172.21.15.84:0/206314595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b08dd634a0 msgr2=0x55b08dd88e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:44.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.071+0000 7fea2d211640 1 --2- 172.21.15.84:0/206314595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b08dd634a0 0x55b08dd88e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.071+0000 7fea2d211640 1 -- 172.21.15.84:0/206314595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b08dd70f40 msgr2=0x55b08dd89390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:44.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.071+0000 7fea2d211640 1 --2- 172.21.15.84:0/206314595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b08dd70f40 0x55b08dd89390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.071+0000 7fea2d211640 1 -- 172.21.15.84:0/206314595 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b08d9990f0 con 0x55b08dd61db0 2024-05-22T23:06:44.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.071+0000 7fea2d211640 1 --2- 172.21.15.84:0/206314595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd61db0 0x55b08dd7ccd0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fea18009230 tx=0x7fea18000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da797cb1520cc76 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:44.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.071+0000 7fea137fe640 1 -- 172.21.15.84:0/206314595 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18035040 con 0x55b08dd61db0 2024-05-22T23:06:44.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.071+0000 7fea137fe640 1 -- 172.21.15.84:0/206314595 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea18052070 con 0x55b08dd61db0 2024-05-22T23:06:44.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.071+0000 7fea137fe640 1 -- 172.21.15.84:0/206314595 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18003970 con 0x55b08dd61db0 2024-05-22T23:06:44.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.072+0000 7fea32988740 1 -- 172.21.15.84:0/206314595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd61db0 msgr2=0x55b08dd7ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:44.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.072+0000 7fea32988740 1 --2- 172.21.15.84:0/206314595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd61db0 0x55b08dd7ccd0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fea18009230 tx=0x7fea18000bc0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.072+0000 7fea32988740 1 -- 172.21.15.84:0/206314595 shutdown_connections 2024-05-22T23:06:44.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.072+0000 7fea32988740 1 --2- 172.21.15.84:0/206314595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b08dd70f40 0x55b08dd89390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.072+0000 7fea32988740 1 --2- 172.21.15.84:0/206314595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b08dd634a0 0x55b08dd88e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.072+0000 7fea32988740 1 --2- 172.21.15.84:0/206314595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd61db0 0x55b08dd7ccd0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.072+0000 7fea32988740 1 -- 172.21.15.84:0/206314595 >> 172.21.15.84:0/206314595 conn(0x55b08dd2a8e0 msgr2=0x55b08dd6ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:44.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.072+0000 7fea32988740 1 -- 172.21.15.84:0/206314595 shutdown_connections 2024-05-22T23:06:44.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.072+0000 7fea32988740 1 -- 172.21.15.84:0/206314595 wait complete. 2024-05-22T23:06:44.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.072+0000 7fea32988740 1 -- start start 2024-05-22T23:06:44.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.072+0000 7fea32988740 1 Processor -- start 2024-05-22T23:06:44.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.072+0000 7fea32988740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd61db0 0x55b08dd6b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.072+0000 7fea32988740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b08dd634a0 0x55b08dd6ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.072+0000 7fea32988740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b08dd70f40 0x55b08dd6a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.072+0000 7fea32988740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fea281aa1d0 con 0x55b08dd61db0 2024-05-22T23:06:44.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.072+0000 7fea32988740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fea281ab640 con 0x55b08dd634a0 2024-05-22T23:06:44.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.072+0000 7fea32988740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fea281aa8a0 con 0x55b08dd70f40 2024-05-22T23:06:44.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.072+0000 7fea32988740 10 client.0 _ll_register_callbacks cb 0x7fea18009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:06:44.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.073+0000 7fea2d211640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd61db0 0x55b08dd6b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.073+0000 7fea2d211640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd61db0 0x55b08dd6b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37802/0 (socket says 172.21.15.84:37802) 2024-05-22T23:06:44.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.073+0000 7fea2d211640 1 -- 172.21.15.84:0/1014599206 learned_addr learned my addr 172.21.15.84:0/1014599206 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:44.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.073+0000 7fea2da12640 1 --2- 172.21.15.84:0/1014599206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b08dd634a0 0x55b08dd6ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.073+0000 7fea2d211640 1 --2- 172.21.15.84:0/1014599206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b08dd70f40 0x55b08dd6a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.073+0000 7fea2d211640 1 -- 172.21.15.84:0/1014599206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b08dd70f40 msgr2=0x55b08dd6a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:44.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.073+0000 7fea2d211640 1 --2- 172.21.15.84:0/1014599206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b08dd70f40 0x55b08dd6a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.073+0000 7fea2d211640 1 -- 172.21.15.84:0/1014599206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b08dd634a0 msgr2=0x55b08dd6ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:44.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.073+0000 7fea2d211640 1 --2- 172.21.15.84:0/1014599206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b08dd634a0 0x55b08dd6ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.073+0000 7fea2d211640 1 -- 172.21.15.84:0/1014599206 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b08dc84fe0 con 0x55b08dd61db0 2024-05-22T23:06:44.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.073+0000 7fea2d211640 1 --2- 172.21.15.84:0/1014599206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd61db0 0x55b08dd6b510 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fea18009430 tx=0x7fea18005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948b68c146e9b059 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:44.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.073+0000 7fea117fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:06:44.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.073+0000 7fea117fa640 1 -- 172.21.15.84:0/1014599206 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1805c070 con 0x55b08dd61db0 2024-05-22T23:06:44.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.073+0000 7fea117fa640 1 -- 172.21.15.84:0/1014599206 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fea1804d7d0 con 0x55b08dd61db0 2024-05-22T23:06:44.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.073+0000 7fea32988740 1 -- 172.21.15.84:0/1014599206 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea18052070 con 0x55b08dd61db0 2024-05-22T23:06:44.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.073+0000 7fea117fa640 1 -- 172.21.15.84:0/1014599206 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fea18068050 con 0x55b08dd61db0 2024-05-22T23:06:44.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.073+0000 7fea32988740 10 client.16188 fetch_fsmap learned FSMap version 18 2024-05-22T23:06:44.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.073+0000 7fea32988740 1 -- 172.21.15.84:0/1014599206 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55b08dc561c0 con 0x55b08dd61db0 2024-05-22T23:06:44.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.073+0000 7fea117fa640 1 -- 172.21.15.84:0/1014599206 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea18052070 con 0x55b08dd61db0 2024-05-22T23:06:44.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.074+0000 7fea117fa640 1 -- 172.21.15.84:0/1014599206 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fea18058680 con 0x55b08dd61db0 2024-05-22T23:06:44.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.074+0000 7fea32988740 10 client.16188 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:06:44.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.074+0000 7fea32988740 10 client.16188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:06:44.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.074+0000 7fea32988740 20 client.16188 populate_metadata read hostname 'smithi084' 2024-05-22T23:06:44.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.074+0000 7fea32988740 1 --2- 172.21.15.84:0/1014599206 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b08dde6f90 0x55b08dde93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.074+0000 7fea32988740 4 client.16188 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:06:44.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.074+0000 7fea32988740 1 -- 172.21.15.84:0/1014599206 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b08dccd830 con 0x55b08dde6f90 2024-05-22T23:06:44.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.074+0000 7fea2da12640 1 --2- 172.21.15.84:0/1014599206 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b08dde6f90 0x55b08dde93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.074+0000 7fea2da12640 1 --2- 172.21.15.84:0/1014599206 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b08dde6f90 0x55b08dde93b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84c32150ba76267e server_cookie=e67ce7e18ac72a3e in_seq=0 out_seq=0 2024-05-22T23:06:44.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.074+0000 7fea117fa640 10 client.16188 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:44.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.074+0000 7fea117fa640 1 -- 172.21.15.84:0/1014599206 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b08dccd830 con 0x55b08dde6f90 2024-05-22T23:06:44.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.074+0000 7fea117fa640 10 client.16188 handle_command_reply: tid=0 2024-05-22T23:06:44.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.074+0000 7fea32988740 1 client.16188 shutdown 2024-05-22T23:06:44.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.074+0000 7fea32988740 1 -- 172.21.15.84:0/1014599206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd61db0 msgr2=0x55b08dd6b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:44.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.074+0000 7fea32988740 1 --2- 172.21.15.84:0/1014599206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd61db0 0x55b08dd6b510 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fea18009430 tx=0x7fea18005bd0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.075+0000 7fea32988740 1 -- 172.21.15.84:0/1014599206 shutdown_connections 2024-05-22T23:06:44.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.075+0000 7fea32988740 1 --2- 172.21.15.84:0/1014599206 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b08dde6f90 0x55b08dde93b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.075+0000 7fea32988740 1 --2- 172.21.15.84:0/1014599206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b08dd70f40 0x55b08dd6a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.075+0000 7fea32988740 1 --2- 172.21.15.84:0/1014599206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b08dd634a0 0x55b08dd6ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.075+0000 7fea32988740 1 --2- 172.21.15.84:0/1014599206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b08dd61db0 0x55b08dd6b510 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.075+0000 7fea117fa640 0 client.16188 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:44.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.075+0000 7fea32988740 1 -- 172.21.15.84:0/1014599206 >> 172.21.15.84:0/1014599206 conn(0x55b08dd2a8e0 msgr2=0x55b08dd63880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:44.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.075+0000 7fea32988740 1 -- 172.21.15.84:0/1014599206 shutdown_connections 2024-05-22T23:06:44.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.075+0000 7fea32988740 1 -- 172.21.15.84:0/1014599206 wait complete. 2024-05-22T23:06:44.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.075+0000 7fea32988740 20 client.16188 trim_cache size 0 max 16384 2024-05-22T23:06:44.087 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:06:44.087 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "no active scrubs running", 2024-05-22T23:06:44.087 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": {} 2024-05-22T23:06:44.088 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:06:44.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.075+0000 7fea2fc76640 1 -- 172.21.15.84:0/2860812275 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fea140a2d00 msgr2=0x7fea140a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:44.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.075+0000 7fea2fc76640 1 --2- 172.21.15.84:0/2860812275 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fea140a2d00 0x7fea140a51c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fea2009ce50 tx=0x7fea20004340 comp rx=0 tx=0).stop 2024-05-22T23:06:44.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.075+0000 7fea2fc76640 1 -- 172.21.15.84:0/2860812275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea28106750 msgr2=0x7fea2819b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:44.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.075+0000 7fea2fc76640 1 --2- 172.21.15.84:0/2860812275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea28106750 0x7fea2819b980 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fea20009fd0 tx=0x7fea20007590 comp rx=0 tx=0).stop 2024-05-22T23:06:44.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.075+0000 7fea2fc76640 1 -- 172.21.15.84:0/2860812275 shutdown_connections 2024-05-22T23:06:44.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.075+0000 7fea2fc76640 1 --2- 172.21.15.84:0/2860812275 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fea140a2d00 0x7fea140a51c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.075+0000 7fea2fc76640 1 --2- 172.21.15.84:0/2860812275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea28106750 0x7fea2819b980 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.075+0000 7fea2fc76640 1 --2- 172.21.15.84:0/2860812275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea28101100 0x7fea28197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.075+0000 7fea2fc76640 1 --2- 172.21.15.84:0/2860812275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea28100740 0x7fea281970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.075+0000 7fea2fc76640 1 -- 172.21.15.84:0/2860812275 >> 172.21.15.84:0/2860812275 conn(0x7fea28077870 msgr2=0x7fea281030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:44.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.075+0000 7fea2fc76640 1 -- 172.21.15.84:0/2860812275 shutdown_connections 2024-05-22T23:06:44.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.076+0000 7fea2fc76640 1 -- 172.21.15.84:0/2860812275 wait complete. 2024-05-22T23:06:44.092 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-22T23:06:44.092 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:06:44.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.194+0000 7f0f1238b640 1 -- 172.21.15.84:0/2396919914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f0c102b00 msgr2=0x7f0f0c102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:44.200 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.194+0000 7f0f1238b640 1 --2- 172.21.15.84:0/2396919914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f0c102b00 0x7f0f0c102ee0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009900 tx=0x7f0efc02a510 comp rx=0 tx=0).stop 2024-05-22T23:06:44.200 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.194+0000 7f0f1238b640 1 -- 172.21.15.84:0/2396919914 shutdown_connections 2024-05-22T23:06:44.200 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.194+0000 7f0f1238b640 1 --2- 172.21.15.84:0/2396919914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f0c10be40 0x7f0f0c10e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.200 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.194+0000 7f0f1238b640 1 --2- 172.21.15.84:0/2396919914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f0c103420 0x7f0f0c10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.200 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.194+0000 7f0f1238b640 1 --2- 172.21.15.84:0/2396919914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f0c102b00 0x7f0f0c102ee0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.200 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.194+0000 7f0f1238b640 1 -- 172.21.15.84:0/2396919914 >> 172.21.15.84:0/2396919914 conn(0x7f0f0c077870 msgr2=0x7f0f0c077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:44.200 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.194+0000 7f0f1238b640 1 -- 172.21.15.84:0/2396919914 shutdown_connections 2024-05-22T23:06:44.200 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.194+0000 7f0f1238b640 1 -- 172.21.15.84:0/2396919914 wait complete. 2024-05-22T23:06:44.200 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.195+0000 7f0f1238b640 1 Processor -- start 2024-05-22T23:06:44.200 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.195+0000 7f0f1238b640 1 -- start start 2024-05-22T23:06:44.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.195+0000 7f0f1238b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f0c102b00 0x7f0f0c198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.195+0000 7f0f1238b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f0c103420 0x7f0f0c198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.195+0000 7f0f1238b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f0c10be40 0x7f0f0c19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.195+0000 7f0f1238b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c111c40 con 0x7f0f0c10be40 2024-05-22T23:06:44.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.195+0000 7f0f1238b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c111ac0 con 0x7f0f0c102b00 2024-05-22T23:06:44.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.195+0000 7f0f1238b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c111dc0 con 0x7f0f0c103420 2024-05-22T23:06:44.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.195+0000 7f0f10928640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f0c10be40 0x7f0f0c19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.195+0000 7f0f0b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f0c103420 0x7f0f0c198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.195+0000 7f0f10928640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f0c10be40 0x7f0f0c19c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37814/0 (socket says 172.21.15.84:37814) 2024-05-22T23:06:44.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.195+0000 7f0f0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f0c102b00 0x7f0f0c198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.195+0000 7f0f10928640 1 -- 172.21.15.84:0/725007902 learned_addr learned my addr 172.21.15.84:0/725007902 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:44.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.196+0000 7f0f10928640 1 -- 172.21.15.84:0/725007902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f0c103420 msgr2=0x7f0f0c198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:44.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.196+0000 7f0f10928640 1 --2- 172.21.15.84:0/725007902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f0c103420 0x7f0f0c198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.196+0000 7f0f10928640 1 -- 172.21.15.84:0/725007902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f0c102b00 msgr2=0x7f0f0c198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:44.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.196+0000 7f0f10928640 1 --2- 172.21.15.84:0/725007902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f0c102b00 0x7f0f0c198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.196+0000 7f0f10928640 1 -- 172.21.15.84:0/725007902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0c19d110 con 0x7f0f0c10be40 2024-05-22T23:06:44.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.196+0000 7f0f10928640 1 --2- 172.21.15.84:0/725007902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f0c10be40 0x7f0f0c19c9e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400eaa0 tx=0x7f0f0400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:44.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.196+0000 7f0f097fa640 1 -- 172.21.15.84:0/725007902 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0400cdb0 con 0x7f0f0c10be40 2024-05-22T23:06:44.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.196+0000 7f0f1238b640 1 -- 172.21.15.84:0/725007902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f0c19d400 con 0x7f0f0c10be40 2024-05-22T23:06:44.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.196+0000 7f0f097fa640 1 -- 172.21.15.84:0/725007902 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f04005020 con 0x7f0f0c10be40 2024-05-22T23:06:44.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.196+0000 7f0f097fa640 1 -- 172.21.15.84:0/725007902 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f040116b0 con 0x7f0f0c10be40 2024-05-22T23:06:44.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.196+0000 7f0f1238b640 1 -- 172.21.15.84:0/725007902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f0c1a4c90 con 0x7f0f0c10be40 2024-05-22T23:06:44.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.197+0000 7f0f1238b640 1 -- 172.21.15.84:0/725007902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f0c068690 con 0x7f0f0c10be40 2024-05-22T23:06:44.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.198+0000 7f0f097fa640 1 -- 172.21.15.84:0/725007902 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0f04005560 con 0x7f0f0c10be40 2024-05-22T23:06:44.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.199+0000 7f0f097fa640 1 --2- 172.21.15.84:0/725007902 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f0ef40a2d00 0x7f0ef40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.199+0000 7f0f097fa640 1 -- 172.21.15.84:0/725007902 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f0f040cf940 con 0x7f0f0c10be40 2024-05-22T23:06:44.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.199+0000 7f0f0bfff640 1 --2- 172.21.15.84:0/725007902 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f0ef40a2d00 0x7f0ef40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.199+0000 7f0f0bfff640 1 --2- 172.21.15.84:0/725007902 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f0ef40a2d00 0x7f0ef40a51c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009230 tx=0x7f0efc0058d0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:44.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.203+0000 7f0f097fa640 1 -- 172.21.15.84:0/725007902 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0f04016030 con 0x7f0f0c10be40 2024-05-22T23:06:44.434 INFO:tasks.workunit.client.0.smithi084.stdout: 1 606140 15.39 MB/sec execute 90 sec latency 16.337 ms 2024-05-22T23:06:44.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.453+0000 7f0f1238b640 1 -- 172.21.15.84:0/725007902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f0c107020 con 0x7f0f0c10be40 2024-05-22T23:06:44.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.454+0000 7f0f097fa640 1 -- 172.21.15.84:0/725007902 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7f0f0409bf10 con 0x7f0f0c10be40 2024-05-22T23:06:44.459 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:06:44.459 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:06:44.459 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:06:44.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.456+0000 7f0f1238b640 1 -- 172.21.15.84:0/725007902 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f0ef40a2d00 msgr2=0x7f0ef40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:44.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.456+0000 7f0f1238b640 1 --2- 172.21.15.84:0/725007902 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f0ef40a2d00 0x7f0ef40a51c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009230 tx=0x7f0efc0058d0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.456+0000 7f0f1238b640 1 -- 172.21.15.84:0/725007902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f0c10be40 msgr2=0x7f0f0c19c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:44.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.456+0000 7f0f1238b640 1 --2- 172.21.15.84:0/725007902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f0c10be40 0x7f0f0c19c9e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400eaa0 tx=0x7f0f0400ef60 comp rx=0 tx=0).stop 2024-05-22T23:06:44.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.456+0000 7f0f1238b640 1 -- 172.21.15.84:0/725007902 shutdown_connections 2024-05-22T23:06:44.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.456+0000 7f0f1238b640 1 --2- 172.21.15.84:0/725007902 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f0ef40a2d00 0x7f0ef40a51c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.456+0000 7f0f1238b640 1 --2- 172.21.15.84:0/725007902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f0c10be40 0x7f0f0c19c9e0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.456+0000 7f0f1238b640 1 --2- 172.21.15.84:0/725007902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f0c103420 0x7f0f0c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.456+0000 7f0f1238b640 1 --2- 172.21.15.84:0/725007902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f0c102b00 0x7f0f0c198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.456+0000 7f0f1238b640 1 -- 172.21.15.84:0/725007902 >> 172.21.15.84:0/725007902 conn(0x7f0f0c077870 msgr2=0x7f0f0c0fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:44.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.456+0000 7f0f1238b640 1 -- 172.21.15.84:0/725007902 shutdown_connections 2024-05-22T23:06:44.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.456+0000 7f0f1238b640 1 -- 172.21.15.84:0/725007902 wait complete. 2024-05-22T23:06:44.473 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-22T23:06:44.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.595+0000 7f604e29a640 1 -- 172.21.15.84:0/1730419204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6048101850 msgr2=0x7f6048101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:44.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.595+0000 7f604e29a640 1 --2- 172.21.15.84:0/1730419204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6048101850 0x7f6048101cd0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f603c009930 tx=0x7f603c02a420 comp rx=0 tx=0).stop 2024-05-22T23:06:44.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.595+0000 7f604e29a640 1 -- 172.21.15.84:0/1730419204 shutdown_connections 2024-05-22T23:06:44.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.595+0000 7f604e29a640 1 --2- 172.21.15.84:0/1730419204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6048102210 0x7f604810e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.595+0000 7f604e29a640 1 --2- 172.21.15.84:0/1730419204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6048101850 0x7f6048101cd0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.595+0000 7f604e29a640 1 --2- 172.21.15.84:0/1730419204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6048107860 0x7f6048107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.595+0000 7f604e29a640 1 -- 172.21.15.84:0/1730419204 >> 172.21.15.84:0/1730419204 conn(0x7f6048078c70 msgr2=0x7f6048079080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:44.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.595+0000 7f604e29a640 1 -- 172.21.15.84:0/1730419204 shutdown_connections 2024-05-22T23:06:44.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.595+0000 7f604e29a640 1 -- 172.21.15.84:0/1730419204 wait complete. 2024-05-22T23:06:44.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.595+0000 7f604e29a640 1 Processor -- start 2024-05-22T23:06:44.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.595+0000 7f604e29a640 1 -- start start 2024-05-22T23:06:44.602 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.596+0000 7f604e29a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6048101850 0x7f60481981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.603 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.596+0000 7f604e29a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6048102210 0x7f6048198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.603 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.596+0000 7f604e29a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6048107860 0x7f604819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.603 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.596+0000 7f604e29a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6048111c70 con 0x7f6048107860 2024-05-22T23:06:44.603 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.596+0000 7f604e29a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6048111af0 con 0x7f6048101850 2024-05-22T23:06:44.603 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.596+0000 7f604e29a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6048111df0 con 0x7f6048102210 2024-05-22T23:06:44.603 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.596+0000 7f6047fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6048101850 0x7f60481981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.603 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.596+0000 7f6047fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6048101850 0x7f60481981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:42708/0 (socket says 172.21.15.84:42708) 2024-05-22T23:06:44.603 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.596+0000 7f604c837640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6048107860 0x7f604819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.603 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.596+0000 7f6047fff640 1 -- 172.21.15.84:0/1797042444 learned_addr learned my addr 172.21.15.84:0/1797042444 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:44.603 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.596+0000 7f60477fe640 1 --2- 172.21.15.84:0/1797042444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6048102210 0x7f6048198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.603 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.596+0000 7f604c837640 1 -- 172.21.15.84:0/1797042444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6048102210 msgr2=0x7f6048198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:44.603 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.596+0000 7f604c837640 1 --2- 172.21.15.84:0/1797042444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6048102210 0x7f6048198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.604 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.596+0000 7f604c837640 1 -- 172.21.15.84:0/1797042444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6048101850 msgr2=0x7f60481981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:44.604 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.596+0000 7f604c837640 1 --2- 172.21.15.84:0/1797042444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6048101850 0x7f60481981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.604 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.596+0000 7f604c837640 1 -- 172.21.15.84:0/1797042444 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604819d1c0 con 0x7f6048107860 2024-05-22T23:06:44.604 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.596+0000 7f6047fff640 1 --2- 172.21.15.84:0/1797042444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6048101850 0x7f60481981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:06:44.604 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.596+0000 7f604c837640 1 --2- 172.21.15.84:0/1797042444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6048107860 0x7f604819ca90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f6040009fd0 tx=0x7f604000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:44.604 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.596+0000 7f60457fa640 1 -- 172.21.15.84:0/1797042444 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604000ce20 con 0x7f6048107860 2024-05-22T23:06:44.604 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.596+0000 7f60457fa640 1 -- 172.21.15.84:0/1797042444 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6040005020 con 0x7f6048107860 2024-05-22T23:06:44.604 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.596+0000 7f604e29a640 1 -- 172.21.15.84:0/1797042444 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604819d4b0 con 0x7f6048107860 2024-05-22T23:06:44.604 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.596+0000 7f60457fa640 1 -- 172.21.15.84:0/1797042444 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040010720 con 0x7f6048107860 2024-05-22T23:06:44.604 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.596+0000 7f604e29a640 1 -- 172.21.15.84:0/1797042444 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60481a4d40 con 0x7f6048107860 2024-05-22T23:06:44.610 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.598+0000 7f60457fa640 1 -- 172.21.15.84:0/1797042444 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6040005540 con 0x7f6048107860 2024-05-22T23:06:44.610 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.599+0000 7f6050fa4740 1 Processor -- start 2024-05-22T23:06:44.610 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.599+0000 7f6050fa4740 1 -- start start 2024-05-22T23:06:44.610 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.599+0000 7f6050fa4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56344268ef80 0x56344268f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.610 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.599+0000 7f6050fa4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56344268f8a0 0x563442699a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.599+0000 7f6050fa4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563442699f70 0x56344269c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.599+0000 7f6050fa4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5634425f3db0 con 0x56344268ef80 2024-05-22T23:06:44.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.599+0000 7f6050fa4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563442592400 con 0x56344268f8a0 2024-05-22T23:06:44.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.599+0000 7f6050fa4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5634425efd70 con 0x563442699f70 2024-05-22T23:06:44.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.599+0000 7f60457fa640 1 --2- 172.21.15.84:0/1797042444 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f60300a2d00 0x7f60300a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.599+0000 7f60477fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56344268ef80 0x56344268f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.599+0000 7f60477fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56344268ef80 0x56344268f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37844/0 (socket says 172.21.15.84:37844) 2024-05-22T23:06:44.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.599+0000 7f60477fe640 1 -- 172.21.15.84:0/2739447410 learned_addr learned my addr 172.21.15.84:0/2739447410 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:44.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.599+0000 7f6047fff640 1 --2- 172.21.15.84:0/2739447410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56344268f8a0 0x563442699a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.599+0000 7f604c837640 1 --2- 172.21.15.84:0/1797042444 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f60300a2d00 0x7f60300a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.599+0000 7f60477fe640 1 --2- 172.21.15.84:0/2739447410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563442699f70 0x56344269c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.599+0000 7f60457fa640 1 -- 172.21.15.84:0/1797042444 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f60400cfb60 con 0x7f6048107860 2024-05-22T23:06:44.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.599+0000 7f6047fff640 1 -- 172.21.15.84:0/2739447410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563442699f70 msgr2=0x56344269c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:44.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.599+0000 7f6047fff640 1 --2- 172.21.15.84:0/2739447410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563442699f70 0x56344269c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.599+0000 7f6047fff640 1 -- 172.21.15.84:0/2739447410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56344268ef80 msgr2=0x56344268f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:44.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.599+0000 7f6047fff640 1 --2- 172.21.15.84:0/2739447410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56344268ef80 0x56344268f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.599+0000 7f6047fff640 1 -- 172.21.15.84:0/2739447410 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634425842f0 con 0x56344268f8a0 2024-05-22T23:06:44.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.599+0000 7f60477fe640 1 --2- 172.21.15.84:0/2739447410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56344268ef80 0x56344268f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:06:44.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.600+0000 7f6047fff640 1 --2- 172.21.15.84:0/2739447410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56344268f8a0 0x563442699a30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6038010480 tx=0x7f603800bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b2e3371dfb08f43 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:44.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.600+0000 7f604c837640 1 --2- 172.21.15.84:0/1797042444 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f60300a2d00 0x7f60300a51c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f60400cd360 tx=0x7f6040091510 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:44.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.600+0000 7f602e7fc640 1 -- 172.21.15.84:0/2739447410 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038011820 con 0x56344268f8a0 2024-05-22T23:06:44.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.600+0000 7f602e7fc640 1 -- 172.21.15.84:0/2739447410 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f603801a380 con 0x56344268f8a0 2024-05-22T23:06:44.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.600+0000 7f602e7fc640 1 -- 172.21.15.84:0/2739447410 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603801b640 con 0x56344268f8a0 2024-05-22T23:06:44.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.601+0000 7f6050fa4740 1 -- 172.21.15.84:0/2739447410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56344268f8a0 msgr2=0x563442699a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:44.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.601+0000 7f6050fa4740 1 --2- 172.21.15.84:0/2739447410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56344268f8a0 0x563442699a30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6038010480 tx=0x7f603800bf90 comp rx=0 tx=0).stop 2024-05-22T23:06:44.612 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.601+0000 7f6050fa4740 1 -- 172.21.15.84:0/2739447410 shutdown_connections 2024-05-22T23:06:44.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.601+0000 7f6050fa4740 1 --2- 172.21.15.84:0/2739447410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563442699f70 0x56344269c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.601+0000 7f6050fa4740 1 --2- 172.21.15.84:0/2739447410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56344268f8a0 0x563442699a30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.601+0000 7f6050fa4740 1 --2- 172.21.15.84:0/2739447410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56344268ef80 0x56344268f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.601+0000 7f6050fa4740 1 -- 172.21.15.84:0/2739447410 >> 172.21.15.84:0/2739447410 conn(0x56344268b2a0 msgr2=0x56344268b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:44.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.601+0000 7f6050fa4740 1 -- 172.21.15.84:0/2739447410 shutdown_connections 2024-05-22T23:06:44.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.601+0000 7f6050fa4740 1 -- 172.21.15.84:0/2739447410 wait complete. 2024-05-22T23:06:44.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.601+0000 7f6050fa4740 1 -- start start 2024-05-22T23:06:44.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.601+0000 7f6050fa4740 1 Processor -- start 2024-05-22T23:06:44.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.601+0000 7f6050fa4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56344268ef80 0x563442696eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.601+0000 7f6050fa4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56344268f8a0 0x5634426973f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.601+0000 7f6050fa4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563442699f70 0x563442697930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.601+0000 7f6050fa4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56344269cd60 con 0x56344268ef80 2024-05-22T23:06:44.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.601+0000 7f6050fa4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5634425ae2e0 con 0x56344268f8a0 2024-05-22T23:06:44.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.601+0000 7f6050fa4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56344269cee0 con 0x563442699f70 2024-05-22T23:06:44.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.601+0000 7f6050fa4740 10 client.0 _ll_register_callbacks cb 0x5634425b7a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:06:44.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.602+0000 7f60477fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56344268ef80 0x563442696eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.602+0000 7f6047fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56344268f8a0 0x5634426973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.602+0000 7f60477fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56344268ef80 0x563442696eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37848/0 (socket says 172.21.15.84:37848) 2024-05-22T23:06:44.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.602+0000 7f60477fe640 1 -- 172.21.15.84:0/1722572706 learned_addr learned my addr 172.21.15.84:0/1722572706 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:44.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.602+0000 7f60477fe640 1 --2- 172.21.15.84:0/1722572706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563442699f70 0x563442697930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.602+0000 7f6047fff640 1 -- 172.21.15.84:0/1722572706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563442699f70 msgr2=0x563442697930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:44.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.602+0000 7f6047fff640 1 --2- 172.21.15.84:0/1722572706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563442699f70 0x563442697930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.602+0000 7f6047fff640 1 -- 172.21.15.84:0/1722572706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56344268ef80 msgr2=0x563442696eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:44.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.602+0000 7f6047fff640 1 --2- 172.21.15.84:0/1722572706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56344268ef80 0x563442696eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.602+0000 7f6047fff640 1 -- 172.21.15.84:0/1722572706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634423073f0 con 0x56344268f8a0 2024-05-22T23:06:44.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.602+0000 7f60477fe640 1 --2- 172.21.15.84:0/1722572706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56344268ef80 0x563442696eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:06:44.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.602+0000 7f6047fff640 1 --2- 172.21.15.84:0/1722572706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56344268f8a0 0x5634426973f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6038010450 tx=0x7f603801a760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4124d21a2c2952ef server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:44.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.602+0000 7f60277fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:06:44.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.602+0000 7f60277fe640 1 -- 172.21.15.84:0/1722572706 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603802b060 con 0x56344268f8a0 2024-05-22T23:06:44.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.602+0000 7f60277fe640 1 -- 172.21.15.84:0/1722572706 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f6038011830 con 0x56344268f8a0 2024-05-22T23:06:44.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.602+0000 7f6050fa4740 1 -- 172.21.15.84:0/1722572706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f603801a380 con 0x56344268f8a0 2024-05-22T23:06:44.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.602+0000 7f60277fe640 1 -- 172.21.15.84:0/1722572706 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f6038011a30 con 0x56344268f8a0 2024-05-22T23:06:44.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.602+0000 7f6050fa4740 10 client.25778 fetch_fsmap learned FSMap version 18 2024-05-22T23:06:44.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.602+0000 7f6050fa4740 1 -- 172.21.15.84:0/1722572706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5634423d17d0 con 0x56344268f8a0 2024-05-22T23:06:44.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.603+0000 7f60277fe640 1 -- 172.21.15.84:0/1722572706 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f603801a380 con 0x56344268f8a0 2024-05-22T23:06:44.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.603+0000 7f60277fe640 1 -- 172.21.15.84:0/1722572706 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f603801c960 con 0x56344268f8a0 2024-05-22T23:06:44.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.603+0000 7f6050fa4740 10 client.25778 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:06:44.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.603+0000 7f6050fa4740 10 client.25778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:06:44.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.603+0000 7f6050fa4740 20 client.25778 populate_metadata read hostname 'smithi084' 2024-05-22T23:06:44.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.603+0000 7f6050fa4740 1 --2- 172.21.15.84:0/1722572706 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5634426aa1b0 0x5634426ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.603+0000 7f6050fa4740 4 client.25778 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:06:44.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.603+0000 7f6050fa4740 1 -- 172.21.15.84:0/1722572706 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634426014e0 con 0x5634426aa1b0 2024-05-22T23:06:44.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.603+0000 7f60477fe640 1 --2- 172.21.15.84:0/1722572706 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5634426aa1b0 0x5634426ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.603+0000 7f60477fe640 1 --2- 172.21.15.84:0/1722572706 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5634426aa1b0 0x5634426ac5a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2376001ca22b4569 server_cookie=dff93288c2e82cd5 in_seq=0 out_seq=0 2024-05-22T23:06:44.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.603+0000 7f60277fe640 10 client.25778 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:44.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.605+0000 7f60277fe640 1 -- 172.21.15.84:0/1722572706 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5634426014e0 con 0x5634426aa1b0 2024-05-22T23:06:44.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.605+0000 7f60277fe640 10 client.25778 handle_command_reply: tid=0 2024-05-22T23:06:44.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.605+0000 7f6050fa4740 1 client.25778 shutdown 2024-05-22T23:06:44.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.605+0000 7f6050fa4740 1 -- 172.21.15.84:0/1722572706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56344268f8a0 msgr2=0x5634426973f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:44.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.605+0000 7f6050fa4740 1 --2- 172.21.15.84:0/1722572706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56344268f8a0 0x5634426973f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6038010450 tx=0x7f603801a760 comp rx=0 tx=0).stop 2024-05-22T23:06:44.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.605+0000 7f6050fa4740 1 -- 172.21.15.84:0/1722572706 shutdown_connections 2024-05-22T23:06:44.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.605+0000 7f6050fa4740 1 --2- 172.21.15.84:0/1722572706 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5634426aa1b0 0x5634426ac5a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.605+0000 7f6050fa4740 1 --2- 172.21.15.84:0/1722572706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563442699f70 0x563442697930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.605+0000 7f6050fa4740 1 --2- 172.21.15.84:0/1722572706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56344268f8a0 0x5634426973f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.605+0000 7f60277fe640 0 client.25778 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:44.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.605+0000 7f6050fa4740 1 --2- 172.21.15.84:0/1722572706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56344268ef80 0x563442696eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.605+0000 7f6050fa4740 1 -- 172.21.15.84:0/1722572706 >> 172.21.15.84:0/1722572706 conn(0x56344268b2a0 msgr2=0x563442690fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:44.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.605+0000 7f6050fa4740 1 -- 172.21.15.84:0/1722572706 shutdown_connections 2024-05-22T23:06:44.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.605+0000 7f6050fa4740 1 -- 172.21.15.84:0/1722572706 wait complete. 2024-05-22T23:06:44.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.605+0000 7f6050fa4740 20 client.25778 trim_cache size 0 max 16384 2024-05-22T23:06:44.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.619+0000 7f6050fa4740 1 Processor -- start 2024-05-22T23:06:44.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.619+0000 7f6050fa4740 1 -- start start 2024-05-22T23:06:44.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.619+0000 7f6050fa4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56344268ef80 0x5634426995e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.619+0000 7f6050fa4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56344268f8a0 0x5634426a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.619+0000 7f6050fa4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563442699f70 0x563442696050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.619+0000 7f6050fa4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f60481ab670 con 0x563442699f70 2024-05-22T23:06:44.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.619+0000 7f6050fa4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f60481adef0 con 0x56344268ef80 2024-05-22T23:06:44.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.619+0000 7f6050fa4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f60481abce0 con 0x56344268f8a0 2024-05-22T23:06:44.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.619+0000 7f60477fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563442699f70 0x563442696050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.619+0000 7f60477fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563442699f70 0x563442696050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37854/0 (socket says 172.21.15.84:37854) 2024-05-22T23:06:44.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.619+0000 7f60477fe640 1 -- 172.21.15.84:0/470133704 learned_addr learned my addr 172.21.15.84:0/470133704 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:44.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.619+0000 7f60477fe640 1 --2- 172.21.15.84:0/470133704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56344268ef80 0x5634426995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.619+0000 7f6047fff640 1 --2- 172.21.15.84:0/470133704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56344268f8a0 0x5634426a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.619+0000 7f60477fe640 1 -- 172.21.15.84:0/470133704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56344268f8a0 msgr2=0x5634426a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:44.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.619+0000 7f60477fe640 1 --2- 172.21.15.84:0/470133704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56344268f8a0 0x5634426a5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.619+0000 7f60477fe640 1 -- 172.21.15.84:0/470133704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56344268ef80 msgr2=0x5634426995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:44.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.619+0000 7f60477fe640 1 --2- 172.21.15.84:0/470133704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56344268ef80 0x5634426995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.619+0000 7f60477fe640 1 -- 172.21.15.84:0/470133704 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563442695440 con 0x563442699f70 2024-05-22T23:06:44.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.619+0000 7f6047fff640 1 --2- 172.21.15.84:0/470133704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56344268f8a0 0x5634426a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:06:44.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.620+0000 7f60477fe640 1 --2- 172.21.15.84:0/470133704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563442699f70 0x563442696050 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f603c000c00 tx=0x7f603c03f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cefe99621e18e124 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:44.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.620+0000 7f602dffb640 1 -- 172.21.15.84:0/470133704 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c035040 con 0x563442699f70 2024-05-22T23:06:44.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.620+0000 7f602dffb640 1 -- 172.21.15.84:0/470133704 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f603c005510 con 0x563442699f70 2024-05-22T23:06:44.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.620+0000 7f602dffb640 1 -- 172.21.15.84:0/470133704 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c0049d0 con 0x563442699f70 2024-05-22T23:06:44.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.620+0000 7f6050fa4740 1 -- 172.21.15.84:0/470133704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563442699f70 msgr2=0x563442696050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:44.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.620+0000 7f6050fa4740 1 --2- 172.21.15.84:0/470133704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563442699f70 0x563442696050 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f603c000c00 tx=0x7f603c03f040 comp rx=0 tx=0).stop 2024-05-22T23:06:44.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.620+0000 7f6050fa4740 1 -- 172.21.15.84:0/470133704 shutdown_connections 2024-05-22T23:06:44.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.620+0000 7f6050fa4740 1 --2- 172.21.15.84:0/470133704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563442699f70 0x563442696050 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.620+0000 7f6050fa4740 1 --2- 172.21.15.84:0/470133704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56344268f8a0 0x5634426a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.620+0000 7f6050fa4740 1 --2- 172.21.15.84:0/470133704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56344268ef80 0x5634426995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.620+0000 7f6050fa4740 1 -- 172.21.15.84:0/470133704 >> 172.21.15.84:0/470133704 conn(0x56344268b2a0 msgr2=0x563442696e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:44.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.620+0000 7f6050fa4740 1 -- 172.21.15.84:0/470133704 shutdown_connections 2024-05-22T23:06:44.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.620+0000 7f6050fa4740 1 -- 172.21.15.84:0/470133704 wait complete. 2024-05-22T23:06:44.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.621+0000 7f6050fa4740 1 -- start start 2024-05-22T23:06:44.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.621+0000 7f6050fa4740 1 Processor -- start 2024-05-22T23:06:44.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.621+0000 7f6050fa4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56344268ef80 0x563442693eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.621+0000 7f6050fa4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56344268f8a0 0x5634426943f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.621+0000 7f6050fa4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563442699f70 0x563442694970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.621+0000 7f6050fa4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f60481ac3b0 con 0x563442699f70 2024-05-22T23:06:44.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.621+0000 7f6050fa4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f60481ad820 con 0x56344268f8a0 2024-05-22T23:06:44.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.621+0000 7f6050fa4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f60481aca80 con 0x56344268ef80 2024-05-22T23:06:44.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.621+0000 7f6050fa4740 10 client.0 _ll_register_callbacks cb 0x7f603c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:06:44.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.621+0000 7f60477fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563442699f70 0x563442694970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.621+0000 7f60477fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563442699f70 0x563442694970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37858/0 (socket says 172.21.15.84:37858) 2024-05-22T23:06:44.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.621+0000 7f60477fe640 1 -- 172.21.15.84:0/659685237 learned_addr learned my addr 172.21.15.84:0/659685237 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:44.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.621+0000 7f6047fff640 1 --2- 172.21.15.84:0/659685237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56344268f8a0 0x5634426943f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.621+0000 7f60477fe640 1 --2- 172.21.15.84:0/659685237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56344268ef80 0x563442693eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.621+0000 7f6047fff640 1 -- 172.21.15.84:0/659685237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56344268ef80 msgr2=0x563442693eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:44.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.621+0000 7f6047fff640 1 --2- 172.21.15.84:0/659685237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56344268ef80 0x563442693eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.622+0000 7f6047fff640 1 -- 172.21.15.84:0/659685237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563442699f70 msgr2=0x563442694970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:44.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.622+0000 7f6047fff640 1 --2- 172.21.15.84:0/659685237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563442699f70 0x563442694970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.622+0000 7f6047fff640 1 -- 172.21.15.84:0/659685237 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56344270b710 con 0x56344268f8a0 2024-05-22T23:06:44.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.622+0000 7f60477fe640 1 --2- 172.21.15.84:0/659685237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563442699f70 0x563442694970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:06:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.622+0000 7f6047fff640 1 --2- 172.21.15.84:0/659685237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56344268f8a0 0x5634426943f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f603800b6c0 tx=0x7f603800be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca9b88e2280497a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.622+0000 7f60277fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:06:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.622+0000 7f60277fe640 1 -- 172.21.15.84:0/659685237 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603802b070 con 0x56344268f8a0 2024-05-22T23:06:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.622+0000 7f6050fa4740 1 -- 172.21.15.84:0/659685237 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f603c005510 con 0x56344268f8a0 2024-05-22T23:06:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.622+0000 7f60277fe640 1 -- 172.21.15.84:0/659685237 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f603801a9e0 con 0x56344268f8a0 2024-05-22T23:06:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.622+0000 7f60277fe640 1 -- 172.21.15.84:0/659685237 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f603803e050 con 0x56344268f8a0 2024-05-22T23:06:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.622+0000 7f6050fa4740 10 client.25784 fetch_fsmap learned FSMap version 18 2024-05-22T23:06:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.622+0000 7f6050fa4740 1 -- 172.21.15.84:0/659685237 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x563442719250 con 0x56344268f8a0 2024-05-22T23:06:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.622+0000 7f60277fe640 1 -- 172.21.15.84:0/659685237 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f603c005510 con 0x56344268f8a0 2024-05-22T23:06:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.622+0000 7f60277fe640 1 -- 172.21.15.84:0/659685237 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f6038004680 con 0x56344268f8a0 2024-05-22T23:06:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.622+0000 7f6050fa4740 10 client.25784 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:06:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.622+0000 7f6050fa4740 10 client.25784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:06:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.622+0000 7f6050fa4740 20 client.25784 populate_metadata read hostname 'smithi084' 2024-05-22T23:06:44.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.622+0000 7f6050fa4740 1 --2- 172.21.15.84:0/659685237 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5634427198d0 0x56344271bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:44.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.622+0000 7f6050fa4740 4 client.25784 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-22T23:06:44.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.622+0000 7f6050fa4740 1 -- 172.21.15.84:0/659685237 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5634425fb830 con 0x5634427198d0 2024-05-22T23:06:44.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.623+0000 7f60477fe640 1 --2- 172.21.15.84:0/659685237 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5634427198d0 0x56344271bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:44.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.623+0000 7f60277fe640 10 client.25784 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:44.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.623+0000 7f60477fe640 1 --2- 172.21.15.84:0/659685237 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5634427198d0 0x56344271bcf0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=877c3e643dd98b6c server_cookie=54f18d0028fb1fba in_seq=0 out_seq=0 2024-05-22T23:06:44.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.624+0000 7f60277fe640 1 -- 172.21.15.84:0/659685237 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5634425fb830 con 0x5634427198d0 2024-05-22T23:06:44.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.624+0000 7f60277fe640 10 client.25784 handle_command_reply: tid=0 2024-05-22T23:06:44.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.624+0000 7f6050fa4740 1 client.25784 shutdown 2024-05-22T23:06:44.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.624+0000 7f6050fa4740 1 -- 172.21.15.84:0/659685237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56344268f8a0 msgr2=0x5634426943f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:44.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.624+0000 7f6050fa4740 1 --2- 172.21.15.84:0/659685237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56344268f8a0 0x5634426943f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f603800b6c0 tx=0x7f603800be90 comp rx=0 tx=0).stop 2024-05-22T23:06:44.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.624+0000 7f6050fa4740 1 -- 172.21.15.84:0/659685237 shutdown_connections 2024-05-22T23:06:44.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.624+0000 7f6050fa4740 1 --2- 172.21.15.84:0/659685237 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5634427198d0 0x56344271bcf0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.624+0000 7f6050fa4740 1 --2- 172.21.15.84:0/659685237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563442699f70 0x563442694970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.624+0000 7f60277fe640 0 client.25784 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:44.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.624+0000 7f6050fa4740 1 --2- 172.21.15.84:0/659685237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56344268f8a0 0x5634426943f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.624+0000 7f6050fa4740 1 --2- 172.21.15.84:0/659685237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56344268ef80 0x563442693eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.624+0000 7f6050fa4740 1 -- 172.21.15.84:0/659685237 >> 172.21.15.84:0/659685237 conn(0x56344268b2a0 msgr2=0x563442695ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:44.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.624+0000 7f6050fa4740 1 -- 172.21.15.84:0/659685237 shutdown_connections 2024-05-22T23:06:44.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.624+0000 7f6050fa4740 1 -- 172.21.15.84:0/659685237 wait complete. 2024-05-22T23:06:44.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.624+0000 7f6050fa4740 20 client.25784 trim_cache size 0 max 16384 2024-05-22T23:06:44.635 INFO:teuthology.orchestra.run.smithi084.stdout:[] 2024-05-22T23:06:44.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.625+0000 7f604e29a640 1 -- 172.21.15.84:0/1797042444 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f60300a2d00 msgr2=0x7f60300a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:44.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.625+0000 7f604e29a640 1 --2- 172.21.15.84:0/1797042444 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f60300a2d00 0x7f60300a51c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f60400cd360 tx=0x7f6040091510 comp rx=0 tx=0).stop 2024-05-22T23:06:44.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.625+0000 7f604e29a640 1 -- 172.21.15.84:0/1797042444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6048107860 msgr2=0x7f604819ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:44.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.625+0000 7f604e29a640 1 --2- 172.21.15.84:0/1797042444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6048107860 0x7f604819ca90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f6040009fd0 tx=0x7f604000ef10 comp rx=0 tx=0).stop 2024-05-22T23:06:44.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.625+0000 7f604e29a640 1 -- 172.21.15.84:0/1797042444 shutdown_connections 2024-05-22T23:06:44.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.625+0000 7f604e29a640 1 --2- 172.21.15.84:0/1797042444 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f60300a2d00 0x7f60300a51c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.625+0000 7f604e29a640 1 --2- 172.21.15.84:0/1797042444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6048107860 0x7f604819ca90 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.625+0000 7f604e29a640 1 --2- 172.21.15.84:0/1797042444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6048102210 0x7f6048198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.625+0000 7f604e29a640 1 --2- 172.21.15.84:0/1797042444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6048101850 0x7f60481981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:44.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.625+0000 7f604e29a640 1 -- 172.21.15.84:0/1797042444 >> 172.21.15.84:0/1797042444 conn(0x7f6048078c70 msgr2=0x7f6048104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:44.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.625+0000 7f604e29a640 1 -- 172.21.15.84:0/1797042444 shutdown_connections 2024-05-22T23:06:44.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:44.625+0000 7f604e29a640 1 -- 172.21.15.84:0/1797042444 wait complete. 2024-05-22T23:06:44.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:44 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/725007902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:06:44.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:44 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/725007902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:06:44.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:44 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/725007902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:06:45.434 INFO:tasks.workunit.client.0.smithi084.stdout: 1 608135 15.36 MB/sec execute 91 sec latency 98.068 ms 2024-05-22T23:06:45.642 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-22T23:06:45.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:06:45.643 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-22T23:06:45.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.753+0000 7effb6407640 1 -- 172.21.15.84:0/3916546580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effb0100740 msgr2=0x7effb0100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:45.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.753+0000 7effb6407640 1 --2- 172.21.15.84:0/3916546580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effb0100740 0x7effb0100bc0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7eff98009930 tx=0x7eff9802a420 comp rx=0 tx=0).stop 2024-05-22T23:06:45.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.753+0000 7effb6407640 1 -- 172.21.15.84:0/3916546580 shutdown_connections 2024-05-22T23:06:45.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.753+0000 7effb6407640 1 --2- 172.21.15.84:0/3916546580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effb0101100 0x7effb010d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.753+0000 7effb6407640 1 --2- 172.21.15.84:0/3916546580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effb0100740 0x7effb0100bc0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.753+0000 7effb6407640 1 --2- 172.21.15.84:0/3916546580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effb0106750 0x7effb0106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.753+0000 7effb6407640 1 -- 172.21.15.84:0/3916546580 >> 172.21.15.84:0/3916546580 conn(0x7effb0077870 msgr2=0x7effb0077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:45.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.753+0000 7effb6407640 1 -- 172.21.15.84:0/3916546580 shutdown_connections 2024-05-22T23:06:45.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.753+0000 7effb6407640 1 -- 172.21.15.84:0/3916546580 wait complete. 2024-05-22T23:06:45.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.754+0000 7effb6407640 1 Processor -- start 2024-05-22T23:06:45.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.754+0000 7effb6407640 1 -- start start 2024-05-22T23:06:45.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.754+0000 7effb6407640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effb0100740 0x7effb0199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.754+0000 7effb6407640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effb0101100 0x7effb0199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.754+0000 7effb6407640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effb0106750 0x7effb01a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.754+0000 7effb6407640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7effb0110b60 con 0x7effb0106750 2024-05-22T23:06:45.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.754+0000 7effb6407640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7effb01109e0 con 0x7effb0101100 2024-05-22T23:06:45.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.754+0000 7effb6407640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7effb0110ce0 con 0x7effb0100740 2024-05-22T23:06:45.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.754+0000 7effb49a4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effb0106750 0x7effb01a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.754+0000 7effa7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effb0101100 0x7effb0199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.754+0000 7effb49a4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effb0106750 0x7effb01a4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37884/0 (socket says 172.21.15.84:37884) 2024-05-22T23:06:45.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.754+0000 7effb49a4640 1 -- 172.21.15.84:0/2589735393 learned_addr learned my addr 172.21.15.84:0/2589735393 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:45.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.754+0000 7effaffff640 1 --2- 172.21.15.84:0/2589735393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effb0100740 0x7effb0199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.754+0000 7effb49a4640 1 -- 172.21.15.84:0/2589735393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effb0100740 msgr2=0x7effb0199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:45.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.754+0000 7effb49a4640 1 --2- 172.21.15.84:0/2589735393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effb0100740 0x7effb0199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.754+0000 7effb49a4640 1 -- 172.21.15.84:0/2589735393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effb0101100 msgr2=0x7effb0199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:45.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.754+0000 7effb49a4640 1 --2- 172.21.15.84:0/2589735393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effb0101100 0x7effb0199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.754+0000 7effb49a4640 1 -- 172.21.15.84:0/2589735393 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb01a45d0 con 0x7effb0106750 2024-05-22T23:06:45.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.754+0000 7effa7fff640 1 --2- 172.21.15.84:0/2589735393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effb0101100 0x7effb0199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:06:45.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.755+0000 7effb49a4640 1 --2- 172.21.15.84:0/2589735393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effb0106750 0x7effb01a4090 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7effa800eb10 tx=0x7effa800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:45.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.755+0000 7effadffb640 1 -- 172.21.15.84:0/2589735393 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8004b20 con 0x7effb0106750 2024-05-22T23:06:45.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.755+0000 7effb6407640 1 -- 172.21.15.84:0/2589735393 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb01a47a0 con 0x7effb0106750 2024-05-22T23:06:45.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.755+0000 7effadffb640 1 -- 172.21.15.84:0/2589735393 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa8004cc0 con 0x7effb0106750 2024-05-22T23:06:45.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.755+0000 7effadffb640 1 -- 172.21.15.84:0/2589735393 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8010760 con 0x7effb0106750 2024-05-22T23:06:45.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.755+0000 7effb6407640 1 -- 172.21.15.84:0/2589735393 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb01a4c70 con 0x7effb0106750 2024-05-22T23:06:45.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.757+0000 7effadffb640 1 -- 172.21.15.84:0/2589735393 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7effa8005020 con 0x7effb0106750 2024-05-22T23:06:45.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.758+0000 7effb91f2740 1 Processor -- start 2024-05-22T23:06:45.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.758+0000 7effb91f2740 1 -- start start 2024-05-22T23:06:45.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.758+0000 7effb91f2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde4fc50 0x55d1bde50030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.758+0000 7effb91f2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde50570 0x55d1bde5a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.758+0000 7effb91f2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1bde5ac40 0x55d1bde5d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.758+0000 7effb91f2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d1bddb3a40 con 0x55d1bde4fc50 2024-05-22T23:06:45.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.758+0000 7effb91f2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d1bdd52440 con 0x55d1bde50570 2024-05-22T23:06:45.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.758+0000 7effb91f2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d1bddafa00 con 0x55d1bde5ac40 2024-05-22T23:06:45.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.758+0000 7effadffb640 1 --2- 172.21.15.84:0/2589735393 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7eff9c0a2d00 0x7eff9c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.758+0000 7effa7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde4fc50 0x55d1bde50030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.758+0000 7effaffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde50570 0x55d1bde5a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.758+0000 7effa7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde4fc50 0x55d1bde50030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37892/0 (socket says 172.21.15.84:37892) 2024-05-22T23:06:45.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.758+0000 7effa7fff640 1 -- 172.21.15.84:0/401889933 learned_addr learned my addr 172.21.15.84:0/401889933 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:45.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.758+0000 7effaffff640 1 --2- 172.21.15.84:0/401889933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde50570 0x55d1bde5a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:42778/0 (socket says 172.21.15.84:42778) 2024-05-22T23:06:45.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.758+0000 7effb49a4640 1 --2- 172.21.15.84:0/2589735393 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7eff9c0a2d00 0x7eff9c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.758+0000 7effa7fff640 1 --2- 172.21.15.84:0/401889933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1bde5ac40 0x55d1bde5d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.759+0000 7effaffff640 1 -- 172.21.15.84:0/401889933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1bde5ac40 msgr2=0x55d1bde5d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:45.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.759+0000 7effaffff640 1 --2- 172.21.15.84:0/401889933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1bde5ac40 0x55d1bde5d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.759+0000 7effaffff640 1 -- 172.21.15.84:0/401889933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde4fc50 msgr2=0x55d1bde50030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:45.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.759+0000 7effaffff640 1 --2- 172.21.15.84:0/401889933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde4fc50 0x55d1bde50030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.759+0000 7effaffff640 1 -- 172.21.15.84:0/401889933 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1bdd5c2d0 con 0x55d1bde50570 2024-05-22T23:06:45.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.759+0000 7effadffb640 1 -- 172.21.15.84:0/2589735393 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7effa80cfa00 con 0x7effb0106750 2024-05-22T23:06:45.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.759+0000 7effaffff640 1 --2- 172.21.15.84:0/401889933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde50570 0x55d1bde5a700 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7effa0009c00 tx=0x7effa0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f38f2a59aa11d656 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:45.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.759+0000 7effb49a4640 1 --2- 172.21.15.84:0/2589735393 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7eff9c0a2d00 0x7eff9c0a51c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7effa8062fd0 tx=0x7effa806ef90 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:45.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.759+0000 7effa67fc640 1 -- 172.21.15.84:0/401889933 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0004050 con 0x55d1bde50570 2024-05-22T23:06:45.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.759+0000 7effa67fc640 1 -- 172.21.15.84:0/401889933 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa0005500 con 0x55d1bde50570 2024-05-22T23:06:45.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.759+0000 7effa67fc640 1 -- 172.21.15.84:0/401889933 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0004d60 con 0x55d1bde50570 2024-05-22T23:06:45.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.760+0000 7effb91f2740 1 -- 172.21.15.84:0/401889933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde50570 msgr2=0x55d1bde5a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:45.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.760+0000 7effb91f2740 1 --2- 172.21.15.84:0/401889933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde50570 0x55d1bde5a700 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7effa0009c00 tx=0x7effa0011810 comp rx=0 tx=0).stop 2024-05-22T23:06:45.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.760+0000 7effb91f2740 1 -- 172.21.15.84:0/401889933 shutdown_connections 2024-05-22T23:06:45.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.760+0000 7effb91f2740 1 --2- 172.21.15.84:0/401889933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1bde5ac40 0x55d1bde5d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.760+0000 7effb91f2740 1 --2- 172.21.15.84:0/401889933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde50570 0x55d1bde5a700 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.760+0000 7effb91f2740 1 --2- 172.21.15.84:0/401889933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde4fc50 0x55d1bde50030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.760+0000 7effb91f2740 1 -- 172.21.15.84:0/401889933 >> 172.21.15.84:0/401889933 conn(0x55d1bde4bf70 msgr2=0x55d1bde4c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:45.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.760+0000 7effb91f2740 1 -- 172.21.15.84:0/401889933 shutdown_connections 2024-05-22T23:06:45.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.760+0000 7effb91f2740 1 -- 172.21.15.84:0/401889933 wait complete. 2024-05-22T23:06:45.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.760+0000 7effb91f2740 1 -- start start 2024-05-22T23:06:45.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.760+0000 7effb91f2740 1 Processor -- start 2024-05-22T23:06:45.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.760+0000 7effb91f2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde4fc50 0x55d1bde57b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.760+0000 7effb91f2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1bde50570 0x55d1bde580d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.761+0000 7effb91f2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde5ac40 0x55d1bde58610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.761+0000 7effb91f2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d1bde5da30 con 0x55d1bde5ac40 2024-05-22T23:06:45.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.761+0000 7effb91f2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d1bdd3d5f0 con 0x55d1bde4fc50 2024-05-22T23:06:45.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.761+0000 7effb91f2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d1bde5dbb0 con 0x55d1bde50570 2024-05-22T23:06:45.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.761+0000 7effb91f2740 10 client.0 _ll_register_callbacks cb 0x55d1bdd38cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:06:45.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.761+0000 7effa7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde5ac40 0x55d1bde58610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.761+0000 7effa7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde5ac40 0x55d1bde58610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37898/0 (socket says 172.21.15.84:37898) 2024-05-22T23:06:45.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.761+0000 7effa7fff640 1 -- 172.21.15.84:0/606959000 learned_addr learned my addr 172.21.15.84:0/606959000 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:45.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.761+0000 7effa7fff640 1 --2- 172.21.15.84:0/606959000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde4fc50 0x55d1bde57b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.761+0000 7effaffff640 1 --2- 172.21.15.84:0/606959000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1bde50570 0x55d1bde580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.761+0000 7effa7fff640 1 -- 172.21.15.84:0/606959000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1bde50570 msgr2=0x55d1bde580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:45.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.761+0000 7effa7fff640 1 --2- 172.21.15.84:0/606959000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1bde50570 0x55d1bde580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.761+0000 7effa7fff640 1 -- 172.21.15.84:0/606959000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde4fc50 msgr2=0x55d1bde57b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:45.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.761+0000 7effa7fff640 1 --2- 172.21.15.84:0/606959000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde4fc50 0x55d1bde57b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.761+0000 7effa7fff640 1 -- 172.21.15.84:0/606959000 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1bdb8f7e0 con 0x55d1bde5ac40 2024-05-22T23:06:45.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.761+0000 7effa7fff640 1 --2- 172.21.15.84:0/606959000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde5ac40 0x55d1bde58610 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7eff98009900 tx=0x7eff980058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a617048dccb730 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:45.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.761+0000 7eff8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:06:45.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.761+0000 7eff8f7fe640 1 -- 172.21.15.84:0/606959000 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98035040 con 0x55d1bde5ac40 2024-05-22T23:06:45.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.761+0000 7effb91f2740 1 -- 172.21.15.84:0/606959000 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effa0005500 con 0x55d1bde5ac40 2024-05-22T23:06:45.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.761+0000 7eff8f7fe640 1 -- 172.21.15.84:0/606959000 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7eff980043b0 con 0x55d1bde5ac40 2024-05-22T23:06:45.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.762+0000 7eff8f7fe640 1 -- 172.21.15.84:0/606959000 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7eff98005020 con 0x55d1bde5ac40 2024-05-22T23:06:45.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.762+0000 7effb91f2740 10 client.16224 fetch_fsmap learned FSMap version 18 2024-05-22T23:06:45.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.762+0000 7effb91f2740 1 -- 172.21.15.84:0/606959000 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55d1bdbeff70 con 0x55d1bde5ac40 2024-05-22T23:06:45.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.762+0000 7eff8f7fe640 1 -- 172.21.15.84:0/606959000 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa0005500 con 0x55d1bde5ac40 2024-05-22T23:06:45.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.762+0000 7eff8f7fe640 1 -- 172.21.15.84:0/606959000 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7eff9804a030 con 0x55d1bde5ac40 2024-05-22T23:06:45.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.762+0000 7effb91f2740 10 client.16224 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:06:45.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.762+0000 7effb91f2740 10 client.16224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:06:45.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.762+0000 7effb91f2740 20 client.16224 populate_metadata read hostname 'smithi084' 2024-05-22T23:06:45.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.762+0000 7effb91f2740 1 --2- 172.21.15.84:0/606959000 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d1bde6b2f0 0x55d1bde6d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.762+0000 7effb91f2740 4 client.16224 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:06:45.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.762+0000 7effb91f2740 1 -- 172.21.15.84:0/606959000 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1bddc7460 con 0x55d1bde6b2f0 2024-05-22T23:06:45.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.762+0000 7effaffff640 1 --2- 172.21.15.84:0/606959000 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d1bde6b2f0 0x55d1bde6d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.763+0000 7eff8f7fe640 10 client.16224 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:45.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.763+0000 7effaffff640 1 --2- 172.21.15.84:0/606959000 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d1bde6b2f0 0x55d1bde6d740 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26dc62df024d5c9f server_cookie=41195a2aca791928 in_seq=0 out_seq=0 2024-05-22T23:06:45.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.765+0000 7eff8f7fe640 1 -- 172.21.15.84:0/606959000 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d1bddc7460 con 0x55d1bde6b2f0 2024-05-22T23:06:45.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.765+0000 7eff8f7fe640 10 client.16224 handle_command_reply: tid=0 2024-05-22T23:06:45.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.765+0000 7effb91f2740 1 client.16224 shutdown 2024-05-22T23:06:45.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.765+0000 7effb91f2740 1 -- 172.21.15.84:0/606959000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde5ac40 msgr2=0x55d1bde58610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:45.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.765+0000 7effb91f2740 1 --2- 172.21.15.84:0/606959000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde5ac40 0x55d1bde58610 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7eff98009900 tx=0x7eff980058d0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.765+0000 7effb91f2740 1 -- 172.21.15.84:0/606959000 shutdown_connections 2024-05-22T23:06:45.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.765+0000 7effb91f2740 1 --2- 172.21.15.84:0/606959000 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d1bde6b2f0 0x55d1bde6d740 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.765+0000 7effb91f2740 1 --2- 172.21.15.84:0/606959000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde5ac40 0x55d1bde58610 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.765+0000 7effb91f2740 1 --2- 172.21.15.84:0/606959000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1bde50570 0x55d1bde580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.765+0000 7eff8f7fe640 0 client.16224 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:45.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.765+0000 7effb91f2740 1 --2- 172.21.15.84:0/606959000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde4fc50 0x55d1bde57b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.765+0000 7effb91f2740 1 -- 172.21.15.84:0/606959000 >> 172.21.15.84:0/606959000 conn(0x55d1bde4bf70 msgr2=0x55d1bde4b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:45.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.765+0000 7effb91f2740 1 -- 172.21.15.84:0/606959000 shutdown_connections 2024-05-22T23:06:45.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.765+0000 7effb91f2740 1 -- 172.21.15.84:0/606959000 wait complete. 2024-05-22T23:06:45.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.766+0000 7effb91f2740 20 client.16224 trim_cache size 0 max 16384 2024-05-22T23:06:45.791 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.784+0000 7effb91f2740 1 Processor -- start 2024-05-22T23:06:45.791 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.784+0000 7effb91f2740 1 -- start start 2024-05-22T23:06:45.791 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.785+0000 7effb91f2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde4fc50 0x55d1bde668f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.791 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.785+0000 7effb91f2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde50570 0x55d1bde51ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.785+0000 7effb91f2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1bde5ac40 0x55d1bde73010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.785+0000 7effb91f2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7effb01ab570 con 0x55d1bde50570 2024-05-22T23:06:45.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.785+0000 7effb91f2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7effb01addf0 con 0x55d1bde4fc50 2024-05-22T23:06:45.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.785+0000 7effb91f2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7effb01abbe0 con 0x55d1bde5ac40 2024-05-22T23:06:45.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.785+0000 7effaffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde50570 0x55d1bde51ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.785+0000 7effa7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde4fc50 0x55d1bde668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.785+0000 7effaffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde50570 0x55d1bde51ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37910/0 (socket says 172.21.15.84:37910) 2024-05-22T23:06:45.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.785+0000 7effaffff640 1 -- 172.21.15.84:0/2621096657 learned_addr learned my addr 172.21.15.84:0/2621096657 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:45.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.785+0000 7effa7fff640 1 --2- 172.21.15.84:0/2621096657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1bde5ac40 0x55d1bde73010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.785+0000 7effaffff640 1 -- 172.21.15.84:0/2621096657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1bde5ac40 msgr2=0x55d1bde73010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:45.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.785+0000 7effaffff640 1 --2- 172.21.15.84:0/2621096657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1bde5ac40 0x55d1bde73010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.785+0000 7effaffff640 1 -- 172.21.15.84:0/2621096657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde4fc50 msgr2=0x55d1bde668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:45.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.785+0000 7effaffff640 1 --2- 172.21.15.84:0/2621096657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde4fc50 0x55d1bde668f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.785+0000 7effaffff640 1 -- 172.21.15.84:0/2621096657 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1bda87170 con 0x55d1bde50570 2024-05-22T23:06:45.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.785+0000 7effa7fff640 1 --2- 172.21.15.84:0/2621096657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde4fc50 0x55d1bde668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:06:45.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.785+0000 7effaffff640 1 --2- 172.21.15.84:0/2621096657 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde50570 0x55d1bde51ab0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7effa0009230 tx=0x7effa00049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22224960e9763db4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:45.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.785+0000 7effa5ffb640 1 -- 172.21.15.84:0/2621096657 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0004240 con 0x55d1bde50570 2024-05-22T23:06:45.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.785+0000 7effa5ffb640 1 -- 172.21.15.84:0/2621096657 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa00043e0 con 0x55d1bde50570 2024-05-22T23:06:45.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.785+0000 7effa5ffb640 1 -- 172.21.15.84:0/2621096657 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa00038e0 con 0x55d1bde50570 2024-05-22T23:06:45.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.786+0000 7effb91f2740 1 -- 172.21.15.84:0/2621096657 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde50570 msgr2=0x55d1bde51ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:45.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.786+0000 7effb91f2740 1 --2- 172.21.15.84:0/2621096657 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde50570 0x55d1bde51ab0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7effa0009230 tx=0x7effa00049e0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.786+0000 7effb91f2740 1 -- 172.21.15.84:0/2621096657 shutdown_connections 2024-05-22T23:06:45.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.786+0000 7effb91f2740 1 --2- 172.21.15.84:0/2621096657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1bde5ac40 0x55d1bde73010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.786+0000 7effb91f2740 1 --2- 172.21.15.84:0/2621096657 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde50570 0x55d1bde51ab0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.786+0000 7effb91f2740 1 --2- 172.21.15.84:0/2621096657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde4fc50 0x55d1bde668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.794 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.786+0000 7effb91f2740 1 -- 172.21.15.84:0/2621096657 >> 172.21.15.84:0/2621096657 conn(0x55d1bde4bf70 msgr2=0x55d1bde513a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:45.794 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.786+0000 7effb91f2740 1 -- 172.21.15.84:0/2621096657 shutdown_connections 2024-05-22T23:06:45.794 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.786+0000 7effb91f2740 1 -- 172.21.15.84:0/2621096657 wait complete. 2024-05-22T23:06:45.794 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.787+0000 7effb91f2740 1 -- start start 2024-05-22T23:06:45.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.787+0000 7effb91f2740 1 Processor -- start 2024-05-22T23:06:45.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.787+0000 7effb91f2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde4fc50 0x55d1bde5ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.787+0000 7effb91f2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1bde50570 0x55d1bde5ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.787+0000 7effb91f2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde5ac40 0x55d1bde5f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.787+0000 7effb91f2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7effb01ac2b0 con 0x55d1bde5ac40 2024-05-22T23:06:45.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.787+0000 7effb91f2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7effb01ad720 con 0x55d1bde4fc50 2024-05-22T23:06:45.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.787+0000 7effb91f2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7effb01ac980 con 0x55d1bde50570 2024-05-22T23:06:45.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.787+0000 7effb91f2740 10 client.0 _ll_register_callbacks cb 0x7eff98009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:06:45.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.787+0000 7effa7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde4fc50 0x55d1bde5ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.787+0000 7effa7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde4fc50 0x55d1bde5ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:42818/0 (socket says 172.21.15.84:42818) 2024-05-22T23:06:45.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.787+0000 7effa7fff640 1 -- 172.21.15.84:0/3136950217 learned_addr learned my addr 172.21.15.84:0/3136950217 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:45.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.787+0000 7effa7fff640 1 --2- 172.21.15.84:0/3136950217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde5ac40 0x55d1bde5f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.787+0000 7effaffff640 1 --2- 172.21.15.84:0/3136950217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1bde50570 0x55d1bde5ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.787+0000 7effa7fff640 1 -- 172.21.15.84:0/3136950217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1bde50570 msgr2=0x55d1bde5ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:45.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.787+0000 7effa7fff640 1 --2- 172.21.15.84:0/3136950217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1bde50570 0x55d1bde5ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.787+0000 7effa7fff640 1 -- 172.21.15.84:0/3136950217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde5ac40 msgr2=0x55d1bde5f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:45.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.787+0000 7effa7fff640 1 --2- 172.21.15.84:0/3136950217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde5ac40 0x55d1bde5f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.787+0000 7effa7fff640 1 -- 172.21.15.84:0/3136950217 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1bdd7c930 con 0x55d1bde4fc50 2024-05-22T23:06:45.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.788+0000 7effa7fff640 1 --2- 172.21.15.84:0/3136950217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde4fc50 0x55d1bde5ea10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7eff980059d0 tx=0x7eff98005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5e3dd2f2e1005ff server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:45.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.788+0000 7eff8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:06:45.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.788+0000 7eff8f7fe640 1 -- 172.21.15.84:0/3136950217 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98035040 con 0x55d1bde4fc50 2024-05-22T23:06:45.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.788+0000 7eff8f7fe640 1 -- 172.21.15.84:0/3136950217 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7eff980051c0 con 0x55d1bde4fc50 2024-05-22T23:06:45.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.788+0000 7effb91f2740 1 -- 172.21.15.84:0/3136950217 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effa00043e0 con 0x55d1bde4fc50 2024-05-22T23:06:45.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.788+0000 7eff8f7fe640 1 -- 172.21.15.84:0/3136950217 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7eff980053c0 con 0x55d1bde4fc50 2024-05-22T23:06:45.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.788+0000 7effb91f2740 10 client.25802 fetch_fsmap learned FSMap version 18 2024-05-22T23:06:45.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.788+0000 7effb91f2740 1 -- 172.21.15.84:0/3136950217 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55d1bded1bd0 con 0x55d1bde4fc50 2024-05-22T23:06:45.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.788+0000 7eff8f7fe640 1 -- 172.21.15.84:0/3136950217 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff9805b070 con 0x55d1bde4fc50 2024-05-22T23:06:45.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.789+0000 7eff8f7fe640 1 -- 172.21.15.84:0/3136950217 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7eff9804a030 con 0x55d1bde4fc50 2024-05-22T23:06:45.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.789+0000 7effb91f2740 10 client.25802 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:06:45.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.789+0000 7effb91f2740 10 client.25802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:06:45.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.789+0000 7effb91f2740 20 client.25802 populate_metadata read hostname 'smithi084' 2024-05-22T23:06:45.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.789+0000 7effb91f2740 1 --2- 172.21.15.84:0/3136950217 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d1bded2290 0x55d1bded4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.789+0000 7effb91f2740 4 client.25802 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-22T23:06:45.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.789+0000 7effb91f2740 1 -- 172.21.15.84:0/3136950217 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7effa00043e0 con 0x55d1bded2290 2024-05-22T23:06:45.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.789+0000 7effaffff640 1 --2- 172.21.15.84:0/3136950217 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d1bded2290 0x55d1bded4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.789+0000 7effaffff640 1 --2- 172.21.15.84:0/3136950217 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d1bded2290 0x55d1bded4680 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8708af27fa266f9e server_cookie=6588db06e1a553c5 in_seq=0 out_seq=0 2024-05-22T23:06:45.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.789+0000 7eff8f7fe640 10 client.25802 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:45.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.791+0000 7eff8f7fe640 1 -- 172.21.15.84:0/3136950217 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7effa00043e0 con 0x55d1bded2290 2024-05-22T23:06:45.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.791+0000 7eff8f7fe640 10 client.25802 handle_command_reply: tid=0 2024-05-22T23:06:45.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.791+0000 7effb91f2740 1 client.25802 shutdown 2024-05-22T23:06:45.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.791+0000 7effb91f2740 1 -- 172.21.15.84:0/3136950217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde4fc50 msgr2=0x55d1bde5ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:45.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.791+0000 7effb91f2740 1 --2- 172.21.15.84:0/3136950217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde4fc50 0x55d1bde5ea10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7eff980059d0 tx=0x7eff98005fd0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.791+0000 7effb91f2740 1 -- 172.21.15.84:0/3136950217 shutdown_connections 2024-05-22T23:06:45.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.791+0000 7effb91f2740 1 --2- 172.21.15.84:0/3136950217 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d1bded2290 0x55d1bded4680 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.791+0000 7eff8f7fe640 0 client.25802 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:45.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.791+0000 7effb91f2740 1 --2- 172.21.15.84:0/3136950217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1bde5ac40 0x55d1bde5f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.791+0000 7effb91f2740 1 --2- 172.21.15.84:0/3136950217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1bde50570 0x55d1bde5ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.791+0000 7effb91f2740 1 --2- 172.21.15.84:0/3136950217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1bde4fc50 0x55d1bde5ea10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.791+0000 7effb91f2740 1 -- 172.21.15.84:0/3136950217 >> 172.21.15.84:0/3136950217 conn(0x55d1bde4bf70 msgr2=0x55d1bde51c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:45.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.791+0000 7effb91f2740 1 -- 172.21.15.84:0/3136950217 shutdown_connections 2024-05-22T23:06:45.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.791+0000 7effb91f2740 1 -- 172.21.15.84:0/3136950217 wait complete. 2024-05-22T23:06:45.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.791+0000 7effb91f2740 20 client.25802 trim_cache size 0 max 16384 2024-05-22T23:06:45.802 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:06:45.802 INFO:teuthology.orchestra.run.smithi084.stdout: "return_code": 0, 2024-05-22T23:06:45.802 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_tag": "3f8d387b-3b3b-4fb8-ae56-ae0ea4e37900", 2024-05-22T23:06:45.802 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": "asynchronous" 2024-05-22T23:06:45.802 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:06:45.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.792+0000 7effb6407640 1 -- 172.21.15.84:0/2589735393 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7eff9c0a2d00 msgr2=0x7eff9c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:45.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.792+0000 7effb6407640 1 --2- 172.21.15.84:0/2589735393 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7eff9c0a2d00 0x7eff9c0a51c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7effa8062fd0 tx=0x7effa806ef90 comp rx=0 tx=0).stop 2024-05-22T23:06:45.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.792+0000 7effb6407640 1 -- 172.21.15.84:0/2589735393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effb0106750 msgr2=0x7effb01a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:45.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.792+0000 7effb6407640 1 --2- 172.21.15.84:0/2589735393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effb0106750 0x7effb01a4090 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7effa800eb10 tx=0x7effa800c2d0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.792+0000 7effb6407640 1 -- 172.21.15.84:0/2589735393 shutdown_connections 2024-05-22T23:06:45.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.792+0000 7effb6407640 1 --2- 172.21.15.84:0/2589735393 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7eff9c0a2d00 0x7eff9c0a51c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.792+0000 7effb6407640 1 --2- 172.21.15.84:0/2589735393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effb0106750 0x7effb01a4090 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.792+0000 7effb6407640 1 --2- 172.21.15.84:0/2589735393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effb0101100 0x7effb0199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.792+0000 7effb6407640 1 --2- 172.21.15.84:0/2589735393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effb0100740 0x7effb0199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.792+0000 7effb6407640 1 -- 172.21.15.84:0/2589735393 >> 172.21.15.84:0/2589735393 conn(0x7effb0077870 msgr2=0x7effb01030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:45.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.792+0000 7effb6407640 1 -- 172.21.15.84:0/2589735393 shutdown_connections 2024-05-22T23:06:45.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.792+0000 7effb6407640 1 -- 172.21.15.84:0/2589735393 wait complete. 2024-05-22T23:06:45.804 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:45 smithi096 ceph-mon[36392]: pgmap v150: 97 pgs: 97 active+clean; 334 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 254 op/s 2024-05-22T23:06:45.808 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:45 smithi084 ceph-mon[32186]: pgmap v150: 97 pgs: 97 active+clean; 334 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 254 op/s 2024-05-22T23:06:45.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:06:45.809 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:06:45.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:45 smithi204 ceph-mon[36211]: pgmap v150: 97 pgs: 97 active+clean; 334 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 254 op/s 2024-05-22T23:06:45.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.915+0000 7f14d9bf4640 1 -- 172.21.15.84:0/2431507927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14d4106750 msgr2=0x7f14d4106b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:45.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.915+0000 7f14d9bf4640 1 --2- 172.21.15.84:0/2431507927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14d4106750 0x7f14d4106b30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009930 tx=0x7f14c402a420 comp rx=0 tx=0).stop 2024-05-22T23:06:45.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.916+0000 7f14d9bf4640 1 -- 172.21.15.84:0/2431507927 shutdown_connections 2024-05-22T23:06:45.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.916+0000 7f14d9bf4640 1 --2- 172.21.15.84:0/2431507927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14d4101100 0x7f14d410d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.916+0000 7f14d9bf4640 1 --2- 172.21.15.84:0/2431507927 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14d4100740 0x7f14d4100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.916+0000 7f14d9bf4640 1 --2- 172.21.15.84:0/2431507927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14d4106750 0x7f14d4106b30 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.916+0000 7f14d9bf4640 1 -- 172.21.15.84:0/2431507927 >> 172.21.15.84:0/2431507927 conn(0x7f14d4077870 msgr2=0x7f14d4077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:45.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.916+0000 7f14d9bf4640 1 -- 172.21.15.84:0/2431507927 shutdown_connections 2024-05-22T23:06:45.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.916+0000 7f14d9bf4640 1 -- 172.21.15.84:0/2431507927 wait complete. 2024-05-22T23:06:45.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.916+0000 7f14d9bf4640 1 Processor -- start 2024-05-22T23:06:45.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.916+0000 7f14d9bf4640 1 -- start start 2024-05-22T23:06:45.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.917+0000 7f14d9bf4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14d4100740 0x7f14d4198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.917+0000 7f14d9bf4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14d4101100 0x7f14d4198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.917+0000 7f14d9bf4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14d4106750 0x7f14d419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.917+0000 7f14d9bf4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4110b60 con 0x7f14d4100740 2024-05-22T23:06:45.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.917+0000 7f14d9bf4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41109e0 con 0x7f14d4106750 2024-05-22T23:06:45.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.917+0000 7f14d9bf4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4110ce0 con 0x7f14d4101100 2024-05-22T23:06:45.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.917+0000 7f14d37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14d4100740 0x7f14d4198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.917+0000 7f14d37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14d4100740 0x7f14d4198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37934/0 (socket says 172.21.15.84:37934) 2024-05-22T23:06:45.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.917+0000 7f14d37fe640 1 -- 172.21.15.84:0/95689962 learned_addr learned my addr 172.21.15.84:0/95689962 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:45.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.917+0000 7f14d3fff640 1 --2- 172.21.15.84:0/95689962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14d4106750 0x7f14d419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.917+0000 7f14d2ffd640 1 --2- 172.21.15.84:0/95689962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14d4101100 0x7f14d4198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.917+0000 7f14d37fe640 1 -- 172.21.15.84:0/95689962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14d4101100 msgr2=0x7f14d4198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:45.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.917+0000 7f14d37fe640 1 --2- 172.21.15.84:0/95689962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14d4101100 0x7f14d4198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.917+0000 7f14d37fe640 1 -- 172.21.15.84:0/95689962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14d4106750 msgr2=0x7f14d419ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:45.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.917+0000 7f14d37fe640 1 --2- 172.21.15.84:0/95689962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14d4106750 0x7f14d419ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.917+0000 7f14d37fe640 1 -- 172.21.15.84:0/95689962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d419d130 con 0x7f14d4100740 2024-05-22T23:06:45.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.917+0000 7f14d37fe640 1 --2- 172.21.15.84:0/95689962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14d4100740 0x7f14d4198140 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009230 tx=0x7f14c4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:45.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.917+0000 7f14d0ff9640 1 -- 172.21.15.84:0/95689962 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c403f030 con 0x7f14d4100740 2024-05-22T23:06:45.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.917+0000 7f14d0ff9640 1 -- 172.21.15.84:0/95689962 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c4003ad0 con 0x7f14d4100740 2024-05-22T23:06:45.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.917+0000 7f14d9bf4640 1 -- 172.21.15.84:0/95689962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d419d3c0 con 0x7f14d4100740 2024-05-22T23:06:45.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.917+0000 7f14d0ff9640 1 -- 172.21.15.84:0/95689962 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4035070 con 0x7f14d4100740 2024-05-22T23:06:45.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.918+0000 7f14d9bf4640 1 -- 172.21.15.84:0/95689962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d41a4cb0 con 0x7f14d4100740 2024-05-22T23:06:45.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.920+0000 7f14d0ff9640 1 -- 172.21.15.84:0/95689962 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f14c4042030 con 0x7f14d4100740 2024-05-22T23:06:45.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.921+0000 7f14dc9d6740 1 Processor -- start 2024-05-22T23:06:45.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.921+0000 7f14dc9d6740 1 -- start start 2024-05-22T23:06:45.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.921+0000 7f14d0ff9640 1 --2- 172.21.15.84:0/95689962 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f14bc0a2d80 0x7f14bc0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.921+0000 7f14dc9d6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9c9a494a0 0x55b9c9a477e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.921+0000 7f14dc9d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9c9a47db0 0x55b9c9a48230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.921+0000 7f14dc9d6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a56f40 0x55b9c9a59330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.921+0000 7f14dc9d6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b9c99abdb0 con 0x55b9c9a56f40 2024-05-22T23:06:45.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.921+0000 7f14dc9d6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b9c994a360 con 0x55b9c9a494a0 2024-05-22T23:06:45.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.921+0000 7f14dc9d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b9c99a7d70 con 0x55b9c9a47db0 2024-05-22T23:06:45.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.921+0000 7f14d0ff9640 1 -- 172.21.15.84:0/95689962 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f14c40ef9e0 con 0x7f14d4100740 2024-05-22T23:06:45.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.921+0000 7f14d3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a56f40 0x55b9c9a59330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.921+0000 7f14d2ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9c9a47db0 0x55b9c9a48230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.921+0000 7f14d2ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9c9a47db0 0x55b9c9a48230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:54364/0 (socket says 172.21.15.84:54364) 2024-05-22T23:06:45.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.921+0000 7f14d3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a56f40 0x55b9c9a59330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37950/0 (socket says 172.21.15.84:37950) 2024-05-22T23:06:45.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.921+0000 7f14d2ffd640 1 -- 172.21.15.84:0/2703116116 learned_addr learned my addr 172.21.15.84:0/2703116116 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:45.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.921+0000 7f14d37fe640 1 --2- 172.21.15.84:0/2703116116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9c9a494a0 0x55b9c9a477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.921+0000 7f14d3fff640 1 --2- 172.21.15.84:0/95689962 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f14bc0a2d80 0x7f14bc0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.922+0000 7f14d2ffd640 1 -- 172.21.15.84:0/2703116116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9c9a494a0 msgr2=0x55b9c9a477e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:45.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.922+0000 7f14d2ffd640 1 --2- 172.21.15.84:0/2703116116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9c9a494a0 0x55b9c9a477e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.922+0000 7f14d2ffd640 1 -- 172.21.15.84:0/2703116116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a56f40 msgr2=0x55b9c9a59330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:45.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.922+0000 7f14d2ffd640 1 --2- 172.21.15.84:0/2703116116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a56f40 0x55b9c9a59330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.922+0000 7f14d2ffd640 1 -- 172.21.15.84:0/2703116116 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9c996da60 con 0x55b9c9a47db0 2024-05-22T23:06:45.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.922+0000 7f14d3fff640 1 --2- 172.21.15.84:0/2703116116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a56f40 0x55b9c9a59330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:06:45.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.922+0000 7f14d3fff640 1 --2- 172.21.15.84:0/95689962 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f14bc0a2d80 0x7f14bc0a5240 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55b9c9a454a0 tx=0x7f14cc009600 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:45.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.922+0000 7f14d2ffd640 1 --2- 172.21.15.84:0/2703116116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9c9a47db0 0x55b9c9a48230 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009c20 tx=0x7f14c80117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=158b396c78a1769d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:45.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.922+0000 7f14b9ffb640 1 -- 172.21.15.84:0/2703116116 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8012450 con 0x55b9c9a47db0 2024-05-22T23:06:45.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.922+0000 7f14b9ffb640 1 -- 172.21.15.84:0/2703116116 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c8012ad0 con 0x55b9c9a47db0 2024-05-22T23:06:45.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.922+0000 7f14b9ffb640 1 -- 172.21.15.84:0/2703116116 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c800fb00 con 0x55b9c9a47db0 2024-05-22T23:06:45.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.923+0000 7f14dc9d6740 1 -- 172.21.15.84:0/2703116116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9c9a47db0 msgr2=0x55b9c9a48230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:45.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.923+0000 7f14dc9d6740 1 --2- 172.21.15.84:0/2703116116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9c9a47db0 0x55b9c9a48230 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009c20 tx=0x7f14c80117b0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.923+0000 7f14dc9d6740 1 -- 172.21.15.84:0/2703116116 shutdown_connections 2024-05-22T23:06:45.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.923+0000 7f14dc9d6740 1 --2- 172.21.15.84:0/2703116116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a56f40 0x55b9c9a59330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.923+0000 7f14dc9d6740 1 --2- 172.21.15.84:0/2703116116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9c9a47db0 0x55b9c9a48230 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.923+0000 7f14dc9d6740 1 --2- 172.21.15.84:0/2703116116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9c9a494a0 0x55b9c9a477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.923+0000 7f14dc9d6740 1 -- 172.21.15.84:0/2703116116 >> 172.21.15.84:0/2703116116 conn(0x55b9c9a108e0 msgr2=0x55b9c9a44310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:45.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.923+0000 7f14dc9d6740 1 -- 172.21.15.84:0/2703116116 shutdown_connections 2024-05-22T23:06:45.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.923+0000 7f14dc9d6740 1 -- 172.21.15.84:0/2703116116 wait complete. 2024-05-22T23:06:45.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.923+0000 7f14dc9d6740 1 -- start start 2024-05-22T23:06:45.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.923+0000 7f14dc9d6740 1 Processor -- start 2024-05-22T23:06:45.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.924+0000 7f14dc9d6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a47db0 0x55b9c9a53d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.924+0000 7f14dc9d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9c9a494a0 0x55b9c9a542a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.924+0000 7f14dc9d6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9c9a56f40 0x55b9c9a547e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.924+0000 7f14dc9d6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b9c9a59dd0 con 0x55b9c9a47db0 2024-05-22T23:06:45.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.924+0000 7f14dc9d6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b9c99660f0 con 0x55b9c9a56f40 2024-05-22T23:06:45.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.924+0000 7f14dc9d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b9c9a59f50 con 0x55b9c9a494a0 2024-05-22T23:06:45.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.924+0000 7f14dc9d6740 10 client.0 _ll_register_callbacks cb 0x55b9c996af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:06:45.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.924+0000 7f14d37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a47db0 0x55b9c9a53d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.924+0000 7f14d2ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9c9a494a0 0x55b9c9a542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.924+0000 7f14d37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a47db0 0x55b9c9a53d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37956/0 (socket says 172.21.15.84:37956) 2024-05-22T23:06:45.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.924+0000 7f14d2ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9c9a494a0 0x55b9c9a542a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:54368/0 (socket says 172.21.15.84:54368) 2024-05-22T23:06:45.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.924+0000 7f14d37fe640 1 -- 172.21.15.84:0/439619092 learned_addr learned my addr 172.21.15.84:0/439619092 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:45.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.924+0000 7f14d3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9c9a56f40 0x55b9c9a547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.924+0000 7f14d37fe640 1 -- 172.21.15.84:0/439619092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9c9a494a0 msgr2=0x55b9c9a542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:45.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.924+0000 7f14d37fe640 1 --2- 172.21.15.84:0/439619092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9c9a494a0 0x55b9c9a542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.924+0000 7f14d37fe640 1 -- 172.21.15.84:0/439619092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9c9a56f40 msgr2=0x55b9c9a547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:45.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.924+0000 7f14d37fe640 1 --2- 172.21.15.84:0/439619092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9c9a56f40 0x55b9c9a547e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.924+0000 7f14d37fe640 1 -- 172.21.15.84:0/439619092 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9c97897d0 con 0x55b9c9a47db0 2024-05-22T23:06:45.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.924+0000 7f14d37fe640 1 --2- 172.21.15.84:0/439619092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a47db0 0x55b9c9a53d60 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f14c40bbc00 tx=0x7f14c40bbb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec6fd683934281e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:45.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.924+0000 7f14b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:06:45.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.924+0000 7f14b2ffd640 1 -- 172.21.15.84:0/439619092 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4049070 con 0x55b9c9a47db0 2024-05-22T23:06:45.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.924+0000 7f14b2ffd640 1 -- 172.21.15.84:0/439619092 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f14c403bd60 con 0x55b9c9a47db0 2024-05-22T23:06:45.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.924+0000 7f14dc9d6740 1 -- 172.21.15.84:0/439619092 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14c8012ad0 con 0x55b9c9a47db0 2024-05-22T23:06:45.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.925+0000 7f14b2ffd640 1 -- 172.21.15.84:0/439619092 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f14c40bbf00 con 0x55b9c9a47db0 2024-05-22T23:06:45.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.925+0000 7f14dc9d6740 10 client.16242 fetch_fsmap learned FSMap version 18 2024-05-22T23:06:45.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.925+0000 7f14dc9d6740 1 -- 172.21.15.84:0/439619092 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55b9c96bf3f0 con 0x55b9c9a47db0 2024-05-22T23:06:45.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.925+0000 7f14b2ffd640 1 -- 172.21.15.84:0/439619092 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c8012ad0 con 0x55b9c9a47db0 2024-05-22T23:06:45.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.925+0000 7f14b2ffd640 1 -- 172.21.15.84:0/439619092 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f14c40ed3b0 con 0x55b9c9a47db0 2024-05-22T23:06:45.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.925+0000 7f14dc9d6740 10 client.16242 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:06:45.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.925+0000 7f14dc9d6740 10 client.16242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:06:45.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.925+0000 7f14dc9d6740 20 client.16242 populate_metadata read hostname 'smithi084' 2024-05-22T23:06:45.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.925+0000 7f14dc9d6740 1 --2- 172.21.15.84:0/439619092 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b9c9a67270 0x55b9c9a69680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.925+0000 7f14dc9d6740 4 client.16242 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:06:45.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.925+0000 7f14dc9d6740 1 -- 172.21.15.84:0/439619092 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9c99b94e0 con 0x55b9c9a67270 2024-05-22T23:06:45.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.925+0000 7f14d2ffd640 1 --2- 172.21.15.84:0/439619092 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b9c9a67270 0x55b9c9a69680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.925+0000 7f14b2ffd640 10 client.16242 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:45.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.926+0000 7f14d2ffd640 1 --2- 172.21.15.84:0/439619092 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b9c9a67270 0x55b9c9a69680 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5048e67c03056ccb server_cookie=44530dab4001c11c in_seq=0 out_seq=0 2024-05-22T23:06:45.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.927+0000 7f14b2ffd640 1 -- 172.21.15.84:0/439619092 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b9c99b94e0 con 0x55b9c9a67270 2024-05-22T23:06:45.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.927+0000 7f14b2ffd640 10 client.16242 handle_command_reply: tid=0 2024-05-22T23:06:45.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.927+0000 7f14dc9d6740 1 client.16242 shutdown 2024-05-22T23:06:45.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.927+0000 7f14dc9d6740 1 -- 172.21.15.84:0/439619092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a47db0 msgr2=0x55b9c9a53d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:45.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.927+0000 7f14dc9d6740 1 --2- 172.21.15.84:0/439619092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a47db0 0x55b9c9a53d60 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f14c40bbc00 tx=0x7f14c40bbb00 comp rx=0 tx=0).stop 2024-05-22T23:06:45.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.927+0000 7f14dc9d6740 1 -- 172.21.15.84:0/439619092 shutdown_connections 2024-05-22T23:06:45.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.927+0000 7f14dc9d6740 1 --2- 172.21.15.84:0/439619092 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b9c9a67270 0x55b9c9a69680 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.927+0000 7f14dc9d6740 1 --2- 172.21.15.84:0/439619092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9c9a56f40 0x55b9c9a547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.927+0000 7f14b2ffd640 0 client.16242 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:45.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.927+0000 7f14dc9d6740 1 --2- 172.21.15.84:0/439619092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9c9a494a0 0x55b9c9a542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.927+0000 7f14dc9d6740 1 --2- 172.21.15.84:0/439619092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a47db0 0x55b9c9a53d60 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.927+0000 7f14dc9d6740 1 -- 172.21.15.84:0/439619092 >> 172.21.15.84:0/439619092 conn(0x55b9c9a108e0 msgr2=0x55b9c9a4a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:45.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.927+0000 7f14dc9d6740 1 -- 172.21.15.84:0/439619092 shutdown_connections 2024-05-22T23:06:45.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.927+0000 7f14dc9d6740 1 -- 172.21.15.84:0/439619092 wait complete. 2024-05-22T23:06:45.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.927+0000 7f14dc9d6740 20 client.16242 trim_cache size 0 max 16384 2024-05-22T23:06:45.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.947+0000 7f14dc9d6740 1 Processor -- start 2024-05-22T23:06:45.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.947+0000 7f14dc9d6740 1 -- start start 2024-05-22T23:06:45.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.947+0000 7f14dc9d6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9c9a47db0 0x55b9c9a62d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.947+0000 7f14dc9d6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a494a0 0x55b9c9a3f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.947+0000 7f14dc9d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9c9a56f40 0x55b9c9a52ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.947+0000 7f14dc9d6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41a9330 con 0x55b9c9a494a0 2024-05-22T23:06:45.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.947+0000 7f14dc9d6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41abbb0 con 0x55b9c9a47db0 2024-05-22T23:06:45.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.947+0000 7f14dc9d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41a99a0 con 0x55b9c9a56f40 2024-05-22T23:06:45.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.947+0000 7f14d2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a494a0 0x55b9c9a3f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.947+0000 7f14d37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9c9a47db0 0x55b9c9a62d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.947+0000 7f14d3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9c9a56f40 0x55b9c9a52ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.947+0000 7f14d2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a494a0 0x55b9c9a3f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37966/0 (socket says 172.21.15.84:37966) 2024-05-22T23:06:45.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.947+0000 7f14d2ffd640 1 -- 172.21.15.84:0/2405430853 learned_addr learned my addr 172.21.15.84:0/2405430853 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:45.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.948+0000 7f14d2ffd640 1 -- 172.21.15.84:0/2405430853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9c9a56f40 msgr2=0x55b9c9a52ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:45.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.948+0000 7f14d2ffd640 1 --2- 172.21.15.84:0/2405430853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9c9a56f40 0x55b9c9a52ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.948+0000 7f14d2ffd640 1 -- 172.21.15.84:0/2405430853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9c9a47db0 msgr2=0x55b9c9a62d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:45.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.948+0000 7f14d2ffd640 1 --2- 172.21.15.84:0/2405430853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9c9a47db0 0x55b9c9a62d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.948+0000 7f14d2ffd640 1 -- 172.21.15.84:0/2405430853 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9c967f0f0 con 0x55b9c9a494a0 2024-05-22T23:06:45.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.948+0000 7f14d2ffd640 1 --2- 172.21.15.84:0/2405430853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a494a0 0x55b9c9a3f160 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009c20 tx=0x7f14c80093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc39d7b94241528 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:45.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.948+0000 7f14b97fa640 1 -- 172.21.15.84:0/2405430853 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8009650 con 0x55b9c9a494a0 2024-05-22T23:06:45.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.948+0000 7f14b97fa640 1 -- 172.21.15.84:0/2405430853 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c8029070 con 0x55b9c9a494a0 2024-05-22T23:06:45.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.948+0000 7f14b97fa640 1 -- 172.21.15.84:0/2405430853 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c80035f0 con 0x55b9c9a494a0 2024-05-22T23:06:45.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.949+0000 7f14dc9d6740 1 -- 172.21.15.84:0/2405430853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a494a0 msgr2=0x55b9c9a3f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:45.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.949+0000 7f14dc9d6740 1 --2- 172.21.15.84:0/2405430853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a494a0 0x55b9c9a3f160 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009c20 tx=0x7f14c80093b0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.949+0000 7f14dc9d6740 1 -- 172.21.15.84:0/2405430853 shutdown_connections 2024-05-22T23:06:45.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.949+0000 7f14dc9d6740 1 --2- 172.21.15.84:0/2405430853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9c9a56f40 0x55b9c9a52ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.949+0000 7f14dc9d6740 1 --2- 172.21.15.84:0/2405430853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a494a0 0x55b9c9a3f160 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.949+0000 7f14dc9d6740 1 --2- 172.21.15.84:0/2405430853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9c9a47db0 0x55b9c9a62d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.949+0000 7f14dc9d6740 1 -- 172.21.15.84:0/2405430853 >> 172.21.15.84:0/2405430853 conn(0x55b9c9a108e0 msgr2=0x55b9c9a55590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:45.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.949+0000 7f14dc9d6740 1 -- 172.21.15.84:0/2405430853 shutdown_connections 2024-05-22T23:06:45.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.949+0000 7f14dc9d6740 1 -- 172.21.15.84:0/2405430853 wait complete. 2024-05-22T23:06:45.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.949+0000 7f14dc9d6740 1 -- start start 2024-05-22T23:06:45.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.949+0000 7f14dc9d6740 1 Processor -- start 2024-05-22T23:06:45.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.950+0000 7f14dc9d6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9c9a47db0 0x55b9c9a5f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.950+0000 7f14dc9d6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a494a0 0x55b9c9a50fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.950+0000 7f14dc9d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9c9a56f40 0x55b9c9a51580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.950+0000 7f14dc9d6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41aa070 con 0x55b9c9a494a0 2024-05-22T23:06:45.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.950+0000 7f14dc9d6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41ab4e0 con 0x55b9c9a47db0 2024-05-22T23:06:45.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.950+0000 7f14dc9d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41aa740 con 0x55b9c9a56f40 2024-05-22T23:06:45.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.950+0000 7f14dc9d6740 10 client.0 _ll_register_callbacks cb 0x7f14c4009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:06:45.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.950+0000 7f14d2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a494a0 0x55b9c9a50fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.950+0000 7f14d37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9c9a47db0 0x55b9c9a5f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.950+0000 7f14d2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a494a0 0x55b9c9a50fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37972/0 (socket says 172.21.15.84:37972) 2024-05-22T23:06:45.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.950+0000 7f14d2ffd640 1 -- 172.21.15.84:0/783528256 learned_addr learned my addr 172.21.15.84:0/783528256 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:06:45.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.950+0000 7f14d3fff640 1 --2- 172.21.15.84:0/783528256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9c9a56f40 0x55b9c9a51580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.950+0000 7f14d37fe640 1 -- 172.21.15.84:0/783528256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9c9a56f40 msgr2=0x55b9c9a51580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:45.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.950+0000 7f14d37fe640 1 --2- 172.21.15.84:0/783528256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9c9a56f40 0x55b9c9a51580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.950+0000 7f14d37fe640 1 -- 172.21.15.84:0/783528256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a494a0 msgr2=0x55b9c9a50fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:06:45.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.950+0000 7f14d37fe640 1 --2- 172.21.15.84:0/783528256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a494a0 0x55b9c9a50fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.950+0000 7f14d37fe640 1 -- 172.21.15.84:0/783528256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9c996afe0 con 0x55b9c9a47db0 2024-05-22T23:06:45.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.950+0000 7f14d2ffd640 1 --2- 172.21.15.84:0/783528256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a494a0 0x55b9c9a50fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:06:45.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.950+0000 7f14d37fe640 1 --2- 172.21.15.84:0/783528256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9c9a47db0 0x55b9c9a5f690 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f14c4038020 tx=0x7f14c40bbb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ed06c830fc709c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:06:45.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.950+0000 7f14b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:06:45.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.950+0000 7f14b2ffd640 1 -- 172.21.15.84:0/783528256 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4049040 con 0x55b9c9a47db0 2024-05-22T23:06:45.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.950+0000 7f14dc9d6740 1 -- 172.21.15.84:0/783528256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14c8029070 con 0x55b9c9a47db0 2024-05-22T23:06:45.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.950+0000 7f14b2ffd640 1 -- 172.21.15.84:0/783528256 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f14c4003c80 con 0x55b9c9a47db0 2024-05-22T23:06:45.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.951+0000 7f14b2ffd640 1 -- 172.21.15.84:0/783528256 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f14c40c02b0 con 0x55b9c9a47db0 2024-05-22T23:06:45.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.951+0000 7f14dc9d6740 10 client.25820 fetch_fsmap learned FSMap version 18 2024-05-22T23:06:45.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.951+0000 7f14dc9d6740 1 -- 172.21.15.84:0/783528256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55b9c993c1c0 con 0x55b9c9a47db0 2024-05-22T23:06:45.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.951+0000 7f14b2ffd640 1 -- 172.21.15.84:0/783528256 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c8029070 con 0x55b9c9a47db0 2024-05-22T23:06:45.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.951+0000 7f14b2ffd640 1 -- 172.21.15.84:0/783528256 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f14c4044fe0 con 0x55b9c9a47db0 2024-05-22T23:06:45.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.951+0000 7f14dc9d6740 10 client.25820 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:06:45.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.951+0000 7f14dc9d6740 10 client.25820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:06:45.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.951+0000 7f14dc9d6740 20 client.25820 populate_metadata read hostname 'smithi084' 2024-05-22T23:06:45.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.951+0000 7f14dc9d6740 1 --2- 172.21.15.84:0/783528256 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b9c9a67270 0x55b9c9acfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:06:45.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.951+0000 7f14dc9d6740 4 client.25820 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:06:45.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.951+0000 7f14dc9d6740 1 -- 172.21.15.84:0/783528256 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b9c99b3830 con 0x55b9c9a67270 2024-05-22T23:06:45.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.951+0000 7f14d2ffd640 1 --2- 172.21.15.84:0/783528256 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b9c9a67270 0x55b9c9acfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:06:45.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.951+0000 7f14d2ffd640 1 --2- 172.21.15.84:0/783528256 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b9c9a67270 0x55b9c9acfb30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f577d3d5460d4a24 server_cookie=d1e228e1664e9d76 in_seq=0 out_seq=0 2024-05-22T23:06:45.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.951+0000 7f14b2ffd640 10 client.25820 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:45.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.952+0000 7f14b2ffd640 1 -- 172.21.15.84:0/783528256 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b9c99b3830 con 0x55b9c9a67270 2024-05-22T23:06:45.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.952+0000 7f14b2ffd640 10 client.25820 handle_command_reply: tid=0 2024-05-22T23:06:45.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.952+0000 7f14dc9d6740 1 client.25820 shutdown 2024-05-22T23:06:45.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.952+0000 7f14dc9d6740 1 -- 172.21.15.84:0/783528256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9c9a47db0 msgr2=0x55b9c9a5f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:45.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.952+0000 7f14dc9d6740 1 --2- 172.21.15.84:0/783528256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9c9a47db0 0x55b9c9a5f690 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f14c4038020 tx=0x7f14c40bbb70 comp rx=0 tx=0).stop 2024-05-22T23:06:45.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.952+0000 7f14dc9d6740 1 -- 172.21.15.84:0/783528256 shutdown_connections 2024-05-22T23:06:45.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.952+0000 7f14dc9d6740 1 --2- 172.21.15.84:0/783528256 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55b9c9a67270 0x55b9c9acfb30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.952+0000 7f14dc9d6740 1 --2- 172.21.15.84:0/783528256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9c9a56f40 0x55b9c9a51580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.952+0000 7f14b2ffd640 0 client.25820 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:06:45.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.952+0000 7f14dc9d6740 1 --2- 172.21.15.84:0/783528256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b9c9a494a0 0x55b9c9a50fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.952+0000 7f14dc9d6740 1 --2- 172.21.15.84:0/783528256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9c9a47db0 0x55b9c9a5f690 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.952+0000 7f14dc9d6740 1 -- 172.21.15.84:0/783528256 >> 172.21.15.84:0/783528256 conn(0x55b9c9a108e0 msgr2=0x55b9c9a49880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:45.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.952+0000 7f14dc9d6740 1 -- 172.21.15.84:0/783528256 shutdown_connections 2024-05-22T23:06:45.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.952+0000 7f14dc9d6740 1 -- 172.21.15.84:0/783528256 wait complete. 2024-05-22T23:06:45.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.952+0000 7f14dc9d6740 20 client.25820 trim_cache size 0 max 16384 2024-05-22T23:06:45.963 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:06:45.964 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-22T23:06:45.964 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": { 2024-05-22T23:06:45.964 INFO:teuthology.orchestra.run.smithi084.stdout: "3f8d387b-3b3b-4fb8-ae56-ae0ea4e37900": { 2024-05-22T23:06:45.964 INFO:teuthology.orchestra.run.smithi084.stdout: "path": "/", 2024-05-22T23:06:45.964 INFO:teuthology.orchestra.run.smithi084.stdout: "tag": "3f8d387b-3b3b-4fb8-ae56-ae0ea4e37900", 2024-05-22T23:06:45.964 INFO:teuthology.orchestra.run.smithi084.stdout: "options": "recursive,force" 2024-05-22T23:06:45.964 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:06:45.964 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:06:45.964 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:06:45.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.953+0000 7f14d9bf4640 1 -- 172.21.15.84:0/95689962 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f14bc0a2d80 msgr2=0x7f14bc0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:45.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.953+0000 7f14d9bf4640 1 --2- 172.21.15.84:0/95689962 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f14bc0a2d80 0x7f14bc0a5240 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55b9c9a454a0 tx=0x7f14cc009600 comp rx=0 tx=0).stop 2024-05-22T23:06:45.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.953+0000 7f14d9bf4640 1 -- 172.21.15.84:0/95689962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14d4100740 msgr2=0x7f14d4198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:06:45.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.953+0000 7f14d9bf4640 1 --2- 172.21.15.84:0/95689962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14d4100740 0x7f14d4198140 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009230 tx=0x7f14c4004270 comp rx=0 tx=0).stop 2024-05-22T23:06:45.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.953+0000 7f14d9bf4640 1 -- 172.21.15.84:0/95689962 shutdown_connections 2024-05-22T23:06:45.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.953+0000 7f14d9bf4640 1 --2- 172.21.15.84:0/95689962 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f14bc0a2d80 0x7f14bc0a5240 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.953+0000 7f14d9bf4640 1 --2- 172.21.15.84:0/95689962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14d4106750 0x7f14d419ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.953+0000 7f14d9bf4640 1 --2- 172.21.15.84:0/95689962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14d4101100 0x7f14d4198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.953+0000 7f14d9bf4640 1 --2- 172.21.15.84:0/95689962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14d4100740 0x7f14d4198140 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:06:45.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.953+0000 7f14d9bf4640 1 -- 172.21.15.84:0/95689962 >> 172.21.15.84:0/95689962 conn(0x7f14d4077870 msgr2=0x7f14d4068d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:06:45.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.953+0000 7f14d9bf4640 1 -- 172.21.15.84:0/95689962 shutdown_connections 2024-05-22T23:06:45.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:06:45.953+0000 7f14d9bf4640 1 -- 172.21.15.84:0/95689962 wait complete. 2024-05-22T23:06:45.969 INFO:tasks.cephfs.filesystem:scrub status for tag:3f8d387b-3b3b-4fb8-ae56-ae0ea4e37900 - {'path': '/', 'tag': '3f8d387b-3b3b-4fb8-ae56-ae0ea4e37900', 'options': 'recursive,force'} 2024-05-22T23:06:46.435 INFO:tasks.workunit.client.0.smithi084.stdout: 1 611279 15.40 MB/sec execute 92 sec latency 21.866 ms 2024-05-22T23:06:46.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:46 smithi096 ceph-mon[36392]: scrub queued for path: / 2024-05-22T23:06:46.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:46 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:06:46.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:46 smithi096 ceph-mon[36392]: scrub summary: active paths [/] 2024-05-22T23:06:46.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:46 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:06:46.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:46 smithi096 ceph-mon[36392]: scrub summary: idle 2024-05-22T23:06:46.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:46 smithi084 ceph-mon[32186]: scrub queued for path: / 2024-05-22T23:06:46.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:46 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:06:46.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:46 smithi084 ceph-mon[32186]: scrub summary: active paths [/] 2024-05-22T23:06:46.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:46 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:06:46.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:46 smithi084 ceph-mon[32186]: scrub summary: idle 2024-05-22T23:06:46.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:46 smithi204 ceph-mon[36211]: scrub queued for path: / 2024-05-22T23:06:46.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:46 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:06:46.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:46 smithi204 ceph-mon[36211]: scrub summary: active paths [/] 2024-05-22T23:06:46.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:46 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:06:46.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:46 smithi204 ceph-mon[36211]: scrub summary: idle 2024-05-22T23:06:47.434 INFO:tasks.workunit.client.0.smithi084.stdout: 1 614546 15.42 MB/sec execute 93 sec latency 17.460 ms 2024-05-22T23:06:47.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:47 smithi096 ceph-mon[36392]: pgmap v151: 97 pgs: 97 active+clean; 342 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.8 MiB/s wr, 380 op/s 2024-05-22T23:06:47.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:47 smithi084 ceph-mon[32186]: pgmap v151: 97 pgs: 97 active+clean; 342 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.8 MiB/s wr, 380 op/s 2024-05-22T23:06:47.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:47 smithi204 ceph-mon[36211]: pgmap v151: 97 pgs: 97 active+clean; 342 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.8 MiB/s wr, 380 op/s 2024-05-22T23:06:48.435 INFO:tasks.workunit.client.0.smithi084.stdout: 1 617365 15.41 MB/sec execute 94 sec latency 22.367 ms 2024-05-22T23:06:49.435 INFO:tasks.workunit.client.0.smithi084.stdout: 1 619950 15.37 MB/sec execute 95 sec latency 19.101 ms 2024-05-22T23:06:49.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:49 smithi096 ceph-mon[36392]: pgmap v152: 97 pgs: 97 active+clean; 342 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 257 op/s 2024-05-22T23:06:49.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:49 smithi084 ceph-mon[32186]: pgmap v152: 97 pgs: 97 active+clean; 342 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 257 op/s 2024-05-22T23:06:49.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:49 smithi204 ceph-mon[36211]: pgmap v152: 97 pgs: 97 active+clean; 342 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 257 op/s 2024-05-22T23:06:50.435 INFO:tasks.workunit.client.0.smithi084.stdout: 1 622204 15.36 MB/sec execute 96 sec latency 95.766 ms 2024-05-22T23:06:51.435 INFO:tasks.workunit.client.0.smithi084.stdout: 1 625315 15.39 MB/sec execute 97 sec latency 25.970 ms 2024-05-22T23:06:51.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:51 smithi096 ceph-mon[36392]: pgmap v153: 97 pgs: 97 active+clean; 342 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 257 op/s 2024-05-22T23:06:51.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:51 smithi084 ceph-mon[32186]: pgmap v153: 97 pgs: 97 active+clean; 342 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 257 op/s 2024-05-22T23:06:51.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:51 smithi204 ceph-mon[36211]: pgmap v153: 97 pgs: 97 active+clean; 342 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 257 op/s 2024-05-22T23:06:52.435 INFO:tasks.workunit.client.0.smithi084.stdout: 1 628436 15.38 MB/sec execute 98 sec latency 14.973 ms 2024-05-22T23:06:53.435 INFO:tasks.workunit.client.0.smithi084.stdout: 1 631103 15.35 MB/sec execute 99 sec latency 19.876 ms 2024-05-22T23:06:53.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:53 smithi096 ceph-mon[36392]: pgmap v154: 97 pgs: 97 active+clean; 351 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.9 MiB/s wr, 381 op/s 2024-05-22T23:06:53.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:53 smithi084 ceph-mon[32186]: pgmap v154: 97 pgs: 97 active+clean; 351 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.9 MiB/s wr, 381 op/s 2024-05-22T23:06:53.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:53 smithi204 ceph-mon[36211]: pgmap v154: 97 pgs: 97 active+clean; 351 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.9 MiB/s wr, 381 op/s 2024-05-22T23:06:54.435 INFO:tasks.workunit.client.0.smithi084.stdout: 1 633675 15.36 MB/sec execute 100 sec latency 17.220 ms 2024-05-22T23:06:55.435 INFO:tasks.workunit.client.0.smithi084.stdout: 1 636263 15.35 MB/sec execute 101 sec latency 114.245 ms 2024-05-22T23:06:55.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:55 smithi096 ceph-mon[36392]: pgmap v155: 97 pgs: 97 active+clean; 351 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.9 MiB/s wr, 250 op/s 2024-05-22T23:06:55.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:55 smithi084 ceph-mon[32186]: pgmap v155: 97 pgs: 97 active+clean; 351 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.9 MiB/s wr, 250 op/s 2024-05-22T23:06:55.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:55 smithi204 ceph-mon[36211]: pgmap v155: 97 pgs: 97 active+clean; 351 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.9 MiB/s wr, 250 op/s 2024-05-22T23:06:56.435 INFO:tasks.workunit.client.0.smithi084.stdout: 1 639482 15.39 MB/sec execute 102 sec latency 37.666 ms 2024-05-22T23:06:57.435 INFO:tasks.workunit.client.0.smithi084.stdout: 1 642267 15.36 MB/sec execute 103 sec latency 14.777 ms 2024-05-22T23:06:57.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:57 smithi096 ceph-mon[36392]: pgmap v156: 97 pgs: 97 active+clean; 360 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.9 MiB/s wr, 376 op/s 2024-05-22T23:06:57.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:57 smithi084 ceph-mon[32186]: pgmap v156: 97 pgs: 97 active+clean; 360 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.9 MiB/s wr, 376 op/s 2024-05-22T23:06:57.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:57 smithi204 ceph-mon[36211]: pgmap v156: 97 pgs: 97 active+clean; 360 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.9 MiB/s wr, 376 op/s 2024-05-22T23:06:58.435 INFO:tasks.workunit.client.0.smithi084.stdout: 1 645135 15.34 MB/sec execute 104 sec latency 14.794 ms 2024-05-22T23:06:59.435 INFO:tasks.workunit.client.0.smithi084.stdout: 1 647571 15.33 MB/sec execute 105 sec latency 14.134 ms 2024-05-22T23:06:59.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:06:59 smithi096 ceph-mon[36392]: pgmap v157: 97 pgs: 97 active+clean; 360 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 250 op/s 2024-05-22T23:06:59.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:06:59 smithi084 ceph-mon[32186]: pgmap v157: 97 pgs: 97 active+clean; 360 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 250 op/s 2024-05-22T23:06:59.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:06:59 smithi204 ceph-mon[36211]: pgmap v157: 97 pgs: 97 active+clean; 360 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 250 op/s 2024-05-22T23:07:00.435 INFO:tasks.workunit.client.0.smithi084.stdout: 1 650123 15.35 MB/sec execute 106 sec latency 126.672 ms 2024-05-22T23:07:00.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:00 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:07:00.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:00 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:07:00.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:00 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:07:01.435 INFO:tasks.workunit.client.0.smithi084.stdout: 1 651655 15.26 MB/sec execute 107 sec latency 33.681 ms 2024-05-22T23:07:01.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:01 smithi096 ceph-mon[36392]: pgmap v158: 97 pgs: 97 active+clean; 360 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 250 op/s 2024-05-22T23:07:01.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:01 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:07:01.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:01 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:07:01.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:01 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:07:01.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:01 smithi084 ceph-mon[32186]: pgmap v158: 97 pgs: 97 active+clean; 360 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 250 op/s 2024-05-22T23:07:01.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:01 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:07:01.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:01 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:07:01.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:01 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:07:01.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:01 smithi204 ceph-mon[36211]: pgmap v158: 97 pgs: 97 active+clean; 360 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 250 op/s 2024-05-22T23:07:01.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:01 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:07:01.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:01 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:07:01.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:01 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:07:02.435 INFO:tasks.workunit.client.0.smithi084.stdout: 1 654324 15.25 MB/sec execute 108 sec latency 13.157 ms 2024-05-22T23:07:03.435 INFO:tasks.workunit.client.0.smithi084.stdout: 1 657400 15.28 MB/sec execute 109 sec latency 15.956 ms 2024-05-22T23:07:03.803 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:03 smithi096 ceph-mon[36392]: pgmap v159: 97 pgs: 97 active+clean; 340 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.9 MiB/s wr, 386 op/s 2024-05-22T23:07:03.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:03 smithi084 ceph-mon[32186]: pgmap v159: 97 pgs: 97 active+clean; 340 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.9 MiB/s wr, 386 op/s 2024-05-22T23:07:03.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:03 smithi204 ceph-mon[36211]: pgmap v159: 97 pgs: 97 active+clean; 340 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.9 MiB/s wr, 386 op/s 2024-05-22T23:07:04.436 INFO:tasks.workunit.client.0.smithi084.stdout: 1 660646 15.30 MB/sec execute 110 sec latency 14.714 ms 2024-05-22T23:07:05.435 INFO:tasks.workunit.client.0.smithi084.stdout: 1 662920 15.24 MB/sec execute 111 sec latency 57.590 ms 2024-05-22T23:07:05.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:05 smithi084 ceph-mon[32186]: pgmap v160: 97 pgs: 97 active+clean; 340 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 261 op/s 2024-05-22T23:07:05.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:05 smithi204 ceph-mon[36211]: pgmap v160: 97 pgs: 97 active+clean; 340 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 261 op/s 2024-05-22T23:07:06.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:05 smithi096 ceph-mon[36392]: pgmap v160: 97 pgs: 97 active+clean; 340 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 261 op/s 2024-05-22T23:07:06.435 INFO:tasks.workunit.client.0.smithi084.stdout: 1 665052 15.22 MB/sec execute 112 sec latency 27.317 ms 2024-05-22T23:07:07.435 INFO:tasks.workunit.client.0.smithi084.stdout: 1 668186 15.24 MB/sec execute 113 sec latency 18.211 ms 2024-05-22T23:07:07.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:07 smithi084 ceph-mon[32186]: pgmap v161: 97 pgs: 97 active+clean; 348 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.0 MiB/s wr, 398 op/s 2024-05-22T23:07:07.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:07 smithi204 ceph-mon[36211]: pgmap v161: 97 pgs: 97 active+clean; 348 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.0 MiB/s wr, 398 op/s 2024-05-22T23:07:08.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:07 smithi096 ceph-mon[36392]: pgmap v161: 97 pgs: 97 active+clean; 348 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.0 MiB/s wr, 398 op/s 2024-05-22T23:07:08.436 INFO:tasks.workunit.client.0.smithi084.stdout: 1 671460 15.28 MB/sec execute 114 sec latency 24.267 ms 2024-05-22T23:07:09.436 INFO:tasks.workunit.client.0.smithi084.stdout: 1 674575 15.27 MB/sec execute 115 sec latency 18.663 ms 2024-05-22T23:07:09.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:09 smithi084 ceph-mon[32186]: pgmap v162: 97 pgs: 97 active+clean; 348 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 272 op/s 2024-05-22T23:07:09.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:09 smithi204 ceph-mon[36211]: pgmap v162: 97 pgs: 97 active+clean; 348 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 272 op/s 2024-05-22T23:07:10.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:09 smithi096 ceph-mon[36392]: pgmap v162: 97 pgs: 97 active+clean; 348 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 272 op/s 2024-05-22T23:07:10.436 INFO:tasks.workunit.client.0.smithi084.stdout: 1 676434 15.23 MB/sec execute 116 sec latency 94.394 ms 2024-05-22T23:07:11.436 INFO:tasks.workunit.client.0.smithi084.stdout: 1 679044 15.22 MB/sec execute 117 sec latency 30.782 ms 2024-05-22T23:07:11.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:11 smithi084 ceph-mon[32186]: pgmap v163: 97 pgs: 97 active+clean; 348 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 272 op/s 2024-05-22T23:07:11.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:11 smithi204 ceph-mon[36211]: pgmap v163: 97 pgs: 97 active+clean; 348 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 272 op/s 2024-05-22T23:07:12.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:11 smithi096 ceph-mon[36392]: pgmap v163: 97 pgs: 97 active+clean; 348 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 272 op/s 2024-05-22T23:07:12.436 INFO:tasks.workunit.client.0.smithi084.stdout: 1 682246 15.25 MB/sec execute 118 sec latency 11.069 ms 2024-05-22T23:07:13.436 INFO:tasks.workunit.client.0.smithi084.stdout: 1 685391 15.27 MB/sec execute 119 sec latency 14.766 ms 2024-05-22T23:07:13.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:13 smithi084 ceph-mon[32186]: pgmap v164: 97 pgs: 97 active+clean; 358 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.0 MiB/s wr, 395 op/s 2024-05-22T23:07:13.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:13 smithi204 ceph-mon[36211]: pgmap v164: 97 pgs: 97 active+clean; 358 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.0 MiB/s wr, 395 op/s 2024-05-22T23:07:14.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:13 smithi096 ceph-mon[36392]: pgmap v164: 97 pgs: 97 active+clean; 358 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.0 MiB/s wr, 395 op/s 2024-05-22T23:07:14.436 INFO:tasks.workunit.client.0.smithi084.stdout: 1 687961 15.22 MB/sec execute 120 sec latency 15.207 ms 2024-05-22T23:07:15.436 INFO:tasks.workunit.client.0.smithi084.stdout: 1 690090 15.20 MB/sec execute 121 sec latency 76.146 ms 2024-05-22T23:07:15.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:15 smithi084 ceph-mon[32186]: pgmap v165: 97 pgs: 97 active+clean; 358 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 258 op/s 2024-05-22T23:07:15.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:15 smithi204 ceph-mon[36211]: pgmap v165: 97 pgs: 97 active+clean; 358 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 258 op/s 2024-05-22T23:07:15.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:07:15.970 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:07:16.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:15 smithi096 ceph-mon[36392]: pgmap v165: 97 pgs: 97 active+clean; 358 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 258 op/s 2024-05-22T23:07:16.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.073+0000 7fedc2d7e640 1 -- 172.21.15.84:0/3393054606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc06a8b0 msgr2=0x7fedbc10e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:16.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.073+0000 7fedc2d7e640 1 --2- 172.21.15.84:0/3393054606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc06a8b0 0x7fedbc10e720 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fedb4009cd0 tx=0x7fedb402a8e0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.073+0000 7fedc2d7e640 1 -- 172.21.15.84:0/3393054606 shutdown_connections 2024-05-22T23:07:16.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.073+0000 7fedc2d7e640 1 --2- 172.21.15.84:0/3393054606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc06a8b0 0x7fedbc10e720 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.073+0000 7fedc2d7e640 1 --2- 172.21.15.84:0/3393054606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fedbc069ef0 0x7fedbc06a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.073+0000 7fedc2d7e640 1 --2- 172.21.15.84:0/3393054606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedbc107860 0x7fedbc107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.073+0000 7fedc2d7e640 1 -- 172.21.15.84:0/3393054606 >> 172.21.15.84:0/3393054606 conn(0x7fedbc0777d0 msgr2=0x7fedbc077be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:16.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.073+0000 7fedc2d7e640 1 -- 172.21.15.84:0/3393054606 shutdown_connections 2024-05-22T23:07:16.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.073+0000 7fedc2d7e640 1 -- 172.21.15.84:0/3393054606 wait complete. 2024-05-22T23:07:16.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.074+0000 7fedc2d7e640 1 Processor -- start 2024-05-22T23:07:16.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.074+0000 7fedc2d7e640 1 -- start start 2024-05-22T23:07:16.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.074+0000 7fedc2d7e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fedbc069ef0 0x7fedbc198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.074+0000 7fedc2d7e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc06a8b0 0x7fedbc1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.074+0000 7fedc2d7e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedbc107860 0x7fedbc19c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.074+0000 7fedc2d7e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc111c70 con 0x7fedbc06a8b0 2024-05-22T23:07:16.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.074+0000 7fedc2d7e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc111af0 con 0x7fedbc069ef0 2024-05-22T23:07:16.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.074+0000 7fedc2d7e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc111df0 con 0x7fedbc107860 2024-05-22T23:07:16.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.074+0000 7fedbbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc06a8b0 0x7fedbc1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.074+0000 7fedbbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc06a8b0 0x7fedbc1985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51054/0 (socket says 172.21.15.84:51054) 2024-05-22T23:07:16.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.074+0000 7fedc0b1a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fedbc069ef0 0x7fedbc198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.074+0000 7fedbbfff640 1 -- 172.21.15.84:0/1508148498 learned_addr learned my addr 172.21.15.84:0/1508148498 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:16.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.074+0000 7fedc131b640 1 --2- 172.21.15.84:0/1508148498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedbc107860 0x7fedbc19c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.074+0000 7fedbbfff640 1 -- 172.21.15.84:0/1508148498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedbc107860 msgr2=0x7fedbc19c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:16.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.074+0000 7fedbbfff640 1 --2- 172.21.15.84:0/1508148498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedbc107860 0x7fedbc19c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.074+0000 7fedbbfff640 1 -- 172.21.15.84:0/1508148498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fedbc069ef0 msgr2=0x7fedbc198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:16.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.074+0000 7fedbbfff640 1 --2- 172.21.15.84:0/1508148498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fedbc069ef0 0x7fedbc198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.074+0000 7fedbbfff640 1 -- 172.21.15.84:0/1508148498 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedbc19d080 con 0x7fedbc06a8b0 2024-05-22T23:07:16.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.074+0000 7fedc131b640 1 --2- 172.21.15.84:0/1508148498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedbc107860 0x7fedbc19c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:07:16.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.074+0000 7fedbbfff640 1 --2- 172.21.15.84:0/1508148498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc06a8b0 0x7fedbc1985d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7feda800e9f0 tx=0x7feda800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:16.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.074+0000 7fedb9ffb640 1 -- 172.21.15.84:0/1508148498 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda800cd80 con 0x7fedbc06a8b0 2024-05-22T23:07:16.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.075+0000 7fedb9ffb640 1 -- 172.21.15.84:0/1508148498 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feda8010910 con 0x7fedbc06a8b0 2024-05-22T23:07:16.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.075+0000 7fedc2d7e640 1 -- 172.21.15.84:0/1508148498 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedbc19d370 con 0x7fedbc06a8b0 2024-05-22T23:07:16.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.075+0000 7fedb9ffb640 1 -- 172.21.15.84:0/1508148498 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda8018990 con 0x7fedbc06a8b0 2024-05-22T23:07:16.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.075+0000 7fedc2d7e640 1 -- 172.21.15.84:0/1508148498 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedbc1a4c00 con 0x7fedbc06a8b0 2024-05-22T23:07:16.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.077+0000 7fedb9ffb640 1 -- 172.21.15.84:0/1508148498 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7feda8010ac0 con 0x7fedbc06a8b0 2024-05-22T23:07:16.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.077+0000 7fedc5b6d740 1 Processor -- start 2024-05-22T23:07:16.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.077+0000 7fedc5b6d740 1 -- start start 2024-05-22T23:07:16.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.077+0000 7fedb9ffb640 1 --2- 172.21.15.84:0/1508148498 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fedac0a2d00 0x7fedac0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.078+0000 7fedc5b6d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5d04a0 0x55f87d5ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.078+0000 7fedc5b6d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5cedb0 0x55f87d5cf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.078+0000 7fedc5b6d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f87d5d9b30 0x55f87d5dbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.078+0000 7fedc5b6d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f87d532db0 con 0x55f87d5cedb0 2024-05-22T23:07:16.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.078+0000 7fedc5b6d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f87d4d1360 con 0x55f87d5d04a0 2024-05-22T23:07:16.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.078+0000 7fedc5b6d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f87d52ed70 con 0x55f87d5d9b30 2024-05-22T23:07:16.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.078+0000 7fedb9ffb640 1 -- 172.21.15.84:0/1508148498 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7feda80cf7f0 con 0x7fedbc06a8b0 2024-05-22T23:07:16.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.078+0000 7fedc131b640 1 --2- 172.21.15.84:0/1508148498 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fedac0a2d00 0x7fedac0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.078+0000 7fedc0b1a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5d04a0 0x55f87d5ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.078+0000 7fedbbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5cedb0 0x55f87d5cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.078+0000 7fedc0b1a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5d04a0 0x55f87d5ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:36604/0 (socket says 172.21.15.84:36604) 2024-05-22T23:07:16.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.078+0000 7fedc0b1a640 1 -- 172.21.15.84:0/4126516055 learned_addr learned my addr 172.21.15.84:0/4126516055 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:16.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.078+0000 7fedc131b640 1 --2- 172.21.15.84:0/4126516055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f87d5d9b30 0x55f87d5dbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.078+0000 7fedc0b1a640 1 -- 172.21.15.84:0/4126516055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f87d5d9b30 msgr2=0x55f87d5dbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:16.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.078+0000 7fedc0b1a640 1 --2- 172.21.15.84:0/4126516055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f87d5d9b30 0x55f87d5dbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.078+0000 7fedc0b1a640 1 -- 172.21.15.84:0/4126516055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5cedb0 msgr2=0x55f87d5cf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:16.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.078+0000 7fedc0b1a640 1 --2- 172.21.15.84:0/4126516055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5cedb0 0x55f87d5cf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.078+0000 7fedc0b1a640 1 -- 172.21.15.84:0/4126516055 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f87d4f4a60 con 0x55f87d5d04a0 2024-05-22T23:07:16.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.078+0000 7fedc131b640 1 --2- 172.21.15.84:0/1508148498 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fedac0a2d00 0x7fedac0a51c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fedb4006010 tx=0x7fedb4031640 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:16.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.078+0000 7fedc0b1a640 1 --2- 172.21.15.84:0/4126516055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5d04a0 0x55f87d5ce7e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fedb0009c20 tx=0x7fedb0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bdc8d7d0a299404 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:16.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.078+0000 7feda6ffd640 1 -- 172.21.15.84:0/4126516055 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb001e070 con 0x55f87d5d04a0 2024-05-22T23:07:16.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.078+0000 7feda6ffd640 1 -- 172.21.15.84:0/4126516055 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedb0019b10 con 0x55f87d5d04a0 2024-05-22T23:07:16.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.079+0000 7feda6ffd640 1 -- 172.21.15.84:0/4126516055 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb000e9e0 con 0x55f87d5d04a0 2024-05-22T23:07:16.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.079+0000 7fedc5b6d740 1 -- 172.21.15.84:0/4126516055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5d04a0 msgr2=0x55f87d5ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:16.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.079+0000 7fedc5b6d740 1 --2- 172.21.15.84:0/4126516055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5d04a0 0x55f87d5ce7e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fedb0009c20 tx=0x7fedb0011810 comp rx=0 tx=0).stop 2024-05-22T23:07:16.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.079+0000 7fedc5b6d740 1 -- 172.21.15.84:0/4126516055 shutdown_connections 2024-05-22T23:07:16.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.079+0000 7fedc5b6d740 1 --2- 172.21.15.84:0/4126516055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f87d5d9b30 0x55f87d5dbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.079+0000 7fedc5b6d740 1 --2- 172.21.15.84:0/4126516055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5cedb0 0x55f87d5cf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.079+0000 7fedc5b6d740 1 --2- 172.21.15.84:0/4126516055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5d04a0 0x55f87d5ce7e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.079+0000 7fedc5b6d740 1 -- 172.21.15.84:0/4126516055 >> 172.21.15.84:0/4126516055 conn(0x55f87d5978e0 msgr2=0x55f87d5cb310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:16.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.079+0000 7fedc5b6d740 1 -- 172.21.15.84:0/4126516055 shutdown_connections 2024-05-22T23:07:16.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.079+0000 7fedc5b6d740 1 -- 172.21.15.84:0/4126516055 wait complete. 2024-05-22T23:07:16.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.080+0000 7fedc5b6d740 1 -- start start 2024-05-22T23:07:16.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.080+0000 7fedc5b6d740 1 Processor -- start 2024-05-22T23:07:16.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.080+0000 7fedc5b6d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f87d5cedb0 0x55f87d5d6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.080+0000 7fedc5b6d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5d04a0 0x55f87d5d6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.080+0000 7fedc5b6d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5d9b30 0x55f87d5d74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.080+0000 7fedc5b6d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f87d5e0dd0 con 0x55f87d5d04a0 2024-05-22T23:07:16.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.080+0000 7fedc5b6d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f87d4ed0f0 con 0x55f87d5d9b30 2024-05-22T23:07:16.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.080+0000 7fedc5b6d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f87d5e0f50 con 0x55f87d5cedb0 2024-05-22T23:07:16.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.080+0000 7fedc5b6d740 10 client.0 _ll_register_callbacks cb 0x55f87d4f1f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:07:16.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.080+0000 7fedbbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5d04a0 0x55f87d5d6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.080+0000 7fedbbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5d04a0 0x55f87d5d6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51068/0 (socket says 172.21.15.84:51068) 2024-05-22T23:07:16.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.080+0000 7fedbbfff640 1 -- 172.21.15.84:0/3215158925 learned_addr learned my addr 172.21.15.84:0/3215158925 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:16.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.080+0000 7fedc131b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5d9b30 0x55f87d5d74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.080+0000 7fedc0b1a640 1 --2- 172.21.15.84:0/3215158925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f87d5cedb0 0x55f87d5d6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.080+0000 7fedbbfff640 1 -- 172.21.15.84:0/3215158925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f87d5cedb0 msgr2=0x55f87d5d6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:16.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.080+0000 7fedbbfff640 1 --2- 172.21.15.84:0/3215158925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f87d5cedb0 0x55f87d5d6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.081+0000 7fedbbfff640 1 -- 172.21.15.84:0/3215158925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5d9b30 msgr2=0x55f87d5d74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:16.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.081+0000 7fedbbfff640 1 --2- 172.21.15.84:0/3215158925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5d9b30 0x55f87d5d74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.081+0000 7fedbbfff640 1 -- 172.21.15.84:0/3215158925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f87d3107d0 con 0x55f87d5d04a0 2024-05-22T23:07:16.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.081+0000 7fedc131b640 1 --2- 172.21.15.84:0/3215158925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5d9b30 0x55f87d5d74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:16.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.081+0000 7fedbbfff640 1 --2- 172.21.15.84:0/3215158925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5d04a0 0x55f87d5d6f90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7feda809bca0 tx=0x7feda809c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269644430c5eaf3f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:16.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.081+0000 7fed9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:07:16.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.081+0000 7fed9bfff640 1 -- 172.21.15.84:0/3215158925 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda8018d30 con 0x55f87d5d04a0 2024-05-22T23:07:16.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.081+0000 7fed9bfff640 1 -- 172.21.15.84:0/3215158925 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7feda809b160 con 0x55f87d5d04a0 2024-05-22T23:07:16.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.081+0000 7fedc5b6d740 1 -- 172.21.15.84:0/3215158925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedb0019b10 con 0x55f87d5d04a0 2024-05-22T23:07:16.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.081+0000 7fed9bfff640 1 -- 172.21.15.84:0/3215158925 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7feda80a4980 con 0x55f87d5d04a0 2024-05-22T23:07:16.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.081+0000 7fedc5b6d740 10 client.16260 fetch_fsmap learned FSMap version 18 2024-05-22T23:07:16.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.081+0000 7fedc5b6d740 1 -- 172.21.15.84:0/3215158925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55f87d2463f0 con 0x55f87d5d04a0 2024-05-22T23:07:16.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.081+0000 7fed9bfff640 1 -- 172.21.15.84:0/3215158925 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedb0019b10 con 0x55f87d5d04a0 2024-05-22T23:07:16.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.081+0000 7fed9bfff640 1 -- 172.21.15.84:0/3215158925 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7feda8097a60 con 0x55f87d5d04a0 2024-05-22T23:07:16.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.081+0000 7fedc5b6d740 10 client.16260 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:07:16.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.081+0000 7fedc5b6d740 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:07:16.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.081+0000 7fedc5b6d740 20 client.16260 populate_metadata read hostname 'smithi084' 2024-05-22T23:07:16.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.081+0000 7fedc5b6d740 1 --2- 172.21.15.84:0/3215158925 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55f87d5ee410 0x55f87d5f0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.081+0000 7fedc5b6d740 4 client.16260 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:07:16.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.081+0000 7fedc5b6d740 1 -- 172.21.15.84:0/3215158925 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f87d5404e0 con 0x55f87d5ee410 2024-05-22T23:07:16.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.082+0000 7fedc0b1a640 1 --2- 172.21.15.84:0/3215158925 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55f87d5ee410 0x55f87d5f0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.082+0000 7fedc0b1a640 1 --2- 172.21.15.84:0/3215158925 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55f87d5ee410 0x55f87d5f0800 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af68243d7ea8b364 server_cookie=278bd4d17c5364ec in_seq=0 out_seq=0 2024-05-22T23:07:16.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.082+0000 7fed9bfff640 10 client.16260 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:16.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.086+0000 7fed9bfff640 1 -- 172.21.15.84:0/3215158925 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f87d5404e0 con 0x55f87d5ee410 2024-05-22T23:07:16.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.086+0000 7fed9bfff640 10 client.16260 handle_command_reply: tid=0 2024-05-22T23:07:16.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.086+0000 7fedc5b6d740 1 client.16260 shutdown 2024-05-22T23:07:16.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.086+0000 7fedc5b6d740 1 -- 172.21.15.84:0/3215158925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5d04a0 msgr2=0x55f87d5d6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:16.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.086+0000 7fedc5b6d740 1 --2- 172.21.15.84:0/3215158925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5d04a0 0x55f87d5d6f90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7feda809bca0 tx=0x7feda809c4d0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.086+0000 7fedc5b6d740 1 -- 172.21.15.84:0/3215158925 shutdown_connections 2024-05-22T23:07:16.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.086+0000 7fedc5b6d740 1 --2- 172.21.15.84:0/3215158925 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55f87d5ee410 0x55f87d5f0800 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.086+0000 7fed9bfff640 0 client.16260 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:16.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.086+0000 7fedc5b6d740 1 --2- 172.21.15.84:0/3215158925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5d9b30 0x55f87d5d74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.086+0000 7fedc5b6d740 1 --2- 172.21.15.84:0/3215158925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5d04a0 0x55f87d5d6f90 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.086+0000 7fedc5b6d740 1 --2- 172.21.15.84:0/3215158925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f87d5cedb0 0x55f87d5d6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.086+0000 7fedc5b6d740 1 -- 172.21.15.84:0/3215158925 >> 172.21.15.84:0/3215158925 conn(0x55f87d5978e0 msgr2=0x55f87d5d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:16.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.086+0000 7fedc5b6d740 1 -- 172.21.15.84:0/3215158925 shutdown_connections 2024-05-22T23:07:16.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.086+0000 7fedc5b6d740 1 -- 172.21.15.84:0/3215158925 wait complete. 2024-05-22T23:07:16.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.086+0000 7fedc5b6d740 20 client.16260 trim_cache size 0 max 16384 2024-05-22T23:07:16.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.105+0000 7fedc5b6d740 1 Processor -- start 2024-05-22T23:07:16.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.105+0000 7fedc5b6d740 1 -- start start 2024-05-22T23:07:16.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.106+0000 7fedc5b6d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5cedb0 0x55f87d5e9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.106+0000 7fedc5b6d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5d04a0 0x55f87d5d8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.106+0000 7fedc5b6d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f87d5d9b30 0x55f87d5f6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.106+0000 7fedc5b6d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc1a93d0 con 0x55f87d5d04a0 2024-05-22T23:07:16.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.106+0000 7fedc5b6d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc1abc50 con 0x55f87d5cedb0 2024-05-22T23:07:16.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.106+0000 7fedc5b6d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc1a9a40 con 0x55f87d5d9b30 2024-05-22T23:07:16.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.106+0000 7fedbbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5d04a0 0x55f87d5d8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.106+0000 7fedc0b1a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5cedb0 0x55f87d5e9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.106+0000 7fedbbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5d04a0 0x55f87d5d8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51078/0 (socket says 172.21.15.84:51078) 2024-05-22T23:07:16.113 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.106+0000 7fedbbfff640 1 -- 172.21.15.84:0/1931081160 learned_addr learned my addr 172.21.15.84:0/1931081160 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:16.113 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.106+0000 7fedc131b640 1 --2- 172.21.15.84:0/1931081160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f87d5d9b30 0x55f87d5f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.113 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.106+0000 7fedbbfff640 1 -- 172.21.15.84:0/1931081160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f87d5d9b30 msgr2=0x55f87d5f6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:16.113 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.106+0000 7fedbbfff640 1 --2- 172.21.15.84:0/1931081160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f87d5d9b30 0x55f87d5f6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.113 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.106+0000 7fedbbfff640 1 -- 172.21.15.84:0/1931081160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5cedb0 msgr2=0x55f87d5e9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:16.113 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.106+0000 7fedbbfff640 1 --2- 172.21.15.84:0/1931081160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5cedb0 0x55f87d5e9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.113 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.106+0000 7fedbbfff640 1 -- 172.21.15.84:0/1931081160 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f87d2060f0 con 0x55f87d5d04a0 2024-05-22T23:07:16.113 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.106+0000 7fedc0b1a640 1 --2- 172.21.15.84:0/1931081160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5cedb0 0x55f87d5e9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:16.113 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.106+0000 7fedbbfff640 1 --2- 172.21.15.84:0/1931081160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5d04a0 0x55f87d5d8990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7feda8099fc0 tx=0x7feda809ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5484212c8b2cb9a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:16.113 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.106+0000 7feda67fc640 1 -- 172.21.15.84:0/1931081160 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda809add0 con 0x55f87d5d04a0 2024-05-22T23:07:16.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.106+0000 7feda67fc640 1 -- 172.21.15.84:0/1931081160 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feda809af70 con 0x55f87d5d04a0 2024-05-22T23:07:16.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.106+0000 7feda67fc640 1 -- 172.21.15.84:0/1931081160 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda8018d90 con 0x55f87d5d04a0 2024-05-22T23:07:16.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.107+0000 7fedc5b6d740 1 -- 172.21.15.84:0/1931081160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5d04a0 msgr2=0x55f87d5d8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:16.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.107+0000 7fedc5b6d740 1 --2- 172.21.15.84:0/1931081160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5d04a0 0x55f87d5d8990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7feda8099fc0 tx=0x7feda809ad60 comp rx=0 tx=0).stop 2024-05-22T23:07:16.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.107+0000 7fedc5b6d740 1 -- 172.21.15.84:0/1931081160 shutdown_connections 2024-05-22T23:07:16.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.107+0000 7fedc5b6d740 1 --2- 172.21.15.84:0/1931081160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f87d5d9b30 0x55f87d5f6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.107+0000 7fedc5b6d740 1 --2- 172.21.15.84:0/1931081160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5d04a0 0x55f87d5d8990 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.107+0000 7fedc5b6d740 1 --2- 172.21.15.84:0/1931081160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5cedb0 0x55f87d5e9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.107+0000 7fedc5b6d740 1 -- 172.21.15.84:0/1931081160 >> 172.21.15.84:0/1931081160 conn(0x55f87d5978e0 msgr2=0x55f87d5d8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:16.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.107+0000 7fedc5b6d740 1 -- 172.21.15.84:0/1931081160 shutdown_connections 2024-05-22T23:07:16.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.107+0000 7fedc5b6d740 1 -- 172.21.15.84:0/1931081160 wait complete. 2024-05-22T23:07:16.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.108+0000 7fedc5b6d740 1 -- start start 2024-05-22T23:07:16.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.108+0000 7fedc5b6d740 1 Processor -- start 2024-05-22T23:07:16.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.108+0000 7fedc5b6d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f87d5cedb0 0x55f87d5e61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.108+0000 7fedc5b6d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5d04a0 0x55f87d5e6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.108+0000 7fedc5b6d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5d9b30 0x55f87d5d2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.108+0000 7fedc5b6d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc1aa110 con 0x55f87d5d9b30 2024-05-22T23:07:16.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.108+0000 7fedc5b6d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc1ab580 con 0x55f87d5d04a0 2024-05-22T23:07:16.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.108+0000 7fedc5b6d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc1aa7e0 con 0x55f87d5cedb0 2024-05-22T23:07:16.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.108+0000 7fedc5b6d740 10 client.0 _ll_register_callbacks cb 0x7feda800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:07:16.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.108+0000 7fedc131b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5d9b30 0x55f87d5d2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.108+0000 7fedc0b1a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f87d5cedb0 0x55f87d5e61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.108+0000 7fedc131b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5d9b30 0x55f87d5d2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51090/0 (socket says 172.21.15.84:51090) 2024-05-22T23:07:16.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.108+0000 7fedc131b640 1 -- 172.21.15.84:0/1760383718 learned_addr learned my addr 172.21.15.84:0/1760383718 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:16.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.108+0000 7fedbbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5d04a0 0x55f87d5e6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.109+0000 7fedc131b640 1 -- 172.21.15.84:0/1760383718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f87d5cedb0 msgr2=0x55f87d5e61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:16.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.109+0000 7fedc131b640 1 --2- 172.21.15.84:0/1760383718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f87d5cedb0 0x55f87d5e61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.109+0000 7fedc131b640 1 -- 172.21.15.84:0/1760383718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5d04a0 msgr2=0x55f87d5e6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:16.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.109+0000 7fedc131b640 1 --2- 172.21.15.84:0/1760383718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5d04a0 0x55f87d5e6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.109+0000 7fedc131b640 1 -- 172.21.15.84:0/1760383718 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f87d4f1fe0 con 0x55f87d5d9b30 2024-05-22T23:07:16.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.109+0000 7fedc0b1a640 1 --2- 172.21.15.84:0/1760383718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f87d5cedb0 0x55f87d5e61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:16.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.109+0000 7fedbbfff640 1 --2- 172.21.15.84:0/1760383718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5d04a0 0x55f87d5e6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:16.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.109+0000 7fedc131b640 1 --2- 172.21.15.84:0/1760383718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5d9b30 0x55f87d5d2cc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fedb4003f00 tx=0x7fedb402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45fc2172c00090fb server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:16.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.109+0000 7fed9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:07:16.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.109+0000 7fed9bfff640 1 -- 172.21.15.84:0/1760383718 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb40307a0 con 0x55f87d5d9b30 2024-05-22T23:07:16.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.109+0000 7fed9bfff640 1 -- 172.21.15.84:0/1760383718 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fedb4030940 con 0x55f87d5d9b30 2024-05-22T23:07:16.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.109+0000 7fedc5b6d740 1 -- 172.21.15.84:0/1760383718 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feda809af70 con 0x55f87d5d9b30 2024-05-22T23:07:16.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.109+0000 7fed9bfff640 1 -- 172.21.15.84:0/1760383718 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fedb4030b40 con 0x55f87d5d9b30 2024-05-22T23:07:16.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.109+0000 7fedc5b6d740 10 client.16266 fetch_fsmap learned FSMap version 18 2024-05-22T23:07:16.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.109+0000 7fedc5b6d740 1 -- 172.21.15.84:0/1760383718 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55f87d4c31c0 con 0x55f87d5d9b30 2024-05-22T23:07:16.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.109+0000 7fed9bfff640 1 -- 172.21.15.84:0/1760383718 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feda809af70 con 0x55f87d5d9b30 2024-05-22T23:07:16.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.110+0000 7fed9bfff640 1 -- 172.21.15.84:0/1760383718 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fedb4049040 con 0x55f87d5d9b30 2024-05-22T23:07:16.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.110+0000 7fedc5b6d740 10 client.16266 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:07:16.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.110+0000 7fedc5b6d740 10 client.16266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:07:16.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.110+0000 7fedc5b6d740 20 client.16266 populate_metadata read hostname 'smithi084' 2024-05-22T23:07:16.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.110+0000 7fedc5b6d740 1 --2- 172.21.15.84:0/1760383718 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55f87d5ee410 0x55f87d656490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.110+0000 7fedc5b6d740 4 client.16266 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:07:16.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.110+0000 7fedc5b6d740 1 -- 172.21.15.84:0/1760383718 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f87d53a830 con 0x55f87d5ee410 2024-05-22T23:07:16.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.110+0000 7fedc0b1a640 1 --2- 172.21.15.84:0/1760383718 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55f87d5ee410 0x55f87d656490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.110+0000 7fed9bfff640 10 client.16266 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:16.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.110+0000 7fedc0b1a640 1 --2- 172.21.15.84:0/1760383718 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55f87d5ee410 0x55f87d656490 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=293abde4fa27c0c9 server_cookie=580b1adf21fad98 in_seq=0 out_seq=0 2024-05-22T23:07:16.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.112+0000 7fed9bfff640 1 -- 172.21.15.84:0/1760383718 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f87d53a830 con 0x55f87d5ee410 2024-05-22T23:07:16.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.112+0000 7fed9bfff640 10 client.16266 handle_command_reply: tid=0 2024-05-22T23:07:16.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.112+0000 7fedc5b6d740 1 client.16266 shutdown 2024-05-22T23:07:16.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.112+0000 7fedc5b6d740 1 -- 172.21.15.84:0/1760383718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5d9b30 msgr2=0x55f87d5d2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:16.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.112+0000 7fedc5b6d740 1 --2- 172.21.15.84:0/1760383718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5d9b30 0x55f87d5d2cc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fedb4003f00 tx=0x7fedb402a8e0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.112+0000 7fedc5b6d740 1 -- 172.21.15.84:0/1760383718 shutdown_connections 2024-05-22T23:07:16.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.112+0000 7fedc5b6d740 1 --2- 172.21.15.84:0/1760383718 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55f87d5ee410 0x55f87d656490 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.112+0000 7fedc5b6d740 1 --2- 172.21.15.84:0/1760383718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f87d5d9b30 0x55f87d5d2cc0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.112+0000 7fedc5b6d740 1 --2- 172.21.15.84:0/1760383718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f87d5d04a0 0x55f87d5e6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.112+0000 7fed9bfff640 0 client.16266 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:16.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.112+0000 7fedc5b6d740 1 --2- 172.21.15.84:0/1760383718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f87d5cedb0 0x55f87d5e61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.112+0000 7fedc5b6d740 1 -- 172.21.15.84:0/1760383718 >> 172.21.15.84:0/1760383718 conn(0x55f87d5978e0 msgr2=0x55f87d5d6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:16.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.112+0000 7fedc5b6d740 1 -- 172.21.15.84:0/1760383718 shutdown_connections 2024-05-22T23:07:16.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.112+0000 7fedc5b6d740 1 -- 172.21.15.84:0/1760383718 wait complete. 2024-05-22T23:07:16.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.112+0000 7fedc5b6d740 20 client.16266 trim_cache size 0 max 16384 2024-05-22T23:07:16.125 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:07:16.125 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "no active scrubs running", 2024-05-22T23:07:16.125 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": {} 2024-05-22T23:07:16.125 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:07:16.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.113+0000 7fedc2d7e640 1 -- 172.21.15.84:0/1508148498 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fedac0a2d00 msgr2=0x7fedac0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:16.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.113+0000 7fedc2d7e640 1 --2- 172.21.15.84:0/1508148498 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fedac0a2d00 0x7fedac0a51c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fedb4006010 tx=0x7fedb4031640 comp rx=0 tx=0).stop 2024-05-22T23:07:16.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.113+0000 7fedc2d7e640 1 -- 172.21.15.84:0/1508148498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc06a8b0 msgr2=0x7fedbc1985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:16.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.113+0000 7fedc2d7e640 1 --2- 172.21.15.84:0/1508148498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc06a8b0 0x7fedbc1985d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7feda800e9f0 tx=0x7feda800eeb0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.113+0000 7fedc2d7e640 1 -- 172.21.15.84:0/1508148498 shutdown_connections 2024-05-22T23:07:16.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.113+0000 7fedc2d7e640 1 --2- 172.21.15.84:0/1508148498 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fedac0a2d00 0x7fedac0a51c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.113+0000 7fedc2d7e640 1 --2- 172.21.15.84:0/1508148498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedbc107860 0x7fedbc19c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.113+0000 7fedc2d7e640 1 --2- 172.21.15.84:0/1508148498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc06a8b0 0x7fedbc1985d0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.113+0000 7fedc2d7e640 1 --2- 172.21.15.84:0/1508148498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fedbc069ef0 0x7fedbc198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.113+0000 7fedc2d7e640 1 -- 172.21.15.84:0/1508148498 >> 172.21.15.84:0/1508148498 conn(0x7fedbc0777d0 msgr2=0x7fedbc0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:16.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.113+0000 7fedc2d7e640 1 -- 172.21.15.84:0/1508148498 shutdown_connections 2024-05-22T23:07:16.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.113+0000 7fedc2d7e640 1 -- 172.21.15.84:0/1508148498 wait complete. 2024-05-22T23:07:16.129 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-22T23:07:16.129 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:07:16.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.233+0000 7f20e9fd5640 1 -- 172.21.15.84:0/2170755628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20e4107860 msgr2=0x7f20e4107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:16.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.233+0000 7f20e9fd5640 1 --2- 172.21.15.84:0/2170755628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20e4107860 0x7f20e4107c40 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009930 tx=0x7f20d402a420 comp rx=0 tx=0).stop 2024-05-22T23:07:16.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.233+0000 7f20e9fd5640 1 -- 172.21.15.84:0/2170755628 shutdown_connections 2024-05-22T23:07:16.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.233+0000 7f20e9fd5640 1 --2- 172.21.15.84:0/2170755628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20e4102210 0x7f20e410e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.233+0000 7f20e9fd5640 1 --2- 172.21.15.84:0/2170755628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20e4101850 0x7f20e4101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.233+0000 7f20e9fd5640 1 --2- 172.21.15.84:0/2170755628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20e4107860 0x7f20e4107c40 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.233+0000 7f20e9fd5640 1 -- 172.21.15.84:0/2170755628 >> 172.21.15.84:0/2170755628 conn(0x7f20e40fd300 msgr2=0x7f20e40ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:16.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.233+0000 7f20e9fd5640 1 -- 172.21.15.84:0/2170755628 shutdown_connections 2024-05-22T23:07:16.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.233+0000 7f20e9fd5640 1 -- 172.21.15.84:0/2170755628 wait complete. 2024-05-22T23:07:16.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.234+0000 7f20e9fd5640 1 Processor -- start 2024-05-22T23:07:16.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.234+0000 7f20e9fd5640 1 -- start start 2024-05-22T23:07:16.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.234+0000 7f20e9fd5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20e4101850 0x7f20e4198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.234+0000 7f20e9fd5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20e4102210 0x7f20e4198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.234+0000 7f20e9fd5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20e4107860 0x7f20e419ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.234+0000 7f20e9fd5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4111c70 con 0x7f20e4102210 2024-05-22T23:07:16.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.234+0000 7f20e9fd5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4111af0 con 0x7f20e4101850 2024-05-22T23:07:16.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.234+0000 7f20e9fd5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4111df0 con 0x7f20e4107860 2024-05-22T23:07:16.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.235+0000 7f20e2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20e4102210 0x7f20e4198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.235+0000 7f20e2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20e4102210 0x7f20e4198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51096/0 (socket says 172.21.15.84:51096) 2024-05-22T23:07:16.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.235+0000 7f20e2ffd640 1 -- 172.21.15.84:0/360755273 learned_addr learned my addr 172.21.15.84:0/360755273 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:16.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.235+0000 7f20e3fff640 1 --2- 172.21.15.84:0/360755273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20e4107860 0x7f20e419ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.235+0000 7f20e37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20e4101850 0x7f20e4198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.235+0000 7f20e2ffd640 1 -- 172.21.15.84:0/360755273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20e4107860 msgr2=0x7f20e419ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:16.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.235+0000 7f20e2ffd640 1 --2- 172.21.15.84:0/360755273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20e4107860 0x7f20e419ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.235+0000 7f20e2ffd640 1 -- 172.21.15.84:0/360755273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20e4101850 msgr2=0x7f20e4198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:16.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.235+0000 7f20e2ffd640 1 --2- 172.21.15.84:0/360755273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20e4101850 0x7f20e4198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.235+0000 7f20e2ffd640 1 -- 172.21.15.84:0/360755273 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e419d140 con 0x7f20e4102210 2024-05-22T23:07:16.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.235+0000 7f20e3fff640 1 --2- 172.21.15.84:0/360755273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20e4107860 0x7f20e419ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:16.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.235+0000 7f20e37fe640 1 --2- 172.21.15.84:0/360755273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20e4101850 0x7f20e4198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:16.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.235+0000 7f20e2ffd640 1 --2- 172.21.15.84:0/360755273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20e4102210 0x7f20e4198690 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f20d800ebc0 tx=0x7f20d800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:16.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.235+0000 7f20e0ff9640 1 -- 172.21.15.84:0/360755273 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8010630 con 0x7f20e4102210 2024-05-22T23:07:16.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.235+0000 7f20e0ff9640 1 -- 172.21.15.84:0/360755273 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20d8010cb0 con 0x7f20e4102210 2024-05-22T23:07:16.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.235+0000 7f20e9fd5640 1 -- 172.21.15.84:0/360755273 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e419d430 con 0x7f20e4102210 2024-05-22T23:07:16.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.235+0000 7f20e0ff9640 1 -- 172.21.15.84:0/360755273 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8018bd0 con 0x7f20e4102210 2024-05-22T23:07:16.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.235+0000 7f20e9fd5640 1 -- 172.21.15.84:0/360755273 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e41a4cc0 con 0x7f20e4102210 2024-05-22T23:07:16.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.237+0000 7f20e9fd5640 1 -- 172.21.15.84:0/360755273 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20e4068690 con 0x7f20e4102210 2024-05-22T23:07:16.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.238+0000 7f20e0ff9640 1 -- 172.21.15.84:0/360755273 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f20d80107d0 con 0x7f20e4102210 2024-05-22T23:07:16.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.238+0000 7f20e0ff9640 1 --2- 172.21.15.84:0/360755273 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f20cc0a2d00 0x7f20cc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.238+0000 7f20e37fe640 1 --2- 172.21.15.84:0/360755273 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f20cc0a2d00 0x7f20cc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.238+0000 7f20e0ff9640 1 -- 172.21.15.84:0/360755273 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f20d80cfa30 con 0x7f20e4102210 2024-05-22T23:07:16.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.239+0000 7f20e37fe640 1 --2- 172.21.15.84:0/360755273 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f20cc0a2d00 0x7f20cc0a51c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009900 tx=0x7f20d40058d0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:16.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.243+0000 7f20e0ff9640 1 -- 172.21.15.84:0/360755273 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f20d809c000 con 0x7f20e4102210 2024-05-22T23:07:16.436 INFO:tasks.workunit.client.0.smithi084.stdout: 1 692968 15.22 MB/sec execute 122 sec latency 29.689 ms 2024-05-22T23:07:16.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.485+0000 7f20e9fd5640 1 -- 172.21.15.84:0/360755273 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20e4109e20 con 0x7f20e4102210 2024-05-22T23:07:16.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.486+0000 7f20e0ff9640 1 -- 172.21.15.84:0/360755273 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7f20d80a0300 con 0x7f20e4102210 2024-05-22T23:07:16.492 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:07:16.492 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:07:16.492 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:07:16.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.488+0000 7f20e9fd5640 1 -- 172.21.15.84:0/360755273 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f20cc0a2d00 msgr2=0x7f20cc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:16.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.488+0000 7f20e9fd5640 1 --2- 172.21.15.84:0/360755273 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f20cc0a2d00 0x7f20cc0a51c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009900 tx=0x7f20d40058d0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.488+0000 7f20e9fd5640 1 -- 172.21.15.84:0/360755273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20e4102210 msgr2=0x7f20e4198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:16.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.488+0000 7f20e9fd5640 1 --2- 172.21.15.84:0/360755273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20e4102210 0x7f20e4198690 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f20d800ebc0 tx=0x7f20d800c6a0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.488+0000 7f20e9fd5640 1 -- 172.21.15.84:0/360755273 shutdown_connections 2024-05-22T23:07:16.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.488+0000 7f20e9fd5640 1 --2- 172.21.15.84:0/360755273 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f20cc0a2d00 0x7f20cc0a51c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.488+0000 7f20e9fd5640 1 --2- 172.21.15.84:0/360755273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20e4107860 0x7f20e419ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.488+0000 7f20e9fd5640 1 --2- 172.21.15.84:0/360755273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20e4102210 0x7f20e4198690 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.488+0000 7f20e9fd5640 1 --2- 172.21.15.84:0/360755273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20e4101850 0x7f20e4198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.488+0000 7f20e9fd5640 1 -- 172.21.15.84:0/360755273 >> 172.21.15.84:0/360755273 conn(0x7f20e40fd300 msgr2=0x7f20e40fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:16.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.488+0000 7f20e9fd5640 1 -- 172.21.15.84:0/360755273 shutdown_connections 2024-05-22T23:07:16.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.489+0000 7f20e9fd5640 1 -- 172.21.15.84:0/360755273 wait complete. 2024-05-22T23:07:16.505 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-22T23:07:16.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.620+0000 7f3ac96b3640 1 -- 172.21.15.84:0/3729865696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ac4107850 msgr2=0x7f3ac4107c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:16.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.620+0000 7f3ac96b3640 1 --2- 172.21.15.84:0/3729865696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ac4107850 0x7f3ac4107c30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009930 tx=0x7f3ab402a420 comp rx=0 tx=0).stop 2024-05-22T23:07:16.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.620+0000 7f3ac96b3640 1 -- 172.21.15.84:0/3729865696 shutdown_connections 2024-05-22T23:07:16.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.620+0000 7f3ac96b3640 1 --2- 172.21.15.84:0/3729865696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ac4102200 0x7f3ac410e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.620+0000 7f3ac96b3640 1 --2- 172.21.15.84:0/3729865696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ac4101840 0x7f3ac4101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.620+0000 7f3ac96b3640 1 --2- 172.21.15.84:0/3729865696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ac4107850 0x7f3ac4107c30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.620+0000 7f3ac96b3640 1 -- 172.21.15.84:0/3729865696 >> 172.21.15.84:0/3729865696 conn(0x7f3ac4078cc0 msgr2=0x7f3ac40790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:16.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.620+0000 7f3ac96b3640 1 -- 172.21.15.84:0/3729865696 shutdown_connections 2024-05-22T23:07:16.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.620+0000 7f3ac96b3640 1 -- 172.21.15.84:0/3729865696 wait complete. 2024-05-22T23:07:16.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.621+0000 7f3ac96b3640 1 Processor -- start 2024-05-22T23:07:16.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.621+0000 7f3ac96b3640 1 -- start start 2024-05-22T23:07:16.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.621+0000 7f3ac96b3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ac4101840 0x7f3ac41a44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.621+0000 7f3ac96b3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ac4102200 0x7f3ac41a49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.621+0000 7f3ac96b3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ac4107850 0x7f3ac410d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.621+0000 7f3ac96b3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac4111cb0 con 0x7f3ac4107850 2024-05-22T23:07:16.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.621+0000 7f3ac96b3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac4111b30 con 0x7f3ac4101840 2024-05-22T23:07:16.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.621+0000 7f3ac96b3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac4111e30 con 0x7f3ac4102200 2024-05-22T23:07:16.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.621+0000 7f3ac37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ac4107850 0x7f3ac410d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.621+0000 7f3ac2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ac4101840 0x7f3ac41a44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.621+0000 7f3ac27fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ac4102200 0x7f3ac41a49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.621+0000 7f3ac37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ac4107850 0x7f3ac410d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51106/0 (socket says 172.21.15.84:51106) 2024-05-22T23:07:16.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.621+0000 7f3ac27fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ac4102200 0x7f3ac41a49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:48344/0 (socket says 172.21.15.84:48344) 2024-05-22T23:07:16.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.621+0000 7f3ac37fe640 1 -- 172.21.15.84:0/1891067020 learned_addr learned my addr 172.21.15.84:0/1891067020 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:16.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.621+0000 7f3ac2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ac4101840 0x7f3ac41a44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:36668/0 (socket says 172.21.15.84:36668) 2024-05-22T23:07:16.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.622+0000 7f3ac37fe640 1 -- 172.21.15.84:0/1891067020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ac4102200 msgr2=0x7f3ac41a49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:16.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.622+0000 7f3ac37fe640 1 --2- 172.21.15.84:0/1891067020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ac4102200 0x7f3ac41a49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.622+0000 7f3ac37fe640 1 -- 172.21.15.84:0/1891067020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ac4101840 msgr2=0x7f3ac41a44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:16.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.622+0000 7f3ac37fe640 1 --2- 172.21.15.84:0/1891067020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ac4101840 0x7f3ac41a44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.622+0000 7f3ac37fe640 1 -- 172.21.15.84:0/1891067020 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac410e160 con 0x7f3ac4107850 2024-05-22T23:07:16.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.622+0000 7f3ac2ffd640 1 --2- 172.21.15.84:0/1891067020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ac4101840 0x7f3ac41a44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:16.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.622+0000 7f3ac27fc640 1 --2- 172.21.15.84:0/1891067020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ac4102200 0x7f3ac41a49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:16.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.622+0000 7f3ac37fe640 1 --2- 172.21.15.84:0/1891067020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ac4107850 0x7f3ac410d8d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00eb10 tx=0x7f3abc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:16.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.622+0000 7f3abbfff640 1 -- 172.21.15.84:0/1891067020 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc004b20 con 0x7f3ac4107850 2024-05-22T23:07:16.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.622+0000 7f3abbfff640 1 -- 172.21.15.84:0/1891067020 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3abc004cc0 con 0x7f3ac4107850 2024-05-22T23:07:16.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.622+0000 7f3ac96b3640 1 -- 172.21.15.84:0/1891067020 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac410e450 con 0x7f3ac4107850 2024-05-22T23:07:16.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.622+0000 7f3abbfff640 1 -- 172.21.15.84:0/1891067020 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc010760 con 0x7f3ac4107850 2024-05-22T23:07:16.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.622+0000 7f3ac96b3640 1 -- 172.21.15.84:0/1891067020 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ac41ab440 con 0x7f3ac4107850 2024-05-22T23:07:16.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.624+0000 7f3abbfff640 1 -- 172.21.15.84:0/1891067020 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3abc005020 con 0x7f3ac4107850 2024-05-22T23:07:16.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.625+0000 7f3acc3c2740 1 Processor -- start 2024-05-22T23:07:16.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.625+0000 7f3acc3c2740 1 -- start start 2024-05-22T23:07:16.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.625+0000 7f3acc3c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563710d39f80 0x563710d3a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.625+0000 7f3acc3c2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d3a8a0 0x563710d44a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.625+0000 7f3acc3c2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563710d44f70 0x563710d47360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.625+0000 7f3acc3c2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563710c9edb0 con 0x563710d3a8a0 2024-05-22T23:07:16.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.625+0000 7f3acc3c2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563710c3d400 con 0x563710d44f70 2024-05-22T23:07:16.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.625+0000 7f3acc3c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563710c9ad70 con 0x563710d39f80 2024-05-22T23:07:16.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.625+0000 7f3abbfff640 1 --2- 172.21.15.84:0/1891067020 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f3ab00a2d00 0x7f3ab00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.625+0000 7f3ac37fe640 1 --2- 172.21.15.84:0/1891067020 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f3ab00a2d00 0x7f3ab00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.626+0000 7f3ac27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563710d44f70 0x563710d47360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.625+0000 7f3ac2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d3a8a0 0x563710d44a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.626+0000 7f3ac27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563710d44f70 0x563710d47360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:36684/0 (socket says 172.21.15.84:36684) 2024-05-22T23:07:16.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.626+0000 7f3ac27fc640 1 -- 172.21.15.84:0/138072176 learned_addr learned my addr 172.21.15.84:0/138072176 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:16.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.626+0000 7f3ac27fc640 1 --2- 172.21.15.84:0/138072176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563710d39f80 0x563710d3a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.626+0000 7f3abbfff640 1 -- 172.21.15.84:0/1891067020 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f3abc0cfa90 con 0x7f3ac4107850 2024-05-22T23:07:16.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.626+0000 7f3ac27fc640 1 -- 172.21.15.84:0/138072176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563710d39f80 msgr2=0x563710d3a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:16.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.626+0000 7f3ac27fc640 1 --2- 172.21.15.84:0/138072176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563710d39f80 0x563710d3a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.626+0000 7f3ac27fc640 1 -- 172.21.15.84:0/138072176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d3a8a0 msgr2=0x563710d44a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:16.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.626+0000 7f3ac27fc640 1 --2- 172.21.15.84:0/138072176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d3a8a0 0x563710d44a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.626+0000 7f3ac27fc640 1 -- 172.21.15.84:0/138072176 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563710c2f2f0 con 0x563710d44f70 2024-05-22T23:07:16.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.626+0000 7f3ac37fe640 1 --2- 172.21.15.84:0/1891067020 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f3ab00a2d00 0x7f3ab00a51c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3abc09bcb0 tx=0x7f3abc00efd0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:16.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.626+0000 7f3ac27fc640 1 --2- 172.21.15.84:0/138072176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563710d44f70 0x563710d47360 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3aac011bb0 tx=0x7f3aac004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b33f7f994ea933ed server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:16.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.626+0000 7f3ab97fa640 1 -- 172.21.15.84:0/138072176 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac00fc50 con 0x563710d44f70 2024-05-22T23:07:16.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.626+0000 7f3ab97fa640 1 -- 172.21.15.84:0/138072176 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aac00fdf0 con 0x563710d44f70 2024-05-22T23:07:16.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.626+0000 7f3ab97fa640 1 -- 172.21.15.84:0/138072176 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac01e3a0 con 0x563710d44f70 2024-05-22T23:07:16.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.627+0000 7f3acc3c2740 1 -- 172.21.15.84:0/138072176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563710d44f70 msgr2=0x563710d47360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:16.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.627+0000 7f3acc3c2740 1 --2- 172.21.15.84:0/138072176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563710d44f70 0x563710d47360 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3aac011bb0 tx=0x7f3aac004730 comp rx=0 tx=0).stop 2024-05-22T23:07:16.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.627+0000 7f3acc3c2740 1 -- 172.21.15.84:0/138072176 shutdown_connections 2024-05-22T23:07:16.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.627+0000 7f3acc3c2740 1 --2- 172.21.15.84:0/138072176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563710d44f70 0x563710d47360 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.627+0000 7f3acc3c2740 1 --2- 172.21.15.84:0/138072176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d3a8a0 0x563710d44a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.627+0000 7f3acc3c2740 1 --2- 172.21.15.84:0/138072176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563710d39f80 0x563710d3a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.627+0000 7f3acc3c2740 1 -- 172.21.15.84:0/138072176 >> 172.21.15.84:0/138072176 conn(0x563710d362a0 msgr2=0x563710d36680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:16.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.627+0000 7f3acc3c2740 1 -- 172.21.15.84:0/138072176 shutdown_connections 2024-05-22T23:07:16.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.627+0000 7f3acc3c2740 1 -- 172.21.15.84:0/138072176 wait complete. 2024-05-22T23:07:16.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.627+0000 7f3acc3c2740 1 -- start start 2024-05-22T23:07:16.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.627+0000 7f3acc3c2740 1 Processor -- start 2024-05-22T23:07:16.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.628+0000 7f3acc3c2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563710d39f80 0x563710d41e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.628+0000 7f3acc3c2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d3a8a0 0x563710d423d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.628+0000 7f3acc3c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563710d44f70 0x563710d42910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.628+0000 7f3acc3c2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563710d47d60 con 0x563710d3a8a0 2024-05-22T23:07:16.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.628+0000 7f3acc3c2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563710c592e0 con 0x563710d39f80 2024-05-22T23:07:16.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.628+0000 7f3acc3c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563710d47ee0 con 0x563710d44f70 2024-05-22T23:07:16.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.628+0000 7f3ac27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563710d39f80 0x563710d41e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.628+0000 7f3ac27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563710d39f80 0x563710d41e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:36700/0 (socket says 172.21.15.84:36700) 2024-05-22T23:07:16.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.628+0000 7f3ac27fc640 1 -- 172.21.15.84:0/1119781061 learned_addr learned my addr 172.21.15.84:0/1119781061 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:16.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.628+0000 7f3ac2ffd640 1 --2- 172.21.15.84:0/1119781061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d3a8a0 0x563710d423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.628+0000 7f3acc3c2740 10 client.0 _ll_register_callbacks cb 0x563710c62a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:07:16.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.628+0000 7f3ac27fc640 1 -- 172.21.15.84:0/1119781061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563710d44f70 msgr2=0x563710d42910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:16.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.628+0000 7f3ac27fc640 1 --2- 172.21.15.84:0/1119781061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563710d44f70 0x563710d42910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.628+0000 7f3ac27fc640 1 -- 172.21.15.84:0/1119781061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d3a8a0 msgr2=0x563710d423d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:16.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.628+0000 7f3ac27fc640 1 --2- 172.21.15.84:0/1119781061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d3a8a0 0x563710d423d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.628+0000 7f3ac27fc640 1 -- 172.21.15.84:0/1119781061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637109b23f0 con 0x563710d39f80 2024-05-22T23:07:16.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.628+0000 7f3ac2ffd640 1 --2- 172.21.15.84:0/1119781061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d3a8a0 0x563710d423d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:16.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.628+0000 7f3ac27fc640 1 --2- 172.21.15.84:0/1119781061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563710d39f80 0x563710d41e90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3aac005880 tx=0x7f3aac00ffa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df100f15325fd203 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:16.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.628+0000 7f3aaa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:07:16.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.629+0000 7f3aaa7fc640 1 -- 172.21.15.84:0/1119781061 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac027070 con 0x563710d39f80 2024-05-22T23:07:16.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.629+0000 7f3acc3c2740 1 -- 172.21.15.84:0/1119781061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aac00fdf0 con 0x563710d39f80 2024-05-22T23:07:16.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.629+0000 7f3aaa7fc640 1 -- 172.21.15.84:0/1119781061 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f3aac003f50 con 0x563710d39f80 2024-05-22T23:07:16.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.629+0000 7f3aaa7fc640 1 -- 172.21.15.84:0/1119781061 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f3aac00fc20 con 0x563710d39f80 2024-05-22T23:07:16.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.629+0000 7f3acc3c2740 10 client.25856 fetch_fsmap learned FSMap version 18 2024-05-22T23:07:16.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.629+0000 7f3acc3c2740 1 -- 172.21.15.84:0/1119781061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x563710a7c7d0 con 0x563710d39f80 2024-05-22T23:07:16.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.629+0000 7f3aaa7fc640 1 -- 172.21.15.84:0/1119781061 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aac00fdf0 con 0x563710d39f80 2024-05-22T23:07:16.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.630+0000 7f3aaa7fc640 1 -- 172.21.15.84:0/1119781061 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f3aac01fc80 con 0x563710d39f80 2024-05-22T23:07:16.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.630+0000 7f3acc3c2740 10 client.25856 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:07:16.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.630+0000 7f3acc3c2740 10 client.25856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:07:16.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.630+0000 7f3acc3c2740 20 client.25856 populate_metadata read hostname 'smithi084' 2024-05-22T23:07:16.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.630+0000 7f3acc3c2740 1 --2- 172.21.15.84:0/1119781061 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563710d550e0 0x563710d574d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.630+0000 7f3acc3c2740 4 client.25856 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:07:16.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.630+0000 7f3acc3c2740 1 -- 172.21.15.84:0/1119781061 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563710cac4e0 con 0x563710d550e0 2024-05-22T23:07:16.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.630+0000 7f3ac2ffd640 1 --2- 172.21.15.84:0/1119781061 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563710d550e0 0x563710d574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.630+0000 7f3ac2ffd640 1 --2- 172.21.15.84:0/1119781061 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563710d550e0 0x563710d574d0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6e07fdcde5627b9 server_cookie=195dc598d138c4e6 in_seq=0 out_seq=0 2024-05-22T23:07:16.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.630+0000 7f3aaa7fc640 10 client.25856 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:16.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.631+0000 7f3aaa7fc640 1 -- 172.21.15.84:0/1119781061 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563710cac4e0 con 0x563710d550e0 2024-05-22T23:07:16.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.631+0000 7f3aaa7fc640 10 client.25856 handle_command_reply: tid=0 2024-05-22T23:07:16.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.631+0000 7f3acc3c2740 1 client.25856 shutdown 2024-05-22T23:07:16.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.631+0000 7f3acc3c2740 1 -- 172.21.15.84:0/1119781061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563710d39f80 msgr2=0x563710d41e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:16.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.631+0000 7f3acc3c2740 1 --2- 172.21.15.84:0/1119781061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563710d39f80 0x563710d41e90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3aac005880 tx=0x7f3aac00ffa0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.631+0000 7f3acc3c2740 1 -- 172.21.15.84:0/1119781061 shutdown_connections 2024-05-22T23:07:16.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.631+0000 7f3acc3c2740 1 --2- 172.21.15.84:0/1119781061 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563710d550e0 0x563710d574d0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.631+0000 7f3acc3c2740 1 --2- 172.21.15.84:0/1119781061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563710d44f70 0x563710d42910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.631+0000 7f3aaa7fc640 0 client.25856 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:16.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.631+0000 7f3acc3c2740 1 --2- 172.21.15.84:0/1119781061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d3a8a0 0x563710d423d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.631+0000 7f3acc3c2740 1 --2- 172.21.15.84:0/1119781061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563710d39f80 0x563710d41e90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.631+0000 7f3acc3c2740 1 -- 172.21.15.84:0/1119781061 >> 172.21.15.84:0/1119781061 conn(0x563710d362a0 msgr2=0x563710d354c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:16.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.631+0000 7f3acc3c2740 1 -- 172.21.15.84:0/1119781061 shutdown_connections 2024-05-22T23:07:16.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.631+0000 7f3acc3c2740 1 -- 172.21.15.84:0/1119781061 wait complete. 2024-05-22T23:07:16.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.631+0000 7f3acc3c2740 20 client.25856 trim_cache size 0 max 16384 2024-05-22T23:07:16.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.645+0000 7f3acc3c2740 1 Processor -- start 2024-05-22T23:07:16.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.646+0000 7f3acc3c2740 1 -- start start 2024-05-22T23:07:16.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.646+0000 7f3acc3c2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563710d39f80 0x563710d50c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.646+0000 7f3acc3c2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d3a8a0 0x563710d44550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.646+0000 7f3acc3c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563710d44f70 0x563710d3c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.646+0000 7f3acc3c2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac41b1cf0 con 0x563710d3a8a0 2024-05-22T23:07:16.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.646+0000 7f3acc3c2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac41b4570 con 0x563710d39f80 2024-05-22T23:07:16.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.646+0000 7f3acc3c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac41b2360 con 0x563710d44f70 2024-05-22T23:07:16.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.646+0000 7f3ac2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d3a8a0 0x563710d44550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.646+0000 7f3ac2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d3a8a0 0x563710d44550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51132/0 (socket says 172.21.15.84:51132) 2024-05-22T23:07:16.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.646+0000 7f3ac27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563710d39f80 0x563710d50c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.646+0000 7f3ac2ffd640 1 -- 172.21.15.84:0/1876021349 learned_addr learned my addr 172.21.15.84:0/1876021349 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:16.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.646+0000 7f3ac27fc640 1 --2- 172.21.15.84:0/1876021349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563710d44f70 0x563710d3c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.646+0000 7f3ac2ffd640 1 -- 172.21.15.84:0/1876021349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563710d44f70 msgr2=0x563710d3c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:16.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.646+0000 7f3ac2ffd640 1 --2- 172.21.15.84:0/1876021349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563710d44f70 0x563710d3c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.646+0000 7f3ac2ffd640 1 -- 172.21.15.84:0/1876021349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563710d39f80 msgr2=0x563710d50c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:16.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.646+0000 7f3ac2ffd640 1 --2- 172.21.15.84:0/1876021349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563710d39f80 0x563710d50c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.646+0000 7f3ac2ffd640 1 -- 172.21.15.84:0/1876021349 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563710d31a30 con 0x563710d3a8a0 2024-05-22T23:07:16.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.647+0000 7f3ac2ffd640 1 --2- 172.21.15.84:0/1876021349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d3a8a0 0x563710d44550 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3ab402aa10 tx=0x7f3ab4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211eb197c7c08190 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:16.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.647+0000 7f3ab8ff9640 1 -- 172.21.15.84:0/1876021349 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab4035040 con 0x563710d3a8a0 2024-05-22T23:07:16.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.647+0000 7f3ab8ff9640 1 -- 172.21.15.84:0/1876021349 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ab404b070 con 0x563710d3a8a0 2024-05-22T23:07:16.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.647+0000 7f3ab8ff9640 1 -- 172.21.15.84:0/1876021349 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab4004bf0 con 0x563710d3a8a0 2024-05-22T23:07:16.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.647+0000 7f3acc3c2740 1 -- 172.21.15.84:0/1876021349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d3a8a0 msgr2=0x563710d44550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:16.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.647+0000 7f3acc3c2740 1 --2- 172.21.15.84:0/1876021349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d3a8a0 0x563710d44550 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3ab402aa10 tx=0x7f3ab4000bc0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.648+0000 7f3acc3c2740 1 -- 172.21.15.84:0/1876021349 shutdown_connections 2024-05-22T23:07:16.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.648+0000 7f3acc3c2740 1 --2- 172.21.15.84:0/1876021349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563710d44f70 0x563710d3c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.648+0000 7f3acc3c2740 1 --2- 172.21.15.84:0/1876021349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d3a8a0 0x563710d44550 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.648+0000 7f3acc3c2740 1 --2- 172.21.15.84:0/1876021349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563710d39f80 0x563710d50c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.648+0000 7f3acc3c2740 1 -- 172.21.15.84:0/1876021349 >> 172.21.15.84:0/1876021349 conn(0x563710d362a0 msgr2=0x563710d43e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:16.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.648+0000 7f3acc3c2740 1 -- 172.21.15.84:0/1876021349 shutdown_connections 2024-05-22T23:07:16.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.648+0000 7f3acc3c2740 1 -- 172.21.15.84:0/1876021349 wait complete. 2024-05-22T23:07:16.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.648+0000 7f3acc3c2740 1 -- start start 2024-05-22T23:07:16.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.648+0000 7f3acc3c2740 1 Processor -- start 2024-05-22T23:07:16.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.648+0000 7f3acc3c2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d39f80 0x563710d4d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.648+0000 7f3acc3c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563710d3a8a0 0x563710d3e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.648+0000 7f3acc3c2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563710d44f70 0x563710d3ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.648+0000 7f3acc3c2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac41b2a30 con 0x563710d39f80 2024-05-22T23:07:16.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.648+0000 7f3acc3c2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac41b3ea0 con 0x563710d44f70 2024-05-22T23:07:16.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.648+0000 7f3acc3c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac41b3100 con 0x563710d3a8a0 2024-05-22T23:07:16.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.648+0000 7f3acc3c2740 10 client.0 _ll_register_callbacks cb 0x7f3abc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:07:16.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.648+0000 7f3ac27fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d39f80 0x563710d4d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.648+0000 7f3ac2ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563710d3a8a0 0x563710d3e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.648+0000 7f3ac27fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d39f80 0x563710d4d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51136/0 (socket says 172.21.15.84:51136) 2024-05-22T23:07:16.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.648+0000 7f3ac27fc640 1 -- 172.21.15.84:0/1168762259 learned_addr learned my addr 172.21.15.84:0/1168762259 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:16.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.649+0000 7f3ac27fc640 1 --2- 172.21.15.84:0/1168762259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563710d44f70 0x563710d3ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.649+0000 7f3ac27fc640 1 -- 172.21.15.84:0/1168762259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563710d3a8a0 msgr2=0x563710d3e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:16.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.649+0000 7f3ac27fc640 1 --2- 172.21.15.84:0/1168762259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563710d3a8a0 0x563710d3e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.649+0000 7f3ac27fc640 1 -- 172.21.15.84:0/1168762259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563710d44f70 msgr2=0x563710d3ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:16.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.649+0000 7f3ac27fc640 1 --2- 172.21.15.84:0/1168762259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563710d44f70 0x563710d3ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.649+0000 7f3ac27fc640 1 -- 172.21.15.84:0/1168762259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563710d3f1c0 con 0x563710d39f80 2024-05-22T23:07:16.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.649+0000 7f3ac27fc640 1 --2- 172.21.15.84:0/1168762259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d39f80 0x563710d4d350 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f3aac001eb0 tx=0x7f3aac003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed920241e36032e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:16.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.649+0000 7f3aaa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:07:16.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.649+0000 7f3aaa7fc640 1 -- 172.21.15.84:0/1168762259 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac027070 con 0x563710d39f80 2024-05-22T23:07:16.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.649+0000 7f3aaa7fc640 1 -- 172.21.15.84:0/1168762259 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f3aac0054d0 con 0x563710d39f80 2024-05-22T23:07:16.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.649+0000 7f3acc3c2740 1 -- 172.21.15.84:0/1168762259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ab404b070 con 0x563710d39f80 2024-05-22T23:07:16.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.649+0000 7f3aaa7fc640 1 -- 172.21.15.84:0/1168762259 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f3aac00fdc0 con 0x563710d39f80 2024-05-22T23:07:16.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.649+0000 7f3acc3c2740 10 client.16290 fetch_fsmap learned FSMap version 18 2024-05-22T23:07:16.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.649+0000 7f3acc3c2740 1 -- 172.21.15.84:0/1168762259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x563710dbb1c0 con 0x563710d39f80 2024-05-22T23:07:16.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.650+0000 7f3aaa7fc640 1 -- 172.21.15.84:0/1168762259 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ab404b070 con 0x563710d39f80 2024-05-22T23:07:16.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.650+0000 7f3aaa7fc640 1 -- 172.21.15.84:0/1168762259 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f3aac038410 con 0x563710d39f80 2024-05-22T23:07:16.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.650+0000 7f3acc3c2740 10 client.16290 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:07:16.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.650+0000 7f3acc3c2740 10 client.16290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:07:16.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.650+0000 7f3acc3c2740 20 client.16290 populate_metadata read hostname 'smithi084' 2024-05-22T23:07:16.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.650+0000 7f3acc3c2740 1 --2- 172.21.15.84:0/1168762259 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563710dbb8c0 0x563710dbdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:16.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.650+0000 7f3acc3c2740 4 client.16290 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-22T23:07:16.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.650+0000 7f3acc3c2740 1 -- 172.21.15.84:0/1168762259 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563710ca6830 con 0x563710dbb8c0 2024-05-22T23:07:16.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.650+0000 7f3ac2ffd640 1 --2- 172.21.15.84:0/1168762259 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563710dbb8c0 0x563710dbdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:16.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.650+0000 7f3aaa7fc640 10 client.16290 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:16.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.650+0000 7f3ac2ffd640 1 --2- 172.21.15.84:0/1168762259 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563710dbb8c0 0x563710dbdce0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=620a190681bb3c30 server_cookie=e524db9b190b88fc in_seq=0 out_seq=0 2024-05-22T23:07:16.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.652+0000 7f3aaa7fc640 1 -- 172.21.15.84:0/1168762259 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563710ca6830 con 0x563710dbb8c0 2024-05-22T23:07:16.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.652+0000 7f3aaa7fc640 10 client.16290 handle_command_reply: tid=0 2024-05-22T23:07:16.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.652+0000 7f3acc3c2740 1 client.16290 shutdown 2024-05-22T23:07:16.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.652+0000 7f3acc3c2740 1 -- 172.21.15.84:0/1168762259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d39f80 msgr2=0x563710d4d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:16.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.652+0000 7f3acc3c2740 1 --2- 172.21.15.84:0/1168762259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d39f80 0x563710d4d350 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f3aac001eb0 tx=0x7f3aac003c60 comp rx=0 tx=0).stop 2024-05-22T23:07:16.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.652+0000 7f3acc3c2740 1 -- 172.21.15.84:0/1168762259 shutdown_connections 2024-05-22T23:07:16.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.652+0000 7f3acc3c2740 1 --2- 172.21.15.84:0/1168762259 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563710dbb8c0 0x563710dbdce0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.652+0000 7f3acc3c2740 1 --2- 172.21.15.84:0/1168762259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563710d44f70 0x563710d3ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.652+0000 7f3acc3c2740 1 --2- 172.21.15.84:0/1168762259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563710d3a8a0 0x563710d3e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.652+0000 7f3aaa7fc640 0 client.16290 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:16.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.652+0000 7f3acc3c2740 1 --2- 172.21.15.84:0/1168762259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563710d39f80 0x563710d4d350 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.652+0000 7f3acc3c2740 1 -- 172.21.15.84:0/1168762259 >> 172.21.15.84:0/1168762259 conn(0x563710d362a0 msgr2=0x563710d43520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:16.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.652+0000 7f3acc3c2740 1 -- 172.21.15.84:0/1168762259 shutdown_connections 2024-05-22T23:07:16.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.652+0000 7f3acc3c2740 1 -- 172.21.15.84:0/1168762259 wait complete. 2024-05-22T23:07:16.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.652+0000 7f3acc3c2740 20 client.16290 trim_cache size 0 max 16384 2024-05-22T23:07:16.662 INFO:teuthology.orchestra.run.smithi084.stdout:[] 2024-05-22T23:07:16.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.653+0000 7f3ac96b3640 1 -- 172.21.15.84:0/1891067020 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f3ab00a2d00 msgr2=0x7f3ab00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:16.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.653+0000 7f3ac96b3640 1 --2- 172.21.15.84:0/1891067020 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f3ab00a2d00 0x7f3ab00a51c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3abc09bcb0 tx=0x7f3abc00efd0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.653+0000 7f3ac96b3640 1 -- 172.21.15.84:0/1891067020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ac4107850 msgr2=0x7f3ac410d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:16.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.653+0000 7f3ac96b3640 1 --2- 172.21.15.84:0/1891067020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ac4107850 0x7f3ac410d8d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00eb10 tx=0x7f3abc00c2d0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.653+0000 7f3ac96b3640 1 -- 172.21.15.84:0/1891067020 shutdown_connections 2024-05-22T23:07:16.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.653+0000 7f3ac96b3640 1 --2- 172.21.15.84:0/1891067020 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f3ab00a2d00 0x7f3ab00a51c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.653+0000 7f3ac96b3640 1 --2- 172.21.15.84:0/1891067020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ac4107850 0x7f3ac410d8d0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.653+0000 7f3ac96b3640 1 --2- 172.21.15.84:0/1891067020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ac4102200 0x7f3ac41a49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.653+0000 7f3ac96b3640 1 --2- 172.21.15.84:0/1891067020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ac4101840 0x7f3ac41a44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:16.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.653+0000 7f3ac96b3640 1 -- 172.21.15.84:0/1891067020 >> 172.21.15.84:0/1891067020 conn(0x7f3ac4078cc0 msgr2=0x7f3ac40ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:16.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.653+0000 7f3ac96b3640 1 -- 172.21.15.84:0/1891067020 shutdown_connections 2024-05-22T23:07:16.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:16.653+0000 7f3ac96b3640 1 -- 172.21.15.84:0/1891067020 wait complete. 2024-05-22T23:07:16.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:16 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/360755273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:07:16.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:16 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/360755273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:07:17.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:16 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/360755273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:07:17.436 INFO:tasks.workunit.client.0.smithi084.stdout: 1 696276 15.25 MB/sec execute 123 sec latency 15.835 ms 2024-05-22T23:07:17.670 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-22T23:07:17.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:07:17.670 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-22T23:07:17.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.780+0000 7f677c04d640 1 -- 172.21.15.84:0/4055563620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6774101100 msgr2=0x7f677410d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:17.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.780+0000 7f677c04d640 1 --2- 172.21.15.84:0/4055563620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6774101100 0x7f677410d610 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f676c009cd0 tx=0x7f676c02a8e0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.780+0000 7f677c04d640 1 -- 172.21.15.84:0/4055563620 shutdown_connections 2024-05-22T23:07:17.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.780+0000 7f677c04d640 1 --2- 172.21.15.84:0/4055563620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6774101100 0x7f677410d610 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.780+0000 7f677c04d640 1 --2- 172.21.15.84:0/4055563620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6774100740 0x7f6774100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.780+0000 7f677c04d640 1 --2- 172.21.15.84:0/4055563620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6774106750 0x7f6774106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.780+0000 7f677c04d640 1 -- 172.21.15.84:0/4055563620 >> 172.21.15.84:0/4055563620 conn(0x7f6774077870 msgr2=0x7f6774077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:17.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.780+0000 7f677c04d640 1 -- 172.21.15.84:0/4055563620 shutdown_connections 2024-05-22T23:07:17.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.780+0000 7f677c04d640 1 -- 172.21.15.84:0/4055563620 wait complete. 2024-05-22T23:07:17.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.781+0000 7f677c04d640 1 Processor -- start 2024-05-22T23:07:17.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.781+0000 7f677c04d640 1 -- start start 2024-05-22T23:07:17.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.781+0000 7f677c04d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6774100740 0x7f6774198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.781+0000 7f677c04d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6774101100 0x7f67741986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.781+0000 7f677c04d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6774106750 0x7f677419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.781+0000 7f677c04d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6774110b20 con 0x7f6774100740 2024-05-22T23:07:17.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.781+0000 7f677c04d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f67741109a0 con 0x7f6774106750 2024-05-22T23:07:17.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.781+0000 7f677c04d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6774110ca0 con 0x7f6774101100 2024-05-22T23:07:17.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.781+0000 7f6779de9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6774100740 0x7f6774198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.781+0000 7f6779de9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6774100740 0x7f6774198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51158/0 (socket says 172.21.15.84:51158) 2024-05-22T23:07:17.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.781+0000 7f67795e8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6774101100 0x7f67741986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.781+0000 7f6779de9640 1 -- 172.21.15.84:0/3662085473 learned_addr learned my addr 172.21.15.84:0/3662085473 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:17.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.781+0000 7f677a5ea640 1 --2- 172.21.15.84:0/3662085473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6774106750 0x7f677419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.782+0000 7f6779de9640 1 -- 172.21.15.84:0/3662085473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6774101100 msgr2=0x7f67741986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:17.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.782+0000 7f6779de9640 1 --2- 172.21.15.84:0/3662085473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6774101100 0x7f67741986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.782+0000 7f6779de9640 1 -- 172.21.15.84:0/3662085473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6774106750 msgr2=0x7f677419ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:17.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.782+0000 7f6779de9640 1 --2- 172.21.15.84:0/3662085473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6774106750 0x7f677419ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.782+0000 7f6779de9640 1 -- 172.21.15.84:0/3662085473 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677419d160 con 0x7f6774100740 2024-05-22T23:07:17.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.782+0000 7f67795e8640 1 --2- 172.21.15.84:0/3662085473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6774101100 0x7f67741986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:17.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.782+0000 7f677a5ea640 1 --2- 172.21.15.84:0/3662085473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6774106750 0x7f677419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:17.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.782+0000 7f6779de9640 1 --2- 172.21.15.84:0/3662085473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6774100740 0x7f6774198170 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f677000ea60 tx=0x7f677000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:17.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.782+0000 7f676affd640 1 -- 172.21.15.84:0/3662085473 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677000cde0 con 0x7f6774100740 2024-05-22T23:07:17.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.782+0000 7f677c04d640 1 -- 172.21.15.84:0/3662085473 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677419d450 con 0x7f6774100740 2024-05-22T23:07:17.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.782+0000 7f676affd640 1 -- 172.21.15.84:0/3662085473 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67700035f0 con 0x7f6774100740 2024-05-22T23:07:17.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.782+0000 7f676affd640 1 -- 172.21.15.84:0/3662085473 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6770010670 con 0x7f6774100740 2024-05-22T23:07:17.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.782+0000 7f677c04d640 1 -- 172.21.15.84:0/3662085473 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67741a4ce0 con 0x7f6774100740 2024-05-22T23:07:17.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.784+0000 7f676affd640 1 -- 172.21.15.84:0/3662085473 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f67700037a0 con 0x7f6774100740 2024-05-22T23:07:17.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.785+0000 7f677ee3a740 1 Processor -- start 2024-05-22T23:07:17.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.785+0000 7f677ee3a740 1 -- start start 2024-05-22T23:07:17.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.785+0000 7f677ee3a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bef8b50 0x55c57bef8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.785+0000 7f677ee3a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57bef9470 0x55c57bf07b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.785+0000 7f677ee3a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bf08050 0x55c57bf0a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.785+0000 7f677ee3a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c57be5ca40 con 0x55c57bef8b50 2024-05-22T23:07:17.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.785+0000 7f677ee3a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c57bdfb440 con 0x55c57bf08050 2024-05-22T23:07:17.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.785+0000 7f677ee3a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c57be58a00 con 0x55c57bef9470 2024-05-22T23:07:17.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.785+0000 7f676affd640 1 --2- 172.21.15.84:0/3662085473 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f67600a2d00 0x7f67600a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.785+0000 7f677a5ea640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bef8b50 0x55c57bef8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.785+0000 7f677a5ea640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bef8b50 0x55c57bef8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51162/0 (socket says 172.21.15.84:51162) 2024-05-22T23:07:17.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.785+0000 7f677a5ea640 1 -- 172.21.15.84:0/3343501168 learned_addr learned my addr 172.21.15.84:0/3343501168 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:17.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.785+0000 7f6779de9640 1 --2- 172.21.15.84:0/3343501168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57bef9470 0x55c57bf07b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.785+0000 7f67795e8640 1 --2- 172.21.15.84:0/3343501168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bf08050 0x55c57bf0a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.785+0000 7f676affd640 1 -- 172.21.15.84:0/3662085473 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f67700cf6c0 con 0x7f6774100740 2024-05-22T23:07:17.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.785+0000 7f677a5ea640 1 --2- 172.21.15.84:0/3662085473 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f67600a2d00 0x7f67600a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.785+0000 7f67795e8640 1 -- 172.21.15.84:0/3343501168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57bef9470 msgr2=0x55c57bf07b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:17.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.785+0000 7f67795e8640 1 --2- 172.21.15.84:0/3343501168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57bef9470 0x55c57bf07b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.785+0000 7f67795e8640 1 -- 172.21.15.84:0/3343501168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bef8b50 msgr2=0x55c57bef8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:17.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.785+0000 7f67795e8640 1 --2- 172.21.15.84:0/3343501168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bef8b50 0x55c57bef8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.786+0000 7f67795e8640 1 -- 172.21.15.84:0/3343501168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c57be052d0 con 0x55c57bf08050 2024-05-22T23:07:17.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.786+0000 7f67795e8640 1 --2- 172.21.15.84:0/3343501168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bf08050 0x55c57bf0a440 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6764009fd0 tx=0x7f67640118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb19f0eddbcd4280 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:17.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.786+0000 7f677a5ea640 1 --2- 172.21.15.84:0/3662085473 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f67600a2d00 0x7f67600a51c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f676c009a60 tx=0x7f676c02a8e0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:17.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.786+0000 7f675ffff640 1 -- 172.21.15.84:0/3343501168 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764004030 con 0x55c57bf08050 2024-05-22T23:07:17.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.786+0000 7f675ffff640 1 -- 172.21.15.84:0/3343501168 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6764005500 con 0x55c57bf08050 2024-05-22T23:07:17.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.786+0000 7f675ffff640 1 -- 172.21.15.84:0/3343501168 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764004d80 con 0x55c57bf08050 2024-05-22T23:07:17.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.787+0000 7f677ee3a740 1 -- 172.21.15.84:0/3343501168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bf08050 msgr2=0x55c57bf0a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:17.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.787+0000 7f677ee3a740 1 --2- 172.21.15.84:0/3343501168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bf08050 0x55c57bf0a440 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6764009fd0 tx=0x7f67640118d0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.787+0000 7f677ee3a740 1 -- 172.21.15.84:0/3343501168 shutdown_connections 2024-05-22T23:07:17.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.787+0000 7f677ee3a740 1 --2- 172.21.15.84:0/3343501168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bf08050 0x55c57bf0a440 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.787+0000 7f677ee3a740 1 --2- 172.21.15.84:0/3343501168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57bef9470 0x55c57bf07b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.787+0000 7f677ee3a740 1 --2- 172.21.15.84:0/3343501168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bef8b50 0x55c57bef8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.787+0000 7f677ee3a740 1 -- 172.21.15.84:0/3343501168 >> 172.21.15.84:0/3343501168 conn(0x55c57bef4f70 msgr2=0x55c57bef5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:17.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.787+0000 7f677ee3a740 1 -- 172.21.15.84:0/3343501168 shutdown_connections 2024-05-22T23:07:17.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.787+0000 7f677ee3a740 1 -- 172.21.15.84:0/3343501168 wait complete. 2024-05-22T23:07:17.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.787+0000 7f677ee3a740 1 -- start start 2024-05-22T23:07:17.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.787+0000 7f677ee3a740 1 Processor -- start 2024-05-22T23:07:17.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.788+0000 7f677ee3a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57bef8b50 0x55c57bf04ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.788+0000 7f677ee3a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bef9470 0x55c57bf05420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.788+0000 7f677ee3a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bf08050 0x55c57bf05960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.788+0000 7f677ee3a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c57bf0ae10 con 0x55c57bf08050 2024-05-22T23:07:17.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.788+0000 7f677ee3a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c57bde65f0 con 0x55c57bef9470 2024-05-22T23:07:17.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.788+0000 7f677ee3a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c57bf0af90 con 0x55c57bef8b50 2024-05-22T23:07:17.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.788+0000 7f677a5ea640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bf08050 0x55c57bf05960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.788+0000 7f677a5ea640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bf08050 0x55c57bf05960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51174/0 (socket says 172.21.15.84:51174) 2024-05-22T23:07:17.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.788+0000 7f67795e8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bef9470 0x55c57bf05420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.788+0000 7f677a5ea640 1 -- 172.21.15.84:0/3854516842 learned_addr learned my addr 172.21.15.84:0/3854516842 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:17.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.788+0000 7f677ee3a740 10 client.0 _ll_register_callbacks cb 0x55c57bde1cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:07:17.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.788+0000 7f6779de9640 1 --2- 172.21.15.84:0/3854516842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57bef8b50 0x55c57bf04ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.788+0000 7f67795e8640 1 -- 172.21.15.84:0/3854516842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57bef8b50 msgr2=0x55c57bf04ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:17.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.788+0000 7f67795e8640 1 --2- 172.21.15.84:0/3854516842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57bef8b50 0x55c57bf04ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.788+0000 7f67795e8640 1 -- 172.21.15.84:0/3854516842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bf08050 msgr2=0x55c57bf05960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:17.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.788+0000 7f67795e8640 1 --2- 172.21.15.84:0/3854516842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bf08050 0x55c57bf05960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.788+0000 7f67795e8640 1 -- 172.21.15.84:0/3854516842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c57bc387e0 con 0x55c57bef9470 2024-05-22T23:07:17.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.788+0000 7f677a5ea640 1 --2- 172.21.15.84:0/3854516842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bf08050 0x55c57bf05960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:17.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.789+0000 7f67795e8640 1 --2- 172.21.15.84:0/3854516842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bef9470 0x55c57bf05420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6764011dd0 tx=0x7f67640038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8223b85565a9189a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:17.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.789+0000 7f675d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:07:17.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.789+0000 7f675d7fa640 1 -- 172.21.15.84:0/3854516842 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764004460 con 0x55c57bef9470 2024-05-22T23:07:17.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.789+0000 7f677ee3a740 1 -- 172.21.15.84:0/3854516842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6764005500 con 0x55c57bef9470 2024-05-22T23:07:17.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.789+0000 7f675d7fa640 1 -- 172.21.15.84:0/3854516842 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f6764003ca0 con 0x55c57bef9470 2024-05-22T23:07:17.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.789+0000 7f675d7fa640 1 -- 172.21.15.84:0/3854516842 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f6764004a80 con 0x55c57bef9470 2024-05-22T23:07:17.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.789+0000 7f677ee3a740 10 client.25874 fetch_fsmap learned FSMap version 18 2024-05-22T23:07:17.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.789+0000 7f677ee3a740 1 -- 172.21.15.84:0/3854516842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55c57bc98f70 con 0x55c57bef9470 2024-05-22T23:07:17.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.790+0000 7f675d7fa640 1 -- 172.21.15.84:0/3854516842 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6764005500 con 0x55c57bef9470 2024-05-22T23:07:17.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.790+0000 7f675d7fa640 1 -- 172.21.15.84:0/3854516842 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f676400f5a0 con 0x55c57bef9470 2024-05-22T23:07:17.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.790+0000 7f677ee3a740 10 client.25874 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:07:17.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.790+0000 7f677ee3a740 10 client.25874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:07:17.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.790+0000 7f677ee3a740 20 client.25874 populate_metadata read hostname 'smithi084' 2024-05-22T23:07:17.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.790+0000 7f677ee3a740 1 --2- 172.21.15.84:0/3854516842 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c57bf18230 0x55c57bf1a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.790+0000 7f677ee3a740 4 client.25874 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:07:17.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.790+0000 7f677ee3a740 1 -- 172.21.15.84:0/3854516842 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c57be70460 con 0x55c57bf18230 2024-05-22T23:07:17.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.790+0000 7f6779de9640 1 --2- 172.21.15.84:0/3854516842 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c57bf18230 0x55c57bf1a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.790+0000 7f6779de9640 1 --2- 172.21.15.84:0/3854516842 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c57bf18230 0x55c57bf1a620 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f585b85665ff5a7c server_cookie=5cadf5e34f1de3ed in_seq=0 out_seq=0 2024-05-22T23:07:17.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.790+0000 7f675d7fa640 10 client.25874 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:17.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.791+0000 7f675d7fa640 1 -- 172.21.15.84:0/3854516842 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c57be70460 con 0x55c57bf18230 2024-05-22T23:07:17.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.791+0000 7f675d7fa640 10 client.25874 handle_command_reply: tid=0 2024-05-22T23:07:17.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.791+0000 7f677ee3a740 1 client.25874 shutdown 2024-05-22T23:07:17.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.791+0000 7f677ee3a740 1 -- 172.21.15.84:0/3854516842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bef9470 msgr2=0x55c57bf05420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:17.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.791+0000 7f677ee3a740 1 --2- 172.21.15.84:0/3854516842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bef9470 0x55c57bf05420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6764011dd0 tx=0x7f67640038a0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.792+0000 7f677ee3a740 1 -- 172.21.15.84:0/3854516842 shutdown_connections 2024-05-22T23:07:17.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.792+0000 7f677ee3a740 1 --2- 172.21.15.84:0/3854516842 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c57bf18230 0x55c57bf1a620 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.792+0000 7f677ee3a740 1 --2- 172.21.15.84:0/3854516842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bf08050 0x55c57bf05960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.792+0000 7f677ee3a740 1 --2- 172.21.15.84:0/3854516842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bef9470 0x55c57bf05420 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.792+0000 7f677ee3a740 1 --2- 172.21.15.84:0/3854516842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57bef8b50 0x55c57bf04ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.792+0000 7f675d7fa640 0 client.25874 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:17.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.792+0000 7f677ee3a740 1 -- 172.21.15.84:0/3854516842 >> 172.21.15.84:0/3854516842 conn(0x55c57bef4f70 msgr2=0x55c57befade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:17.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.792+0000 7f677ee3a740 1 -- 172.21.15.84:0/3854516842 shutdown_connections 2024-05-22T23:07:17.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.792+0000 7f677ee3a740 1 -- 172.21.15.84:0/3854516842 wait complete. 2024-05-22T23:07:17.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.792+0000 7f677ee3a740 20 client.25874 trim_cache size 0 max 16384 2024-05-22T23:07:17.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.812+0000 7f677ee3a740 1 Processor -- start 2024-05-22T23:07:17.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.812+0000 7f677ee3a740 1 -- start start 2024-05-22T23:07:17.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.812+0000 7f677ee3a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bef8b50 0x55c57bf13d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.812+0000 7f677ee3a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57bef9470 0x55c57bf055b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.812+0000 7f677ee3a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bf08050 0x55c57bf1fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.812+0000 7f677ee3a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f67741a94d0 con 0x55c57bf08050 2024-05-22T23:07:17.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.812+0000 7f677ee3a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f67741abd50 con 0x55c57bef8b50 2024-05-22T23:07:17.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.812+0000 7f677ee3a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f67741a9b40 con 0x55c57bef9470 2024-05-22T23:07:17.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.813+0000 7f677a5ea640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bf08050 0x55c57bf1fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.813+0000 7f6779de9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bef8b50 0x55c57bf13d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.813+0000 7f677a5ea640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bf08050 0x55c57bf1fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51188/0 (socket says 172.21.15.84:51188) 2024-05-22T23:07:17.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.813+0000 7f6779de9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bef8b50 0x55c57bf13d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:36756/0 (socket says 172.21.15.84:36756) 2024-05-22T23:07:17.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.813+0000 7f677a5ea640 1 -- 172.21.15.84:0/3853404597 learned_addr learned my addr 172.21.15.84:0/3853404597 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:17.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.813+0000 7f67795e8640 1 --2- 172.21.15.84:0/3853404597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57bef9470 0x55c57bf055b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.813+0000 7f677a5ea640 1 -- 172.21.15.84:0/3853404597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57bef9470 msgr2=0x55c57bf055b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:17.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.813+0000 7f677a5ea640 1 --2- 172.21.15.84:0/3853404597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57bef9470 0x55c57bf055b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.813+0000 7f677a5ea640 1 -- 172.21.15.84:0/3853404597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bef8b50 msgr2=0x55c57bf13d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:17.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.813+0000 7f677a5ea640 1 --2- 172.21.15.84:0/3853404597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bef8b50 0x55c57bf13d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.813+0000 7f677a5ea640 1 -- 172.21.15.84:0/3853404597 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c57bb30170 con 0x55c57bf08050 2024-05-22T23:07:17.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.813+0000 7f6779de9640 1 --2- 172.21.15.84:0/3853404597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bef8b50 0x55c57bf13d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:17.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.813+0000 7f67795e8640 1 --2- 172.21.15.84:0/3853404597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57bef9470 0x55c57bf055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:07:17.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.813+0000 7f677a5ea640 1 --2- 172.21.15.84:0/3853404597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bf08050 0x55c57bf1fe80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f676c003aa0 tx=0x7f676c005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac860b1e768bf3b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:17.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.813+0000 7f675f7fe640 1 -- 172.21.15.84:0/3853404597 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c0047c0 con 0x55c57bf08050 2024-05-22T23:07:17.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.813+0000 7f675f7fe640 1 -- 172.21.15.84:0/3853404597 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f676c036050 con 0x55c57bf08050 2024-05-22T23:07:17.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.813+0000 7f675f7fe640 1 -- 172.21.15.84:0/3853404597 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c04b9d0 con 0x55c57bf08050 2024-05-22T23:07:17.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.814+0000 7f677ee3a740 1 -- 172.21.15.84:0/3853404597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bf08050 msgr2=0x55c57bf1fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:17.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.814+0000 7f677ee3a740 1 --2- 172.21.15.84:0/3853404597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bf08050 0x55c57bf1fe80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f676c003aa0 tx=0x7f676c005f70 comp rx=0 tx=0).stop 2024-05-22T23:07:17.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.814+0000 7f677ee3a740 1 -- 172.21.15.84:0/3853404597 shutdown_connections 2024-05-22T23:07:17.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.814+0000 7f677ee3a740 1 --2- 172.21.15.84:0/3853404597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bf08050 0x55c57bf1fe80 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.814+0000 7f677ee3a740 1 --2- 172.21.15.84:0/3853404597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57bef9470 0x55c57bf055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.814+0000 7f677ee3a740 1 --2- 172.21.15.84:0/3853404597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bef8b50 0x55c57bf13d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.814+0000 7f677ee3a740 1 -- 172.21.15.84:0/3853404597 >> 172.21.15.84:0/3853404597 conn(0x55c57bef4f70 msgr2=0x55c57bf04ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:17.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.814+0000 7f677ee3a740 1 -- 172.21.15.84:0/3853404597 shutdown_connections 2024-05-22T23:07:17.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.814+0000 7f677ee3a740 1 -- 172.21.15.84:0/3853404597 wait complete. 2024-05-22T23:07:17.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.814+0000 7f677ee3a740 1 -- start start 2024-05-22T23:07:17.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.814+0000 7f677ee3a740 1 Processor -- start 2024-05-22T23:07:17.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.815+0000 7f677ee3a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57bef8b50 0x55c57bf106d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.815+0000 7f677ee3a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bef9470 0x55c57bf0ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.815+0000 7f677ee3a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bf08050 0x55c57bf0c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.815+0000 7f677ee3a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f67741aa210 con 0x55c57bef9470 2024-05-22T23:07:17.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.815+0000 7f677ee3a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f67741ab680 con 0x55c57bf08050 2024-05-22T23:07:17.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.815+0000 7f677ee3a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f67741aa8e0 con 0x55c57bef8b50 2024-05-22T23:07:17.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.815+0000 7f677ee3a740 10 client.0 _ll_register_callbacks cb 0x7f6764005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:07:17.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.815+0000 7f67795e8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bef9470 0x55c57bf0ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.815+0000 7f6779de9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57bef8b50 0x55c57bf106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.815+0000 7f67795e8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bef9470 0x55c57bf0ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51194/0 (socket says 172.21.15.84:51194) 2024-05-22T23:07:17.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.815+0000 7f67795e8640 1 -- 172.21.15.84:0/2686002676 learned_addr learned my addr 172.21.15.84:0/2686002676 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:17.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.815+0000 7f677a5ea640 1 --2- 172.21.15.84:0/2686002676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bf08050 0x55c57bf0c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.815+0000 7f677a5ea640 1 -- 172.21.15.84:0/2686002676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57bef8b50 msgr2=0x55c57bf106d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:17.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.815+0000 7f677a5ea640 1 --2- 172.21.15.84:0/2686002676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57bef8b50 0x55c57bf106d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.815+0000 7f677a5ea640 1 -- 172.21.15.84:0/2686002676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bef9470 msgr2=0x55c57bf0ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:17.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.815+0000 7f677a5ea640 1 --2- 172.21.15.84:0/2686002676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bef9470 0x55c57bf0ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.815+0000 7f677a5ea640 1 -- 172.21.15.84:0/2686002676 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c57be25930 con 0x55c57bf08050 2024-05-22T23:07:17.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.816+0000 7f677a5ea640 1 --2- 172.21.15.84:0/2686002676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bf08050 0x55c57bf0c030 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f676c02ae00 tx=0x7f676c04b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a881cf0f227d415f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:17.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.816+0000 7f675d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:07:17.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.816+0000 7f675d7fa640 1 -- 172.21.15.84:0/2686002676 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c05c020 con 0x55c57bf08050 2024-05-22T23:07:17.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.816+0000 7f675d7fa640 1 -- 172.21.15.84:0/2686002676 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f676c0305a0 con 0x55c57bf08050 2024-05-22T23:07:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.816+0000 7f677ee3a740 1 -- 172.21.15.84:0/2686002676 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f676c036050 con 0x55c57bf08050 2024-05-22T23:07:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.816+0000 7f675d7fa640 1 -- 172.21.15.84:0/2686002676 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f676c067050 con 0x55c57bf08050 2024-05-22T23:07:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.816+0000 7f677ee3a740 10 client.25880 fetch_fsmap learned FSMap version 18 2024-05-22T23:07:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.816+0000 7f677ee3a740 1 -- 172.21.15.84:0/2686002676 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55c57befb6a0 con 0x55c57bf08050 2024-05-22T23:07:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.816+0000 7f675d7fa640 1 -- 172.21.15.84:0/2686002676 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f676c036050 con 0x55c57bf08050 2024-05-22T23:07:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.816+0000 7f675d7fa640 1 -- 172.21.15.84:0/2686002676 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f676c030070 con 0x55c57bf08050 2024-05-22T23:07:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.816+0000 7f677ee3a740 10 client.25880 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:07:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.816+0000 7f677ee3a740 10 client.25880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:07:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.816+0000 7f677ee3a740 20 client.25880 populate_metadata read hostname 'smithi084' 2024-05-22T23:07:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.816+0000 7f677ee3a740 1 --2- 172.21.15.84:0/2686002676 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c57befbe40 0x55c57befe260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.816+0000 7f677ee3a740 4 client.25880 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-22T23:07:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.816+0000 7f677ee3a740 1 -- 172.21.15.84:0/2686002676 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c57be6a170 con 0x55c57befbe40 2024-05-22T23:07:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.817+0000 7f6779de9640 1 --2- 172.21.15.84:0/2686002676 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c57befbe40 0x55c57befe260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.817+0000 7f675d7fa640 10 client.25880 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.817+0000 7f6779de9640 1 --2- 172.21.15.84:0/2686002676 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c57befbe40 0x55c57befe260 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cf47275f53bd776 server_cookie=8dc197ce0aa6a87f in_seq=0 out_seq=0 2024-05-22T23:07:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.819+0000 7f675d7fa640 1 -- 172.21.15.84:0/2686002676 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c57be6a170 con 0x55c57befbe40 2024-05-22T23:07:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.819+0000 7f675d7fa640 10 client.25880 handle_command_reply: tid=0 2024-05-22T23:07:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.819+0000 7f677ee3a740 1 client.25880 shutdown 2024-05-22T23:07:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.819+0000 7f677ee3a740 1 -- 172.21.15.84:0/2686002676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bf08050 msgr2=0x55c57bf0c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.819+0000 7f677ee3a740 1 --2- 172.21.15.84:0/2686002676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bf08050 0x55c57bf0c030 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f676c02ae00 tx=0x7f676c04b900 comp rx=0 tx=0).stop 2024-05-22T23:07:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.819+0000 7f677ee3a740 1 -- 172.21.15.84:0/2686002676 shutdown_connections 2024-05-22T23:07:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.819+0000 7f677ee3a740 1 --2- 172.21.15.84:0/2686002676 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55c57befbe40 0x55c57befe260 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.819+0000 7f677ee3a740 1 --2- 172.21.15.84:0/2686002676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c57bf08050 0x55c57bf0c030 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.819+0000 7f677ee3a740 1 --2- 172.21.15.84:0/2686002676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c57bef9470 0x55c57bf0ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.819+0000 7f677ee3a740 1 --2- 172.21.15.84:0/2686002676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57bef8b50 0x55c57bf106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.819+0000 7f675d7fa640 0 client.25880 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.819+0000 7f677ee3a740 1 -- 172.21.15.84:0/2686002676 >> 172.21.15.84:0/2686002676 conn(0x55c57bef4f70 msgr2=0x55c57befade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.819+0000 7f677ee3a740 1 -- 172.21.15.84:0/2686002676 shutdown_connections 2024-05-22T23:07:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.819+0000 7f677ee3a740 1 -- 172.21.15.84:0/2686002676 wait complete. 2024-05-22T23:07:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.819+0000 7f677ee3a740 20 client.25880 trim_cache size 0 max 16384 2024-05-22T23:07:17.830 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:07:17.830 INFO:teuthology.orchestra.run.smithi084.stdout: "return_code": 0, 2024-05-22T23:07:17.830 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_tag": "0755f526-bf73-48b4-a6b6-78f4916fd1ae", 2024-05-22T23:07:17.830 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": "asynchronous" 2024-05-22T23:07:17.830 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:07:17.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.820+0000 7f677c04d640 1 -- 172.21.15.84:0/3662085473 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f67600a2d00 msgr2=0x7f67600a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:17.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.820+0000 7f677c04d640 1 --2- 172.21.15.84:0/3662085473 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f67600a2d00 0x7f67600a51c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f676c009a60 tx=0x7f676c02a8e0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.820+0000 7f677c04d640 1 -- 172.21.15.84:0/3662085473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6774100740 msgr2=0x7f6774198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:17.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.820+0000 7f677c04d640 1 --2- 172.21.15.84:0/3662085473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6774100740 0x7f6774198170 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f677000ea60 tx=0x7f677000ef20 comp rx=0 tx=0).stop 2024-05-22T23:07:17.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.820+0000 7f677c04d640 1 -- 172.21.15.84:0/3662085473 shutdown_connections 2024-05-22T23:07:17.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.820+0000 7f677c04d640 1 --2- 172.21.15.84:0/3662085473 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f67600a2d00 0x7f67600a51c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.820+0000 7f677c04d640 1 --2- 172.21.15.84:0/3662085473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6774106750 0x7f677419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.820+0000 7f677c04d640 1 --2- 172.21.15.84:0/3662085473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6774101100 0x7f67741986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.820+0000 7f677c04d640 1 --2- 172.21.15.84:0/3662085473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6774100740 0x7f6774198170 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.820+0000 7f677c04d640 1 -- 172.21.15.84:0/3662085473 >> 172.21.15.84:0/3662085473 conn(0x7f6774077870 msgr2=0x7f6774068d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:17.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.820+0000 7f677c04d640 1 -- 172.21.15.84:0/3662085473 shutdown_connections 2024-05-22T23:07:17.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.820+0000 7f677c04d640 1 -- 172.21.15.84:0/3662085473 wait complete. 2024-05-22T23:07:17.836 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:17 smithi084 ceph-mon[32186]: pgmap v166: 97 pgs: 97 active+clean; 367 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.1 MiB/s wr, 380 op/s 2024-05-22T23:07:17.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:07:17.838 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:07:17.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:17 smithi204 ceph-mon[36211]: pgmap v166: 97 pgs: 97 active+clean; 367 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.1 MiB/s wr, 380 op/s 2024-05-22T23:07:17.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.946+0000 7f7a03dee640 1 -- 172.21.15.84:0/1987915816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79fc100740 msgr2=0x7f79fc100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:17.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.946+0000 7f7a03dee640 1 --2- 172.21.15.84:0/1987915816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79fc100740 0x7f79fc100bc0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009930 tx=0x7f79ec02a420 comp rx=0 tx=0).stop 2024-05-22T23:07:17.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.946+0000 7f7a03dee640 1 -- 172.21.15.84:0/1987915816 shutdown_connections 2024-05-22T23:07:17.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.946+0000 7f7a03dee640 1 --2- 172.21.15.84:0/1987915816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79fc101100 0x7f79fc10d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.946+0000 7f7a03dee640 1 --2- 172.21.15.84:0/1987915816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79fc100740 0x7f79fc100bc0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.946+0000 7f7a03dee640 1 --2- 172.21.15.84:0/1987915816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79fc106750 0x7f79fc106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.946+0000 7f7a03dee640 1 -- 172.21.15.84:0/1987915816 >> 172.21.15.84:0/1987915816 conn(0x7f79fc077870 msgr2=0x7f79fc077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:17.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.946+0000 7f7a03dee640 1 -- 172.21.15.84:0/1987915816 shutdown_connections 2024-05-22T23:07:17.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.946+0000 7f7a03dee640 1 -- 172.21.15.84:0/1987915816 wait complete. 2024-05-22T23:07:17.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.947+0000 7f7a03dee640 1 Processor -- start 2024-05-22T23:07:17.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.947+0000 7f7a03dee640 1 -- start start 2024-05-22T23:07:17.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.947+0000 7f7a03dee640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79fc100740 0x7f79fc198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.947+0000 7f7a03dee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79fc101100 0x7f79fc1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.947+0000 7f7a03dee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79fc106750 0x7f79fc19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.947+0000 7f7a03dee640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc111c60 con 0x7f79fc101100 2024-05-22T23:07:17.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.947+0000 7f7a03dee640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc111ae0 con 0x7f79fc106750 2024-05-22T23:07:17.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.947+0000 7f7a03dee640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc111de0 con 0x7f79fc100740 2024-05-22T23:07:17.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.947+0000 7f7a01389640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79fc101100 0x7f79fc1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.947+0000 7f7a01389640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79fc101100 0x7f79fc1986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51212/0 (socket says 172.21.15.84:51212) 2024-05-22T23:07:17.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.947+0000 7f7a01389640 1 -- 172.21.15.84:0/2960234102 learned_addr learned my addr 172.21.15.84:0/2960234102 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:17.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.948+0000 7f7a0238b640 1 --2- 172.21.15.84:0/2960234102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79fc106750 0x7f79fc19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.948+0000 7f7a01b8a640 1 --2- 172.21.15.84:0/2960234102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79fc100740 0x7f79fc198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.948+0000 7f7a0238b640 1 -- 172.21.15.84:0/2960234102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79fc100740 msgr2=0x7f79fc198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:17.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.948+0000 7f7a0238b640 1 --2- 172.21.15.84:0/2960234102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79fc100740 0x7f79fc198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.948+0000 7f7a0238b640 1 -- 172.21.15.84:0/2960234102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79fc101100 msgr2=0x7f79fc1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:17.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.948+0000 7f7a0238b640 1 --2- 172.21.15.84:0/2960234102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79fc101100 0x7f79fc1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.948+0000 7f7a0238b640 1 -- 172.21.15.84:0/2960234102 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79fc19d120 con 0x7f79fc106750 2024-05-22T23:07:17.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.948+0000 7f7a01389640 1 --2- 172.21.15.84:0/2960234102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79fc101100 0x7f79fc1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:17.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.948+0000 7f7a0238b640 1 --2- 172.21.15.84:0/2960234102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79fc106750 0x7f79fc19c9f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f79f4009fd0 tx=0x7f79f400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:17.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.948+0000 7f79f2ffd640 1 -- 172.21.15.84:0/2960234102 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f400ce20 con 0x7f79fc106750 2024-05-22T23:07:17.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.948+0000 7f7a03dee640 1 -- 172.21.15.84:0/2960234102 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79fc19d410 con 0x7f79fc106750 2024-05-22T23:07:17.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.948+0000 7f79f2ffd640 1 -- 172.21.15.84:0/2960234102 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79f4005020 con 0x7f79fc106750 2024-05-22T23:07:17.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.948+0000 7f79f2ffd640 1 -- 172.21.15.84:0/2960234102 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f4010720 con 0x7f79fc106750 2024-05-22T23:07:17.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.948+0000 7f7a03dee640 1 -- 172.21.15.84:0/2960234102 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79fc10dfd0 con 0x7f79fc106750 2024-05-22T23:07:17.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.951+0000 7f79f2ffd640 1 -- 172.21.15.84:0/2960234102 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f79f4005540 con 0x7f79fc106750 2024-05-22T23:07:17.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.951+0000 7f7a06bdc740 1 Processor -- start 2024-05-22T23:07:17.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.951+0000 7f7a06bdc740 1 -- start start 2024-05-22T23:07:17.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.952+0000 7f7a06bdc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2440ff4a0 0x55d2440fd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.952+0000 7f7a06bdc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2440fddb0 0x55d2440fe230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.952+0000 7f7a06bdc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d244108b30 0x55d24410af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.952+0000 7f7a06bdc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d244061db0 con 0x55d244108b30 2024-05-22T23:07:17.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.952+0000 7f7a06bdc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d244000360 con 0x55d2440ff4a0 2024-05-22T23:07:17.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.952+0000 7f7a06bdc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d24405dd70 con 0x55d2440fddb0 2024-05-22T23:07:17.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.952+0000 7f7a01389640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2440ff4a0 0x55d2440fd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.952+0000 7f7a01389640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2440ff4a0 0x55d2440fd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:36798/0 (socket says 172.21.15.84:36798) 2024-05-22T23:07:17.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.952+0000 7f7a01389640 1 -- 172.21.15.84:0/95033032 learned_addr learned my addr 172.21.15.84:0/95033032 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:17.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.952+0000 7f7a01b8a640 1 --2- 172.21.15.84:0/95033032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2440fddb0 0x55d2440fe230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.952+0000 7f79f2ffd640 1 --2- 172.21.15.84:0/2960234102 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f79e80a2d00 0x7f79e80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.952+0000 7f7a01389640 1 --2- 172.21.15.84:0/95033032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d244108b30 0x55d24410af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.952+0000 7f7a0238b640 1 --2- 172.21.15.84:0/2960234102 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f79e80a2d00 0x7f79e80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.952+0000 7f7a01389640 1 -- 172.21.15.84:0/95033032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2440fddb0 msgr2=0x55d2440fe230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:17.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.952+0000 7f7a01389640 1 --2- 172.21.15.84:0/95033032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2440fddb0 0x55d2440fe230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.952+0000 7f7a01389640 1 -- 172.21.15.84:0/95033032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d244108b30 msgr2=0x55d24410af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:17.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.952+0000 7f7a01389640 1 --2- 172.21.15.84:0/95033032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d244108b30 0x55d24410af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.952+0000 7f7a01389640 1 -- 172.21.15.84:0/95033032 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d244023a60 con 0x55d2440ff4a0 2024-05-22T23:07:17.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.952+0000 7f7a01b8a640 1 --2- 172.21.15.84:0/95033032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2440fddb0 0x55d2440fe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:17.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.952+0000 7f79f2ffd640 1 -- 172.21.15.84:0/2960234102 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f79f40cfab0 con 0x7f79fc106750 2024-05-22T23:07:17.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.952+0000 7f7a01389640 1 --2- 172.21.15.84:0/95033032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2440ff4a0 0x55d2440fd7e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009230 tx=0x7f79ec009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed953fca8cccd5df server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:17.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.953+0000 7f79e7fff640 1 -- 172.21.15.84:0/95033032 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec035070 con 0x55d2440ff4a0 2024-05-22T23:07:17.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.953+0000 7f79e7fff640 1 -- 172.21.15.84:0/95033032 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79ec0319e0 con 0x55d2440ff4a0 2024-05-22T23:07:17.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.953+0000 7f79e7fff640 1 -- 172.21.15.84:0/95033032 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0047a0 con 0x55d2440ff4a0 2024-05-22T23:07:17.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.953+0000 7f7a06bdc740 1 -- 172.21.15.84:0/95033032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2440ff4a0 msgr2=0x55d2440fd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:17.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.953+0000 7f7a0238b640 1 --2- 172.21.15.84:0/2960234102 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f79e80a2d00 0x7f79e80a51c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f79f409bf00 tx=0x7f79f40bc040 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:17.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.953+0000 7f7a06bdc740 1 --2- 172.21.15.84:0/95033032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2440ff4a0 0x55d2440fd7e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009230 tx=0x7f79ec009260 comp rx=0 tx=0).stop 2024-05-22T23:07:17.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.953+0000 7f7a06bdc740 1 -- 172.21.15.84:0/95033032 shutdown_connections 2024-05-22T23:07:17.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.954+0000 7f7a06bdc740 1 --2- 172.21.15.84:0/95033032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d244108b30 0x55d24410af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.954+0000 7f7a06bdc740 1 --2- 172.21.15.84:0/95033032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2440fddb0 0x55d2440fe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.954+0000 7f7a06bdc740 1 --2- 172.21.15.84:0/95033032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2440ff4a0 0x55d2440fd7e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.954+0000 7f7a06bdc740 1 -- 172.21.15.84:0/95033032 >> 172.21.15.84:0/95033032 conn(0x55d2440c68e0 msgr2=0x55d2440fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:17.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.954+0000 7f7a06bdc740 1 -- 172.21.15.84:0/95033032 shutdown_connections 2024-05-22T23:07:17.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.954+0000 7f7a06bdc740 1 -- 172.21.15.84:0/95033032 wait complete. 2024-05-22T23:07:17.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.954+0000 7f7a06bdc740 1 -- start start 2024-05-22T23:07:17.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.954+0000 7f7a06bdc740 1 Processor -- start 2024-05-22T23:07:17.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.954+0000 7f7a06bdc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2440fddb0 0x55d244105a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.954+0000 7f7a06bdc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2440ff4a0 0x55d244105f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.954+0000 7f7a06bdc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d244108b30 0x55d244106490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.954+0000 7f7a06bdc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d24410b960 con 0x55d244108b30 2024-05-22T23:07:17.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.955+0000 7f7a06bdc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d24401c0f0 con 0x55d2440fddb0 2024-05-22T23:07:17.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.955+0000 7f7a06bdc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d24410bae0 con 0x55d2440ff4a0 2024-05-22T23:07:17.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.955+0000 7f7a06bdc740 10 client.0 _ll_register_callbacks cb 0x55d244020f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:07:17.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.955+0000 7f7a01389640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d244108b30 0x55d244106490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.955+0000 7f7a01389640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d244108b30 0x55d244106490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51224/0 (socket says 172.21.15.84:51224) 2024-05-22T23:07:17.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.955+0000 7f7a01389640 1 -- 172.21.15.84:0/2164503708 learned_addr learned my addr 172.21.15.84:0/2164503708 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:17.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.955+0000 7f7a01389640 1 --2- 172.21.15.84:0/2164503708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2440fddb0 0x55d244105a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.955+0000 7f7a01b8a640 1 --2- 172.21.15.84:0/2164503708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2440ff4a0 0x55d244105f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.955+0000 7f7a01389640 1 -- 172.21.15.84:0/2164503708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2440ff4a0 msgr2=0x55d244105f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:17.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.955+0000 7f7a01389640 1 --2- 172.21.15.84:0/2164503708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2440ff4a0 0x55d244105f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.955+0000 7f7a01389640 1 -- 172.21.15.84:0/2164503708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2440fddb0 msgr2=0x55d244105a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:17.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.955+0000 7f7a01389640 1 --2- 172.21.15.84:0/2164503708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2440fddb0 0x55d244105a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.955+0000 7f7a01389640 1 -- 172.21.15.84:0/2164503708 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d243e3f7d0 con 0x55d244108b30 2024-05-22T23:07:17.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.955+0000 7f7a01389640 1 --2- 172.21.15.84:0/2164503708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d244108b30 0x55d244106490 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009900 tx=0x7f79ec005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689a8b5e51ff16b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:17.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.955+0000 7f79e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:07:17.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.955+0000 7f79e57fa640 1 -- 172.21.15.84:0/2164503708 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec035030 con 0x55d244108b30 2024-05-22T23:07:17.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.955+0000 7f79e57fa640 1 -- 172.21.15.84:0/2164503708 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f79ec004050 con 0x55d244108b30 2024-05-22T23:07:17.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.955+0000 7f7a06bdc740 1 -- 172.21.15.84:0/2164503708 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79ec0319e0 con 0x55d244108b30 2024-05-22T23:07:17.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.955+0000 7f79e57fa640 1 -- 172.21.15.84:0/2164503708 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f79ec05d050 con 0x55d244108b30 2024-05-22T23:07:17.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.955+0000 7f7a06bdc740 10 client.16320 fetch_fsmap learned FSMap version 18 2024-05-22T23:07:17.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.956+0000 7f7a06bdc740 1 -- 172.21.15.84:0/2164503708 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55d243d753f0 con 0x55d244108b30 2024-05-22T23:07:17.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.956+0000 7f79e57fa640 1 -- 172.21.15.84:0/2164503708 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79ec0319e0 con 0x55d244108b30 2024-05-22T23:07:17.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.956+0000 7f79e57fa640 1 -- 172.21.15.84:0/2164503708 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f79ec004770 con 0x55d244108b30 2024-05-22T23:07:17.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.956+0000 7f7a06bdc740 10 client.16320 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:07:17.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.956+0000 7f7a06bdc740 10 client.16320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:07:17.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.956+0000 7f7a06bdc740 20 client.16320 populate_metadata read hostname 'smithi084' 2024-05-22T23:07:17.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.956+0000 7f7a06bdc740 1 --2- 172.21.15.84:0/2164503708 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d244118d80 0x55d24411b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.956+0000 7f7a06bdc740 4 client.16320 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:07:17.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.956+0000 7f7a06bdc740 1 -- 172.21.15.84:0/2164503708 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d24406f4e0 con 0x55d244118d80 2024-05-22T23:07:17.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.956+0000 7f7a01b8a640 1 --2- 172.21.15.84:0/2164503708 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d244118d80 0x55d24411b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.956+0000 7f79e57fa640 10 client.16320 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:17.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.957+0000 7f7a01b8a640 1 --2- 172.21.15.84:0/2164503708 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d244118d80 0x55d24411b170 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e7b0f8ae88858c0 server_cookie=bcf98ea15422ae37 in_seq=0 out_seq=0 2024-05-22T23:07:17.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.958+0000 7f79e57fa640 1 -- 172.21.15.84:0/2164503708 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d24406f4e0 con 0x55d244118d80 2024-05-22T23:07:17.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.958+0000 7f79e57fa640 10 client.16320 handle_command_reply: tid=0 2024-05-22T23:07:17.969 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.958+0000 7f7a06bdc740 1 client.16320 shutdown 2024-05-22T23:07:17.969 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.958+0000 7f7a06bdc740 1 -- 172.21.15.84:0/2164503708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d244108b30 msgr2=0x55d244106490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:17.969 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.958+0000 7f7a06bdc740 1 --2- 172.21.15.84:0/2164503708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d244108b30 0x55d244106490 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009900 tx=0x7f79ec005a40 comp rx=0 tx=0).stop 2024-05-22T23:07:17.969 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.958+0000 7f7a06bdc740 1 -- 172.21.15.84:0/2164503708 shutdown_connections 2024-05-22T23:07:17.969 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.958+0000 7f7a06bdc740 1 --2- 172.21.15.84:0/2164503708 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d244118d80 0x55d24411b170 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.969 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.958+0000 7f7a06bdc740 1 --2- 172.21.15.84:0/2164503708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d244108b30 0x55d244106490 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.969 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.958+0000 7f7a06bdc740 1 --2- 172.21.15.84:0/2164503708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2440ff4a0 0x55d244105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.969 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.958+0000 7f79e57fa640 0 client.16320 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:17.969 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.958+0000 7f7a06bdc740 1 --2- 172.21.15.84:0/2164503708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2440fddb0 0x55d244105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.969 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.958+0000 7f7a06bdc740 1 -- 172.21.15.84:0/2164503708 >> 172.21.15.84:0/2164503708 conn(0x55d2440c68e0 msgr2=0x55d244100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:17.969 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.958+0000 7f7a06bdc740 1 -- 172.21.15.84:0/2164503708 shutdown_connections 2024-05-22T23:07:17.969 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.958+0000 7f7a06bdc740 1 -- 172.21.15.84:0/2164503708 wait complete. 2024-05-22T23:07:17.969 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.958+0000 7f7a06bdc740 20 client.16320 trim_cache size 0 max 16384 2024-05-22T23:07:17.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.979+0000 7f7a06bdc740 1 Processor -- start 2024-05-22T23:07:17.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.979+0000 7f7a06bdc740 1 -- start start 2024-05-22T23:07:17.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.979+0000 7f7a06bdc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2440fddb0 0x55d244114480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.979+0000 7f7a06bdc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2440ff4a0 0x55d2440f9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.979+0000 7f7a06bdc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d244108b30 0x55d2440fa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.979+0000 7f7a06bdc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc1a9370 con 0x55d2440ff4a0 2024-05-22T23:07:17.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.979+0000 7f7a06bdc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc1abbf0 con 0x55d2440fddb0 2024-05-22T23:07:17.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.979+0000 7f7a06bdc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc1a99e0 con 0x55d244108b30 2024-05-22T23:07:17.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.979+0000 7f7a01b8a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2440ff4a0 0x55d2440f9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.979+0000 7f7a01389640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2440fddb0 0x55d244114480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.979+0000 7f7a01389640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2440fddb0 0x55d244114480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:36818/0 (socket says 172.21.15.84:36818) 2024-05-22T23:07:17.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.979+0000 7f7a01389640 1 -- 172.21.15.84:0/305054761 learned_addr learned my addr 172.21.15.84:0/305054761 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:17.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.979+0000 7f7a01b8a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2440ff4a0 0x55d2440f9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51240/0 (socket says 172.21.15.84:51240) 2024-05-22T23:07:17.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.979+0000 7f7a01389640 1 --2- 172.21.15.84:0/305054761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d244108b30 0x55d2440fa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.979+0000 7f7a01389640 1 -- 172.21.15.84:0/305054761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d244108b30 msgr2=0x55d2440fa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:17.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.979+0000 7f7a01389640 1 --2- 172.21.15.84:0/305054761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d244108b30 0x55d2440fa130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.979+0000 7f7a01389640 1 -- 172.21.15.84:0/305054761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2440ff4a0 msgr2=0x55d2440f9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:17.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.979+0000 7f7a01389640 1 --2- 172.21.15.84:0/305054761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2440ff4a0 0x55d2440f9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.980+0000 7f7a01389640 1 -- 172.21.15.84:0/305054761 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d243d350f0 con 0x55d2440fddb0 2024-05-22T23:07:17.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.980+0000 7f7a01b8a640 1 --2- 172.21.15.84:0/305054761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2440ff4a0 0x55d2440f9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:17.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.980+0000 7f7a01389640 1 --2- 172.21.15.84:0/305054761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2440fddb0 0x55d244114480 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55d2440fb630 tx=0x7f79ec03f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eb1436754bd3c9e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:17.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.980+0000 7f79e77fe640 1 -- 172.21.15.84:0/305054761 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec035070 con 0x55d2440fddb0 2024-05-22T23:07:17.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.980+0000 7f79e77fe640 1 -- 172.21.15.84:0/305054761 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79ec03dc40 con 0x55d2440fddb0 2024-05-22T23:07:17.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.980+0000 7f79e77fe640 1 -- 172.21.15.84:0/305054761 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec031e40 con 0x55d2440fddb0 2024-05-22T23:07:17.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.981+0000 7f7a06bdc740 1 -- 172.21.15.84:0/305054761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2440fddb0 msgr2=0x55d244114480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:17.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.981+0000 7f7a06bdc740 1 --2- 172.21.15.84:0/305054761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2440fddb0 0x55d244114480 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55d2440fb630 tx=0x7f79ec03f040 comp rx=0 tx=0).stop 2024-05-22T23:07:17.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.981+0000 7f7a06bdc740 1 -- 172.21.15.84:0/305054761 shutdown_connections 2024-05-22T23:07:17.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.981+0000 7f7a06bdc740 1 --2- 172.21.15.84:0/305054761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d244108b30 0x55d2440fa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.981+0000 7f7a06bdc740 1 --2- 172.21.15.84:0/305054761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2440ff4a0 0x55d2440f9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.981+0000 7f7a06bdc740 1 --2- 172.21.15.84:0/305054761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2440fddb0 0x55d244114480 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.981+0000 7f7a06bdc740 1 -- 172.21.15.84:0/305054761 >> 172.21.15.84:0/305054761 conn(0x55d2440c68e0 msgr2=0x55d244113d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:17.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.981+0000 7f7a06bdc740 1 -- 172.21.15.84:0/305054761 shutdown_connections 2024-05-22T23:07:17.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.981+0000 7f7a06bdc740 1 -- 172.21.15.84:0/305054761 wait complete. 2024-05-22T23:07:17.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.981+0000 7f7a06bdc740 1 -- start start 2024-05-22T23:07:17.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.981+0000 7f7a06bdc740 1 Processor -- start 2024-05-22T23:07:17.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.981+0000 7f7a06bdc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2440fddb0 0x55d24411beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f7a06bdc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2440ff4a0 0x55d24411c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f7a06bdc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d244108b30 0x55d24411c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f7a06bdc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc1aa0b0 con 0x55d2440fddb0 2024-05-22T23:07:17.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f7a06bdc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc1ab520 con 0x55d244108b30 2024-05-22T23:07:17.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f7a06bdc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc1aa780 con 0x55d2440ff4a0 2024-05-22T23:07:17.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f7a06bdc740 10 client.0 _ll_register_callbacks cb 0x7f79ec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:07:17.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f7a01389640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2440fddb0 0x55d24411beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f7a01389640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2440fddb0 0x55d24411beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51252/0 (socket says 172.21.15.84:51252) 2024-05-22T23:07:17.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f7a01389640 1 -- 172.21.15.84:0/846236014 learned_addr learned my addr 172.21.15.84:0/846236014 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:17.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f7a01389640 1 --2- 172.21.15.84:0/846236014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d244108b30 0x55d24411c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f7a01b8a640 1 --2- 172.21.15.84:0/846236014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2440ff4a0 0x55d24411c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f7a01389640 1 -- 172.21.15.84:0/846236014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2440ff4a0 msgr2=0x55d24411c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:17.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f7a01389640 1 --2- 172.21.15.84:0/846236014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2440ff4a0 0x55d24411c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f7a01389640 1 -- 172.21.15.84:0/846236014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d244108b30 msgr2=0x55d24411c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:17.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f7a01389640 1 --2- 172.21.15.84:0/846236014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d244108b30 0x55d24411c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f7a01389640 1 -- 172.21.15.84:0/846236014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d244020fe0 con 0x55d2440fddb0 2024-05-22T23:07:17.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f7a01b8a640 1 --2- 172.21.15.84:0/846236014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2440ff4a0 0x55d24411c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:07:17.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f7a01389640 1 --2- 172.21.15.84:0/846236014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2440fddb0 0x55d24411beb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x55d2440f5e30 tx=0x7f79ec005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef9303db7b86fd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:17.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f79e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:07:17.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f79e57fa640 1 -- 172.21.15.84:0/846236014 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec035070 con 0x55d2440fddb0 2024-05-22T23:07:17.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f79e57fa640 1 -- 172.21.15.84:0/846236014 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f79ec0315b0 con 0x55d2440fddb0 2024-05-22T23:07:17.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f7a06bdc740 1 -- 172.21.15.84:0/846236014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79ec03dc40 con 0x55d2440fddb0 2024-05-22T23:07:17.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f79e57fa640 1 -- 172.21.15.84:0/846236014 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f79ec0046e0 con 0x55d2440fddb0 2024-05-22T23:07:17.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f7a06bdc740 10 client.16326 fetch_fsmap learned FSMap version 18 2024-05-22T23:07:17.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f7a06bdc740 1 -- 172.21.15.84:0/846236014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55d243ff21c0 con 0x55d2440fddb0 2024-05-22T23:07:17.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.982+0000 7f79e57fa640 1 -- 172.21.15.84:0/846236014 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79ec03dc40 con 0x55d2440fddb0 2024-05-22T23:07:17.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.983+0000 7f79e57fa640 1 -- 172.21.15.84:0/846236014 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f79ec0597b0 con 0x55d2440fddb0 2024-05-22T23:07:17.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.983+0000 7f7a06bdc740 10 client.16326 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:07:17.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.983+0000 7f7a06bdc740 10 client.16326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:07:17.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.983+0000 7f7a06bdc740 20 client.16326 populate_metadata read hostname 'smithi084' 2024-05-22T23:07:17.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.983+0000 7f7a06bdc740 1 --2- 172.21.15.84:0/846236014 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d24417f5d0 0x55d2441819c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:17.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.983+0000 7f7a06bdc740 4 client.16326 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:07:17.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.983+0000 7f7a06bdc740 1 -- 172.21.15.84:0/846236014 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d244069830 con 0x55d24417f5d0 2024-05-22T23:07:17.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.983+0000 7f7a01b8a640 1 --2- 172.21.15.84:0/846236014 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d24417f5d0 0x55d2441819c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:17.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.983+0000 7f79e57fa640 10 client.16326 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:17.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.983+0000 7f7a01b8a640 1 --2- 172.21.15.84:0/846236014 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d24417f5d0 0x55d2441819c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ae47b8991456c42 server_cookie=b279f33afeba4451 in_seq=0 out_seq=0 2024-05-22T23:07:17.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.984+0000 7f79e57fa640 1 -- 172.21.15.84:0/846236014 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d244069830 con 0x55d24417f5d0 2024-05-22T23:07:17.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.984+0000 7f79e57fa640 10 client.16326 handle_command_reply: tid=0 2024-05-22T23:07:17.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.984+0000 7f7a06bdc740 1 client.16326 shutdown 2024-05-22T23:07:17.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.984+0000 7f7a06bdc740 1 -- 172.21.15.84:0/846236014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2440fddb0 msgr2=0x55d24411beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:17.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.984+0000 7f7a06bdc740 1 --2- 172.21.15.84:0/846236014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2440fddb0 0x55d24411beb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x55d2440f5e30 tx=0x7f79ec005060 comp rx=0 tx=0).stop 2024-05-22T23:07:17.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.984+0000 7f7a06bdc740 1 -- 172.21.15.84:0/846236014 shutdown_connections 2024-05-22T23:07:17.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.984+0000 7f7a06bdc740 1 --2- 172.21.15.84:0/846236014 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55d24417f5d0 0x55d2441819c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.984+0000 7f7a06bdc740 1 --2- 172.21.15.84:0/846236014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d244108b30 0x55d24411c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.984+0000 7f79e57fa640 0 client.16326 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:17.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.984+0000 7f7a06bdc740 1 --2- 172.21.15.84:0/846236014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2440ff4a0 0x55d24411c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.984+0000 7f7a06bdc740 1 --2- 172.21.15.84:0/846236014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2440fddb0 0x55d24411beb0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.984+0000 7f7a06bdc740 1 -- 172.21.15.84:0/846236014 >> 172.21.15.84:0/846236014 conn(0x55d2440c68e0 msgr2=0x55d244106260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:17.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.984+0000 7f7a06bdc740 1 -- 172.21.15.84:0/846236014 shutdown_connections 2024-05-22T23:07:17.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.984+0000 7f7a06bdc740 1 -- 172.21.15.84:0/846236014 wait complete. 2024-05-22T23:07:17.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.984+0000 7f7a06bdc740 20 client.16326 trim_cache size 0 max 16384 2024-05-22T23:07:17.996 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:07:17.996 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-22T23:07:17.996 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": { 2024-05-22T23:07:17.997 INFO:teuthology.orchestra.run.smithi084.stdout: "0755f526-bf73-48b4-a6b6-78f4916fd1ae": { 2024-05-22T23:07:17.997 INFO:teuthology.orchestra.run.smithi084.stdout: "path": "/", 2024-05-22T23:07:17.997 INFO:teuthology.orchestra.run.smithi084.stdout: "tag": "0755f526-bf73-48b4-a6b6-78f4916fd1ae", 2024-05-22T23:07:17.997 INFO:teuthology.orchestra.run.smithi084.stdout: "options": "recursive,force" 2024-05-22T23:07:17.997 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:07:17.997 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:07:17.997 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:07:17.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.984+0000 7f7a03dee640 1 -- 172.21.15.84:0/2960234102 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f79e80a2d00 msgr2=0x7f79e80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:17.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.984+0000 7f7a03dee640 1 --2- 172.21.15.84:0/2960234102 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f79e80a2d00 0x7f79e80a51c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f79f409bf00 tx=0x7f79f40bc040 comp rx=0 tx=0).stop 2024-05-22T23:07:17.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.984+0000 7f7a03dee640 1 -- 172.21.15.84:0/2960234102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79fc106750 msgr2=0x7f79fc19c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:17.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.984+0000 7f7a03dee640 1 --2- 172.21.15.84:0/2960234102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79fc106750 0x7f79fc19c9f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f79f4009fd0 tx=0x7f79f400ef10 comp rx=0 tx=0).stop 2024-05-22T23:07:17.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.984+0000 7f7a03dee640 1 -- 172.21.15.84:0/2960234102 shutdown_connections 2024-05-22T23:07:17.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.984+0000 7f7a03dee640 1 --2- 172.21.15.84:0/2960234102 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f79e80a2d00 0x7f79e80a51c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.984+0000 7f7a03dee640 1 --2- 172.21.15.84:0/2960234102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79fc106750 0x7f79fc19c9f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.984+0000 7f7a03dee640 1 --2- 172.21.15.84:0/2960234102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79fc101100 0x7f79fc1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.985+0000 7f7a03dee640 1 --2- 172.21.15.84:0/2960234102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79fc100740 0x7f79fc198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:17.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.985+0000 7f7a03dee640 1 -- 172.21.15.84:0/2960234102 >> 172.21.15.84:0/2960234102 conn(0x7f79fc077870 msgr2=0x7f79fc102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:17.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.985+0000 7f7a03dee640 1 -- 172.21.15.84:0/2960234102 shutdown_connections 2024-05-22T23:07:17.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:17.985+0000 7f7a03dee640 1 -- 172.21.15.84:0/2960234102 wait complete. 2024-05-22T23:07:18.001 INFO:tasks.cephfs.filesystem:scrub status for tag:0755f526-bf73-48b4-a6b6-78f4916fd1ae - {'path': '/', 'tag': '0755f526-bf73-48b4-a6b6-78f4916fd1ae', 'options': 'recursive,force'} 2024-05-22T23:07:18.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:17 smithi096 ceph-mon[36392]: pgmap v166: 97 pgs: 97 active+clean; 367 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.1 MiB/s wr, 380 op/s 2024-05-22T23:07:18.436 INFO:tasks.workunit.client.0.smithi084.stdout: 1 698971 15.23 MB/sec execute 124 sec latency 14.663 ms 2024-05-22T23:07:18.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:18 smithi084 ceph-mon[32186]: scrub queued for path: / 2024-05-22T23:07:18.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:18 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:07:18.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:18 smithi084 ceph-mon[32186]: scrub summary: active paths [/] 2024-05-22T23:07:18.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:18 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:07:18.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:18 smithi204 ceph-mon[36211]: scrub queued for path: / 2024-05-22T23:07:18.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:18 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:07:18.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:18 smithi204 ceph-mon[36211]: scrub summary: active paths [/] 2024-05-22T23:07:18.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:18 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:07:19.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:18 smithi096 ceph-mon[36392]: scrub queued for path: / 2024-05-22T23:07:19.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:18 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:07:19.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:18 smithi096 ceph-mon[36392]: scrub summary: active paths [/] 2024-05-22T23:07:19.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:18 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:07:19.436 INFO:tasks.workunit.client.0.smithi084.stdout: 1 701563 15.20 MB/sec execute 125 sec latency 14.668 ms 2024-05-22T23:07:19.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:19 smithi084 ceph-mon[32186]: pgmap v167: 97 pgs: 97 active+clean; 367 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 244 op/s 2024-05-22T23:07:19.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:19 smithi204 ceph-mon[36211]: pgmap v167: 97 pgs: 97 active+clean; 367 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 244 op/s 2024-05-22T23:07:20.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:19 smithi096 ceph-mon[36392]: pgmap v167: 97 pgs: 97 active+clean; 367 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 244 op/s 2024-05-22T23:07:20.436 INFO:tasks.workunit.client.0.smithi084.stdout: 1 703950 15.19 MB/sec execute 126 sec latency 72.054 ms 2024-05-22T23:07:21.436 INFO:tasks.workunit.client.0.smithi084.stdout: 1 706918 15.22 MB/sec execute 127 sec latency 91.775 ms 2024-05-22T23:07:21.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:21 smithi084 ceph-mon[32186]: pgmap v168: 97 pgs: 97 active+clean; 367 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 244 op/s 2024-05-22T23:07:21.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:21 smithi084 ceph-mon[32186]: scrub summary: idle 2024-05-22T23:07:21.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:21 smithi204 ceph-mon[36211]: pgmap v168: 97 pgs: 97 active+clean; 367 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 244 op/s 2024-05-22T23:07:21.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:21 smithi204 ceph-mon[36211]: scrub summary: idle 2024-05-22T23:07:22.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:21 smithi096 ceph-mon[36392]: pgmap v168: 97 pgs: 97 active+clean; 367 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 244 op/s 2024-05-22T23:07:22.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:21 smithi096 ceph-mon[36392]: scrub summary: idle 2024-05-22T23:07:22.436 INFO:tasks.workunit.client.0.smithi084.stdout: 1 709841 15.20 MB/sec execute 128 sec latency 18.807 ms 2024-05-22T23:07:23.436 INFO:tasks.workunit.client.0.smithi084.stdout: 1 712428 15.18 MB/sec execute 129 sec latency 23.793 ms 2024-05-22T23:07:23.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:23 smithi084 ceph-mon[32186]: pgmap v169: 97 pgs: 97 active+clean; 377 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.1 MiB/s wr, 366 op/s 2024-05-22T23:07:23.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:23 smithi204 ceph-mon[36211]: pgmap v169: 97 pgs: 97 active+clean; 377 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.1 MiB/s wr, 366 op/s 2024-05-22T23:07:24.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:23 smithi096 ceph-mon[36392]: pgmap v169: 97 pgs: 97 active+clean; 377 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.1 MiB/s wr, 366 op/s 2024-05-22T23:07:24.436 INFO:tasks.workunit.client.0.smithi084.stdout: 1 715002 15.18 MB/sec execute 130 sec latency 14.328 ms 2024-05-22T23:07:25.437 INFO:tasks.workunit.client.0.smithi084.stdout: 1 717788 15.19 MB/sec execute 131 sec latency 92.841 ms 2024-05-22T23:07:25.854 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:25 smithi096 ceph-mon[36392]: pgmap v170: 97 pgs: 97 active+clean; 377 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 244 op/s 2024-05-22T23:07:25.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:25 smithi084 ceph-mon[32186]: pgmap v170: 97 pgs: 97 active+clean; 377 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 244 op/s 2024-05-22T23:07:25.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:25 smithi204 ceph-mon[36211]: pgmap v170: 97 pgs: 97 active+clean; 377 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 244 op/s 2024-05-22T23:07:26.437 INFO:tasks.workunit.client.0.smithi084.stdout: 1 720508 15.18 MB/sec execute 132 sec latency 71.975 ms 2024-05-22T23:07:27.437 INFO:tasks.workunit.client.0.smithi084.stdout: 1 722997 15.15 MB/sec execute 133 sec latency 24.231 ms 2024-05-22T23:07:27.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:27 smithi084 ceph-mon[32186]: pgmap v171: 97 pgs: 97 active+clean; 372 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.2 MiB/s wr, 348 op/s 2024-05-22T23:07:27.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:27 smithi204 ceph-mon[36211]: pgmap v171: 97 pgs: 97 active+clean; 372 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.2 MiB/s wr, 348 op/s 2024-05-22T23:07:28.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:27 smithi096 ceph-mon[36392]: pgmap v171: 97 pgs: 97 active+clean; 372 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.2 MiB/s wr, 348 op/s 2024-05-22T23:07:28.437 INFO:tasks.workunit.client.0.smithi084.stdout: 1 725571 15.15 MB/sec execute 134 sec latency 14.576 ms 2024-05-22T23:07:29.437 INFO:tasks.workunit.client.0.smithi084.stdout: 1 728502 15.17 MB/sec execute 135 sec latency 20.373 ms 2024-05-22T23:07:29.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:29 smithi084 ceph-mon[32186]: pgmap v172: 97 pgs: 97 active+clean; 372 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 226 op/s 2024-05-22T23:07:29.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:29 smithi204 ceph-mon[36211]: pgmap v172: 97 pgs: 97 active+clean; 372 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 226 op/s 2024-05-22T23:07:30.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:29 smithi096 ceph-mon[36392]: pgmap v172: 97 pgs: 97 active+clean; 372 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 226 op/s 2024-05-22T23:07:30.437 INFO:tasks.workunit.client.0.smithi084.stdout: 1 731720 15.19 MB/sec execute 136 sec latency 23.524 ms 2024-05-22T23:07:31.437 INFO:tasks.workunit.client.0.smithi084.stdout: 1 733898 15.14 MB/sec execute 137 sec latency 123.747 ms 2024-05-22T23:07:31.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:31 smithi084 ceph-mon[32186]: pgmap v173: 97 pgs: 97 active+clean; 372 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 226 op/s 2024-05-22T23:07:31.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:31 smithi204 ceph-mon[36211]: pgmap v173: 97 pgs: 97 active+clean; 372 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 226 op/s 2024-05-22T23:07:32.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:31 smithi096 ceph-mon[36392]: pgmap v173: 97 pgs: 97 active+clean; 372 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 226 op/s 2024-05-22T23:07:32.437 INFO:tasks.workunit.client.0.smithi084.stdout: 1 736221 15.13 MB/sec execute 138 sec latency 23.323 ms 2024-05-22T23:07:33.437 INFO:tasks.workunit.client.0.smithi084.stdout: 1 739040 15.14 MB/sec execute 139 sec latency 10.475 ms 2024-05-22T23:07:33.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:33 smithi084 ceph-mon[32186]: pgmap v174: 97 pgs: 97 active+clean; 381 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.1 MiB/s wr, 344 op/s 2024-05-22T23:07:33.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:33 smithi204 ceph-mon[36211]: pgmap v174: 97 pgs: 97 active+clean; 381 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.1 MiB/s wr, 344 op/s 2024-05-22T23:07:34.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:33 smithi096 ceph-mon[36392]: pgmap v174: 97 pgs: 97 active+clean; 381 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.1 MiB/s wr, 344 op/s 2024-05-22T23:07:34.437 INFO:tasks.workunit.client.0.smithi084.stdout: 1 742250 15.16 MB/sec execute 140 sec latency 21.579 ms 2024-05-22T23:07:35.437 INFO:tasks.workunit.client.0.smithi084.stdout: 1 745006 15.14 MB/sec execute 141 sec latency 16.762 ms 2024-05-22T23:07:35.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:35 smithi084 ceph-mon[32186]: pgmap v175: 97 pgs: 97 active+clean; 381 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 222 op/s 2024-05-22T23:07:35.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:35 smithi204 ceph-mon[36211]: pgmap v175: 97 pgs: 97 active+clean; 381 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 222 op/s 2024-05-22T23:07:36.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:35 smithi096 ceph-mon[36392]: pgmap v175: 97 pgs: 97 active+clean; 381 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 222 op/s 2024-05-22T23:07:36.437 INFO:tasks.workunit.client.0.smithi084.stdout: 1 747235 15.13 MB/sec execute 142 sec latency 84.093 ms 2024-05-22T23:07:37.437 INFO:tasks.workunit.client.0.smithi084.stdout: 1 749974 15.12 MB/sec execute 143 sec latency 32.938 ms 2024-05-22T23:07:37.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:37 smithi084 ceph-mon[32186]: pgmap v176: 97 pgs: 97 active+clean; 389 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.0 MiB/s wr, 342 op/s 2024-05-22T23:07:37.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:37 smithi204 ceph-mon[36211]: pgmap v176: 97 pgs: 97 active+clean; 389 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.0 MiB/s wr, 342 op/s 2024-05-22T23:07:38.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:37 smithi096 ceph-mon[36392]: pgmap v176: 97 pgs: 97 active+clean; 389 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.0 MiB/s wr, 342 op/s 2024-05-22T23:07:38.437 INFO:tasks.workunit.client.0.smithi084.stdout: 1 753167 15.15 MB/sec execute 144 sec latency 17.536 ms 2024-05-22T23:07:39.437 INFO:tasks.workunit.client.0.smithi084.stdout: 1 756390 15.17 MB/sec execute 145 sec latency 17.187 ms 2024-05-22T23:07:39.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:39 smithi084 ceph-mon[32186]: pgmap v177: 97 pgs: 97 active+clean; 389 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 238 op/s 2024-05-22T23:07:39.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:39 smithi204 ceph-mon[36211]: pgmap v177: 97 pgs: 97 active+clean; 389 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 238 op/s 2024-05-22T23:07:40.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:39 smithi096 ceph-mon[36392]: pgmap v177: 97 pgs: 97 active+clean; 389 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 238 op/s 2024-05-22T23:07:40.437 INFO:tasks.workunit.client.0.smithi084.stdout: 1 758914 15.13 MB/sec execute 146 sec latency 58.798 ms 2024-05-22T23:07:41.437 INFO:tasks.workunit.client.0.smithi084.stdout: 1 760781 15.11 MB/sec execute 147 sec latency 93.988 ms 2024-05-22T23:07:41.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:41 smithi084 ceph-mon[32186]: pgmap v178: 97 pgs: 97 active+clean; 389 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 238 op/s 2024-05-22T23:07:41.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:41 smithi204 ceph-mon[36211]: pgmap v178: 97 pgs: 97 active+clean; 389 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 238 op/s 2024-05-22T23:07:42.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:41 smithi096 ceph-mon[36392]: pgmap v178: 97 pgs: 97 active+clean; 389 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 238 op/s 2024-05-22T23:07:42.437 INFO:tasks.workunit.client.0.smithi084.stdout: 1 763842 15.13 MB/sec execute 148 sec latency 36.660 ms 2024-05-22T23:07:43.437 INFO:tasks.workunit.client.0.smithi084.stdout: 1 767025 15.14 MB/sec execute 149 sec latency 17.424 ms 2024-05-22T23:07:43.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:43 smithi084 ceph-mon[32186]: pgmap v179: 97 pgs: 97 active+clean; 399 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.0 MiB/s wr, 366 op/s 2024-05-22T23:07:43.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:43 smithi204 ceph-mon[36211]: pgmap v179: 97 pgs: 97 active+clean; 399 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.0 MiB/s wr, 366 op/s 2024-05-22T23:07:44.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:43 smithi096 ceph-mon[36392]: pgmap v179: 97 pgs: 97 active+clean; 399 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.0 MiB/s wr, 366 op/s 2024-05-22T23:07:44.437 INFO:tasks.workunit.client.0.smithi084.stdout: 1 769453 15.11 MB/sec execute 150 sec latency 25.520 ms 2024-05-22T23:07:45.438 INFO:tasks.workunit.client.0.smithi084.stdout: 1 772274 15.12 MB/sec execute 151 sec latency 13.869 ms 2024-05-22T23:07:45.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:45 smithi084 ceph-mon[32186]: pgmap v180: 97 pgs: 97 active+clean; 399 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.0 MiB/s wr, 248 op/s 2024-05-22T23:07:45.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:45 smithi204 ceph-mon[36211]: pgmap v180: 97 pgs: 97 active+clean; 399 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.0 MiB/s wr, 248 op/s 2024-05-22T23:07:46.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:45 smithi096 ceph-mon[36392]: pgmap v180: 97 pgs: 97 active+clean; 399 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.0 MiB/s wr, 248 op/s 2024-05-22T23:07:46.438 INFO:tasks.workunit.client.0.smithi084.stdout: 1 774632 15.11 MB/sec execute 152 sec latency 70.037 ms 2024-05-22T23:07:47.438 INFO:tasks.workunit.client.0.smithi084.stdout: 1 777750 15.12 MB/sec execute 153 sec latency 36.780 ms 2024-05-22T23:07:47.878 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:47 smithi084 ceph-mon[32186]: pgmap v181: 97 pgs: 97 active+clean; 401 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.4 MiB/s wr, 363 op/s 2024-05-22T23:07:47.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:47 smithi204 ceph-mon[36211]: pgmap v181: 97 pgs: 97 active+clean; 401 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.4 MiB/s wr, 363 op/s 2024-05-22T23:07:48.003 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:07:48.003 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:07:48.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:47 smithi096 ceph-mon[36392]: pgmap v181: 97 pgs: 97 active+clean; 401 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.4 MiB/s wr, 363 op/s 2024-05-22T23:07:48.113 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.107+0000 7f5be90ca640 1 -- 172.21.15.84:0/3267255084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be4103410 msgr2=0x7f5be410b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:48.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.107+0000 7f5be90ca640 1 --2- 172.21.15.84:0/3267255084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be4103410 0x7f5be410b8f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009900 tx=0x7f5bcc02a510 comp rx=0 tx=0).stop 2024-05-22T23:07:48.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.107+0000 7f5be90ca640 1 -- 172.21.15.84:0/3267255084 shutdown_connections 2024-05-22T23:07:48.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.107+0000 7f5be90ca640 1 --2- 172.21.15.84:0/3267255084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5be410be30 0x7f5be410e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.107+0000 7f5be90ca640 1 --2- 172.21.15.84:0/3267255084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be4103410 0x7f5be410b8f0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.107+0000 7f5be90ca640 1 --2- 172.21.15.84:0/3267255084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5be4102af0 0x7f5be4102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.108+0000 7f5be90ca640 1 -- 172.21.15.84:0/3267255084 >> 172.21.15.84:0/3267255084 conn(0x7f5be4077870 msgr2=0x7f5be4077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:48.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.108+0000 7f5be90ca640 1 -- 172.21.15.84:0/3267255084 shutdown_connections 2024-05-22T23:07:48.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.108+0000 7f5be90ca640 1 -- 172.21.15.84:0/3267255084 wait complete. 2024-05-22T23:07:48.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.108+0000 7f5be90ca640 1 Processor -- start 2024-05-22T23:07:48.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.108+0000 7f5be90ca640 1 -- start start 2024-05-22T23:07:48.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.109+0000 7f5be90ca640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5be4102af0 0x7f5be4100740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.109+0000 7f5be90ca640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5be4103410 0x7f5be4100c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.109+0000 7f5be90ca640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be410be30 0x7f5be41011c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.109+0000 7f5be90ca640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4111c30 con 0x7f5be410be30 2024-05-22T23:07:48.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.109+0000 7f5be90ca640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4111ab0 con 0x7f5be4102af0 2024-05-22T23:07:48.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.109+0000 7f5be90ca640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4111db0 con 0x7f5be4103410 2024-05-22T23:07:48.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.109+0000 7f5be359e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be410be30 0x7f5be41011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.109+0000 7f5be359e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be410be30 0x7f5be41011c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50108/0 (socket says 172.21.15.84:50108) 2024-05-22T23:07:48.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.109+0000 7f5be2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5be4102af0 0x7f5be4100740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.109+0000 7f5be359e640 1 -- 172.21.15.84:0/2517633350 learned_addr learned my addr 172.21.15.84:0/2517633350 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:48.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.109+0000 7f5bda59c640 1 --2- 172.21.15.84:0/2517633350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5be4103410 0x7f5be4100c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.109+0000 7f5be359e640 1 -- 172.21.15.84:0/2517633350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5be4103410 msgr2=0x7f5be4100c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:48.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.109+0000 7f5be359e640 1 --2- 172.21.15.84:0/2517633350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5be4103410 0x7f5be4100c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.109+0000 7f5be359e640 1 -- 172.21.15.84:0/2517633350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5be4102af0 msgr2=0x7f5be4100740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:48.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.109+0000 7f5be359e640 1 --2- 172.21.15.84:0/2517633350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5be4102af0 0x7f5be4100740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.109+0000 7f5be359e640 1 -- 172.21.15.84:0/2517633350 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be41a6610 con 0x7f5be410be30 2024-05-22T23:07:48.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.109+0000 7f5be2d9d640 1 --2- 172.21.15.84:0/2517633350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5be4102af0 0x7f5be4100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:48.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.109+0000 7f5be359e640 1 --2- 172.21.15.84:0/2517633350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be410be30 0x7f5be41011c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00eb10 tx=0x7f5bdc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:48.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.109+0000 7f5be0d99640 1 -- 172.21.15.84:0/2517633350 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc004b20 con 0x7f5be410be30 2024-05-22T23:07:48.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.110+0000 7f5be0d99640 1 -- 172.21.15.84:0/2517633350 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bdc004cc0 con 0x7f5be410be30 2024-05-22T23:07:48.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.110+0000 7f5be0d99640 1 -- 172.21.15.84:0/2517633350 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc010760 con 0x7f5be410be30 2024-05-22T23:07:48.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.110+0000 7f5be90ca640 1 -- 172.21.15.84:0/2517633350 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be41a6900 con 0x7f5be410be30 2024-05-22T23:07:48.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.110+0000 7f5be90ca640 1 -- 172.21.15.84:0/2517633350 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be41a6dd0 con 0x7f5be410be30 2024-05-22T23:07:48.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.112+0000 7f5be0d99640 1 -- 172.21.15.84:0/2517633350 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5bdc005020 con 0x7f5be410be30 2024-05-22T23:07:48.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.112+0000 7f5bebdd6740 1 Processor -- start 2024-05-22T23:07:48.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.112+0000 7f5bebdd6740 1 -- start start 2024-05-22T23:07:48.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.112+0000 7f5be0d99640 1 --2- 172.21.15.84:0/2517633350 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f5bd00a2d00 0x7f5bd00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.112+0000 7f5bebdd6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b004a0 0x561313afe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.112+0000 7f5bebdd6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561313afedb0 0x561313aff230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.113+0000 7f5bebdd6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561313b09b30 0x561313b0bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.113+0000 7f5bebdd6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561313a62db0 con 0x561313b004a0 2024-05-22T23:07:48.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.113+0000 7f5bebdd6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561313a01360 con 0x561313b09b30 2024-05-22T23:07:48.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.113+0000 7f5bebdd6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561313a5ed70 con 0x561313afedb0 2024-05-22T23:07:48.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.113+0000 7f5be2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b004a0 0x561313afe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.113+0000 7f5bda59c640 1 --2- 172.21.15.84:0/2517633350 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f5bd00a2d00 0x7f5bd00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.113+0000 7f5be2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b004a0 0x561313afe7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50112/0 (socket says 172.21.15.84:50112) 2024-05-22T23:07:48.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.113+0000 7f5be2d9d640 1 -- 172.21.15.84:0/3595923660 learned_addr learned my addr 172.21.15.84:0/3595923660 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:48.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.113+0000 7f5be359e640 1 --2- 172.21.15.84:0/3595923660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561313b09b30 0x561313b0bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.113+0000 7f5bda59c640 1 --2- 172.21.15.84:0/3595923660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561313afedb0 0x561313aff230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.113+0000 7f5be0d99640 1 -- 172.21.15.84:0/2517633350 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f5bdc0cf940 con 0x7f5be410be30 2024-05-22T23:07:48.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.113+0000 7f5be2d9d640 1 -- 172.21.15.84:0/3595923660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561313afedb0 msgr2=0x561313aff230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:48.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.113+0000 7f5be2d9d640 1 --2- 172.21.15.84:0/3595923660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561313afedb0 0x561313aff230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.113+0000 7f5be2d9d640 1 -- 172.21.15.84:0/3595923660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561313b09b30 msgr2=0x561313b0bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:48.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.113+0000 7f5be2d9d640 1 --2- 172.21.15.84:0/3595923660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561313b09b30 0x561313b0bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.113+0000 7f5be2d9d640 1 -- 172.21.15.84:0/3595923660 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561313a24a60 con 0x561313b004a0 2024-05-22T23:07:48.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.113+0000 7f5be359e640 1 --2- 172.21.15.84:0/3595923660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561313b09b30 0x561313b0bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:48.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.113+0000 7f5be2d9d640 1 --2- 172.21.15.84:0/3595923660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b004a0 0x561313afe7e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40115f0 tx=0x7f5bd40119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df78d1d9ec52acbb server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:48.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.113+0000 7f5bda59c640 1 --2- 172.21.15.84:0/2517633350 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f5bd00a2d00 0x7f5bd00a51c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009230 tx=0x7f5bcc005d50 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:48.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.113+0000 7f5bd959a640 1 -- 172.21.15.84:0/3595923660 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd401e070 con 0x561313b004a0 2024-05-22T23:07:48.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.113+0000 7f5bd959a640 1 -- 172.21.15.84:0/3595923660 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bd400fa40 con 0x561313b004a0 2024-05-22T23:07:48.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.113+0000 7f5bd959a640 1 -- 172.21.15.84:0/3595923660 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4019bc0 con 0x561313b004a0 2024-05-22T23:07:48.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.114+0000 7f5bebdd6740 1 -- 172.21.15.84:0/3595923660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b004a0 msgr2=0x561313afe7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:48.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.114+0000 7f5bebdd6740 1 --2- 172.21.15.84:0/3595923660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b004a0 0x561313afe7e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40115f0 tx=0x7f5bd40119f0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.114+0000 7f5bebdd6740 1 -- 172.21.15.84:0/3595923660 shutdown_connections 2024-05-22T23:07:48.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.114+0000 7f5bebdd6740 1 --2- 172.21.15.84:0/3595923660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561313b09b30 0x561313b0bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.114+0000 7f5bebdd6740 1 --2- 172.21.15.84:0/3595923660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561313afedb0 0x561313aff230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.114+0000 7f5bebdd6740 1 --2- 172.21.15.84:0/3595923660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b004a0 0x561313afe7e0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.114+0000 7f5bebdd6740 1 -- 172.21.15.84:0/3595923660 >> 172.21.15.84:0/3595923660 conn(0x561313ac78e0 msgr2=0x561313afb310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:48.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.114+0000 7f5bebdd6740 1 -- 172.21.15.84:0/3595923660 shutdown_connections 2024-05-22T23:07:48.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.114+0000 7f5bebdd6740 1 -- 172.21.15.84:0/3595923660 wait complete. 2024-05-22T23:07:48.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.115+0000 7f5bebdd6740 1 -- start start 2024-05-22T23:07:48.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.115+0000 7f5bebdd6740 1 Processor -- start 2024-05-22T23:07:48.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.115+0000 7f5bebdd6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561313afedb0 0x561313b06a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.115+0000 7f5bebdd6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561313b004a0 0x561313b06f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.115+0000 7f5bebdd6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b09b30 0x561313b074c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.115+0000 7f5bebdd6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561313b0c9c0 con 0x561313b09b30 2024-05-22T23:07:48.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.115+0000 7f5bebdd6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561313a1d0f0 con 0x561313b004a0 2024-05-22T23:07:48.129 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.115+0000 7f5bebdd6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561313b0cb40 con 0x561313afedb0 2024-05-22T23:07:48.129 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.115+0000 7f5bebdd6740 10 client.0 _ll_register_callbacks cb 0x561313a21f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:07:48.129 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.115+0000 7f5be359e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b09b30 0x561313b074c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.129 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.115+0000 7f5bda59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561313b004a0 0x561313b06f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.129 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.115+0000 7f5be359e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b09b30 0x561313b074c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50122/0 (socket says 172.21.15.84:50122) 2024-05-22T23:07:48.129 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.115+0000 7f5be359e640 1 -- 172.21.15.84:0/2987975630 learned_addr learned my addr 172.21.15.84:0/2987975630 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:48.129 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.115+0000 7f5be2d9d640 1 --2- 172.21.15.84:0/2987975630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561313afedb0 0x561313b06a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.129 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.115+0000 7f5be359e640 1 -- 172.21.15.84:0/2987975630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561313afedb0 msgr2=0x561313b06a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:48.129 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.115+0000 7f5be359e640 1 --2- 172.21.15.84:0/2987975630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561313afedb0 0x561313b06a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.129 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.115+0000 7f5be359e640 1 -- 172.21.15.84:0/2987975630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561313b004a0 msgr2=0x561313b06f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:48.129 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.115+0000 7f5be359e640 1 --2- 172.21.15.84:0/2987975630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561313b004a0 0x561313b06f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.129 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.115+0000 7f5be359e640 1 -- 172.21.15.84:0/2987975630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613138407d0 con 0x561313b09b30 2024-05-22T23:07:48.129 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.115+0000 7f5bda59c640 1 --2- 172.21.15.84:0/2987975630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561313b004a0 0x561313b06f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:48.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.116+0000 7f5be359e640 1 --2- 172.21.15.84:0/2987975630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b09b30 0x561313b074c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc09bb60 tx=0x7f5bdc01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1737a1542ddc276 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:48.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.116+0000 7f5bca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:07:48.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.116+0000 7f5bca7fc640 1 -- 172.21.15.84:0/2987975630 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc09be60 con 0x561313b09b30 2024-05-22T23:07:48.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.116+0000 7f5bca7fc640 1 -- 172.21.15.84:0/2987975630 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f5bdc0bd290 con 0x561313b09b30 2024-05-22T23:07:48.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.116+0000 7f5bebdd6740 1 -- 172.21.15.84:0/2987975630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bd400fa40 con 0x561313b09b30 2024-05-22T23:07:48.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.116+0000 7f5bca7fc640 1 -- 172.21.15.84:0/2987975630 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f5bdc0a0210 con 0x561313b09b30 2024-05-22T23:07:48.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.116+0000 7f5bebdd6740 10 client.16338 fetch_fsmap learned FSMap version 18 2024-05-22T23:07:48.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.116+0000 7f5bebdd6740 1 -- 172.21.15.84:0/2987975630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5613137763f0 con 0x561313b09b30 2024-05-22T23:07:48.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.116+0000 7f5bca7fc640 1 -- 172.21.15.84:0/2987975630 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bd400fa40 con 0x561313b09b30 2024-05-22T23:07:48.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.116+0000 7f5bca7fc640 1 -- 172.21.15.84:0/2987975630 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f5bdc0765d0 con 0x561313b09b30 2024-05-22T23:07:48.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.116+0000 7f5bebdd6740 10 client.16338 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:07:48.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.116+0000 7f5bebdd6740 10 client.16338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:07:48.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.116+0000 7f5bebdd6740 20 client.16338 populate_metadata read hostname 'smithi084' 2024-05-22T23:07:48.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.116+0000 7f5bebdd6740 1 --2- 172.21.15.84:0/2987975630 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x561313b1a280 0x561313b1c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.116+0000 7f5bebdd6740 4 client.16338 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:07:48.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.116+0000 7f5bebdd6740 1 -- 172.21.15.84:0/2987975630 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561313a704e0 con 0x561313b1a280 2024-05-22T23:07:48.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.116+0000 7f5be2d9d640 1 --2- 172.21.15.84:0/2987975630 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x561313b1a280 0x561313b1c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.117+0000 7f5bca7fc640 10 client.16338 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:48.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.117+0000 7f5be2d9d640 1 --2- 172.21.15.84:0/2987975630 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x561313b1a280 0x561313b1c670 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7711ed6e97ebf19 server_cookie=1b216331ed6c722 in_seq=0 out_seq=0 2024-05-22T23:07:48.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.118+0000 7f5bca7fc640 1 -- 172.21.15.84:0/2987975630 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561313a704e0 con 0x561313b1a280 2024-05-22T23:07:48.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.118+0000 7f5bca7fc640 10 client.16338 handle_command_reply: tid=0 2024-05-22T23:07:48.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.118+0000 7f5bebdd6740 1 client.16338 shutdown 2024-05-22T23:07:48.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.118+0000 7f5bebdd6740 1 -- 172.21.15.84:0/2987975630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b09b30 msgr2=0x561313b074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:48.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.118+0000 7f5bebdd6740 1 --2- 172.21.15.84:0/2987975630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b09b30 0x561313b074c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc09bb60 tx=0x7f5bdc01d000 comp rx=0 tx=0).stop 2024-05-22T23:07:48.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.118+0000 7f5bebdd6740 1 -- 172.21.15.84:0/2987975630 shutdown_connections 2024-05-22T23:07:48.131 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.118+0000 7f5bebdd6740 1 --2- 172.21.15.84:0/2987975630 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x561313b1a280 0x561313b1c670 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.132 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.118+0000 7f5bebdd6740 1 --2- 172.21.15.84:0/2987975630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b09b30 0x561313b074c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.132 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.118+0000 7f5bebdd6740 1 --2- 172.21.15.84:0/2987975630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561313b004a0 0x561313b06f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.132 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.118+0000 7f5bebdd6740 1 --2- 172.21.15.84:0/2987975630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561313afedb0 0x561313b06a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.132 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.118+0000 7f5bca7fc640 0 client.16338 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:48.132 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.118+0000 7f5bebdd6740 1 -- 172.21.15.84:0/2987975630 >> 172.21.15.84:0/2987975630 conn(0x561313ac78e0 msgr2=0x561313b01110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:48.132 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.118+0000 7f5bebdd6740 1 -- 172.21.15.84:0/2987975630 shutdown_connections 2024-05-22T23:07:48.132 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.118+0000 7f5bebdd6740 1 -- 172.21.15.84:0/2987975630 wait complete. 2024-05-22T23:07:48.132 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.118+0000 7f5bebdd6740 20 client.16338 trim_cache size 0 max 16384 2024-05-22T23:07:48.143 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.137+0000 7f5bebdd6740 1 Processor -- start 2024-05-22T23:07:48.143 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.137+0000 7f5bebdd6740 1 -- start start 2024-05-22T23:07:48.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.138+0000 7f5bebdd6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561313afedb0 0x561313b158c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.138+0000 7f5bebdd6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b004a0 0x561313af6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.138+0000 7f5bebdd6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561313b09b30 0x561313afa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.138+0000 7f5bebdd6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41ad770 con 0x561313b004a0 2024-05-22T23:07:48.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.138+0000 7f5bebdd6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41afff0 con 0x561313afedb0 2024-05-22T23:07:48.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.138+0000 7f5bebdd6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41adde0 con 0x561313b09b30 2024-05-22T23:07:48.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.138+0000 7f5bda59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b004a0 0x561313af6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.138+0000 7f5be359e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561313b09b30 0x561313afa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.138+0000 7f5be2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561313afedb0 0x561313b158c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.138+0000 7f5bda59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b004a0 0x561313af6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50128/0 (socket says 172.21.15.84:50128) 2024-05-22T23:07:48.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.138+0000 7f5bda59c640 1 -- 172.21.15.84:0/607689678 learned_addr learned my addr 172.21.15.84:0/607689678 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:48.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.138+0000 7f5be359e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561313b09b30 0x561313afa2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:48482/0 (socket says 172.21.15.84:48482) 2024-05-22T23:07:48.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.138+0000 7f5be359e640 1 -- 172.21.15.84:0/607689678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561313afedb0 msgr2=0x561313b158c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:48.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.138+0000 7f5be359e640 1 --2- 172.21.15.84:0/607689678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561313afedb0 0x561313b158c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.138+0000 7f5be359e640 1 -- 172.21.15.84:0/607689678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b004a0 msgr2=0x561313af6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:48.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.138+0000 7f5be359e640 1 --2- 172.21.15.84:0/607689678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b004a0 0x561313af6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.138+0000 7f5be359e640 1 -- 172.21.15.84:0/607689678 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561313b059f0 con 0x561313b09b30 2024-05-22T23:07:48.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.138+0000 7f5bda59c640 1 --2- 172.21.15.84:0/607689678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b004a0 0x561313af6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:48.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.138+0000 7f5be2d9d640 1 --2- 172.21.15.84:0/607689678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561313afedb0 0x561313b158c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:48.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.138+0000 7f5be359e640 1 --2- 172.21.15.84:0/607689678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561313b09b30 0x561313afa2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc09bdf0 tx=0x7f5bdc076490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=799f9504516425cd server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:48.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.139+0000 7f5bd8d99640 1 -- 172.21.15.84:0/607689678 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc09b7c0 con 0x561313b09b30 2024-05-22T23:07:48.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.139+0000 7f5bd8d99640 1 -- 172.21.15.84:0/607689678 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bdc09be60 con 0x561313b09b30 2024-05-22T23:07:48.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.139+0000 7f5bd8d99640 1 -- 172.21.15.84:0/607689678 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc0765d0 con 0x561313b09b30 2024-05-22T23:07:48.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.139+0000 7f5bebdd6740 1 -- 172.21.15.84:0/607689678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561313b09b30 msgr2=0x561313afa2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:48.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.139+0000 7f5bebdd6740 1 --2- 172.21.15.84:0/607689678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561313b09b30 0x561313afa2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc09bdf0 tx=0x7f5bdc076490 comp rx=0 tx=0).stop 2024-05-22T23:07:48.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.140+0000 7f5bebdd6740 1 -- 172.21.15.84:0/607689678 shutdown_connections 2024-05-22T23:07:48.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.140+0000 7f5bebdd6740 1 --2- 172.21.15.84:0/607689678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561313b09b30 0x561313afa2f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.140+0000 7f5bebdd6740 1 --2- 172.21.15.84:0/607689678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b004a0 0x561313af6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.140+0000 7f5bebdd6740 1 --2- 172.21.15.84:0/607689678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561313afedb0 0x561313b158c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.140+0000 7f5bebdd6740 1 -- 172.21.15.84:0/607689678 >> 172.21.15.84:0/607689678 conn(0x561313ac78e0 msgr2=0x561313af9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:48.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.140+0000 7f5bebdd6740 1 -- 172.21.15.84:0/607689678 shutdown_connections 2024-05-22T23:07:48.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.140+0000 7f5bebdd6740 1 -- 172.21.15.84:0/607689678 wait complete. 2024-05-22T23:07:48.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.140+0000 7f5bebdd6740 1 -- start start 2024-05-22T23:07:48.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.140+0000 7f5bebdd6740 1 Processor -- start 2024-05-22T23:07:48.151 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.140+0000 7f5bebdd6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561313afedb0 0x561313b0d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.152 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.140+0000 7f5bebdd6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b004a0 0x561313b0dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.152 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.140+0000 7f5bebdd6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561313b09b30 0x561313b0e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.152 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.140+0000 7f5bebdd6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41ae4b0 con 0x561313b004a0 2024-05-22T23:07:48.152 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.140+0000 7f5bebdd6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41af920 con 0x561313b09b30 2024-05-22T23:07:48.152 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.140+0000 7f5bebdd6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41aeb80 con 0x561313afedb0 2024-05-22T23:07:48.152 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.140+0000 7f5bebdd6740 10 client.0 _ll_register_callbacks cb 0x7f5bdc0057d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:07:48.152 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.140+0000 7f5bda59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b004a0 0x561313b0dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.152 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.140+0000 7f5be2d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561313afedb0 0x561313b0d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.152 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.140+0000 7f5be2d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561313afedb0 0x561313b0d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:48490/0 (socket says 172.21.15.84:48490) 2024-05-22T23:07:48.152 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.140+0000 7f5bda59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b004a0 0x561313b0dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50130/0 (socket says 172.21.15.84:50130) 2024-05-22T23:07:48.152 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.140+0000 7f5be2d9d640 1 -- 172.21.15.84:0/920358563 learned_addr learned my addr 172.21.15.84:0/920358563 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:48.152 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.140+0000 7f5be359e640 1 --2- 172.21.15.84:0/920358563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561313b09b30 0x561313b0e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.152 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.141+0000 7f5be2d9d640 1 -- 172.21.15.84:0/920358563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561313b09b30 msgr2=0x561313b0e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:48.153 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.141+0000 7f5be2d9d640 1 --2- 172.21.15.84:0/920358563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561313b09b30 0x561313b0e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.153 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.141+0000 7f5be2d9d640 1 -- 172.21.15.84:0/920358563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b004a0 msgr2=0x561313b0dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:48.153 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.141+0000 7f5be2d9d640 1 --2- 172.21.15.84:0/920358563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b004a0 0x561313b0dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.153 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.141+0000 7f5be2d9d640 1 -- 172.21.15.84:0/920358563 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613137360f0 con 0x561313afedb0 2024-05-22T23:07:48.153 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.141+0000 7f5bda59c640 1 --2- 172.21.15.84:0/920358563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b004a0 0x561313b0dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:48.153 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.141+0000 7f5be359e640 1 --2- 172.21.15.84:0/920358563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561313b09b30 0x561313b0e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:48.153 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.141+0000 7f5be2d9d640 1 --2- 172.21.15.84:0/920358563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561313afedb0 0x561313b0d800 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4005fd0 tx=0x7f5bd4019820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4da92c0e04733c4e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:48.153 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.141+0000 7f5bca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-22T23:07:48.153 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.141+0000 7f5bca7fc640 1 -- 172.21.15.84:0/920358563 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd401e040 con 0x561313afedb0 2024-05-22T23:07:48.153 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.141+0000 7f5bebdd6740 1 -- 172.21.15.84:0/920358563 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bdc09be60 con 0x561313afedb0 2024-05-22T23:07:48.153 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.141+0000 7f5bca7fc640 1 -- 172.21.15.84:0/920358563 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f5bd400f560 con 0x561313afedb0 2024-05-22T23:07:48.153 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.141+0000 7f5bca7fc640 1 -- 172.21.15.84:0/920358563 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f5bd4038050 con 0x561313afedb0 2024-05-22T23:07:48.153 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.141+0000 7f5bebdd6740 10 client.25756 fetch_fsmap learned FSMap version 18 2024-05-22T23:07:48.154 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.141+0000 7f5bebdd6740 1 -- 172.21.15.84:0/920358563 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x561313a21fe0 con 0x561313afedb0 2024-05-22T23:07:48.154 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.141+0000 7f5bca7fc640 1 -- 172.21.15.84:0/920358563 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bdc09be60 con 0x561313afedb0 2024-05-22T23:07:48.154 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.141+0000 7f5bca7fc640 1 -- 172.21.15.84:0/920358563 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f5bd4019890 con 0x561313afedb0 2024-05-22T23:07:48.154 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.142+0000 7f5bebdd6740 10 client.25756 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:07:48.154 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.142+0000 7f5bebdd6740 10 client.25756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:07:48.154 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.142+0000 7f5bebdd6740 20 client.25756 populate_metadata read hostname 'smithi084' 2024-05-22T23:07:48.154 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.142+0000 7f5bebdd6740 1 --2- 172.21.15.84:0/920358563 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x561313b809e0 0x561313b82e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.154 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.142+0000 7f5bebdd6740 4 client.25756 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:07:48.154 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.142+0000 7f5bebdd6740 1 -- 172.21.15.84:0/920358563 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561313a6a830 con 0x561313b809e0 2024-05-22T23:07:48.154 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.142+0000 7f5bda59c640 1 --2- 172.21.15.84:0/920358563 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x561313b809e0 0x561313b82e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.154 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.142+0000 7f5bca7fc640 10 client.25756 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:48.154 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.142+0000 7f5bda59c640 1 --2- 172.21.15.84:0/920358563 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x561313b809e0 0x561313b82e00 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4df27a2d1c9dae8f server_cookie=9aa93852ccbf2b2e in_seq=0 out_seq=0 2024-05-22T23:07:48.154 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.144+0000 7f5bca7fc640 1 -- 172.21.15.84:0/920358563 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561313a6a830 con 0x561313b809e0 2024-05-22T23:07:48.155 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.144+0000 7f5bca7fc640 10 client.25756 handle_command_reply: tid=0 2024-05-22T23:07:48.155 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.144+0000 7f5bebdd6740 1 client.25756 shutdown 2024-05-22T23:07:48.155 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.144+0000 7f5bebdd6740 1 -- 172.21.15.84:0/920358563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561313afedb0 msgr2=0x561313b0d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:48.155 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.144+0000 7f5bebdd6740 1 --2- 172.21.15.84:0/920358563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561313afedb0 0x561313b0d800 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4005fd0 tx=0x7f5bd4019820 comp rx=0 tx=0).stop 2024-05-22T23:07:48.155 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.144+0000 7f5bebdd6740 1 -- 172.21.15.84:0/920358563 shutdown_connections 2024-05-22T23:07:48.155 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.144+0000 7f5bebdd6740 1 --2- 172.21.15.84:0/920358563 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x561313b809e0 0x561313b82e00 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.155 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.144+0000 7f5bebdd6740 1 --2- 172.21.15.84:0/920358563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561313b09b30 0x561313b0e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.155 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.144+0000 7f5bca7fc640 0 client.25756 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:48.155 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.144+0000 7f5bebdd6740 1 --2- 172.21.15.84:0/920358563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561313b004a0 0x561313b0dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.155 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.144+0000 7f5bebdd6740 1 --2- 172.21.15.84:0/920358563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561313afedb0 0x561313b0d800 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.155 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.144+0000 7f5bebdd6740 1 -- 172.21.15.84:0/920358563 >> 172.21.15.84:0/920358563 conn(0x561313ac78e0 msgr2=0x561313b07290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:48.155 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.144+0000 7f5bebdd6740 1 -- 172.21.15.84:0/920358563 shutdown_connections 2024-05-22T23:07:48.155 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.144+0000 7f5bebdd6740 1 -- 172.21.15.84:0/920358563 wait complete. 2024-05-22T23:07:48.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.144+0000 7f5bebdd6740 20 client.25756 trim_cache size 0 max 16384 2024-05-22T23:07:48.156 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:07:48.156 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "no active scrubs running", 2024-05-22T23:07:48.156 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": {} 2024-05-22T23:07:48.156 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:07:48.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.144+0000 7f5be90ca640 1 -- 172.21.15.84:0/2517633350 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f5bd00a2d00 msgr2=0x7f5bd00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:48.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.144+0000 7f5be90ca640 1 --2- 172.21.15.84:0/2517633350 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f5bd00a2d00 0x7f5bd00a51c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009230 tx=0x7f5bcc005d50 comp rx=0 tx=0).stop 2024-05-22T23:07:48.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.144+0000 7f5be90ca640 1 -- 172.21.15.84:0/2517633350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be410be30 msgr2=0x7f5be41011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:48.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.144+0000 7f5be90ca640 1 --2- 172.21.15.84:0/2517633350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be410be30 0x7f5be41011c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00eb10 tx=0x7f5bdc00c2d0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.145+0000 7f5be90ca640 1 -- 172.21.15.84:0/2517633350 shutdown_connections 2024-05-22T23:07:48.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.145+0000 7f5be90ca640 1 --2- 172.21.15.84:0/2517633350 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f5bd00a2d00 0x7f5bd00a51c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.145+0000 7f5be90ca640 1 --2- 172.21.15.84:0/2517633350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be410be30 0x7f5be41011c0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.145+0000 7f5be90ca640 1 --2- 172.21.15.84:0/2517633350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5be4103410 0x7f5be4100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.145+0000 7f5be90ca640 1 --2- 172.21.15.84:0/2517633350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5be4102af0 0x7f5be4100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.145+0000 7f5be90ca640 1 -- 172.21.15.84:0/2517633350 >> 172.21.15.84:0/2517633350 conn(0x7f5be4077870 msgr2=0x7f5be40fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:48.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.145+0000 7f5be90ca640 1 -- 172.21.15.84:0/2517633350 shutdown_connections 2024-05-22T23:07:48.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.145+0000 7f5be90ca640 1 -- 172.21.15.84:0/2517633350 wait complete. 2024-05-22T23:07:48.161 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-22T23:07:48.161 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:07:48.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.262+0000 7fc762722640 1 -- 172.21.15.84:0/2067055312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc75c106780 msgr2=0x7fc75c106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:48.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.262+0000 7fc762722640 1 --2- 172.21.15.84:0/2067055312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc75c106780 0x7fc75c106b60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009860 tx=0x7fc74c02a340 comp rx=0 tx=0).stop 2024-05-22T23:07:48.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.262+0000 7fc762722640 1 -- 172.21.15.84:0/2067055312 shutdown_connections 2024-05-22T23:07:48.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.262+0000 7fc762722640 1 --2- 172.21.15.84:0/2067055312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc75c101180 0x7fc75c10d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.262+0000 7fc762722640 1 --2- 172.21.15.84:0/2067055312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc75c1007c0 0x7fc75c100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.262+0000 7fc762722640 1 --2- 172.21.15.84:0/2067055312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc75c106780 0x7fc75c106b60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.262+0000 7fc762722640 1 -- 172.21.15.84:0/2067055312 >> 172.21.15.84:0/2067055312 conn(0x7fc75c0778d0 msgr2=0x7fc75c077ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:48.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.262+0000 7fc762722640 1 -- 172.21.15.84:0/2067055312 shutdown_connections 2024-05-22T23:07:48.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.262+0000 7fc762722640 1 -- 172.21.15.84:0/2067055312 wait complete. 2024-05-22T23:07:48.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.263+0000 7fc762722640 1 Processor -- start 2024-05-22T23:07:48.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.263+0000 7fc762722640 1 -- start start 2024-05-22T23:07:48.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.263+0000 7fc762722640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc75c1007c0 0x7fc75c198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.263+0000 7fc762722640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc75c101180 0x7fc75c1994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.263+0000 7fc762722640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc75c106780 0x7fc75c1a4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.263+0000 7fc762722640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c110b90 con 0x7fc75c106780 2024-05-22T23:07:48.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.263+0000 7fc762722640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c110a10 con 0x7fc75c101180 2024-05-22T23:07:48.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.263+0000 7fc762722640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c110d10 con 0x7fc75c1007c0 2024-05-22T23:07:48.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.263+0000 7fc760cbf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc75c106780 0x7fc75c1a4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.264+0000 7fc760cbf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc75c106780 0x7fc75c1a4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50134/0 (socket says 172.21.15.84:50134) 2024-05-22T23:07:48.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.264+0000 7fc75b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc75c101180 0x7fc75c1994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.264+0000 7fc760cbf640 1 -- 172.21.15.84:0/3237340812 learned_addr learned my addr 172.21.15.84:0/3237340812 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:48.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.264+0000 7fc75bfff640 1 --2- 172.21.15.84:0/3237340812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc75c1007c0 0x7fc75c198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.264+0000 7fc760cbf640 1 -- 172.21.15.84:0/3237340812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc75c1007c0 msgr2=0x7fc75c198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:48.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.264+0000 7fc760cbf640 1 --2- 172.21.15.84:0/3237340812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc75c1007c0 0x7fc75c198fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.264+0000 7fc760cbf640 1 -- 172.21.15.84:0/3237340812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc75c101180 msgr2=0x7fc75c1994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:48.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.264+0000 7fc760cbf640 1 --2- 172.21.15.84:0/3237340812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc75c101180 0x7fc75c1994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.264+0000 7fc760cbf640 1 -- 172.21.15.84:0/3237340812 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75c1a4570 con 0x7fc75c106780 2024-05-22T23:07:48.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.264+0000 7fc75b7fe640 1 --2- 172.21.15.84:0/3237340812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc75c101180 0x7fc75c1994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:48.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.264+0000 7fc760cbf640 1 --2- 172.21.15.84:0/3237340812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc75c106780 0x7fc75c1a4030 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc75400ea60 tx=0x7fc75400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:48.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.264+0000 7fc7597fa640 1 -- 172.21.15.84:0/3237340812 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75400ce40 con 0x7fc75c106780 2024-05-22T23:07:48.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.264+0000 7fc762722640 1 -- 172.21.15.84:0/3237340812 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc75c1a4740 con 0x7fc75c106780 2024-05-22T23:07:48.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.264+0000 7fc7597fa640 1 -- 172.21.15.84:0/3237340812 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc754005020 con 0x7fc75c106780 2024-05-22T23:07:48.273 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.264+0000 7fc762722640 1 -- 172.21.15.84:0/3237340812 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc75c1a4ce0 con 0x7fc75c106780 2024-05-22T23:07:48.273 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.264+0000 7fc7597fa640 1 -- 172.21.15.84:0/3237340812 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc754018670 con 0x7fc75c106780 2024-05-22T23:07:48.273 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.265+0000 7fc762722640 1 -- 172.21.15.84:0/3237340812 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc75c068ae0 con 0x7fc75c106780 2024-05-22T23:07:48.273 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.266+0000 7fc7597fa640 1 -- 172.21.15.84:0/3237340812 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc7540051d0 con 0x7fc75c106780 2024-05-22T23:07:48.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.267+0000 7fc7597fa640 1 --2- 172.21.15.84:0/3237340812 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fc7440a2d00 0x7fc7440a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.267+0000 7fc75bfff640 1 --2- 172.21.15.84:0/3237340812 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fc7440a2d00 0x7fc7440a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.267+0000 7fc7597fa640 1 -- 172.21.15.84:0/3237340812 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fc7540cf990 con 0x7fc75c106780 2024-05-22T23:07:48.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.267+0000 7fc75bfff640 1 --2- 172.21.15.84:0/3237340812 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fc7440a2d00 0x7fc7440a51c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc74c02a930 tx=0x7fc74c0058d0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:48.277 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.271+0000 7fc7597fa640 1 -- 172.21.15.84:0/3237340812 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc75409c5b0 con 0x7fc75c106780 2024-05-22T23:07:48.438 INFO:tasks.workunit.client.0.smithi084.stdout: 1 780468 15.11 MB/sec execute 154 sec latency 12.902 ms 2024-05-22T23:07:48.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.509+0000 7fc762722640 1 -- 172.21.15.84:0/3237340812 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc75c108ce0 con 0x7fc75c106780 2024-05-22T23:07:48.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.510+0000 7fc7597fa640 1 -- 172.21.15.84:0/3237340812 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7fc75409beb0 con 0x7fc75c106780 2024-05-22T23:07:48.516 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:07:48.516 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:07:48.516 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:07:48.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.512+0000 7fc762722640 1 -- 172.21.15.84:0/3237340812 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fc7440a2d00 msgr2=0x7fc7440a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:48.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.512+0000 7fc762722640 1 --2- 172.21.15.84:0/3237340812 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fc7440a2d00 0x7fc7440a51c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc74c02a930 tx=0x7fc74c0058d0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.512+0000 7fc762722640 1 -- 172.21.15.84:0/3237340812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc75c106780 msgr2=0x7fc75c1a4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:48.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.512+0000 7fc762722640 1 --2- 172.21.15.84:0/3237340812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc75c106780 0x7fc75c1a4030 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc75400ea60 tx=0x7fc75400ef20 comp rx=0 tx=0).stop 2024-05-22T23:07:48.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.512+0000 7fc762722640 1 -- 172.21.15.84:0/3237340812 shutdown_connections 2024-05-22T23:07:48.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.512+0000 7fc762722640 1 --2- 172.21.15.84:0/3237340812 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fc7440a2d00 0x7fc7440a51c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.512+0000 7fc762722640 1 --2- 172.21.15.84:0/3237340812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc75c106780 0x7fc75c1a4030 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.512+0000 7fc762722640 1 --2- 172.21.15.84:0/3237340812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc75c101180 0x7fc75c1994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.512+0000 7fc762722640 1 --2- 172.21.15.84:0/3237340812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc75c1007c0 0x7fc75c198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.512+0000 7fc762722640 1 -- 172.21.15.84:0/3237340812 >> 172.21.15.84:0/3237340812 conn(0x7fc75c0778d0 msgr2=0x7fc75c103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:48.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.512+0000 7fc762722640 1 -- 172.21.15.84:0/3237340812 shutdown_connections 2024-05-22T23:07:48.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.512+0000 7fc762722640 1 -- 172.21.15.84:0/3237340812 wait complete. 2024-05-22T23:07:48.529 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-22T23:07:48.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.635+0000 7fd412f5a640 1 -- 172.21.15.84:0/3160886219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd40c106750 msgr2=0x7fd40c106b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:48.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.635+0000 7fd412f5a640 1 --2- 172.21.15.84:0/3160886219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd40c106750 0x7fd40c106b30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd408009930 tx=0x7fd40802a420 comp rx=0 tx=0).stop 2024-05-22T23:07:48.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.635+0000 7fd412f5a640 1 -- 172.21.15.84:0/3160886219 shutdown_connections 2024-05-22T23:07:48.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.635+0000 7fd412f5a640 1 --2- 172.21.15.84:0/3160886219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd40c101100 0x7fd40c10d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.635+0000 7fd412f5a640 1 --2- 172.21.15.84:0/3160886219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd40c100740 0x7fd40c100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.635+0000 7fd412f5a640 1 --2- 172.21.15.84:0/3160886219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd40c106750 0x7fd40c106b30 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.635+0000 7fd412f5a640 1 -- 172.21.15.84:0/3160886219 >> 172.21.15.84:0/3160886219 conn(0x7fd40c077870 msgr2=0x7fd40c077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:48.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.635+0000 7fd412f5a640 1 -- 172.21.15.84:0/3160886219 shutdown_connections 2024-05-22T23:07:48.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.635+0000 7fd412f5a640 1 -- 172.21.15.84:0/3160886219 wait complete. 2024-05-22T23:07:48.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.636+0000 7fd412f5a640 1 Processor -- start 2024-05-22T23:07:48.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.636+0000 7fd412f5a640 1 -- start start 2024-05-22T23:07:48.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.636+0000 7fd412f5a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd40c100740 0x7fd40c198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.636+0000 7fd412f5a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd40c101100 0x7fd40c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.636+0000 7fd412f5a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd40c106750 0x7fd40c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.636+0000 7fd412f5a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c110b20 con 0x7fd40c101100 2024-05-22T23:07:48.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.636+0000 7fd412f5a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c1109a0 con 0x7fd40c106750 2024-05-22T23:07:48.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.636+0000 7fd412f5a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c110ca0 con 0x7fd40c100740 2024-05-22T23:07:48.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.636+0000 7fd403fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd40c101100 0x7fd40c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.636+0000 7fd403fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd40c101100 0x7fd40c198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50156/0 (socket says 172.21.15.84:50156) 2024-05-22T23:07:48.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.636+0000 7fd403fff640 1 -- 172.21.15.84:0/1422749566 learned_addr learned my addr 172.21.15.84:0/1422749566 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:48.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.636+0000 7fd4114f7640 1 --2- 172.21.15.84:0/1422749566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd40c106750 0x7fd40c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.636+0000 7fd410cf6640 1 --2- 172.21.15.84:0/1422749566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd40c100740 0x7fd40c198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.636+0000 7fd403fff640 1 -- 172.21.15.84:0/1422749566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd40c100740 msgr2=0x7fd40c198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:48.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.636+0000 7fd403fff640 1 --2- 172.21.15.84:0/1422749566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd40c100740 0x7fd40c198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.636+0000 7fd403fff640 1 -- 172.21.15.84:0/1422749566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd40c106750 msgr2=0x7fd40c19caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:48.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.636+0000 7fd403fff640 1 --2- 172.21.15.84:0/1422749566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd40c106750 0x7fd40c19caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.636+0000 7fd403fff640 1 -- 172.21.15.84:0/1422749566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40c19d1d0 con 0x7fd40c101100 2024-05-22T23:07:48.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.637+0000 7fd403fff640 1 --2- 172.21.15.84:0/1422749566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd40c101100 0x7fd40c198770 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00c890 tx=0x7fd3fc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:48.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.637+0000 7fd401ffb640 1 -- 172.21.15.84:0/1422749566 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc004220 con 0x7fd40c101100 2024-05-22T23:07:48.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.637+0000 7fd401ffb640 1 -- 172.21.15.84:0/1422749566 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3fc0043c0 con 0x7fd40c101100 2024-05-22T23:07:48.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.637+0000 7fd412f5a640 1 -- 172.21.15.84:0/1422749566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd40c19d4c0 con 0x7fd40c101100 2024-05-22T23:07:48.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.637+0000 7fd401ffb640 1 -- 172.21.15.84:0/1422749566 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc0049e0 con 0x7fd40c101100 2024-05-22T23:07:48.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.637+0000 7fd412f5a640 1 -- 172.21.15.84:0/1422749566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd40c1a4d50 con 0x7fd40c101100 2024-05-22T23:07:48.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.639+0000 7fd401ffb640 1 -- 172.21.15.84:0/1422749566 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd3fc0079b0 con 0x7fd40c101100 2024-05-22T23:07:48.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.639+0000 7fd415c66740 1 Processor -- start 2024-05-22T23:07:48.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.640+0000 7fd415c66740 1 -- start start 2024-05-22T23:07:48.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.640+0000 7fd415c66740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558207513f80 0x558207514360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.640+0000 7fd415c66740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582075148a0 0x55820751ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.640+0000 7fd415c66740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55820751ef70 0x558207521360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.640+0000 7fd415c66740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558207478db0 con 0x5582075148a0 2024-05-22T23:07:48.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.640+0000 7fd401ffb640 1 --2- 172.21.15.84:0/1422749566 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd3f40a2d00 0x7fd3f40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.640+0000 7fd415c66740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558207417400 con 0x55820751ef70 2024-05-22T23:07:48.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.640+0000 7fd415c66740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558207474d70 con 0x558207513f80 2024-05-22T23:07:48.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.640+0000 7fd410cf6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582075148a0 0x55820751ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.640+0000 7fd410cf6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582075148a0 0x55820751ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50168/0 (socket says 172.21.15.84:50168) 2024-05-22T23:07:48.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.640+0000 7fd410cf6640 1 -- 172.21.15.84:0/345896069 learned_addr learned my addr 172.21.15.84:0/345896069 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:48.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.640+0000 7fd4114f7640 1 --2- 172.21.15.84:0/1422749566 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd3f40a2d00 0x7fd3f40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.640+0000 7fd401ffb640 1 -- 172.21.15.84:0/1422749566 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fd3fc0cf400 con 0x7fd40c101100 2024-05-22T23:07:48.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.640+0000 7fd403fff640 1 --2- 172.21.15.84:0/345896069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55820751ef70 0x558207521360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.640+0000 7fd4114f7640 1 --2- 172.21.15.84:0/345896069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558207513f80 0x558207514360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.640+0000 7fd410cf6640 1 -- 172.21.15.84:0/345896069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558207513f80 msgr2=0x558207514360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:48.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.640+0000 7fd410cf6640 1 --2- 172.21.15.84:0/345896069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558207513f80 0x558207514360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.640+0000 7fd410cf6640 1 -- 172.21.15.84:0/345896069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55820751ef70 msgr2=0x558207521360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:48.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.640+0000 7fd410cf6640 1 --2- 172.21.15.84:0/345896069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55820751ef70 0x558207521360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.640+0000 7fd410cf6640 1 -- 172.21.15.84:0/345896069 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582074092f0 con 0x5582075148a0 2024-05-22T23:07:48.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.640+0000 7fd403fff640 1 --2- 172.21.15.84:0/345896069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55820751ef70 0x558207521360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:48.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.640+0000 7fd4114f7640 1 --2- 172.21.15.84:0/1422749566 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd3f40a2d00 0x7fd3f40a51c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd404005fd0 tx=0x7fd404009e80 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:48.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.640+0000 7fd410cf6640 1 --2- 172.21.15.84:0/345896069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582075148a0 0x55820751ea30 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd408009930 tx=0x7fd408005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a78e6d0c52a0f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:48.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.641+0000 7fd3f2ffd640 1 -- 172.21.15.84:0/345896069 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408040070 con 0x5582075148a0 2024-05-22T23:07:48.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.641+0000 7fd3f2ffd640 1 -- 172.21.15.84:0/345896069 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4080319f0 con 0x5582075148a0 2024-05-22T23:07:48.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.641+0000 7fd3f2ffd640 1 -- 172.21.15.84:0/345896069 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408035040 con 0x5582075148a0 2024-05-22T23:07:48.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.641+0000 7fd415c66740 1 -- 172.21.15.84:0/345896069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582075148a0 msgr2=0x55820751ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:48.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.641+0000 7fd415c66740 1 --2- 172.21.15.84:0/345896069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582075148a0 0x55820751ea30 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd408009930 tx=0x7fd408005e80 comp rx=0 tx=0).stop 2024-05-22T23:07:48.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.641+0000 7fd415c66740 1 -- 172.21.15.84:0/345896069 shutdown_connections 2024-05-22T23:07:48.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.641+0000 7fd415c66740 1 --2- 172.21.15.84:0/345896069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55820751ef70 0x558207521360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.641+0000 7fd415c66740 1 --2- 172.21.15.84:0/345896069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582075148a0 0x55820751ea30 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.641+0000 7fd415c66740 1 --2- 172.21.15.84:0/345896069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558207513f80 0x558207514360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.641+0000 7fd415c66740 1 -- 172.21.15.84:0/345896069 >> 172.21.15.84:0/345896069 conn(0x5582075102a0 msgr2=0x558207510680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:48.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.641+0000 7fd415c66740 1 -- 172.21.15.84:0/345896069 shutdown_connections 2024-05-22T23:07:48.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.641+0000 7fd415c66740 1 -- 172.21.15.84:0/345896069 wait complete. 2024-05-22T23:07:48.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.642+0000 7fd415c66740 1 -- start start 2024-05-22T23:07:48.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.642+0000 7fd415c66740 1 Processor -- start 2024-05-22T23:07:48.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.642+0000 7fd415c66740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558207513f80 0x55820751be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.642+0000 7fd415c66740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582075148a0 0x55820751c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.642+0000 7fd415c66740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55820751ef70 0x55820751c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.642+0000 7fd415c66740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558207521d00 con 0x55820751ef70 2024-05-22T23:07:48.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.642+0000 7fd415c66740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5582074332e0 con 0x5582075148a0 2024-05-22T23:07:48.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.642+0000 7fd415c66740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558207521e80 con 0x558207513f80 2024-05-22T23:07:48.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.642+0000 7fd415c66740 10 client.0 _ll_register_callbacks cb 0x55820743ca70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:07:48.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.642+0000 7fd4114f7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55820751ef70 0x55820751c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.642+0000 7fd4114f7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55820751ef70 0x55820751c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50180/0 (socket says 172.21.15.84:50180) 2024-05-22T23:07:48.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.642+0000 7fd4114f7640 1 -- 172.21.15.84:0/2376592716 learned_addr learned my addr 172.21.15.84:0/2376592716 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:48.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.642+0000 7fd403fff640 1 --2- 172.21.15.84:0/2376592716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582075148a0 0x55820751c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.642+0000 7fd410cf6640 1 --2- 172.21.15.84:0/2376592716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558207513f80 0x55820751be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.643+0000 7fd403fff640 1 -- 172.21.15.84:0/2376592716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558207513f80 msgr2=0x55820751be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:48.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.643+0000 7fd403fff640 1 --2- 172.21.15.84:0/2376592716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558207513f80 0x55820751be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.643+0000 7fd403fff640 1 -- 172.21.15.84:0/2376592716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55820751ef70 msgr2=0x55820751c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:48.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.643+0000 7fd403fff640 1 --2- 172.21.15.84:0/2376592716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55820751ef70 0x55820751c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.643+0000 7fd403fff640 1 -- 172.21.15.84:0/2376592716 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55820718c3f0 con 0x5582075148a0 2024-05-22T23:07:48.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.643+0000 7fd4114f7640 1 --2- 172.21.15.84:0/2376592716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55820751ef70 0x55820751c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:48.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.643+0000 7fd403fff640 1 --2- 172.21.15.84:0/2376592716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582075148a0 0x55820751c3d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0ccb50 tx=0x7fd3fc09b640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fca399a3beed2ab0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:48.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.643+0000 7fd3ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:07:48.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.643+0000 7fd3ebfff640 1 -- 172.21.15.84:0/2376592716 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc01d040 con 0x5582075148a0 2024-05-22T23:07:48.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.643+0000 7fd415c66740 1 -- 172.21.15.84:0/2376592716 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4080319f0 con 0x5582075148a0 2024-05-22T23:07:48.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.643+0000 7fd3ebfff640 1 -- 172.21.15.84:0/2376592716 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fd3fc0a4100 con 0x5582075148a0 2024-05-22T23:07:48.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.643+0000 7fd3ebfff640 1 -- 172.21.15.84:0/2376592716 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fd3fc0a4590 con 0x5582075148a0 2024-05-22T23:07:48.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.643+0000 7fd415c66740 10 client.25934 fetch_fsmap learned FSMap version 18 2024-05-22T23:07:48.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.643+0000 7fd415c66740 1 -- 172.21.15.84:0/2376592716 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5582072567d0 con 0x5582075148a0 2024-05-22T23:07:48.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.643+0000 7fd3ebfff640 1 -- 172.21.15.84:0/2376592716 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4080319f0 con 0x5582075148a0 2024-05-22T23:07:48.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.643+0000 7fd3ebfff640 1 -- 172.21.15.84:0/2376592716 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fd3fc09ae20 con 0x5582075148a0 2024-05-22T23:07:48.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.643+0000 7fd415c66740 10 client.25934 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:07:48.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.643+0000 7fd415c66740 10 client.25934 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:07:48.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.643+0000 7fd415c66740 20 client.25934 populate_metadata read hostname 'smithi084' 2024-05-22T23:07:48.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.643+0000 7fd415c66740 1 --2- 172.21.15.84:0/2376592716 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55820752f180 0x558207531590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.644+0000 7fd415c66740 4 client.25934 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:07:48.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.644+0000 7fd415c66740 1 -- 172.21.15.84:0/2376592716 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582074864e0 con 0x55820752f180 2024-05-22T23:07:48.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.644+0000 7fd410cf6640 1 --2- 172.21.15.84:0/2376592716 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55820752f180 0x558207531590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.644+0000 7fd3ebfff640 10 client.25934 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:48.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.644+0000 7fd410cf6640 1 --2- 172.21.15.84:0/2376592716 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55820752f180 0x558207531590 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9bbd171cc7aaca9 server_cookie=35fd385aaf0995fc in_seq=0 out_seq=0 2024-05-22T23:07:48.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.645+0000 7fd3ebfff640 1 -- 172.21.15.84:0/2376592716 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5582074864e0 con 0x55820752f180 2024-05-22T23:07:48.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.645+0000 7fd3ebfff640 10 client.25934 handle_command_reply: tid=0 2024-05-22T23:07:48.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.645+0000 7fd415c66740 1 client.25934 shutdown 2024-05-22T23:07:48.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.645+0000 7fd415c66740 1 -- 172.21.15.84:0/2376592716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582075148a0 msgr2=0x55820751c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:48.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.645+0000 7fd415c66740 1 --2- 172.21.15.84:0/2376592716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582075148a0 0x55820751c3d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0ccb50 tx=0x7fd3fc09b640 comp rx=0 tx=0).stop 2024-05-22T23:07:48.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.645+0000 7fd415c66740 1 -- 172.21.15.84:0/2376592716 shutdown_connections 2024-05-22T23:07:48.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.645+0000 7fd415c66740 1 --2- 172.21.15.84:0/2376592716 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55820752f180 0x558207531590 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.645+0000 7fd415c66740 1 --2- 172.21.15.84:0/2376592716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55820751ef70 0x55820751c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.645+0000 7fd3ebfff640 0 client.25934 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:48.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.645+0000 7fd415c66740 1 --2- 172.21.15.84:0/2376592716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582075148a0 0x55820751c3d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.645+0000 7fd415c66740 1 --2- 172.21.15.84:0/2376592716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558207513f80 0x55820751be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.645+0000 7fd415c66740 1 -- 172.21.15.84:0/2376592716 >> 172.21.15.84:0/2376592716 conn(0x5582075102a0 msgr2=0x5582075160a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:48.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.645+0000 7fd415c66740 1 -- 172.21.15.84:0/2376592716 shutdown_connections 2024-05-22T23:07:48.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.645+0000 7fd415c66740 1 -- 172.21.15.84:0/2376592716 wait complete. 2024-05-22T23:07:48.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.645+0000 7fd415c66740 20 client.25934 trim_cache size 0 max 16384 2024-05-22T23:07:48.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.660+0000 7fd415c66740 1 Processor -- start 2024-05-22T23:07:48.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.660+0000 7fd415c66740 1 -- start start 2024-05-22T23:07:48.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.661+0000 7fd415c66740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558207513f80 0x55820752abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.661+0000 7fd415c66740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582075148a0 0x558207516720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.661+0000 7fd415c66740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55820751ef70 0x558207536e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.661+0000 7fd415c66740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c1ab5a0 con 0x55820751ef70 2024-05-22T23:07:48.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.661+0000 7fd415c66740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c1ade20 con 0x558207513f80 2024-05-22T23:07:48.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.661+0000 7fd415c66740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c1abc10 con 0x5582075148a0 2024-05-22T23:07:48.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.661+0000 7fd4114f7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55820751ef70 0x558207536e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.661+0000 7fd410cf6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558207513f80 0x55820752abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.661+0000 7fd4114f7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55820751ef70 0x558207536e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50182/0 (socket says 172.21.15.84:50182) 2024-05-22T23:07:48.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.661+0000 7fd4114f7640 1 -- 172.21.15.84:0/1358784068 learned_addr learned my addr 172.21.15.84:0/1358784068 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:48.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.661+0000 7fd403fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582075148a0 0x558207516720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.661+0000 7fd410cf6640 1 -- 172.21.15.84:0/1358784068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582075148a0 msgr2=0x558207516720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:48.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.661+0000 7fd410cf6640 1 --2- 172.21.15.84:0/1358784068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582075148a0 0x558207516720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.661+0000 7fd410cf6640 1 -- 172.21.15.84:0/1358784068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55820751ef70 msgr2=0x558207536e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:48.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.661+0000 7fd410cf6640 1 --2- 172.21.15.84:0/1358784068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55820751ef70 0x558207536e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.661+0000 7fd410cf6640 1 -- 172.21.15.84:0/1358784068 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582075100d0 con 0x558207513f80 2024-05-22T23:07:48.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.661+0000 7fd403fff640 1 --2- 172.21.15.84:0/1358784068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582075148a0 0x558207516720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:48.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.661+0000 7fd4114f7640 1 --2- 172.21.15.84:0/1358784068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55820751ef70 0x558207536e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:48.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.661+0000 7fd410cf6640 1 --2- 172.21.15.84:0/1358784068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558207513f80 0x55820752abc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd408006010 tx=0x7fd40802aae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=235b82538c250e34 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:48.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.662+0000 7fd3f27fc640 1 -- 172.21.15.84:0/1358784068 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408040050 con 0x558207513f80 2024-05-22T23:07:48.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.662+0000 7fd3f27fc640 1 -- 172.21.15.84:0/1358784068 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd408005520 con 0x558207513f80 2024-05-22T23:07:48.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.662+0000 7fd3f27fc640 1 -- 172.21.15.84:0/1358784068 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408035040 con 0x558207513f80 2024-05-22T23:07:48.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.662+0000 7fd415c66740 1 -- 172.21.15.84:0/1358784068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558207513f80 msgr2=0x55820752abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:48.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.662+0000 7fd415c66740 1 --2- 172.21.15.84:0/1358784068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558207513f80 0x55820752abc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd408006010 tx=0x7fd40802aae0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.662+0000 7fd415c66740 1 -- 172.21.15.84:0/1358784068 shutdown_connections 2024-05-22T23:07:48.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.662+0000 7fd415c66740 1 --2- 172.21.15.84:0/1358784068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55820751ef70 0x558207536e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.662+0000 7fd415c66740 1 --2- 172.21.15.84:0/1358784068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582075148a0 0x558207516720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.662+0000 7fd415c66740 1 --2- 172.21.15.84:0/1358784068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558207513f80 0x55820752abc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.662+0000 7fd415c66740 1 -- 172.21.15.84:0/1358784068 >> 172.21.15.84:0/1358784068 conn(0x5582075102a0 msgr2=0x55820751c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:48.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.662+0000 7fd415c66740 1 -- 172.21.15.84:0/1358784068 shutdown_connections 2024-05-22T23:07:48.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.662+0000 7fd415c66740 1 -- 172.21.15.84:0/1358784068 wait complete. 2024-05-22T23:07:48.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.663+0000 7fd415c66740 1 -- start start 2024-05-22T23:07:48.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.663+0000 7fd415c66740 1 Processor -- start 2024-05-22T23:07:48.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.663+0000 7fd415c66740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558207513f80 0x558207518c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.663+0000 7fd415c66740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582075148a0 0x558207519150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.663+0000 7fd415c66740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55820751ef70 0x558207519690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.663+0000 7fd415c66740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c1ac2e0 con 0x5582075148a0 2024-05-22T23:07:48.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.663+0000 7fd415c66740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c1ad750 con 0x55820751ef70 2024-05-22T23:07:48.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.663+0000 7fd415c66740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c1ac9b0 con 0x558207513f80 2024-05-22T23:07:48.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.663+0000 7fd415c66740 10 client.0 _ll_register_callbacks cb 0x7fd3fc00a580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:07:48.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.664+0000 7fd403fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582075148a0 0x558207519150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.664+0000 7fd403fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582075148a0 0x558207519150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50194/0 (socket says 172.21.15.84:50194) 2024-05-22T23:07:48.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.664+0000 7fd403fff640 1 -- 172.21.15.84:0/3590053116 learned_addr learned my addr 172.21.15.84:0/3590053116 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:48.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.664+0000 7fd4114f7640 1 --2- 172.21.15.84:0/3590053116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55820751ef70 0x558207519690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.664+0000 7fd410cf6640 1 --2- 172.21.15.84:0/3590053116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558207513f80 0x558207518c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.664+0000 7fd403fff640 1 -- 172.21.15.84:0/3590053116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558207513f80 msgr2=0x558207518c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:48.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.664+0000 7fd403fff640 1 --2- 172.21.15.84:0/3590053116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558207513f80 0x558207518c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.664+0000 7fd403fff640 1 -- 172.21.15.84:0/3590053116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55820751ef70 msgr2=0x558207519690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:48.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.664+0000 7fd403fff640 1 --2- 172.21.15.84:0/3590053116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55820751ef70 0x558207519690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.664+0000 7fd403fff640 1 -- 172.21.15.84:0/3590053116 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558207594780 con 0x5582075148a0 2024-05-22T23:07:48.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.664+0000 7fd403fff640 1 --2- 172.21.15.84:0/3590053116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582075148a0 0x558207519150 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc09b960 tx=0x7fd3fc007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb541abd10023528 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:48.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.664+0000 7fd3ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:07:48.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.664+0000 7fd3ebfff640 1 -- 172.21.15.84:0/3590053116 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc01d070 con 0x5582075148a0 2024-05-22T23:07:48.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.664+0000 7fd3ebfff640 1 -- 172.21.15.84:0/3590053116 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7fd3fc0860b0 con 0x5582075148a0 2024-05-22T23:07:48.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.664+0000 7fd415c66740 1 -- 172.21.15.84:0/3590053116 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd408005520 con 0x5582075148a0 2024-05-22T23:07:48.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.664+0000 7fd3ebfff640 1 -- 172.21.15.84:0/3590053116 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fd3fc0b9040 con 0x5582075148a0 2024-05-22T23:07:48.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.664+0000 7fd415c66740 10 client.16368 fetch_fsmap learned FSMap version 18 2024-05-22T23:07:48.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.664+0000 7fd415c66740 1 -- 172.21.15.84:0/3590053116 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x558207595a40 con 0x5582075148a0 2024-05-22T23:07:48.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.665+0000 7fd3ebfff640 1 -- 172.21.15.84:0/3590053116 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd408005520 con 0x5582075148a0 2024-05-22T23:07:48.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.665+0000 7fd3ebfff640 1 -- 172.21.15.84:0/3590053116 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fd3fc097720 con 0x5582075148a0 2024-05-22T23:07:48.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.665+0000 7fd415c66740 10 client.16368 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:07:48.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.665+0000 7fd415c66740 10 client.16368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:07:48.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.665+0000 7fd415c66740 20 client.16368 populate_metadata read hostname 'smithi084' 2024-05-22T23:07:48.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.665+0000 7fd415c66740 1 --2- 172.21.15.84:0/3590053116 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558207596130 0x558207598550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:48.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.665+0000 7fd415c66740 4 client.16368 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-22T23:07:48.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.665+0000 7fd415c66740 1 -- 172.21.15.84:0/3590053116 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558207480830 con 0x558207596130 2024-05-22T23:07:48.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.667+0000 7fd410cf6640 1 --2- 172.21.15.84:0/3590053116 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558207596130 0x558207598550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:48.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.671+0000 7fd410cf6640 1 --2- 172.21.15.84:0/3590053116 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558207596130 0x558207598550 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6032b03cbed791e8 server_cookie=5811f92ca7b56ef2 in_seq=0 out_seq=0 2024-05-22T23:07:48.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.672+0000 7fd3ebfff640 10 client.16368 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:48.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.672+0000 7fd3ebfff640 1 -- 172.21.15.84:0/3590053116 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558207480830 con 0x558207596130 2024-05-22T23:07:48.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.672+0000 7fd3ebfff640 10 client.16368 handle_command_reply: tid=0 2024-05-22T23:07:48.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.673+0000 7fd415c66740 1 client.16368 shutdown 2024-05-22T23:07:48.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.673+0000 7fd415c66740 1 -- 172.21.15.84:0/3590053116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582075148a0 msgr2=0x558207519150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:48.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.673+0000 7fd415c66740 1 --2- 172.21.15.84:0/3590053116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582075148a0 0x558207519150 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc09b960 tx=0x7fd3fc007eb0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.673+0000 7fd415c66740 1 -- 172.21.15.84:0/3590053116 shutdown_connections 2024-05-22T23:07:48.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.673+0000 7fd415c66740 1 --2- 172.21.15.84:0/3590053116 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558207596130 0x558207598550 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.673+0000 7fd415c66740 1 --2- 172.21.15.84:0/3590053116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55820751ef70 0x558207519690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.673+0000 7fd415c66740 1 --2- 172.21.15.84:0/3590053116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582075148a0 0x558207519150 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.673+0000 7fd415c66740 1 --2- 172.21.15.84:0/3590053116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558207513f80 0x558207518c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.673+0000 7fd415c66740 1 -- 172.21.15.84:0/3590053116 >> 172.21.15.84:0/3590053116 conn(0x5582075102a0 msgr2=0x55820750ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:48.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.673+0000 7fd3ebfff640 0 client.16368 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:48.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.673+0000 7fd415c66740 1 -- 172.21.15.84:0/3590053116 shutdown_connections 2024-05-22T23:07:48.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.673+0000 7fd415c66740 1 -- 172.21.15.84:0/3590053116 wait complete. 2024-05-22T23:07:48.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.673+0000 7fd415c66740 20 client.16368 trim_cache size 0 max 16384 2024-05-22T23:07:48.680 INFO:teuthology.orchestra.run.smithi084.stdout:[] 2024-05-22T23:07:48.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.674+0000 7fd412f5a640 1 -- 172.21.15.84:0/1422749566 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd3f40a2d00 msgr2=0x7fd3f40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:48.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.674+0000 7fd412f5a640 1 --2- 172.21.15.84:0/1422749566 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd3f40a2d00 0x7fd3f40a51c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd404005fd0 tx=0x7fd404009e80 comp rx=0 tx=0).stop 2024-05-22T23:07:48.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.674+0000 7fd412f5a640 1 -- 172.21.15.84:0/1422749566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd40c101100 msgr2=0x7fd40c198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:48.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.674+0000 7fd412f5a640 1 --2- 172.21.15.84:0/1422749566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd40c101100 0x7fd40c198770 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00c890 tx=0x7fd3fc00cd50 comp rx=0 tx=0).stop 2024-05-22T23:07:48.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.674+0000 7fd412f5a640 1 -- 172.21.15.84:0/1422749566 shutdown_connections 2024-05-22T23:07:48.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.674+0000 7fd412f5a640 1 --2- 172.21.15.84:0/1422749566 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd3f40a2d00 0x7fd3f40a51c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.674+0000 7fd412f5a640 1 --2- 172.21.15.84:0/1422749566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd40c106750 0x7fd40c19caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.674+0000 7fd412f5a640 1 --2- 172.21.15.84:0/1422749566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd40c101100 0x7fd40c198770 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.674+0000 7fd412f5a640 1 --2- 172.21.15.84:0/1422749566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd40c100740 0x7fd40c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:48.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.674+0000 7fd412f5a640 1 -- 172.21.15.84:0/1422749566 >> 172.21.15.84:0/1422749566 conn(0x7fd40c077870 msgr2=0x7fd40c068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:48.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.674+0000 7fd412f5a640 1 -- 172.21.15.84:0/1422749566 shutdown_connections 2024-05-22T23:07:48.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:48.674+0000 7fd412f5a640 1 -- 172.21.15.84:0/1422749566 wait complete. 2024-05-22T23:07:48.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:48 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3237340812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:07:48.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:48 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3237340812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:07:49.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:48 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3237340812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:07:49.438 INFO:tasks.workunit.client.0.smithi084.stdout: 1 783054 15.10 MB/sec execute 155 sec latency 15.416 ms 2024-05-22T23:07:49.692 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-22T23:07:49.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:07:49.693 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-22T23:07:49.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.802+0000 7f4805015640 1 -- 172.21.15.84:0/3492192989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f480010be30 msgr2=0x7f480010e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:49.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.802+0000 7f4805015640 1 --2- 172.21.15.84:0/3492192989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f480010be30 0x7f480010e2d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009ca0 tx=0x7f47f802a9e0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.802+0000 7f4805015640 1 -- 172.21.15.84:0/3492192989 shutdown_connections 2024-05-22T23:07:49.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.802+0000 7f4805015640 1 --2- 172.21.15.84:0/3492192989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f480010be30 0x7f480010e2d0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.802+0000 7f4805015640 1 --2- 172.21.15.84:0/3492192989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4800103410 0x7f480010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.802+0000 7f4805015640 1 --2- 172.21.15.84:0/3492192989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4800102af0 0x7f4800102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.802+0000 7f4805015640 1 -- 172.21.15.84:0/3492192989 >> 172.21.15.84:0/3492192989 conn(0x7f4800077870 msgr2=0x7f4800077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:49.810 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.802+0000 7f4805015640 1 -- 172.21.15.84:0/3492192989 shutdown_connections 2024-05-22T23:07:49.810 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.802+0000 7f4805015640 1 -- 172.21.15.84:0/3492192989 wait complete. 2024-05-22T23:07:49.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.803+0000 7f4805015640 1 Processor -- start 2024-05-22T23:07:49.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.803+0000 7f4805015640 1 -- start start 2024-05-22T23:07:49.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.804+0000 7f4805015640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4800102af0 0x7f48001980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.804+0000 7f4805015640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4800103410 0x7f4800198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.804+0000 7f4805015640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f480010be30 0x7f480019c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.804+0000 7f4805015640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4800111c30 con 0x7f480010be30 2024-05-22T23:07:49.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.804+0000 7f4805015640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4800111ab0 con 0x7f4800103410 2024-05-22T23:07:49.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.804+0000 7f4805015640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4800111db0 con 0x7f4800102af0 2024-05-22T23:07:49.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.804+0000 7f47ff59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f480010be30 0x7f480019c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.804+0000 7f47ff59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f480010be30 0x7f480019c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50204/0 (socket says 172.21.15.84:50204) 2024-05-22T23:07:49.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.804+0000 7f47ff59e640 1 -- 172.21.15.84:0/2810889856 learned_addr learned my addr 172.21.15.84:0/2810889856 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:49.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.804+0000 7f47fe59c640 1 --2- 172.21.15.84:0/2810889856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4800103410 0x7f4800198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.804+0000 7f47fed9d640 1 --2- 172.21.15.84:0/2810889856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4800102af0 0x7f48001980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.804+0000 7f47ff59e640 1 -- 172.21.15.84:0/2810889856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4800102af0 msgr2=0x7f48001980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:49.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.804+0000 7f47ff59e640 1 --2- 172.21.15.84:0/2810889856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4800102af0 0x7f48001980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.804+0000 7f47ff59e640 1 -- 172.21.15.84:0/2810889856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4800103410 msgr2=0x7f4800198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:49.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.804+0000 7f47ff59e640 1 --2- 172.21.15.84:0/2810889856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4800103410 0x7f4800198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.804+0000 7f47ff59e640 1 -- 172.21.15.84:0/2810889856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480019d0e0 con 0x7f480010be30 2024-05-22T23:07:49.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.804+0000 7f47fed9d640 1 --2- 172.21.15.84:0/2810889856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4800102af0 0x7f48001980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:07:49.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.804+0000 7f47fe59c640 1 --2- 172.21.15.84:0/2810889856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4800103410 0x7f4800198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:49.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.804+0000 7f47ff59e640 1 --2- 172.21.15.84:0/2810889856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f480010be30 0x7f480019c9b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f47f8006010 tx=0x7f47f8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:49.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.804+0000 7f47e7fff640 1 -- 172.21.15.84:0/2810889856 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8047070 con 0x7f480010be30 2024-05-22T23:07:49.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.805+0000 7f4805015640 1 -- 172.21.15.84:0/2810889856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480019d370 con 0x7f480010be30 2024-05-22T23:07:49.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.805+0000 7f47e7fff640 1 -- 172.21.15.84:0/2810889856 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47f802fb20 con 0x7f480010be30 2024-05-22T23:07:49.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.805+0000 7f47e7fff640 1 -- 172.21.15.84:0/2810889856 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8033050 con 0x7f480010be30 2024-05-22T23:07:49.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.805+0000 7f4805015640 1 -- 172.21.15.84:0/2810889856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48001a4c60 con 0x7f480010be30 2024-05-22T23:07:49.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.807+0000 7f47e7fff640 1 -- 172.21.15.84:0/2810889856 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f47f804a070 con 0x7f480010be30 2024-05-22T23:07:49.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.807+0000 7f4807e00740 1 Processor -- start 2024-05-22T23:07:49.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.807+0000 7f4807e00740 1 -- start start 2024-05-22T23:07:49.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.807+0000 7f4807e00740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ca4964c50 0x561ca4965030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.807+0000 7f4807e00740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca4965570 0x561ca496f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.807+0000 7f4807e00740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ca496fc40 0x561ca4972030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.807+0000 7f4807e00740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561ca48c8a40 con 0x561ca4965570 2024-05-22T23:07:49.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.807+0000 7f4807e00740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561ca4867440 con 0x561ca4964c50 2024-05-22T23:07:49.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.807+0000 7f4807e00740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561ca48c4a00 con 0x561ca496fc40 2024-05-22T23:07:49.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.807+0000 7f47e7fff640 1 --2- 172.21.15.84:0/2810889856 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f47e80a2d00 0x7f47e80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.808+0000 7f47fed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca4965570 0x561ca496f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.808+0000 7f47fe59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ca4964c50 0x561ca4965030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.808+0000 7f47fed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca4965570 0x561ca496f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50208/0 (socket says 172.21.15.84:50208) 2024-05-22T23:07:49.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.808+0000 7f47fed9d640 1 -- 172.21.15.84:0/4278174194 learned_addr learned my addr 172.21.15.84:0/4278174194 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:49.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.808+0000 7f47ff59e640 1 --2- 172.21.15.84:0/2810889856 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f47e80a2d00 0x7f47e80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.808+0000 7f47fe59c640 1 --2- 172.21.15.84:0/4278174194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ca496fc40 0x561ca4972030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.808+0000 7f47e7fff640 1 -- 172.21.15.84:0/2810889856 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f47f80eff40 con 0x7f480010be30 2024-05-22T23:07:49.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.808+0000 7f47fe59c640 1 -- 172.21.15.84:0/4278174194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ca496fc40 msgr2=0x561ca4972030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:49.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.808+0000 7f47fe59c640 1 --2- 172.21.15.84:0/4278174194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ca496fc40 0x561ca4972030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.808+0000 7f47fe59c640 1 -- 172.21.15.84:0/4278174194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca4965570 msgr2=0x561ca496f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:49.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.808+0000 7f47fe59c640 1 --2- 172.21.15.84:0/4278174194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca4965570 0x561ca496f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.808+0000 7f47fe59c640 1 -- 172.21.15.84:0/4278174194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ca48712d0 con 0x561ca4964c50 2024-05-22T23:07:49.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.808+0000 7f47fe59c640 1 --2- 172.21.15.84:0/4278174194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ca4964c50 0x561ca4965030 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4800100140 tx=0x7f47f40096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6dc6e4f882d49d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:49.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.808+0000 7f47ff59e640 1 --2- 172.21.15.84:0/2810889856 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f47e80a2d00 0x7f47e80a51c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f47f80ed740 tx=0x7f47f80095c0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:49.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.808+0000 7f47e57fa640 1 -- 172.21.15.84:0/4278174194 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f4013e60 con 0x561ca4964c50 2024-05-22T23:07:49.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.808+0000 7f47e57fa640 1 -- 172.21.15.84:0/4278174194 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47f400f940 con 0x561ca4964c50 2024-05-22T23:07:49.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.808+0000 7f47e57fa640 1 -- 172.21.15.84:0/4278174194 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f4003950 con 0x561ca4964c50 2024-05-22T23:07:49.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.809+0000 7f4807e00740 1 -- 172.21.15.84:0/4278174194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ca4964c50 msgr2=0x561ca4965030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:49.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.809+0000 7f4807e00740 1 --2- 172.21.15.84:0/4278174194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ca4964c50 0x561ca4965030 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4800100140 tx=0x7f47f40096c0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.809+0000 7f4807e00740 1 -- 172.21.15.84:0/4278174194 shutdown_connections 2024-05-22T23:07:49.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.809+0000 7f4807e00740 1 --2- 172.21.15.84:0/4278174194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ca496fc40 0x561ca4972030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.809+0000 7f4807e00740 1 --2- 172.21.15.84:0/4278174194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca4965570 0x561ca496f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.809+0000 7f4807e00740 1 --2- 172.21.15.84:0/4278174194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ca4964c50 0x561ca4965030 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.809+0000 7f4807e00740 1 -- 172.21.15.84:0/4278174194 >> 172.21.15.84:0/4278174194 conn(0x561ca4960f70 msgr2=0x561ca4961350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:49.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.809+0000 7f4807e00740 1 -- 172.21.15.84:0/4278174194 shutdown_connections 2024-05-22T23:07:49.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.809+0000 7f4807e00740 1 -- 172.21.15.84:0/4278174194 wait complete. 2024-05-22T23:07:49.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.809+0000 7f4807e00740 1 -- start start 2024-05-22T23:07:49.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.810+0000 7f4807e00740 1 Processor -- start 2024-05-22T23:07:49.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.810+0000 7f4807e00740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ca4964c50 0x561ca496cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.810+0000 7f4807e00740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca4965570 0x561ca496d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.810+0000 7f4807e00740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ca496fc40 0x561ca496d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.810+0000 7f4807e00740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561ca4972a30 con 0x561ca4965570 2024-05-22T23:07:49.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.810+0000 7f4807e00740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561ca48525f0 con 0x561ca496fc40 2024-05-22T23:07:49.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.810+0000 7f4807e00740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561ca4972bb0 con 0x561ca4964c50 2024-05-22T23:07:49.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.810+0000 7f4807e00740 10 client.0 _ll_register_callbacks cb 0x561ca484dcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:07:49.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.810+0000 7f47fed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca4965570 0x561ca496d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.810+0000 7f47fed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca4965570 0x561ca496d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50224/0 (socket says 172.21.15.84:50224) 2024-05-22T23:07:49.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.810+0000 7f47fed9d640 1 -- 172.21.15.84:0/4144142563 learned_addr learned my addr 172.21.15.84:0/4144142563 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:49.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.810+0000 7f47fe59c640 1 --2- 172.21.15.84:0/4144142563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ca496fc40 0x561ca496d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.810+0000 7f47fe59c640 1 --2- 172.21.15.84:0/4144142563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ca4964c50 0x561ca496cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.810+0000 7f47fed9d640 1 -- 172.21.15.84:0/4144142563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ca4964c50 msgr2=0x561ca496cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:49.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.810+0000 7f47fed9d640 1 --2- 172.21.15.84:0/4144142563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ca4964c50 0x561ca496cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.810+0000 7f47fed9d640 1 -- 172.21.15.84:0/4144142563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ca496fc40 msgr2=0x561ca496d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:49.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.810+0000 7f47fed9d640 1 --2- 172.21.15.84:0/4144142563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ca496fc40 0x561ca496d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.810+0000 7f47fed9d640 1 -- 172.21.15.84:0/4144142563 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ca46a47e0 con 0x561ca4965570 2024-05-22T23:07:49.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.810+0000 7f47fe59c640 1 --2- 172.21.15.84:0/4144142563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ca496fc40 0x561ca496d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:49.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.810+0000 7f47fed9d640 1 --2- 172.21.15.84:0/4144142563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca4965570 0x561ca496d110 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f47f0013710 tx=0x7f47f0013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=153cccecc070daf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:49.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.810+0000 7f47e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:07:49.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.811+0000 7f47e2ff5640 1 -- 172.21.15.84:0/4144142563 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f0011dd0 con 0x561ca4965570 2024-05-22T23:07:49.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.811+0000 7f4807e00740 1 -- 172.21.15.84:0/4144142563 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47f400f940 con 0x561ca4965570 2024-05-22T23:07:49.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.811+0000 7f47e2ff5640 1 -- 172.21.15.84:0/4144142563 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f47f0004030 con 0x561ca4965570 2024-05-22T23:07:49.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.811+0000 7f47e2ff5640 1 -- 172.21.15.84:0/4144142563 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f47f0004230 con 0x561ca4965570 2024-05-22T23:07:49.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.811+0000 7f4807e00740 10 client.16380 fetch_fsmap learned FSMap version 18 2024-05-22T23:07:49.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.811+0000 7f4807e00740 1 -- 172.21.15.84:0/4144142563 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x561ca4704f70 con 0x561ca4965570 2024-05-22T23:07:49.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.811+0000 7f47e2ff5640 1 -- 172.21.15.84:0/4144142563 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47f400f940 con 0x561ca4965570 2024-05-22T23:07:49.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.811+0000 7f47e2ff5640 1 -- 172.21.15.84:0/4144142563 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f47f000f040 con 0x561ca4965570 2024-05-22T23:07:49.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.811+0000 7f4807e00740 10 client.16380 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:07:49.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.811+0000 7f4807e00740 10 client.16380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:07:49.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.811+0000 7f4807e00740 20 client.16380 populate_metadata read hostname 'smithi084' 2024-05-22T23:07:49.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.811+0000 7f4807e00740 1 --2- 172.21.15.84:0/4144142563 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x561ca4980040 0x561ca4982460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.811+0000 7f4807e00740 4 client.16380 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:07:49.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.811+0000 7f4807e00740 1 -- 172.21.15.84:0/4144142563 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ca48dc460 con 0x561ca4980040 2024-05-22T23:07:49.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.811+0000 7f47fe59c640 1 --2- 172.21.15.84:0/4144142563 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x561ca4980040 0x561ca4982460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.812+0000 7f47e2ff5640 10 client.16380 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:49.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.812+0000 7f47fe59c640 1 --2- 172.21.15.84:0/4144142563 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x561ca4980040 0x561ca4982460 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b24b9458b6c4ef28 server_cookie=3dcca858974ef76b in_seq=0 out_seq=0 2024-05-22T23:07:49.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.813+0000 7f47e2ff5640 1 -- 172.21.15.84:0/4144142563 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561ca48dc460 con 0x561ca4980040 2024-05-22T23:07:49.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.813+0000 7f47e2ff5640 10 client.16380 handle_command_reply: tid=0 2024-05-22T23:07:49.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.813+0000 7f4807e00740 1 client.16380 shutdown 2024-05-22T23:07:49.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.813+0000 7f4807e00740 1 -- 172.21.15.84:0/4144142563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca4965570 msgr2=0x561ca496d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:49.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.813+0000 7f4807e00740 1 --2- 172.21.15.84:0/4144142563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca4965570 0x561ca496d110 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f47f0013710 tx=0x7f47f0013bd0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.813+0000 7f4807e00740 1 -- 172.21.15.84:0/4144142563 shutdown_connections 2024-05-22T23:07:49.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.813+0000 7f4807e00740 1 --2- 172.21.15.84:0/4144142563 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x561ca4980040 0x561ca4982460 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.813+0000 7f4807e00740 1 --2- 172.21.15.84:0/4144142563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ca496fc40 0x561ca496d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.813+0000 7f4807e00740 1 --2- 172.21.15.84:0/4144142563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca4965570 0x561ca496d110 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.813+0000 7f4807e00740 1 --2- 172.21.15.84:0/4144142563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ca4964c50 0x561ca496cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.813+0000 7f47e2ff5640 0 client.16380 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:49.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.813+0000 7f4807e00740 1 -- 172.21.15.84:0/4144142563 >> 172.21.15.84:0/4144142563 conn(0x561ca4960f70 msgr2=0x561ca49601b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:49.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.813+0000 7f4807e00740 1 -- 172.21.15.84:0/4144142563 shutdown_connections 2024-05-22T23:07:49.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.813+0000 7f4807e00740 1 -- 172.21.15.84:0/4144142563 wait complete. 2024-05-22T23:07:49.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.813+0000 7f4807e00740 20 client.16380 trim_cache size 0 max 16384 2024-05-22T23:07:49.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.831+0000 7f4807e00740 1 Processor -- start 2024-05-22T23:07:49.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.831+0000 7f4807e00740 1 -- start start 2024-05-22T23:07:49.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.831+0000 7f4807e00740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ca4964c50 0x561ca497b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.831+0000 7f4807e00740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca4965570 0x561ca4987d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.831+0000 7f4807e00740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ca496fc40 0x561ca4988250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.831+0000 7f4807e00740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f48001ab620 con 0x561ca4965570 2024-05-22T23:07:49.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.831+0000 7f4807e00740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f48001adea0 con 0x561ca496fc40 2024-05-22T23:07:49.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.831+0000 7f4807e00740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f48001abc90 con 0x561ca4964c50 2024-05-22T23:07:49.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.831+0000 7f47fed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca4965570 0x561ca4987d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.831+0000 7f47fed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca4965570 0x561ca4987d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50240/0 (socket says 172.21.15.84:50240) 2024-05-22T23:07:49.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.831+0000 7f47fed9d640 1 -- 172.21.15.84:0/3376101211 learned_addr learned my addr 172.21.15.84:0/3376101211 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:49.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.832+0000 7f47fe59c640 1 --2- 172.21.15.84:0/3376101211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ca496fc40 0x561ca4988250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.832+0000 7f47fe59c640 1 --2- 172.21.15.84:0/3376101211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ca4964c50 0x561ca497b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.832+0000 7f47fed9d640 1 -- 172.21.15.84:0/3376101211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ca4964c50 msgr2=0x561ca497b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:49.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.832+0000 7f47fed9d640 1 --2- 172.21.15.84:0/3376101211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ca4964c50 0x561ca497b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.832+0000 7f47fed9d640 1 -- 172.21.15.84:0/3376101211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ca496fc40 msgr2=0x561ca4988250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:49.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.832+0000 7f47fed9d640 1 --2- 172.21.15.84:0/3376101211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ca496fc40 0x561ca4988250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.832+0000 7f47fed9d640 1 -- 172.21.15.84:0/3376101211 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ca459c170 con 0x561ca4965570 2024-05-22T23:07:49.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.832+0000 7f47fed9d640 1 --2- 172.21.15.84:0/3376101211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca4965570 0x561ca4987d10 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f47f001c8b0 tx=0x7f47f0009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa39c8814650b416 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:49.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.832+0000 7f47e4ff9640 1 -- 172.21.15.84:0/3376101211 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f0031070 con 0x561ca4965570 2024-05-22T23:07:49.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.832+0000 7f47e4ff9640 1 -- 172.21.15.84:0/3376101211 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47f001ce10 con 0x561ca4965570 2024-05-22T23:07:49.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.832+0000 7f47e4ff9640 1 -- 172.21.15.84:0/3376101211 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f0029070 con 0x561ca4965570 2024-05-22T23:07:49.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.833+0000 7f4807e00740 1 -- 172.21.15.84:0/3376101211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca4965570 msgr2=0x561ca4987d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:49.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.833+0000 7f4807e00740 1 --2- 172.21.15.84:0/3376101211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca4965570 0x561ca4987d10 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f47f001c8b0 tx=0x7f47f0009dd0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.833+0000 7f4807e00740 1 -- 172.21.15.84:0/3376101211 shutdown_connections 2024-05-22T23:07:49.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.833+0000 7f4807e00740 1 --2- 172.21.15.84:0/3376101211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ca496fc40 0x561ca4988250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.833+0000 7f4807e00740 1 --2- 172.21.15.84:0/3376101211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca4965570 0x561ca4987d10 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.833+0000 7f4807e00740 1 --2- 172.21.15.84:0/3376101211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ca4964c50 0x561ca497b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.833+0000 7f4807e00740 1 -- 172.21.15.84:0/3376101211 >> 172.21.15.84:0/3376101211 conn(0x561ca4960f70 msgr2=0x561ca4966c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:49.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.833+0000 7f4807e00740 1 -- 172.21.15.84:0/3376101211 shutdown_connections 2024-05-22T23:07:49.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.833+0000 7f4807e00740 1 -- 172.21.15.84:0/3376101211 wait complete. 2024-05-22T23:07:49.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.833+0000 7f4807e00740 1 -- start start 2024-05-22T23:07:49.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.833+0000 7f4807e00740 1 Processor -- start 2024-05-22T23:07:49.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.833+0000 7f4807e00740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ca4964c50 0x561ca4977af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.833+0000 7f4807e00740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ca4965570 0x561ca4978030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.833+0000 7f4807e00740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca496fc40 0x561ca4973f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.833+0000 7f4807e00740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f48001ac360 con 0x561ca496fc40 2024-05-22T23:07:49.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.833+0000 7f4807e00740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f48001ad7d0 con 0x561ca4964c50 2024-05-22T23:07:49.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.833+0000 7f4807e00740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f48001aca30 con 0x561ca4965570 2024-05-22T23:07:49.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.833+0000 7f4807e00740 10 client.0 _ll_register_callbacks cb 0x7f47f0005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:07:49.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.834+0000 7f47fe59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca496fc40 0x561ca4973f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.834+0000 7f47fe59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca496fc40 0x561ca4973f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50242/0 (socket says 172.21.15.84:50242) 2024-05-22T23:07:49.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.834+0000 7f47fe59c640 1 -- 172.21.15.84:0/2600627839 learned_addr learned my addr 172.21.15.84:0/2600627839 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:49.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.834+0000 7f47fe59c640 1 --2- 172.21.15.84:0/2600627839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ca4964c50 0x561ca4977af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.834+0000 7f47fed9d640 1 --2- 172.21.15.84:0/2600627839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ca4965570 0x561ca4978030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.834+0000 7f47fe59c640 1 -- 172.21.15.84:0/2600627839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ca4965570 msgr2=0x561ca4978030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:49.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.834+0000 7f47fe59c640 1 --2- 172.21.15.84:0/2600627839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ca4965570 0x561ca4978030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.834+0000 7f47fe59c640 1 -- 172.21.15.84:0/2600627839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ca4964c50 msgr2=0x561ca4977af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:49.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.834+0000 7f47fe59c640 1 --2- 172.21.15.84:0/2600627839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ca4964c50 0x561ca4977af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.834+0000 7f47fe59c640 1 -- 172.21.15.84:0/2600627839 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ca4891930 con 0x561ca496fc40 2024-05-22T23:07:49.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.834+0000 7f47fed9d640 1 --2- 172.21.15.84:0/2600627839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ca4965570 0x561ca4978030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:07:49.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.834+0000 7f47fe59c640 1 --2- 172.21.15.84:0/2600627839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca496fc40 0x561ca4973f90 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009ca0 tx=0x7f47f4021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd419d62889a259 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:49.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.834+0000 7f47e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:07:49.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.834+0000 7f47e2ff5640 1 -- 172.21.15.84:0/2600627839 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f4013e60 con 0x561ca496fc40 2024-05-22T23:07:49.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.834+0000 7f4807e00740 1 -- 172.21.15.84:0/2600627839 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47f001ce10 con 0x561ca496fc40 2024-05-22T23:07:49.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.834+0000 7f47e2ff5640 1 -- 172.21.15.84:0/2600627839 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f47f400fd30 con 0x561ca496fc40 2024-05-22T23:07:49.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.834+0000 7f47e2ff5640 1 -- 172.21.15.84:0/2600627839 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f47f40039e0 con 0x561ca496fc40 2024-05-22T23:07:49.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.834+0000 7f4807e00740 10 client.16386 fetch_fsmap learned FSMap version 18 2024-05-22T23:07:49.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.834+0000 7f4807e00740 1 -- 172.21.15.84:0/2600627839 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x561ca49e6e10 con 0x561ca496fc40 2024-05-22T23:07:49.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.834+0000 7f47e2ff5640 1 -- 172.21.15.84:0/2600627839 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47f001ce10 con 0x561ca496fc40 2024-05-22T23:07:49.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.835+0000 7f47e2ff5640 1 -- 172.21.15.84:0/2600627839 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f47f401e070 con 0x561ca496fc40 2024-05-22T23:07:49.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.835+0000 7f4807e00740 10 client.16386 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:07:49.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.835+0000 7f4807e00740 10 client.16386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:07:49.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.835+0000 7f4807e00740 20 client.16386 populate_metadata read hostname 'smithi084' 2024-05-22T23:07:49.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.835+0000 7f4807e00740 1 --2- 172.21.15.84:0/2600627839 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x561ca49e7390 0x561ca49e9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.835+0000 7f4807e00740 4 client.16386 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-22T23:07:49.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.835+0000 7f4807e00740 1 -- 172.21.15.84:0/2600627839 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561ca48d6170 con 0x561ca49e7390 2024-05-22T23:07:49.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.835+0000 7f47fed9d640 1 --2- 172.21.15.84:0/2600627839 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x561ca49e7390 0x561ca49e9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.835+0000 7f47fed9d640 1 --2- 172.21.15.84:0/2600627839 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x561ca49e7390 0x561ca49e9780 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9566ef97dc5e6ebb server_cookie=ea9f5892dc66964b in_seq=0 out_seq=0 2024-05-22T23:07:49.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.835+0000 7f47e2ff5640 10 client.16386 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:49.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.836+0000 7f47e2ff5640 1 -- 172.21.15.84:0/2600627839 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561ca48d6170 con 0x561ca49e7390 2024-05-22T23:07:49.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.836+0000 7f47e2ff5640 10 client.16386 handle_command_reply: tid=0 2024-05-22T23:07:49.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.836+0000 7f4807e00740 1 client.16386 shutdown 2024-05-22T23:07:49.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.836+0000 7f4807e00740 1 -- 172.21.15.84:0/2600627839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca496fc40 msgr2=0x561ca4973f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:49.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.836+0000 7f4807e00740 1 --2- 172.21.15.84:0/2600627839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca496fc40 0x561ca4973f90 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009ca0 tx=0x7f47f4021040 comp rx=0 tx=0).stop 2024-05-22T23:07:49.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.836+0000 7f4807e00740 1 -- 172.21.15.84:0/2600627839 shutdown_connections 2024-05-22T23:07:49.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.836+0000 7f4807e00740 1 --2- 172.21.15.84:0/2600627839 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x561ca49e7390 0x561ca49e9780 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.836+0000 7f4807e00740 1 --2- 172.21.15.84:0/2600627839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ca496fc40 0x561ca4973f90 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.836+0000 7f4807e00740 1 --2- 172.21.15.84:0/2600627839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ca4965570 0x561ca4978030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.836+0000 7f47e2ff5640 0 client.16386 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:49.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.836+0000 7f4807e00740 1 --2- 172.21.15.84:0/2600627839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ca4964c50 0x561ca4977af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.836+0000 7f4807e00740 1 -- 172.21.15.84:0/2600627839 >> 172.21.15.84:0/2600627839 conn(0x561ca4960f70 msgr2=0x561ca495f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:49.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.836+0000 7f4807e00740 1 -- 172.21.15.84:0/2600627839 shutdown_connections 2024-05-22T23:07:49.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.836+0000 7f4807e00740 1 -- 172.21.15.84:0/2600627839 wait complete. 2024-05-22T23:07:49.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.837+0000 7f4807e00740 20 client.16386 trim_cache size 0 max 16384 2024-05-22T23:07:49.859 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:07:49.859 INFO:teuthology.orchestra.run.smithi084.stdout: "return_code": 0, 2024-05-22T23:07:49.860 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_tag": "d1e59321-3a77-4d7b-a41c-e55437bda5be", 2024-05-22T23:07:49.860 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": "asynchronous" 2024-05-22T23:07:49.860 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:07:49.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.837+0000 7f4805015640 1 -- 172.21.15.84:0/2810889856 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f47e80a2d00 msgr2=0x7f47e80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:49.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.837+0000 7f4805015640 1 --2- 172.21.15.84:0/2810889856 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f47e80a2d00 0x7f47e80a51c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f47f80ed740 tx=0x7f47f80095c0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.837+0000 7f4805015640 1 -- 172.21.15.84:0/2810889856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f480010be30 msgr2=0x7f480019c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:49.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.837+0000 7f4805015640 1 --2- 172.21.15.84:0/2810889856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f480010be30 0x7f480019c9b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f47f8006010 tx=0x7f47f8003ba0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.837+0000 7f4805015640 1 -- 172.21.15.84:0/2810889856 shutdown_connections 2024-05-22T23:07:49.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.837+0000 7f4805015640 1 --2- 172.21.15.84:0/2810889856 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f47e80a2d00 0x7f47e80a51c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.837+0000 7f4805015640 1 --2- 172.21.15.84:0/2810889856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f480010be30 0x7f480019c9b0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.837+0000 7f4805015640 1 --2- 172.21.15.84:0/2810889856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4800103410 0x7f4800198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.837+0000 7f4805015640 1 --2- 172.21.15.84:0/2810889856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4800102af0 0x7f48001980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.837+0000 7f4805015640 1 -- 172.21.15.84:0/2810889856 >> 172.21.15.84:0/2810889856 conn(0x7f4800077870 msgr2=0x7f48000fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:49.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.837+0000 7f4805015640 1 -- 172.21.15.84:0/2810889856 shutdown_connections 2024-05-22T23:07:49.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.837+0000 7f4805015640 1 -- 172.21.15.84:0/2810889856 wait complete. 2024-05-22T23:07:49.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:07:49.863 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:07:49.881 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:49 smithi084 ceph-mon[32186]: pgmap v182: 97 pgs: 97 active+clean; 401 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 243 op/s 2024-05-22T23:07:49.916 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:49 smithi204 ceph-mon[36211]: pgmap v182: 97 pgs: 97 active+clean; 401 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 243 op/s 2024-05-22T23:07:49.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.969+0000 7f4fec516640 1 -- 172.21.15.84:0/2272082412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe4100740 msgr2=0x7f4fe4100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:49.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.969+0000 7f4fec516640 1 --2- 172.21.15.84:0/2272082412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe4100740 0x7f4fe4100bc0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009930 tx=0x7f4fd402a420 comp rx=0 tx=0).stop 2024-05-22T23:07:49.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.970+0000 7f4fec516640 1 -- 172.21.15.84:0/2272082412 shutdown_connections 2024-05-22T23:07:49.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.970+0000 7f4fec516640 1 --2- 172.21.15.84:0/2272082412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fe4101100 0x7f4fe410d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.970+0000 7f4fec516640 1 --2- 172.21.15.84:0/2272082412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe4100740 0x7f4fe4100bc0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.970+0000 7f4fec516640 1 --2- 172.21.15.84:0/2272082412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fe4106750 0x7f4fe4106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.970+0000 7f4fec516640 1 -- 172.21.15.84:0/2272082412 >> 172.21.15.84:0/2272082412 conn(0x7f4fe4077870 msgr2=0x7f4fe4077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:49.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.970+0000 7f4fec516640 1 -- 172.21.15.84:0/2272082412 shutdown_connections 2024-05-22T23:07:49.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.970+0000 7f4fec516640 1 -- 172.21.15.84:0/2272082412 wait complete. 2024-05-22T23:07:49.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.970+0000 7f4fec516640 1 Processor -- start 2024-05-22T23:07:49.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.970+0000 7f4fec516640 1 -- start start 2024-05-22T23:07:49.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.970+0000 7f4fec516640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fe4100740 0x7f4fe41990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.971+0000 7f4fec516640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe4101100 0x7f4fe41995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.971+0000 7f4fec516640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fe4106750 0x7f4fe41a4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.971+0000 7f4fec516640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe4110bb0 con 0x7f4fe4101100 2024-05-22T23:07:49.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.971+0000 7f4fec516640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe4110a30 con 0x7f4fe4100740 2024-05-22T23:07:49.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.971+0000 7f4fec516640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe4110d30 con 0x7f4fe4106750 2024-05-22T23:07:49.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.971+0000 7f4fe9ab1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe4101100 0x7f4fe41995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.971+0000 7f4fe9ab1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe4101100 0x7f4fe41995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50268/0 (socket says 172.21.15.84:50268) 2024-05-22T23:07:49.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.971+0000 7f4fe9ab1640 1 -- 172.21.15.84:0/1516364543 learned_addr learned my addr 172.21.15.84:0/1516364543 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:49.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.971+0000 7f4fea2b2640 1 --2- 172.21.15.84:0/1516364543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fe4100740 0x7f4fe41990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.971+0000 7f4fe9ab1640 1 -- 172.21.15.84:0/1516364543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fe4106750 msgr2=0x7f4fe41a4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:49.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.971+0000 7f4fe9ab1640 1 --2- 172.21.15.84:0/1516364543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fe4106750 0x7f4fe41a4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.971+0000 7f4fe9ab1640 1 -- 172.21.15.84:0/1516364543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fe4100740 msgr2=0x7f4fe41990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:49.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.971+0000 7f4fe9ab1640 1 --2- 172.21.15.84:0/1516364543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fe4100740 0x7f4fe41990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.971+0000 7f4fe9ab1640 1 -- 172.21.15.84:0/1516364543 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe41a4670 con 0x7f4fe4101100 2024-05-22T23:07:49.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.971+0000 7f4fe9ab1640 1 --2- 172.21.15.84:0/1516364543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe4101100 0x7f4fe41995f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4fd402a940 tx=0x7f4fd4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:49.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.971+0000 7f4fdb7fe640 1 -- 172.21.15.84:0/1516364543 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd403f070 con 0x7f4fe4101100 2024-05-22T23:07:49.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.971+0000 7f4fdb7fe640 1 -- 172.21.15.84:0/1516364543 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fd4031c90 con 0x7f4fe4101100 2024-05-22T23:07:49.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.971+0000 7f4fec516640 1 -- 172.21.15.84:0/1516364543 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe41a4840 con 0x7f4fe4101100 2024-05-22T23:07:49.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.971+0000 7f4fdb7fe640 1 -- 172.21.15.84:0/1516364543 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4035030 con 0x7f4fe4101100 2024-05-22T23:07:49.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.971+0000 7f4fec516640 1 -- 172.21.15.84:0/1516364543 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe41a4ca0 con 0x7f4fe4101100 2024-05-22T23:07:49.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.974+0000 7f4fdb7fe640 1 -- 172.21.15.84:0/1516364543 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4fd4042070 con 0x7f4fe4101100 2024-05-22T23:07:49.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.974+0000 7f4fef21b740 1 Processor -- start 2024-05-22T23:07:49.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.974+0000 7f4fef21b740 1 -- start start 2024-05-22T23:07:49.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.974+0000 7f4fef21b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556051bf84a0 0x556051bf67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.974+0000 7f4fef21b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051bf6db0 0x556051bf7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.974+0000 7f4fef21b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556051c01b30 0x556051c03f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.974+0000 7f4fef21b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556051b5adb0 con 0x556051bf6db0 2024-05-22T23:07:49.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.974+0000 7f4fef21b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556051af9360 con 0x556051c01b30 2024-05-22T23:07:49.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.974+0000 7f4fef21b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556051b56d70 con 0x556051bf84a0 2024-05-22T23:07:49.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.975+0000 7f4fea2b2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051bf6db0 0x556051bf7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.975+0000 7f4fe9ab1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556051c01b30 0x556051c03f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.975+0000 7f4fea2b2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051bf6db0 0x556051bf7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50284/0 (socket says 172.21.15.84:50284) 2024-05-22T23:07:49.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.975+0000 7f4fea2b2640 1 -- 172.21.15.84:0/248000315 learned_addr learned my addr 172.21.15.84:0/248000315 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:49.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.975+0000 7f4feaab3640 1 --2- 172.21.15.84:0/248000315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556051bf84a0 0x556051bf67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.975+0000 7f4fea2b2640 1 -- 172.21.15.84:0/248000315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556051bf84a0 msgr2=0x556051bf67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:49.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.975+0000 7f4fea2b2640 1 --2- 172.21.15.84:0/248000315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556051bf84a0 0x556051bf67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.975+0000 7f4fea2b2640 1 -- 172.21.15.84:0/248000315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556051c01b30 msgr2=0x556051c03f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:49.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.975+0000 7f4fdb7fe640 1 --2- 172.21.15.84:0/1516364543 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f4fd00a2d00 0x7f4fd00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.975+0000 7f4fea2b2640 1 --2- 172.21.15.84:0/248000315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556051c01b30 0x556051c03f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.975+0000 7f4fea2b2640 1 -- 172.21.15.84:0/248000315 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556051b1ca60 con 0x556051bf6db0 2024-05-22T23:07:49.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.975+0000 7f4feaab3640 1 --2- 172.21.15.84:0/1516364543 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f4fd00a2d00 0x7f4fd00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.975+0000 7f4fea2b2640 1 --2- 172.21.15.84:0/248000315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051bf6db0 0x556051bf7230 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0009c70 tx=0x7f4fe0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418b2918284272c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:49.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.975+0000 7f4fdb7fe640 1 -- 172.21.15.84:0/1516364543 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f4fd40efbe0 con 0x7f4fe4101100 2024-05-22T23:07:49.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.975+0000 7f4fd8bfe640 1 -- 172.21.15.84:0/248000315 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0004230 con 0x556051bf6db0 2024-05-22T23:07:49.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.975+0000 7f4fd8bfe640 1 -- 172.21.15.84:0/248000315 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe00043d0 con 0x556051bf6db0 2024-05-22T23:07:49.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.975+0000 7f4fd8bfe640 1 -- 172.21.15.84:0/248000315 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe00197a0 con 0x556051bf6db0 2024-05-22T23:07:49.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.976+0000 7f4feaab3640 1 --2- 172.21.15.84:0/1516364543 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f4fd00a2d00 0x7f4fd00a51c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009270 tx=0x7f4fdc0094e0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:49.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.976+0000 7f4fef21b740 1 -- 172.21.15.84:0/248000315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051bf6db0 msgr2=0x556051bf7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:49.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.976+0000 7f4fef21b740 1 --2- 172.21.15.84:0/248000315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051bf6db0 0x556051bf7230 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0009c70 tx=0x7f4fe0011810 comp rx=0 tx=0).stop 2024-05-22T23:07:49.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.976+0000 7f4fef21b740 1 -- 172.21.15.84:0/248000315 shutdown_connections 2024-05-22T23:07:49.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.976+0000 7f4fef21b740 1 --2- 172.21.15.84:0/248000315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556051c01b30 0x556051c03f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.976+0000 7f4fef21b740 1 --2- 172.21.15.84:0/248000315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051bf6db0 0x556051bf7230 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.976+0000 7f4fef21b740 1 --2- 172.21.15.84:0/248000315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556051bf84a0 0x556051bf67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.976+0000 7f4fef21b740 1 -- 172.21.15.84:0/248000315 >> 172.21.15.84:0/248000315 conn(0x556051bbf8e0 msgr2=0x556051bf3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:49.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.976+0000 7f4fef21b740 1 -- 172.21.15.84:0/248000315 shutdown_connections 2024-05-22T23:07:49.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.976+0000 7f4fef21b740 1 -- 172.21.15.84:0/248000315 wait complete. 2024-05-22T23:07:49.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.977+0000 7f4fef21b740 1 -- start start 2024-05-22T23:07:49.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.977+0000 7f4fef21b740 1 Processor -- start 2024-05-22T23:07:49.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.977+0000 7f4fef21b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556051bf6db0 0x556051c014a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.977+0000 7f4fef21b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051bf84a0 0x556051bfe9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.977+0000 7f4fef21b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556051c01b30 0x556051bfef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.977+0000 7f4fef21b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556051c049c0 con 0x556051bf84a0 2024-05-22T23:07:49.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.977+0000 7f4fef21b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556051b150f0 con 0x556051bf6db0 2024-05-22T23:07:49.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.977+0000 7f4fef21b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556051c04b40 con 0x556051c01b30 2024-05-22T23:07:49.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.977+0000 7f4fef21b740 10 client.0 _ll_register_callbacks cb 0x556051b19f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:07:49.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.977+0000 7f4fe9ab1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051bf84a0 0x556051bfe9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.977+0000 7f4fe9ab1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051bf84a0 0x556051bfe9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50296/0 (socket says 172.21.15.84:50296) 2024-05-22T23:07:49.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.977+0000 7f4fe9ab1640 1 -- 172.21.15.84:0/256832255 learned_addr learned my addr 172.21.15.84:0/256832255 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:49.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.977+0000 7f4fea2b2640 1 --2- 172.21.15.84:0/256832255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556051bf6db0 0x556051c014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.977+0000 7f4fe9ab1640 1 -- 172.21.15.84:0/256832255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556051c01b30 msgr2=0x556051bfef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:49.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.977+0000 7f4fe9ab1640 1 --2- 172.21.15.84:0/256832255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556051c01b30 0x556051bfef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.977+0000 7f4fe9ab1640 1 -- 172.21.15.84:0/256832255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556051bf6db0 msgr2=0x556051c014a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:49.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.977+0000 7f4fe9ab1640 1 --2- 172.21.15.84:0/256832255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556051bf6db0 0x556051c014a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:49.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.977+0000 7f4fe9ab1640 1 -- 172.21.15.84:0/256832255 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560519387d0 con 0x556051bf84a0 2024-05-22T23:07:49.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.977+0000 7f4fe9ab1640 1 --2- 172.21.15.84:0/256832255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051bf84a0 0x556051bfe9f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4000c00 tx=0x7f4fd4031490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66e497a0b7e69c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:49.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.977+0000 7f4fcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:07:49.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.978+0000 7f4fcdffb640 1 -- 172.21.15.84:0/256832255 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4049070 con 0x556051bf84a0 2024-05-22T23:07:49.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.978+0000 7f4fcdffb640 1 -- 172.21.15.84:0/256832255 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f4fd403b460 con 0x556051bf84a0 2024-05-22T23:07:49.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.978+0000 7f4fef21b740 1 -- 172.21.15.84:0/256832255 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fe00043d0 con 0x556051bf84a0 2024-05-22T23:07:49.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.978+0000 7f4fcdffb640 1 -- 172.21.15.84:0/256832255 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f4fd40bbed0 con 0x556051bf84a0 2024-05-22T23:07:49.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.978+0000 7f4fef21b740 10 client.16398 fetch_fsmap learned FSMap version 18 2024-05-22T23:07:49.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.978+0000 7f4fef21b740 1 -- 172.21.15.84:0/256832255 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55605186e3f0 con 0x556051bf84a0 2024-05-22T23:07:49.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.978+0000 7f4fcdffb640 1 -- 172.21.15.84:0/256832255 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe00043d0 con 0x556051bf84a0 2024-05-22T23:07:49.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.978+0000 7f4fcdffb640 1 -- 172.21.15.84:0/256832255 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f4fd40ed5b0 con 0x556051bf84a0 2024-05-22T23:07:49.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.978+0000 7f4fef21b740 10 client.16398 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:07:49.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.978+0000 7f4fef21b740 10 client.16398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:07:49.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.978+0000 7f4fef21b740 20 client.16398 populate_metadata read hostname 'smithi084' 2024-05-22T23:07:49.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.978+0000 7f4fef21b740 1 --2- 172.21.15.84:0/256832255 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556051c1a6b0 0x556051c1cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:49.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.978+0000 7f4fef21b740 4 client.16398 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:07:49.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.978+0000 7f4fef21b740 1 -- 172.21.15.84:0/256832255 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556051b684e0 con 0x556051c1a6b0 2024-05-22T23:07:49.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.978+0000 7f4fea2b2640 1 --2- 172.21.15.84:0/256832255 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556051c1a6b0 0x556051c1cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:49.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.979+0000 7f4fcdffb640 10 client.16398 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:49.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.979+0000 7f4fea2b2640 1 --2- 172.21.15.84:0/256832255 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556051c1a6b0 0x556051c1cae0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d94579785b4135b0 server_cookie=e6e43e5589e38430 in_seq=0 out_seq=0 2024-05-22T23:07:49.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.979+0000 7f4fcdffb640 1 -- 172.21.15.84:0/256832255 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556051b684e0 con 0x556051c1a6b0 2024-05-22T23:07:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.979+0000 7f4fcdffb640 10 client.16398 handle_command_reply: tid=0 2024-05-22T23:07:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.980+0000 7f4fef21b740 1 client.16398 shutdown 2024-05-22T23:07:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.980+0000 7f4fef21b740 1 -- 172.21.15.84:0/256832255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051bf84a0 msgr2=0x556051bfe9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.980+0000 7f4fef21b740 1 --2- 172.21.15.84:0/256832255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051bf84a0 0x556051bfe9f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4000c00 tx=0x7f4fd4031490 comp rx=0 tx=0).stop 2024-05-22T23:07:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.980+0000 7f4fef21b740 1 -- 172.21.15.84:0/256832255 shutdown_connections 2024-05-22T23:07:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.980+0000 7f4fef21b740 1 --2- 172.21.15.84:0/256832255 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556051c1a6b0 0x556051c1cae0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.980+0000 7f4fef21b740 1 --2- 172.21.15.84:0/256832255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556051c01b30 0x556051bfef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.980+0000 7f4fef21b740 1 --2- 172.21.15.84:0/256832255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051bf84a0 0x556051bfe9f0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.980+0000 7f4fef21b740 1 --2- 172.21.15.84:0/256832255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556051bf6db0 0x556051c014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.980+0000 7f4fef21b740 1 -- 172.21.15.84:0/256832255 >> 172.21.15.84:0/256832255 conn(0x556051bbf8e0 msgr2=0x556051bf9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.980+0000 7f4fcdffb640 0 client.16398 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.980+0000 7f4fef21b740 1 -- 172.21.15.84:0/256832255 shutdown_connections 2024-05-22T23:07:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.980+0000 7f4fef21b740 1 -- 172.21.15.84:0/256832255 wait complete. 2024-05-22T23:07:50.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.980+0000 7f4fef21b740 20 client.16398 trim_cache size 0 max 16384 2024-05-22T23:07:50.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.998+0000 7f4fef21b740 1 Processor -- start 2024-05-22T23:07:50.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.998+0000 7f4fef21b740 1 -- start start 2024-05-22T23:07:50.005 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.998+0000 7f4fef21b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556051bf6db0 0x556051bf3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:50.006 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.998+0000 7f4fef21b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051c01b30 0x556051bf9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:50.006 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.998+0000 7f4fef21b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556051c223f0 0x556051c22870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:50.006 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.998+0000 7f4fef21b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41a9350 con 0x556051c01b30 2024-05-22T23:07:50.006 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.998+0000 7f4fef21b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41abbd0 con 0x556051bf6db0 2024-05-22T23:07:50.006 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.998+0000 7f4fef21b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41a99c0 con 0x556051c223f0 2024-05-22T23:07:50.006 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.999+0000 7f4fe9ab1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051c01b30 0x556051bf9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:50.006 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.999+0000 7f4fe9ab1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051c01b30 0x556051bf9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50310/0 (socket says 172.21.15.84:50310) 2024-05-22T23:07:50.006 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.999+0000 7f4fe9ab1640 1 -- 172.21.15.84:0/452855820 learned_addr learned my addr 172.21.15.84:0/452855820 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:50.006 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.999+0000 7f4fea2b2640 1 --2- 172.21.15.84:0/452855820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556051bf6db0 0x556051bf3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:50.007 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.999+0000 7f4feaab3640 1 --2- 172.21.15.84:0/452855820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556051c223f0 0x556051c22870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:50.007 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.999+0000 7f4fe9ab1640 1 -- 172.21.15.84:0/452855820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556051c223f0 msgr2=0x556051c22870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:50.007 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.999+0000 7f4fe9ab1640 1 --2- 172.21.15.84:0/452855820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556051c223f0 0x556051c22870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:50.007 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.999+0000 7f4fe9ab1640 1 -- 172.21.15.84:0/452855820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556051bf6db0 msgr2=0x556051bf3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:50.007 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.999+0000 7f4fe9ab1640 1 --2- 172.21.15.84:0/452855820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556051bf6db0 0x556051bf3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:50.007 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.999+0000 7f4fe9ab1640 1 -- 172.21.15.84:0/452855820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556051bfd9c0 con 0x556051c01b30 2024-05-22T23:07:50.007 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.999+0000 7f4fe9ab1640 1 --2- 172.21.15.84:0/452855820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051c01b30 0x556051bf9d80 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40ba3b0 tx=0x7f4fd40ba610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3175b6e988fc1c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:50.007 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.999+0000 7f4fcffff640 1 -- 172.21.15.84:0/452855820 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4049070 con 0x556051c01b30 2024-05-22T23:07:50.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.999+0000 7f4fcffff640 1 -- 172.21.15.84:0/452855820 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fd40bc800 con 0x556051c01b30 2024-05-22T23:07:50.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:49.999+0000 7f4fcffff640 1 -- 172.21.15.84:0/452855820 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd40bba60 con 0x556051c01b30 2024-05-22T23:07:50.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.000+0000 7f4fef21b740 1 -- 172.21.15.84:0/452855820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051c01b30 msgr2=0x556051bf9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:50.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.000+0000 7f4fef21b740 1 --2- 172.21.15.84:0/452855820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051c01b30 0x556051bf9d80 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40ba3b0 tx=0x7f4fd40ba610 comp rx=0 tx=0).stop 2024-05-22T23:07:50.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.000+0000 7f4fef21b740 1 -- 172.21.15.84:0/452855820 shutdown_connections 2024-05-22T23:07:50.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.000+0000 7f4fef21b740 1 --2- 172.21.15.84:0/452855820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556051c223f0 0x556051c22870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:50.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.000+0000 7f4fef21b740 1 --2- 172.21.15.84:0/452855820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051c01b30 0x556051bf9d80 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:50.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.000+0000 7f4fef21b740 1 --2- 172.21.15.84:0/452855820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556051bf6db0 0x556051bf3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:50.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.000+0000 7f4fef21b740 1 -- 172.21.15.84:0/452855820 >> 172.21.15.84:0/452855820 conn(0x556051bbf8e0 msgr2=0x556051bf2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:50.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.000+0000 7f4fef21b740 1 -- 172.21.15.84:0/452855820 shutdown_connections 2024-05-22T23:07:50.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.000+0000 7f4fef21b740 1 -- 172.21.15.84:0/452855820 wait complete. 2024-05-22T23:07:50.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.000+0000 7f4fef21b740 1 -- start start 2024-05-22T23:07:50.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.000+0000 7f4fef21b740 1 Processor -- start 2024-05-22T23:07:50.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4fef21b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556051bf6db0 0x556051c13a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:50.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4fef21b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556051c01b30 0x556051c13f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:50.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4fef21b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051c223f0 0x556051c144c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:50.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4fef21b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41aa090 con 0x556051c223f0 2024-05-22T23:07:50.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4fef21b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41ab500 con 0x556051c01b30 2024-05-22T23:07:50.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4fef21b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41aa760 con 0x556051bf6db0 2024-05-22T23:07:50.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4fef21b740 10 client.0 _ll_register_callbacks cb 0x7f4fd4009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:07:50.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4feaab3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051c223f0 0x556051c144c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:50.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4feaab3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051c223f0 0x556051c144c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50322/0 (socket says 172.21.15.84:50322) 2024-05-22T23:07:50.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4feaab3640 1 -- 172.21.15.84:0/1099898196 learned_addr learned my addr 172.21.15.84:0/1099898196 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:07:50.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4fe9ab1640 1 --2- 172.21.15.84:0/1099898196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556051c01b30 0x556051c13f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:50.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4fe9ab1640 1 -- 172.21.15.84:0/1099898196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556051bf6db0 msgr2=0x556051c13a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:50.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4fea2b2640 1 --2- 172.21.15.84:0/1099898196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556051bf6db0 0x556051c13a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:50.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4fe9ab1640 1 --2- 172.21.15.84:0/1099898196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556051bf6db0 0x556051c13a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:50.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4fe9ab1640 1 -- 172.21.15.84:0/1099898196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051c223f0 msgr2=0x556051c144c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:07:50.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4fe9ab1640 1 --2- 172.21.15.84:0/1099898196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051c223f0 0x556051c144c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:50.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4fe9ab1640 1 -- 172.21.15.84:0/1099898196 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55605182e0f0 con 0x556051c01b30 2024-05-22T23:07:50.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4feaab3640 1 --2- 172.21.15.84:0/1099898196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051c223f0 0x556051c144c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:07:50.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4fea2b2640 1 --2- 172.21.15.84:0/1099898196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556051bf6db0 0x556051c13a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:07:50.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4fe9ab1640 1 --2- 172.21.15.84:0/1099898196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556051c01b30 0x556051c13f80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x556051bfaa60 tx=0x7f4fd4005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c4aceab5563db86 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:07:50.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4fcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:07:50.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4fcdffb640 1 -- 172.21.15.84:0/1099898196 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4049070 con 0x556051c01b30 2024-05-22T23:07:50.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4fcdffb640 1 -- 172.21.15.84:0/1099898196 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8527+0+0 (secure 0 0 0) 0x7f4fd40c48e0 con 0x556051c01b30 2024-05-22T23:07:50.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4fef21b740 1 -- 172.21.15.84:0/1099898196 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fd40bc800 con 0x556051c01b30 2024-05-22T23:07:50.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4fcdffb640 1 -- 172.21.15.84:0/1099898196 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f4fd40bbc40 con 0x556051c01b30 2024-05-22T23:07:50.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4fef21b740 10 client.25976 fetch_fsmap learned FSMap version 18 2024-05-22T23:07:50.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.001+0000 7f4fef21b740 1 -- 172.21.15.84:0/1099898196 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x556051b19fe0 con 0x556051c01b30 2024-05-22T23:07:50.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.002+0000 7f4fcdffb640 1 -- 172.21.15.84:0/1099898196 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fd40bc800 con 0x556051c01b30 2024-05-22T23:07:50.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.002+0000 7f4fcdffb640 1 -- 172.21.15.84:0/1099898196 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f4fd40bb550 con 0x556051c01b30 2024-05-22T23:07:50.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.002+0000 7f4fef21b740 10 client.25976 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:07:50.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.002+0000 7f4fef21b740 10 client.25976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:07:50.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.002+0000 7f4fef21b740 20 client.25976 populate_metadata read hostname 'smithi084' 2024-05-22T23:07:50.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.002+0000 7f4fef21b740 1 --2- 172.21.15.84:0/1099898196 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556051c81fa0 0x556051c84390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:07:50.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.002+0000 7f4fef21b740 4 client.25976 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:07:50.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.002+0000 7f4fef21b740 1 -- 172.21.15.84:0/1099898196 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556051c178b0 con 0x556051c81fa0 2024-05-22T23:07:50.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.002+0000 7f4fea2b2640 1 --2- 172.21.15.84:0/1099898196 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556051c81fa0 0x556051c84390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:07:50.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.002+0000 7f4fcdffb640 10 client.25976 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:50.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.002+0000 7f4fea2b2640 1 --2- 172.21.15.84:0/1099898196 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556051c81fa0 0x556051c84390 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a3f9e3e445e268e server_cookie=32e1800299b0ffb6 in_seq=0 out_seq=0 2024-05-22T23:07:50.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.004+0000 7f4fcdffb640 1 -- 172.21.15.84:0/1099898196 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556051c178b0 con 0x556051c81fa0 2024-05-22T23:07:50.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.004+0000 7f4fcdffb640 10 client.25976 handle_command_reply: tid=0 2024-05-22T23:07:50.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.004+0000 7f4fef21b740 1 client.25976 shutdown 2024-05-22T23:07:50.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.004+0000 7f4fef21b740 1 -- 172.21.15.84:0/1099898196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556051c01b30 msgr2=0x556051c13f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:50.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.004+0000 7f4fef21b740 1 --2- 172.21.15.84:0/1099898196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556051c01b30 0x556051c13f80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x556051bfaa60 tx=0x7f4fd4005e50 comp rx=0 tx=0).stop 2024-05-22T23:07:50.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.004+0000 7f4fef21b740 1 -- 172.21.15.84:0/1099898196 shutdown_connections 2024-05-22T23:07:50.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.004+0000 7f4fef21b740 1 --2- 172.21.15.84:0/1099898196 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556051c81fa0 0x556051c84390 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:50.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.004+0000 7f4fef21b740 1 --2- 172.21.15.84:0/1099898196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556051c223f0 0x556051c144c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:50.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.004+0000 7f4fcdffb640 0 client.25976 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:07:50.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.004+0000 7f4fef21b740 1 --2- 172.21.15.84:0/1099898196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556051c01b30 0x556051c13f80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:50.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.004+0000 7f4fef21b740 1 --2- 172.21.15.84:0/1099898196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556051bf6db0 0x556051c13a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:50.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.004+0000 7f4fef21b740 1 -- 172.21.15.84:0/1099898196 >> 172.21.15.84:0/1099898196 conn(0x556051bbf8e0 msgr2=0x556051c06da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:50.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.004+0000 7f4fef21b740 1 -- 172.21.15.84:0/1099898196 shutdown_connections 2024-05-22T23:07:50.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.004+0000 7f4fef21b740 1 -- 172.21.15.84:0/1099898196 wait complete. 2024-05-22T23:07:50.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.004+0000 7f4fef21b740 20 client.25976 trim_cache size 0 max 16384 2024-05-22T23:07:50.020 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:07:50.020 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-22T23:07:50.020 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": { 2024-05-22T23:07:50.020 INFO:teuthology.orchestra.run.smithi084.stdout: "d1e59321-3a77-4d7b-a41c-e55437bda5be": { 2024-05-22T23:07:50.020 INFO:teuthology.orchestra.run.smithi084.stdout: "path": "/", 2024-05-22T23:07:50.020 INFO:teuthology.orchestra.run.smithi084.stdout: "tag": "d1e59321-3a77-4d7b-a41c-e55437bda5be", 2024-05-22T23:07:50.020 INFO:teuthology.orchestra.run.smithi084.stdout: "options": "recursive,force" 2024-05-22T23:07:50.021 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:07:50.021 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:07:50.021 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:07:50.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.004+0000 7f4fec516640 1 -- 172.21.15.84:0/1516364543 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f4fd00a2d00 msgr2=0x7f4fd00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:50.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.004+0000 7f4fec516640 1 --2- 172.21.15.84:0/1516364543 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f4fd00a2d00 0x7f4fd00a51c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009270 tx=0x7f4fdc0094e0 comp rx=0 tx=0).stop 2024-05-22T23:07:50.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.004+0000 7f4fec516640 1 -- 172.21.15.84:0/1516364543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe4101100 msgr2=0x7f4fe41995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:07:50.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.004+0000 7f4fec516640 1 --2- 172.21.15.84:0/1516364543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe4101100 0x7f4fe41995f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4fd402a940 tx=0x7f4fd4003b90 comp rx=0 tx=0).stop 2024-05-22T23:07:50.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.004+0000 7f4fec516640 1 -- 172.21.15.84:0/1516364543 shutdown_connections 2024-05-22T23:07:50.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.004+0000 7f4fec516640 1 --2- 172.21.15.84:0/1516364543 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f4fd00a2d00 0x7f4fd00a51c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:50.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.004+0000 7f4fec516640 1 --2- 172.21.15.84:0/1516364543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fe4106750 0x7f4fe41a4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:50.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.004+0000 7f4fec516640 1 --2- 172.21.15.84:0/1516364543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe4101100 0x7f4fe41995f0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:50.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.005+0000 7f4fec516640 1 --2- 172.21.15.84:0/1516364543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fe4100740 0x7f4fe41990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:07:50.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.005+0000 7f4fec516640 1 -- 172.21.15.84:0/1516364543 >> 172.21.15.84:0/1516364543 conn(0x7f4fe4077870 msgr2=0x7f4fe4102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:07:50.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.005+0000 7f4fec516640 1 -- 172.21.15.84:0/1516364543 shutdown_connections 2024-05-22T23:07:50.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:07:50.005+0000 7f4fec516640 1 -- 172.21.15.84:0/1516364543 wait complete. 2024-05-22T23:07:50.023 INFO:tasks.cephfs.filesystem:scrub status for tag:d1e59321-3a77-4d7b-a41c-e55437bda5be - {'path': '/', 'tag': 'd1e59321-3a77-4d7b-a41c-e55437bda5be', 'options': 'recursive,force'} 2024-05-22T23:07:50.060 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:49 smithi096 ceph-mon[36392]: pgmap v182: 97 pgs: 97 active+clean; 401 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 243 op/s 2024-05-22T23:07:50.438 INFO:tasks.workunit.client.0.smithi084.stdout: 1 785661 15.09 MB/sec execute 156 sec latency 16.849 ms 2024-05-22T23:07:50.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:50 smithi084 ceph-mon[32186]: scrub queued for path: / 2024-05-22T23:07:50.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:50 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:07:50.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:50 smithi084 ceph-mon[32186]: scrub summary: active paths [/] 2024-05-22T23:07:50.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:50 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:07:50.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:50 smithi204 ceph-mon[36211]: scrub queued for path: / 2024-05-22T23:07:50.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:50 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:07:50.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:50 smithi204 ceph-mon[36211]: scrub summary: active paths [/] 2024-05-22T23:07:50.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:50 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:07:51.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:50 smithi096 ceph-mon[36392]: scrub queued for path: / 2024-05-22T23:07:51.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:50 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:07:51.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:50 smithi096 ceph-mon[36392]: scrub summary: active paths [/] 2024-05-22T23:07:51.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:50 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:07:51.438 INFO:tasks.workunit.client.0.smithi084.stdout: 1 788478 15.11 MB/sec execute 157 sec latency 65.570 ms 2024-05-22T23:07:51.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:51 smithi084 ceph-mon[32186]: pgmap v183: 97 pgs: 97 active+clean; 401 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 243 op/s 2024-05-22T23:07:51.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:51 smithi084 ceph-mon[32186]: scrub summary: idle 2024-05-22T23:07:51.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:51 smithi204 ceph-mon[36211]: pgmap v183: 97 pgs: 97 active+clean; 401 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 243 op/s 2024-05-22T23:07:51.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:51 smithi204 ceph-mon[36211]: scrub summary: idle 2024-05-22T23:07:52.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:51 smithi096 ceph-mon[36392]: pgmap v183: 97 pgs: 97 active+clean; 401 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 243 op/s 2024-05-22T23:07:52.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:51 smithi096 ceph-mon[36392]: scrub summary: idle 2024-05-22T23:07:52.438 INFO:tasks.workunit.client.0.smithi084.stdout: 1 791211 15.10 MB/sec execute 158 sec latency 47.298 ms 2024-05-22T23:07:53.438 INFO:tasks.workunit.client.0.smithi084.stdout: 1 793633 15.08 MB/sec execute 159 sec latency 20.105 ms 2024-05-22T23:07:53.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:53 smithi084 ceph-mon[32186]: pgmap v184: 97 pgs: 97 active+clean; 406 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.1 MiB/s wr, 362 op/s 2024-05-22T23:07:53.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:53 smithi204 ceph-mon[36211]: pgmap v184: 97 pgs: 97 active+clean; 406 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.1 MiB/s wr, 362 op/s 2024-05-22T23:07:54.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:53 smithi096 ceph-mon[36392]: pgmap v184: 97 pgs: 97 active+clean; 406 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.1 MiB/s wr, 362 op/s 2024-05-22T23:07:54.438 INFO:tasks.workunit.client.0.smithi084.stdout: 1 796379 15.08 MB/sec execute 160 sec latency 14.503 ms 2024-05-22T23:07:55.439 INFO:tasks.workunit.client.0.smithi084.stdout: 1 799312 15.09 MB/sec execute 161 sec latency 19.598 ms 2024-05-22T23:07:55.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:55 smithi084 ceph-mon[32186]: pgmap v185: 97 pgs: 97 active+clean; 406 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 234 op/s 2024-05-22T23:07:55.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:55 smithi204 ceph-mon[36211]: pgmap v185: 97 pgs: 97 active+clean; 406 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 234 op/s 2024-05-22T23:07:56.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:55 smithi096 ceph-mon[36392]: pgmap v185: 97 pgs: 97 active+clean; 406 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 234 op/s 2024-05-22T23:07:56.439 INFO:tasks.workunit.client.0.smithi084.stdout: 1 802070 15.08 MB/sec execute 162 sec latency 101.903 ms 2024-05-22T23:07:57.438 INFO:tasks.workunit.client.0.smithi084.stdout: 1 804738 15.07 MB/sec execute 163 sec latency 34.173 ms 2024-05-22T23:07:57.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:57 smithi204 ceph-mon[36211]: pgmap v186: 97 pgs: 97 active+clean; 384 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.0 MiB/s wr, 355 op/s 2024-05-22T23:07:58.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:57 smithi096 ceph-mon[36392]: pgmap v186: 97 pgs: 97 active+clean; 384 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.0 MiB/s wr, 355 op/s 2024-05-22T23:07:58.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:57 smithi084 ceph-mon[32186]: pgmap v186: 97 pgs: 97 active+clean; 384 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.0 MiB/s wr, 355 op/s 2024-05-22T23:07:58.438 INFO:tasks.workunit.client.0.smithi084.stdout: 1 807138 15.06 MB/sec execute 164 sec latency 22.322 ms 2024-05-22T23:07:59.438 INFO:tasks.workunit.client.0.smithi084.stdout: 1 810236 15.08 MB/sec execute 165 sec latency 13.933 ms 2024-05-22T23:07:59.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:07:59 smithi204 ceph-mon[36211]: pgmap v187: 97 pgs: 97 active+clean; 384 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 240 op/s 2024-05-22T23:07:59.930 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:07:59 smithi084 ceph-mon[32186]: pgmap v187: 97 pgs: 97 active+clean; 384 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 240 op/s 2024-05-22T23:08:00.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:07:59 smithi096 ceph-mon[36392]: pgmap v187: 97 pgs: 97 active+clean; 384 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 240 op/s 2024-05-22T23:08:00.438 INFO:tasks.workunit.client.0.smithi084.stdout: 1 813332 15.10 MB/sec execute 166 sec latency 85.178 ms 2024-05-22T23:08:01.438 INFO:tasks.workunit.client.0.smithi084.stdout: 1 815329 15.05 MB/sec execute 167 sec latency 66.326 ms 2024-05-22T23:08:01.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:01 smithi204 ceph-mon[36211]: pgmap v188: 97 pgs: 97 active+clean; 384 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 240 op/s 2024-05-22T23:08:01.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:01 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:08:01.916 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:01 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:08:01.916 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:01 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:08:01.916 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:01 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:08:01.952 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:01 smithi096 ceph-mon[36392]: pgmap v188: 97 pgs: 97 active+clean; 384 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 240 op/s 2024-05-22T23:08:01.952 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:01 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:08:01.952 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:01 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:08:01.952 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:01 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:08:01.952 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:01 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:08:02.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:01 smithi084 ceph-mon[32186]: pgmap v188: 97 pgs: 97 active+clean; 384 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 240 op/s 2024-05-22T23:08:02.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:01 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:08:02.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:01 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:08:02.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:01 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:08:02.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:01 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:08:02.438 INFO:tasks.workunit.client.0.smithi084.stdout: 1 817195 15.03 MB/sec execute 168 sec latency 76.060 ms 2024-05-22T23:08:03.438 INFO:tasks.workunit.client.0.smithi084.stdout: 1 820414 15.06 MB/sec execute 169 sec latency 23.452 ms 2024-05-22T23:08:03.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:03 smithi204 ceph-mon[36211]: pgmap v189: 97 pgs: 97 active+clean; 394 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.7 MiB/s wr, 363 op/s 2024-05-22T23:08:04.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:03 smithi096 ceph-mon[36392]: pgmap v189: 97 pgs: 97 active+clean; 394 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.7 MiB/s wr, 363 op/s 2024-05-22T23:08:04.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:03 smithi084 ceph-mon[32186]: pgmap v189: 97 pgs: 97 active+clean; 394 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.7 MiB/s wr, 363 op/s 2024-05-22T23:08:04.438 INFO:tasks.workunit.client.0.smithi084.stdout: 1 823507 15.05 MB/sec execute 170 sec latency 21.535 ms 2024-05-22T23:08:05.438 INFO:tasks.workunit.client.0.smithi084.stdout: 1 826265 15.04 MB/sec execute 171 sec latency 18.798 ms 2024-05-22T23:08:05.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:05 smithi204 ceph-mon[36211]: pgmap v190: 97 pgs: 97 active+clean; 394 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.0 MiB/s wr, 244 op/s 2024-05-22T23:08:06.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:05 smithi096 ceph-mon[36392]: pgmap v190: 97 pgs: 97 active+clean; 394 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.0 MiB/s wr, 244 op/s 2024-05-22T23:08:06.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:05 smithi084 ceph-mon[32186]: pgmap v190: 97 pgs: 97 active+clean; 394 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.0 MiB/s wr, 244 op/s 2024-05-22T23:08:06.438 INFO:tasks.workunit.client.0.smithi084.stdout: 1 828448 15.03 MB/sec execute 172 sec latency 23.649 ms 2024-05-22T23:08:07.439 INFO:tasks.workunit.client.0.smithi084.stdout: 1 831139 15.04 MB/sec execute 173 sec latency 84.757 ms 2024-05-22T23:08:07.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:07 smithi204 ceph-mon[36211]: pgmap v191: 97 pgs: 97 active+clean; 403 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.2 MiB/s wr, 396 op/s 2024-05-22T23:08:07.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:07 smithi204 ceph-mon[36211]: osdmap e90: 12 total, 12 up, 12 in 2024-05-22T23:08:08.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:07 smithi096 ceph-mon[36392]: pgmap v191: 97 pgs: 97 active+clean; 403 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.2 MiB/s wr, 396 op/s 2024-05-22T23:08:08.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:07 smithi096 ceph-mon[36392]: osdmap e90: 12 total, 12 up, 12 in 2024-05-22T23:08:08.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:07 smithi084 ceph-mon[32186]: pgmap v191: 97 pgs: 97 active+clean; 403 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.2 MiB/s wr, 396 op/s 2024-05-22T23:08:08.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:07 smithi084 ceph-mon[32186]: osdmap e90: 12 total, 12 up, 12 in 2024-05-22T23:08:08.439 INFO:tasks.workunit.client.0.smithi084.stdout: 1 834379 15.06 MB/sec execute 174 sec latency 16.823 ms 2024-05-22T23:08:09.439 INFO:tasks.workunit.client.0.smithi084.stdout: 1 836959 15.03 MB/sec execute 175 sec latency 25.552 ms 2024-05-22T23:08:09.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:09 smithi204 ceph-mon[36211]: pgmap v193: 97 pgs: 97 active+clean; 403 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.1 MiB/s wr, 329 op/s 2024-05-22T23:08:10.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:09 smithi096 ceph-mon[36392]: pgmap v193: 97 pgs: 97 active+clean; 403 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.1 MiB/s wr, 329 op/s 2024-05-22T23:08:10.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:09 smithi084 ceph-mon[32186]: pgmap v193: 97 pgs: 97 active+clean; 403 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.1 MiB/s wr, 329 op/s 2024-05-22T23:08:10.439 INFO:tasks.workunit.client.0.smithi084.stdout: 1 839556 15.03 MB/sec execute 176 sec latency 16.087 ms 2024-05-22T23:08:11.439 INFO:tasks.workunit.client.0.smithi084.stdout: 1 842238 15.03 MB/sec execute 177 sec latency 14.277 ms 2024-05-22T23:08:11.918 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:11 smithi204 ceph-mon[36211]: pgmap v194: 97 pgs: 97 active+clean; 403 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.1 MiB/s wr, 329 op/s 2024-05-22T23:08:12.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:11 smithi096 ceph-mon[36392]: pgmap v194: 97 pgs: 97 active+clean; 403 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.1 MiB/s wr, 329 op/s 2024-05-22T23:08:12.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:11 smithi084 ceph-mon[32186]: pgmap v194: 97 pgs: 97 active+clean; 403 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.1 MiB/s wr, 329 op/s 2024-05-22T23:08:12.439 INFO:tasks.workunit.client.0.smithi084.stdout: 1 845133 15.04 MB/sec execute 178 sec latency 77.996 ms 2024-05-22T23:08:13.439 INFO:tasks.workunit.client.0.smithi084.stdout: 1 847999 15.04 MB/sec execute 179 sec latency 17.470 ms 2024-05-22T23:08:13.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:13 smithi204 ceph-mon[36211]: pgmap v195: 97 pgs: 97 active+clean; 414 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.2 MiB/s wr, 325 op/s 2024-05-22T23:08:14.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:13 smithi096 ceph-mon[36392]: pgmap v195: 97 pgs: 97 active+clean; 414 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.2 MiB/s wr, 325 op/s 2024-05-22T23:08:14.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:13 smithi084 ceph-mon[32186]: pgmap v195: 97 pgs: 97 active+clean; 414 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.2 MiB/s wr, 325 op/s 2024-05-22T23:08:14.439 INFO:tasks.workunit.client.0.smithi084.stdout: 1 850953 15.03 MB/sec execute 180 sec latency 14.347 ms 2024-05-22T23:08:15.439 INFO:tasks.workunit.client.0.smithi084.stdout: 1 853219 15.02 MB/sec execute 181 sec latency 14.482 ms 2024-05-22T23:08:15.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:15 smithi204 ceph-mon[36211]: pgmap v196: 97 pgs: 97 active+clean; 414 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.2 MiB/s wr, 325 op/s 2024-05-22T23:08:16.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:15 smithi096 ceph-mon[36392]: pgmap v196: 97 pgs: 97 active+clean; 414 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.2 MiB/s wr, 325 op/s 2024-05-22T23:08:16.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:15 smithi084 ceph-mon[32186]: pgmap v196: 97 pgs: 97 active+clean; 414 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.2 MiB/s wr, 325 op/s 2024-05-22T23:08:16.439 INFO:tasks.workunit.client.0.smithi084.stdout: 1 856202 15.03 MB/sec execute 182 sec latency 15.154 ms 2024-05-22T23:08:17.439 INFO:tasks.workunit.client.0.smithi084.stdout: 1 858826 15.02 MB/sec execute 183 sec latency 106.213 ms 2024-05-22T23:08:17.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:17 smithi204 ceph-mon[36211]: pgmap v197: 97 pgs: 97 active+clean; 425 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 288 op/s 2024-05-22T23:08:17.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:17 smithi204 ceph-mon[36211]: osdmap e91: 12 total, 12 up, 12 in 2024-05-22T23:08:18.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:17 smithi096 ceph-mon[36392]: pgmap v197: 97 pgs: 97 active+clean; 425 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 288 op/s 2024-05-22T23:08:18.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:17 smithi096 ceph-mon[36392]: osdmap e91: 12 total, 12 up, 12 in 2024-05-22T23:08:18.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:17 smithi084 ceph-mon[32186]: pgmap v197: 97 pgs: 97 active+clean; 425 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 288 op/s 2024-05-22T23:08:18.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:17 smithi084 ceph-mon[32186]: osdmap e91: 12 total, 12 up, 12 in 2024-05-22T23:08:18.439 INFO:tasks.workunit.client.0.smithi084.stdout: 1 861078 15.00 MB/sec execute 184 sec latency 20.845 ms 2024-05-22T23:08:19.439 INFO:tasks.workunit.client.0.smithi084.stdout: 1 863547 15.00 MB/sec execute 185 sec latency 23.440 ms 2024-05-22T23:08:19.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:19 smithi204 ceph-mon[36211]: pgmap v199: 97 pgs: 97 active+clean; 425 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 288 op/s 2024-05-22T23:08:20.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:08:20.024 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:08:20.046 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:19 smithi084 ceph-mon[32186]: pgmap v199: 97 pgs: 97 active+clean; 425 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 288 op/s 2024-05-22T23:08:20.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:19 smithi096 ceph-mon[36392]: pgmap v199: 97 pgs: 97 active+clean; 425 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 288 op/s 2024-05-22T23:08:20.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.125+0000 7f73b4f14640 1 -- 172.21.15.84:0/3759830616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73b0106750 msgr2=0x7f73b0106b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:20.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.125+0000 7f73b4f14640 1 --2- 172.21.15.84:0/3759830616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73b0106750 0x7f73b0106b30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009930 tx=0x7f73a002a420 comp rx=0 tx=0).stop 2024-05-22T23:08:20.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.125+0000 7f73b4f14640 1 -- 172.21.15.84:0/3759830616 shutdown_connections 2024-05-22T23:08:20.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.125+0000 7f73b4f14640 1 --2- 172.21.15.84:0/3759830616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73b0101100 0x7f73b010d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.125+0000 7f73b4f14640 1 --2- 172.21.15.84:0/3759830616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73b0100740 0x7f73b0100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.125+0000 7f73b4f14640 1 --2- 172.21.15.84:0/3759830616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73b0106750 0x7f73b0106b30 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.125+0000 7f73b4f14640 1 -- 172.21.15.84:0/3759830616 >> 172.21.15.84:0/3759830616 conn(0x7f73b0077870 msgr2=0x7f73b0077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:20.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.125+0000 7f73b4f14640 1 -- 172.21.15.84:0/3759830616 shutdown_connections 2024-05-22T23:08:20.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.125+0000 7f73b4f14640 1 -- 172.21.15.84:0/3759830616 wait complete. 2024-05-22T23:08:20.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.126+0000 7f73b4f14640 1 Processor -- start 2024-05-22T23:08:20.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.126+0000 7f73b4f14640 1 -- start start 2024-05-22T23:08:20.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.126+0000 7f73b4f14640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73b0100740 0x7f73b0198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.126+0000 7f73b4f14640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73b0101100 0x7f73b0198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.126+0000 7f73b4f14640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73b0106750 0x7f73b019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.126+0000 7f73b4f14640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f73b0110b20 con 0x7f73b0101100 2024-05-22T23:08:20.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.126+0000 7f73b4f14640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f73b01109a0 con 0x7f73b0106750 2024-05-22T23:08:20.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.126+0000 7f73b4f14640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f73b0110ca0 con 0x7f73b0100740 2024-05-22T23:08:20.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.126+0000 7f73a7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73b0101100 0x7f73b0198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.127+0000 7f73a7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73b0101100 0x7f73b0198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36694/0 (socket says 172.21.15.84:36694) 2024-05-22T23:08:20.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.127+0000 7f73a7fff640 1 -- 172.21.15.84:0/1577572168 learned_addr learned my addr 172.21.15.84:0/1577572168 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:20.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.127+0000 7f73aed9d640 1 --2- 172.21.15.84:0/1577572168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73b0106750 0x7f73b019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.127+0000 7f73ae59c640 1 --2- 172.21.15.84:0/1577572168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73b0100740 0x7f73b0198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.127+0000 7f73a7fff640 1 -- 172.21.15.84:0/1577572168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73b0100740 msgr2=0x7f73b0198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:20.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.127+0000 7f73a7fff640 1 --2- 172.21.15.84:0/1577572168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73b0100740 0x7f73b0198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.127+0000 7f73a7fff640 1 -- 172.21.15.84:0/1577572168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73b0106750 msgr2=0x7f73b019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:20.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.127+0000 7f73a7fff640 1 --2- 172.21.15.84:0/1577572168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73b0106750 0x7f73b019caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.127+0000 7f73a7fff640 1 -- 172.21.15.84:0/1577572168 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b019d1d0 con 0x7f73b0101100 2024-05-22T23:08:20.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.127+0000 7f73aed9d640 1 --2- 172.21.15.84:0/1577572168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73b0106750 0x7f73b019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:08:20.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.127+0000 7f73a7fff640 1 --2- 172.21.15.84:0/1577572168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73b0101100 0x7f73b0198770 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f739800e9f0 tx=0x7f739800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:20.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.127+0000 7f73a77fe640 1 -- 172.21.15.84:0/1577572168 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739800cd80 con 0x7f73b0101100 2024-05-22T23:08:20.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.127+0000 7f73a77fe640 1 -- 172.21.15.84:0/1577572168 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7398010910 con 0x7f73b0101100 2024-05-22T23:08:20.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.127+0000 7f73b4f14640 1 -- 172.21.15.84:0/1577572168 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b019d4c0 con 0x7f73b0101100 2024-05-22T23:08:20.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.127+0000 7f73a77fe640 1 -- 172.21.15.84:0/1577572168 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7398018990 con 0x7f73b0101100 2024-05-22T23:08:20.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.127+0000 7f73b4f14640 1 -- 172.21.15.84:0/1577572168 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b01a4d50 con 0x7f73b0101100 2024-05-22T23:08:20.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.129+0000 7f73a77fe640 1 -- 172.21.15.84:0/1577572168 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7398010ac0 con 0x7f73b0101100 2024-05-22T23:08:20.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.130+0000 7f73b7c28740 1 Processor -- start 2024-05-22T23:08:20.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.130+0000 7f73b7c28740 1 -- start start 2024-05-22T23:08:20.143 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.130+0000 7f73b7c28740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a159afd4a0 0x55a159afb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.143 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.130+0000 7f73a77fe640 1 --2- 172.21.15.84:0/1577572168 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f739c0a2d00 0x7f739c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.143 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.130+0000 7f73b7c28740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a159afbdb0 0x55a159afc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.143 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.130+0000 7f73b7c28740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159b06b30 0x55a159b08f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.143 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.130+0000 7f73b7c28740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a159a5fdb0 con 0x55a159b06b30 2024-05-22T23:08:20.143 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.130+0000 7f73b7c28740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a1599fe360 con 0x55a159afbdb0 2024-05-22T23:08:20.143 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.130+0000 7f73b7c28740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a159a5bd70 con 0x55a159afd4a0 2024-05-22T23:08:20.143 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.130+0000 7f73aed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159b06b30 0x55a159b08f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.143 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.130+0000 7f73aed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159b06b30 0x55a159b08f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36708/0 (socket says 172.21.15.84:36708) 2024-05-22T23:08:20.143 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.130+0000 7f73ae59c640 1 --2- 172.21.15.84:0/1577572168 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f739c0a2d00 0x7f739c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.143 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.130+0000 7f73aed9d640 1 -- 172.21.15.84:0/1074012206 learned_addr learned my addr 172.21.15.84:0/1074012206 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:20.143 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.130+0000 7f73a7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a159afbdb0 0x55a159afc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.143 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.131+0000 7f73a77fe640 1 -- 172.21.15.84:0/1577572168 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f73980cf820 con 0x7f73b0101100 2024-05-22T23:08:20.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.131+0000 7f73aed9d640 1 --2- 172.21.15.84:0/1074012206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a159afd4a0 0x55a159afb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.131+0000 7f73a7fff640 1 -- 172.21.15.84:0/1074012206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a159afd4a0 msgr2=0x55a159afb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:20.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.131+0000 7f73a7fff640 1 --2- 172.21.15.84:0/1074012206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a159afd4a0 0x55a159afb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.131+0000 7f73a7fff640 1 -- 172.21.15.84:0/1074012206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159b06b30 msgr2=0x55a159b08f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:20.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.131+0000 7f73a7fff640 1 --2- 172.21.15.84:0/1074012206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159b06b30 0x55a159b08f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.131+0000 7f73a7fff640 1 -- 172.21.15.84:0/1074012206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a159a21a60 con 0x55a159afbdb0 2024-05-22T23:08:20.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.131+0000 7f73aed9d640 1 --2- 172.21.15.84:0/1074012206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a159afd4a0 0x55a159afb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:08:20.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.131+0000 7f73a7fff640 1 --2- 172.21.15.84:0/1074012206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a159afbdb0 0x55a159afc230 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f73980ccf40 tx=0x7f7398025360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=532ac1c5eb85b124 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:20.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.131+0000 7f73ae59c640 1 --2- 172.21.15.84:0/1577572168 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f739c0a2d00 0x7f739c0a51c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f73a0000c00 tx=0x7f73a0005a20 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:20.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.131+0000 7f73a4ff9640 1 -- 172.21.15.84:0/1074012206 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73980a42e0 con 0x55a159afbdb0 2024-05-22T23:08:20.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.131+0000 7f73a4ff9640 1 -- 172.21.15.84:0/1074012206 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f739801ae30 con 0x55a159afbdb0 2024-05-22T23:08:20.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.131+0000 7f73a4ff9640 1 -- 172.21.15.84:0/1074012206 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73980a3680 con 0x55a159afbdb0 2024-05-22T23:08:20.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.132+0000 7f73b7c28740 1 -- 172.21.15.84:0/1074012206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a159afbdb0 msgr2=0x55a159afc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:20.144 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.132+0000 7f73b7c28740 1 --2- 172.21.15.84:0/1074012206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a159afbdb0 0x55a159afc230 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f73980ccf40 tx=0x7f7398025360 comp rx=0 tx=0).stop 2024-05-22T23:08:20.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.132+0000 7f73b7c28740 1 -- 172.21.15.84:0/1074012206 shutdown_connections 2024-05-22T23:08:20.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.132+0000 7f73b7c28740 1 --2- 172.21.15.84:0/1074012206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159b06b30 0x55a159b08f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.132+0000 7f73b7c28740 1 --2- 172.21.15.84:0/1074012206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a159afbdb0 0x55a159afc230 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.132+0000 7f73b7c28740 1 --2- 172.21.15.84:0/1074012206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a159afd4a0 0x55a159afb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.132+0000 7f73b7c28740 1 -- 172.21.15.84:0/1074012206 >> 172.21.15.84:0/1074012206 conn(0x55a159ac48e0 msgr2=0x55a159af8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:20.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.132+0000 7f73b7c28740 1 -- 172.21.15.84:0/1074012206 shutdown_connections 2024-05-22T23:08:20.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.132+0000 7f73b7c28740 1 -- 172.21.15.84:0/1074012206 wait complete. 2024-05-22T23:08:20.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.132+0000 7f73b7c28740 1 -- start start 2024-05-22T23:08:20.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.132+0000 7f73b7c28740 1 Processor -- start 2024-05-22T23:08:20.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f73b7c28740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a159afbdb0 0x55a159b03a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f73b7c28740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159afd4a0 0x55a159b03f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f73b7c28740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a159b06b30 0x55a159b04490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f73b7c28740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a159b099c0 con 0x55a159afd4a0 2024-05-22T23:08:20.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f73b7c28740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a159a1a0f0 con 0x55a159afbdb0 2024-05-22T23:08:20.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f73b7c28740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a159b09b40 con 0x55a159b06b30 2024-05-22T23:08:20.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f73b7c28740 10 client.0 _ll_register_callbacks cb 0x55a159a1ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:08:20.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f73a7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159afd4a0 0x55a159b03f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f73a7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159afd4a0 0x55a159b03f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36714/0 (socket says 172.21.15.84:36714) 2024-05-22T23:08:20.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f73a7fff640 1 -- 172.21.15.84:0/3102722512 learned_addr learned my addr 172.21.15.84:0/3102722512 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:20.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f73ae59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a159afbdb0 0x55a159b03a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f73aed9d640 1 --2- 172.21.15.84:0/3102722512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a159b06b30 0x55a159b04490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f73a7fff640 1 -- 172.21.15.84:0/3102722512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a159b06b30 msgr2=0x55a159b04490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:20.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f73a7fff640 1 --2- 172.21.15.84:0/3102722512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a159b06b30 0x55a159b04490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f73a7fff640 1 -- 172.21.15.84:0/3102722512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a159afbdb0 msgr2=0x55a159b03a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:20.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f73a7fff640 1 --2- 172.21.15.84:0/3102722512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a159afbdb0 0x55a159b03a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f73a7fff640 1 -- 172.21.15.84:0/3102722512 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a15983d7d0 con 0x55a159afd4a0 2024-05-22T23:08:20.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f73aed9d640 1 --2- 172.21.15.84:0/3102722512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a159b06b30 0x55a159b04490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:08:20.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f73a7fff640 1 --2- 172.21.15.84:0/3102722512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159afd4a0 0x55a159b03f50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f7398098170 tx=0x7f73980a4350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5182c4c37e1181 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:20.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f7395ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:08:20.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f7395ffb640 1 -- 172.21.15.84:0/3102722512 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73980a40f0 con 0x55a159afd4a0 2024-05-22T23:08:20.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f7395ffb640 1 -- 172.21.15.84:0/3102722512 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f73980a3050 con 0x55a159afd4a0 2024-05-22T23:08:20.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f73b7c28740 1 -- 172.21.15.84:0/3102722512 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f739801ae30 con 0x55a159afd4a0 2024-05-22T23:08:20.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f7395ffb640 1 -- 172.21.15.84:0/3102722512 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f73980a00c0 con 0x55a159afd4a0 2024-05-22T23:08:20.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f73b7c28740 10 client.16416 fetch_fsmap learned FSMap version 18 2024-05-22T23:08:20.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.133+0000 7f73b7c28740 1 -- 172.21.15.84:0/3102722512 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55a1597733f0 con 0x55a159afd4a0 2024-05-22T23:08:20.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.134+0000 7f7395ffb640 1 -- 172.21.15.84:0/3102722512 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f739801ae30 con 0x55a159afd4a0 2024-05-22T23:08:20.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.134+0000 7f7395ffb640 1 -- 172.21.15.84:0/3102722512 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f739805e070 con 0x55a159afd4a0 2024-05-22T23:08:20.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.134+0000 7f73b7c28740 10 client.16416 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:08:20.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.134+0000 7f73b7c28740 10 client.16416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:08:20.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.134+0000 7f73b7c28740 20 client.16416 populate_metadata read hostname 'smithi084' 2024-05-22T23:08:20.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.134+0000 7f73b7c28740 1 --2- 172.21.15.84:0/3102722512 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55a159b16e30 0x55a159b19220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.134+0000 7f73b7c28740 4 client.16416 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:08:20.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.134+0000 7f73b7c28740 1 -- 172.21.15.84:0/3102722512 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a159a6d4e0 con 0x55a159b16e30 2024-05-22T23:08:20.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.134+0000 7f73ae59c640 1 --2- 172.21.15.84:0/3102722512 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55a159b16e30 0x55a159b19220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.134+0000 7f7395ffb640 10 client.16416 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:20.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.134+0000 7f73ae59c640 1 --2- 172.21.15.84:0/3102722512 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55a159b16e30 0x55a159b19220 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8364e08d9a13a3b9 server_cookie=52cc918b9d9dd05a in_seq=0 out_seq=0 2024-05-22T23:08:20.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.135+0000 7f7395ffb640 1 -- 172.21.15.84:0/3102722512 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a159a6d4e0 con 0x55a159b16e30 2024-05-22T23:08:20.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.135+0000 7f7395ffb640 10 client.16416 handle_command_reply: tid=0 2024-05-22T23:08:20.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.135+0000 7f73b7c28740 1 client.16416 shutdown 2024-05-22T23:08:20.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.135+0000 7f73b7c28740 1 -- 172.21.15.84:0/3102722512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159afd4a0 msgr2=0x55a159b03f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:20.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.135+0000 7f73b7c28740 1 --2- 172.21.15.84:0/3102722512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159afd4a0 0x55a159b03f50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f7398098170 tx=0x7f73980a4350 comp rx=0 tx=0).stop 2024-05-22T23:08:20.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.135+0000 7f73b7c28740 1 -- 172.21.15.84:0/3102722512 shutdown_connections 2024-05-22T23:08:20.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.135+0000 7f73b7c28740 1 --2- 172.21.15.84:0/3102722512 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55a159b16e30 0x55a159b19220 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.149 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.135+0000 7f73b7c28740 1 --2- 172.21.15.84:0/3102722512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a159b06b30 0x55a159b04490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.149 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.135+0000 7f7395ffb640 0 client.16416 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:20.149 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.135+0000 7f73b7c28740 1 --2- 172.21.15.84:0/3102722512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159afd4a0 0x55a159b03f50 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.149 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.135+0000 7f73b7c28740 1 --2- 172.21.15.84:0/3102722512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a159afbdb0 0x55a159b03a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.149 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.135+0000 7f73b7c28740 1 -- 172.21.15.84:0/3102722512 >> 172.21.15.84:0/3102722512 conn(0x55a159ac48e0 msgr2=0x55a159afe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:20.149 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.136+0000 7f73b7c28740 1 -- 172.21.15.84:0/3102722512 shutdown_connections 2024-05-22T23:08:20.149 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.136+0000 7f73b7c28740 1 -- 172.21.15.84:0/3102722512 wait complete. 2024-05-22T23:08:20.149 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.136+0000 7f73b7c28740 20 client.16416 trim_cache size 0 max 16384 2024-05-22T23:08:20.164 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.154+0000 7f73b7c28740 1 Processor -- start 2024-05-22T23:08:20.164 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.154+0000 7f73b7c28740 1 -- start start 2024-05-22T23:08:20.164 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.154+0000 7f73b7c28740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a159afbdb0 0x55a159b128f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.164 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.154+0000 7f73b7c28740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a159afd4a0 0x55a159b1e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.154+0000 7f73b7c28740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159b06b30 0x55a159b1ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.154+0000 7f73b7c28740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f73b01ab720 con 0x55a159b06b30 2024-05-22T23:08:20.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.154+0000 7f73b7c28740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f73b01adf80 con 0x55a159afbdb0 2024-05-22T23:08:20.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.154+0000 7f73b7c28740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f73b01abd70 con 0x55a159afd4a0 2024-05-22T23:08:20.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.154+0000 7f73aed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159b06b30 0x55a159b1ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.154+0000 7f73aed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159b06b30 0x55a159b1ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36716/0 (socket says 172.21.15.84:36716) 2024-05-22T23:08:20.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.154+0000 7f73aed9d640 1 -- 172.21.15.84:0/631959303 learned_addr learned my addr 172.21.15.84:0/631959303 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:20.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.154+0000 7f73ae59c640 1 --2- 172.21.15.84:0/631959303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a159afbdb0 0x55a159b128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.155+0000 7f73a7fff640 1 --2- 172.21.15.84:0/631959303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a159afd4a0 0x55a159b1e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.155+0000 7f73aed9d640 1 -- 172.21.15.84:0/631959303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a159afd4a0 msgr2=0x55a159b1e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:20.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.155+0000 7f73aed9d640 1 --2- 172.21.15.84:0/631959303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a159afd4a0 0x55a159b1e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.155+0000 7f73aed9d640 1 -- 172.21.15.84:0/631959303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a159afbdb0 msgr2=0x55a159b128f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:20.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.155+0000 7f73aed9d640 1 --2- 172.21.15.84:0/631959303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a159afbdb0 0x55a159b128f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.155+0000 7f73aed9d640 1 -- 172.21.15.84:0/631959303 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1597330f0 con 0x55a159b06b30 2024-05-22T23:08:20.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.155+0000 7f73a7fff640 1 --2- 172.21.15.84:0/631959303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a159afd4a0 0x55a159b1e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:08:20.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.155+0000 7f73aed9d640 1 --2- 172.21.15.84:0/631959303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159b06b30 0x55a159b1ed60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f73a8003ca0 tx=0x7f73a8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8a0e99cf63c5b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:20.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.155+0000 7f7397fff640 1 -- 172.21.15.84:0/631959303 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a8004540 con 0x55a159b06b30 2024-05-22T23:08:20.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.155+0000 7f7397fff640 1 -- 172.21.15.84:0/631959303 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a80046e0 con 0x55a159b06b30 2024-05-22T23:08:20.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.155+0000 7f7397fff640 1 -- 172.21.15.84:0/631959303 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a8005990 con 0x55a159b06b30 2024-05-22T23:08:20.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.156+0000 7f73b7c28740 1 -- 172.21.15.84:0/631959303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159b06b30 msgr2=0x55a159b1ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:20.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.156+0000 7f73b7c28740 1 --2- 172.21.15.84:0/631959303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159b06b30 0x55a159b1ed60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f73a8003ca0 tx=0x7f73a8004ab0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.156+0000 7f73b7c28740 1 -- 172.21.15.84:0/631959303 shutdown_connections 2024-05-22T23:08:20.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.156+0000 7f73b7c28740 1 --2- 172.21.15.84:0/631959303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159b06b30 0x55a159b1ed60 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.156+0000 7f73b7c28740 1 --2- 172.21.15.84:0/631959303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a159afd4a0 0x55a159b1e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.156+0000 7f73b7c28740 1 --2- 172.21.15.84:0/631959303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a159afbdb0 0x55a159b128f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.156+0000 7f73b7c28740 1 -- 172.21.15.84:0/631959303 >> 172.21.15.84:0/631959303 conn(0x55a159ac48e0 msgr2=0x55a159b05ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:20.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.156+0000 7f73b7c28740 1 -- 172.21.15.84:0/631959303 shutdown_connections 2024-05-22T23:08:20.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.156+0000 7f73b7c28740 1 -- 172.21.15.84:0/631959303 wait complete. 2024-05-22T23:08:20.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.156+0000 7f73b7c28740 1 -- start start 2024-05-22T23:08:20.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.156+0000 7f73b7c28740 1 Processor -- start 2024-05-22T23:08:20.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.156+0000 7f73b7c28740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a159afbdb0 0x55a159b0ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.156+0000 7f73b7c28740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159afd4a0 0x55a159b0f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.156+0000 7f73b7c28740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a159b06b30 0x55a159b00240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.156+0000 7f73b7c28740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f73b01ac440 con 0x55a159afd4a0 2024-05-22T23:08:20.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.156+0000 7f73b7c28740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f73b01ad8b0 con 0x55a159afbdb0 2024-05-22T23:08:20.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.156+0000 7f73b7c28740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f73b01acb10 con 0x55a159b06b30 2024-05-22T23:08:20.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.156+0000 7f73b7c28740 10 client.0 _ll_register_callbacks cb 0x7f73a8009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:08:20.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.156+0000 7f73a7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159afd4a0 0x55a159b0f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.156+0000 7f73a7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159afd4a0 0x55a159b0f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36726/0 (socket says 172.21.15.84:36726) 2024-05-22T23:08:20.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.156+0000 7f73ae59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a159afbdb0 0x55a159b0ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.156+0000 7f73a7fff640 1 -- 172.21.15.84:0/3990684 learned_addr learned my addr 172.21.15.84:0/3990684 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:20.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.156+0000 7f73aed9d640 1 --2- 172.21.15.84:0/3990684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a159b06b30 0x55a159b00240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.157+0000 7f73a7fff640 1 -- 172.21.15.84:0/3990684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a159b06b30 msgr2=0x55a159b00240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:20.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.157+0000 7f73a7fff640 1 --2- 172.21.15.84:0/3990684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a159b06b30 0x55a159b00240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.157+0000 7f73a7fff640 1 -- 172.21.15.84:0/3990684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a159afbdb0 msgr2=0x55a159b0ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:20.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.157+0000 7f73a7fff640 1 --2- 172.21.15.84:0/3990684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a159afbdb0 0x55a159b0ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.157+0000 7f73a7fff640 1 -- 172.21.15.84:0/3990684 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a159a1efe0 con 0x55a159afd4a0 2024-05-22T23:08:20.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.157+0000 7f73a7fff640 1 --2- 172.21.15.84:0/3990684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159afd4a0 0x55a159b0f340 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7398099fc0 tx=0x7f7398009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aaffbef04cbe62d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:20.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.157+0000 7f7395ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:08:20.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.157+0000 7f7395ffb640 1 -- 172.21.15.84:0/3990684 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739809b670 con 0x55a159afd4a0 2024-05-22T23:08:20.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.157+0000 7f7395ffb640 1 -- 172.21.15.84:0/3990684 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f73980a44f0 con 0x55a159afd4a0 2024-05-22T23:08:20.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.157+0000 7f73b7c28740 1 -- 172.21.15.84:0/3990684 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73a80046e0 con 0x55a159afd4a0 2024-05-22T23:08:20.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.157+0000 7f7395ffb640 1 -- 172.21.15.84:0/3990684 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f739801ae30 con 0x55a159afd4a0 2024-05-22T23:08:20.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.157+0000 7f73b7c28740 10 client.16422 fetch_fsmap learned FSMap version 18 2024-05-22T23:08:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.157+0000 7f73b7c28740 1 -- 172.21.15.84:0/3990684 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55a1599f01c0 con 0x55a159afd4a0 2024-05-22T23:08:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.157+0000 7f7395ffb640 1 -- 172.21.15.84:0/3990684 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a80046e0 con 0x55a159afd4a0 2024-05-22T23:08:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.157+0000 7f7395ffb640 1 -- 172.21.15.84:0/3990684 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f73980aa030 con 0x55a159afd4a0 2024-05-22T23:08:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.157+0000 7f73b7c28740 10 client.16422 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:08:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.157+0000 7f73b7c28740 10 client.16422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:08:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.157+0000 7f73b7c28740 20 client.16422 populate_metadata read hostname 'smithi084' 2024-05-22T23:08:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.157+0000 7f73b7c28740 1 --2- 172.21.15.84:0/3990684 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55a159b7cd40 0x55a159b7f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.157+0000 7f73b7c28740 4 client.16422 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:08:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.157+0000 7f73b7c28740 1 -- 172.21.15.84:0/3990684 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a159a67830 con 0x55a159b7cd40 2024-05-22T23:08:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.158+0000 7f73ae59c640 1 --2- 172.21.15.84:0/3990684 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55a159b7cd40 0x55a159b7f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.158+0000 7f73ae59c640 1 --2- 172.21.15.84:0/3990684 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55a159b7cd40 0x55a159b7f160 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a185cdd62904b4bb server_cookie=bbe9305da52ae31d in_seq=0 out_seq=0 2024-05-22T23:08:20.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.158+0000 7f7395ffb640 10 client.16422 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:20.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.160+0000 7f7395ffb640 1 -- 172.21.15.84:0/3990684 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a159a67830 con 0x55a159b7cd40 2024-05-22T23:08:20.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.160+0000 7f7395ffb640 10 client.16422 handle_command_reply: tid=0 2024-05-22T23:08:20.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.160+0000 7f73b7c28740 1 client.16422 shutdown 2024-05-22T23:08:20.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.160+0000 7f73b7c28740 1 -- 172.21.15.84:0/3990684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159afd4a0 msgr2=0x55a159b0f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:20.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.160+0000 7f73b7c28740 1 --2- 172.21.15.84:0/3990684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159afd4a0 0x55a159b0f340 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7398099fc0 tx=0x7f7398009770 comp rx=0 tx=0).stop 2024-05-22T23:08:20.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.160+0000 7f73b7c28740 1 -- 172.21.15.84:0/3990684 shutdown_connections 2024-05-22T23:08:20.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.160+0000 7f73b7c28740 1 --2- 172.21.15.84:0/3990684 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55a159b7cd40 0x55a159b7f160 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.160+0000 7f73b7c28740 1 --2- 172.21.15.84:0/3990684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a159b06b30 0x55a159b00240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.160+0000 7f73b7c28740 1 --2- 172.21.15.84:0/3990684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a159afd4a0 0x55a159b0f340 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.160+0000 7f7395ffb640 0 client.16422 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:20.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.160+0000 7f73b7c28740 1 --2- 172.21.15.84:0/3990684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a159afbdb0 0x55a159b0ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.160+0000 7f73b7c28740 1 -- 172.21.15.84:0/3990684 >> 172.21.15.84:0/3990684 conn(0x55a159ac48e0 msgr2=0x55a159b021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:20.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.160+0000 7f73b7c28740 1 -- 172.21.15.84:0/3990684 shutdown_connections 2024-05-22T23:08:20.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.160+0000 7f73b7c28740 1 -- 172.21.15.84:0/3990684 wait complete. 2024-05-22T23:08:20.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.160+0000 7f73b7c28740 20 client.16422 trim_cache size 0 max 16384 2024-05-22T23:08:20.172 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:08:20.172 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "no active scrubs running", 2024-05-22T23:08:20.172 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": {} 2024-05-22T23:08:20.172 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:08:20.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.161+0000 7f73b4f14640 1 -- 172.21.15.84:0/1577572168 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f739c0a2d00 msgr2=0x7f739c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:20.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.161+0000 7f73b4f14640 1 --2- 172.21.15.84:0/1577572168 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f739c0a2d00 0x7f739c0a51c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f73a0000c00 tx=0x7f73a0005a20 comp rx=0 tx=0).stop 2024-05-22T23:08:20.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.161+0000 7f73b4f14640 1 -- 172.21.15.84:0/1577572168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73b0101100 msgr2=0x7f73b0198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:20.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.161+0000 7f73b4f14640 1 --2- 172.21.15.84:0/1577572168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73b0101100 0x7f73b0198770 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f739800e9f0 tx=0x7f739800eeb0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.161+0000 7f73b4f14640 1 -- 172.21.15.84:0/1577572168 shutdown_connections 2024-05-22T23:08:20.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.161+0000 7f73b4f14640 1 --2- 172.21.15.84:0/1577572168 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f739c0a2d00 0x7f739c0a51c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.161+0000 7f73b4f14640 1 --2- 172.21.15.84:0/1577572168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73b0106750 0x7f73b019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.161+0000 7f73b4f14640 1 --2- 172.21.15.84:0/1577572168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73b0101100 0x7f73b0198770 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.161+0000 7f73b4f14640 1 --2- 172.21.15.84:0/1577572168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73b0100740 0x7f73b0198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.161+0000 7f73b4f14640 1 -- 172.21.15.84:0/1577572168 >> 172.21.15.84:0/1577572168 conn(0x7f73b0077870 msgr2=0x7f73b0068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:20.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.161+0000 7f73b4f14640 1 -- 172.21.15.84:0/1577572168 shutdown_connections 2024-05-22T23:08:20.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.161+0000 7f73b4f14640 1 -- 172.21.15.84:0/1577572168 wait complete. 2024-05-22T23:08:20.177 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-22T23:08:20.177 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:08:20.323 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.315+0000 7f34be631640 1 -- 172.21.15.84:0/682872385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34b8103420 msgr2=0x7f34b810b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:20.324 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.315+0000 7f34be631640 1 --2- 172.21.15.84:0/682872385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34b8103420 0x7f34b810b900 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009900 tx=0x7f34ac02a510 comp rx=0 tx=0).stop 2024-05-22T23:08:20.324 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.315+0000 7f34be631640 1 -- 172.21.15.84:0/682872385 shutdown_connections 2024-05-22T23:08:20.324 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.315+0000 7f34be631640 1 --2- 172.21.15.84:0/682872385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34b810be40 0x7f34b810e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.324 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.315+0000 7f34be631640 1 --2- 172.21.15.84:0/682872385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34b8103420 0x7f34b810b900 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.324 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.315+0000 7f34be631640 1 --2- 172.21.15.84:0/682872385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34b8102b00 0x7f34b8102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.324 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.316+0000 7f34be631640 1 -- 172.21.15.84:0/682872385 >> 172.21.15.84:0/682872385 conn(0x7f34b8077870 msgr2=0x7f34b8077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:20.324 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.316+0000 7f34be631640 1 -- 172.21.15.84:0/682872385 shutdown_connections 2024-05-22T23:08:20.324 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.316+0000 7f34be631640 1 -- 172.21.15.84:0/682872385 wait complete. 2024-05-22T23:08:20.324 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.316+0000 7f34be631640 1 Processor -- start 2024-05-22T23:08:20.324 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.316+0000 7f34be631640 1 -- start start 2024-05-22T23:08:20.324 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.316+0000 7f34be631640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34b8102b00 0x7f34b8100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.324 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.316+0000 7f34be631640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34b8103420 0x7f34b8100c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.324 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.316+0000 7f34be631640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34b810be40 0x7f34b81011d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.316+0000 7f34be631640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f34b8111c80 con 0x7f34b8102b00 2024-05-22T23:08:20.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.316+0000 7f34be631640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f34b8111b00 con 0x7f34b810be40 2024-05-22T23:08:20.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.316+0000 7f34be631640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f34b8111e00 con 0x7f34b8103420 2024-05-22T23:08:20.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.317+0000 7f34b7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34b8102b00 0x7f34b8100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.317+0000 7f34b7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34b8102b00 0x7f34b8100750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36758/0 (socket says 172.21.15.84:36758) 2024-05-22T23:08:20.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.317+0000 7f34b7fff640 1 -- 172.21.15.84:0/4240861739 learned_addr learned my addr 172.21.15.84:0/4240861739 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:20.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.317+0000 7f34bcbce640 1 --2- 172.21.15.84:0/4240861739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34b810be40 0x7f34b81011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.317+0000 7f34b77fe640 1 --2- 172.21.15.84:0/4240861739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34b8103420 0x7f34b8100c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.317+0000 7f34b7fff640 1 -- 172.21.15.84:0/4240861739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34b8103420 msgr2=0x7f34b8100c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:20.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.317+0000 7f34b7fff640 1 --2- 172.21.15.84:0/4240861739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34b8103420 0x7f34b8100c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.317+0000 7f34b7fff640 1 -- 172.21.15.84:0/4240861739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34b810be40 msgr2=0x7f34b81011d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:20.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.317+0000 7f34b7fff640 1 --2- 172.21.15.84:0/4240861739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34b810be40 0x7f34b81011d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.317+0000 7f34b7fff640 1 -- 172.21.15.84:0/4240861739 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b81a6640 con 0x7f34b8102b00 2024-05-22T23:08:20.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.317+0000 7f34b77fe640 1 --2- 172.21.15.84:0/4240861739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34b8103420 0x7f34b8100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:08:20.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.317+0000 7f34b7fff640 1 --2- 172.21.15.84:0/4240861739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34b8102b00 0x7f34b8100750 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f34a800e9f0 tx=0x7f34a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:20.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.317+0000 7f34b57fa640 1 -- 172.21.15.84:0/4240861739 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a800cd80 con 0x7f34b8102b00 2024-05-22T23:08:20.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.317+0000 7f34be631640 1 -- 172.21.15.84:0/4240861739 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b81a6930 con 0x7f34b8102b00 2024-05-22T23:08:20.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.317+0000 7f34b57fa640 1 -- 172.21.15.84:0/4240861739 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34a8010910 con 0x7f34b8102b00 2024-05-22T23:08:20.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.317+0000 7f34b57fa640 1 -- 172.21.15.84:0/4240861739 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a80188f0 con 0x7f34b8102b00 2024-05-22T23:08:20.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.317+0000 7f34be631640 1 -- 172.21.15.84:0/4240861739 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b81a6ed0 con 0x7f34b8102b00 2024-05-22T23:08:20.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.318+0000 7f34be631640 1 -- 172.21.15.84:0/4240861739 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b8068690 con 0x7f34b8102b00 2024-05-22T23:08:20.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.319+0000 7f34b57fa640 1 -- 172.21.15.84:0/4240861739 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f34a8010ac0 con 0x7f34b8102b00 2024-05-22T23:08:20.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.320+0000 7f34b57fa640 1 --2- 172.21.15.84:0/4240861739 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f34a00a2d30 0x7f34a00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.320+0000 7f34b77fe640 1 --2- 172.21.15.84:0/4240861739 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f34a00a2d30 0x7f34a00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.320+0000 7f34b57fa640 1 -- 172.21.15.84:0/4240861739 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f34a80cfc90 con 0x7f34b8102b00 2024-05-22T23:08:20.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.321+0000 7f34b77fe640 1 --2- 172.21.15.84:0/4240861739 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f34a00a2d30 0x7f34a00a51f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f34ac000c00 tx=0x7f34ac032c70 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:20.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.324+0000 7f34b57fa640 1 -- 172.21.15.84:0/4240861739 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f34a809c880 con 0x7f34b8102b00 2024-05-22T23:08:20.439 INFO:tasks.workunit.client.0.smithi084.stdout: 1 866652 15.02 MB/sec execute 186 sec latency 13.336 ms 2024-05-22T23:08:20.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.548+0000 7f34be631640 1 -- 172.21.15.84:0/4240861739 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34b8106fc0 con 0x7f34b8102b00 2024-05-22T23:08:20.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.548+0000 7f34b57fa640 1 -- 172.21.15.84:0/4240861739 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7f34a809c180 con 0x7f34b8102b00 2024-05-22T23:08:20.554 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:08:20.555 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:08:20.555 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:08:20.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.550+0000 7f34be631640 1 -- 172.21.15.84:0/4240861739 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f34a00a2d30 msgr2=0x7f34a00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:20.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.550+0000 7f34be631640 1 --2- 172.21.15.84:0/4240861739 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f34a00a2d30 0x7f34a00a51f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f34ac000c00 tx=0x7f34ac032c70 comp rx=0 tx=0).stop 2024-05-22T23:08:20.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.550+0000 7f34be631640 1 -- 172.21.15.84:0/4240861739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34b8102b00 msgr2=0x7f34b8100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:20.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.550+0000 7f34be631640 1 --2- 172.21.15.84:0/4240861739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34b8102b00 0x7f34b8100750 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f34a800e9f0 tx=0x7f34a800eeb0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.550+0000 7f34be631640 1 -- 172.21.15.84:0/4240861739 shutdown_connections 2024-05-22T23:08:20.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.550+0000 7f34be631640 1 --2- 172.21.15.84:0/4240861739 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f34a00a2d30 0x7f34a00a51f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.550+0000 7f34be631640 1 --2- 172.21.15.84:0/4240861739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34b810be40 0x7f34b81011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.550+0000 7f34be631640 1 --2- 172.21.15.84:0/4240861739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34b8103420 0x7f34b8100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.550+0000 7f34be631640 1 --2- 172.21.15.84:0/4240861739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34b8102b00 0x7f34b8100750 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.550+0000 7f34be631640 1 -- 172.21.15.84:0/4240861739 >> 172.21.15.84:0/4240861739 conn(0x7f34b8077870 msgr2=0x7f34b80fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:20.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.550+0000 7f34be631640 1 -- 172.21.15.84:0/4240861739 shutdown_connections 2024-05-22T23:08:20.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.550+0000 7f34be631640 1 -- 172.21.15.84:0/4240861739 wait complete. 2024-05-22T23:08:20.567 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-22T23:08:20.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.667+0000 7f82a3fff640 1 -- 172.21.15.84:0/3075757531 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82940319b0 con 0x7f82a410be30 2024-05-22T23:08:20.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.668+0000 7f82ab542640 1 -- 172.21.15.84:0/3075757531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82a410be30 msgr2=0x7f82a410e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:20.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.668+0000 7f82ab542640 1 --2- 172.21.15.84:0/3075757531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82a410be30 0x7f82a410e2d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8294009900 tx=0x7f829402a510 comp rx=0 tx=0).stop 2024-05-22T23:08:20.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.668+0000 7f82ab542640 1 -- 172.21.15.84:0/3075757531 shutdown_connections 2024-05-22T23:08:20.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.668+0000 7f82ab542640 1 --2- 172.21.15.84:0/3075757531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82a410be30 0x7f82a410e2d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.668+0000 7f82ab542640 1 --2- 172.21.15.84:0/3075757531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82a4103410 0x7f82a410b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.668+0000 7f82ab542640 1 --2- 172.21.15.84:0/3075757531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82a4102af0 0x7f82a4102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.668+0000 7f82ab542640 1 -- 172.21.15.84:0/3075757531 >> 172.21.15.84:0/3075757531 conn(0x7f82a4077870 msgr2=0x7f82a4077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:20.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.668+0000 7f82ab542640 1 -- 172.21.15.84:0/3075757531 shutdown_connections 2024-05-22T23:08:20.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.668+0000 7f82ab542640 1 -- 172.21.15.84:0/3075757531 wait complete. 2024-05-22T23:08:20.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.668+0000 7f82ab542640 1 Processor -- start 2024-05-22T23:08:20.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.668+0000 7f82ab542640 1 -- start start 2024-05-22T23:08:20.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.669+0000 7f82ab542640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82a4102af0 0x7f82a41981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.669+0000 7f82ab542640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82a4103410 0x7f82a4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.669+0000 7f82ab542640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82a410be30 0x7f82a419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.669+0000 7f82ab542640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f82a4111c90 con 0x7f82a4103410 2024-05-22T23:08:20.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.669+0000 7f82ab542640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82a4111b10 con 0x7f82a4102af0 2024-05-22T23:08:20.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.669+0000 7f82ab542640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82a4111e10 con 0x7f82a410be30 2024-05-22T23:08:20.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.669+0000 7f82a8add640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82a4103410 0x7f82a4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.669+0000 7f82a92de640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82a4102af0 0x7f82a41981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.669+0000 7f82a8add640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82a4103410 0x7f82a4198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36780/0 (socket says 172.21.15.84:36780) 2024-05-22T23:08:20.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.669+0000 7f82a8add640 1 -- 172.21.15.84:0/4017888363 learned_addr learned my addr 172.21.15.84:0/4017888363 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:20.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.669+0000 7f82a9adf640 1 --2- 172.21.15.84:0/4017888363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82a410be30 0x7f82a419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.669+0000 7f82a8add640 1 -- 172.21.15.84:0/4017888363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82a410be30 msgr2=0x7f82a419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:20.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.669+0000 7f82a8add640 1 --2- 172.21.15.84:0/4017888363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82a410be30 0x7f82a419ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.669+0000 7f82a8add640 1 -- 172.21.15.84:0/4017888363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82a4102af0 msgr2=0x7f82a41981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:20.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.669+0000 7f82a8add640 1 --2- 172.21.15.84:0/4017888363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82a4102af0 0x7f82a41981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.669+0000 7f82a8add640 1 -- 172.21.15.84:0/4017888363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a419d170 con 0x7f82a4103410 2024-05-22T23:08:20.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.669+0000 7f82a8add640 1 --2- 172.21.15.84:0/4017888363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82a4103410 0x7f82a4198710 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f829800ea80 tx=0x7f829800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:20.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.669+0000 7f82a27fc640 1 -- 172.21.15.84:0/4017888363 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829800cd80 con 0x7f82a4103410 2024-05-22T23:08:20.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.669+0000 7f82a27fc640 1 -- 172.21.15.84:0/4017888363 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8298005020 con 0x7f82a4103410 2024-05-22T23:08:20.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.669+0000 7f82ab542640 1 -- 172.21.15.84:0/4017888363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a419d460 con 0x7f82a4103410 2024-05-22T23:08:20.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.669+0000 7f82a27fc640 1 -- 172.21.15.84:0/4017888363 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82980116d0 con 0x7f82a4103410 2024-05-22T23:08:20.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.669+0000 7f82ab542640 1 -- 172.21.15.84:0/4017888363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a41a4cf0 con 0x7f82a4103410 2024-05-22T23:08:20.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.672+0000 7f82a27fc640 1 -- 172.21.15.84:0/4017888363 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8298005560 con 0x7f82a4103410 2024-05-22T23:08:20.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.672+0000 7f82ae253740 1 Processor -- start 2024-05-22T23:08:20.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.672+0000 7f82ae253740 1 -- start start 2024-05-22T23:08:20.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.672+0000 7f82ae253740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6260cd0 0x5649c62610b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.672+0000 7f82ae253740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6261680 0x5649c626fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.672+0000 7f82ae253740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649c6270380 0x5649c6272770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.672+0000 7f82ae253740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5649c61c5db0 con 0x5649c6260cd0 2024-05-22T23:08:20.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.672+0000 7f82ae253740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5649c6164400 con 0x5649c6261680 2024-05-22T23:08:20.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.672+0000 7f82ae253740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5649c61c1d70 con 0x5649c6270380 2024-05-22T23:08:20.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.673+0000 7f82a9adf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6260cd0 0x5649c62610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.673+0000 7f82a92de640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6261680 0x5649c626fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.673+0000 7f82a9adf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6260cd0 0x5649c62610b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36784/0 (socket says 172.21.15.84:36784) 2024-05-22T23:08:20.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.673+0000 7f82a9adf640 1 -- 172.21.15.84:0/734235093 learned_addr learned my addr 172.21.15.84:0/734235093 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:20.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.673+0000 7f82a8add640 1 --2- 172.21.15.84:0/734235093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649c6270380 0x5649c6272770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.673+0000 7f82a27fc640 1 --2- 172.21.15.84:0/4017888363 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f828c0a2d00 0x7f828c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.673+0000 7f82a92de640 1 -- 172.21.15.84:0/734235093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649c6270380 msgr2=0x5649c6272770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:20.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.673+0000 7f82a92de640 1 --2- 172.21.15.84:0/734235093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649c6270380 0x5649c6272770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.673+0000 7f82a92de640 1 -- 172.21.15.84:0/734235093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6260cd0 msgr2=0x5649c62610b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:20.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.673+0000 7f82a92de640 1 --2- 172.21.15.84:0/734235093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6260cd0 0x5649c62610b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.673+0000 7f82a92de640 1 -- 172.21.15.84:0/734235093 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649c61562f0 con 0x5649c6261680 2024-05-22T23:08:20.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.673+0000 7f82a9adf640 1 --2- 172.21.15.84:0/4017888363 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f828c0a2d00 0x7f828c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.673+0000 7f82a27fc640 1 -- 172.21.15.84:0/4017888363 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f82980cfa90 con 0x7f82a4103410 2024-05-22T23:08:20.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.673+0000 7f82a92de640 1 --2- 172.21.15.84:0/734235093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6261680 0x5649c626fe40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f829c0099a0 tx=0x7f829c009e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7870da70906879ce server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:20.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.673+0000 7f828b7fe640 1 -- 172.21.15.84:0/734235093 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c017070 con 0x5649c6261680 2024-05-22T23:08:20.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.673+0000 7f82a9adf640 1 --2- 172.21.15.84:0/4017888363 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f828c0a2d00 0x7f828c0a51c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8294031550 tx=0x7f8294009260 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:20.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.673+0000 7f828b7fe640 1 -- 172.21.15.84:0/734235093 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f829c013470 con 0x5649c6261680 2024-05-22T23:08:20.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.673+0000 7f828b7fe640 1 -- 172.21.15.84:0/734235093 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c012630 con 0x5649c6261680 2024-05-22T23:08:20.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.674+0000 7f82ae253740 1 -- 172.21.15.84:0/734235093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6261680 msgr2=0x5649c626fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:20.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.674+0000 7f82ae253740 1 --2- 172.21.15.84:0/734235093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6261680 0x5649c626fe40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f829c0099a0 tx=0x7f829c009e60 comp rx=0 tx=0).stop 2024-05-22T23:08:20.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.674+0000 7f82ae253740 1 -- 172.21.15.84:0/734235093 shutdown_connections 2024-05-22T23:08:20.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.674+0000 7f82ae253740 1 --2- 172.21.15.84:0/734235093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649c6270380 0x5649c6272770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.674+0000 7f82ae253740 1 --2- 172.21.15.84:0/734235093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6261680 0x5649c626fe40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.674+0000 7f82ae253740 1 --2- 172.21.15.84:0/734235093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6260cd0 0x5649c62610b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.674+0000 7f82ae253740 1 -- 172.21.15.84:0/734235093 >> 172.21.15.84:0/734235093 conn(0x5649c625d2a0 msgr2=0x5649c625d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:20.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.674+0000 7f82ae253740 1 -- 172.21.15.84:0/734235093 shutdown_connections 2024-05-22T23:08:20.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.674+0000 7f82ae253740 1 -- 172.21.15.84:0/734235093 wait complete. 2024-05-22T23:08:20.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.675+0000 7f82ae253740 1 -- start start 2024-05-22T23:08:20.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.675+0000 7f82ae253740 1 Processor -- start 2024-05-22T23:08:20.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.675+0000 7f82ae253740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6260cd0 0x5649c626d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.675+0000 7f82ae253740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649c6261680 0x5649c626d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.675+0000 7f82ae253740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6270380 0x5649c626ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.675+0000 7f82ae253740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5649c6273140 con 0x5649c6260cd0 2024-05-22T23:08:20.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.675+0000 7f82ae253740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5649c61802e0 con 0x5649c6270380 2024-05-22T23:08:20.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.675+0000 7f82ae253740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5649c62732c0 con 0x5649c6261680 2024-05-22T23:08:20.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.675+0000 7f82ae253740 10 client.0 _ll_register_callbacks cb 0x5649c6189a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:08:20.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.675+0000 7f82a92de640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6260cd0 0x5649c626d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.675+0000 7f82a92de640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6260cd0 0x5649c626d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36792/0 (socket says 172.21.15.84:36792) 2024-05-22T23:08:20.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.675+0000 7f82a9adf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6270380 0x5649c626ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.675+0000 7f82a92de640 1 -- 172.21.15.84:0/2652439898 learned_addr learned my addr 172.21.15.84:0/2652439898 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:20.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.675+0000 7f82a8add640 1 --2- 172.21.15.84:0/2652439898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649c6261680 0x5649c626d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.676+0000 7f82a92de640 1 -- 172.21.15.84:0/2652439898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649c6261680 msgr2=0x5649c626d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:20.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.676+0000 7f82a92de640 1 --2- 172.21.15.84:0/2652439898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649c6261680 0x5649c626d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.676+0000 7f82a92de640 1 -- 172.21.15.84:0/2652439898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6270380 msgr2=0x5649c626ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:20.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.676+0000 7f82a92de640 1 --2- 172.21.15.84:0/2652439898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6270380 0x5649c626ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.676+0000 7f82a92de640 1 -- 172.21.15.84:0/2652439898 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649c5ed93f0 con 0x5649c6260cd0 2024-05-22T23:08:20.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.676+0000 7f82a9adf640 1 --2- 172.21.15.84:0/2652439898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6270380 0x5649c626ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:08:20.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.676+0000 7f82a92de640 1 --2- 172.21.15.84:0/2652439898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6260cd0 0x5649c626d350 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f829c00ef70 tx=0x7f829c013620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff41bac06b94551c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:20.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.676+0000 7f8288ff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:08:20.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.676+0000 7f8288ff9640 1 -- 172.21.15.84:0/2652439898 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c017050 con 0x5649c6260cd0 2024-05-22T23:08:20.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.676+0000 7f82ae253740 1 -- 172.21.15.84:0/2652439898 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f829c013470 con 0x5649c6260cd0 2024-05-22T23:08:20.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.676+0000 7f8288ff9640 1 -- 172.21.15.84:0/2652439898 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f829c013c50 con 0x5649c6260cd0 2024-05-22T23:08:20.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.676+0000 7f8288ff9640 1 -- 172.21.15.84:0/2652439898 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f829c01c450 con 0x5649c6260cd0 2024-05-22T23:08:20.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.676+0000 7f82ae253740 10 client.16440 fetch_fsmap learned FSMap version 18 2024-05-22T23:08:20.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.676+0000 7f82ae253740 1 -- 172.21.15.84:0/2652439898 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5649c5fa37d0 con 0x5649c6260cd0 2024-05-22T23:08:20.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.676+0000 7f8288ff9640 1 -- 172.21.15.84:0/2652439898 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f829c013470 con 0x5649c6260cd0 2024-05-22T23:08:20.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.676+0000 7f8288ff9640 1 -- 172.21.15.84:0/2652439898 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f829c02d070 con 0x5649c6260cd0 2024-05-22T23:08:20.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.676+0000 7f82ae253740 10 client.16440 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:08:20.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.676+0000 7f82ae253740 10 client.16440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:08:20.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.676+0000 7f82ae253740 20 client.16440 populate_metadata read hostname 'smithi084' 2024-05-22T23:08:20.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.676+0000 7f82ae253740 1 --2- 172.21.15.84:0/2652439898 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5649c62805f0 0x5649c62829e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.676+0000 7f82ae253740 4 client.16440 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:08:20.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.676+0000 7f82ae253740 1 -- 172.21.15.84:0/2652439898 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649c61d34e0 con 0x5649c62805f0 2024-05-22T23:08:20.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.677+0000 7f82a8add640 1 --2- 172.21.15.84:0/2652439898 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5649c62805f0 0x5649c62829e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.677+0000 7f8288ff9640 10 client.16440 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:20.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.677+0000 7f82a8add640 1 --2- 172.21.15.84:0/2652439898 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5649c62805f0 0x5649c62829e0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aded10de9644dea server_cookie=9936647a9f266542 in_seq=0 out_seq=0 2024-05-22T23:08:20.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.678+0000 7f8288ff9640 1 -- 172.21.15.84:0/2652439898 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5649c61d34e0 con 0x5649c62805f0 2024-05-22T23:08:20.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.678+0000 7f8288ff9640 10 client.16440 handle_command_reply: tid=0 2024-05-22T23:08:20.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.678+0000 7f82ae253740 1 client.16440 shutdown 2024-05-22T23:08:20.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.678+0000 7f82ae253740 1 -- 172.21.15.84:0/2652439898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6260cd0 msgr2=0x5649c626d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:20.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.678+0000 7f82ae253740 1 --2- 172.21.15.84:0/2652439898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6260cd0 0x5649c626d350 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f829c00ef70 tx=0x7f829c013620 comp rx=0 tx=0).stop 2024-05-22T23:08:20.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.678+0000 7f82ae253740 1 -- 172.21.15.84:0/2652439898 shutdown_connections 2024-05-22T23:08:20.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.678+0000 7f82ae253740 1 --2- 172.21.15.84:0/2652439898 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5649c62805f0 0x5649c62829e0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.678+0000 7f82ae253740 1 --2- 172.21.15.84:0/2652439898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6270380 0x5649c626ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.678+0000 7f8288ff9640 0 client.16440 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:20.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.678+0000 7f82ae253740 1 --2- 172.21.15.84:0/2652439898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649c6261680 0x5649c626d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.678+0000 7f82ae253740 1 --2- 172.21.15.84:0/2652439898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6260cd0 0x5649c626d350 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.678+0000 7f82ae253740 1 -- 172.21.15.84:0/2652439898 >> 172.21.15.84:0/2652439898 conn(0x5649c625d2a0 msgr2=0x5649c6263110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:20.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.678+0000 7f82ae253740 1 -- 172.21.15.84:0/2652439898 shutdown_connections 2024-05-22T23:08:20.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.678+0000 7f82ae253740 1 -- 172.21.15.84:0/2652439898 wait complete. 2024-05-22T23:08:20.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.678+0000 7f82ae253740 20 client.16440 trim_cache size 0 max 16384 2024-05-22T23:08:20.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.692+0000 7f82ae253740 1 Processor -- start 2024-05-22T23:08:20.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.692+0000 7f82ae253740 1 -- start start 2024-05-22T23:08:20.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.692+0000 7f82ae253740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6260cd0 0x5649c626fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.692+0000 7f82ae253740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649c6261680 0x5649c627c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.692+0000 7f82ae253740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6270380 0x5649c6288010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.692+0000 7f82ae253740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41a9530 con 0x5649c6270380 2024-05-22T23:08:20.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.692+0000 7f82ae253740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41abd90 con 0x5649c6260cd0 2024-05-22T23:08:20.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.692+0000 7f82ae253740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41a9b80 con 0x5649c6261680 2024-05-22T23:08:20.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.692+0000 7f82a9adf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6270380 0x5649c6288010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.692+0000 7f82a92de640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6260cd0 0x5649c626fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.692+0000 7f82a9adf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6270380 0x5649c6288010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36796/0 (socket says 172.21.15.84:36796) 2024-05-22T23:08:20.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.692+0000 7f82a9adf640 1 -- 172.21.15.84:0/1512018357 learned_addr learned my addr 172.21.15.84:0/1512018357 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:20.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.692+0000 7f82a92de640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6260cd0 0x5649c626fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:38336/0 (socket says 172.21.15.84:38336) 2024-05-22T23:08:20.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.692+0000 7f82a8add640 1 --2- 172.21.15.84:0/1512018357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649c6261680 0x5649c627c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.692+0000 7f82a9adf640 1 -- 172.21.15.84:0/1512018357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649c6261680 msgr2=0x5649c627c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:20.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.692+0000 7f82a9adf640 1 --2- 172.21.15.84:0/1512018357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649c6261680 0x5649c627c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.692+0000 7f82a9adf640 1 -- 172.21.15.84:0/1512018357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6260cd0 msgr2=0x5649c626fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:20.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.692+0000 7f82a9adf640 1 --2- 172.21.15.84:0/1512018357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6260cd0 0x5649c626fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.692+0000 7f82a9adf640 1 -- 172.21.15.84:0/1512018357 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649c625d0d0 con 0x5649c6270380 2024-05-22T23:08:20.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.692+0000 7f82a8add640 1 --2- 172.21.15.84:0/1512018357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649c6261680 0x5649c627c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:08:20.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.692+0000 7f82a92de640 1 --2- 172.21.15.84:0/1512018357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6260cd0 0x5649c626fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:08:20.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.692+0000 7f82a9adf640 1 --2- 172.21.15.84:0/1512018357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6270380 0x5649c6288010 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8294033000 tx=0x7f829403e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376172d25feda474 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:20.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.693+0000 7f828affd640 1 -- 172.21.15.84:0/1512018357 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294035040 con 0x5649c6270380 2024-05-22T23:08:20.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.693+0000 7f828affd640 1 -- 172.21.15.84:0/1512018357 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8294003d40 con 0x5649c6270380 2024-05-22T23:08:20.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.693+0000 7f828affd640 1 -- 172.21.15.84:0/1512018357 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829404a690 con 0x5649c6270380 2024-05-22T23:08:20.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.693+0000 7f82ae253740 1 -- 172.21.15.84:0/1512018357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6270380 msgr2=0x5649c6288010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:20.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.693+0000 7f82ae253740 1 --2- 172.21.15.84:0/1512018357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6270380 0x5649c6288010 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8294033000 tx=0x7f829403e000 comp rx=0 tx=0).stop 2024-05-22T23:08:20.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.693+0000 7f82ae253740 1 -- 172.21.15.84:0/1512018357 shutdown_connections 2024-05-22T23:08:20.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.693+0000 7f82ae253740 1 --2- 172.21.15.84:0/1512018357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6270380 0x5649c6288010 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.693+0000 7f82ae253740 1 --2- 172.21.15.84:0/1512018357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649c6261680 0x5649c627c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.693+0000 7f82ae253740 1 --2- 172.21.15.84:0/1512018357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6260cd0 0x5649c626fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.693+0000 7f82ae253740 1 -- 172.21.15.84:0/1512018357 >> 172.21.15.84:0/1512018357 conn(0x5649c625d2a0 msgr2=0x5649c62630a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:20.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.693+0000 7f82ae253740 1 -- 172.21.15.84:0/1512018357 shutdown_connections 2024-05-22T23:08:20.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.693+0000 7f82ae253740 1 -- 172.21.15.84:0/1512018357 wait complete. 2024-05-22T23:08:20.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.694+0000 7f82ae253740 1 -- start start 2024-05-22T23:08:20.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.694+0000 7f82ae253740 1 Processor -- start 2024-05-22T23:08:20.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.694+0000 7f82ae253740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649c6260cd0 0x5649c62644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.694+0000 7f82ae253740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6261680 0x5649c62649f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.694+0000 7f82ae253740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6270380 0x5649c6277540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.694+0000 7f82ae253740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41aa250 con 0x5649c6270380 2024-05-22T23:08:20.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.694+0000 7f82ae253740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41ab6c0 con 0x5649c6261680 2024-05-22T23:08:20.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.694+0000 7f82ae253740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41aa920 con 0x5649c6260cd0 2024-05-22T23:08:20.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.694+0000 7f82ae253740 10 client.0 _ll_register_callbacks cb 0x7f8294009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:08:20.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.694+0000 7f82a9adf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6270380 0x5649c6277540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.694+0000 7f82a8add640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6261680 0x5649c62649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.694+0000 7f82a9adf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6270380 0x5649c6277540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36806/0 (socket says 172.21.15.84:36806) 2024-05-22T23:08:20.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.694+0000 7f82a9adf640 1 -- 172.21.15.84:0/2207104588 learned_addr learned my addr 172.21.15.84:0/2207104588 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:20.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.694+0000 7f82a92de640 1 --2- 172.21.15.84:0/2207104588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649c6260cd0 0x5649c62644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.695+0000 7f82a8add640 1 -- 172.21.15.84:0/2207104588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649c6260cd0 msgr2=0x5649c62644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:20.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.695+0000 7f82a8add640 1 --2- 172.21.15.84:0/2207104588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649c6260cd0 0x5649c62644b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.695+0000 7f82a8add640 1 -- 172.21.15.84:0/2207104588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6270380 msgr2=0x5649c6277540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:20.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.695+0000 7f82a8add640 1 --2- 172.21.15.84:0/2207104588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6270380 0x5649c6277540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.695+0000 7f82a8add640 1 -- 172.21.15.84:0/2207104588 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649c6272770 con 0x5649c6261680 2024-05-22T23:08:20.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.695+0000 7f82a9adf640 1 --2- 172.21.15.84:0/2207104588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6270380 0x5649c6277540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:08:20.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.695+0000 7f82a92de640 1 --2- 172.21.15.84:0/2207104588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649c6260cd0 0x5649c62644b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:08:20.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.695+0000 7f82a8add640 1 --2- 172.21.15.84:0/2207104588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6261680 0x5649c62649f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f82980cd1b0 tx=0x7f82980054d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7e4d8b5630ee27f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:20.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.695+0000 7f8288ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:08:20.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.695+0000 7f8288ff9640 1 -- 172.21.15.84:0/2207104588 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829801f040 con 0x5649c6261680 2024-05-22T23:08:20.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.695+0000 7f82ae253740 1 -- 172.21.15.84:0/2207104588 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8294003d40 con 0x5649c6261680 2024-05-22T23:08:20.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.695+0000 7f8288ff9640 1 -- 172.21.15.84:0/2207104588 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f82980b00d0 con 0x5649c6261680 2024-05-22T23:08:20.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.695+0000 7f8288ff9640 1 -- 172.21.15.84:0/2207104588 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f829809bf80 con 0x5649c6261680 2024-05-22T23:08:20.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.695+0000 7f82ae253740 10 client.26015 fetch_fsmap learned FSMap version 18 2024-05-22T23:08:20.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.695+0000 7f82ae253740 1 -- 172.21.15.84:0/2207104588 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5649c6278bf0 con 0x5649c6261680 2024-05-22T23:08:20.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.695+0000 7f8288ff9640 1 -- 172.21.15.84:0/2207104588 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8294003d40 con 0x5649c6261680 2024-05-22T23:08:20.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.695+0000 7f8288ff9640 1 -- 172.21.15.84:0/2207104588 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f829800f040 con 0x5649c6261680 2024-05-22T23:08:20.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.696+0000 7f82ae253740 10 client.26015 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:08:20.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.696+0000 7f82ae253740 10 client.26015 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:08:20.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.696+0000 7f82ae253740 20 client.26015 populate_metadata read hostname 'smithi084' 2024-05-22T23:08:20.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.696+0000 7f82ae253740 1 --2- 172.21.15.84:0/2207104588 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5649c62eb3a0 0x5649c62ed7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:20.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.696+0000 7f82ae253740 4 client.26015 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-22T23:08:20.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.696+0000 7f82ae253740 1 -- 172.21.15.84:0/2207104588 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5649c61cd830 con 0x5649c62eb3a0 2024-05-22T23:08:20.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.696+0000 7f82a92de640 1 --2- 172.21.15.84:0/2207104588 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5649c62eb3a0 0x5649c62ed7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:20.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.696+0000 7f82a92de640 1 --2- 172.21.15.84:0/2207104588 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5649c62eb3a0 0x5649c62ed7c0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d72f2f70d8edf6e1 server_cookie=64d2533fc5673e84 in_seq=0 out_seq=0 2024-05-22T23:08:20.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.696+0000 7f8288ff9640 10 client.26015 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:20.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.696+0000 7f8288ff9640 1 -- 172.21.15.84:0/2207104588 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5649c61cd830 con 0x5649c62eb3a0 2024-05-22T23:08:20.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.696+0000 7f8288ff9640 10 client.26015 handle_command_reply: tid=0 2024-05-22T23:08:20.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.696+0000 7f82ae253740 1 client.26015 shutdown 2024-05-22T23:08:20.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.696+0000 7f82ae253740 1 -- 172.21.15.84:0/2207104588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6261680 msgr2=0x5649c62649f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:20.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.696+0000 7f82ae253740 1 --2- 172.21.15.84:0/2207104588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6261680 0x5649c62649f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f82980cd1b0 tx=0x7f82980054d0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.696+0000 7f82ae253740 1 -- 172.21.15.84:0/2207104588 shutdown_connections 2024-05-22T23:08:20.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.696+0000 7f82ae253740 1 --2- 172.21.15.84:0/2207104588 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5649c62eb3a0 0x5649c62ed7c0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.696+0000 7f82ae253740 1 --2- 172.21.15.84:0/2207104588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649c6270380 0x5649c6277540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.696+0000 7f82ae253740 1 --2- 172.21.15.84:0/2207104588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c6261680 0x5649c62649f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.696+0000 7f8288ff9640 0 client.26015 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:20.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.696+0000 7f82ae253740 1 --2- 172.21.15.84:0/2207104588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649c6260cd0 0x5649c62644b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.696+0000 7f82ae253740 1 -- 172.21.15.84:0/2207104588 >> 172.21.15.84:0/2207104588 conn(0x5649c625d2a0 msgr2=0x5649c625bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:20.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.697+0000 7f82ae253740 1 -- 172.21.15.84:0/2207104588 shutdown_connections 2024-05-22T23:08:20.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.697+0000 7f82ae253740 1 -- 172.21.15.84:0/2207104588 wait complete. 2024-05-22T23:08:20.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.697+0000 7f82ae253740 20 client.26015 trim_cache size 0 max 16384 2024-05-22T23:08:20.709 INFO:teuthology.orchestra.run.smithi084.stdout:[] 2024-05-22T23:08:20.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.697+0000 7f82ab542640 1 -- 172.21.15.84:0/4017888363 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f828c0a2d00 msgr2=0x7f828c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:20.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.697+0000 7f82ab542640 1 --2- 172.21.15.84:0/4017888363 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f828c0a2d00 0x7f828c0a51c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8294031550 tx=0x7f8294009260 comp rx=0 tx=0).stop 2024-05-22T23:08:20.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.697+0000 7f82ab542640 1 -- 172.21.15.84:0/4017888363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82a4103410 msgr2=0x7f82a4198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:20.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.697+0000 7f82ab542640 1 --2- 172.21.15.84:0/4017888363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82a4103410 0x7f82a4198710 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f829800ea80 tx=0x7f829800ee80 comp rx=0 tx=0).stop 2024-05-22T23:08:20.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.697+0000 7f82ab542640 1 -- 172.21.15.84:0/4017888363 shutdown_connections 2024-05-22T23:08:20.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.697+0000 7f82ab542640 1 --2- 172.21.15.84:0/4017888363 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f828c0a2d00 0x7f828c0a51c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.697+0000 7f82ab542640 1 --2- 172.21.15.84:0/4017888363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82a410be30 0x7f82a419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.697+0000 7f82ab542640 1 --2- 172.21.15.84:0/4017888363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82a4103410 0x7f82a4198710 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.697+0000 7f82ab542640 1 --2- 172.21.15.84:0/4017888363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82a4102af0 0x7f82a41981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:20.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.697+0000 7f82ab542640 1 -- 172.21.15.84:0/4017888363 >> 172.21.15.84:0/4017888363 conn(0x7f82a4077870 msgr2=0x7f82a40fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:20.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.697+0000 7f82ab542640 1 -- 172.21.15.84:0/4017888363 shutdown_connections 2024-05-22T23:08:20.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:20.697+0000 7f82ab542640 1 -- 172.21.15.84:0/4017888363 wait complete. 2024-05-22T23:08:20.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:20 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/4240861739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:08:21.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:20 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/4240861739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:08:21.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:20 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/4240861739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:08:21.440 INFO:tasks.workunit.client.0.smithi084.stdout: 1 869313 15.01 MB/sec execute 187 sec latency 23.380 ms 2024-05-22T23:08:21.716 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-22T23:08:21.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:08:21.716 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-22T23:08:21.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.846+0000 7f699de66640 1 -- 172.21.15.84:0/1440810554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6998107850 msgr2=0x7f6998107c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:21.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.846+0000 7f699de66640 1 --2- 172.21.15.84:0/1440810554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6998107850 0x7f6998107c30 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6988009930 tx=0x7f698802a420 comp rx=0 tx=0).stop 2024-05-22T23:08:21.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.847+0000 7f699de66640 1 -- 172.21.15.84:0/1440810554 shutdown_connections 2024-05-22T23:08:21.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.847+0000 7f699de66640 1 --2- 172.21.15.84:0/1440810554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6998102200 0x7f699810e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.847+0000 7f699de66640 1 --2- 172.21.15.84:0/1440810554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6998101840 0x7f6998101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.847+0000 7f699de66640 1 --2- 172.21.15.84:0/1440810554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6998107850 0x7f6998107c30 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.847+0000 7f699de66640 1 -- 172.21.15.84:0/1440810554 >> 172.21.15.84:0/1440810554 conn(0x7f6998078cc0 msgr2=0x7f69980790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:21.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.847+0000 7f699de66640 1 -- 172.21.15.84:0/1440810554 shutdown_connections 2024-05-22T23:08:21.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.847+0000 7f699de66640 1 -- 172.21.15.84:0/1440810554 wait complete. 2024-05-22T23:08:21.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.847+0000 7f699de66640 1 Processor -- start 2024-05-22T23:08:21.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.847+0000 7f699de66640 1 -- start start 2024-05-22T23:08:21.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.848+0000 7f699de66640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6998101840 0x7f6998198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:21.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.848+0000 7f699de66640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6998102200 0x7f69981986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:21.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.848+0000 7f699de66640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6998107850 0x7f699819ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:21.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.848+0000 7f699de66640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6998111c60 con 0x7f6998107850 2024-05-22T23:08:21.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.848+0000 7f699de66640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6998111ae0 con 0x7f6998101840 2024-05-22T23:08:21.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.848+0000 7f699de66640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6998111de0 con 0x7f6998102200 2024-05-22T23:08:21.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.848+0000 7f69977fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6998101840 0x7f6998198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:21.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.848+0000 7f6997fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6998107850 0x7f699819ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:21.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.848+0000 7f69977fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6998101840 0x7f6998198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:33852/0 (socket says 172.21.15.84:33852) 2024-05-22T23:08:21.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.848+0000 7f69977fe640 1 -- 172.21.15.84:0/2493668091 learned_addr learned my addr 172.21.15.84:0/2493668091 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:21.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.848+0000 7f6996ffd640 1 --2- 172.21.15.84:0/2493668091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6998102200 0x7f69981986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:21.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.848+0000 7f6997fff640 1 -- 172.21.15.84:0/2493668091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6998102200 msgr2=0x7f69981986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:21.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.848+0000 7f6997fff640 1 --2- 172.21.15.84:0/2493668091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6998102200 0x7f69981986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.848+0000 7f6997fff640 1 -- 172.21.15.84:0/2493668091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6998101840 msgr2=0x7f6998198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:21.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.848+0000 7f6997fff640 1 --2- 172.21.15.84:0/2493668091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6998101840 0x7f6998198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.848+0000 7f6997fff640 1 -- 172.21.15.84:0/2493668091 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699819d150 con 0x7f6998107850 2024-05-22T23:08:21.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.848+0000 7f69977fe640 1 --2- 172.21.15.84:0/2493668091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6998101840 0x7f6998198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:08:21.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.848+0000 7f6997fff640 1 --2- 172.21.15.84:0/2493668091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6998107850 0x7f699819ca20 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f699000e940 tx=0x7f699000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:21.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.848+0000 7f6994ff9640 1 -- 172.21.15.84:0/2493668091 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990004d20 con 0x7f6998107850 2024-05-22T23:08:21.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.848+0000 7f699de66640 1 -- 172.21.15.84:0/2493668091 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699819d440 con 0x7f6998107850 2024-05-22T23:08:21.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.848+0000 7f6994ff9640 1 -- 172.21.15.84:0/2493668091 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6990005020 con 0x7f6998107850 2024-05-22T23:08:21.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.848+0000 7f6994ff9640 1 -- 172.21.15.84:0/2493668091 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69900116d0 con 0x7f6998107850 2024-05-22T23:08:21.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.848+0000 7f699de66640 1 -- 172.21.15.84:0/2493668091 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69981a4cd0 con 0x7f6998107850 2024-05-22T23:08:21.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.851+0000 7f6994ff9640 1 -- 172.21.15.84:0/2493668091 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f69900054d0 con 0x7f6998107850 2024-05-22T23:08:21.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.851+0000 7f69a0b70740 1 Processor -- start 2024-05-22T23:08:21.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.851+0000 7f69a0b70740 1 -- start start 2024-05-22T23:08:21.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.851+0000 7f69a0b70740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449bc50 0x556d5449c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:21.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.851+0000 7f69a0b70740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d5449c570 0x556d544a6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:21.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.851+0000 7f69a0b70740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d544a6c40 0x556d544a9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:21.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.851+0000 7f69a0b70740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556d543ffa40 con 0x556d5449bc50 2024-05-22T23:08:21.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.851+0000 7f69a0b70740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556d5439e440 con 0x556d544a6c40 2024-05-22T23:08:21.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.851+0000 7f69a0b70740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556d543fba00 con 0x556d5449c570 2024-05-22T23:08:21.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.851+0000 7f6994ff9640 1 --2- 172.21.15.84:0/2493668091 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f69800a2d00 0x7f69800a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:21.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.851+0000 7f6996ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449bc50 0x556d5449c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:21.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.851+0000 7f6996ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449bc50 0x556d5449c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39594/0 (socket says 172.21.15.84:39594) 2024-05-22T23:08:21.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.851+0000 7f6996ffd640 1 -- 172.21.15.84:0/2541223199 learned_addr learned my addr 172.21.15.84:0/2541223199 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:21.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.851+0000 7f69977fe640 1 --2- 172.21.15.84:0/2541223199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d5449c570 0x556d544a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:21.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.851+0000 7f6996ffd640 1 --2- 172.21.15.84:0/2541223199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d544a6c40 0x556d544a9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:21.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.851+0000 7f6997fff640 1 --2- 172.21.15.84:0/2493668091 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f69800a2d00 0x7f69800a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:21.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.851+0000 7f6994ff9640 1 -- 172.21.15.84:0/2493668091 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f69900cf780 con 0x7f6998107850 2024-05-22T23:08:21.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.852+0000 7f6996ffd640 1 -- 172.21.15.84:0/2541223199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d5449c570 msgr2=0x556d544a6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:21.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.852+0000 7f6996ffd640 1 --2- 172.21.15.84:0/2541223199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d5449c570 0x556d544a6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.852+0000 7f6996ffd640 1 -- 172.21.15.84:0/2541223199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d544a6c40 msgr2=0x556d544a9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:21.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.852+0000 7f6996ffd640 1 --2- 172.21.15.84:0/2541223199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d544a6c40 0x556d544a9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.852+0000 7f6996ffd640 1 -- 172.21.15.84:0/2541223199 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d543a82d0 con 0x556d5449bc50 2024-05-22T23:08:21.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.852+0000 7f69977fe640 1 --2- 172.21.15.84:0/2541223199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d5449c570 0x556d544a6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:08:21.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.852+0000 7f6996ffd640 1 --2- 172.21.15.84:0/2541223199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449bc50 0x556d5449c030 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f698c01a980 tx=0x7f698c0097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458d4b9568866602 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:21.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.852+0000 7f6997fff640 1 --2- 172.21.15.84:0/2493668091 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f69800a2d00 0x7f69800a51c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f69900ccf50 tx=0x7f69900911b0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:21.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.852+0000 7f697dffb640 1 -- 172.21.15.84:0/2541223199 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c004030 con 0x556d5449bc50 2024-05-22T23:08:21.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.852+0000 7f697dffb640 1 -- 172.21.15.84:0/2541223199 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f698c011cc0 con 0x556d5449bc50 2024-05-22T23:08:21.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.852+0000 7f697dffb640 1 -- 172.21.15.84:0/2541223199 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c0038e0 con 0x556d5449bc50 2024-05-22T23:08:21.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.853+0000 7f69a0b70740 1 -- 172.21.15.84:0/2541223199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449bc50 msgr2=0x556d5449c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:21.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.853+0000 7f69a0b70740 1 --2- 172.21.15.84:0/2541223199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449bc50 0x556d5449c030 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f698c01a980 tx=0x7f698c0097a0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.853+0000 7f69a0b70740 1 -- 172.21.15.84:0/2541223199 shutdown_connections 2024-05-22T23:08:21.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.853+0000 7f69a0b70740 1 --2- 172.21.15.84:0/2541223199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d544a6c40 0x556d544a9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.853+0000 7f69a0b70740 1 --2- 172.21.15.84:0/2541223199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d5449c570 0x556d544a6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.853+0000 7f69a0b70740 1 --2- 172.21.15.84:0/2541223199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449bc50 0x556d5449c030 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.853+0000 7f69a0b70740 1 -- 172.21.15.84:0/2541223199 >> 172.21.15.84:0/2541223199 conn(0x556d54497f70 msgr2=0x556d54498350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:21.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.853+0000 7f69a0b70740 1 -- 172.21.15.84:0/2541223199 shutdown_connections 2024-05-22T23:08:21.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.853+0000 7f69a0b70740 1 -- 172.21.15.84:0/2541223199 wait complete. 2024-05-22T23:08:21.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.853+0000 7f69a0b70740 1 -- start start 2024-05-22T23:08:21.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.853+0000 7f69a0b70740 1 Processor -- start 2024-05-22T23:08:21.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.853+0000 7f69a0b70740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d5449bc50 0x556d544a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:21.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.854+0000 7f69a0b70740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449c570 0x556d544a4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:21.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.854+0000 7f69a0b70740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d544a6c40 0x556d544a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:21.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.854+0000 7f69a0b70740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556d544a99d0 con 0x556d5449c570 2024-05-22T23:08:21.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.854+0000 7f69a0b70740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556d543895f0 con 0x556d5449bc50 2024-05-22T23:08:21.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.854+0000 7f69a0b70740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556d544a9b50 con 0x556d544a6c40 2024-05-22T23:08:21.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.854+0000 7f69a0b70740 10 client.0 _ll_register_callbacks cb 0x556d54384cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:08:21.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.854+0000 7f69977fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449c570 0x556d544a4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:21.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.854+0000 7f6996ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d5449bc50 0x556d544a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:21.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.854+0000 7f69977fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449c570 0x556d544a4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39598/0 (socket says 172.21.15.84:39598) 2024-05-22T23:08:21.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.854+0000 7f69977fe640 1 -- 172.21.15.84:0/1585335779 learned_addr learned my addr 172.21.15.84:0/1585335779 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:21.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.854+0000 7f6996ffd640 1 --2- 172.21.15.84:0/1585335779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d544a6c40 0x556d544a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:21.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.854+0000 7f69977fe640 1 -- 172.21.15.84:0/1585335779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d544a6c40 msgr2=0x556d544a45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:21.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.854+0000 7f69977fe640 1 --2- 172.21.15.84:0/1585335779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d544a6c40 0x556d544a45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.854+0000 7f69977fe640 1 -- 172.21.15.84:0/1585335779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d5449bc50 msgr2=0x556d544a3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:21.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.854+0000 7f69977fe640 1 --2- 172.21.15.84:0/1585335779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d5449bc50 0x556d544a3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.854+0000 7f69977fe640 1 -- 172.21.15.84:0/1585335779 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d541db7e0 con 0x556d5449c570 2024-05-22T23:08:21.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.854+0000 7f6996ffd640 1 --2- 172.21.15.84:0/1585335779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d5449bc50 0x556d544a3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:08:21.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.854+0000 7f69977fe640 1 --2- 172.21.15.84:0/1585335779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449c570 0x556d544a4070 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6988006010 tx=0x7f6988031a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88dd29fd3c6cf210 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:21.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.854+0000 7f6976ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:08:21.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.854+0000 7f6976ffd640 1 -- 172.21.15.84:0/1585335779 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6988035040 con 0x556d5449c570 2024-05-22T23:08:21.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.854+0000 7f6976ffd640 1 -- 172.21.15.84:0/1585335779 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f6988004250 con 0x556d5449c570 2024-05-22T23:08:21.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.854+0000 7f69a0b70740 1 -- 172.21.15.84:0/1585335779 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f698c011cc0 con 0x556d5449c570 2024-05-22T23:08:21.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.855+0000 7f6976ffd640 1 -- 172.21.15.84:0/1585335779 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f6988004450 con 0x556d5449c570 2024-05-22T23:08:21.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.855+0000 7f69a0b70740 10 client.16458 fetch_fsmap learned FSMap version 18 2024-05-22T23:08:21.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.855+0000 7f69a0b70740 1 -- 172.21.15.84:0/1585335779 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x556d5423bf70 con 0x556d5449c570 2024-05-22T23:08:21.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.855+0000 7f6976ffd640 1 -- 172.21.15.84:0/1585335779 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f698c011cc0 con 0x556d5449c570 2024-05-22T23:08:21.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.855+0000 7f6976ffd640 1 -- 172.21.15.84:0/1585335779 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f698804d070 con 0x556d5449c570 2024-05-22T23:08:21.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.855+0000 7f69a0b70740 10 client.16458 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:08:21.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.855+0000 7f69a0b70740 10 client.16458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:08:21.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.855+0000 7f69a0b70740 20 client.16458 populate_metadata read hostname 'smithi084' 2024-05-22T23:08:21.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.855+0000 7f69a0b70740 1 --2- 172.21.15.84:0/1585335779 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556d544b6ec0 0x556d544b92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:21.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.855+0000 7f69a0b70740 4 client.16458 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:08:21.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.855+0000 7f69a0b70740 1 -- 172.21.15.84:0/1585335779 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d54413460 con 0x556d544b6ec0 2024-05-22T23:08:21.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.855+0000 7f6996ffd640 1 --2- 172.21.15.84:0/1585335779 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556d544b6ec0 0x556d544b92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:21.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.856+0000 7f6976ffd640 10 client.16458 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:21.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.856+0000 7f6996ffd640 1 --2- 172.21.15.84:0/1585335779 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556d544b6ec0 0x556d544b92d0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1864601a6a34426 server_cookie=56955639f24bfa2d in_seq=0 out_seq=0 2024-05-22T23:08:21.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.857+0000 7f6976ffd640 1 -- 172.21.15.84:0/1585335779 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556d54413460 con 0x556d544b6ec0 2024-05-22T23:08:21.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.857+0000 7f6976ffd640 10 client.16458 handle_command_reply: tid=0 2024-05-22T23:08:21.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.857+0000 7f69a0b70740 1 client.16458 shutdown 2024-05-22T23:08:21.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.857+0000 7f69a0b70740 1 -- 172.21.15.84:0/1585335779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449c570 msgr2=0x556d544a4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:21.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.857+0000 7f69a0b70740 1 --2- 172.21.15.84:0/1585335779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449c570 0x556d544a4070 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6988006010 tx=0x7f6988031a40 comp rx=0 tx=0).stop 2024-05-22T23:08:21.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.857+0000 7f69a0b70740 1 -- 172.21.15.84:0/1585335779 shutdown_connections 2024-05-22T23:08:21.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.857+0000 7f69a0b70740 1 --2- 172.21.15.84:0/1585335779 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556d544b6ec0 0x556d544b92d0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.857+0000 7f69a0b70740 1 --2- 172.21.15.84:0/1585335779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d544a6c40 0x556d544a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.857+0000 7f6976ffd640 0 client.16458 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:21.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.857+0000 7f69a0b70740 1 --2- 172.21.15.84:0/1585335779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449c570 0x556d544a4070 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.857+0000 7f69a0b70740 1 --2- 172.21.15.84:0/1585335779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d5449bc50 0x556d544a3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.857+0000 7f69a0b70740 1 -- 172.21.15.84:0/1585335779 >> 172.21.15.84:0/1585335779 conn(0x556d54497f70 msgr2=0x556d5449dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:21.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.857+0000 7f69a0b70740 1 -- 172.21.15.84:0/1585335779 shutdown_connections 2024-05-22T23:08:21.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.857+0000 7f69a0b70740 1 -- 172.21.15.84:0/1585335779 wait complete. 2024-05-22T23:08:21.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.857+0000 7f69a0b70740 20 client.16458 trim_cache size 0 max 16384 2024-05-22T23:08:21.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.876+0000 7f69a0b70740 1 Processor -- start 2024-05-22T23:08:21.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.876+0000 7f69a0b70740 1 -- start start 2024-05-22T23:08:21.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.876+0000 7f69a0b70740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d5449bc50 0x556d544b2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:21.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.876+0000 7f69a0b70740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449c570 0x556d5449e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:21.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.876+0000 7f69a0b70740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d544a6c40 0x556d544bebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:21.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.876+0000 7f69a0b70740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f69981a9510 con 0x556d5449c570 2024-05-22T23:08:21.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.876+0000 7f69a0b70740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f69981abd70 con 0x556d544a6c40 2024-05-22T23:08:21.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.876+0000 7f69a0b70740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f69981a9b60 con 0x556d5449bc50 2024-05-22T23:08:21.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.876+0000 7f69977fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449c570 0x556d5449e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:21.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.876+0000 7f69977fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449c570 0x556d5449e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39608/0 (socket says 172.21.15.84:39608) 2024-05-22T23:08:21.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.876+0000 7f69977fe640 1 -- 172.21.15.84:0/3269663632 learned_addr learned my addr 172.21.15.84:0/3269663632 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:21.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.876+0000 7f6996ffd640 1 --2- 172.21.15.84:0/3269663632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d544a6c40 0x556d544bebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:21.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.876+0000 7f6996ffd640 1 --2- 172.21.15.84:0/3269663632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d5449bc50 0x556d544b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:21.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.876+0000 7f69977fe640 1 -- 172.21.15.84:0/3269663632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d5449bc50 msgr2=0x556d544b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:21.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.876+0000 7f69977fe640 1 --2- 172.21.15.84:0/3269663632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d5449bc50 0x556d544b2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.876+0000 7f69977fe640 1 -- 172.21.15.84:0/3269663632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d544a6c40 msgr2=0x556d544bebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:21.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.876+0000 7f69977fe640 1 --2- 172.21.15.84:0/3269663632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d544a6c40 0x556d544bebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.876+0000 7f69977fe640 1 -- 172.21.15.84:0/3269663632 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d540d3170 con 0x556d5449c570 2024-05-22T23:08:21.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.876+0000 7f6996ffd640 1 --2- 172.21.15.84:0/3269663632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d5449bc50 0x556d544b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:08:21.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.876+0000 7f69977fe640 1 --2- 172.21.15.84:0/3269663632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449c570 0x556d5449e3b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6988000c00 tx=0x7f6988046610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a87f2dd5c95420c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:21.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.876+0000 7f697d7fa640 1 -- 172.21.15.84:0/3269663632 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6988053020 con 0x556d5449c570 2024-05-22T23:08:21.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.876+0000 7f697d7fa640 1 -- 172.21.15.84:0/3269663632 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6988046cd0 con 0x556d5449c570 2024-05-22T23:08:21.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.876+0000 7f697d7fa640 1 -- 172.21.15.84:0/3269663632 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6988035070 con 0x556d5449c570 2024-05-22T23:08:21.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.877+0000 7f69a0b70740 1 -- 172.21.15.84:0/3269663632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449c570 msgr2=0x556d5449e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:21.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.877+0000 7f69a0b70740 1 --2- 172.21.15.84:0/3269663632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449c570 0x556d5449e3b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6988000c00 tx=0x7f6988046610 comp rx=0 tx=0).stop 2024-05-22T23:08:21.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.877+0000 7f69a0b70740 1 -- 172.21.15.84:0/3269663632 shutdown_connections 2024-05-22T23:08:21.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.877+0000 7f69a0b70740 1 --2- 172.21.15.84:0/3269663632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d544a6c40 0x556d544bebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.877+0000 7f69a0b70740 1 --2- 172.21.15.84:0/3269663632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449c570 0x556d5449e3b0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.877+0000 7f69a0b70740 1 --2- 172.21.15.84:0/3269663632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d5449bc50 0x556d544b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.877+0000 7f69a0b70740 1 -- 172.21.15.84:0/3269663632 >> 172.21.15.84:0/3269663632 conn(0x556d54497f70 msgr2=0x556d544a4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:21.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.877+0000 7f69a0b70740 1 -- 172.21.15.84:0/3269663632 shutdown_connections 2024-05-22T23:08:21.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.877+0000 7f69a0b70740 1 -- 172.21.15.84:0/3269663632 wait complete. 2024-05-22T23:08:21.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.877+0000 7f69a0b70740 1 -- start start 2024-05-22T23:08:21.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.878+0000 7f69a0b70740 1 Processor -- start 2024-05-22T23:08:21.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.878+0000 7f69a0b70740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d5449bc50 0x556d544aae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:21.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.878+0000 7f69a0b70740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449c570 0x556d544ab350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:21.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.878+0000 7f69a0b70740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d544a6c40 0x556d5451c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:21.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.878+0000 7f69a0b70740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f69981aa230 con 0x556d5449c570 2024-05-22T23:08:21.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.878+0000 7f69a0b70740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f69981ab6a0 con 0x556d544a6c40 2024-05-22T23:08:21.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.878+0000 7f69a0b70740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f69981aa900 con 0x556d5449bc50 2024-05-22T23:08:21.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.878+0000 7f69a0b70740 10 client.0 _ll_register_callbacks cb 0x7f6988009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:08:21.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.878+0000 7f69977fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449c570 0x556d544ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:21.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.878+0000 7f69977fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449c570 0x556d544ab350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39624/0 (socket says 172.21.15.84:39624) 2024-05-22T23:08:21.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.878+0000 7f69977fe640 1 -- 172.21.15.84:0/2068893690 learned_addr learned my addr 172.21.15.84:0/2068893690 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:21.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.878+0000 7f6996ffd640 1 --2- 172.21.15.84:0/2068893690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d544a6c40 0x556d5451c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:21.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.878+0000 7f6996ffd640 1 --2- 172.21.15.84:0/2068893690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d5449bc50 0x556d544aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:21.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.878+0000 7f69977fe640 1 -- 172.21.15.84:0/2068893690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d5449bc50 msgr2=0x556d544aae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:21.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.878+0000 7f69977fe640 1 --2- 172.21.15.84:0/2068893690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d5449bc50 0x556d544aae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.878+0000 7f69977fe640 1 -- 172.21.15.84:0/2068893690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d544a6c40 msgr2=0x556d5451c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:21.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.878+0000 7f69977fe640 1 --2- 172.21.15.84:0/2068893690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d544a6c40 0x556d5451c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.878+0000 7f69977fe640 1 -- 172.21.15.84:0/2068893690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d543c8930 con 0x556d5449c570 2024-05-22T23:08:21.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.878+0000 7f6996ffd640 1 --2- 172.21.15.84:0/2068893690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d544a6c40 0x556d5451c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:08:21.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.878+0000 7f69977fe640 1 --2- 172.21.15.84:0/2068893690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449c570 0x556d544ab350 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6988046a50 tx=0x7f698804f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451ed0f63fd7969b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:21.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.878+0000 7f6976ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:08:21.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.879+0000 7f6976ffd640 1 -- 172.21.15.84:0/2068893690 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6988053030 con 0x556d5449c570 2024-05-22T23:08:21.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.879+0000 7f6976ffd640 1 -- 172.21.15.84:0/2068893690 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f698804fa90 con 0x556d5449c570 2024-05-22T23:08:21.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.879+0000 7f69a0b70740 1 -- 172.21.15.84:0/2068893690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6988046cd0 con 0x556d5449c570 2024-05-22T23:08:21.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.879+0000 7f6976ffd640 1 -- 172.21.15.84:0/2068893690 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f6988068050 con 0x556d5449c570 2024-05-22T23:08:21.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.879+0000 7f69a0b70740 10 client.16464 fetch_fsmap learned FSMap version 18 2024-05-22T23:08:21.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.879+0000 7f69a0b70740 1 -- 172.21.15.84:0/2068893690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x556d5451d9a0 con 0x556d5449c570 2024-05-22T23:08:21.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.879+0000 7f6976ffd640 1 -- 172.21.15.84:0/2068893690 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6988046cd0 con 0x556d5449c570 2024-05-22T23:08:21.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.879+0000 7f6976ffd640 1 -- 172.21.15.84:0/2068893690 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f698804d030 con 0x556d5449c570 2024-05-22T23:08:21.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.879+0000 7f69a0b70740 10 client.16464 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:08:21.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.879+0000 7f69a0b70740 10 client.16464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:08:21.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.879+0000 7f69a0b70740 20 client.16464 populate_metadata read hostname 'smithi084' 2024-05-22T23:08:21.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.879+0000 7f69a0b70740 1 --2- 172.21.15.84:0/2068893690 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556d5451dfb0 0x556d545203d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:21.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.879+0000 7f69a0b70740 4 client.16464 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-22T23:08:21.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.879+0000 7f69a0b70740 1 -- 172.21.15.84:0/2068893690 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556d5440d170 con 0x556d5451dfb0 2024-05-22T23:08:21.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.879+0000 7f6996ffd640 1 --2- 172.21.15.84:0/2068893690 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556d5451dfb0 0x556d545203d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:21.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.880+0000 7f6976ffd640 10 client.16464 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:21.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.880+0000 7f6996ffd640 1 --2- 172.21.15.84:0/2068893690 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556d5451dfb0 0x556d545203d0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4e5d9bfc8e1acc8 server_cookie=d817a58e1090f331 in_seq=0 out_seq=0 2024-05-22T23:08:21.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.882+0000 7f6976ffd640 1 -- 172.21.15.84:0/2068893690 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556d5440d170 con 0x556d5451dfb0 2024-05-22T23:08:21.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.882+0000 7f6976ffd640 10 client.16464 handle_command_reply: tid=0 2024-05-22T23:08:21.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.882+0000 7f69a0b70740 1 client.16464 shutdown 2024-05-22T23:08:21.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.882+0000 7f69a0b70740 1 -- 172.21.15.84:0/2068893690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449c570 msgr2=0x556d544ab350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:21.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.882+0000 7f69a0b70740 1 --2- 172.21.15.84:0/2068893690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449c570 0x556d544ab350 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6988046a50 tx=0x7f698804f690 comp rx=0 tx=0).stop 2024-05-22T23:08:21.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.882+0000 7f69a0b70740 1 -- 172.21.15.84:0/2068893690 shutdown_connections 2024-05-22T23:08:21.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.882+0000 7f69a0b70740 1 --2- 172.21.15.84:0/2068893690 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556d5451dfb0 0x556d545203d0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.882+0000 7f69a0b70740 1 --2- 172.21.15.84:0/2068893690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d544a6c40 0x556d5451c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.882+0000 7f69a0b70740 1 --2- 172.21.15.84:0/2068893690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d5449c570 0x556d544ab350 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.882+0000 7f6976ffd640 0 client.16464 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:21.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.882+0000 7f69a0b70740 1 --2- 172.21.15.84:0/2068893690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d5449bc50 0x556d544aae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.882+0000 7f69a0b70740 1 -- 172.21.15.84:0/2068893690 >> 172.21.15.84:0/2068893690 conn(0x556d54497f70 msgr2=0x556d54496920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:21.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.882+0000 7f69a0b70740 1 -- 172.21.15.84:0/2068893690 shutdown_connections 2024-05-22T23:08:21.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.882+0000 7f69a0b70740 1 -- 172.21.15.84:0/2068893690 wait complete. 2024-05-22T23:08:21.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.882+0000 7f69a0b70740 20 client.16464 trim_cache size 0 max 16384 2024-05-22T23:08:21.894 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:08:21.894 INFO:teuthology.orchestra.run.smithi084.stdout: "return_code": 0, 2024-05-22T23:08:21.894 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_tag": "3c5c2c9f-98ef-4174-a92a-93666f13d22c", 2024-05-22T23:08:21.894 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": "asynchronous" 2024-05-22T23:08:21.894 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:08:21.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.883+0000 7f699de66640 1 -- 172.21.15.84:0/2493668091 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f69800a2d00 msgr2=0x7f69800a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:21.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.883+0000 7f699de66640 1 --2- 172.21.15.84:0/2493668091 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f69800a2d00 0x7f69800a51c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f69900ccf50 tx=0x7f69900911b0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.883+0000 7f699de66640 1 -- 172.21.15.84:0/2493668091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6998107850 msgr2=0x7f699819ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:21.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.883+0000 7f699de66640 1 --2- 172.21.15.84:0/2493668091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6998107850 0x7f699819ca20 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f699000e940 tx=0x7f699000ee00 comp rx=0 tx=0).stop 2024-05-22T23:08:21.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.883+0000 7f699de66640 1 -- 172.21.15.84:0/2493668091 shutdown_connections 2024-05-22T23:08:21.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.883+0000 7f699de66640 1 --2- 172.21.15.84:0/2493668091 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f69800a2d00 0x7f69800a51c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.883+0000 7f699de66640 1 --2- 172.21.15.84:0/2493668091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6998107850 0x7f699819ca20 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.883+0000 7f699de66640 1 --2- 172.21.15.84:0/2493668091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6998102200 0x7f69981986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.883+0000 7f699de66640 1 --2- 172.21.15.84:0/2493668091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6998101840 0x7f6998198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:21.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.883+0000 7f699de66640 1 -- 172.21.15.84:0/2493668091 >> 172.21.15.84:0/2493668091 conn(0x7f6998078cc0 msgr2=0x7f69980ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:21.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.883+0000 7f699de66640 1 -- 172.21.15.84:0/2493668091 shutdown_connections 2024-05-22T23:08:21.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:21.883+0000 7f699de66640 1 -- 172.21.15.84:0/2493668091 wait complete. 2024-05-22T23:08:21.900 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:08:21.900 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:08:21.921 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:21 smithi084 ceph-mon[32186]: pgmap v200: 97 pgs: 97 active+clean; 425 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 288 op/s 2024-05-22T23:08:22.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.004+0000 7f0c383b7640 1 -- 172.21.15.84:0/86627175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c30101850 msgr2=0x7f0c30101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:22.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.004+0000 7f0c383b7640 1 --2- 172.21.15.84:0/86627175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c30101850 0x7f0c30101cd0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009930 tx=0x7f0c2002a420 comp rx=0 tx=0).stop 2024-05-22T23:08:22.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.004+0000 7f0c383b7640 1 -- 172.21.15.84:0/86627175 shutdown_connections 2024-05-22T23:08:22.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.004+0000 7f0c383b7640 1 --2- 172.21.15.84:0/86627175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c30102210 0x7f0c3010e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.004+0000 7f0c383b7640 1 --2- 172.21.15.84:0/86627175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c30101850 0x7f0c30101cd0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.004+0000 7f0c383b7640 1 --2- 172.21.15.84:0/86627175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c30107860 0x7f0c30107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.004+0000 7f0c383b7640 1 -- 172.21.15.84:0/86627175 >> 172.21.15.84:0/86627175 conn(0x7f0c30078c70 msgr2=0x7f0c30079080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:22.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.004+0000 7f0c383b7640 1 -- 172.21.15.84:0/86627175 shutdown_connections 2024-05-22T23:08:22.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.004+0000 7f0c383b7640 1 -- 172.21.15.84:0/86627175 wait complete. 2024-05-22T23:08:22.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.005+0000 7f0c383b7640 1 Processor -- start 2024-05-22T23:08:22.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.005+0000 7f0c383b7640 1 -- start start 2024-05-22T23:08:22.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.005+0000 7f0c383b7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c30101850 0x7f0c30198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:22.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.005+0000 7f0c383b7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c30102210 0x7f0c301986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:22.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.005+0000 7f0c383b7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c30107860 0x7f0c3019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:22.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.005+0000 7f0c383b7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0c30111c70 con 0x7f0c30102210 2024-05-22T23:08:22.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.005+0000 7f0c383b7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0c30111af0 con 0x7f0c30101850 2024-05-22T23:08:22.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.005+0000 7f0c383b7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c30111df0 con 0x7f0c30107860 2024-05-22T23:08:22.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.005+0000 7f0c35952640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c30102210 0x7f0c301986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:22.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.005+0000 7f0c35952640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c30102210 0x7f0c301986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39634/0 (socket says 172.21.15.84:39634) 2024-05-22T23:08:22.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.005+0000 7f0c35952640 1 -- 172.21.15.84:0/4076452294 learned_addr learned my addr 172.21.15.84:0/4076452294 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:22.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.005+0000 7f0c36153640 1 --2- 172.21.15.84:0/4076452294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c30101850 0x7f0c30198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:22.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.005+0000 7f0c36954640 1 --2- 172.21.15.84:0/4076452294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c30107860 0x7f0c3019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:22.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.005+0000 7f0c35952640 1 -- 172.21.15.84:0/4076452294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c30107860 msgr2=0x7f0c3019ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:22.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.005+0000 7f0c35952640 1 --2- 172.21.15.84:0/4076452294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c30107860 0x7f0c3019ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.005+0000 7f0c35952640 1 -- 172.21.15.84:0/4076452294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c30101850 msgr2=0x7f0c30198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:22.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.005+0000 7f0c35952640 1 --2- 172.21.15.84:0/4076452294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c30101850 0x7f0c30198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.005+0000 7f0c35952640 1 -- 172.21.15.84:0/4076452294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3019d180 con 0x7f0c30102210 2024-05-22T23:08:22.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.005+0000 7f0c36954640 1 --2- 172.21.15.84:0/4076452294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c30107860 0x7f0c3019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:08:22.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.006+0000 7f0c35952640 1 --2- 172.21.15.84:0/4076452294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c30102210 0x7f0c301986d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009230 tx=0x7f0c20004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:22.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.006+0000 7f0c277fe640 1 -- 172.21.15.84:0/4076452294 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2003f030 con 0x7f0c30102210 2024-05-22T23:08:22.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.006+0000 7f0c383b7640 1 -- 172.21.15.84:0/4076452294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c3019d410 con 0x7f0c30102210 2024-05-22T23:08:22.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.006+0000 7f0c277fe640 1 -- 172.21.15.84:0/4076452294 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c20003ad0 con 0x7f0c30102210 2024-05-22T23:08:22.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.006+0000 7f0c277fe640 1 -- 172.21.15.84:0/4076452294 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20035070 con 0x7f0c30102210 2024-05-22T23:08:22.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.006+0000 7f0c383b7640 1 -- 172.21.15.84:0/4076452294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c301a4d00 con 0x7f0c30102210 2024-05-22T23:08:22.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.008+0000 7f0c277fe640 1 -- 172.21.15.84:0/4076452294 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0c20042030 con 0x7f0c30102210 2024-05-22T23:08:22.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.009+0000 7f0c3b1a1740 1 Processor -- start 2024-05-22T23:08:22.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.009+0000 7f0c3b1a1740 1 -- start start 2024-05-22T23:08:22.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.009+0000 7f0c3b1a1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ea61b14a0 0x564ea61af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:22.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.009+0000 7f0c3b1a1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61afdb0 0x564ea61b0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:22.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.009+0000 7f0c3b1a1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ea61bab30 0x564ea61bcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:22.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.009+0000 7f0c3b1a1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564ea6113db0 con 0x564ea61afdb0 2024-05-22T23:08:22.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.009+0000 7f0c3b1a1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564ea60b2360 con 0x564ea61b14a0 2024-05-22T23:08:22.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.009+0000 7f0c3b1a1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564ea610fd70 con 0x564ea61bab30 2024-05-22T23:08:22.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.010+0000 7f0c36153640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61afdb0 0x564ea61b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:22.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.010+0000 7f0c36954640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ea61b14a0 0x564ea61af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:22.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.010+0000 7f0c36153640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61afdb0 0x564ea61b0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39640/0 (socket says 172.21.15.84:39640) 2024-05-22T23:08:22.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.010+0000 7f0c36153640 1 -- 172.21.15.84:0/727983167 learned_addr learned my addr 172.21.15.84:0/727983167 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:22.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.010+0000 7f0c277fe640 1 --2- 172.21.15.84:0/4076452294 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f0c1c0a2d00 0x7f0c1c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:22.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.010+0000 7f0c36954640 1 --2- 172.21.15.84:0/4076452294 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f0c1c0a2d00 0x7f0c1c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:22.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.010+0000 7f0c277fe640 1 -- 172.21.15.84:0/4076452294 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f0c200ef5c0 con 0x7f0c30102210 2024-05-22T23:08:22.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.010+0000 7f0c35952640 1 --2- 172.21.15.84:0/727983167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ea61bab30 0x564ea61bcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:22.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.010+0000 7f0c36153640 1 -- 172.21.15.84:0/727983167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ea61bab30 msgr2=0x564ea61bcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:22.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.010+0000 7f0c36153640 1 --2- 172.21.15.84:0/727983167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ea61bab30 0x564ea61bcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.010+0000 7f0c36153640 1 -- 172.21.15.84:0/727983167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ea61b14a0 msgr2=0x564ea61af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:22.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.010+0000 7f0c36153640 1 --2- 172.21.15.84:0/727983167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ea61b14a0 0x564ea61af7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.010+0000 7f0c36153640 1 -- 172.21.15.84:0/727983167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ea60d5a60 con 0x564ea61afdb0 2024-05-22T23:08:22.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.010+0000 7f0c35952640 1 --2- 172.21.15.84:0/727983167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ea61bab30 0x564ea61bcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:08:22.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.010+0000 7f0c36954640 1 --2- 172.21.15.84:0/727983167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ea61b14a0 0x564ea61af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:08:22.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.010+0000 7f0c36153640 1 --2- 172.21.15.84:0/727983167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61afdb0 0x564ea61b0230 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009c70 tx=0x7f0c2c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bae1b8b986cf884 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:22.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.010+0000 7f0c36954640 1 --2- 172.21.15.84:0/4076452294 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f0c1c0a2d00 0x7f0c1c0a51c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0c28005fd0 tx=0x7f0c28004ab0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:22.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.010+0000 7f0c24bfe640 1 -- 172.21.15.84:0/727983167 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c004230 con 0x564ea61afdb0 2024-05-22T23:08:22.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.010+0000 7f0c24bfe640 1 -- 172.21.15.84:0/727983167 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c2c0043d0 con 0x564ea61afdb0 2024-05-22T23:08:22.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.010+0000 7f0c24bfe640 1 -- 172.21.15.84:0/727983167 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c019740 con 0x564ea61afdb0 2024-05-22T23:08:22.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.011+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/727983167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61afdb0 msgr2=0x564ea61b0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:22.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.011+0000 7f0c3b1a1740 1 --2- 172.21.15.84:0/727983167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61afdb0 0x564ea61b0230 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009c70 tx=0x7f0c2c011810 comp rx=0 tx=0).stop 2024-05-22T23:08:22.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.011+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/727983167 shutdown_connections 2024-05-22T23:08:22.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.011+0000 7f0c3b1a1740 1 --2- 172.21.15.84:0/727983167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ea61bab30 0x564ea61bcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.011+0000 7f0c3b1a1740 1 --2- 172.21.15.84:0/727983167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61afdb0 0x564ea61b0230 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.011+0000 7f0c3b1a1740 1 --2- 172.21.15.84:0/727983167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ea61b14a0 0x564ea61af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.011+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/727983167 >> 172.21.15.84:0/727983167 conn(0x564ea61788e0 msgr2=0x564ea61ac310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:22.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.011+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/727983167 shutdown_connections 2024-05-22T23:08:22.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.011+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/727983167 wait complete. 2024-05-22T23:08:22.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.011+0000 7f0c3b1a1740 1 -- start start 2024-05-22T23:08:22.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.012+0000 7f0c3b1a1740 1 Processor -- start 2024-05-22T23:08:22.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.012+0000 7f0c3b1a1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ea61afdb0 0x564ea61b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:22.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.012+0000 7f0c3b1a1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61b14a0 0x564ea61b7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:22.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.012+0000 7f0c3b1a1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ea61bab30 0x564ea61b8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:22.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.012+0000 7f0c3b1a1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564ea61bd9c0 con 0x564ea61b14a0 2024-05-22T23:08:22.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.012+0000 7f0c3b1a1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564ea60ce0f0 con 0x564ea61afdb0 2024-05-22T23:08:22.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.012+0000 7f0c3b1a1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564ea61bdb40 con 0x564ea61bab30 2024-05-22T23:08:22.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.012+0000 7f0c3b1a1740 10 client.0 _ll_register_callbacks cb 0x564ea60d2f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:08:22.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.012+0000 7f0c35952640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61b14a0 0x564ea61b7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:22.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.012+0000 7f0c35952640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61b14a0 0x564ea61b7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39646/0 (socket says 172.21.15.84:39646) 2024-05-22T23:08:22.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.012+0000 7f0c35952640 1 -- 172.21.15.84:0/3501958489 learned_addr learned my addr 172.21.15.84:0/3501958489 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:22.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.012+0000 7f0c36153640 1 --2- 172.21.15.84:0/3501958489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ea61afdb0 0x564ea61b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:22.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.012+0000 7f0c36954640 1 --2- 172.21.15.84:0/3501958489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ea61bab30 0x564ea61b8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:22.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.012+0000 7f0c35952640 1 -- 172.21.15.84:0/3501958489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ea61bab30 msgr2=0x564ea61b8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:22.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.012+0000 7f0c35952640 1 --2- 172.21.15.84:0/3501958489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ea61bab30 0x564ea61b8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.012+0000 7f0c35952640 1 -- 172.21.15.84:0/3501958489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ea61afdb0 msgr2=0x564ea61b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:22.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.012+0000 7f0c35952640 1 --2- 172.21.15.84:0/3501958489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ea61afdb0 0x564ea61b79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.012+0000 7f0c35952640 1 -- 172.21.15.84:0/3501958489 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ea5ef17d0 con 0x564ea61b14a0 2024-05-22T23:08:22.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.012+0000 7f0c36153640 1 --2- 172.21.15.84:0/3501958489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ea61afdb0 0x564ea61b79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:08:22.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.012+0000 7f0c35952640 1 --2- 172.21.15.84:0/3501958489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61b14a0 0x564ea61b7ee0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0c20000c00 tx=0x7f0c200bb4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b90e66f9dce4a4e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:22.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.012+0000 7f0c19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:08:22.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.012+0000 7f0c19ffb640 1 -- 172.21.15.84:0/3501958489 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20049040 con 0x564ea61b14a0 2024-05-22T23:08:22.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.012+0000 7f0c19ffb640 1 -- 172.21.15.84:0/3501958489 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f0c20003c80 con 0x564ea61b14a0 2024-05-22T23:08:22.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.012+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/3501958489 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c2c0043d0 con 0x564ea61b14a0 2024-05-22T23:08:22.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.013+0000 7f0c19ffb640 1 -- 172.21.15.84:0/3501958489 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f0c200bba90 con 0x564ea61b14a0 2024-05-22T23:08:22.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.013+0000 7f0c3b1a1740 10 client.16476 fetch_fsmap learned FSMap version 18 2024-05-22T23:08:22.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.013+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/3501958489 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x564ea5e273f0 con 0x564ea61b14a0 2024-05-22T23:08:22.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.013+0000 7f0c19ffb640 1 -- 172.21.15.84:0/3501958489 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c2c0043d0 con 0x564ea61b14a0 2024-05-22T23:08:22.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.013+0000 7f0c19ffb640 1 -- 172.21.15.84:0/3501958489 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f0c2004c060 con 0x564ea61b14a0 2024-05-22T23:08:22.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.013+0000 7f0c3b1a1740 10 client.16476 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:08:22.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.013+0000 7f0c3b1a1740 10 client.16476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:08:22.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.013+0000 7f0c3b1a1740 20 client.16476 populate_metadata read hostname 'smithi084' 2024-05-22T23:08:22.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.013+0000 7f0c3b1a1740 1 --2- 172.21.15.84:0/3501958489 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564ea61cad70 0x564ea61cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:22.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.013+0000 7f0c3b1a1740 4 client.16476 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:08:22.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.013+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/3501958489 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ea61214e0 con 0x564ea61cad70 2024-05-22T23:08:22.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.013+0000 7f0c36153640 1 --2- 172.21.15.84:0/3501958489 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564ea61cad70 0x564ea61cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:22.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.013+0000 7f0c19ffb640 10 client.16476 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:22.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.013+0000 7f0c36153640 1 --2- 172.21.15.84:0/3501958489 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564ea61cad70 0x564ea61cd1c0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecd5bc3c551f240f server_cookie=53a1a874e5efed34 in_seq=0 out_seq=0 2024-05-22T23:08:22.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.015+0000 7f0c19ffb640 1 -- 172.21.15.84:0/3501958489 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564ea61214e0 con 0x564ea61cad70 2024-05-22T23:08:22.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.015+0000 7f0c19ffb640 10 client.16476 handle_command_reply: tid=0 2024-05-22T23:08:22.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.015+0000 7f0c3b1a1740 1 client.16476 shutdown 2024-05-22T23:08:22.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.015+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/3501958489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61b14a0 msgr2=0x564ea61b7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:22.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.015+0000 7f0c3b1a1740 1 --2- 172.21.15.84:0/3501958489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61b14a0 0x564ea61b7ee0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0c20000c00 tx=0x7f0c200bb4b0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.015+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/3501958489 shutdown_connections 2024-05-22T23:08:22.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.015+0000 7f0c3b1a1740 1 --2- 172.21.15.84:0/3501958489 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564ea61cad70 0x564ea61cd1c0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.015+0000 7f0c3b1a1740 1 --2- 172.21.15.84:0/3501958489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ea61bab30 0x564ea61b8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.015+0000 7f0c3b1a1740 1 --2- 172.21.15.84:0/3501958489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61b14a0 0x564ea61b7ee0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.015+0000 7f0c3b1a1740 1 --2- 172.21.15.84:0/3501958489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ea61afdb0 0x564ea61b79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.015+0000 7f0c19ffb640 0 client.16476 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:22.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.015+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/3501958489 >> 172.21.15.84:0/3501958489 conn(0x564ea61788e0 msgr2=0x564ea61b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:22.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.015+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/3501958489 shutdown_connections 2024-05-22T23:08:22.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.015+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/3501958489 wait complete. 2024-05-22T23:08:22.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.015+0000 7f0c3b1a1740 20 client.16476 trim_cache size 0 max 16384 2024-05-22T23:08:22.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.034+0000 7f0c3b1a1740 1 Processor -- start 2024-05-22T23:08:22.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.034+0000 7f0c3b1a1740 1 -- start start 2024-05-22T23:08:22.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.034+0000 7f0c3b1a1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61afdb0 0x564ea61c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:22.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.034+0000 7f0c3b1a1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ea61b14a0 0x564ea61a7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:22.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.034+0000 7f0c3b1a1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ea61bab30 0x564ea61b98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:22.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.034+0000 7f0c3b1a1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301ab6f0 con 0x564ea61afdb0 2024-05-22T23:08:22.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.034+0000 7f0c3b1a1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301adf50 con 0x564ea61bab30 2024-05-22T23:08:22.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.034+0000 7f0c3b1a1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301abd40 con 0x564ea61b14a0 2024-05-22T23:08:22.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.034+0000 7f0c36153640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61afdb0 0x564ea61c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:22.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.034+0000 7f0c36153640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61afdb0 0x564ea61c68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39654/0 (socket says 172.21.15.84:39654) 2024-05-22T23:08:22.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.034+0000 7f0c36153640 1 -- 172.21.15.84:0/1557174395 learned_addr learned my addr 172.21.15.84:0/1557174395 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:22.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.034+0000 7f0c35952640 1 --2- 172.21.15.84:0/1557174395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ea61b14a0 0x564ea61a7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:22.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.034+0000 7f0c36954640 1 --2- 172.21.15.84:0/1557174395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ea61bab30 0x564ea61b98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:22.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.035+0000 7f0c36954640 1 -- 172.21.15.84:0/1557174395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ea61b14a0 msgr2=0x564ea61a7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:22.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.035+0000 7f0c36954640 1 --2- 172.21.15.84:0/1557174395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ea61b14a0 0x564ea61a7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.035+0000 7f0c36954640 1 -- 172.21.15.84:0/1557174395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61afdb0 msgr2=0x564ea61c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:22.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.035+0000 7f0c36954640 1 --2- 172.21.15.84:0/1557174395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61afdb0 0x564ea61c68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.035+0000 7f0c36954640 1 -- 172.21.15.84:0/1557174395 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ea61b6950 con 0x564ea61bab30 2024-05-22T23:08:22.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.035+0000 7f0c36153640 1 --2- 172.21.15.84:0/1557174395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61afdb0 0x564ea61c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:08:22.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.035+0000 7f0c36954640 1 --2- 172.21.15.84:0/1557174395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ea61bab30 0x564ea61b98e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0c28012880 tx=0x7f0c28009900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c04bd51a641251a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:22.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.035+0000 7f0c1bfff640 1 -- 172.21.15.84:0/1557174395 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2801a070 con 0x564ea61bab30 2024-05-22T23:08:22.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.035+0000 7f0c1bfff640 1 -- 172.21.15.84:0/1557174395 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c28014220 con 0x564ea61bab30 2024-05-22T23:08:22.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.035+0000 7f0c1bfff640 1 -- 172.21.15.84:0/1557174395 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c28013df0 con 0x564ea61bab30 2024-05-22T23:08:22.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.036+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/1557174395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ea61bab30 msgr2=0x564ea61b98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:22.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.036+0000 7f0c3b1a1740 1 --2- 172.21.15.84:0/1557174395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ea61bab30 0x564ea61b98e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0c28012880 tx=0x7f0c28009900 comp rx=0 tx=0).stop 2024-05-22T23:08:22.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.036+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/1557174395 shutdown_connections 2024-05-22T23:08:22.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.036+0000 7f0c3b1a1740 1 --2- 172.21.15.84:0/1557174395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ea61bab30 0x564ea61b98e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.036+0000 7f0c3b1a1740 1 --2- 172.21.15.84:0/1557174395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ea61b14a0 0x564ea61a7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.036+0000 7f0c3b1a1740 1 --2- 172.21.15.84:0/1557174395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61afdb0 0x564ea61c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.036+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/1557174395 >> 172.21.15.84:0/1557174395 conn(0x564ea61788e0 msgr2=0x564ea61b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:22.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.036+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/1557174395 shutdown_connections 2024-05-22T23:08:22.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.036+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/1557174395 wait complete. 2024-05-22T23:08:22.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.036+0000 7f0c3b1a1740 1 -- start start 2024-05-22T23:08:22.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.036+0000 7f0c3b1a1740 1 Processor -- start 2024-05-22T23:08:22.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.036+0000 7f0c3b1a1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61afdb0 0x564ea61bb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:22.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.036+0000 7f0c3b1a1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ea61b14a0 0x564ea61bbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:22.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.036+0000 7f0c3b1a1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ea61bab30 0x564ea61bc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:22.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.036+0000 7f0c3b1a1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301ac410 con 0x564ea61afdb0 2024-05-22T23:08:22.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.036+0000 7f0c3b1a1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301ad880 con 0x564ea61bab30 2024-05-22T23:08:22.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.037+0000 7f0c3b1a1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301acae0 con 0x564ea61b14a0 2024-05-22T23:08:22.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.037+0000 7f0c3b1a1740 10 client.0 _ll_register_callbacks cb 0x7f0c20005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:08:22.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.037+0000 7f0c36153640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61afdb0 0x564ea61bb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:22.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.037+0000 7f0c36153640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61afdb0 0x564ea61bb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39658/0 (socket says 172.21.15.84:39658) 2024-05-22T23:08:22.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.037+0000 7f0c36153640 1 -- 172.21.15.84:0/3280456288 learned_addr learned my addr 172.21.15.84:0/3280456288 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:22.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.037+0000 7f0c36954640 1 --2- 172.21.15.84:0/3280456288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ea61bab30 0x564ea61bc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:22.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.037+0000 7f0c35952640 1 --2- 172.21.15.84:0/3280456288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ea61b14a0 0x564ea61bbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:22.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.037+0000 7f0c36153640 1 -- 172.21.15.84:0/3280456288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ea61b14a0 msgr2=0x564ea61bbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:22.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.037+0000 7f0c36153640 1 --2- 172.21.15.84:0/3280456288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ea61b14a0 0x564ea61bbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.037+0000 7f0c36153640 1 -- 172.21.15.84:0/3280456288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ea61bab30 msgr2=0x564ea61bc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:22.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.037+0000 7f0c36153640 1 --2- 172.21.15.84:0/3280456288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ea61bab30 0x564ea61bc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.037+0000 7f0c36153640 1 -- 172.21.15.84:0/3280456288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ea61a79e0 con 0x564ea61afdb0 2024-05-22T23:08:22.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.037+0000 7f0c36153640 1 --2- 172.21.15.84:0/3280456288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61afdb0 0x564ea61bb840 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c000c00 tx=0x7f0c2c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e6cd95adee4c47 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:22.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.037+0000 7f0c19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:08:22.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.037+0000 7f0c19ffb640 1 -- 172.21.15.84:0/3280456288 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c005410 con 0x564ea61afdb0 2024-05-22T23:08:22.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.037+0000 7f0c19ffb640 1 -- 172.21.15.84:0/3280456288 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f0c2c0055b0 con 0x564ea61afdb0 2024-05-22T23:08:22.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.037+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/3280456288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c28014220 con 0x564ea61afdb0 2024-05-22T23:08:22.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.037+0000 7f0c19ffb640 1 -- 172.21.15.84:0/3280456288 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f0c2c0035f0 con 0x564ea61afdb0 2024-05-22T23:08:22.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.037+0000 7f0c3b1a1740 10 client.16482 fetch_fsmap learned FSMap version 18 2024-05-22T23:08:22.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.037+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/3280456288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x564ea5de70f0 con 0x564ea61afdb0 2024-05-22T23:08:22.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.037+0000 7f0c19ffb640 1 -- 172.21.15.84:0/3280456288 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c28014220 con 0x564ea61afdb0 2024-05-22T23:08:22.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.038+0000 7f0c19ffb640 1 -- 172.21.15.84:0/3280456288 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f0c2c018070 con 0x564ea61afdb0 2024-05-22T23:08:22.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.038+0000 7f0c3b1a1740 10 client.16482 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:08:22.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.038+0000 7f0c3b1a1740 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:08:22.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.038+0000 7f0c3b1a1740 20 client.16482 populate_metadata read hostname 'smithi084' 2024-05-22T23:08:22.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.038+0000 7f0c3b1a1740 1 --2- 172.21.15.84:0/3280456288 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564ea61cad70 0x564ea6237ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:22.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.038+0000 7f0c3b1a1740 4 client.16482 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:08:22.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.038+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/3280456288 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564ea611b830 con 0x564ea61cad70 2024-05-22T23:08:22.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.038+0000 7f0c35952640 1 --2- 172.21.15.84:0/3280456288 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564ea61cad70 0x564ea6237ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:22.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.038+0000 7f0c35952640 1 --2- 172.21.15.84:0/3280456288 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564ea61cad70 0x564ea6237ea0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4798f4c4f778dffc server_cookie=cd7a013fd3eb7fc2 in_seq=0 out_seq=0 2024-05-22T23:08:22.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.038+0000 7f0c19ffb640 10 client.16482 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:22.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.040+0000 7f0c19ffb640 1 -- 172.21.15.84:0/3280456288 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564ea611b830 con 0x564ea61cad70 2024-05-22T23:08:22.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.040+0000 7f0c19ffb640 10 client.16482 handle_command_reply: tid=0 2024-05-22T23:08:22.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.040+0000 7f0c3b1a1740 1 client.16482 shutdown 2024-05-22T23:08:22.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.040+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/3280456288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61afdb0 msgr2=0x564ea61bb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:22.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.040+0000 7f0c3b1a1740 1 --2- 172.21.15.84:0/3280456288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61afdb0 0x564ea61bb840 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c000c00 tx=0x7f0c2c009360 comp rx=0 tx=0).stop 2024-05-22T23:08:22.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.040+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/3280456288 shutdown_connections 2024-05-22T23:08:22.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.040+0000 7f0c3b1a1740 1 --2- 172.21.15.84:0/3280456288 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x564ea61cad70 0x564ea6237ea0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.040+0000 7f0c3b1a1740 1 --2- 172.21.15.84:0/3280456288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ea61bab30 0x564ea61bc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.040+0000 7f0c19ffb640 0 client.16482 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:22.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.040+0000 7f0c3b1a1740 1 --2- 172.21.15.84:0/3280456288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ea61b14a0 0x564ea61bbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.040+0000 7f0c3b1a1740 1 --2- 172.21.15.84:0/3280456288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ea61afdb0 0x564ea61bb840 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.040+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/3280456288 >> 172.21.15.84:0/3280456288 conn(0x564ea61788e0 msgr2=0x564ea61b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:22.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.040+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/3280456288 shutdown_connections 2024-05-22T23:08:22.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.040+0000 7f0c3b1a1740 1 -- 172.21.15.84:0/3280456288 wait complete. 2024-05-22T23:08:22.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.040+0000 7f0c3b1a1740 20 client.16482 trim_cache size 0 max 16384 2024-05-22T23:08:22.052 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:08:22.052 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-22T23:08:22.052 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": { 2024-05-22T23:08:22.052 INFO:teuthology.orchestra.run.smithi084.stdout: "3c5c2c9f-98ef-4174-a92a-93666f13d22c": { 2024-05-22T23:08:22.052 INFO:teuthology.orchestra.run.smithi084.stdout: "path": "/", 2024-05-22T23:08:22.052 INFO:teuthology.orchestra.run.smithi084.stdout: "tag": "3c5c2c9f-98ef-4174-a92a-93666f13d22c", 2024-05-22T23:08:22.053 INFO:teuthology.orchestra.run.smithi084.stdout: "options": "recursive,force" 2024-05-22T23:08:22.053 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:08:22.053 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:08:22.053 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:08:22.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.041+0000 7f0c383b7640 1 -- 172.21.15.84:0/4076452294 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f0c1c0a2d00 msgr2=0x7f0c1c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:22.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.041+0000 7f0c383b7640 1 --2- 172.21.15.84:0/4076452294 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f0c1c0a2d00 0x7f0c1c0a51c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0c28005fd0 tx=0x7f0c28004ab0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.041+0000 7f0c383b7640 1 -- 172.21.15.84:0/4076452294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c30102210 msgr2=0x7f0c301986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:22.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.041+0000 7f0c383b7640 1 --2- 172.21.15.84:0/4076452294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c30102210 0x7f0c301986d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009230 tx=0x7f0c20004270 comp rx=0 tx=0).stop 2024-05-22T23:08:22.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.041+0000 7f0c383b7640 1 -- 172.21.15.84:0/4076452294 shutdown_connections 2024-05-22T23:08:22.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.041+0000 7f0c383b7640 1 --2- 172.21.15.84:0/4076452294 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f0c1c0a2d00 0x7f0c1c0a51c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.041+0000 7f0c383b7640 1 --2- 172.21.15.84:0/4076452294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c30107860 0x7f0c3019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.041+0000 7f0c383b7640 1 --2- 172.21.15.84:0/4076452294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c30102210 0x7f0c301986d0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.041+0000 7f0c383b7640 1 --2- 172.21.15.84:0/4076452294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c30101850 0x7f0c30198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:22.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.041+0000 7f0c383b7640 1 -- 172.21.15.84:0/4076452294 >> 172.21.15.84:0/4076452294 conn(0x7f0c30078c70 msgr2=0x7f0c301042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:22.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.041+0000 7f0c383b7640 1 -- 172.21.15.84:0/4076452294 shutdown_connections 2024-05-22T23:08:22.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:22.041+0000 7f0c383b7640 1 -- 172.21.15.84:0/4076452294 wait complete. 2024-05-22T23:08:22.055 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:21 smithi096 ceph-mon[36392]: pgmap v200: 97 pgs: 97 active+clean; 425 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 288 op/s 2024-05-22T23:08:22.057 INFO:tasks.cephfs.filesystem:scrub status for tag:3c5c2c9f-98ef-4174-a92a-93666f13d22c - {'path': '/', 'tag': '3c5c2c9f-98ef-4174-a92a-93666f13d22c', 'options': 'recursive,force'} 2024-05-22T23:08:22.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:21 smithi204 ceph-mon[36211]: pgmap v200: 97 pgs: 97 active+clean; 425 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 288 op/s 2024-05-22T23:08:22.440 INFO:tasks.workunit.client.0.smithi084.stdout: 1 871093 14.97 MB/sec execute 188 sec latency 77.200 ms 2024-05-22T23:08:23.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:22 smithi096 ceph-mon[36392]: scrub queued for path: / 2024-05-22T23:08:23.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:22 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:08:23.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:22 smithi096 ceph-mon[36392]: scrub summary: active paths [/] 2024-05-22T23:08:23.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:22 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:08:23.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:22 smithi084 ceph-mon[32186]: scrub queued for path: / 2024-05-22T23:08:23.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:22 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:08:23.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:22 smithi084 ceph-mon[32186]: scrub summary: active paths [/] 2024-05-22T23:08:23.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:22 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:08:23.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:22 smithi204 ceph-mon[36211]: scrub queued for path: / 2024-05-22T23:08:23.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:22 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:08:23.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:22 smithi204 ceph-mon[36211]: scrub summary: active paths [/] 2024-05-22T23:08:23.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:22 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:08:23.440 INFO:tasks.workunit.client.0.smithi084.stdout: 1 873951 14.98 MB/sec execute 189 sec latency 18.180 ms 2024-05-22T23:08:24.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:23 smithi096 ceph-mon[36392]: pgmap v201: 97 pgs: 97 active+clean; 423 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 288 op/s 2024-05-22T23:08:24.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:23 smithi084 ceph-mon[32186]: pgmap v201: 97 pgs: 97 active+clean; 423 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 288 op/s 2024-05-22T23:08:24.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:23 smithi204 ceph-mon[36211]: pgmap v201: 97 pgs: 97 active+clean; 423 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 288 op/s 2024-05-22T23:08:24.440 INFO:tasks.workunit.client.0.smithi084.stdout: 1 877050 15.00 MB/sec execute 190 sec latency 21.608 ms 2024-05-22T23:08:25.440 INFO:tasks.workunit.client.0.smithi084.stdout: 1 879533 14.97 MB/sec execute 191 sec latency 14.158 ms 2024-05-22T23:08:25.967 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:25 smithi204 ceph-mon[36211]: pgmap v202: 97 pgs: 97 active+clean; 423 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 288 op/s 2024-05-22T23:08:26.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:25 smithi096 ceph-mon[36392]: pgmap v202: 97 pgs: 97 active+clean; 423 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 288 op/s 2024-05-22T23:08:26.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:25 smithi084 ceph-mon[32186]: pgmap v202: 97 pgs: 97 active+clean; 423 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 288 op/s 2024-05-22T23:08:26.441 INFO:tasks.workunit.client.0.smithi084.stdout: 1 881911 14.97 MB/sec execute 192 sec latency 24.251 ms 2024-05-22T23:08:27.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:26 smithi096 ceph-mon[36392]: scrub summary: idle 2024-05-22T23:08:27.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:26 smithi084 ceph-mon[32186]: scrub summary: idle 2024-05-22T23:08:27.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:26 smithi204 ceph-mon[36211]: scrub summary: idle 2024-05-22T23:08:27.440 INFO:tasks.workunit.client.0.smithi084.stdout: 1 884427 14.97 MB/sec execute 193 sec latency 80.642 ms 2024-05-22T23:08:28.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:27 smithi096 ceph-mon[36392]: pgmap v203: 97 pgs: 97 active+clean; 409 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.0 MiB/s wr, 285 op/s 2024-05-22T23:08:28.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:27 smithi084 ceph-mon[32186]: pgmap v203: 97 pgs: 97 active+clean; 409 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.0 MiB/s wr, 285 op/s 2024-05-22T23:08:28.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:27 smithi204 ceph-mon[36211]: pgmap v203: 97 pgs: 97 active+clean; 409 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.0 MiB/s wr, 285 op/s 2024-05-22T23:08:28.440 INFO:tasks.workunit.client.0.smithi084.stdout: 1 887225 14.96 MB/sec execute 194 sec latency 20.818 ms 2024-05-22T23:08:29.440 INFO:tasks.workunit.client.0.smithi084.stdout: 1 889963 14.95 MB/sec execute 195 sec latency 14.649 ms 2024-05-22T23:08:30.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:29 smithi096 ceph-mon[36392]: pgmap v204: 97 pgs: 97 active+clean; 409 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 261 op/s 2024-05-22T23:08:30.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:29 smithi084 ceph-mon[32186]: pgmap v204: 97 pgs: 97 active+clean; 409 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 261 op/s 2024-05-22T23:08:30.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:29 smithi204 ceph-mon[36211]: pgmap v204: 97 pgs: 97 active+clean; 409 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 261 op/s 2024-05-22T23:08:30.440 INFO:tasks.workunit.client.0.smithi084.stdout: 1 892367 14.95 MB/sec execute 196 sec latency 24.451 ms 2024-05-22T23:08:31.440 INFO:tasks.workunit.client.0.smithi084.stdout: 1 895417 14.96 MB/sec execute 197 sec latency 14.491 ms 2024-05-22T23:08:32.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:31 smithi096 ceph-mon[36392]: pgmap v205: 97 pgs: 97 active+clean; 409 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 238 op/s 2024-05-22T23:08:32.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:31 smithi084 ceph-mon[32186]: pgmap v205: 97 pgs: 97 active+clean; 409 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 238 op/s 2024-05-22T23:08:32.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:31 smithi204 ceph-mon[36211]: pgmap v205: 97 pgs: 97 active+clean; 409 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 238 op/s 2024-05-22T23:08:32.440 INFO:tasks.workunit.client.0.smithi084.stdout: 1 898211 14.97 MB/sec execute 198 sec latency 82.643 ms 2024-05-22T23:08:33.441 INFO:tasks.workunit.client.0.smithi084.stdout: 1 900801 14.94 MB/sec execute 199 sec latency 23.197 ms 2024-05-22T23:08:34.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:33 smithi096 ceph-mon[36392]: pgmap v206: 97 pgs: 97 active+clean; 421 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.4 MiB/s wr, 357 op/s 2024-05-22T23:08:34.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:33 smithi084 ceph-mon[32186]: pgmap v206: 97 pgs: 97 active+clean; 421 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.4 MiB/s wr, 357 op/s 2024-05-22T23:08:34.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:33 smithi204 ceph-mon[36211]: pgmap v206: 97 pgs: 97 active+clean; 421 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.4 MiB/s wr, 357 op/s 2024-05-22T23:08:34.440 INFO:tasks.workunit.client.0.smithi084.stdout: 1 903452 14.95 MB/sec execute 200 sec latency 18.456 ms 2024-05-22T23:08:35.440 INFO:tasks.workunit.client.0.smithi084.stdout: 1 906234 14.95 MB/sec execute 201 sec latency 13.663 ms 2024-05-22T23:08:36.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:35 smithi096 ceph-mon[36392]: pgmap v207: 97 pgs: 97 active+clean; 421 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 238 op/s 2024-05-22T23:08:36.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:35 smithi084 ceph-mon[32186]: pgmap v207: 97 pgs: 97 active+clean; 421 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 238 op/s 2024-05-22T23:08:36.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:35 smithi204 ceph-mon[36211]: pgmap v207: 97 pgs: 97 active+clean; 421 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 238 op/s 2024-05-22T23:08:36.440 INFO:tasks.workunit.client.0.smithi084.stdout: 1 909447 14.96 MB/sec execute 202 sec latency 10.869 ms 2024-05-22T23:08:37.440 INFO:tasks.workunit.client.0.smithi084.stdout: 1 912470 14.98 MB/sec execute 203 sec latency 81.332 ms 2024-05-22T23:08:38.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:37 smithi096 ceph-mon[36392]: pgmap v208: 97 pgs: 97 active+clean; 432 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.4 MiB/s wr, 360 op/s 2024-05-22T23:08:38.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:37 smithi084 ceph-mon[32186]: pgmap v208: 97 pgs: 97 active+clean; 432 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.4 MiB/s wr, 360 op/s 2024-05-22T23:08:38.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:37 smithi204 ceph-mon[36211]: pgmap v208: 97 pgs: 97 active+clean; 432 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.4 MiB/s wr, 360 op/s 2024-05-22T23:08:38.440 INFO:tasks.workunit.client.0.smithi084.stdout: 1 915148 14.97 MB/sec execute 204 sec latency 39.584 ms 2024-05-22T23:08:39.440 INFO:tasks.workunit.client.0.smithi084.stdout: 1 917053 15.00 MB/sec execute 205 sec latency 31.656 ms 2024-05-22T23:08:40.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:39 smithi096 ceph-mon[36392]: pgmap v209: 97 pgs: 97 active+clean; 432 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.3 MiB/s wr, 241 op/s 2024-05-22T23:08:40.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:39 smithi084 ceph-mon[32186]: pgmap v209: 97 pgs: 97 active+clean; 432 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.3 MiB/s wr, 241 op/s 2024-05-22T23:08:40.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:39 smithi204 ceph-mon[36211]: pgmap v209: 97 pgs: 97 active+clean; 432 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.3 MiB/s wr, 241 op/s 2024-05-22T23:08:40.440 INFO:tasks.workunit.client.0.smithi084.stdout: 1 920351 15.05 MB/sec execute 206 sec latency 21.192 ms 2024-05-22T23:08:41.440 INFO:tasks.workunit.client.0.smithi084.stdout: 1 923145 15.04 MB/sec execute 207 sec latency 13.983 ms 2024-05-22T23:08:42.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:41 smithi096 ceph-mon[36392]: pgmap v210: 97 pgs: 97 active+clean; 432 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.3 MiB/s wr, 241 op/s 2024-05-22T23:08:42.130 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:41 smithi084 ceph-mon[32186]: pgmap v210: 97 pgs: 97 active+clean; 432 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.3 MiB/s wr, 241 op/s 2024-05-22T23:08:42.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:41 smithi204 ceph-mon[36211]: pgmap v210: 97 pgs: 97 active+clean; 432 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.3 MiB/s wr, 241 op/s 2024-05-22T23:08:42.440 INFO:tasks.workunit.client.0.smithi084.stdout: 1 925010 15.01 MB/sec execute 208 sec latency 52.621 ms 2024-05-22T23:08:43.441 INFO:tasks.workunit.client.0.smithi084.stdout: 1 927998 15.02 MB/sec execute 209 sec latency 25.154 ms 2024-05-22T23:08:44.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:43 smithi096 ceph-mon[36392]: pgmap v211: 97 pgs: 97 active+clean; 443 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.5 MiB/s wr, 363 op/s 2024-05-22T23:08:44.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:43 smithi084 ceph-mon[32186]: pgmap v211: 97 pgs: 97 active+clean; 443 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.5 MiB/s wr, 363 op/s 2024-05-22T23:08:44.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:43 smithi204 ceph-mon[36211]: pgmap v211: 97 pgs: 97 active+clean; 443 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.5 MiB/s wr, 363 op/s 2024-05-22T23:08:44.441 INFO:tasks.workunit.client.0.smithi084.stdout: 1 931100 15.04 MB/sec execute 210 sec latency 52.894 ms 2024-05-22T23:08:45.441 INFO:tasks.workunit.client.0.smithi084.stdout: 1 933182 15.00 MB/sec execute 211 sec latency 23.820 ms 2024-05-22T23:08:46.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:45 smithi096 ceph-mon[36392]: pgmap v212: 97 pgs: 97 active+clean; 443 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.3 MiB/s wr, 245 op/s 2024-05-22T23:08:46.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:45 smithi084 ceph-mon[32186]: pgmap v212: 97 pgs: 97 active+clean; 443 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.3 MiB/s wr, 245 op/s 2024-05-22T23:08:46.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:45 smithi204 ceph-mon[36211]: pgmap v212: 97 pgs: 97 active+clean; 443 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.3 MiB/s wr, 245 op/s 2024-05-22T23:08:46.441 INFO:tasks.workunit.client.0.smithi084.stdout: 1 935711 15.00 MB/sec execute 212 sec latency 21.647 ms 2024-05-22T23:08:47.441 INFO:tasks.workunit.client.0.smithi084.stdout: 1 938337 15.01 MB/sec execute 213 sec latency 42.390 ms 2024-05-22T23:08:48.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:47 smithi096 ceph-mon[36392]: pgmap v213: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.4 MiB/s wr, 361 op/s 2024-05-22T23:08:48.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:47 smithi084 ceph-mon[32186]: pgmap v213: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.4 MiB/s wr, 361 op/s 2024-05-22T23:08:48.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:47 smithi204 ceph-mon[36211]: pgmap v213: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.4 MiB/s wr, 361 op/s 2024-05-22T23:08:48.441 INFO:tasks.workunit.client.0.smithi084.stdout: 1 940348 14.97 MB/sec execute 214 sec latency 93.982 ms 2024-05-22T23:08:49.441 INFO:tasks.workunit.client.0.smithi084.stdout: 1 942987 14.98 MB/sec execute 215 sec latency 13.741 ms 2024-05-22T23:08:50.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:49 smithi096 ceph-mon[36392]: pgmap v214: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 238 op/s 2024-05-22T23:08:50.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:49 smithi084 ceph-mon[32186]: pgmap v214: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 238 op/s 2024-05-22T23:08:50.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:49 smithi204 ceph-mon[36211]: pgmap v214: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 238 op/s 2024-05-22T23:08:50.441 INFO:tasks.workunit.client.0.smithi084.stdout: 1 945561 14.98 MB/sec execute 216 sec latency 16.255 ms 2024-05-22T23:08:51.441 INFO:tasks.workunit.client.0.smithi084.stdout: 1 948027 14.97 MB/sec execute 217 sec latency 41.354 ms 2024-05-22T23:08:52.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:51 smithi096 ceph-mon[36392]: pgmap v215: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 238 op/s 2024-05-22T23:08:52.058 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:08:52.058 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:08:52.081 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:51 smithi084 ceph-mon[32186]: pgmap v215: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 238 op/s 2024-05-22T23:08:52.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:51 smithi204 ceph-mon[36211]: pgmap v215: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 238 op/s 2024-05-22T23:08:52.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.160+0000 7f6c56f90640 1 -- 172.21.15.84:0/1111100835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c50103410 msgr2=0x7f6c5010b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.160+0000 7f6c56f90640 1 --2- 172.21.15.84:0/1111100835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c50103410 0x7f6c5010b8f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009900 tx=0x7f6c3c02a510 comp rx=0 tx=0).stop 2024-05-22T23:08:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.160+0000 7f6c56f90640 1 -- 172.21.15.84:0/1111100835 shutdown_connections 2024-05-22T23:08:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.160+0000 7f6c56f90640 1 --2- 172.21.15.84:0/1111100835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c5010be30 0x7f6c5010e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.160+0000 7f6c56f90640 1 --2- 172.21.15.84:0/1111100835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c50103410 0x7f6c5010b8f0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.160+0000 7f6c56f90640 1 --2- 172.21.15.84:0/1111100835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c50102af0 0x7f6c50102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.160+0000 7f6c56f90640 1 -- 172.21.15.84:0/1111100835 >> 172.21.15.84:0/1111100835 conn(0x7f6c50077870 msgr2=0x7f6c50077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.160+0000 7f6c56f90640 1 -- 172.21.15.84:0/1111100835 shutdown_connections 2024-05-22T23:08:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.160+0000 7f6c56f90640 1 -- 172.21.15.84:0/1111100835 wait complete. 2024-05-22T23:08:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.161+0000 7f6c56f90640 1 Processor -- start 2024-05-22T23:08:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.161+0000 7f6c56f90640 1 -- start start 2024-05-22T23:08:52.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.161+0000 7f6c56f90640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c50102af0 0x7f6c50198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.161+0000 7f6c56f90640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c50103410 0x7f6c50198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.161+0000 7f6c56f90640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c5010be30 0x7f6c5019ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.161+0000 7f6c56f90640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6c50111c30 con 0x7f6c50102af0 2024-05-22T23:08:52.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.161+0000 7f6c56f90640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6c50111ab0 con 0x7f6c50103410 2024-05-22T23:08:52.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.161+0000 7f6c56f90640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6c50111db0 con 0x7f6c5010be30 2024-05-22T23:08:52.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.161+0000 7f6c54d2c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c50102af0 0x7f6c50198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.161+0000 7f6c54d2c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c50102af0 0x7f6c50198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49430/0 (socket says 172.21.15.84:49430) 2024-05-22T23:08:52.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.161+0000 7f6c54d2c640 1 -- 172.21.15.84:0/1954499622 learned_addr learned my addr 172.21.15.84:0/1954499622 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:52.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.161+0000 7f6c4ffff640 1 --2- 172.21.15.84:0/1954499622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c50103410 0x7f6c50198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.161+0000 7f6c5552d640 1 --2- 172.21.15.84:0/1954499622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c5010be30 0x7f6c5019ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.161+0000 7f6c54d2c640 1 -- 172.21.15.84:0/1954499622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c5010be30 msgr2=0x7f6c5019ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:52.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.161+0000 7f6c54d2c640 1 --2- 172.21.15.84:0/1954499622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c5010be30 0x7f6c5019ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.161+0000 7f6c54d2c640 1 -- 172.21.15.84:0/1954499622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c50103410 msgr2=0x7f6c50198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:52.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.161+0000 7f6c54d2c640 1 --2- 172.21.15.84:0/1954499622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c50103410 0x7f6c50198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.161+0000 7f6c54d2c640 1 -- 172.21.15.84:0/1954499622 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5019d140 con 0x7f6c50102af0 2024-05-22T23:08:52.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.162+0000 7f6c54d2c640 1 --2- 172.21.15.84:0/1954499622 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c50102af0 0x7f6c50198150 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400d950 tx=0x7f6c4400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:52.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.162+0000 7f6c4dffb640 1 -- 172.21.15.84:0/1954499622 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44017070 con 0x7f6c50102af0 2024-05-22T23:08:52.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.162+0000 7f6c4dffb640 1 -- 172.21.15.84:0/1954499622 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c4400de80 con 0x7f6c50102af0 2024-05-22T23:08:52.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.162+0000 7f6c56f90640 1 -- 172.21.15.84:0/1954499622 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5019d430 con 0x7f6c50102af0 2024-05-22T23:08:52.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.162+0000 7f6c4dffb640 1 -- 172.21.15.84:0/1954499622 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4400f620 con 0x7f6c50102af0 2024-05-22T23:08:52.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.162+0000 7f6c56f90640 1 -- 172.21.15.84:0/1954499622 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c501a4cc0 con 0x7f6c50102af0 2024-05-22T23:08:52.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.165+0000 7f6c4dffb640 1 -- 172.21.15.84:0/1954499622 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6c44009c30 con 0x7f6c50102af0 2024-05-22T23:08:52.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.165+0000 7f6c59d7b740 1 Processor -- start 2024-05-22T23:08:52.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.165+0000 7f6c59d7b740 1 -- start start 2024-05-22T23:08:52.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.165+0000 7f6c59d7b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c194f4a0 0x5557c194d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.165+0000 7f6c59d7b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557c194ddb0 0x5557c194e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.165+0000 7f6c59d7b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557c1958b30 0x5557c195af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.165+0000 7f6c59d7b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5557c18b1db0 con 0x5557c194f4a0 2024-05-22T23:08:52.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.165+0000 7f6c59d7b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5557c1850360 con 0x5557c194ddb0 2024-05-22T23:08:52.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.165+0000 7f6c59d7b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5557c18add70 con 0x5557c1958b30 2024-05-22T23:08:52.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.165+0000 7f6c5552d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c194f4a0 0x5557c194d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.165+0000 7f6c54d2c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557c194ddb0 0x5557c194e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.165+0000 7f6c5552d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c194f4a0 0x5557c194d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49438/0 (socket says 172.21.15.84:49438) 2024-05-22T23:08:52.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.165+0000 7f6c5552d640 1 -- 172.21.15.84:0/21451355 learned_addr learned my addr 172.21.15.84:0/21451355 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:52.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.165+0000 7f6c4ffff640 1 --2- 172.21.15.84:0/21451355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557c1958b30 0x5557c195af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.166+0000 7f6c4dffb640 1 --2- 172.21.15.84:0/1954499622 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6c400a2d00 0x7f6c400a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.166+0000 7f6c5552d640 1 -- 172.21.15.84:0/21451355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557c1958b30 msgr2=0x5557c195af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:52.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.166+0000 7f6c5552d640 1 --2- 172.21.15.84:0/21451355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557c1958b30 0x5557c195af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.166+0000 7f6c5552d640 1 -- 172.21.15.84:0/21451355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557c194ddb0 msgr2=0x5557c194e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:52.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.166+0000 7f6c5552d640 1 --2- 172.21.15.84:0/21451355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557c194ddb0 0x5557c194e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.166+0000 7f6c5552d640 1 -- 172.21.15.84:0/21451355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557c1873a60 con 0x5557c194f4a0 2024-05-22T23:08:52.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.166+0000 7f6c4dffb640 1 -- 172.21.15.84:0/1954499622 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f6c440cf040 con 0x7f6c50102af0 2024-05-22T23:08:52.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.166+0000 7f6c5552d640 1 --2- 172.21.15.84:0/21451355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c194f4a0 0x5557c194d7e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009f60 tx=0x7f6c48009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0f3af753961b9d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:52.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.166+0000 7f6c5552d640 1 --2- 172.21.15.84:0/1954499622 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6c400a2d00 0x7f6c400a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.166+0000 7f6c3affd640 1 -- 172.21.15.84:0/21451355 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48004250 con 0x5557c194f4a0 2024-05-22T23:08:52.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.166+0000 7f6c3affd640 1 -- 172.21.15.84:0/21451355 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c4800fe80 con 0x5557c194f4a0 2024-05-22T23:08:52.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.166+0000 7f6c3affd640 1 -- 172.21.15.84:0/21451355 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4801bbc0 con 0x5557c194f4a0 2024-05-22T23:08:52.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.167+0000 7f6c59d7b740 1 -- 172.21.15.84:0/21451355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c194f4a0 msgr2=0x5557c194d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:52.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.167+0000 7f6c59d7b740 1 --2- 172.21.15.84:0/21451355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c194f4a0 0x5557c194d7e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009f60 tx=0x7f6c48009f90 comp rx=0 tx=0).stop 2024-05-22T23:08:52.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.167+0000 7f6c5552d640 1 --2- 172.21.15.84:0/1954499622 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6c400a2d00 0x7f6c400a51c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6c48020000 tx=0x7f6c48021000 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:52.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.167+0000 7f6c59d7b740 1 -- 172.21.15.84:0/21451355 shutdown_connections 2024-05-22T23:08:52.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.167+0000 7f6c59d7b740 1 --2- 172.21.15.84:0/21451355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557c1958b30 0x5557c195af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.167+0000 7f6c59d7b740 1 --2- 172.21.15.84:0/21451355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557c194ddb0 0x5557c194e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.167+0000 7f6c59d7b740 1 --2- 172.21.15.84:0/21451355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c194f4a0 0x5557c194d7e0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.167+0000 7f6c59d7b740 1 -- 172.21.15.84:0/21451355 >> 172.21.15.84:0/21451355 conn(0x5557c19168e0 msgr2=0x5557c194a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:52.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.167+0000 7f6c59d7b740 1 -- 172.21.15.84:0/21451355 shutdown_connections 2024-05-22T23:08:52.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.167+0000 7f6c59d7b740 1 -- 172.21.15.84:0/21451355 wait complete. 2024-05-22T23:08:52.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.167+0000 7f6c59d7b740 1 -- start start 2024-05-22T23:08:52.177 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.167+0000 7f6c59d7b740 1 Processor -- start 2024-05-22T23:08:52.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.168+0000 7f6c59d7b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557c194ddb0 0x5557c1955980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.168+0000 7f6c59d7b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557c194f4a0 0x5557c1955ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.168+0000 7f6c59d7b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c1958b30 0x5557c1956400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.168+0000 7f6c59d7b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5557c195b9c0 con 0x5557c1958b30 2024-05-22T23:08:52.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.168+0000 7f6c59d7b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5557c186c0f0 con 0x5557c194f4a0 2024-05-22T23:08:52.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.168+0000 7f6c59d7b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5557c195bb40 con 0x5557c194ddb0 2024-05-22T23:08:52.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.168+0000 7f6c59d7b740 10 client.0 _ll_register_callbacks cb 0x5557c1870f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:08:52.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.168+0000 7f6c5552d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c1958b30 0x5557c1956400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.168+0000 7f6c4ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557c194f4a0 0x5557c1955ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.168+0000 7f6c4ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557c194f4a0 0x5557c1955ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:35834/0 (socket says 172.21.15.84:35834) 2024-05-22T23:08:52.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.168+0000 7f6c54d2c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557c194ddb0 0x5557c1955980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.168+0000 7f6c4ffff640 1 -- 172.21.15.84:0/2637686366 learned_addr learned my addr 172.21.15.84:0/2637686366 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:52.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.168+0000 7f6c5552d640 1 -- 172.21.15.84:0/2637686366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557c194ddb0 msgr2=0x5557c1955980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:52.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.168+0000 7f6c5552d640 1 --2- 172.21.15.84:0/2637686366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557c194ddb0 0x5557c1955980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.168+0000 7f6c5552d640 1 -- 172.21.15.84:0/2637686366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557c194f4a0 msgr2=0x5557c1955ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:52.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.168+0000 7f6c5552d640 1 --2- 172.21.15.84:0/2637686366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557c194f4a0 0x5557c1955ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.168+0000 7f6c5552d640 1 -- 172.21.15.84:0/2637686366 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557c168f7d0 con 0x5557c1958b30 2024-05-22T23:08:52.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.168+0000 7f6c5552d640 1 --2- 172.21.15.84:0/2637686366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c1958b30 0x5557c1956400 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6c48011d60 tx=0x7f6c4801b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbbd670250792e0d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:52.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.168+0000 7f6c2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:08:52.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.169+0000 7f6c2ffff640 1 -- 172.21.15.84:0/2637686366 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48033020 con 0x5557c1958b30 2024-05-22T23:08:52.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.169+0000 7f6c2ffff640 1 -- 172.21.15.84:0/2637686366 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f6c48004dd0 con 0x5557c1958b30 2024-05-22T23:08:52.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.169+0000 7f6c59d7b740 1 -- 172.21.15.84:0/2637686366 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5557c18bf4e0 con 0x5557c1958b30 2024-05-22T23:08:52.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.169+0000 7f6c2ffff640 1 -- 172.21.15.84:0/2637686366 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f6c4803e050 con 0x5557c1958b30 2024-05-22T23:08:52.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.169+0000 7f6c59d7b740 10 client.16494 fetch_fsmap learned FSMap version 18 2024-05-22T23:08:52.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.169+0000 7f6c59d7b740 1 -- 172.21.15.84:0/2637686366 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5557c15c53f0 con 0x5557c1958b30 2024-05-22T23:08:52.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.169+0000 7f6c2ffff640 1 -- 172.21.15.84:0/2637686366 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5557c18bf4e0 con 0x5557c1958b30 2024-05-22T23:08:52.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.169+0000 7f6c2ffff640 1 -- 172.21.15.84:0/2637686366 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f6c48005040 con 0x5557c1958b30 2024-05-22T23:08:52.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.169+0000 7f6c59d7b740 10 client.16494 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:08:52.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.169+0000 7f6c59d7b740 10 client.16494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:08:52.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.169+0000 7f6c59d7b740 20 client.16494 populate_metadata read hostname 'smithi084' 2024-05-22T23:08:52.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.169+0000 7f6c59d7b740 1 --2- 172.21.15.84:0/2637686366 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5557c1968e10 0x5557c196b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.169+0000 7f6c59d7b740 4 client.16494 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:08:52.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.169+0000 7f6c59d7b740 1 -- 172.21.15.84:0/2637686366 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557c18b9830 con 0x5557c1968e10 2024-05-22T23:08:52.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.169+0000 7f6c54d2c640 1 --2- 172.21.15.84:0/2637686366 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5557c1968e10 0x5557c196b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.170+0000 7f6c2ffff640 10 client.16494 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:52.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.170+0000 7f6c54d2c640 1 --2- 172.21.15.84:0/2637686366 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5557c1968e10 0x5557c196b200 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12f757cf0d6ea3f1 server_cookie=48daca5584c98e8b in_seq=0 out_seq=0 2024-05-22T23:08:52.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.171+0000 7f6c2ffff640 1 -- 172.21.15.84:0/2637686366 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557c18b9830 con 0x5557c1968e10 2024-05-22T23:08:52.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.171+0000 7f6c2ffff640 10 client.16494 handle_command_reply: tid=0 2024-05-22T23:08:52.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.171+0000 7f6c59d7b740 1 client.16494 shutdown 2024-05-22T23:08:52.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.171+0000 7f6c59d7b740 1 -- 172.21.15.84:0/2637686366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c1958b30 msgr2=0x5557c1956400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:52.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.171+0000 7f6c59d7b740 1 --2- 172.21.15.84:0/2637686366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c1958b30 0x5557c1956400 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6c48011d60 tx=0x7f6c4801b8a0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.171+0000 7f6c59d7b740 1 -- 172.21.15.84:0/2637686366 shutdown_connections 2024-05-22T23:08:52.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.171+0000 7f6c59d7b740 1 --2- 172.21.15.84:0/2637686366 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5557c1968e10 0x5557c196b200 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.171+0000 7f6c59d7b740 1 --2- 172.21.15.84:0/2637686366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c1958b30 0x5557c1956400 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.171+0000 7f6c59d7b740 1 --2- 172.21.15.84:0/2637686366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557c194f4a0 0x5557c1955ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.171+0000 7f6c2ffff640 0 client.16494 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:52.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.171+0000 7f6c59d7b740 1 --2- 172.21.15.84:0/2637686366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557c194ddb0 0x5557c1955980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.171+0000 7f6c59d7b740 1 -- 172.21.15.84:0/2637686366 >> 172.21.15.84:0/2637686366 conn(0x5557c19168e0 msgr2=0x5557c194f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:52.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.171+0000 7f6c59d7b740 1 -- 172.21.15.84:0/2637686366 shutdown_connections 2024-05-22T23:08:52.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.171+0000 7f6c59d7b740 1 -- 172.21.15.84:0/2637686366 wait complete. 2024-05-22T23:08:52.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.171+0000 7f6c59d7b740 20 client.16494 trim_cache size 0 max 16384 2024-05-22T23:08:52.201 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.189+0000 7f6c59d7b740 1 Processor -- start 2024-05-22T23:08:52.201 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.189+0000 7f6c59d7b740 1 -- start start 2024-05-22T23:08:52.201 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.189+0000 7f6c59d7b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557c194ddb0 0x5557c19648c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.201 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.189+0000 7f6c59d7b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557c194f4a0 0x5557c1954960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.201 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.189+0000 7f6c59d7b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c1958b30 0x5557c1970a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.201 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.189+0000 7f6c59d7b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501ab740 con 0x5557c1958b30 2024-05-22T23:08:52.201 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.189+0000 7f6c59d7b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501adfa0 con 0x5557c194ddb0 2024-05-22T23:08:52.201 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.189+0000 7f6c59d7b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501abd90 con 0x5557c194f4a0 2024-05-22T23:08:52.201 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.189+0000 7f6c5552d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c1958b30 0x5557c1970a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.201 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.189+0000 7f6c5552d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c1958b30 0x5557c1970a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49466/0 (socket says 172.21.15.84:49466) 2024-05-22T23:08:52.201 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.189+0000 7f6c5552d640 1 -- 172.21.15.84:0/304167612 learned_addr learned my addr 172.21.15.84:0/304167612 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:52.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.189+0000 7f6c54d2c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557c194ddb0 0x5557c19648c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.189+0000 7f6c4ffff640 1 --2- 172.21.15.84:0/304167612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557c194f4a0 0x5557c1954960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.189+0000 7f6c5552d640 1 -- 172.21.15.84:0/304167612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557c194f4a0 msgr2=0x5557c1954960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:52.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.189+0000 7f6c5552d640 1 --2- 172.21.15.84:0/304167612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557c194f4a0 0x5557c1954960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.189+0000 7f6c5552d640 1 -- 172.21.15.84:0/304167612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557c194ddb0 msgr2=0x5557c19648c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:52.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.189+0000 7f6c5552d640 1 --2- 172.21.15.84:0/304167612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557c194ddb0 0x5557c19648c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.190+0000 7f6c5552d640 1 -- 172.21.15.84:0/304167612 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557c15850f0 con 0x5557c1958b30 2024-05-22T23:08:52.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.190+0000 7f6c4ffff640 1 --2- 172.21.15.84:0/304167612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557c194f4a0 0x5557c1954960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:08:52.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.190+0000 7f6c5552d640 1 --2- 172.21.15.84:0/304167612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c1958b30 0x5557c1970a30 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009690 tx=0x7f6c48005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907acb1b196fe68c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:52.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.190+0000 7f6c3a7fc640 1 -- 172.21.15.84:0/304167612 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48033050 con 0x5557c1958b30 2024-05-22T23:08:52.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.190+0000 7f6c3a7fc640 1 -- 172.21.15.84:0/304167612 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c4802a040 con 0x5557c1958b30 2024-05-22T23:08:52.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.190+0000 7f6c3a7fc640 1 -- 172.21.15.84:0/304167612 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48029030 con 0x5557c1958b30 2024-05-22T23:08:52.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.190+0000 7f6c59d7b740 1 -- 172.21.15.84:0/304167612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c1958b30 msgr2=0x5557c1970a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.190+0000 7f6c59d7b740 1 --2- 172.21.15.84:0/304167612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c1958b30 0x5557c1970a30 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009690 tx=0x7f6c48005b00 comp rx=0 tx=0).stop 2024-05-22T23:08:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.190+0000 7f6c59d7b740 1 -- 172.21.15.84:0/304167612 shutdown_connections 2024-05-22T23:08:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.190+0000 7f6c59d7b740 1 --2- 172.21.15.84:0/304167612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c1958b30 0x5557c1970a30 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.190+0000 7f6c59d7b740 1 --2- 172.21.15.84:0/304167612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557c194f4a0 0x5557c1954960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.190+0000 7f6c59d7b740 1 --2- 172.21.15.84:0/304167612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557c194ddb0 0x5557c19648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.190+0000 7f6c59d7b740 1 -- 172.21.15.84:0/304167612 >> 172.21.15.84:0/304167612 conn(0x5557c19168e0 msgr2=0x5557c1954250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.190+0000 7f6c59d7b740 1 -- 172.21.15.84:0/304167612 shutdown_connections 2024-05-22T23:08:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.190+0000 7f6c59d7b740 1 -- 172.21.15.84:0/304167612 wait complete. 2024-05-22T23:08:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.191+0000 7f6c59d7b740 1 -- start start 2024-05-22T23:08:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.191+0000 7f6c59d7b740 1 Processor -- start 2024-05-22T23:08:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.191+0000 7f6c59d7b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557c194ddb0 0x5557c1960dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.191+0000 7f6c59d7b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557c194f4a0 0x5557c1961310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.191+0000 7f6c59d7b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c1958b30 0x5557c1952c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.191+0000 7f6c59d7b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501ac460 con 0x5557c1958b30 2024-05-22T23:08:52.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.191+0000 7f6c59d7b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501ad8d0 con 0x5557c194ddb0 2024-05-22T23:08:52.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.191+0000 7f6c59d7b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501acb30 con 0x5557c194f4a0 2024-05-22T23:08:52.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.191+0000 7f6c59d7b740 10 client.0 _ll_register_callbacks cb 0x7f6c440d2a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:08:52.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.191+0000 7f6c5552d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c1958b30 0x5557c1952c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.191+0000 7f6c5552d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c1958b30 0x5557c1952c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49468/0 (socket says 172.21.15.84:49468) 2024-05-22T23:08:52.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.191+0000 7f6c5552d640 1 -- 172.21.15.84:0/4199067573 learned_addr learned my addr 172.21.15.84:0/4199067573 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:52.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.191+0000 7f6c54d2c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557c194ddb0 0x5557c1960dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.191+0000 7f6c4ffff640 1 --2- 172.21.15.84:0/4199067573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557c194f4a0 0x5557c1961310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.192+0000 7f6c5552d640 1 -- 172.21.15.84:0/4199067573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557c194f4a0 msgr2=0x5557c1961310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:52.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.192+0000 7f6c5552d640 1 --2- 172.21.15.84:0/4199067573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557c194f4a0 0x5557c1961310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.192+0000 7f6c5552d640 1 -- 172.21.15.84:0/4199067573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557c194ddb0 msgr2=0x5557c1960dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:52.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.192+0000 7f6c5552d640 1 --2- 172.21.15.84:0/4199067573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557c194ddb0 0x5557c1960dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.192+0000 7f6c5552d640 1 -- 172.21.15.84:0/4199067573 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557c1870fe0 con 0x5557c1958b30 2024-05-22T23:08:52.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.192+0000 7f6c5552d640 1 --2- 172.21.15.84:0/4199067573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c1958b30 0x5557c1952c00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6c4801b950 tx=0x7f6c480098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94041f04e54a0c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:52.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.192+0000 7f6c2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:08:52.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.192+0000 7f6c2ffff640 1 -- 172.21.15.84:0/4199067573 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48029030 con 0x5557c1958b30 2024-05-22T23:08:52.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.192+0000 7f6c2ffff640 1 -- 172.21.15.84:0/4199067573 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f6c4801b430 con 0x5557c1958b30 2024-05-22T23:08:52.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.192+0000 7f6c59d7b740 1 -- 172.21.15.84:0/4199067573 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c4802a040 con 0x5557c1958b30 2024-05-22T23:08:52.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.192+0000 7f6c2ffff640 1 -- 172.21.15.84:0/4199067573 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f6c48033070 con 0x5557c1958b30 2024-05-22T23:08:52.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.192+0000 7f6c59d7b740 10 client.16500 fetch_fsmap learned FSMap version 18 2024-05-22T23:08:52.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.192+0000 7f6c59d7b740 1 -- 172.21.15.84:0/4199067573 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5557c18421c0 con 0x5557c1958b30 2024-05-22T23:08:52.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.192+0000 7f6c2ffff640 1 -- 172.21.15.84:0/4199067573 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c4802a040 con 0x5557c1958b30 2024-05-22T23:08:52.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.192+0000 7f6c2ffff640 1 -- 172.21.15.84:0/4199067573 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f6c48052070 con 0x5557c1958b30 2024-05-22T23:08:52.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.192+0000 7f6c59d7b740 10 client.16500 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:08:52.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.192+0000 7f6c59d7b740 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:08:52.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.192+0000 7f6c59d7b740 20 client.16500 populate_metadata read hostname 'smithi084' 2024-05-22T23:08:52.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.192+0000 7f6c59d7b740 1 --2- 172.21.15.84:0/4199067573 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5557c1959ae0 0x5557c19d9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.192+0000 7f6c59d7b740 4 client.16500 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:08:52.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.192+0000 7f6c59d7b740 1 -- 172.21.15.84:0/4199067573 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5557c18366f0 con 0x5557c1959ae0 2024-05-22T23:08:52.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.193+0000 7f6c54d2c640 1 --2- 172.21.15.84:0/4199067573 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5557c1959ae0 0x5557c19d9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.193+0000 7f6c2ffff640 10 client.16500 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:52.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.193+0000 7f6c54d2c640 1 --2- 172.21.15.84:0/4199067573 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5557c1959ae0 0x5557c19d9ac0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bb3cb5bc30aa3ba server_cookie=6d0171193f4c3448 in_seq=0 out_seq=0 2024-05-22T23:08:52.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.193+0000 7f6c2ffff640 1 -- 172.21.15.84:0/4199067573 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5557c18366f0 con 0x5557c1959ae0 2024-05-22T23:08:52.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.193+0000 7f6c2ffff640 10 client.16500 handle_command_reply: tid=0 2024-05-22T23:08:52.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.193+0000 7f6c59d7b740 1 client.16500 shutdown 2024-05-22T23:08:52.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.193+0000 7f6c59d7b740 1 -- 172.21.15.84:0/4199067573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c1958b30 msgr2=0x5557c1952c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:52.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.193+0000 7f6c59d7b740 1 --2- 172.21.15.84:0/4199067573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c1958b30 0x5557c1952c00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6c4801b950 tx=0x7f6c480098b0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.193+0000 7f6c59d7b740 1 -- 172.21.15.84:0/4199067573 shutdown_connections 2024-05-22T23:08:52.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.193+0000 7f6c59d7b740 1 --2- 172.21.15.84:0/4199067573 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5557c1959ae0 0x5557c19d9ac0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.193+0000 7f6c59d7b740 1 --2- 172.21.15.84:0/4199067573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557c1958b30 0x5557c1952c00 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.193+0000 7f6c59d7b740 1 --2- 172.21.15.84:0/4199067573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557c194f4a0 0x5557c1961310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.193+0000 7f6c59d7b740 1 --2- 172.21.15.84:0/4199067573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557c194ddb0 0x5557c1960dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.193+0000 7f6c2ffff640 0 client.16500 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:52.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.193+0000 7f6c59d7b740 1 -- 172.21.15.84:0/4199067573 >> 172.21.15.84:0/4199067573 conn(0x5557c19168e0 msgr2=0x5557c1955920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:52.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.194+0000 7f6c59d7b740 1 -- 172.21.15.84:0/4199067573 shutdown_connections 2024-05-22T23:08:52.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.194+0000 7f6c59d7b740 1 -- 172.21.15.84:0/4199067573 wait complete. 2024-05-22T23:08:52.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.194+0000 7f6c59d7b740 20 client.16500 trim_cache size 0 max 16384 2024-05-22T23:08:52.207 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:08:52.207 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "no active scrubs running", 2024-05-22T23:08:52.207 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": {} 2024-05-22T23:08:52.208 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:08:52.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.194+0000 7f6c56f90640 1 -- 172.21.15.84:0/1954499622 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6c400a2d00 msgr2=0x7f6c400a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:52.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.194+0000 7f6c56f90640 1 --2- 172.21.15.84:0/1954499622 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6c400a2d00 0x7f6c400a51c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6c48020000 tx=0x7f6c48021000 comp rx=0 tx=0).stop 2024-05-22T23:08:52.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.194+0000 7f6c56f90640 1 -- 172.21.15.84:0/1954499622 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c50102af0 msgr2=0x7f6c50198150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:52.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.194+0000 7f6c56f90640 1 --2- 172.21.15.84:0/1954499622 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c50102af0 0x7f6c50198150 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400d950 tx=0x7f6c4400dd50 comp rx=0 tx=0).stop 2024-05-22T23:08:52.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.194+0000 7f6c56f90640 1 -- 172.21.15.84:0/1954499622 shutdown_connections 2024-05-22T23:08:52.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.194+0000 7f6c56f90640 1 --2- 172.21.15.84:0/1954499622 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6c400a2d00 0x7f6c400a51c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.194+0000 7f6c56f90640 1 --2- 172.21.15.84:0/1954499622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c5010be30 0x7f6c5019ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.194+0000 7f6c56f90640 1 --2- 172.21.15.84:0/1954499622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c50103410 0x7f6c50198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.194+0000 7f6c56f90640 1 --2- 172.21.15.84:0/1954499622 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c50102af0 0x7f6c50198150 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.194+0000 7f6c56f90640 1 -- 172.21.15.84:0/1954499622 >> 172.21.15.84:0/1954499622 conn(0x7f6c50077870 msgr2=0x7f6c500fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:52.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.194+0000 7f6c56f90640 1 -- 172.21.15.84:0/1954499622 shutdown_connections 2024-05-22T23:08:52.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.194+0000 7f6c56f90640 1 -- 172.21.15.84:0/1954499622 wait complete. 2024-05-22T23:08:52.212 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-22T23:08:52.212 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:08:52.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.354+0000 7f8bacf1b640 1 -- 172.21.15.84:0/2659918920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba8100430 msgr2=0x7f8ba8100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:52.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.354+0000 7f8bacf1b640 1 --2- 172.21.15.84:0/2659918920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba8100430 0x7f8ba8100810 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f8b98009900 tx=0x7f8b9802a410 comp rx=0 tx=0).stop 2024-05-22T23:08:52.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.354+0000 7f8bacf1b640 1 -- 172.21.15.84:0/2659918920 shutdown_connections 2024-05-22T23:08:52.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.355+0000 7f8bacf1b640 1 --2- 172.21.15.84:0/2659918920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ba8106bf0 0x7f8ba810e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.355+0000 7f8bacf1b640 1 --2- 172.21.15.84:0/2659918920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba8100de0 0x7f8ba8106510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.355+0000 7f8bacf1b640 1 --2- 172.21.15.84:0/2659918920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba8100430 0x7f8ba8100810 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.355+0000 7f8bacf1b640 1 -- 172.21.15.84:0/2659918920 >> 172.21.15.84:0/2659918920 conn(0x7f8ba8075a50 msgr2=0x7f8ba8075e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:52.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.355+0000 7f8bacf1b640 1 -- 172.21.15.84:0/2659918920 shutdown_connections 2024-05-22T23:08:52.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.355+0000 7f8bacf1b640 1 -- 172.21.15.84:0/2659918920 wait complete. 2024-05-22T23:08:52.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.355+0000 7f8bacf1b640 1 Processor -- start 2024-05-22T23:08:52.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.355+0000 7f8bacf1b640 1 -- start start 2024-05-22T23:08:52.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.356+0000 7f8bacf1b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba8100430 0x7f8ba81981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.356+0000 7f8bacf1b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ba8106bf0 0x7f8ba8198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.356+0000 7f8bacf1b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba819caa0 0x7f8ba819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.356+0000 7f8bacf1b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba8111cc0 con 0x7f8ba819caa0 2024-05-22T23:08:52.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.356+0000 7f8bacf1b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba8111b40 con 0x7f8ba8100430 2024-05-22T23:08:52.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.356+0000 7f8bacf1b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba8111e40 con 0x7f8ba8106bf0 2024-05-22T23:08:52.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.356+0000 7f8ba6d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba819caa0 0x7f8ba819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.356+0000 7f8ba6d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba819caa0 0x7f8ba819cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49496/0 (socket says 172.21.15.84:49496) 2024-05-22T23:08:52.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.356+0000 7f8ba6d9d640 1 -- 172.21.15.84:0/1008414642 learned_addr learned my addr 172.21.15.84:0/1008414642 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:52.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.356+0000 7f8ba659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba8100430 0x7f8ba81981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.356+0000 7f8ba5d9b640 1 --2- 172.21.15.84:0/1008414642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ba8106bf0 0x7f8ba8198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.356+0000 7f8ba6d9d640 1 -- 172.21.15.84:0/1008414642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ba8106bf0 msgr2=0x7f8ba8198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:52.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.356+0000 7f8ba6d9d640 1 --2- 172.21.15.84:0/1008414642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ba8106bf0 0x7f8ba8198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.356+0000 7f8ba6d9d640 1 -- 172.21.15.84:0/1008414642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba8100430 msgr2=0x7f8ba81981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:52.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.356+0000 7f8ba6d9d640 1 --2- 172.21.15.84:0/1008414642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba8100430 0x7f8ba81981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.356+0000 7f8ba6d9d640 1 -- 172.21.15.84:0/1008414642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba819d590 con 0x7f8ba819caa0 2024-05-22T23:08:52.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.356+0000 7f8ba6d9d640 1 --2- 172.21.15.84:0/1008414642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba819caa0 0x7f8ba819cf50 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0009fd0 tx=0x7f8ba000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:52.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.356+0000 7f8b977fe640 1 -- 172.21.15.84:0/1008414642 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0017070 con 0x7f8ba819caa0 2024-05-22T23:08:52.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.356+0000 7f8bacf1b640 1 -- 172.21.15.84:0/1008414642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba819d880 con 0x7f8ba819caa0 2024-05-22T23:08:52.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.356+0000 7f8b977fe640 1 -- 172.21.15.84:0/1008414642 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ba0003e50 con 0x7f8ba819caa0 2024-05-22T23:08:52.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.356+0000 7f8b977fe640 1 -- 172.21.15.84:0/1008414642 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba00106c0 con 0x7f8ba819caa0 2024-05-22T23:08:52.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.356+0000 7f8bacf1b640 1 -- 172.21.15.84:0/1008414642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba81a5020 con 0x7f8ba819caa0 2024-05-22T23:08:52.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.357+0000 7f8bacf1b640 1 -- 172.21.15.84:0/1008414642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba8069a00 con 0x7f8ba819caa0 2024-05-22T23:08:52.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.358+0000 7f8b977fe640 1 -- 172.21.15.84:0/1008414642 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8ba0003970 con 0x7f8ba819caa0 2024-05-22T23:08:52.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.359+0000 7f8b977fe640 1 --2- 172.21.15.84:0/1008414642 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f8b900a2d00 0x7f8b900a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.359+0000 7f8ba659c640 1 --2- 172.21.15.84:0/1008414642 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f8b900a2d00 0x7f8b900a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.359+0000 7f8b977fe640 1 -- 172.21.15.84:0/1008414642 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f8ba00cf090 con 0x7f8ba819caa0 2024-05-22T23:08:52.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.359+0000 7f8ba659c640 1 --2- 172.21.15.84:0/1008414642 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f8b900a2d00 0x7f8b900a51c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8b98009230 tx=0x7f8b980058d0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:52.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.363+0000 7f8b977fe640 1 -- 172.21.15.84:0/1008414642 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8ba009b630 con 0x7f8ba819caa0 2024-05-22T23:08:52.442 INFO:tasks.workunit.client.0.smithi084.stdout: 1 950175 14.95 MB/sec execute 218 sec latency 22.444 ms 2024-05-22T23:08:52.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.609+0000 7f8bacf1b640 1 -- 172.21.15.84:0/1008414642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ba8068ae0 con 0x7f8ba819caa0 2024-05-22T23:08:52.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.610+0000 7f8b977fe640 1 -- 172.21.15.84:0/1008414642 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7f8ba8068ae0 con 0x7f8ba819caa0 2024-05-22T23:08:52.616 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:08:52.616 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:08:52.616 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:08:52.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.612+0000 7f8bacf1b640 1 -- 172.21.15.84:0/1008414642 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f8b900a2d00 msgr2=0x7f8b900a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:52.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.612+0000 7f8bacf1b640 1 --2- 172.21.15.84:0/1008414642 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f8b900a2d00 0x7f8b900a51c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8b98009230 tx=0x7f8b980058d0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.612+0000 7f8bacf1b640 1 -- 172.21.15.84:0/1008414642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba819caa0 msgr2=0x7f8ba819cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:52.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.612+0000 7f8bacf1b640 1 --2- 172.21.15.84:0/1008414642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba819caa0 0x7f8ba819cf50 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0009fd0 tx=0x7f8ba000b260 comp rx=0 tx=0).stop 2024-05-22T23:08:52.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.612+0000 7f8bacf1b640 1 -- 172.21.15.84:0/1008414642 shutdown_connections 2024-05-22T23:08:52.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.612+0000 7f8bacf1b640 1 --2- 172.21.15.84:0/1008414642 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f8b900a2d00 0x7f8b900a51c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.612+0000 7f8bacf1b640 1 --2- 172.21.15.84:0/1008414642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba819caa0 0x7f8ba819cf50 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.612+0000 7f8bacf1b640 1 --2- 172.21.15.84:0/1008414642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ba8106bf0 0x7f8ba8198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.612+0000 7f8bacf1b640 1 --2- 172.21.15.84:0/1008414642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba8100430 0x7f8ba81981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.612+0000 7f8bacf1b640 1 -- 172.21.15.84:0/1008414642 >> 172.21.15.84:0/1008414642 conn(0x7f8ba8075a50 msgr2=0x7f8ba80fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:52.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.612+0000 7f8bacf1b640 1 -- 172.21.15.84:0/1008414642 shutdown_connections 2024-05-22T23:08:52.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.612+0000 7f8bacf1b640 1 -- 172.21.15.84:0/1008414642 wait complete. 2024-05-22T23:08:52.630 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-22T23:08:52.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.732+0000 7f78606da640 1 -- 172.21.15.84:0/2489360790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7858100ac0 msgr2=0x7f7858100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:52.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.732+0000 7f78606da640 1 --2- 172.21.15.84:0/2489360790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7858100ac0 0x7f7858100f20 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7848009900 tx=0x7f784802a410 comp rx=0 tx=0).stop 2024-05-22T23:08:52.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.732+0000 7f78606da640 1 -- 172.21.15.84:0/2489360790 shutdown_connections 2024-05-22T23:08:52.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.732+0000 7f78606da640 1 --2- 172.21.15.84:0/2489360790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7858101460 0x7f785810d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.732+0000 7f78606da640 1 --2- 172.21.15.84:0/2489360790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7858100ac0 0x7f7858100f20 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.732+0000 7f78606da640 1 --2- 172.21.15.84:0/2489360790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78581067c0 0x7f7858106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.732+0000 7f78606da640 1 -- 172.21.15.84:0/2489360790 >> 172.21.15.84:0/2489360790 conn(0x7f7858076570 msgr2=0x7f7858076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:52.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.732+0000 7f78606da640 1 -- 172.21.15.84:0/2489360790 shutdown_connections 2024-05-22T23:08:52.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.732+0000 7f78606da640 1 -- 172.21.15.84:0/2489360790 wait complete. 2024-05-22T23:08:52.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.733+0000 7f78606da640 1 Processor -- start 2024-05-22T23:08:52.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.733+0000 7f78606da640 1 -- start start 2024-05-22T23:08:52.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.733+0000 7f78606da640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7858100ac0 0x7f7858198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.733+0000 7f78606da640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7858101460 0x7f7858198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.733+0000 7f78606da640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78581067c0 0x7f785819c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.733+0000 7f78606da640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7858111c20 con 0x7f78581067c0 2024-05-22T23:08:52.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.733+0000 7f78606da640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7858111aa0 con 0x7f7858100ac0 2024-05-22T23:08:52.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.733+0000 7f78606da640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7858111da0 con 0x7f7858101460 2024-05-22T23:08:52.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.733+0000 7f785ec77640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78581067c0 0x7f785819c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.733+0000 7f785e476640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7858100ac0 0x7f7858198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.733+0000 7f785ec77640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78581067c0 0x7f785819c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49508/0 (socket says 172.21.15.84:49508) 2024-05-22T23:08:52.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.733+0000 7f785ec77640 1 -- 172.21.15.84:0/1118444879 learned_addr learned my addr 172.21.15.84:0/1118444879 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:52.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.733+0000 7f785dc75640 1 --2- 172.21.15.84:0/1118444879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7858101460 0x7f7858198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.733+0000 7f785ec77640 1 -- 172.21.15.84:0/1118444879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7858101460 msgr2=0x7f7858198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:52.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.733+0000 7f785ec77640 1 --2- 172.21.15.84:0/1118444879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7858101460 0x7f7858198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.733+0000 7f785ec77640 1 -- 172.21.15.84:0/1118444879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7858100ac0 msgr2=0x7f7858198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:52.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.733+0000 7f785ec77640 1 --2- 172.21.15.84:0/1118444879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7858100ac0 0x7f7858198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.733+0000 7f785ec77640 1 -- 172.21.15.84:0/1118444879 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785819d0c0 con 0x7f78581067c0 2024-05-22T23:08:52.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.733+0000 7f785dc75640 1 --2- 172.21.15.84:0/1118444879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7858101460 0x7f7858198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:08:52.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.733+0000 7f785ec77640 1 --2- 172.21.15.84:0/1118444879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78581067c0 0x7f785819c990 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f785000b970 tx=0x7f785000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:52.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.734+0000 7f784f7fe640 1 -- 172.21.15.84:0/1118444879 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850019070 con 0x7f78581067c0 2024-05-22T23:08:52.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.734+0000 7f78606da640 1 -- 172.21.15.84:0/1118444879 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f785819d3b0 con 0x7f78581067c0 2024-05-22T23:08:52.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.734+0000 7f784f7fe640 1 -- 172.21.15.84:0/1118444879 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f785000cb40 con 0x7f78581067c0 2024-05-22T23:08:52.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.734+0000 7f784f7fe640 1 -- 172.21.15.84:0/1118444879 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850007650 con 0x7f78581067c0 2024-05-22T23:08:52.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.734+0000 7f78606da640 1 -- 172.21.15.84:0/1118444879 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78581a4c40 con 0x7f78581067c0 2024-05-22T23:08:52.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.736+0000 7f784f7fe640 1 -- 172.21.15.84:0/1118444879 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f785001c070 con 0x7f78581067c0 2024-05-22T23:08:52.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.736+0000 7f78633eb740 1 Processor -- start 2024-05-22T23:08:52.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.736+0000 7f78633eb740 1 -- start start 2024-05-22T23:08:52.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.737+0000 7f78633eb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639c2a1ef80 0x5639c2a1f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.737+0000 7f78633eb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a1f8a0 0x5639c2a29a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.737+0000 7f78633eb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639c2a29f70 0x5639c2a2c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.737+0000 7f78633eb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5639c2983db0 con 0x5639c2a1f8a0 2024-05-22T23:08:52.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.737+0000 7f78633eb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5639c2922400 con 0x5639c2a1ef80 2024-05-22T23:08:52.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.737+0000 7f78633eb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5639c297fd70 con 0x5639c2a29f70 2024-05-22T23:08:52.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.737+0000 7f785e476640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a1f8a0 0x5639c2a29a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.737+0000 7f785e476640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a1f8a0 0x5639c2a29a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49510/0 (socket says 172.21.15.84:49510) 2024-05-22T23:08:52.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.737+0000 7f785e476640 1 -- 172.21.15.84:0/1819643953 learned_addr learned my addr 172.21.15.84:0/1819643953 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:52.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.737+0000 7f785dc75640 1 --2- 172.21.15.84:0/1819643953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639c2a1ef80 0x5639c2a1f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.737+0000 7f785dc75640 1 --2- 172.21.15.84:0/1819643953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639c2a29f70 0x5639c2a2c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.737+0000 7f784f7fe640 1 --2- 172.21.15.84:0/1118444879 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f78440a2d00 0x7f78440a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.737+0000 7f785e476640 1 -- 172.21.15.84:0/1819643953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639c2a29f70 msgr2=0x5639c2a2c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:52.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.737+0000 7f785e476640 1 --2- 172.21.15.84:0/1819643953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639c2a29f70 0x5639c2a2c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.737+0000 7f785e476640 1 -- 172.21.15.84:0/1819643953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639c2a1ef80 msgr2=0x5639c2a1f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:52.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.737+0000 7f785e476640 1 --2- 172.21.15.84:0/1819643953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639c2a1ef80 0x5639c2a1f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.737+0000 7f785e476640 1 -- 172.21.15.84:0/1819643953 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639c29142f0 con 0x5639c2a1f8a0 2024-05-22T23:08:52.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.737+0000 7f785ec77640 1 --2- 172.21.15.84:0/1118444879 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f78440a2d00 0x7f78440a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.737+0000 7f784f7fe640 1 -- 172.21.15.84:0/1118444879 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f78500ce970 con 0x7f78581067c0 2024-05-22T23:08:52.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.737+0000 7f785e476640 1 --2- 172.21.15.84:0/1819643953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a1f8a0 0x5639c2a29a30 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f785400bd50 tx=0x7f78540107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f46c850e2f80e35 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:52.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.738+0000 7f785ec77640 1 --2- 172.21.15.84:0/1118444879 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f78440a2d00 0x7f78440a51c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7850097420 tx=0x7f785006cf80 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:52.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.738+0000 7f784cff9640 1 -- 172.21.15.84:0/1819643953 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7854005020 con 0x5639c2a1f8a0 2024-05-22T23:08:52.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.738+0000 7f784cff9640 1 -- 172.21.15.84:0/1819643953 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78540047a0 con 0x5639c2a1f8a0 2024-05-22T23:08:52.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.738+0000 7f784cff9640 1 -- 172.21.15.84:0/1819643953 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78540117c0 con 0x5639c2a1f8a0 2024-05-22T23:08:52.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.738+0000 7f78633eb740 1 -- 172.21.15.84:0/1819643953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a1f8a0 msgr2=0x5639c2a29a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:52.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.738+0000 7f78633eb740 1 --2- 172.21.15.84:0/1819643953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a1f8a0 0x5639c2a29a30 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f785400bd50 tx=0x7f78540107a0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.738+0000 7f78633eb740 1 -- 172.21.15.84:0/1819643953 shutdown_connections 2024-05-22T23:08:52.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.738+0000 7f78633eb740 1 --2- 172.21.15.84:0/1819643953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639c2a29f70 0x5639c2a2c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.738+0000 7f78633eb740 1 --2- 172.21.15.84:0/1819643953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a1f8a0 0x5639c2a29a30 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.738+0000 7f78633eb740 1 --2- 172.21.15.84:0/1819643953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639c2a1ef80 0x5639c2a1f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.738+0000 7f78633eb740 1 -- 172.21.15.84:0/1819643953 >> 172.21.15.84:0/1819643953 conn(0x5639c2a1b2a0 msgr2=0x5639c2a1b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:52.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.738+0000 7f78633eb740 1 -- 172.21.15.84:0/1819643953 shutdown_connections 2024-05-22T23:08:52.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.738+0000 7f78633eb740 1 -- 172.21.15.84:0/1819643953 wait complete. 2024-05-22T23:08:52.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.739+0000 7f78633eb740 1 -- start start 2024-05-22T23:08:52.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.739+0000 7f78633eb740 1 Processor -- start 2024-05-22T23:08:52.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.739+0000 7f78633eb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639c2a1ef80 0x5639c2a26eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.739+0000 7f78633eb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a1f8a0 0x5639c2a273f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.739+0000 7f78633eb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639c2a29f70 0x5639c2a27930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.739+0000 7f78633eb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5639c2a2cd60 con 0x5639c2a1f8a0 2024-05-22T23:08:52.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.739+0000 7f78633eb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5639c293e2e0 con 0x5639c2a1ef80 2024-05-22T23:08:52.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.739+0000 7f78633eb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5639c2a2cee0 con 0x5639c2a29f70 2024-05-22T23:08:52.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.739+0000 7f78633eb740 10 client.0 _ll_register_callbacks cb 0x5639c2947a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:08:52.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.739+0000 7f785e476640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a1f8a0 0x5639c2a273f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.739+0000 7f785dc75640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639c2a1ef80 0x5639c2a26eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.739+0000 7f785e476640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a1f8a0 0x5639c2a273f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49516/0 (socket says 172.21.15.84:49516) 2024-05-22T23:08:52.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.739+0000 7f785e476640 1 -- 172.21.15.84:0/2260718460 learned_addr learned my addr 172.21.15.84:0/2260718460 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:52.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.739+0000 7f785dc75640 1 --2- 172.21.15.84:0/2260718460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639c2a29f70 0x5639c2a27930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.739+0000 7f785e476640 1 -- 172.21.15.84:0/2260718460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639c2a29f70 msgr2=0x5639c2a27930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:52.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.739+0000 7f785e476640 1 --2- 172.21.15.84:0/2260718460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639c2a29f70 0x5639c2a27930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.739+0000 7f785e476640 1 -- 172.21.15.84:0/2260718460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639c2a1ef80 msgr2=0x5639c2a26eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:52.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.739+0000 7f785e476640 1 --2- 172.21.15.84:0/2260718460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639c2a1ef80 0x5639c2a26eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.739+0000 7f785e476640 1 -- 172.21.15.84:0/2260718460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639c26973f0 con 0x5639c2a1f8a0 2024-05-22T23:08:52.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.740+0000 7f785e476640 1 --2- 172.21.15.84:0/2260718460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a1f8a0 0x5639c2a273f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f785400bfa0 tx=0x7f78540046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2857fed4b26033 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:52.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.740+0000 7f7841ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:08:52.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.740+0000 7f7841ffb640 1 -- 172.21.15.84:0/2260718460 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7854004b60 con 0x5639c2a1f8a0 2024-05-22T23:08:52.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.740+0000 7f7841ffb640 1 -- 172.21.15.84:0/2260718460 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f7854004020 con 0x5639c2a1f8a0 2024-05-22T23:08:52.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.740+0000 7f78633eb740 1 -- 172.21.15.84:0/2260718460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78540047a0 con 0x5639c2a1f8a0 2024-05-22T23:08:52.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.740+0000 7f7841ffb640 1 -- 172.21.15.84:0/2260718460 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f7854004220 con 0x5639c2a1f8a0 2024-05-22T23:08:52.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.740+0000 7f78633eb740 10 client.16518 fetch_fsmap learned FSMap version 18 2024-05-22T23:08:52.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.740+0000 7f78633eb740 1 -- 172.21.15.84:0/2260718460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5639c27617d0 con 0x5639c2a1f8a0 2024-05-22T23:08:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.740+0000 7f7841ffb640 1 -- 172.21.15.84:0/2260718460 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78540047a0 con 0x5639c2a1f8a0 2024-05-22T23:08:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.740+0000 7f7841ffb640 1 -- 172.21.15.84:0/2260718460 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f785402d070 con 0x5639c2a1f8a0 2024-05-22T23:08:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.740+0000 7f78633eb740 10 client.16518 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:08:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.740+0000 7f78633eb740 10 client.16518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:08:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.740+0000 7f78633eb740 20 client.16518 populate_metadata read hostname 'smithi084' 2024-05-22T23:08:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.740+0000 7f78633eb740 1 --2- 172.21.15.84:0/2260718460 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5639c2a3a1e0 0x5639c2a3c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.740+0000 7f78633eb740 4 client.16518 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:08:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.740+0000 7f78633eb740 1 -- 172.21.15.84:0/2260718460 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639c29914e0 con 0x5639c2a3a1e0 2024-05-22T23:08:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.741+0000 7f785dc75640 1 --2- 172.21.15.84:0/2260718460 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5639c2a3a1e0 0x5639c2a3c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.741+0000 7f7841ffb640 10 client.16518 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:52.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.741+0000 7f785dc75640 1 --2- 172.21.15.84:0/2260718460 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5639c2a3a1e0 0x5639c2a3c5d0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23196d32a83b0f66 server_cookie=1b3d5a6644fef2b9 in_seq=0 out_seq=0 2024-05-22T23:08:52.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.742+0000 7f7841ffb640 1 -- 172.21.15.84:0/2260718460 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5639c29914e0 con 0x5639c2a3a1e0 2024-05-22T23:08:52.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.742+0000 7f7841ffb640 10 client.16518 handle_command_reply: tid=0 2024-05-22T23:08:52.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.742+0000 7f78633eb740 1 client.16518 shutdown 2024-05-22T23:08:52.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.742+0000 7f78633eb740 1 -- 172.21.15.84:0/2260718460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a1f8a0 msgr2=0x5639c2a273f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:52.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.742+0000 7f78633eb740 1 --2- 172.21.15.84:0/2260718460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a1f8a0 0x5639c2a273f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f785400bfa0 tx=0x7f78540046f0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.742+0000 7f78633eb740 1 -- 172.21.15.84:0/2260718460 shutdown_connections 2024-05-22T23:08:52.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.742+0000 7f78633eb740 1 --2- 172.21.15.84:0/2260718460 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5639c2a3a1e0 0x5639c2a3c5d0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.742+0000 7f78633eb740 1 --2- 172.21.15.84:0/2260718460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639c2a29f70 0x5639c2a27930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.742+0000 7f78633eb740 1 --2- 172.21.15.84:0/2260718460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a1f8a0 0x5639c2a273f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.742+0000 7f78633eb740 1 --2- 172.21.15.84:0/2260718460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639c2a1ef80 0x5639c2a26eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.742+0000 7f7841ffb640 0 client.16518 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:52.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.742+0000 7f78633eb740 1 -- 172.21.15.84:0/2260718460 >> 172.21.15.84:0/2260718460 conn(0x5639c2a1b2a0 msgr2=0x5639c2a1a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:52.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.742+0000 7f78633eb740 1 -- 172.21.15.84:0/2260718460 shutdown_connections 2024-05-22T23:08:52.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.742+0000 7f78633eb740 1 -- 172.21.15.84:0/2260718460 wait complete. 2024-05-22T23:08:52.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.742+0000 7f78633eb740 20 client.16518 trim_cache size 0 max 16384 2024-05-22T23:08:52.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.756+0000 7f78633eb740 1 Processor -- start 2024-05-22T23:08:52.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.756+0000 7f78633eb740 1 -- start start 2024-05-22T23:08:52.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.756+0000 7f78633eb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639c2a1ef80 0x5639c2a295e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.756+0000 7f78633eb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639c2a1f8a0 0x5639c2a35c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.756+0000 7f78633eb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a29f70 0x5639c2a27580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.756+0000 7f78633eb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f78581ab6d0 con 0x5639c2a29f70 2024-05-22T23:08:52.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.756+0000 7f78633eb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f78581adf30 con 0x5639c2a1ef80 2024-05-22T23:08:52.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.756+0000 7f78633eb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f78581abd20 con 0x5639c2a1f8a0 2024-05-22T23:08:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.756+0000 7f785dc75640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a29f70 0x5639c2a27580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.756+0000 7f785dc75640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a29f70 0x5639c2a27580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49518/0 (socket says 172.21.15.84:49518) 2024-05-22T23:08:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.756+0000 7f785dc75640 1 -- 172.21.15.84:0/153163017 learned_addr learned my addr 172.21.15.84:0/153163017 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.756+0000 7f785dc75640 1 --2- 172.21.15.84:0/153163017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639c2a1ef80 0x5639c2a295e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.756+0000 7f785e476640 1 --2- 172.21.15.84:0/153163017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639c2a1f8a0 0x5639c2a35c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.757+0000 7f785dc75640 1 -- 172.21.15.84:0/153163017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639c2a1f8a0 msgr2=0x5639c2a35c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.757+0000 7f785dc75640 1 --2- 172.21.15.84:0/153163017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639c2a1f8a0 0x5639c2a35c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.757+0000 7f785dc75640 1 -- 172.21.15.84:0/153163017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639c2a1ef80 msgr2=0x5639c2a295e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.757+0000 7f785dc75640 1 --2- 172.21.15.84:0/153163017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639c2a1ef80 0x5639c2a295e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.757+0000 7f785dc75640 1 -- 172.21.15.84:0/153163017 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639c2a2cec0 con 0x5639c2a29f70 2024-05-22T23:08:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.757+0000 7f785dc75640 1 --2- 172.21.15.84:0/153163017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a29f70 0x5639c2a27580 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7848005ec0 tx=0x7f7848009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276e4d935795513c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.757+0000 7f7843fff640 1 -- 172.21.15.84:0/153163017 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848035040 con 0x5639c2a29f70 2024-05-22T23:08:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.757+0000 7f7843fff640 1 -- 172.21.15.84:0/153163017 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7848042030 con 0x5639c2a29f70 2024-05-22T23:08:52.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.757+0000 7f7843fff640 1 -- 172.21.15.84:0/153163017 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784803e400 con 0x5639c2a29f70 2024-05-22T23:08:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.757+0000 7f78633eb740 1 -- 172.21.15.84:0/153163017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a29f70 msgr2=0x5639c2a27580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.757+0000 7f78633eb740 1 --2- 172.21.15.84:0/153163017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a29f70 0x5639c2a27580 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7848005ec0 tx=0x7f7848009260 comp rx=0 tx=0).stop 2024-05-22T23:08:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.757+0000 7f78633eb740 1 -- 172.21.15.84:0/153163017 shutdown_connections 2024-05-22T23:08:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.757+0000 7f78633eb740 1 --2- 172.21.15.84:0/153163017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a29f70 0x5639c2a27580 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.757+0000 7f78633eb740 1 --2- 172.21.15.84:0/153163017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639c2a1f8a0 0x5639c2a35c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.757+0000 7f78633eb740 1 --2- 172.21.15.84:0/153163017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639c2a1ef80 0x5639c2a295e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.757+0000 7f78633eb740 1 -- 172.21.15.84:0/153163017 >> 172.21.15.84:0/153163017 conn(0x5639c2a1b2a0 msgr2=0x5639c2a26e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.758+0000 7f78633eb740 1 -- 172.21.15.84:0/153163017 shutdown_connections 2024-05-22T23:08:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.758+0000 7f78633eb740 1 -- 172.21.15.84:0/153163017 wait complete. 2024-05-22T23:08:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.758+0000 7f78633eb740 1 -- start start 2024-05-22T23:08:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.758+0000 7f78633eb740 1 Processor -- start 2024-05-22T23:08:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.758+0000 7f78633eb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a1ef80 0x5639c2a32510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.758+0000 7f78633eb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639c2a1f8a0 0x5639c2a23850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.758+0000 7f78633eb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639c2a29f70 0x5639c2a23d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.758+0000 7f78633eb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f78581ac3f0 con 0x5639c2a1ef80 2024-05-22T23:08:52.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.758+0000 7f78633eb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f78581ad860 con 0x5639c2a1f8a0 2024-05-22T23:08:52.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.758+0000 7f78633eb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f78581acac0 con 0x5639c2a29f70 2024-05-22T23:08:52.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.758+0000 7f78633eb740 10 client.0 _ll_register_callbacks cb 0x7f7848009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:08:52.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.758+0000 7f785dc75640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a1ef80 0x5639c2a32510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.758+0000 7f785dc75640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a1ef80 0x5639c2a32510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49532/0 (socket says 172.21.15.84:49532) 2024-05-22T23:08:52.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.758+0000 7f785dc75640 1 -- 172.21.15.84:0/2116944646 learned_addr learned my addr 172.21.15.84:0/2116944646 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:52.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.758+0000 7f785e476640 1 --2- 172.21.15.84:0/2116944646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639c2a1f8a0 0x5639c2a23850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.758+0000 7f785dc75640 1 -- 172.21.15.84:0/2116944646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639c2a29f70 msgr2=0x5639c2a23d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:52.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.758+0000 7f785dc75640 1 --2- 172.21.15.84:0/2116944646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639c2a29f70 0x5639c2a23d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.759+0000 7f785dc75640 1 -- 172.21.15.84:0/2116944646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639c2a1f8a0 msgr2=0x5639c2a23850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:52.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.759+0000 7f785dc75640 1 --2- 172.21.15.84:0/2116944646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639c2a1f8a0 0x5639c2a23850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.759+0000 7f785dc75640 1 -- 172.21.15.84:0/2116944646 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639c2a24590 con 0x5639c2a1ef80 2024-05-22T23:08:52.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.759+0000 7f785dc75640 1 --2- 172.21.15.84:0/2116944646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a1ef80 0x5639c2a32510 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7848000c00 tx=0x7f7848003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f853d75aaf73a22a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:52.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.759+0000 7f7841ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:08:52.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.759+0000 7f7841ffb640 1 -- 172.21.15.84:0/2116944646 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848035040 con 0x5639c2a1ef80 2024-05-22T23:08:52.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.759+0000 7f7841ffb640 1 -- 172.21.15.84:0/2116944646 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f78480035f0 con 0x5639c2a1ef80 2024-05-22T23:08:52.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.759+0000 7f78633eb740 1 -- 172.21.15.84:0/2116944646 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7848042030 con 0x5639c2a1ef80 2024-05-22T23:08:52.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.759+0000 7f7841ffb640 1 -- 172.21.15.84:0/2116944646 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f7848068050 con 0x5639c2a1ef80 2024-05-22T23:08:52.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.759+0000 7f78633eb740 10 client.16524 fetch_fsmap learned FSMap version 18 2024-05-22T23:08:52.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.759+0000 7f78633eb740 1 -- 172.21.15.84:0/2116944646 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5639c2aa0410 con 0x5639c2a1ef80 2024-05-22T23:08:52.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.759+0000 7f7841ffb640 1 -- 172.21.15.84:0/2116944646 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7848042030 con 0x5639c2a1ef80 2024-05-22T23:08:52.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.759+0000 7f7841ffb640 1 -- 172.21.15.84:0/2116944646 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f784804dce0 con 0x5639c2a1ef80 2024-05-22T23:08:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.759+0000 7f78633eb740 10 client.16524 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:08:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.759+0000 7f78633eb740 10 client.16524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:08:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.759+0000 7f78633eb740 20 client.16524 populate_metadata read hostname 'smithi084' 2024-05-22T23:08:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.759+0000 7f78633eb740 1 --2- 172.21.15.84:0/2116944646 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5639c2aa09c0 0x5639c2aa2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.759+0000 7f78633eb740 4 client.16524 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-22T23:08:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.759+0000 7f78633eb740 1 -- 172.21.15.84:0/2116944646 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5639c298b830 con 0x5639c2aa09c0 2024-05-22T23:08:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.760+0000 7f785e476640 1 --2- 172.21.15.84:0/2116944646 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5639c2aa09c0 0x5639c2aa2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.760+0000 7f785e476640 1 --2- 172.21.15.84:0/2116944646 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5639c2aa09c0 0x5639c2aa2db0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57c7022ab5a40211 server_cookie=63f4ef993b6a3c00 in_seq=0 out_seq=0 2024-05-22T23:08:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.760+0000 7f7841ffb640 10 client.16524 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.760+0000 7f7841ffb640 1 -- 172.21.15.84:0/2116944646 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5639c298b830 con 0x5639c2aa09c0 2024-05-22T23:08:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.760+0000 7f7841ffb640 10 client.16524 handle_command_reply: tid=0 2024-05-22T23:08:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.760+0000 7f78633eb740 1 client.16524 shutdown 2024-05-22T23:08:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.760+0000 7f78633eb740 1 -- 172.21.15.84:0/2116944646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a1ef80 msgr2=0x5639c2a32510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:52.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.760+0000 7f78633eb740 1 --2- 172.21.15.84:0/2116944646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a1ef80 0x5639c2a32510 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7848000c00 tx=0x7f7848003b00 comp rx=0 tx=0).stop 2024-05-22T23:08:52.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.760+0000 7f78633eb740 1 -- 172.21.15.84:0/2116944646 shutdown_connections 2024-05-22T23:08:52.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.760+0000 7f78633eb740 1 --2- 172.21.15.84:0/2116944646 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5639c2aa09c0 0x5639c2aa2db0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.760+0000 7f78633eb740 1 --2- 172.21.15.84:0/2116944646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639c2a29f70 0x5639c2a23d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.760+0000 7f78633eb740 1 --2- 172.21.15.84:0/2116944646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639c2a1f8a0 0x5639c2a23850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.760+0000 7f78633eb740 1 --2- 172.21.15.84:0/2116944646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639c2a1ef80 0x5639c2a32510 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.760+0000 7f78633eb740 1 -- 172.21.15.84:0/2116944646 >> 172.21.15.84:0/2116944646 conn(0x5639c2a1b2a0 msgr2=0x5639c2a1a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:52.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.760+0000 7f7841ffb640 0 client.16524 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:52.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.760+0000 7f78633eb740 1 -- 172.21.15.84:0/2116944646 shutdown_connections 2024-05-22T23:08:52.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.760+0000 7f78633eb740 1 -- 172.21.15.84:0/2116944646 wait complete. 2024-05-22T23:08:52.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.760+0000 7f78633eb740 20 client.16524 trim_cache size 0 max 16384 2024-05-22T23:08:52.773 INFO:teuthology.orchestra.run.smithi084.stdout:[] 2024-05-22T23:08:52.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.761+0000 7f78606da640 1 -- 172.21.15.84:0/1118444879 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f78440a2d00 msgr2=0x7f78440a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:52.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.761+0000 7f78606da640 1 --2- 172.21.15.84:0/1118444879 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f78440a2d00 0x7f78440a51c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7850097420 tx=0x7f785006cf80 comp rx=0 tx=0).stop 2024-05-22T23:08:52.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.761+0000 7f78606da640 1 -- 172.21.15.84:0/1118444879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78581067c0 msgr2=0x7f785819c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:52.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.761+0000 7f78606da640 1 --2- 172.21.15.84:0/1118444879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78581067c0 0x7f785819c990 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f785000b970 tx=0x7f785000be30 comp rx=0 tx=0).stop 2024-05-22T23:08:52.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.761+0000 7f78606da640 1 -- 172.21.15.84:0/1118444879 shutdown_connections 2024-05-22T23:08:52.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.761+0000 7f78606da640 1 --2- 172.21.15.84:0/1118444879 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f78440a2d00 0x7f78440a51c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.761+0000 7f78606da640 1 --2- 172.21.15.84:0/1118444879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78581067c0 0x7f785819c990 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.761+0000 7f78606da640 1 --2- 172.21.15.84:0/1118444879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7858101460 0x7f7858198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.761+0000 7f78606da640 1 --2- 172.21.15.84:0/1118444879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7858100ac0 0x7f7858198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:52.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.761+0000 7f78606da640 1 -- 172.21.15.84:0/1118444879 >> 172.21.15.84:0/1118444879 conn(0x7f7858076570 msgr2=0x7f78580ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:52.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.761+0000 7f78606da640 1 -- 172.21.15.84:0/1118444879 shutdown_connections 2024-05-22T23:08:52.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:52.761+0000 7f78606da640 1 -- 172.21.15.84:0/1118444879 wait complete. 2024-05-22T23:08:52.879 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:52 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1008414642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:08:53.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:52 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1008414642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:08:53.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:52 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1008414642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:08:53.441 INFO:tasks.workunit.client.0.smithi084.stdout: 1 952452 14.95 MB/sec execute 219 sec latency 124.839 ms 2024-05-22T23:08:53.779 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-22T23:08:53.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:08:53.780 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-22T23:08:53.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.885+0000 7fce67e5d640 1 -- 172.21.15.84:0/3721484719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce601018c0 msgr2=0x7fce60101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:53.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.885+0000 7fce67e5d640 1 --2- 172.21.15.84:0/3721484719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce601018c0 0x7fce60101d40 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009930 tx=0x7fce5c02a420 comp rx=0 tx=0).stop 2024-05-22T23:08:53.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.885+0000 7fce67e5d640 1 -- 172.21.15.84:0/3721484719 shutdown_connections 2024-05-22T23:08:53.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.885+0000 7fce67e5d640 1 --2- 172.21.15.84:0/3721484719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce60102280 0x7fce6010e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.885+0000 7fce67e5d640 1 --2- 172.21.15.84:0/3721484719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce601018c0 0x7fce60101d40 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.885+0000 7fce67e5d640 1 --2- 172.21.15.84:0/3721484719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce601078d0 0x7fce60107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.885+0000 7fce67e5d640 1 -- 172.21.15.84:0/3721484719 >> 172.21.15.84:0/3721484719 conn(0x7fce600fd3d0 msgr2=0x7fce600ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:53.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.885+0000 7fce67e5d640 1 -- 172.21.15.84:0/3721484719 shutdown_connections 2024-05-22T23:08:53.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.885+0000 7fce67e5d640 1 -- 172.21.15.84:0/3721484719 wait complete. 2024-05-22T23:08:53.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.886+0000 7fce67e5d640 1 Processor -- start 2024-05-22T23:08:53.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.886+0000 7fce67e5d640 1 -- start start 2024-05-22T23:08:53.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.886+0000 7fce67e5d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce601018c0 0x7fce60198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:53.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.886+0000 7fce67e5d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce60102280 0x7fce60198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:53.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.886+0000 7fce67e5d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce601078d0 0x7fce6019cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:53.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.886+0000 7fce67e5d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fce60111ce0 con 0x7fce601078d0 2024-05-22T23:08:53.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.886+0000 7fce67e5d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fce60111b60 con 0x7fce601018c0 2024-05-22T23:08:53.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.886+0000 7fce67e5d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fce60111e60 con 0x7fce60102280 2024-05-22T23:08:53.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.886+0000 7fce663fa640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce601078d0 0x7fce6019cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:53.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.886+0000 7fce663fa640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce601078d0 0x7fce6019cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49554/0 (socket says 172.21.15.84:49554) 2024-05-22T23:08:53.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.886+0000 7fce663fa640 1 -- 172.21.15.84:0/1835347844 learned_addr learned my addr 172.21.15.84:0/1835347844 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:53.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.886+0000 7fce65bf9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce601018c0 0x7fce60198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:53.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.886+0000 7fce653f8640 1 --2- 172.21.15.84:0/1835347844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce60102280 0x7fce60198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:53.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.886+0000 7fce663fa640 1 -- 172.21.15.84:0/1835347844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce60102280 msgr2=0x7fce60198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:53.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.886+0000 7fce663fa640 1 --2- 172.21.15.84:0/1835347844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce60102280 0x7fce60198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.886+0000 7fce663fa640 1 -- 172.21.15.84:0/1835347844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce601018c0 msgr2=0x7fce60198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:53.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.886+0000 7fce663fa640 1 --2- 172.21.15.84:0/1835347844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce601018c0 0x7fce60198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.886+0000 7fce663fa640 1 -- 172.21.15.84:0/1835347844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce6019d230 con 0x7fce601078d0 2024-05-22T23:08:53.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.886+0000 7fce663fa640 1 --2- 172.21.15.84:0/1835347844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce601078d0 0x7fce6019cb00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fce5800eb10 tx=0x7fce5800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:53.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.886+0000 7fce52ffd640 1 -- 172.21.15.84:0/1835347844 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58004b20 con 0x7fce601078d0 2024-05-22T23:08:53.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.887+0000 7fce52ffd640 1 -- 172.21.15.84:0/1835347844 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce58004cc0 con 0x7fce601078d0 2024-05-22T23:08:53.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.887+0000 7fce52ffd640 1 -- 172.21.15.84:0/1835347844 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58010760 con 0x7fce601078d0 2024-05-22T23:08:53.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.887+0000 7fce67e5d640 1 -- 172.21.15.84:0/1835347844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce6019d520 con 0x7fce601078d0 2024-05-22T23:08:53.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.887+0000 7fce67e5d640 1 -- 172.21.15.84:0/1835347844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce601a4db0 con 0x7fce601078d0 2024-05-22T23:08:53.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.889+0000 7fce52ffd640 1 -- 172.21.15.84:0/1835347844 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fce58005020 con 0x7fce601078d0 2024-05-22T23:08:53.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.889+0000 7fce6ab6d740 1 Processor -- start 2024-05-22T23:08:53.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.889+0000 7fce6ab6d740 1 -- start start 2024-05-22T23:08:53.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.890+0000 7fce6ab6d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b5fab50 0x56351b5faf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:53.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.890+0000 7fce6ab6d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56351b5fb470 0x56351b609b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:53.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.890+0000 7fce6ab6d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56351b60a050 0x56351b60c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:53.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.890+0000 7fce6ab6d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56351b55ea40 con 0x56351b5fab50 2024-05-22T23:08:53.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.890+0000 7fce6ab6d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56351b4fd440 con 0x56351b5fb470 2024-05-22T23:08:53.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.890+0000 7fce6ab6d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56351b55aa00 con 0x56351b60a050 2024-05-22T23:08:53.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.890+0000 7fce653f8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b5fab50 0x56351b5faf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:53.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.890+0000 7fce52ffd640 1 --2- 172.21.15.84:0/1835347844 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fce4c0a2d00 0x7fce4c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:53.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.890+0000 7fce653f8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b5fab50 0x56351b5faf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49564/0 (socket says 172.21.15.84:49564) 2024-05-22T23:08:53.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.890+0000 7fce653f8640 1 -- 172.21.15.84:0/2973410024 learned_addr learned my addr 172.21.15.84:0/2973410024 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:53.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.890+0000 7fce65bf9640 1 --2- 172.21.15.84:0/2973410024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56351b5fb470 0x56351b609b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:53.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.890+0000 7fce663fa640 1 --2- 172.21.15.84:0/1835347844 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fce4c0a2d00 0x7fce4c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:53.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.890+0000 7fce653f8640 1 -- 172.21.15.84:0/2973410024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56351b60a050 msgr2=0x56351b60c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:53.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.890+0000 7fce653f8640 1 --2- 172.21.15.84:0/2973410024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56351b60a050 0x56351b60c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.890+0000 7fce653f8640 1 -- 172.21.15.84:0/2973410024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56351b5fb470 msgr2=0x56351b609b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:53.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.890+0000 7fce653f8640 1 --2- 172.21.15.84:0/2973410024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56351b5fb470 0x56351b609b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.890+0000 7fce653f8640 1 -- 172.21.15.84:0/2973410024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56351b5072d0 con 0x56351b5fab50 2024-05-22T23:08:53.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.890+0000 7fce65bf9640 1 --2- 172.21.15.84:0/2973410024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56351b5fb470 0x56351b609b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:08:53.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.890+0000 7fce52ffd640 1 -- 172.21.15.84:0/1835347844 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7fce580cfa30 con 0x7fce601078d0 2024-05-22T23:08:53.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.890+0000 7fce653f8640 1 --2- 172.21.15.84:0/2973410024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b5fab50 0x56351b5faf30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fce5c02a940 tx=0x7fce5c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0231b902177daa server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:53.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.890+0000 7fce663fa640 1 --2- 172.21.15.84:0/1835347844 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fce4c0a2d00 0x7fce4c0a51c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fce5809beb0 tx=0x7fce58066ff0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:53.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.890+0000 7fce4bfff640 1 -- 172.21.15.84:0/2973410024 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c035040 con 0x56351b5fab50 2024-05-22T23:08:53.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.891+0000 7fce4bfff640 1 -- 172.21.15.84:0/2973410024 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce5c031830 con 0x56351b5fab50 2024-05-22T23:08:53.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.891+0000 7fce4bfff640 1 -- 172.21.15.84:0/2973410024 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c004a00 con 0x56351b5fab50 2024-05-22T23:08:53.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.891+0000 7fce6ab6d740 1 -- 172.21.15.84:0/2973410024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b5fab50 msgr2=0x56351b5faf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:53.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.891+0000 7fce6ab6d740 1 --2- 172.21.15.84:0/2973410024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b5fab50 0x56351b5faf30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fce5c02a940 tx=0x7fce5c005d50 comp rx=0 tx=0).stop 2024-05-22T23:08:53.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.891+0000 7fce6ab6d740 1 -- 172.21.15.84:0/2973410024 shutdown_connections 2024-05-22T23:08:53.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.891+0000 7fce6ab6d740 1 --2- 172.21.15.84:0/2973410024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56351b60a050 0x56351b60c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.891+0000 7fce6ab6d740 1 --2- 172.21.15.84:0/2973410024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56351b5fb470 0x56351b609b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.891+0000 7fce6ab6d740 1 --2- 172.21.15.84:0/2973410024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b5fab50 0x56351b5faf30 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.891+0000 7fce6ab6d740 1 -- 172.21.15.84:0/2973410024 >> 172.21.15.84:0/2973410024 conn(0x56351b5f6f70 msgr2=0x56351b5f7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:53.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.891+0000 7fce6ab6d740 1 -- 172.21.15.84:0/2973410024 shutdown_connections 2024-05-22T23:08:53.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.891+0000 7fce6ab6d740 1 -- 172.21.15.84:0/2973410024 wait complete. 2024-05-22T23:08:53.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.892+0000 7fce6ab6d740 1 -- start start 2024-05-22T23:08:53.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.892+0000 7fce6ab6d740 1 Processor -- start 2024-05-22T23:08:53.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.892+0000 7fce6ab6d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56351b5fab50 0x56351b606fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:53.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.892+0000 7fce6ab6d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b5fb470 0x56351b6074f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:53.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.892+0000 7fce6ab6d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56351b60a050 0x56351b607a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:53.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.892+0000 7fce6ab6d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56351b60cdb0 con 0x56351b5fb470 2024-05-22T23:08:53.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.892+0000 7fce6ab6d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56351b4e85f0 con 0x56351b60a050 2024-05-22T23:08:53.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.892+0000 7fce6ab6d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56351b60cf30 con 0x56351b5fab50 2024-05-22T23:08:53.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.892+0000 7fce6ab6d740 10 client.0 _ll_register_callbacks cb 0x56351b4e3cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:08:53.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.892+0000 7fce65bf9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b5fb470 0x56351b6074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:53.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.892+0000 7fce65bf9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b5fb470 0x56351b6074f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49570/0 (socket says 172.21.15.84:49570) 2024-05-22T23:08:53.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.892+0000 7fce65bf9640 1 -- 172.21.15.84:0/2534117025 learned_addr learned my addr 172.21.15.84:0/2534117025 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:53.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.892+0000 7fce653f8640 1 --2- 172.21.15.84:0/2534117025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56351b60a050 0x56351b607a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:53.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.892+0000 7fce653f8640 1 --2- 172.21.15.84:0/2534117025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56351b5fab50 0x56351b606fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:53.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.893+0000 7fce65bf9640 1 -- 172.21.15.84:0/2534117025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56351b5fab50 msgr2=0x56351b606fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:53.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.893+0000 7fce65bf9640 1 --2- 172.21.15.84:0/2534117025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56351b5fab50 0x56351b606fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.893+0000 7fce65bf9640 1 -- 172.21.15.84:0/2534117025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56351b60a050 msgr2=0x56351b607a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:53.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.893+0000 7fce65bf9640 1 --2- 172.21.15.84:0/2534117025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56351b60a050 0x56351b607a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.893+0000 7fce65bf9640 1 -- 172.21.15.84:0/2534117025 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56351b33a7e0 con 0x56351b5fb470 2024-05-22T23:08:53.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.893+0000 7fce65bf9640 1 --2- 172.21.15.84:0/2534117025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b5fb470 0x56351b6074f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fce540126e0 tx=0x7fce54012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878cab512a564eff server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:53.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.893+0000 7fce497fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:08:53.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.893+0000 7fce497fa640 1 -- 172.21.15.84:0/2534117025 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce540120d0 con 0x56351b5fb470 2024-05-22T23:08:53.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.893+0000 7fce497fa640 1 -- 172.21.15.84:0/2534117025 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7fce5400f920 con 0x56351b5fb470 2024-05-22T23:08:53.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.893+0000 7fce6ab6d740 1 -- 172.21.15.84:0/2534117025 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce5c031830 con 0x56351b5fb470 2024-05-22T23:08:53.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.893+0000 7fce497fa640 1 -- 172.21.15.84:0/2534117025 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fce54012270 con 0x56351b5fb470 2024-05-22T23:08:53.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.893+0000 7fce6ab6d740 10 client.16536 fetch_fsmap learned FSMap version 18 2024-05-22T23:08:53.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.893+0000 7fce6ab6d740 1 -- 172.21.15.84:0/2534117025 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x56351b39af70 con 0x56351b5fb470 2024-05-22T23:08:53.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.893+0000 7fce497fa640 1 -- 172.21.15.84:0/2534117025 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce5400fb60 con 0x56351b5fb470 2024-05-22T23:08:53.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.893+0000 7fce497fa640 1 -- 172.21.15.84:0/2534117025 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fce540104c0 con 0x56351b5fb470 2024-05-22T23:08:53.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.893+0000 7fce6ab6d740 10 client.16536 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:08:53.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.893+0000 7fce6ab6d740 10 client.16536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:08:53.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.894+0000 7fce6ab6d740 20 client.16536 populate_metadata read hostname 'smithi084' 2024-05-22T23:08:53.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.894+0000 7fce6ab6d740 1 --2- 172.21.15.84:0/2534117025 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56351b61a4a0 0x56351b61c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:53.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.894+0000 7fce6ab6d740 4 client.16536 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:08:53.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.894+0000 7fce6ab6d740 1 -- 172.21.15.84:0/2534117025 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fce5c031830 con 0x56351b61a4a0 2024-05-22T23:08:53.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.894+0000 7fce653f8640 1 --2- 172.21.15.84:0/2534117025 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56351b61a4a0 0x56351b61c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:53.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.894+0000 7fce497fa640 10 client.16536 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:53.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.894+0000 7fce653f8640 1 --2- 172.21.15.84:0/2534117025 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56351b61a4a0 0x56351b61c8c0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4ebf9182cb0a36b server_cookie=ef4b37a6a93c1138 in_seq=0 out_seq=0 2024-05-22T23:08:53.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.895+0000 7fce497fa640 1 -- 172.21.15.84:0/2534117025 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fce5c031830 con 0x56351b61a4a0 2024-05-22T23:08:53.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.895+0000 7fce497fa640 10 client.16536 handle_command_reply: tid=0 2024-05-22T23:08:53.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.895+0000 7fce6ab6d740 1 client.16536 shutdown 2024-05-22T23:08:53.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.895+0000 7fce6ab6d740 1 -- 172.21.15.84:0/2534117025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b5fb470 msgr2=0x56351b6074f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:53.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.895+0000 7fce6ab6d740 1 --2- 172.21.15.84:0/2534117025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b5fb470 0x56351b6074f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fce540126e0 tx=0x7fce54012ba0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.895+0000 7fce6ab6d740 1 -- 172.21.15.84:0/2534117025 shutdown_connections 2024-05-22T23:08:53.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.895+0000 7fce6ab6d740 1 --2- 172.21.15.84:0/2534117025 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56351b61a4a0 0x56351b61c8c0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.895+0000 7fce6ab6d740 1 --2- 172.21.15.84:0/2534117025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56351b60a050 0x56351b607a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.895+0000 7fce6ab6d740 1 --2- 172.21.15.84:0/2534117025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b5fb470 0x56351b6074f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.895+0000 7fce6ab6d740 1 --2- 172.21.15.84:0/2534117025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56351b5fab50 0x56351b606fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.895+0000 7fce497fa640 0 client.16536 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:53.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.895+0000 7fce6ab6d740 1 -- 172.21.15.84:0/2534117025 >> 172.21.15.84:0/2534117025 conn(0x56351b5f6f70 msgr2=0x56351b5fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:53.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.895+0000 7fce6ab6d740 1 -- 172.21.15.84:0/2534117025 shutdown_connections 2024-05-22T23:08:53.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.895+0000 7fce6ab6d740 1 -- 172.21.15.84:0/2534117025 wait complete. 2024-05-22T23:08:53.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.895+0000 7fce6ab6d740 20 client.16536 trim_cache size 0 max 16384 2024-05-22T23:08:53.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.914+0000 7fce6ab6d740 1 Processor -- start 2024-05-22T23:08:53.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.914+0000 7fce6ab6d740 1 -- start start 2024-05-22T23:08:53.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.914+0000 7fce6ab6d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56351b5fab50 0x56351b615d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:53.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.914+0000 7fce6ab6d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56351b5fb470 0x56351b622170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:53.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.914+0000 7fce6ab6d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b60a050 0x56351b6226b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:53.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.914+0000 7fce6ab6d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fce601a9490 con 0x56351b60a050 2024-05-22T23:08:53.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.914+0000 7fce6ab6d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fce601abcf0 con 0x56351b5fb470 2024-05-22T23:08:53.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.914+0000 7fce6ab6d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fce601a9ae0 con 0x56351b5fab50 2024-05-22T23:08:53.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.914+0000 7fce653f8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b60a050 0x56351b6226b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:53.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.914+0000 7fce653f8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b60a050 0x56351b6226b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49572/0 (socket says 172.21.15.84:49572) 2024-05-22T23:08:53.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.914+0000 7fce65bf9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56351b5fb470 0x56351b622170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:53.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.914+0000 7fce653f8640 1 -- 172.21.15.84:0/3433806703 learned_addr learned my addr 172.21.15.84:0/3433806703 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:53.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.915+0000 7fce653f8640 1 --2- 172.21.15.84:0/3433806703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56351b5fab50 0x56351b615d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:53.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.915+0000 7fce653f8640 1 -- 172.21.15.84:0/3433806703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56351b5fab50 msgr2=0x56351b615d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:53.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.915+0000 7fce653f8640 1 --2- 172.21.15.84:0/3433806703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56351b5fab50 0x56351b615d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.915+0000 7fce653f8640 1 -- 172.21.15.84:0/3433806703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56351b5fb470 msgr2=0x56351b622170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:53.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.915+0000 7fce653f8640 1 --2- 172.21.15.84:0/3433806703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56351b5fb470 0x56351b622170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.915+0000 7fce653f8640 1 -- 172.21.15.84:0/3433806703 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56351b605f60 con 0x56351b60a050 2024-05-22T23:08:53.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.915+0000 7fce65bf9640 1 --2- 172.21.15.84:0/3433806703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56351b5fb470 0x56351b622170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:08:53.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.915+0000 7fce653f8640 1 --2- 172.21.15.84:0/3433806703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b60a050 0x56351b6226b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fce5c031b20 tx=0x7fce5c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888318ad0b880094 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:53.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.915+0000 7fce4b7fe640 1 -- 172.21.15.84:0/3433806703 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c035040 con 0x56351b60a050 2024-05-22T23:08:53.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.915+0000 7fce4b7fe640 1 -- 172.21.15.84:0/3433806703 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce5c054830 con 0x56351b60a050 2024-05-22T23:08:53.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.915+0000 7fce4b7fe640 1 -- 172.21.15.84:0/3433806703 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c004aa0 con 0x56351b60a050 2024-05-22T23:08:53.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.916+0000 7fce6ab6d740 1 -- 172.21.15.84:0/3433806703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b60a050 msgr2=0x56351b6226b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:53.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.916+0000 7fce6ab6d740 1 --2- 172.21.15.84:0/3433806703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b60a050 0x56351b6226b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fce5c031b20 tx=0x7fce5c005fd0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.916+0000 7fce6ab6d740 1 -- 172.21.15.84:0/3433806703 shutdown_connections 2024-05-22T23:08:53.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.916+0000 7fce6ab6d740 1 --2- 172.21.15.84:0/3433806703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b60a050 0x56351b6226b0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.916+0000 7fce6ab6d740 1 --2- 172.21.15.84:0/3433806703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56351b5fb470 0x56351b622170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.916+0000 7fce6ab6d740 1 --2- 172.21.15.84:0/3433806703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56351b5fab50 0x56351b615d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.916+0000 7fce6ab6d740 1 -- 172.21.15.84:0/3433806703 >> 172.21.15.84:0/3433806703 conn(0x56351b5f6f70 msgr2=0x56351b5f61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:53.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.916+0000 7fce6ab6d740 1 -- 172.21.15.84:0/3433806703 shutdown_connections 2024-05-22T23:08:53.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.916+0000 7fce6ab6d740 1 -- 172.21.15.84:0/3433806703 wait complete. 2024-05-22T23:08:53.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.916+0000 7fce6ab6d740 1 -- start start 2024-05-22T23:08:53.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.916+0000 7fce6ab6d740 1 Processor -- start 2024-05-22T23:08:53.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.917+0000 7fce6ab6d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b5fab50 0x56351b60ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:53.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.917+0000 7fce6ab6d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56351b5fb470 0x56351b60f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:53.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.917+0000 7fce6ab6d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56351b60a050 0x56351b60f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:53.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.917+0000 7fce6ab6d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fce601aa1b0 con 0x56351b5fab50 2024-05-22T23:08:53.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.917+0000 7fce6ab6d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fce601ab620 con 0x56351b5fb470 2024-05-22T23:08:53.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.917+0000 7fce6ab6d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fce601aa880 con 0x56351b60a050 2024-05-22T23:08:53.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.917+0000 7fce6ab6d740 10 client.0 _ll_register_callbacks cb 0x7fce54009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:08:53.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.917+0000 7fce653f8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b5fab50 0x56351b60ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:53.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.917+0000 7fce653f8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b5fab50 0x56351b60ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49580/0 (socket says 172.21.15.84:49580) 2024-05-22T23:08:53.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.917+0000 7fce653f8640 1 -- 172.21.15.84:0/4078424862 learned_addr learned my addr 172.21.15.84:0/4078424862 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:53.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.917+0000 7fce65bf9640 1 --2- 172.21.15.84:0/4078424862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56351b5fb470 0x56351b60f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:53.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.917+0000 7fce653f8640 1 -- 172.21.15.84:0/4078424862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56351b60a050 msgr2=0x56351b60f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:53.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.917+0000 7fce653f8640 1 --2- 172.21.15.84:0/4078424862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56351b60a050 0x56351b60f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.917+0000 7fce653f8640 1 -- 172.21.15.84:0/4078424862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56351b5fb470 msgr2=0x56351b60f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:53.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.917+0000 7fce653f8640 1 --2- 172.21.15.84:0/4078424862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56351b5fb470 0x56351b60f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.917+0000 7fce653f8640 1 -- 172.21.15.84:0/4078424862 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56351b5fb2a0 con 0x56351b5fab50 2024-05-22T23:08:53.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.917+0000 7fce653f8640 1 --2- 172.21.15.84:0/4078424862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b5fab50 0x56351b60ebd0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fce5c001eb0 tx=0x7fce5c031aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fbfc93a11988b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:53.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.917+0000 7fce497fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:08:53.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.917+0000 7fce497fa640 1 -- 172.21.15.84:0/4078424862 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c058020 con 0x56351b5fab50 2024-05-22T23:08:53.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.917+0000 7fce497fa640 1 -- 172.21.15.84:0/4078424862 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7fce5c0549e0 con 0x56351b5fab50 2024-05-22T23:08:53.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.917+0000 7fce6ab6d740 1 -- 172.21.15.84:0/4078424862 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce5c054830 con 0x56351b5fab50 2024-05-22T23:08:53.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.917+0000 7fce497fa640 1 -- 172.21.15.84:0/4078424862 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fce5c035040 con 0x56351b5fab50 2024-05-22T23:08:53.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.917+0000 7fce6ab6d740 10 client.16542 fetch_fsmap learned FSMap version 18 2024-05-22T23:08:53.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.917+0000 7fce6ab6d740 1 -- 172.21.15.84:0/4078424862 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x56351b232170 con 0x56351b5fab50 2024-05-22T23:08:53.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.918+0000 7fce497fa640 1 -- 172.21.15.84:0/4078424862 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce5c054830 con 0x56351b5fab50 2024-05-22T23:08:53.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.918+0000 7fce497fa640 1 -- 172.21.15.84:0/4078424862 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fce5c054ce0 con 0x56351b5fab50 2024-05-22T23:08:53.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.918+0000 7fce6ab6d740 10 client.16542 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:08:53.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.918+0000 7fce6ab6d740 10 client.16542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:08:53.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.918+0000 7fce6ab6d740 20 client.16542 populate_metadata read hostname 'smithi084' 2024-05-22T23:08:53.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.918+0000 7fce6ab6d740 1 --2- 172.21.15.84:0/4078424862 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56351b681bd0 0x56351b683fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:53.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.918+0000 7fce6ab6d740 4 client.16542 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-22T23:08:53.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.918+0000 7fce6ab6d740 1 -- 172.21.15.84:0/4078424862 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56351b572460 con 0x56351b681bd0 2024-05-22T23:08:53.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.918+0000 7fce65bf9640 1 --2- 172.21.15.84:0/4078424862 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56351b681bd0 0x56351b683fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:53.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.918+0000 7fce497fa640 10 client.16542 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:53.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.918+0000 7fce65bf9640 1 --2- 172.21.15.84:0/4078424862 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56351b681bd0 0x56351b683fc0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed8ee4cd4a6711b7 server_cookie=206720c72395f30f in_seq=0 out_seq=0 2024-05-22T23:08:53.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.919+0000 7fce497fa640 1 -- 172.21.15.84:0/4078424862 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56351b572460 con 0x56351b681bd0 2024-05-22T23:08:53.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.919+0000 7fce497fa640 10 client.16542 handle_command_reply: tid=0 2024-05-22T23:08:53.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.919+0000 7fce6ab6d740 1 client.16542 shutdown 2024-05-22T23:08:53.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.919+0000 7fce6ab6d740 1 -- 172.21.15.84:0/4078424862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b5fab50 msgr2=0x56351b60ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:53.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.919+0000 7fce6ab6d740 1 --2- 172.21.15.84:0/4078424862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b5fab50 0x56351b60ebd0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fce5c001eb0 tx=0x7fce5c031aa0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.920+0000 7fce6ab6d740 1 -- 172.21.15.84:0/4078424862 shutdown_connections 2024-05-22T23:08:53.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.920+0000 7fce6ab6d740 1 --2- 172.21.15.84:0/4078424862 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56351b681bd0 0x56351b683fc0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.920+0000 7fce6ab6d740 1 --2- 172.21.15.84:0/4078424862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56351b60a050 0x56351b60f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.920+0000 7fce6ab6d740 1 --2- 172.21.15.84:0/4078424862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56351b5fb470 0x56351b60f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.920+0000 7fce6ab6d740 1 --2- 172.21.15.84:0/4078424862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56351b5fab50 0x56351b60ebd0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.920+0000 7fce497fa640 0 client.16542 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:53.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.920+0000 7fce6ab6d740 1 -- 172.21.15.84:0/4078424862 >> 172.21.15.84:0/4078424862 conn(0x56351b5f6f70 msgr2=0x56351b606f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:53.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.920+0000 7fce6ab6d740 1 -- 172.21.15.84:0/4078424862 shutdown_connections 2024-05-22T23:08:53.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.920+0000 7fce6ab6d740 1 -- 172.21.15.84:0/4078424862 wait complete. 2024-05-22T23:08:53.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.920+0000 7fce6ab6d740 20 client.16542 trim_cache size 0 max 16384 2024-05-22T23:08:53.932 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:08:53.932 INFO:teuthology.orchestra.run.smithi084.stdout: "return_code": 0, 2024-05-22T23:08:53.932 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_tag": "eb8deda0-6ecc-44d4-97e8-466c256d9d5d", 2024-05-22T23:08:53.932 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": "asynchronous" 2024-05-22T23:08:53.932 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:08:53.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.920+0000 7fce67e5d640 1 -- 172.21.15.84:0/1835347844 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fce4c0a2d00 msgr2=0x7fce4c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:53.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.920+0000 7fce67e5d640 1 --2- 172.21.15.84:0/1835347844 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fce4c0a2d00 0x7fce4c0a51c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fce5809beb0 tx=0x7fce58066ff0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.920+0000 7fce67e5d640 1 -- 172.21.15.84:0/1835347844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce601078d0 msgr2=0x7fce6019cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:53.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.920+0000 7fce67e5d640 1 --2- 172.21.15.84:0/1835347844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce601078d0 0x7fce6019cb00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fce5800eb10 tx=0x7fce5800c2d0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.920+0000 7fce67e5d640 1 -- 172.21.15.84:0/1835347844 shutdown_connections 2024-05-22T23:08:53.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.920+0000 7fce67e5d640 1 --2- 172.21.15.84:0/1835347844 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fce4c0a2d00 0x7fce4c0a51c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.920+0000 7fce67e5d640 1 --2- 172.21.15.84:0/1835347844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce601078d0 0x7fce6019cb00 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.920+0000 7fce67e5d640 1 --2- 172.21.15.84:0/1835347844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce60102280 0x7fce60198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.920+0000 7fce67e5d640 1 --2- 172.21.15.84:0/1835347844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce601018c0 0x7fce60198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:53.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.920+0000 7fce67e5d640 1 -- 172.21.15.84:0/1835347844 >> 172.21.15.84:0/1835347844 conn(0x7fce600fd3d0 msgr2=0x7fce600fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:53.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.920+0000 7fce67e5d640 1 -- 172.21.15.84:0/1835347844 shutdown_connections 2024-05-22T23:08:53.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:53.920+0000 7fce67e5d640 1 -- 172.21.15.84:0/1835347844 wait complete. 2024-05-22T23:08:53.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:08:53.937 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:08:54.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.038+0000 7f65c5c1b640 1 -- 172.21.15.84:0/1125269175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65c0100ac0 msgr2=0x7f65c0100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:54.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.038+0000 7f65c5c1b640 1 --2- 172.21.15.84:0/1125269175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65c0100ac0 0x7f65c0100f20 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f65b4009900 tx=0x7f65b402a410 comp rx=0 tx=0).stop 2024-05-22T23:08:54.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.038+0000 7f65c5c1b640 1 -- 172.21.15.84:0/1125269175 shutdown_connections 2024-05-22T23:08:54.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.038+0000 7f65c5c1b640 1 --2- 172.21.15.84:0/1125269175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65c0101460 0x7f65c010d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.038+0000 7f65c5c1b640 1 --2- 172.21.15.84:0/1125269175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65c0100ac0 0x7f65c0100f20 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.038+0000 7f65c5c1b640 1 --2- 172.21.15.84:0/1125269175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65c01067c0 0x7f65c0106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.038+0000 7f65c5c1b640 1 -- 172.21.15.84:0/1125269175 >> 172.21.15.84:0/1125269175 conn(0x7f65c0076570 msgr2=0x7f65c0076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:54.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.038+0000 7f65c5c1b640 1 -- 172.21.15.84:0/1125269175 shutdown_connections 2024-05-22T23:08:54.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.038+0000 7f65c5c1b640 1 -- 172.21.15.84:0/1125269175 wait complete. 2024-05-22T23:08:54.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.039+0000 7f65c5c1b640 1 Processor -- start 2024-05-22T23:08:54.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.039+0000 7f65c5c1b640 1 -- start start 2024-05-22T23:08:54.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.039+0000 7f65c5c1b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65c0100ac0 0x7f65c0196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:54.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.039+0000 7f65c5c1b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65c0101460 0x7f65c01974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:54.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.039+0000 7f65c5c1b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65c01067c0 0x7f65c019b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:54.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.039+0000 7f65c5c1b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f65c0110b10 con 0x7f65c0101460 2024-05-22T23:08:54.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.039+0000 7f65c5c1b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f65c0110990 con 0x7f65c01067c0 2024-05-22T23:08:54.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.039+0000 7f65c5c1b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f65c0110c90 con 0x7f65c0100ac0 2024-05-22T23:08:54.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.039+0000 7f65beffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65c0101460 0x7f65c01974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:54.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.039+0000 7f65beffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65c0101460 0x7f65c01974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49594/0 (socket says 172.21.15.84:49594) 2024-05-22T23:08:54.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.039+0000 7f65beffd640 1 -- 172.21.15.84:0/1755013401 learned_addr learned my addr 172.21.15.84:0/1755013401 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:54.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.039+0000 7f65bf7fe640 1 --2- 172.21.15.84:0/1755013401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65c0100ac0 0x7f65c0196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:54.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.039+0000 7f65bffff640 1 --2- 172.21.15.84:0/1755013401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65c01067c0 0x7f65c019b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:54.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.040+0000 7f65beffd640 1 -- 172.21.15.84:0/1755013401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65c0100ac0 msgr2=0x7f65c0196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:54.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.040+0000 7f65beffd640 1 --2- 172.21.15.84:0/1755013401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65c0100ac0 0x7f65c0196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.040+0000 7f65beffd640 1 -- 172.21.15.84:0/1755013401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65c01067c0 msgr2=0x7f65c019b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:54.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.040+0000 7f65beffd640 1 --2- 172.21.15.84:0/1755013401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65c01067c0 0x7f65c019b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.040+0000 7f65beffd640 1 -- 172.21.15.84:0/1755013401 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c019bfa0 con 0x7f65c0101460 2024-05-22T23:08:54.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.040+0000 7f65bf7fe640 1 --2- 172.21.15.84:0/1755013401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65c0100ac0 0x7f65c0196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:08:54.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.040+0000 7f65bffff640 1 --2- 172.21.15.84:0/1755013401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65c01067c0 0x7f65c019b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:08:54.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.040+0000 7f65beffd640 1 --2- 172.21.15.84:0/1755013401 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65c0101460 0x7f65c01974f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f65b4006010 tx=0x7f65b4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:54.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.040+0000 7f65bcff9640 1 -- 172.21.15.84:0/1755013401 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b403f030 con 0x7f65c0101460 2024-05-22T23:08:54.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.040+0000 7f65c5c1b640 1 -- 172.21.15.84:0/1755013401 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c019c230 con 0x7f65c0101460 2024-05-22T23:08:54.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.040+0000 7f65bcff9640 1 -- 172.21.15.84:0/1755013401 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b4004440 con 0x7f65c0101460 2024-05-22T23:08:54.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.040+0000 7f65bcff9640 1 -- 172.21.15.84:0/1755013401 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4035070 con 0x7f65c0101460 2024-05-22T23:08:54.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.040+0000 7f65c5c1b640 1 -- 172.21.15.84:0/1755013401 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c01a3b20 con 0x7f65c0101460 2024-05-22T23:08:54.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.042+0000 7f65bcff9640 1 -- 172.21.15.84:0/1755013401 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f65b4042030 con 0x7f65c0101460 2024-05-22T23:08:54.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.043+0000 7f65c89fd740 1 Processor -- start 2024-05-22T23:08:54.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.043+0000 7f65c89fd740 1 -- start start 2024-05-22T23:08:54.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.043+0000 7f65c89fd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403ba64a0 0x556403ba47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:54.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.043+0000 7f65c89fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556403ba4db0 0x556403ba5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:54.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.043+0000 7f65c89fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556403bafb30 0x556403bb1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:54.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.043+0000 7f65c89fd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556403b08db0 con 0x556403ba64a0 2024-05-22T23:08:54.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.043+0000 7f65c89fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556403aa7360 con 0x556403bafb30 2024-05-22T23:08:54.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.043+0000 7f65c89fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556403b04d70 con 0x556403ba4db0 2024-05-22T23:08:54.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.043+0000 7f65bffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403ba64a0 0x556403ba47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:54.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.043+0000 7f65bffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403ba64a0 0x556403ba47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49598/0 (socket says 172.21.15.84:49598) 2024-05-22T23:08:54.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.043+0000 7f65bffff640 1 -- 172.21.15.84:0/3586785326 learned_addr learned my addr 172.21.15.84:0/3586785326 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:54.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.043+0000 7f65beffd640 1 --2- 172.21.15.84:0/3586785326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556403bafb30 0x556403bb1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:54.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.043+0000 7f65bf7fe640 1 --2- 172.21.15.84:0/3586785326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556403ba4db0 0x556403ba5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:54.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.043+0000 7f65bcff9640 1 --2- 172.21.15.84:0/1755013401 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f65a80a2d00 0x7f65a80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:54.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.043+0000 7f65beffd640 1 -- 172.21.15.84:0/3586785326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556403ba4db0 msgr2=0x556403ba5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:54.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.043+0000 7f65beffd640 1 --2- 172.21.15.84:0/3586785326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556403ba4db0 0x556403ba5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.043+0000 7f65beffd640 1 -- 172.21.15.84:0/3586785326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403ba64a0 msgr2=0x556403ba47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:54.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.043+0000 7f65beffd640 1 --2- 172.21.15.84:0/3586785326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403ba64a0 0x556403ba47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.043+0000 7f65beffd640 1 -- 172.21.15.84:0/3586785326 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556403acaa60 con 0x556403bafb30 2024-05-22T23:08:54.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.043+0000 7f65bffff640 1 --2- 172.21.15.84:0/3586785326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403ba64a0 0x556403ba47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:08:54.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.043+0000 7f65bffff640 1 --2- 172.21.15.84:0/1755013401 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f65a80a2d00 0x7f65a80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:54.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.044+0000 7f65bcff9640 1 -- 172.21.15.84:0/1755013401 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f65b40ef7c0 con 0x7f65c0101460 2024-05-22T23:08:54.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.044+0000 7f65beffd640 1 --2- 172.21.15.84:0/3586785326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556403bafb30 0x556403bb1f20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f65b40ecee0 tx=0x7f65b404b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=209e20925863558 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:54.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.044+0000 7f65bffff640 1 --2- 172.21.15.84:0/1755013401 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f65a80a2d00 0x7f65a80a51c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009fd0 tx=0x7f65b8004a40 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:54.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.044+0000 7f65a5ffb640 1 -- 172.21.15.84:0/3586785326 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b40bb6c0 con 0x556403bafb30 2024-05-22T23:08:54.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.044+0000 7f65a5ffb640 1 -- 172.21.15.84:0/3586785326 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b4042320 con 0x556403bafb30 2024-05-22T23:08:54.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.044+0000 7f65a5ffb640 1 -- 172.21.15.84:0/3586785326 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b40c4920 con 0x556403bafb30 2024-05-22T23:08:54.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.044+0000 7f65c89fd740 1 -- 172.21.15.84:0/3586785326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556403bafb30 msgr2=0x556403bb1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:54.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.044+0000 7f65c89fd740 1 --2- 172.21.15.84:0/3586785326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556403bafb30 0x556403bb1f20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f65b40ecee0 tx=0x7f65b404b000 comp rx=0 tx=0).stop 2024-05-22T23:08:54.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.045+0000 7f65c89fd740 1 -- 172.21.15.84:0/3586785326 shutdown_connections 2024-05-22T23:08:54.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.045+0000 7f65c89fd740 1 --2- 172.21.15.84:0/3586785326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556403bafb30 0x556403bb1f20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.045+0000 7f65c89fd740 1 --2- 172.21.15.84:0/3586785326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556403ba4db0 0x556403ba5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.045+0000 7f65c89fd740 1 --2- 172.21.15.84:0/3586785326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403ba64a0 0x556403ba47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.045+0000 7f65c89fd740 1 -- 172.21.15.84:0/3586785326 >> 172.21.15.84:0/3586785326 conn(0x556403b6d8e0 msgr2=0x556403ba1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:54.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.045+0000 7f65c89fd740 1 -- 172.21.15.84:0/3586785326 shutdown_connections 2024-05-22T23:08:54.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.045+0000 7f65c89fd740 1 -- 172.21.15.84:0/3586785326 wait complete. 2024-05-22T23:08:54.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.045+0000 7f65c89fd740 1 -- start start 2024-05-22T23:08:54.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.045+0000 7f65c89fd740 1 Processor -- start 2024-05-22T23:08:54.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.045+0000 7f65c89fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556403ba4db0 0x556403bb2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:54.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.045+0000 7f65c89fd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403ba64a0 0x556403bb3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:54.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.045+0000 7f65c89fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556403bafb30 0x556403bb3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:54.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.045+0000 7f65c89fd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556403bb2780 con 0x556403ba64a0 2024-05-22T23:08:54.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.045+0000 7f65c89fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556403ac30f0 con 0x556403bafb30 2024-05-22T23:08:54.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.045+0000 7f65c89fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556403bb2900 con 0x556403ba4db0 2024-05-22T23:08:54.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.045+0000 7f65c89fd740 10 client.0 _ll_register_callbacks cb 0x556403ac7f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:08:54.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.045+0000 7f65beffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403ba64a0 0x556403bb3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:54.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.046+0000 7f65bffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556403bafb30 0x556403bb3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:54.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.046+0000 7f65beffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403ba64a0 0x556403bb3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49612/0 (socket says 172.21.15.84:49612) 2024-05-22T23:08:54.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.046+0000 7f65beffd640 1 -- 172.21.15.84:0/3451886773 learned_addr learned my addr 172.21.15.84:0/3451886773 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:54.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.046+0000 7f65bf7fe640 1 --2- 172.21.15.84:0/3451886773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556403ba4db0 0x556403bb2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:54.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.046+0000 7f65beffd640 1 -- 172.21.15.84:0/3451886773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556403ba4db0 msgr2=0x556403bb2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:54.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.046+0000 7f65beffd640 1 --2- 172.21.15.84:0/3451886773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556403ba4db0 0x556403bb2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.046+0000 7f65beffd640 1 -- 172.21.15.84:0/3451886773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556403bafb30 msgr2=0x556403bb3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:54.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.046+0000 7f65beffd640 1 --2- 172.21.15.84:0/3451886773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556403bafb30 0x556403bb3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.046+0000 7f65beffd640 1 -- 172.21.15.84:0/3451886773 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564038e67d0 con 0x556403ba64a0 2024-05-22T23:08:54.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.046+0000 7f65bffff640 1 --2- 172.21.15.84:0/3451886773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556403bafb30 0x556403bb3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:08:54.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.046+0000 7f65beffd640 1 --2- 172.21.15.84:0/3451886773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403ba64a0 0x556403bb3050 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f65b40c43f0 tx=0x7f65b40ba270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a264a026704239 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:54.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.046+0000 7f659effd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:08:54.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.046+0000 7f659effd640 1 -- 172.21.15.84:0/3451886773 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b40bb6c0 con 0x556403ba64a0 2024-05-22T23:08:54.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.046+0000 7f659effd640 1 -- 172.21.15.84:0/3451886773 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f65b40b7bf0 con 0x556403ba64a0 2024-05-22T23:08:54.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.046+0000 7f65c89fd740 1 -- 172.21.15.84:0/3451886773 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b4042320 con 0x556403ba64a0 2024-05-22T23:08:54.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.046+0000 7f659effd640 1 -- 172.21.15.84:0/3451886773 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f65b40d9050 con 0x556403ba64a0 2024-05-22T23:08:54.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.046+0000 7f65c89fd740 10 client.16554 fetch_fsmap learned FSMap version 18 2024-05-22T23:08:54.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.046+0000 7f65c89fd740 1 -- 172.21.15.84:0/3451886773 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55640381c3f0 con 0x556403ba64a0 2024-05-22T23:08:54.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.046+0000 7f659effd640 1 -- 172.21.15.84:0/3451886773 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b4042320 con 0x556403ba64a0 2024-05-22T23:08:54.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.047+0000 7f659effd640 1 -- 172.21.15.84:0/3451886773 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f65b40ed160 con 0x556403ba64a0 2024-05-22T23:08:54.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.047+0000 7f65c89fd740 10 client.16554 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:08:54.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.047+0000 7f65c89fd740 10 client.16554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:08:54.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.047+0000 7f65c89fd740 20 client.16554 populate_metadata read hostname 'smithi084' 2024-05-22T23:08:54.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.047+0000 7f65c89fd740 1 --2- 172.21.15.84:0/3451886773 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556403bc1c60 0x556403bc4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:54.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.047+0000 7f65c89fd740 4 client.16554 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:08:54.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.047+0000 7f65c89fd740 1 -- 172.21.15.84:0/3451886773 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556403b164e0 con 0x556403bc1c60 2024-05-22T23:08:54.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.047+0000 7f65bf7fe640 1 --2- 172.21.15.84:0/3451886773 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556403bc1c60 0x556403bc4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:54.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.047+0000 7f659effd640 10 client.16554 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:54.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.047+0000 7f65bf7fe640 1 --2- 172.21.15.84:0/3451886773 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556403bc1c60 0x556403bc4050 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e842c0bd4bf3003 server_cookie=8e5015041de9be6 in_seq=0 out_seq=0 2024-05-22T23:08:54.061 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:53 smithi096 ceph-mon[36392]: pgmap v216: 97 pgs: 97 active+clean; 461 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.3 MiB/s wr, 346 op/s 2024-05-22T23:08:54.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.049+0000 7f659effd640 1 -- 172.21.15.84:0/3451886773 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556403b164e0 con 0x556403bc1c60 2024-05-22T23:08:54.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.049+0000 7f659effd640 10 client.16554 handle_command_reply: tid=0 2024-05-22T23:08:54.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.049+0000 7f65c89fd740 1 client.16554 shutdown 2024-05-22T23:08:54.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.049+0000 7f65c89fd740 1 -- 172.21.15.84:0/3451886773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403ba64a0 msgr2=0x556403bb3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:54.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.049+0000 7f65c89fd740 1 --2- 172.21.15.84:0/3451886773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403ba64a0 0x556403bb3050 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f65b40c43f0 tx=0x7f65b40ba270 comp rx=0 tx=0).stop 2024-05-22T23:08:54.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.049+0000 7f65c89fd740 1 -- 172.21.15.84:0/3451886773 shutdown_connections 2024-05-22T23:08:54.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.049+0000 7f65c89fd740 1 --2- 172.21.15.84:0/3451886773 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556403bc1c60 0x556403bc4050 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.049+0000 7f65c89fd740 1 --2- 172.21.15.84:0/3451886773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556403bafb30 0x556403bb3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.049+0000 7f65c89fd740 1 --2- 172.21.15.84:0/3451886773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403ba64a0 0x556403bb3050 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.049+0000 7f659effd640 0 client.16554 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:54.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.049+0000 7f65c89fd740 1 --2- 172.21.15.84:0/3451886773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556403ba4db0 0x556403bb2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.049+0000 7f65c89fd740 1 -- 172.21.15.84:0/3451886773 >> 172.21.15.84:0/3451886773 conn(0x556403b6d8e0 msgr2=0x556403ba7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:54.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.049+0000 7f65c89fd740 1 -- 172.21.15.84:0/3451886773 shutdown_connections 2024-05-22T23:08:54.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.049+0000 7f65c89fd740 1 -- 172.21.15.84:0/3451886773 wait complete. 2024-05-22T23:08:54.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.049+0000 7f65c89fd740 20 client.16554 trim_cache size 0 max 16384 2024-05-22T23:08:54.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.067+0000 7f65c89fd740 1 Processor -- start 2024-05-22T23:08:54.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.067+0000 7f65c89fd740 1 -- start start 2024-05-22T23:08:54.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.067+0000 7f65c89fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556403ba4db0 0x556403bbd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:54.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.067+0000 7f65c89fd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403ba64a0 0x556403ba8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:54.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.067+0000 7f65c89fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556403bafb30 0x556403bc9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:54.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.067+0000 7f65c89fd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f65c01a9430 con 0x556403ba64a0 2024-05-22T23:08:54.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.067+0000 7f65c89fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f65c01abc90 con 0x556403ba4db0 2024-05-22T23:08:54.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.067+0000 7f65c89fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f65c01a9a80 con 0x556403bafb30 2024-05-22T23:08:54.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.067+0000 7f65beffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403ba64a0 0x556403ba8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:54.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.068+0000 7f65beffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403ba64a0 0x556403ba8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49620/0 (socket says 172.21.15.84:49620) 2024-05-22T23:08:54.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.068+0000 7f65beffd640 1 -- 172.21.15.84:0/4255840112 learned_addr learned my addr 172.21.15.84:0/4255840112 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:54.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.068+0000 7f65bf7fe640 1 --2- 172.21.15.84:0/4255840112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556403ba4db0 0x556403bbd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:54.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.068+0000 7f65bffff640 1 --2- 172.21.15.84:0/4255840112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556403bafb30 0x556403bc9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:54.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.068+0000 7f65beffd640 1 -- 172.21.15.84:0/4255840112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556403bafb30 msgr2=0x556403bc9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:54.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.068+0000 7f65beffd640 1 --2- 172.21.15.84:0/4255840112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556403bafb30 0x556403bc9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.068+0000 7f65beffd640 1 -- 172.21.15.84:0/4255840112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556403ba4db0 msgr2=0x556403bbd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:54.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.068+0000 7f65beffd640 1 --2- 172.21.15.84:0/4255840112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556403ba4db0 0x556403bbd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.068+0000 7f65beffd640 1 -- 172.21.15.84:0/4255840112 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564037dc0f0 con 0x556403ba64a0 2024-05-22T23:08:54.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.068+0000 7f65bffff640 1 --2- 172.21.15.84:0/4255840112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556403bafb30 0x556403bc9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:08:54.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.068+0000 7f65beffd640 1 --2- 172.21.15.84:0/4255840112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403ba64a0 0x556403ba8a50 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f65b4005d00 tx=0x7f65b40efac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7dac21896cadfa server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:54.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.068+0000 7f65a57fa640 1 -- 172.21.15.84:0/4255840112 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b40bb6c0 con 0x556403ba64a0 2024-05-22T23:08:54.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.068+0000 7f65a57fa640 1 -- 172.21.15.84:0/4255840112 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b40bc240 con 0x556403ba64a0 2024-05-22T23:08:54.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.068+0000 7f65a57fa640 1 -- 172.21.15.84:0/4255840112 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b40b7bf0 con 0x556403ba64a0 2024-05-22T23:08:54.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.069+0000 7f65c89fd740 1 -- 172.21.15.84:0/4255840112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403ba64a0 msgr2=0x556403ba8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:54.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.069+0000 7f65c89fd740 1 --2- 172.21.15.84:0/4255840112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403ba64a0 0x556403ba8a50 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f65b4005d00 tx=0x7f65b40efac0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.069+0000 7f65c89fd740 1 -- 172.21.15.84:0/4255840112 shutdown_connections 2024-05-22T23:08:54.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.069+0000 7f65c89fd740 1 --2- 172.21.15.84:0/4255840112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556403bafb30 0x556403bc9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.069+0000 7f65c89fd740 1 --2- 172.21.15.84:0/4255840112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403ba64a0 0x556403ba8a50 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.069+0000 7f65c89fd740 1 --2- 172.21.15.84:0/4255840112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556403ba4db0 0x556403bbd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.069+0000 7f65c89fd740 1 -- 172.21.15.84:0/4255840112 >> 172.21.15.84:0/4255840112 conn(0x556403b6d8e0 msgr2=0x556403ba0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:54.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.069+0000 7f65c89fd740 1 -- 172.21.15.84:0/4255840112 shutdown_connections 2024-05-22T23:08:54.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.069+0000 7f65c89fd740 1 -- 172.21.15.84:0/4255840112 wait complete. 2024-05-22T23:08:54.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.069+0000 7f65c89fd740 1 -- start start 2024-05-22T23:08:54.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.069+0000 7f65c89fd740 1 Processor -- start 2024-05-22T23:08:54.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.070+0000 7f65c89fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556403ba4db0 0x556403bb2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:54.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.070+0000 7f65c89fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556403ba64a0 0x556403bb3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:54.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.070+0000 7f65c89fd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403bafb30 0x556403bb35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:54.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.070+0000 7f65c89fd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f65c01aa150 con 0x556403bafb30 2024-05-22T23:08:54.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.070+0000 7f65c89fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f65c01ab5c0 con 0x556403ba64a0 2024-05-22T23:08:54.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.070+0000 7f65c89fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f65c01aa820 con 0x556403ba4db0 2024-05-22T23:08:54.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.070+0000 7f65c89fd740 10 client.0 _ll_register_callbacks cb 0x7f65b8009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:08:54.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.070+0000 7f65bffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403bafb30 0x556403bb35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:54.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.070+0000 7f65beffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556403ba64a0 0x556403bb3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:54.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.070+0000 7f65beffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556403ba64a0 0x556403bb3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:36044/0 (socket says 172.21.15.84:36044) 2024-05-22T23:08:54.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.070+0000 7f65bffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403bafb30 0x556403bb35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49626/0 (socket says 172.21.15.84:49626) 2024-05-22T23:08:54.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.070+0000 7f65beffd640 1 -- 172.21.15.84:0/1512314814 learned_addr learned my addr 172.21.15.84:0/1512314814 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:08:54.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.070+0000 7f65bf7fe640 1 --2- 172.21.15.84:0/1512314814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556403ba4db0 0x556403bb2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:54.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.070+0000 7f65beffd640 1 -- 172.21.15.84:0/1512314814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556403ba4db0 msgr2=0x556403bb2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:54.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.070+0000 7f65beffd640 1 --2- 172.21.15.84:0/1512314814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556403ba4db0 0x556403bb2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.070+0000 7f65beffd640 1 -- 172.21.15.84:0/1512314814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403bafb30 msgr2=0x556403bb35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:08:54.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.070+0000 7f65beffd640 1 --2- 172.21.15.84:0/1512314814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403bafb30 0x556403bb35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.070+0000 7f65beffd640 1 -- 172.21.15.84:0/1512314814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556403ac7fe0 con 0x556403ba64a0 2024-05-22T23:08:54.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.070+0000 7f65beffd640 1 --2- 172.21.15.84:0/1512314814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556403ba64a0 0x556403bb3050 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f65b40b8730 tx=0x7f65b40ba270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2022055c329444da server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:08:54.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.070+0000 7f659effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:08:54.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.070+0000 7f659effd640 1 -- 172.21.15.84:0/1512314814 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b40bb6c0 con 0x556403ba64a0 2024-05-22T23:08:54.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.070+0000 7f65c89fd740 1 -- 172.21.15.84:0/1512314814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b40bc240 con 0x556403ba64a0 2024-05-22T23:08:54.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.070+0000 7f659effd640 1 -- 172.21.15.84:0/1512314814 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f65b40f5180 con 0x556403ba64a0 2024-05-22T23:08:54.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.071+0000 7f659effd640 1 -- 172.21.15.84:0/1512314814 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f65b40bbd60 con 0x556403ba64a0 2024-05-22T23:08:54.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.071+0000 7f65c89fd740 10 client.26129 fetch_fsmap learned FSMap version 18 2024-05-22T23:08:54.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.071+0000 7f65c89fd740 1 -- 172.21.15.84:0/1512314814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x556403a991c0 con 0x556403ba64a0 2024-05-22T23:08:54.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.071+0000 7f659effd640 1 -- 172.21.15.84:0/1512314814 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b40bc240 con 0x556403ba64a0 2024-05-22T23:08:54.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.071+0000 7f659effd640 1 -- 172.21.15.84:0/1512314814 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f65b40a2030 con 0x556403ba64a0 2024-05-22T23:08:54.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.071+0000 7f65c89fd740 10 client.26129 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:08:54.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.071+0000 7f65c89fd740 10 client.26129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:08:54.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.071+0000 7f65c89fd740 20 client.26129 populate_metadata read hostname 'smithi084' 2024-05-22T23:08:54.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.071+0000 7f65c89fd740 1 --2- 172.21.15.84:0/1512314814 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556403c27550 0x556403c29940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:08:54.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.071+0000 7f65c89fd740 4 client.26129 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:08:54.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.071+0000 7f65c89fd740 1 -- 172.21.15.84:0/1512314814 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556403b10830 con 0x556403c27550 2024-05-22T23:08:54.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.071+0000 7f65bf7fe640 1 --2- 172.21.15.84:0/1512314814 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556403c27550 0x556403c29940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:08:54.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.071+0000 7f659effd640 10 client.26129 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:54.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.071+0000 7f65bf7fe640 1 --2- 172.21.15.84:0/1512314814 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556403c27550 0x556403c29940 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4fb7a6d805a32fc server_cookie=8e44379f0830ce6f in_seq=0 out_seq=0 2024-05-22T23:08:54.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.072+0000 7f659effd640 1 -- 172.21.15.84:0/1512314814 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556403b10830 con 0x556403c27550 2024-05-22T23:08:54.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.072+0000 7f659effd640 10 client.26129 handle_command_reply: tid=0 2024-05-22T23:08:54.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.072+0000 7f65c89fd740 1 client.26129 shutdown 2024-05-22T23:08:54.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.072+0000 7f65c89fd740 1 -- 172.21.15.84:0/1512314814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556403ba64a0 msgr2=0x556403bb3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:54.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.072+0000 7f65c89fd740 1 --2- 172.21.15.84:0/1512314814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556403ba64a0 0x556403bb3050 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f65b40b8730 tx=0x7f65b40ba270 comp rx=0 tx=0).stop 2024-05-22T23:08:54.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.072+0000 7f65c89fd740 1 -- 172.21.15.84:0/1512314814 shutdown_connections 2024-05-22T23:08:54.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.072+0000 7f65c89fd740 1 --2- 172.21.15.84:0/1512314814 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556403c27550 0x556403c29940 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.072+0000 7f65c89fd740 1 --2- 172.21.15.84:0/1512314814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556403bafb30 0x556403bb35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.072+0000 7f65c89fd740 1 --2- 172.21.15.84:0/1512314814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556403ba64a0 0x556403bb3050 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.072+0000 7f65c89fd740 1 --2- 172.21.15.84:0/1512314814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556403ba4db0 0x556403bb2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.072+0000 7f659effd640 0 client.26129 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:08:54.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.072+0000 7f65c89fd740 1 -- 172.21.15.84:0/1512314814 >> 172.21.15.84:0/1512314814 conn(0x556403b6d8e0 msgr2=0x556403bbc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:54.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.072+0000 7f65c89fd740 1 -- 172.21.15.84:0/1512314814 shutdown_connections 2024-05-22T23:08:54.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.072+0000 7f65c89fd740 1 -- 172.21.15.84:0/1512314814 wait complete. 2024-05-22T23:08:54.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.072+0000 7f65c89fd740 20 client.26129 trim_cache size 0 max 16384 2024-05-22T23:08:54.085 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:08:54.085 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-22T23:08:54.085 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": { 2024-05-22T23:08:54.085 INFO:teuthology.orchestra.run.smithi084.stdout: "eb8deda0-6ecc-44d4-97e8-466c256d9d5d": { 2024-05-22T23:08:54.085 INFO:teuthology.orchestra.run.smithi084.stdout: "path": "/", 2024-05-22T23:08:54.085 INFO:teuthology.orchestra.run.smithi084.stdout: "tag": "eb8deda0-6ecc-44d4-97e8-466c256d9d5d", 2024-05-22T23:08:54.085 INFO:teuthology.orchestra.run.smithi084.stdout: "options": "recursive,force" 2024-05-22T23:08:54.085 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:08:54.085 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:08:54.085 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:08:54.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.072+0000 7f65c5c1b640 1 -- 172.21.15.84:0/1755013401 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f65a80a2d00 msgr2=0x7f65a80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:54.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.072+0000 7f65c5c1b640 1 --2- 172.21.15.84:0/1755013401 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f65a80a2d00 0x7f65a80a51c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009fd0 tx=0x7f65b8004a40 comp rx=0 tx=0).stop 2024-05-22T23:08:54.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.072+0000 7f65c5c1b640 1 -- 172.21.15.84:0/1755013401 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65c0101460 msgr2=0x7f65c01974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:08:54.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.072+0000 7f65c5c1b640 1 --2- 172.21.15.84:0/1755013401 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65c0101460 0x7f65c01974f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f65b4006010 tx=0x7f65b4004040 comp rx=0 tx=0).stop 2024-05-22T23:08:54.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.072+0000 7f65c5c1b640 1 -- 172.21.15.84:0/1755013401 shutdown_connections 2024-05-22T23:08:54.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.072+0000 7f65c5c1b640 1 --2- 172.21.15.84:0/1755013401 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f65a80a2d00 0x7f65a80a51c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.072+0000 7f65c5c1b640 1 --2- 172.21.15.84:0/1755013401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65c01067c0 0x7f65c019b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.072+0000 7f65c5c1b640 1 --2- 172.21.15.84:0/1755013401 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65c0101460 0x7f65c01974f0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.073+0000 7f65c5c1b640 1 --2- 172.21.15.84:0/1755013401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65c0100ac0 0x7f65c0196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:08:54.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.073+0000 7f65c5c1b640 1 -- 172.21.15.84:0/1755013401 >> 172.21.15.84:0/1755013401 conn(0x7f65c0076570 msgr2=0x7f65c00ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:08:54.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.073+0000 7f65c5c1b640 1 -- 172.21.15.84:0/1755013401 shutdown_connections 2024-05-22T23:08:54.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:08:54.073+0000 7f65c5c1b640 1 -- 172.21.15.84:0/1755013401 wait complete. 2024-05-22T23:08:54.088 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:53 smithi084 ceph-mon[32186]: pgmap v216: 97 pgs: 97 active+clean; 461 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.3 MiB/s wr, 346 op/s 2024-05-22T23:08:54.090 INFO:tasks.cephfs.filesystem:scrub status for tag:eb8deda0-6ecc-44d4-97e8-466c256d9d5d - {'path': '/', 'tag': 'eb8deda0-6ecc-44d4-97e8-466c256d9d5d', 'options': 'recursive,force'} 2024-05-22T23:08:54.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:53 smithi204 ceph-mon[36211]: pgmap v216: 97 pgs: 97 active+clean; 461 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.3 MiB/s wr, 346 op/s 2024-05-22T23:08:54.441 INFO:tasks.workunit.client.0.smithi084.stdout: 1 954693 14.92 MB/sec execute 220 sec latency 19.408 ms 2024-05-22T23:08:55.026 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:54 smithi096 ceph-mon[36392]: scrub queued for path: / 2024-05-22T23:08:55.026 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:54 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:08:55.027 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:54 smithi096 ceph-mon[36392]: scrub summary: active paths [/] 2024-05-22T23:08:55.027 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:54 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:08:55.066 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:54 smithi204 ceph-mon[36211]: scrub queued for path: / 2024-05-22T23:08:55.066 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:54 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:08:55.066 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:54 smithi204 ceph-mon[36211]: scrub summary: active paths [/] 2024-05-22T23:08:55.066 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:54 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:08:55.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:54 smithi084 ceph-mon[32186]: scrub queued for path: / 2024-05-22T23:08:55.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:54 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:08:55.130 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:54 smithi084 ceph-mon[32186]: scrub summary: active paths [/] 2024-05-22T23:08:55.130 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:54 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:08:55.441 INFO:tasks.workunit.client.0.smithi084.stdout: 1 957338 14.92 MB/sec execute 221 sec latency 15.682 ms 2024-05-22T23:08:56.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:55 smithi096 ceph-mon[36392]: pgmap v217: 97 pgs: 97 active+clean; 461 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 223 op/s 2024-05-22T23:08:56.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:55 smithi084 ceph-mon[32186]: pgmap v217: 97 pgs: 97 active+clean; 461 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 223 op/s 2024-05-22T23:08:56.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:55 smithi204 ceph-mon[36211]: pgmap v217: 97 pgs: 97 active+clean; 461 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 223 op/s 2024-05-22T23:08:56.441 INFO:tasks.workunit.client.0.smithi084.stdout: 1 959927 14.92 MB/sec execute 222 sec latency 15.043 ms 2024-05-22T23:08:57.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:56 smithi096 ceph-mon[36392]: scrub summary: idle 2024-05-22T23:08:57.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:56 smithi084 ceph-mon[32186]: scrub summary: idle 2024-05-22T23:08:57.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:56 smithi204 ceph-mon[36211]: scrub summary: idle 2024-05-22T23:08:57.441 INFO:tasks.workunit.client.0.smithi084.stdout: 1 962307 14.91 MB/sec execute 223 sec latency 48.754 ms 2024-05-22T23:08:58.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:57 smithi096 ceph-mon[36392]: pgmap v218: 97 pgs: 97 active+clean; 471 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.9 MiB/s wr, 325 op/s 2024-05-22T23:08:58.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:57 smithi084 ceph-mon[32186]: pgmap v218: 97 pgs: 97 active+clean; 471 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.9 MiB/s wr, 325 op/s 2024-05-22T23:08:58.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:57 smithi204 ceph-mon[36211]: pgmap v218: 97 pgs: 97 active+clean; 471 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.9 MiB/s wr, 325 op/s 2024-05-22T23:08:58.441 INFO:tasks.workunit.client.0.smithi084.stdout: 1 964270 14.89 MB/sec execute 224 sec latency 69.105 ms 2024-05-22T23:08:59.030 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:08:58 smithi084 ceph-mon[32186]: pgmap v219: 97 pgs: 97 active+clean; 471 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 209 op/s 2024-05-22T23:08:59.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:08:58 smithi096 ceph-mon[36392]: pgmap v219: 97 pgs: 97 active+clean; 471 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 209 op/s 2024-05-22T23:08:59.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:08:58 smithi204 ceph-mon[36211]: pgmap v219: 97 pgs: 97 active+clean; 471 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 209 op/s 2024-05-22T23:08:59.441 INFO:tasks.workunit.client.0.smithi084.stdout: 1 966925 14.89 MB/sec execute 225 sec latency 15.136 ms 2024-05-22T23:09:00.442 INFO:tasks.workunit.client.0.smithi084.stdout: 1 969753 14.89 MB/sec execute 226 sec latency 14.230 ms 2024-05-22T23:09:01.304 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:01 smithi096 ceph-mon[36392]: pgmap v220: 97 pgs: 97 active+clean; 471 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 209 op/s 2024-05-22T23:09:01.379 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:01 smithi084 ceph-mon[32186]: pgmap v220: 97 pgs: 97 active+clean; 471 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 209 op/s 2024-05-22T23:09:01.415 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:01 smithi204 ceph-mon[36211]: pgmap v220: 97 pgs: 97 active+clean; 471 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 209 op/s 2024-05-22T23:09:01.442 INFO:tasks.workunit.client.0.smithi084.stdout: 1 971183 14.85 MB/sec execute 227 sec latency 25.834 ms 2024-05-22T23:09:01.847 INFO:tasks.workunit.client.0.smithi084.stdout:[972188] write failed on handle 13332 (Resource temporarily unavailable) 2024-05-22T23:09:01.847 INFO:tasks.workunit.client.0.smithi084.stdout:Child failed with status 1 2024-05-22T23:09:01.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T23:09:01.848 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-05-22T23:09:01.849 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-22T23:09:02.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:02 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:09:02.146 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_935217f1cb4dc722d5303f5ad364c883e9133376/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_935217f1cb4dc722d5303f5ad364c883e9133376/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_935217f1cb4dc722d5303f5ad364c883e9133376/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi084 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=935217f1cb4dc722d5303f5ad364c883e9133376 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-05-22T23:09:02.391 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=4254f8882f53441ba212b9835c8b500d Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_935217f1cb4dc722d5303f5ad364c883e9133376/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_935217f1cb4dc722d5303f5ad364c883e9133376/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_935217f1cb4dc722d5303f5ad364c883e9133376/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi084 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=935217f1cb4dc722d5303f5ad364c883e9133376 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-05-22T23:09:02.393 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-22T23:09:02.408 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-22T23:09:02.409 INFO:teuthology.task.exec:Executing custom commands... 2024-05-22T23:09:02.409 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi084.front.sepia.ceph.com 2024-05-22T23:09:02.409 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-22T23:09:02.414 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:02 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:09:02.423 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:02 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:09:02.440 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-22T23:09:03.304 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:03 smithi096 ceph-mon[36392]: pgmap v221: 97 pgs: 97 active+clean; 440 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.8 MiB/s wr, 314 op/s 2024-05-22T23:09:03.304 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:03 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:09:03.304 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:03 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:09:03.305 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:03 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:09:03.379 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:03 smithi084 ceph-mon[32186]: pgmap v221: 97 pgs: 97 active+clean; 440 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.8 MiB/s wr, 314 op/s 2024-05-22T23:09:03.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:03 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:09:03.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:03 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:09:03.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:03 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:09:03.415 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:03 smithi204 ceph-mon[36211]: pgmap v221: 97 pgs: 97 active+clean; 440 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.8 MiB/s wr, 314 op/s 2024-05-22T23:09:03.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:03 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:09:03.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:03 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:09:03.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:03 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:09:05.304 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:05 smithi096 ceph-mon[36392]: pgmap v222: 97 pgs: 97 active+clean; 440 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 206 op/s 2024-05-22T23:09:05.379 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:05 smithi084 ceph-mon[32186]: pgmap v222: 97 pgs: 97 active+clean; 440 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 206 op/s 2024-05-22T23:09:05.415 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:05 smithi204 ceph-mon[36211]: pgmap v222: 97 pgs: 97 active+clean; 440 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 206 op/s 2024-05-22T23:09:07.379 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:07 smithi084 ceph-mon[32186]: pgmap v223: 97 pgs: 97 active+clean; 453 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 272 op/s 2024-05-22T23:09:07.379 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:07 smithi084 ceph-mon[32186]: osdmap e92: 12 total, 12 up, 12 in 2024-05-22T23:09:07.415 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:07 smithi204 ceph-mon[36211]: pgmap v223: 97 pgs: 97 active+clean; 453 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 272 op/s 2024-05-22T23:09:07.415 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:07 smithi204 ceph-mon[36211]: osdmap e92: 12 total, 12 up, 12 in 2024-05-22T23:09:07.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:07 smithi096 ceph-mon[36392]: pgmap v223: 97 pgs: 97 active+clean; 453 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 272 op/s 2024-05-22T23:09:07.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:07 smithi096 ceph-mon[36392]: osdmap e92: 12 total, 12 up, 12 in 2024-05-22T23:09:09.379 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:09 smithi084 ceph-mon[32186]: pgmap v225: 97 pgs: 97 active+clean; 453 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 204 op/s 2024-05-22T23:09:09.415 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:09 smithi204 ceph-mon[36211]: pgmap v225: 97 pgs: 97 active+clean; 453 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 204 op/s 2024-05-22T23:09:09.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:09 smithi096 ceph-mon[36392]: pgmap v225: 97 pgs: 97 active+clean; 453 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 204 op/s 2024-05-22T23:09:11.379 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:11 smithi084 ceph-mon[32186]: pgmap v226: 97 pgs: 97 active+clean; 453 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 204 op/s 2024-05-22T23:09:11.415 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:11 smithi204 ceph-mon[36211]: pgmap v226: 97 pgs: 97 active+clean; 453 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 204 op/s 2024-05-22T23:09:11.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:11 smithi096 ceph-mon[36392]: pgmap v226: 97 pgs: 97 active+clean; 453 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 204 op/s 2024-05-22T23:09:13.379 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:13 smithi084 ceph-mon[32186]: pgmap v227: 97 pgs: 97 active+clean; 446 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 2.7 MiB/s wr, 132 op/s 2024-05-22T23:09:13.415 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:13 smithi204 ceph-mon[36211]: pgmap v227: 97 pgs: 97 active+clean; 446 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 2.7 MiB/s wr, 132 op/s 2024-05-22T23:09:13.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:13 smithi096 ceph-mon[36392]: pgmap v227: 97 pgs: 97 active+clean; 446 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 2.7 MiB/s wr, 132 op/s 2024-05-22T23:09:15.379 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:15 smithi084 ceph-mon[32186]: pgmap v228: 97 pgs: 97 active+clean; 446 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 2.7 MiB/s wr, 132 op/s 2024-05-22T23:09:15.415 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:15 smithi204 ceph-mon[36211]: pgmap v228: 97 pgs: 97 active+clean; 446 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 2.7 MiB/s wr, 132 op/s 2024-05-22T23:09:15.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:15 smithi096 ceph-mon[36392]: pgmap v228: 97 pgs: 97 active+clean; 446 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 2.7 MiB/s wr, 132 op/s 2024-05-22T23:09:17.379 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:17 smithi084 ceph-mon[32186]: pgmap v229: 97 pgs: 97 active+clean; 469 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 4.3 MiB/s wr, 96 op/s 2024-05-22T23:09:17.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:17 smithi084 ceph-mon[32186]: osdmap e93: 12 total, 12 up, 12 in 2024-05-22T23:09:17.415 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:17 smithi204 ceph-mon[36211]: pgmap v229: 97 pgs: 97 active+clean; 469 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 4.3 MiB/s wr, 96 op/s 2024-05-22T23:09:17.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:17 smithi204 ceph-mon[36211]: osdmap e93: 12 total, 12 up, 12 in 2024-05-22T23:09:17.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:17 smithi096 ceph-mon[36392]: pgmap v229: 97 pgs: 97 active+clean; 469 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 4.3 MiB/s wr, 96 op/s 2024-05-22T23:09:17.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:17 smithi096 ceph-mon[36392]: osdmap e93: 12 total, 12 up, 12 in 2024-05-22T23:09:19.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:19 smithi084 ceph-mon[32186]: pgmap v231: 97 pgs: 97 active+clean; 469 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 4.3 MiB/s wr, 96 op/s 2024-05-22T23:09:19.415 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:19 smithi204 ceph-mon[36211]: pgmap v231: 97 pgs: 97 active+clean; 469 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 4.3 MiB/s wr, 96 op/s 2024-05-22T23:09:19.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:19 smithi096 ceph-mon[36392]: pgmap v231: 97 pgs: 97 active+clean; 469 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 4.3 MiB/s wr, 96 op/s 2024-05-22T23:09:21.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:21 smithi084 ceph-mon[32186]: pgmap v232: 97 pgs: 97 active+clean; 469 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 4.3 MiB/s wr, 96 op/s 2024-05-22T23:09:21.415 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:21 smithi204 ceph-mon[36211]: pgmap v232: 97 pgs: 97 active+clean; 469 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 4.3 MiB/s wr, 96 op/s 2024-05-22T23:09:21.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:21 smithi096 ceph-mon[36392]: pgmap v232: 97 pgs: 97 active+clean; 469 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 4.3 MiB/s wr, 96 op/s 2024-05-22T23:09:23.379 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:23 smithi084 ceph-mon[32186]: pgmap v233: 97 pgs: 97 active+clean; 466 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 4.1 MiB/s wr, 81 op/s 2024-05-22T23:09:23.415 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:23 smithi204 ceph-mon[36211]: pgmap v233: 97 pgs: 97 active+clean; 466 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 4.1 MiB/s wr, 81 op/s 2024-05-22T23:09:23.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:23 smithi096 ceph-mon[36392]: pgmap v233: 97 pgs: 97 active+clean; 466 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 4.1 MiB/s wr, 81 op/s 2024-05-22T23:09:24.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:09:24.091 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:09:24.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.200+0000 7f13d9590640 1 -- 172.21.15.84:0/4044987069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d41067c0 msgr2=0x7f13d4106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:24.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.200+0000 7f13d9590640 1 --2- 172.21.15.84:0/4044987069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d41067c0 0x7f13d4106ba0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f13c4009900 tx=0x7f13c402a410 comp rx=0 tx=0).stop 2024-05-22T23:09:24.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.200+0000 7f13d9590640 1 -- 172.21.15.84:0/4044987069 shutdown_connections 2024-05-22T23:09:24.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.200+0000 7f13d9590640 1 --2- 172.21.15.84:0/4044987069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13d4101460 0x7f13d410d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.200+0000 7f13d9590640 1 --2- 172.21.15.84:0/4044987069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13d4100ac0 0x7f13d4100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.200+0000 7f13d9590640 1 --2- 172.21.15.84:0/4044987069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d41067c0 0x7f13d4106ba0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.200+0000 7f13d9590640 1 -- 172.21.15.84:0/4044987069 >> 172.21.15.84:0/4044987069 conn(0x7f13d4076570 msgr2=0x7f13d4076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:24.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.200+0000 7f13d9590640 1 -- 172.21.15.84:0/4044987069 shutdown_connections 2024-05-22T23:09:24.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.200+0000 7f13d9590640 1 -- 172.21.15.84:0/4044987069 wait complete. 2024-05-22T23:09:24.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.201+0000 7f13d9590640 1 Processor -- start 2024-05-22T23:09:24.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.201+0000 7f13d9590640 1 -- start start 2024-05-22T23:09:24.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.201+0000 7f13d9590640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13d4100ac0 0x7f13d4198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.201+0000 7f13d9590640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13d4101460 0x7f13d4199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.201+0000 7f13d9590640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d41067c0 0x7f13d41a3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.201+0000 7f13d9590640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f13d4110b10 con 0x7f13d41067c0 2024-05-22T23:09:24.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.201+0000 7f13d9590640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13d4110990 con 0x7f13d4100ac0 2024-05-22T23:09:24.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.201+0000 7f13d9590640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f13d4110c90 con 0x7f13d4101460 2024-05-22T23:09:24.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.201+0000 7f13d37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d41067c0 0x7f13d41a3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.201+0000 7f13d37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d41067c0 0x7f13d41a3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41902/0 (socket says 172.21.15.84:41902) 2024-05-22T23:09:24.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.201+0000 7f13d37fe640 1 -- 172.21.15.84:0/1479892648 learned_addr learned my addr 172.21.15.84:0/1479892648 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:24.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.201+0000 7f13d2ffd640 1 --2- 172.21.15.84:0/1479892648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13d4100ac0 0x7f13d4198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.201+0000 7f13d27fc640 1 --2- 172.21.15.84:0/1479892648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13d4101460 0x7f13d4199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.201+0000 7f13d37fe640 1 -- 172.21.15.84:0/1479892648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13d4101460 msgr2=0x7f13d4199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:24.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.201+0000 7f13d37fe640 1 --2- 172.21.15.84:0/1479892648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13d4101460 0x7f13d4199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.201+0000 7f13d37fe640 1 -- 172.21.15.84:0/1479892648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13d4100ac0 msgr2=0x7f13d4198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:24.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.201+0000 7f13d37fe640 1 --2- 172.21.15.84:0/1479892648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13d4100ac0 0x7f13d4198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.202+0000 7f13d37fe640 1 -- 172.21.15.84:0/1479892648 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d41a4490 con 0x7f13d41067c0 2024-05-22T23:09:24.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.202+0000 7f13d37fe640 1 --2- 172.21.15.84:0/1479892648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d41067c0 0x7f13d41a3f50 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00eaa0 tx=0x7f13cc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:24.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.202+0000 7f13bbfff640 1 -- 172.21.15.84:0/1479892648 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc00cdb0 con 0x7f13d41067c0 2024-05-22T23:09:24.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.202+0000 7f13d9590640 1 -- 172.21.15.84:0/1479892648 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d41a4660 con 0x7f13d41067c0 2024-05-22T23:09:24.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.202+0000 7f13bbfff640 1 -- 172.21.15.84:0/1479892648 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13cc005020 con 0x7f13d41067c0 2024-05-22T23:09:24.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.202+0000 7f13bbfff640 1 -- 172.21.15.84:0/1479892648 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc0116d0 con 0x7f13d41067c0 2024-05-22T23:09:24.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.202+0000 7f13d9590640 1 -- 172.21.15.84:0/1479892648 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d41a4b80 con 0x7f13d41067c0 2024-05-22T23:09:24.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.204+0000 7f13bbfff640 1 -- 172.21.15.84:0/1479892648 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f13cc0054d0 con 0x7f13d41067c0 2024-05-22T23:09:24.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.204+0000 7f13dc37b740 1 Processor -- start 2024-05-22T23:09:24.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.204+0000 7f13dc37b740 1 -- start start 2024-05-22T23:09:24.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.205+0000 7f13dc37b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cdf34c44a0 0x55cdf34c27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.205+0000 7f13dc37b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdf34c2db0 0x55cdf34c3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.205+0000 7f13dc37b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34cdb30 0x55cdf34cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.205+0000 7f13dc37b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55cdf3426db0 con 0x55cdf34cdb30 2024-05-22T23:09:24.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.205+0000 7f13dc37b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cdf33c5360 con 0x55cdf34c2db0 2024-05-22T23:09:24.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.205+0000 7f13dc37b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cdf3422d70 con 0x55cdf34c44a0 2024-05-22T23:09:24.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.205+0000 7f13d27fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34cdb30 0x55cdf34cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.205+0000 7f13d27fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34cdb30 0x55cdf34cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41906/0 (socket says 172.21.15.84:41906) 2024-05-22T23:09:24.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.205+0000 7f13d27fc640 1 -- 172.21.15.84:0/2740399293 learned_addr learned my addr 172.21.15.84:0/2740399293 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:24.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.205+0000 7f13bbfff640 1 --2- 172.21.15.84:0/1479892648 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f13bc0a2d00 0x7f13bc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.205+0000 7f13d27fc640 1 --2- 172.21.15.84:0/2740399293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cdf34c44a0 0x55cdf34c27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.205+0000 7f13d27fc640 1 -- 172.21.15.84:0/2740399293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cdf34c44a0 msgr2=0x55cdf34c27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:24.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.205+0000 7f13d27fc640 1 --2- 172.21.15.84:0/2740399293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cdf34c44a0 0x55cdf34c27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.205+0000 7f13d2ffd640 1 --2- 172.21.15.84:0/2740399293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdf34c2db0 0x55cdf34c3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.205+0000 7f13d27fc640 1 -- 172.21.15.84:0/2740399293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdf34c2db0 msgr2=0x55cdf34c3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:24.218 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.205+0000 7f13d27fc640 1 --2- 172.21.15.84:0/2740399293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdf34c2db0 0x55cdf34c3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.218 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.205+0000 7f13d27fc640 1 -- 172.21.15.84:0/2740399293 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdf33e8a60 con 0x55cdf34cdb30 2024-05-22T23:09:24.218 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.205+0000 7f13d37fe640 1 --2- 172.21.15.84:0/1479892648 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f13bc0a2d00 0x7f13bc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.218 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.205+0000 7f13d27fc640 1 --2- 172.21.15.84:0/2740399293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34cdb30 0x55cdf34cff20 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f13c800ae60 tx=0x7f13c800a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8ac5ac2d47ed83 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:24.218 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.205+0000 7f13bbfff640 1 -- 172.21.15.84:0/1479892648 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7f13cc0cf5b0 con 0x7f13d41067c0 2024-05-22T23:09:24.218 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.205+0000 7f13b97fa640 1 -- 172.21.15.84:0/2740399293 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c8005220 con 0x55cdf34cdb30 2024-05-22T23:09:24.218 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.205+0000 7f13b97fa640 1 -- 172.21.15.84:0/2740399293 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13c8017030 con 0x55cdf34cdb30 2024-05-22T23:09:24.218 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.205+0000 7f13b97fa640 1 -- 172.21.15.84:0/2740399293 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c800f690 con 0x55cdf34cdb30 2024-05-22T23:09:24.218 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.206+0000 7f13dc37b740 1 -- 172.21.15.84:0/2740399293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34cdb30 msgr2=0x55cdf34cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:24.219 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.206+0000 7f13d37fe640 1 --2- 172.21.15.84:0/1479892648 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f13bc0a2d00 0x7f13bc0a51c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f13cc098520 tx=0x7f13cc06aa80 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:24.219 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.206+0000 7f13dc37b740 1 --2- 172.21.15.84:0/2740399293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34cdb30 0x55cdf34cff20 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f13c800ae60 tx=0x7f13c800a750 comp rx=0 tx=0).stop 2024-05-22T23:09:24.219 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.206+0000 7f13dc37b740 1 -- 172.21.15.84:0/2740399293 shutdown_connections 2024-05-22T23:09:24.219 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.206+0000 7f13dc37b740 1 --2- 172.21.15.84:0/2740399293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34cdb30 0x55cdf34cff20 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.219 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.206+0000 7f13dc37b740 1 --2- 172.21.15.84:0/2740399293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdf34c2db0 0x55cdf34c3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.219 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.206+0000 7f13dc37b740 1 --2- 172.21.15.84:0/2740399293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cdf34c44a0 0x55cdf34c27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.219 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.206+0000 7f13dc37b740 1 -- 172.21.15.84:0/2740399293 >> 172.21.15.84:0/2740399293 conn(0x55cdf348b8e0 msgr2=0x55cdf34bf310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:24.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.206+0000 7f13dc37b740 1 -- 172.21.15.84:0/2740399293 shutdown_connections 2024-05-22T23:09:24.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.206+0000 7f13dc37b740 1 -- 172.21.15.84:0/2740399293 wait complete. 2024-05-22T23:09:24.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.206+0000 7f13dc37b740 1 -- start start 2024-05-22T23:09:24.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.207+0000 7f13dc37b740 1 Processor -- start 2024-05-22T23:09:24.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.207+0000 7f13dc37b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cdf34c2db0 0x55cdf34caa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.207+0000 7f13dc37b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34c44a0 0x55cdf34caf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.207+0000 7f13dc37b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdf34cdb30 0x55cdf34cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.207+0000 7f13dc37b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55cdf34d09c0 con 0x55cdf34c44a0 2024-05-22T23:09:24.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.207+0000 7f13dc37b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cdf33e10f0 con 0x55cdf34cdb30 2024-05-22T23:09:24.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.207+0000 7f13dc37b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cdf34d0b40 con 0x55cdf34c2db0 2024-05-22T23:09:24.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.207+0000 7f13d2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34c44a0 0x55cdf34caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.225 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.207+0000 7f13dc37b740 10 client.0 _ll_register_callbacks cb 0x55cdf33e5f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:09:24.225 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.207+0000 7f13d2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34c44a0 0x55cdf34caf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41916/0 (socket says 172.21.15.84:41916) 2024-05-22T23:09:24.225 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.207+0000 7f13d2ffd640 1 -- 172.21.15.84:0/2521689215 learned_addr learned my addr 172.21.15.84:0/2521689215 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:24.225 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.207+0000 7f13d27fc640 1 --2- 172.21.15.84:0/2521689215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cdf34c2db0 0x55cdf34caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.225 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.207+0000 7f13d27fc640 1 --2- 172.21.15.84:0/2521689215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdf34cdb30 0x55cdf34cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.225 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.207+0000 7f13d2ffd640 1 -- 172.21.15.84:0/2521689215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cdf34c2db0 msgr2=0x55cdf34caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:24.225 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.207+0000 7f13d2ffd640 1 --2- 172.21.15.84:0/2521689215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cdf34c2db0 0x55cdf34caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.226 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.207+0000 7f13d2ffd640 1 -- 172.21.15.84:0/2521689215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdf34cdb30 msgr2=0x55cdf34cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:24.226 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.207+0000 7f13d2ffd640 1 --2- 172.21.15.84:0/2521689215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdf34cdb30 0x55cdf34cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.226 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.207+0000 7f13d2ffd640 1 -- 172.21.15.84:0/2521689215 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdf32047d0 con 0x55cdf34c44a0 2024-05-22T23:09:24.226 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.207+0000 7f13d2ffd640 1 --2- 172.21.15.84:0/2521689215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34c44a0 0x55cdf34caf90 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f13c4033000 tx=0x7f13c4031900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4566873b86d7d77c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:24.226 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.207+0000 7f13b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:09:24.226 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.208+0000 7f13b6ff5640 1 -- 172.21.15.84:0/2521689215 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4035040 con 0x55cdf34c44a0 2024-05-22T23:09:24.226 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.208+0000 7f13b6ff5640 1 -- 172.21.15.84:0/2521689215 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7f13c4004460 con 0x55cdf34c44a0 2024-05-22T23:09:24.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.208+0000 7f13dc37b740 1 -- 172.21.15.84:0/2521689215 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13c8017030 con 0x55cdf34c44a0 2024-05-22T23:09:24.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.208+0000 7f13b6ff5640 1 -- 172.21.15.84:0/2521689215 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f13c4005020 con 0x55cdf34c44a0 2024-05-22T23:09:24.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.208+0000 7f13dc37b740 10 client.16572 fetch_fsmap learned FSMap version 18 2024-05-22T23:09:24.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.208+0000 7f13dc37b740 1 -- 172.21.15.84:0/2521689215 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55cdf313a3f0 con 0x55cdf34c44a0 2024-05-22T23:09:24.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.208+0000 7f13b6ff5640 1 -- 172.21.15.84:0/2521689215 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13c8017030 con 0x55cdf34c44a0 2024-05-22T23:09:24.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.208+0000 7f13b6ff5640 1 -- 172.21.15.84:0/2521689215 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f13c40035f0 con 0x55cdf34c44a0 2024-05-22T23:09:24.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.208+0000 7f13dc37b740 10 client.16572 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:09:24.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.208+0000 7f13dc37b740 10 client.16572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:09:24.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.208+0000 7f13dc37b740 20 client.16572 populate_metadata read hostname 'smithi084' 2024-05-22T23:09:24.228 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.208+0000 7f13dc37b740 1 --2- 172.21.15.84:0/2521689215 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55cdf34de080 0x55cdf34e04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.228 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.208+0000 7f13dc37b740 4 client.16572 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:09:24.228 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.208+0000 7f13dc37b740 1 -- 172.21.15.84:0/2521689215 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdf34344e0 con 0x55cdf34de080 2024-05-22T23:09:24.228 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.208+0000 7f13d27fc640 1 --2- 172.21.15.84:0/2521689215 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55cdf34de080 0x55cdf34e04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.228 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.209+0000 7f13d27fc640 1 --2- 172.21.15.84:0/2521689215 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55cdf34de080 0x55cdf34e04a0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cd446e695a52369 server_cookie=8ddd26192c6e2b49 in_seq=0 out_seq=0 2024-05-22T23:09:24.228 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.209+0000 7f13b6ff5640 10 client.16572 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:24.228 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.209+0000 7f13b6ff5640 1 -- 172.21.15.84:0/2521689215 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cdf34344e0 con 0x55cdf34de080 2024-05-22T23:09:24.228 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.209+0000 7f13b6ff5640 10 client.16572 handle_command_reply: tid=0 2024-05-22T23:09:24.228 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.209+0000 7f13dc37b740 1 client.16572 shutdown 2024-05-22T23:09:24.229 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.210+0000 7f13dc37b740 1 -- 172.21.15.84:0/2521689215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34c44a0 msgr2=0x55cdf34caf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:24.229 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.210+0000 7f13dc37b740 1 --2- 172.21.15.84:0/2521689215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34c44a0 0x55cdf34caf90 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f13c4033000 tx=0x7f13c4031900 comp rx=0 tx=0).stop 2024-05-22T23:09:24.229 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.210+0000 7f13dc37b740 1 -- 172.21.15.84:0/2521689215 shutdown_connections 2024-05-22T23:09:24.229 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.210+0000 7f13dc37b740 1 --2- 172.21.15.84:0/2521689215 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55cdf34de080 0x55cdf34e04a0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.229 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.210+0000 7f13dc37b740 1 --2- 172.21.15.84:0/2521689215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdf34cdb30 0x55cdf34cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.229 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.210+0000 7f13b6ff5640 0 client.16572 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:24.229 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.210+0000 7f13dc37b740 1 --2- 172.21.15.84:0/2521689215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34c44a0 0x55cdf34caf90 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.210+0000 7f13dc37b740 1 --2- 172.21.15.84:0/2521689215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cdf34c2db0 0x55cdf34caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.210+0000 7f13dc37b740 1 -- 172.21.15.84:0/2521689215 >> 172.21.15.84:0/2521689215 conn(0x55cdf348b8e0 msgr2=0x55cdf34c5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:24.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.210+0000 7f13dc37b740 1 -- 172.21.15.84:0/2521689215 shutdown_connections 2024-05-22T23:09:24.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.210+0000 7f13dc37b740 1 -- 172.21.15.84:0/2521689215 wait complete. 2024-05-22T23:09:24.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.210+0000 7f13dc37b740 20 client.16572 trim_cache size 0 max 16384 2024-05-22T23:09:24.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.227+0000 7f13dc37b740 1 Processor -- start 2024-05-22T23:09:24.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.227+0000 7f13dc37b740 1 -- start start 2024-05-22T23:09:24.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.227+0000 7f13dc37b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdf34c2db0 0x55cdf34cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.227+0000 7f13dc37b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34c44a0 0x55cdf34cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.227+0000 7f13dc37b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cdf34cdb30 0x55cdf34e5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.227+0000 7f13dc37b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41ab6b0 con 0x55cdf34c44a0 2024-05-22T23:09:24.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.227+0000 7f13dc37b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41adf10 con 0x55cdf34c2db0 2024-05-22T23:09:24.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.227+0000 7f13dc37b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41abd00 con 0x55cdf34cdb30 2024-05-22T23:09:24.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.227+0000 7f13d2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34c44a0 0x55cdf34cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.227+0000 7f13d2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34c44a0 0x55cdf34cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41926/0 (socket says 172.21.15.84:41926) 2024-05-22T23:09:24.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.227+0000 7f13d2ffd640 1 -- 172.21.15.84:0/2619419380 learned_addr learned my addr 172.21.15.84:0/2619419380 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:24.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.228+0000 7f13d27fc640 1 --2- 172.21.15.84:0/2619419380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdf34c2db0 0x55cdf34cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.228+0000 7f13d27fc640 1 --2- 172.21.15.84:0/2619419380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cdf34cdb30 0x55cdf34e5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.228+0000 7f13d2ffd640 1 -- 172.21.15.84:0/2619419380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cdf34cdb30 msgr2=0x55cdf34e5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:24.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.228+0000 7f13d2ffd640 1 --2- 172.21.15.84:0/2619419380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cdf34cdb30 0x55cdf34e5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.228+0000 7f13d2ffd640 1 -- 172.21.15.84:0/2619419380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdf34c2db0 msgr2=0x55cdf34cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:24.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.228+0000 7f13d2ffd640 1 --2- 172.21.15.84:0/2619419380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdf34c2db0 0x55cdf34cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.228+0000 7f13d2ffd640 1 -- 172.21.15.84:0/2619419380 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdf30fa0f0 con 0x55cdf34c44a0 2024-05-22T23:09:24.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.228+0000 7f13d27fc640 1 --2- 172.21.15.84:0/2619419380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdf34c2db0 0x55cdf34cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:09:24.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.228+0000 7f13d2ffd640 1 --2- 172.21.15.84:0/2619419380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34c44a0 0x55cdf34cc990 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f13c4005940 tx=0x7f13c402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89bea3b45ee3089 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:24.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.228+0000 7f13b8ff9640 1 -- 172.21.15.84:0/2619419380 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4050040 con 0x55cdf34c44a0 2024-05-22T23:09:24.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.228+0000 7f13b8ff9640 1 -- 172.21.15.84:0/2619419380 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13c4003e80 con 0x55cdf34c44a0 2024-05-22T23:09:24.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.228+0000 7f13b8ff9640 1 -- 172.21.15.84:0/2619419380 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4035040 con 0x55cdf34c44a0 2024-05-22T23:09:24.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.229+0000 7f13dc37b740 1 -- 172.21.15.84:0/2619419380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34c44a0 msgr2=0x55cdf34cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:24.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.229+0000 7f13dc37b740 1 --2- 172.21.15.84:0/2619419380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34c44a0 0x55cdf34cc990 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f13c4005940 tx=0x7f13c402a410 comp rx=0 tx=0).stop 2024-05-22T23:09:24.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.229+0000 7f13dc37b740 1 -- 172.21.15.84:0/2619419380 shutdown_connections 2024-05-22T23:09:24.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.229+0000 7f13dc37b740 1 --2- 172.21.15.84:0/2619419380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cdf34cdb30 0x55cdf34e5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.229+0000 7f13dc37b740 1 --2- 172.21.15.84:0/2619419380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34c44a0 0x55cdf34cc990 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.229+0000 7f13dc37b740 1 --2- 172.21.15.84:0/2619419380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdf34c2db0 0x55cdf34cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.229+0000 7f13dc37b740 1 -- 172.21.15.84:0/2619419380 >> 172.21.15.84:0/2619419380 conn(0x55cdf348b8e0 msgr2=0x55cdf34cc280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:24.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.229+0000 7f13dc37b740 1 -- 172.21.15.84:0/2619419380 shutdown_connections 2024-05-22T23:09:24.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.229+0000 7f13dc37b740 1 -- 172.21.15.84:0/2619419380 wait complete. 2024-05-22T23:09:24.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.229+0000 7f13dc37b740 1 -- start start 2024-05-22T23:09:24.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.229+0000 7f13dc37b740 1 Processor -- start 2024-05-22T23:09:24.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.229+0000 7f13dc37b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34c2db0 0x55cdf34c8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.229+0000 7f13dc37b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cdf34c44a0 0x55cdf34d5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.229+0000 7f13dc37b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdf34cdb30 0x55cdf34c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.229+0000 7f13dc37b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41ac3d0 con 0x55cdf34c2db0 2024-05-22T23:09:24.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.229+0000 7f13dc37b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41ad840 con 0x55cdf34cdb30 2024-05-22T23:09:24.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.229+0000 7f13dc37b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41acaa0 con 0x55cdf34c44a0 2024-05-22T23:09:24.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.229+0000 7f13dc37b740 10 client.0 _ll_register_callbacks cb 0x7f13c4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:09:24.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.229+0000 7f13d27fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34c2db0 0x55cdf34c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.229+0000 7f13d27fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34c2db0 0x55cdf34c8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41936/0 (socket says 172.21.15.84:41936) 2024-05-22T23:09:24.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.229+0000 7f13d27fc640 1 -- 172.21.15.84:0/738787104 learned_addr learned my addr 172.21.15.84:0/738787104 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:24.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.230+0000 7f13d2ffd640 1 --2- 172.21.15.84:0/738787104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cdf34c44a0 0x55cdf34d5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.230+0000 7f13d27fc640 1 -- 172.21.15.84:0/738787104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cdf34c44a0 msgr2=0x55cdf34d5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:24.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.230+0000 7f13d27fc640 1 --2- 172.21.15.84:0/738787104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cdf34c44a0 0x55cdf34d5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.230+0000 7f13d27fc640 1 -- 172.21.15.84:0/738787104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdf34cdb30 msgr2=0x55cdf34c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:24.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.230+0000 7f13d27fc640 1 --2- 172.21.15.84:0/738787104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdf34cdb30 0x55cdf34c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.230+0000 7f13d27fc640 1 -- 172.21.15.84:0/738787104 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdf33e5fe0 con 0x55cdf34c2db0 2024-05-22T23:09:24.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.230+0000 7f13d27fc640 1 --2- 172.21.15.84:0/738787104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34c2db0 0x55cdf34c8770 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x55cdf34c04a0 tx=0x7f13c800a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf765deb0a49cc39 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:24.246 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.230+0000 7f13b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:09:24.246 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.230+0000 7f13b6ff5640 1 -- 172.21.15.84:0/738787104 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c8025040 con 0x55cdf34c2db0 2024-05-22T23:09:24.246 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.230+0000 7f13b6ff5640 1 -- 172.21.15.84:0/738787104 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7f13c8005070 con 0x55cdf34c2db0 2024-05-22T23:09:24.246 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.230+0000 7f13dc37b740 1 -- 172.21.15.84:0/738787104 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13c4003e80 con 0x55cdf34c2db0 2024-05-22T23:09:24.246 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.230+0000 7f13b6ff5640 1 -- 172.21.15.84:0/738787104 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f13c80054d0 con 0x55cdf34c2db0 2024-05-22T23:09:24.246 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.230+0000 7f13dc37b740 10 client.16578 fetch_fsmap learned FSMap version 18 2024-05-22T23:09:24.246 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.230+0000 7f13dc37b740 1 -- 172.21.15.84:0/738787104 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55cdf33b71c0 con 0x55cdf34c2db0 2024-05-22T23:09:24.246 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.230+0000 7f13b6ff5640 1 -- 172.21.15.84:0/738787104 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13c4003e80 con 0x55cdf34c2db0 2024-05-22T23:09:24.247 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.230+0000 7f13b6ff5640 1 -- 172.21.15.84:0/738787104 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f13c802b040 con 0x55cdf34c2db0 2024-05-22T23:09:24.247 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.231+0000 7f13dc37b740 10 client.16578 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:09:24.247 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.231+0000 7f13dc37b740 10 client.16578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:09:24.247 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.231+0000 7f13dc37b740 20 client.16578 populate_metadata read hostname 'smithi084' 2024-05-22T23:09:24.247 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.231+0000 7f13dc37b740 1 --2- 172.21.15.84:0/738787104 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55cdf35483c0 0x55cdf354a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.247 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.231+0000 7f13dc37b740 4 client.16578 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:09:24.247 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.231+0000 7f13dc37b740 1 -- 172.21.15.84:0/738787104 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cdf342e830 con 0x55cdf35483c0 2024-05-22T23:09:24.247 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.231+0000 7f13d2ffd640 1 --2- 172.21.15.84:0/738787104 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55cdf35483c0 0x55cdf354a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.231+0000 7f13b6ff5640 10 client.16578 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:24.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.231+0000 7f13d2ffd640 1 --2- 172.21.15.84:0/738787104 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55cdf35483c0 0x55cdf354a7e0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b97272a12bdd8ce server_cookie=afa162775a0c5e63 in_seq=0 out_seq=0 2024-05-22T23:09:24.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13b6ff5640 1 -- 172.21.15.84:0/738787104 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cdf342e830 con 0x55cdf35483c0 2024-05-22T23:09:24.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13b6ff5640 10 client.16578 handle_command_reply: tid=0 2024-05-22T23:09:24.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13dc37b740 1 client.16578 shutdown 2024-05-22T23:09:24.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13dc37b740 1 -- 172.21.15.84:0/738787104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34c2db0 msgr2=0x55cdf34c8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:24.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13dc37b740 1 --2- 172.21.15.84:0/738787104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34c2db0 0x55cdf34c8770 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x55cdf34c04a0 tx=0x7f13c800a7c0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13dc37b740 1 -- 172.21.15.84:0/738787104 shutdown_connections 2024-05-22T23:09:24.249 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13dc37b740 1 --2- 172.21.15.84:0/738787104 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55cdf35483c0 0x55cdf354a7e0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.249 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13dc37b740 1 --2- 172.21.15.84:0/738787104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdf34cdb30 0x55cdf34c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.249 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13b6ff5640 0 client.16578 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:24.249 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13dc37b740 1 --2- 172.21.15.84:0/738787104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cdf34c44a0 0x55cdf34d5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.249 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13dc37b740 1 --2- 172.21.15.84:0/738787104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cdf34c2db0 0x55cdf34c8770 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.249 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13dc37b740 1 -- 172.21.15.84:0/738787104 >> 172.21.15.84:0/738787104 conn(0x55cdf348b8e0 msgr2=0x55cdf34c5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:24.249 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13dc37b740 1 -- 172.21.15.84:0/738787104 shutdown_connections 2024-05-22T23:09:24.249 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13dc37b740 1 -- 172.21.15.84:0/738787104 wait complete. 2024-05-22T23:09:24.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13dc37b740 20 client.16578 trim_cache size 0 max 16384 2024-05-22T23:09:24.250 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:09:24.250 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "no active scrubs running", 2024-05-22T23:09:24.251 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": {} 2024-05-22T23:09:24.251 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:09:24.252 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13d9590640 1 -- 172.21.15.84:0/1479892648 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f13bc0a2d00 msgr2=0x7f13bc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:24.252 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13d9590640 1 --2- 172.21.15.84:0/1479892648 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f13bc0a2d00 0x7f13bc0a51c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f13cc098520 tx=0x7f13cc06aa80 comp rx=0 tx=0).stop 2024-05-22T23:09:24.252 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13d9590640 1 -- 172.21.15.84:0/1479892648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d41067c0 msgr2=0x7f13d41a3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:24.253 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13d9590640 1 --2- 172.21.15.84:0/1479892648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d41067c0 0x7f13d41a3f50 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00eaa0 tx=0x7f13cc00ef60 comp rx=0 tx=0).stop 2024-05-22T23:09:24.253 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13d9590640 1 -- 172.21.15.84:0/1479892648 shutdown_connections 2024-05-22T23:09:24.253 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13d9590640 1 --2- 172.21.15.84:0/1479892648 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f13bc0a2d00 0x7f13bc0a51c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.253 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13d9590640 1 --2- 172.21.15.84:0/1479892648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d41067c0 0x7f13d41a3f50 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.253 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13d9590640 1 --2- 172.21.15.84:0/1479892648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13d4101460 0x7f13d4199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.253 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13d9590640 1 --2- 172.21.15.84:0/1479892648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13d4100ac0 0x7f13d4198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.253 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.232+0000 7f13d9590640 1 -- 172.21.15.84:0/1479892648 >> 172.21.15.84:0/1479892648 conn(0x7f13d4076570 msgr2=0x7f13d40ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:24.253 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.233+0000 7f13d9590640 1 -- 172.21.15.84:0/1479892648 shutdown_connections 2024-05-22T23:09:24.254 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.233+0000 7f13d9590640 1 -- 172.21.15.84:0/1479892648 wait complete. 2024-05-22T23:09:24.254 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-22T23:09:24.254 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:09:24.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.348+0000 7fd0b6229640 1 -- 172.21.15.84:0/1487493147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0b0107840 msgr2=0x7fd0b0107c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:24.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.348+0000 7fd0b6229640 1 --2- 172.21.15.84:0/1487493147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0b0107840 0x7fd0b0107c20 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009930 tx=0x7fd0a002a420 comp rx=0 tx=0).stop 2024-05-22T23:09:24.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.348+0000 7fd0b6229640 1 -- 172.21.15.84:0/1487493147 shutdown_connections 2024-05-22T23:09:24.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.348+0000 7fd0b6229640 1 --2- 172.21.15.84:0/1487493147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0b01021f0 0x7fd0b010e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.348+0000 7fd0b6229640 1 --2- 172.21.15.84:0/1487493147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0b0101830 0x7fd0b0101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.348+0000 7fd0b6229640 1 --2- 172.21.15.84:0/1487493147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0b0107840 0x7fd0b0107c20 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.348+0000 7fd0b6229640 1 -- 172.21.15.84:0/1487493147 >> 172.21.15.84:0/1487493147 conn(0x7fd0b0078c70 msgr2=0x7fd0b0079080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:24.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.348+0000 7fd0b6229640 1 -- 172.21.15.84:0/1487493147 shutdown_connections 2024-05-22T23:09:24.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.348+0000 7fd0b6229640 1 -- 172.21.15.84:0/1487493147 wait complete. 2024-05-22T23:09:24.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.349+0000 7fd0b6229640 1 Processor -- start 2024-05-22T23:09:24.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.349+0000 7fd0b6229640 1 -- start start 2024-05-22T23:09:24.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.349+0000 7fd0b6229640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0b0101830 0x7fd0b0198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.349+0000 7fd0b6229640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0b01021f0 0x7fd0b01986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.349+0000 7fd0b6229640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0b0107840 0x7fd0b019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.349+0000 7fd0b6229640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b0111c50 con 0x7fd0b01021f0 2024-05-22T23:09:24.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.349+0000 7fd0b6229640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b0111ad0 con 0x7fd0b0101830 2024-05-22T23:09:24.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.349+0000 7fd0b6229640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b0111dd0 con 0x7fd0b0107840 2024-05-22T23:09:24.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.349+0000 7fd0aeffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0b01021f0 0x7fd0b01986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.361 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.350+0000 7fd0aeffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0b01021f0 0x7fd0b01986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41966/0 (socket says 172.21.15.84:41966) 2024-05-22T23:09:24.361 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.350+0000 7fd0aeffd640 1 -- 172.21.15.84:0/3916607131 learned_addr learned my addr 172.21.15.84:0/3916607131 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:24.361 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.350+0000 7fd0af7fe640 1 --2- 172.21.15.84:0/3916607131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0b0101830 0x7fd0b0198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.361 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.350+0000 7fd0affff640 1 --2- 172.21.15.84:0/3916607131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0b0107840 0x7fd0b019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.361 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.350+0000 7fd0aeffd640 1 -- 172.21.15.84:0/3916607131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0b0107840 msgr2=0x7fd0b019ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:24.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.350+0000 7fd0aeffd640 1 --2- 172.21.15.84:0/3916607131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0b0107840 0x7fd0b019ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.350+0000 7fd0aeffd640 1 -- 172.21.15.84:0/3916607131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0b0101830 msgr2=0x7fd0b0198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:24.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.350+0000 7fd0aeffd640 1 --2- 172.21.15.84:0/3916607131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0b0101830 0x7fd0b0198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.350+0000 7fd0aeffd640 1 -- 172.21.15.84:0/3916607131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b019d160 con 0x7fd0b01021f0 2024-05-22T23:09:24.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.350+0000 7fd0af7fe640 1 --2- 172.21.15.84:0/3916607131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0b0101830 0x7fd0b0198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:09:24.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.350+0000 7fd0aeffd640 1 --2- 172.21.15.84:0/3916607131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0b01021f0 0x7fd0b01986b0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd0a400c970 tx=0x7fd0a400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:24.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.350+0000 7fd0acff9640 1 -- 172.21.15.84:0/3916607131 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a4007bb0 con 0x7fd0b01021f0 2024-05-22T23:09:24.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.350+0000 7fd0b6229640 1 -- 172.21.15.84:0/3916607131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b019d450 con 0x7fd0b01021f0 2024-05-22T23:09:24.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.350+0000 7fd0acff9640 1 -- 172.21.15.84:0/3916607131 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0a4007d50 con 0x7fd0b01021f0 2024-05-22T23:09:24.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.350+0000 7fd0acff9640 1 -- 172.21.15.84:0/3916607131 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a400f450 con 0x7fd0b01021f0 2024-05-22T23:09:24.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.350+0000 7fd0b6229640 1 -- 172.21.15.84:0/3916607131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b01a4ce0 con 0x7fd0b01021f0 2024-05-22T23:09:24.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.351+0000 7fd0b6229640 1 -- 172.21.15.84:0/3916607131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0b0068ae0 con 0x7fd0b01021f0 2024-05-22T23:09:24.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.352+0000 7fd0acff9640 1 -- 172.21.15.84:0/3916607131 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd0a4005020 con 0x7fd0b01021f0 2024-05-22T23:09:24.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.353+0000 7fd0acff9640 1 --2- 172.21.15.84:0/3916607131 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd0980a2d00 0x7fd0980a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.353+0000 7fd0af7fe640 1 --2- 172.21.15.84:0/3916607131 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd0980a2d00 0x7fd0980a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.353+0000 7fd0acff9640 1 -- 172.21.15.84:0/3916607131 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7fd0a40cf320 con 0x7fd0b01021f0 2024-05-22T23:09:24.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.353+0000 7fd0af7fe640 1 --2- 172.21.15.84:0/3916607131 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd0980a2d00 0x7fd0980a51c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0000c00 tx=0x7fd0a0005a20 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:24.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.357+0000 7fd0acff9640 1 -- 172.21.15.84:0/3916607131 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd0a4014030 con 0x7fd0b01021f0 2024-05-22T23:09:24.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.567+0000 7fd0b6229640 1 -- 172.21.15.84:0/3916607131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0b0109e00 con 0x7fd0b01021f0 2024-05-22T23:09:24.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.568+0000 7fd0acff9640 1 -- 172.21.15.84:0/3916607131 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7fd0a409bf10 con 0x7fd0b01021f0 2024-05-22T23:09:24.574 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:09:24.575 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:09:24.575 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:09:24.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.570+0000 7fd0b6229640 1 -- 172.21.15.84:0/3916607131 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd0980a2d00 msgr2=0x7fd0980a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:24.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.570+0000 7fd0b6229640 1 --2- 172.21.15.84:0/3916607131 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd0980a2d00 0x7fd0980a51c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0000c00 tx=0x7fd0a0005a20 comp rx=0 tx=0).stop 2024-05-22T23:09:24.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.570+0000 7fd0b6229640 1 -- 172.21.15.84:0/3916607131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0b01021f0 msgr2=0x7fd0b01986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:24.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.570+0000 7fd0b6229640 1 --2- 172.21.15.84:0/3916607131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0b01021f0 0x7fd0b01986b0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd0a400c970 tx=0x7fd0a400ce30 comp rx=0 tx=0).stop 2024-05-22T23:09:24.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.570+0000 7fd0b6229640 1 -- 172.21.15.84:0/3916607131 shutdown_connections 2024-05-22T23:09:24.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.570+0000 7fd0b6229640 1 --2- 172.21.15.84:0/3916607131 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd0980a2d00 0x7fd0980a51c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.570+0000 7fd0b6229640 1 --2- 172.21.15.84:0/3916607131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0b0107840 0x7fd0b019ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.570+0000 7fd0b6229640 1 --2- 172.21.15.84:0/3916607131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0b01021f0 0x7fd0b01986b0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.570+0000 7fd0b6229640 1 --2- 172.21.15.84:0/3916607131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0b0101830 0x7fd0b0198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.570+0000 7fd0b6229640 1 -- 172.21.15.84:0/3916607131 >> 172.21.15.84:0/3916607131 conn(0x7fd0b0078c70 msgr2=0x7fd0b0104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:24.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.570+0000 7fd0b6229640 1 -- 172.21.15.84:0/3916607131 shutdown_connections 2024-05-22T23:09:24.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.570+0000 7fd0b6229640 1 -- 172.21.15.84:0/3916607131 wait complete. 2024-05-22T23:09:24.588 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-22T23:09:24.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.684+0000 7f542609e640 1 -- 172.21.15.84:0/768768796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5420069e70 msgr2=0x7f542006a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:24.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.684+0000 7f542609e640 1 --2- 172.21.15.84:0/768768796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5420069e70 0x7f542006a2f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5408009930 tx=0x7f540802a420 comp rx=0 tx=0).stop 2024-05-22T23:09:24.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.684+0000 7f542609e640 1 -- 172.21.15.84:0/768768796 shutdown_connections 2024-05-22T23:09:24.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.684+0000 7f542609e640 1 --2- 172.21.15.84:0/768768796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f542006a830 0x7f542010e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.684+0000 7f542609e640 1 --2- 172.21.15.84:0/768768796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5420069e70 0x7f542006a2f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.684+0000 7f542609e640 1 --2- 172.21.15.84:0/768768796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5420107850 0x7f5420107c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.684+0000 7f542609e640 1 -- 172.21.15.84:0/768768796 >> 172.21.15.84:0/768768796 conn(0x7f5420076570 msgr2=0x7f5420076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:24.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.684+0000 7f542609e640 1 -- 172.21.15.84:0/768768796 shutdown_connections 2024-05-22T23:09:24.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.684+0000 7f542609e640 1 -- 172.21.15.84:0/768768796 wait complete. 2024-05-22T23:09:24.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.685+0000 7f542609e640 1 Processor -- start 2024-05-22T23:09:24.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.685+0000 7f542609e640 1 -- start start 2024-05-22T23:09:24.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.685+0000 7f542609e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5420069e70 0x7f5420198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.685+0000 7f542609e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f542006a830 0x7f54201986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.685+0000 7f542609e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5420107850 0x7f542019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.685+0000 7f542609e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5420111c60 con 0x7f5420069e70 2024-05-22T23:09:24.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.685+0000 7f542609e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5420111ae0 con 0x7f542006a830 2024-05-22T23:09:24.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.685+0000 7f542609e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5420111de0 con 0x7f5420107850 2024-05-22T23:09:24.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.685+0000 7f541f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5420069e70 0x7f5420198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.685+0000 7f541f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5420069e70 0x7f5420198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41978/0 (socket says 172.21.15.84:41978) 2024-05-22T23:09:24.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.685+0000 7f541f7fe640 1 -- 172.21.15.84:0/1063104083 learned_addr learned my addr 172.21.15.84:0/1063104083 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:24.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.685+0000 7f5417fff640 1 --2- 172.21.15.84:0/1063104083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f542006a830 0x7f54201986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.685+0000 7f541f7fe640 1 -- 172.21.15.84:0/1063104083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5420107850 msgr2=0x7f542019ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:24.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.685+0000 7f541f7fe640 1 --2- 172.21.15.84:0/1063104083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5420107850 0x7f542019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.685+0000 7f541f7fe640 1 -- 172.21.15.84:0/1063104083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f542006a830 msgr2=0x7f54201986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:24.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.685+0000 7f541f7fe640 1 --2- 172.21.15.84:0/1063104083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f542006a830 0x7f54201986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.685+0000 7f541f7fe640 1 -- 172.21.15.84:0/1063104083 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f542019d170 con 0x7f5420069e70 2024-05-22T23:09:24.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.686+0000 7f541f7fe640 1 --2- 172.21.15.84:0/1063104083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5420069e70 0x7f5420198180 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f541000c890 tx=0x7f541000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:24.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.686+0000 7f541d7fa640 1 -- 172.21.15.84:0/1063104083 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5410004220 con 0x7f5420069e70 2024-05-22T23:09:24.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.686+0000 7f542609e640 1 -- 172.21.15.84:0/1063104083 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f542019d460 con 0x7f5420069e70 2024-05-22T23:09:24.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.686+0000 7f541d7fa640 1 -- 172.21.15.84:0/1063104083 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54100043c0 con 0x7f5420069e70 2024-05-22T23:09:24.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.686+0000 7f541d7fa640 1 -- 172.21.15.84:0/1063104083 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54100049e0 con 0x7f5420069e70 2024-05-22T23:09:24.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.686+0000 7f542609e640 1 -- 172.21.15.84:0/1063104083 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54201a4cf0 con 0x7f5420069e70 2024-05-22T23:09:24.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.688+0000 7f541d7fa640 1 -- 172.21.15.84:0/1063104083 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f54100079b0 con 0x7f5420069e70 2024-05-22T23:09:24.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.688+0000 7f5428dad740 1 Processor -- start 2024-05-22T23:09:24.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.688+0000 7f5428dad740 1 -- start start 2024-05-22T23:09:24.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.688+0000 7f5428dad740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d44f80 0x563530d45360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.688+0000 7f5428dad740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563530d458a0 0x563530d4fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.688+0000 7f541d7fa640 1 --2- 172.21.15.84:0/1063104083 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f540c0a2d00 0x7f540c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.688+0000 7f5428dad740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563530d4ff70 0x563530d52360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.688+0000 7f5428dad740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563530ca9db0 con 0x563530d44f80 2024-05-22T23:09:24.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.688+0000 7f5428dad740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563530c48400 con 0x563530d458a0 2024-05-22T23:09:24.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.688+0000 7f5428dad740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563530ca5d70 con 0x563530d4ff70 2024-05-22T23:09:24.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.689+0000 7f541ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d44f80 0x563530d45360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.689+0000 7f5417fff640 1 --2- 172.21.15.84:0/1063104083 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f540c0a2d00 0x7f540c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.689+0000 7f541ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d44f80 0x563530d45360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41988/0 (socket says 172.21.15.84:41988) 2024-05-22T23:09:24.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.689+0000 7f541ffff640 1 -- 172.21.15.84:0/496706033 learned_addr learned my addr 172.21.15.84:0/496706033 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:24.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.689+0000 7f541d7fa640 1 -- 172.21.15.84:0/1063104083 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7f54100cf550 con 0x7f5420069e70 2024-05-22T23:09:24.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.689+0000 7f541f7fe640 1 --2- 172.21.15.84:0/496706033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563530d458a0 0x563530d4fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.689+0000 7f541ffff640 1 --2- 172.21.15.84:0/496706033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563530d4ff70 0x563530d52360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.689+0000 7f541ffff640 1 -- 172.21.15.84:0/496706033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563530d4ff70 msgr2=0x563530d52360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:24.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.689+0000 7f541ffff640 1 --2- 172.21.15.84:0/496706033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563530d4ff70 0x563530d52360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.689+0000 7f541ffff640 1 -- 172.21.15.84:0/496706033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563530d458a0 msgr2=0x563530d4fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:24.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.689+0000 7f541ffff640 1 --2- 172.21.15.84:0/496706033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563530d458a0 0x563530d4fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.689+0000 7f541ffff640 1 -- 172.21.15.84:0/496706033 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563530c3a2f0 con 0x563530d44f80 2024-05-22T23:09:24.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.689+0000 7f5417fff640 1 --2- 172.21.15.84:0/1063104083 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f540c0a2d00 0x7f540c0a51c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5408000c00 tx=0x7f5408005a20 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:24.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.689+0000 7f541ffff640 1 --2- 172.21.15.84:0/496706033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d44f80 0x563530d45360 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5418009f60 tx=0x7f5418009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e26e0c37c4259c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:24.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.689+0000 7f5415ffb640 1 -- 172.21.15.84:0/496706033 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54180140f0 con 0x563530d44f80 2024-05-22T23:09:24.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.689+0000 7f5415ffb640 1 -- 172.21.15.84:0/496706033 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5418003e50 con 0x563530d44f80 2024-05-22T23:09:24.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.689+0000 7f5415ffb640 1 -- 172.21.15.84:0/496706033 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5418004280 con 0x563530d44f80 2024-05-22T23:09:24.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.690+0000 7f5428dad740 1 -- 172.21.15.84:0/496706033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d44f80 msgr2=0x563530d45360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:24.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.690+0000 7f5428dad740 1 --2- 172.21.15.84:0/496706033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d44f80 0x563530d45360 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5418009f60 tx=0x7f5418009800 comp rx=0 tx=0).stop 2024-05-22T23:09:24.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.690+0000 7f5428dad740 1 -- 172.21.15.84:0/496706033 shutdown_connections 2024-05-22T23:09:24.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.690+0000 7f5428dad740 1 --2- 172.21.15.84:0/496706033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563530d4ff70 0x563530d52360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.690+0000 7f5428dad740 1 --2- 172.21.15.84:0/496706033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563530d458a0 0x563530d4fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.690+0000 7f5428dad740 1 --2- 172.21.15.84:0/496706033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d44f80 0x563530d45360 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.690+0000 7f5428dad740 1 -- 172.21.15.84:0/496706033 >> 172.21.15.84:0/496706033 conn(0x563530d412a0 msgr2=0x563530d41680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:24.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.690+0000 7f5428dad740 1 -- 172.21.15.84:0/496706033 shutdown_connections 2024-05-22T23:09:24.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.690+0000 7f5428dad740 1 -- 172.21.15.84:0/496706033 wait complete. 2024-05-22T23:09:24.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.690+0000 7f5428dad740 1 -- start start 2024-05-22T23:09:24.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.691+0000 7f5428dad740 1 Processor -- start 2024-05-22T23:09:24.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.691+0000 7f5428dad740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563530d44f80 0x563530d4ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.691+0000 7f5428dad740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d458a0 0x563530d4d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.691+0000 7f5428dad740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563530d4ff70 0x563530d4d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.691+0000 7f5428dad740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563530d52d60 con 0x563530d458a0 2024-05-22T23:09:24.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.691+0000 7f5428dad740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563530c642e0 con 0x563530d4ff70 2024-05-22T23:09:24.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.691+0000 7f5428dad740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563530d52ee0 con 0x563530d44f80 2024-05-22T23:09:24.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.691+0000 7f5428dad740 10 client.0 _ll_register_callbacks cb 0x563530c6da70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:09:24.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.691+0000 7f5417fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d458a0 0x563530d4d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.691+0000 7f5417fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d458a0 0x563530d4d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41996/0 (socket says 172.21.15.84:41996) 2024-05-22T23:09:24.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.691+0000 7f5417fff640 1 -- 172.21.15.84:0/597671810 learned_addr learned my addr 172.21.15.84:0/597671810 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:24.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.691+0000 7f541ffff640 1 --2- 172.21.15.84:0/597671810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563530d4ff70 0x563530d4d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.691+0000 7f5417fff640 1 -- 172.21.15.84:0/597671810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563530d44f80 msgr2=0x563530d4ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:24.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.691+0000 7f5417fff640 1 --2- 172.21.15.84:0/597671810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563530d44f80 0x563530d4ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.691+0000 7f5417fff640 1 -- 172.21.15.84:0/597671810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563530d4ff70 msgr2=0x563530d4d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:24.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.691+0000 7f5417fff640 1 --2- 172.21.15.84:0/597671810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563530d4ff70 0x563530d4d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.691+0000 7f5417fff640 1 -- 172.21.15.84:0/597671810 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635309bd3f0 con 0x563530d458a0 2024-05-22T23:09:24.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.691+0000 7f541ffff640 1 --2- 172.21.15.84:0/597671810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563530d4ff70 0x563530d4d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:09:24.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.691+0000 7f5417fff640 1 --2- 172.21.15.84:0/597671810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d458a0 0x563530d4d3d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f540802a940 tx=0x7f5408009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8851dfcd506957 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:24.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.691+0000 7f53feffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:09:24.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.691+0000 7f53feffd640 1 -- 172.21.15.84:0/597671810 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540804b070 con 0x563530d458a0 2024-05-22T23:09:24.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.691+0000 7f53feffd640 1 -- 172.21.15.84:0/597671810 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7f5408003ac0 con 0x563530d458a0 2024-05-22T23:09:24.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.691+0000 7f5428dad740 1 -- 172.21.15.84:0/597671810 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5418003e50 con 0x563530d458a0 2024-05-22T23:09:24.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.692+0000 7f53feffd640 1 -- 172.21.15.84:0/597671810 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f5408057050 con 0x563530d458a0 2024-05-22T23:09:24.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.692+0000 7f5428dad740 10 client.16596 fetch_fsmap learned FSMap version 18 2024-05-22T23:09:24.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.692+0000 7f5428dad740 1 -- 172.21.15.84:0/597671810 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x563530a877d0 con 0x563530d458a0 2024-05-22T23:09:24.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.692+0000 7f53feffd640 1 -- 172.21.15.84:0/597671810 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5418003e50 con 0x563530d458a0 2024-05-22T23:09:24.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.692+0000 7f53feffd640 1 -- 172.21.15.84:0/597671810 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f5408047ca0 con 0x563530d458a0 2024-05-22T23:09:24.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.692+0000 7f5428dad740 10 client.16596 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:09:24.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.692+0000 7f5428dad740 10 client.16596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:09:24.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.692+0000 7f5428dad740 20 client.16596 populate_metadata read hostname 'smithi084' 2024-05-22T23:09:24.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.692+0000 7f5428dad740 1 --2- 172.21.15.84:0/597671810 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563530d601e0 0x563530d62660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.692+0000 7f5428dad740 4 client.16596 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:09:24.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.692+0000 7f5428dad740 1 -- 172.21.15.84:0/597671810 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563530cb74e0 con 0x563530d601e0 2024-05-22T23:09:24.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.692+0000 7f541f7fe640 1 --2- 172.21.15.84:0/597671810 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563530d601e0 0x563530d62660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.693+0000 7f53feffd640 10 client.16596 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:24.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.693+0000 7f541f7fe640 1 --2- 172.21.15.84:0/597671810 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563530d601e0 0x563530d62660 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be78752832c867c6 server_cookie=8f50471f819e9cb in_seq=0 out_seq=0 2024-05-22T23:09:24.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.693+0000 7f53feffd640 1 -- 172.21.15.84:0/597671810 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563530cb74e0 con 0x563530d601e0 2024-05-22T23:09:24.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.693+0000 7f53feffd640 10 client.16596 handle_command_reply: tid=0 2024-05-22T23:09:24.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.693+0000 7f5428dad740 1 client.16596 shutdown 2024-05-22T23:09:24.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.693+0000 7f5428dad740 1 -- 172.21.15.84:0/597671810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d458a0 msgr2=0x563530d4d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:24.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.693+0000 7f5428dad740 1 --2- 172.21.15.84:0/597671810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d458a0 0x563530d4d3d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f540802a940 tx=0x7f5408009260 comp rx=0 tx=0).stop 2024-05-22T23:09:24.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.693+0000 7f5428dad740 1 -- 172.21.15.84:0/597671810 shutdown_connections 2024-05-22T23:09:24.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.693+0000 7f5428dad740 1 --2- 172.21.15.84:0/597671810 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563530d601e0 0x563530d62660 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.693+0000 7f5428dad740 1 --2- 172.21.15.84:0/597671810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563530d4ff70 0x563530d4d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.693+0000 7f5428dad740 1 --2- 172.21.15.84:0/597671810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d458a0 0x563530d4d3d0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.693+0000 7f5428dad740 1 --2- 172.21.15.84:0/597671810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563530d44f80 0x563530d4ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.693+0000 7f53feffd640 0 client.16596 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:24.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.693+0000 7f5428dad740 1 -- 172.21.15.84:0/597671810 >> 172.21.15.84:0/597671810 conn(0x563530d412a0 msgr2=0x563530d404c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:24.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.693+0000 7f5428dad740 1 -- 172.21.15.84:0/597671810 shutdown_connections 2024-05-22T23:09:24.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.693+0000 7f5428dad740 1 -- 172.21.15.84:0/597671810 wait complete. 2024-05-22T23:09:24.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.694+0000 7f5428dad740 20 client.16596 trim_cache size 0 max 16384 2024-05-22T23:09:24.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.707+0000 7f5428dad740 1 Processor -- start 2024-05-22T23:09:24.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.707+0000 7f5428dad740 1 -- start start 2024-05-22T23:09:24.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.707+0000 7f5428dad740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d44f80 0x563530d5bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.707+0000 7f5428dad740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563530d458a0 0x563530d4d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.707+0000 7f5428dad740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563530d4ff70 0x563530d67f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.707+0000 7f5428dad740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f54201ab7d0 con 0x563530d44f80 2024-05-22T23:09:24.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.707+0000 7f5428dad740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f54201ae030 con 0x563530d458a0 2024-05-22T23:09:24.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.707+0000 7f5428dad740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f54201abe20 con 0x563530d4ff70 2024-05-22T23:09:24.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.707+0000 7f541f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d44f80 0x563530d5bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.707+0000 7f541f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d44f80 0x563530d5bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42012/0 (socket says 172.21.15.84:42012) 2024-05-22T23:09:24.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.707+0000 7f541f7fe640 1 -- 172.21.15.84:0/2111973240 learned_addr learned my addr 172.21.15.84:0/2111973240 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:24.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.707+0000 7f541ffff640 1 --2- 172.21.15.84:0/2111973240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563530d4ff70 0x563530d67f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.707+0000 7f5417fff640 1 --2- 172.21.15.84:0/2111973240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563530d458a0 0x563530d4d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.707+0000 7f541f7fe640 1 -- 172.21.15.84:0/2111973240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563530d4ff70 msgr2=0x563530d67f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:24.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.707+0000 7f541f7fe640 1 --2- 172.21.15.84:0/2111973240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563530d4ff70 0x563530d67f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.707+0000 7f541f7fe640 1 -- 172.21.15.84:0/2111973240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563530d458a0 msgr2=0x563530d4d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:24.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.707+0000 7f541f7fe640 1 --2- 172.21.15.84:0/2111973240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563530d458a0 0x563530d4d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.707+0000 7f541f7fe640 1 -- 172.21.15.84:0/2111973240 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563530d45da0 con 0x563530d44f80 2024-05-22T23:09:24.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.707+0000 7f541ffff640 1 --2- 172.21.15.84:0/2111973240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563530d4ff70 0x563530d67f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:09:24.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.707+0000 7f5417fff640 1 --2- 172.21.15.84:0/2111973240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563530d458a0 0x563530d4d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:09:24.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.707+0000 7f541f7fe640 1 --2- 172.21.15.84:0/2111973240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d44f80 0x563530d5bc20 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5410007590 tx=0x7f5410020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcbf13bc5657f8c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:24.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.708+0000 7f54157fa640 1 -- 172.21.15.84:0/2111973240 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541001d040 con 0x563530d44f80 2024-05-22T23:09:24.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.708+0000 7f54157fa640 1 -- 172.21.15.84:0/2111973240 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f541009c140 con 0x563530d44f80 2024-05-22T23:09:24.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.708+0000 7f54157fa640 1 -- 172.21.15.84:0/2111973240 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54100980b0 con 0x563530d44f80 2024-05-22T23:09:24.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.708+0000 7f5428dad740 1 -- 172.21.15.84:0/2111973240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d44f80 msgr2=0x563530d5bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:24.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.708+0000 7f5428dad740 1 --2- 172.21.15.84:0/2111973240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d44f80 0x563530d5bc20 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5410007590 tx=0x7f5410020000 comp rx=0 tx=0).stop 2024-05-22T23:09:24.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.708+0000 7f5428dad740 1 -- 172.21.15.84:0/2111973240 shutdown_connections 2024-05-22T23:09:24.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.708+0000 7f5428dad740 1 --2- 172.21.15.84:0/2111973240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563530d4ff70 0x563530d67f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.708+0000 7f5428dad740 1 --2- 172.21.15.84:0/2111973240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563530d458a0 0x563530d4d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.708+0000 7f5428dad740 1 --2- 172.21.15.84:0/2111973240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d44f80 0x563530d5bc20 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.708+0000 7f5428dad740 1 -- 172.21.15.84:0/2111973240 >> 172.21.15.84:0/2111973240 conn(0x563530d412a0 msgr2=0x563530d46f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:24.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.708+0000 7f5428dad740 1 -- 172.21.15.84:0/2111973240 shutdown_connections 2024-05-22T23:09:24.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.708+0000 7f5428dad740 1 -- 172.21.15.84:0/2111973240 wait complete. 2024-05-22T23:09:24.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.708+0000 7f5428dad740 1 -- start start 2024-05-22T23:09:24.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.709+0000 7f5428dad740 1 Processor -- start 2024-05-22T23:09:24.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.709+0000 7f5428dad740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563530d44f80 0x563530d53e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.709+0000 7f5428dad740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563530d458a0 0x563530d543c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.709+0000 7f5428dad740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d4ff70 0x563530dc57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.709+0000 7f5428dad740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f54201ac4f0 con 0x563530d4ff70 2024-05-22T23:09:24.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.709+0000 7f5428dad740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f54201ad960 con 0x563530d44f80 2024-05-22T23:09:24.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.709+0000 7f5428dad740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f54201acbc0 con 0x563530d458a0 2024-05-22T23:09:24.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.709+0000 7f5428dad740 10 client.0 _ll_register_callbacks cb 0x7f540803f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:09:24.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.709+0000 7f541ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d4ff70 0x563530dc57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.709+0000 7f541ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d4ff70 0x563530dc57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42024/0 (socket says 172.21.15.84:42024) 2024-05-22T23:09:24.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.709+0000 7f541ffff640 1 -- 172.21.15.84:0/307967215 learned_addr learned my addr 172.21.15.84:0/307967215 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:24.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.709+0000 7f5417fff640 1 --2- 172.21.15.84:0/307967215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563530d458a0 0x563530d543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.709+0000 7f541f7fe640 1 --2- 172.21.15.84:0/307967215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563530d44f80 0x563530d53e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.709+0000 7f541ffff640 1 -- 172.21.15.84:0/307967215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563530d458a0 msgr2=0x563530d543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:24.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.709+0000 7f541ffff640 1 --2- 172.21.15.84:0/307967215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563530d458a0 0x563530d543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.709+0000 7f541ffff640 1 -- 172.21.15.84:0/307967215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563530d44f80 msgr2=0x563530d53e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:24.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.709+0000 7f541ffff640 1 --2- 172.21.15.84:0/307967215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563530d44f80 0x563530d53e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.709+0000 7f541ffff640 1 -- 172.21.15.84:0/307967215 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563530d54c60 con 0x563530d4ff70 2024-05-22T23:09:24.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.709+0000 7f5417fff640 1 --2- 172.21.15.84:0/307967215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563530d458a0 0x563530d543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:09:24.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.709+0000 7f541f7fe640 1 --2- 172.21.15.84:0/307967215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563530d44f80 0x563530d53e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:09:24.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.709+0000 7f541ffff640 1 --2- 172.21.15.84:0/307967215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d4ff70 0x563530dc57a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5418009f60 tx=0x7f5418005b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998579d92df1ee64 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:24.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.710+0000 7f53feffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:09:24.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.710+0000 7f53feffd640 1 -- 172.21.15.84:0/307967215 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5418003d80 con 0x563530d4ff70 2024-05-22T23:09:24.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.710+0000 7f53feffd640 1 -- 172.21.15.84:0/307967215 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7f541800f800 con 0x563530d4ff70 2024-05-22T23:09:24.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.710+0000 7f5428dad740 1 -- 172.21.15.84:0/307967215 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f541009c140 con 0x563530d4ff70 2024-05-22T23:09:24.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.710+0000 7f53feffd640 1 -- 172.21.15.84:0/307967215 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f541800fa00 con 0x563530d4ff70 2024-05-22T23:09:24.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.710+0000 7f5428dad740 10 client.16602 fetch_fsmap learned FSMap version 18 2024-05-22T23:09:24.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.710+0000 7f5428dad740 1 -- 172.21.15.84:0/307967215 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x563530dc6d20 con 0x563530d4ff70 2024-05-22T23:09:24.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.710+0000 7f53feffd640 1 -- 172.21.15.84:0/307967215 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f541009c140 con 0x563530d4ff70 2024-05-22T23:09:24.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.710+0000 7f53feffd640 1 -- 172.21.15.84:0/307967215 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f5418030070 con 0x563530d4ff70 2024-05-22T23:09:24.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.710+0000 7f5428dad740 10 client.16602 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:09:24.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.710+0000 7f5428dad740 10 client.16602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:09:24.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.710+0000 7f5428dad740 20 client.16602 populate_metadata read hostname 'smithi084' 2024-05-22T23:09:24.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.710+0000 7f5428dad740 1 --2- 172.21.15.84:0/307967215 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563530dc7300 0x563530dc9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:24.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.710+0000 7f5428dad740 4 client.16602 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-22T23:09:24.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.710+0000 7f5428dad740 1 -- 172.21.15.84:0/307967215 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563530cb1830 con 0x563530dc7300 2024-05-22T23:09:24.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.710+0000 7f541f7fe640 1 --2- 172.21.15.84:0/307967215 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563530dc7300 0x563530dc9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:24.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.711+0000 7f53feffd640 10 client.16602 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:24.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.711+0000 7f541f7fe640 1 --2- 172.21.15.84:0/307967215 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563530dc7300 0x563530dc9720 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc76bb073c09b52d server_cookie=5b4bd0c62fc15120 in_seq=0 out_seq=0 2024-05-22T23:09:24.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.711+0000 7f53feffd640 1 -- 172.21.15.84:0/307967215 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563530cb1830 con 0x563530dc7300 2024-05-22T23:09:24.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.711+0000 7f53feffd640 10 client.16602 handle_command_reply: tid=0 2024-05-22T23:09:24.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.711+0000 7f5428dad740 1 client.16602 shutdown 2024-05-22T23:09:24.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.711+0000 7f5428dad740 1 -- 172.21.15.84:0/307967215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d4ff70 msgr2=0x563530dc57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:24.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.711+0000 7f5428dad740 1 --2- 172.21.15.84:0/307967215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d4ff70 0x563530dc57a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5418009f60 tx=0x7f5418005b30 comp rx=0 tx=0).stop 2024-05-22T23:09:24.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.711+0000 7f5428dad740 1 -- 172.21.15.84:0/307967215 shutdown_connections 2024-05-22T23:09:24.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.711+0000 7f5428dad740 1 --2- 172.21.15.84:0/307967215 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563530dc7300 0x563530dc9720 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.711+0000 7f5428dad740 1 --2- 172.21.15.84:0/307967215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563530d4ff70 0x563530dc57a0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.711+0000 7f5428dad740 1 --2- 172.21.15.84:0/307967215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563530d458a0 0x563530d543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.711+0000 7f5428dad740 1 --2- 172.21.15.84:0/307967215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563530d44f80 0x563530d53e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.711+0000 7f5428dad740 1 -- 172.21.15.84:0/307967215 >> 172.21.15.84:0/307967215 conn(0x563530d412a0 msgr2=0x563530d466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:24.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.711+0000 7f53feffd640 0 client.16602 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:24.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.711+0000 7f5428dad740 1 -- 172.21.15.84:0/307967215 shutdown_connections 2024-05-22T23:09:24.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.711+0000 7f5428dad740 1 -- 172.21.15.84:0/307967215 wait complete. 2024-05-22T23:09:24.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.712+0000 7f5428dad740 20 client.16602 trim_cache size 0 max 16384 2024-05-22T23:09:24.730 INFO:teuthology.orchestra.run.smithi084.stdout:[] 2024-05-22T23:09:24.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.712+0000 7f542609e640 1 -- 172.21.15.84:0/1063104083 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f540c0a2d00 msgr2=0x7f540c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:24.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.712+0000 7f542609e640 1 --2- 172.21.15.84:0/1063104083 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f540c0a2d00 0x7f540c0a51c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5408000c00 tx=0x7f5408005a20 comp rx=0 tx=0).stop 2024-05-22T23:09:24.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.712+0000 7f542609e640 1 -- 172.21.15.84:0/1063104083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5420069e70 msgr2=0x7f5420198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:24.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.712+0000 7f542609e640 1 --2- 172.21.15.84:0/1063104083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5420069e70 0x7f5420198180 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f541000c890 tx=0x7f541000cd50 comp rx=0 tx=0).stop 2024-05-22T23:09:24.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.712+0000 7f542609e640 1 -- 172.21.15.84:0/1063104083 shutdown_connections 2024-05-22T23:09:24.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.712+0000 7f542609e640 1 --2- 172.21.15.84:0/1063104083 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f540c0a2d00 0x7f540c0a51c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.712+0000 7f542609e640 1 --2- 172.21.15.84:0/1063104083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5420107850 0x7f542019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.712+0000 7f542609e640 1 --2- 172.21.15.84:0/1063104083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f542006a830 0x7f54201986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.712+0000 7f542609e640 1 --2- 172.21.15.84:0/1063104083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5420069e70 0x7f5420198180 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:24.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.712+0000 7f542609e640 1 -- 172.21.15.84:0/1063104083 >> 172.21.15.84:0/1063104083 conn(0x7f5420076570 msgr2=0x7f54200fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:24.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.712+0000 7f542609e640 1 -- 172.21.15.84:0/1063104083 shutdown_connections 2024-05-22T23:09:24.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:24.712+0000 7f542609e640 1 -- 172.21.15.84:0/1063104083 wait complete. 2024-05-22T23:09:25.357 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:25 smithi096 ceph-mon[36392]: pgmap v234: 97 pgs: 97 active+clean; 466 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 4.1 MiB/s wr, 81 op/s 2024-05-22T23:09:25.357 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:25 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3916607131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:09:25.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:25 smithi084 ceph-mon[32186]: pgmap v234: 97 pgs: 97 active+clean; 466 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 4.1 MiB/s wr, 81 op/s 2024-05-22T23:09:25.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:25 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3916607131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:09:25.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:25 smithi204 ceph-mon[36211]: pgmap v234: 97 pgs: 97 active+clean; 466 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 4.1 MiB/s wr, 81 op/s 2024-05-22T23:09:25.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:25 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3916607131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:09:25.733 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-22T23:09:25.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:09:25.734 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-22T23:09:25.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.844+0000 7fc5b1f70640 1 -- 172.21.15.84:0/455876741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5ac10be30 msgr2=0x7fc5ac10e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:25.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.844+0000 7fc5b1f70640 1 --2- 172.21.15.84:0/455876741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5ac10be30 0x7fc5ac10e2d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009ca0 tx=0x7fc5a402a9e0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.844+0000 7fc5b1f70640 1 -- 172.21.15.84:0/455876741 shutdown_connections 2024-05-22T23:09:25.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.844+0000 7fc5b1f70640 1 --2- 172.21.15.84:0/455876741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5ac10be30 0x7fc5ac10e2d0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.844+0000 7fc5b1f70640 1 --2- 172.21.15.84:0/455876741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5ac103410 0x7fc5ac10b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.844+0000 7fc5b1f70640 1 --2- 172.21.15.84:0/455876741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5ac102af0 0x7fc5ac102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.844+0000 7fc5b1f70640 1 -- 172.21.15.84:0/455876741 >> 172.21.15.84:0/455876741 conn(0x7fc5ac077870 msgr2=0x7fc5ac077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:25.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.844+0000 7fc5b1f70640 1 -- 172.21.15.84:0/455876741 shutdown_connections 2024-05-22T23:09:25.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.844+0000 7fc5b1f70640 1 -- 172.21.15.84:0/455876741 wait complete. 2024-05-22T23:09:25.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.845+0000 7fc5b1f70640 1 Processor -- start 2024-05-22T23:09:25.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.845+0000 7fc5b1f70640 1 -- start start 2024-05-22T23:09:25.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.845+0000 7fc5b1f70640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5ac102af0 0x7fc5ac100740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:25.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.845+0000 7fc5b1f70640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5ac103410 0x7fc5ac100c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:25.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.845+0000 7fc5b1f70640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5ac10be30 0x7fc5ac1011c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:25.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.845+0000 7fc5b1f70640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac111c30 con 0x7fc5ac103410 2024-05-22T23:09:25.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.845+0000 7fc5b1f70640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac111ab0 con 0x7fc5ac102af0 2024-05-22T23:09:25.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.845+0000 7fc5b1f70640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac111db0 con 0x7fc5ac10be30 2024-05-22T23:09:25.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.845+0000 7fc5a3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5ac103410 0x7fc5ac100c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:25.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.845+0000 7fc5a3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5ac103410 0x7fc5ac100c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42042/0 (socket says 172.21.15.84:42042) 2024-05-22T23:09:25.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.845+0000 7fc5a3fff640 1 -- 172.21.15.84:0/387238220 learned_addr learned my addr 172.21.15.84:0/387238220 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:25.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.845+0000 7fc5ab7fe640 1 --2- 172.21.15.84:0/387238220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5ac102af0 0x7fc5ac100740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:25.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.845+0000 7fc5abfff640 1 --2- 172.21.15.84:0/387238220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5ac10be30 0x7fc5ac1011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:25.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.845+0000 7fc5a3fff640 1 -- 172.21.15.84:0/387238220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5ac10be30 msgr2=0x7fc5ac1011c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:25.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.845+0000 7fc5a3fff640 1 --2- 172.21.15.84:0/387238220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5ac10be30 0x7fc5ac1011c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.845+0000 7fc5a3fff640 1 -- 172.21.15.84:0/387238220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5ac102af0 msgr2=0x7fc5ac100740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.845+0000 7fc5a3fff640 1 --2- 172.21.15.84:0/387238220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5ac102af0 0x7fc5ac100740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.845+0000 7fc5a3fff640 1 -- 172.21.15.84:0/387238220 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ac1a65b0 con 0x7fc5ac103410 2024-05-22T23:09:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.846+0000 7fc5a3fff640 1 --2- 172.21.15.84:0/387238220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5ac103410 0x7fc5ac100c80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc59400c890 tx=0x7fc59400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.846+0000 7fc5a97fa640 1 -- 172.21.15.84:0/387238220 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594004220 con 0x7fc5ac103410 2024-05-22T23:09:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.846+0000 7fc5a97fa640 1 -- 172.21.15.84:0/387238220 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5940043c0 con 0x7fc5ac103410 2024-05-22T23:09:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.846+0000 7fc5a97fa640 1 -- 172.21.15.84:0/387238220 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5940049e0 con 0x7fc5ac103410 2024-05-22T23:09:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.846+0000 7fc5b1f70640 1 -- 172.21.15.84:0/387238220 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5ac1a68a0 con 0x7fc5ac103410 2024-05-22T23:09:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.846+0000 7fc5b1f70640 1 -- 172.21.15.84:0/387238220 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5ac1a6df0 con 0x7fc5ac103410 2024-05-22T23:09:25.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.848+0000 7fc5a97fa640 1 -- 172.21.15.84:0/387238220 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc5940079b0 con 0x7fc5ac103410 2024-05-22T23:09:25.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.848+0000 7fc5b4c83740 1 Processor -- start 2024-05-22T23:09:25.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.848+0000 7fc5b4c83740 1 -- start start 2024-05-22T23:09:25.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.849+0000 7fc5b4c83740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559259e40c50 0x559259e41030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:25.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.849+0000 7fc5b4c83740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e41570 0x559259e4b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:25.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.849+0000 7fc5b4c83740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559259e4bc40 0x559259e4e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:25.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.849+0000 7fc5b4c83740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559259da4a40 con 0x559259e41570 2024-05-22T23:09:25.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.849+0000 7fc5b4c83740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559259d43440 con 0x559259e40c50 2024-05-22T23:09:25.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.849+0000 7fc5b4c83740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559259da0a00 con 0x559259e4bc40 2024-05-22T23:09:25.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.849+0000 7fc5ab7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e41570 0x559259e4b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:25.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.849+0000 7fc5ab7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e41570 0x559259e4b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42050/0 (socket says 172.21.15.84:42050) 2024-05-22T23:09:25.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.849+0000 7fc5ab7fe640 1 -- 172.21.15.84:0/3858556350 learned_addr learned my addr 172.21.15.84:0/3858556350 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:25.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.849+0000 7fc5a97fa640 1 --2- 172.21.15.84:0/387238220 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fc5980a2d30 0x7fc5980a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:25.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.849+0000 7fc5abfff640 1 --2- 172.21.15.84:0/3858556350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559259e40c50 0x559259e41030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:25.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.849+0000 7fc5abfff640 1 --2- 172.21.15.84:0/387238220 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fc5980a2d30 0x7fc5980a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:25.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.849+0000 7fc5ab7fe640 1 -- 172.21.15.84:0/3858556350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559259e4bc40 msgr2=0x559259e4e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:25.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.849+0000 7fc5a97fa640 1 -- 172.21.15.84:0/387238220 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7fc5940cf370 con 0x7fc5ac103410 2024-05-22T23:09:25.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.849+0000 7fc5a3fff640 1 --2- 172.21.15.84:0/3858556350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559259e4bc40 0x559259e4e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:25.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.849+0000 7fc5ab7fe640 1 --2- 172.21.15.84:0/3858556350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559259e4bc40 0x559259e4e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.849+0000 7fc5ab7fe640 1 -- 172.21.15.84:0/3858556350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559259e40c50 msgr2=0x559259e41030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:25.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.849+0000 7fc5ab7fe640 1 --2- 172.21.15.84:0/3858556350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559259e40c50 0x559259e41030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.849+0000 7fc5ab7fe640 1 -- 172.21.15.84:0/3858556350 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559259d4d2d0 con 0x559259e41570 2024-05-22T23:09:25.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.849+0000 7fc5abfff640 1 --2- 172.21.15.84:0/3858556350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559259e40c50 0x559259e41030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:09:25.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.849+0000 7fc5a3fff640 1 --2- 172.21.15.84:0/3858556350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559259e4bc40 0x559259e4e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:09:25.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.849+0000 7fc5ab7fe640 1 --2- 172.21.15.84:0/3858556350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e41570 0x559259e4b700 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00b760 tx=0x7fc59c00bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69910041b70d4bf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:25.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.849+0000 7fc5abfff640 1 --2- 172.21.15.84:0/387238220 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fc5980a2d30 0x7fc5980a51f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4005fe0 tx=0x7fc5a4005c50 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:25.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.850+0000 7fc5a1ffb640 1 -- 172.21.15.84:0/3858556350 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c003dd0 con 0x559259e41570 2024-05-22T23:09:25.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.850+0000 7fc5a1ffb640 1 -- 172.21.15.84:0/3858556350 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc59c015070 con 0x559259e41570 2024-05-22T23:09:25.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.850+0000 7fc5a1ffb640 1 -- 172.21.15.84:0/3858556350 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c004e60 con 0x559259e41570 2024-05-22T23:09:25.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.850+0000 7fc5b4c83740 1 -- 172.21.15.84:0/3858556350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e41570 msgr2=0x559259e4b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:25.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.850+0000 7fc5b4c83740 1 --2- 172.21.15.84:0/3858556350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e41570 0x559259e4b700 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00b760 tx=0x7fc59c00bc20 comp rx=0 tx=0).stop 2024-05-22T23:09:25.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.850+0000 7fc5b4c83740 1 -- 172.21.15.84:0/3858556350 shutdown_connections 2024-05-22T23:09:25.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.850+0000 7fc5b4c83740 1 --2- 172.21.15.84:0/3858556350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559259e4bc40 0x559259e4e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.850+0000 7fc5b4c83740 1 --2- 172.21.15.84:0/3858556350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e41570 0x559259e4b700 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.850+0000 7fc5b4c83740 1 --2- 172.21.15.84:0/3858556350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559259e40c50 0x559259e41030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.850+0000 7fc5b4c83740 1 -- 172.21.15.84:0/3858556350 >> 172.21.15.84:0/3858556350 conn(0x559259e3cf70 msgr2=0x559259e3d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:25.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.850+0000 7fc5b4c83740 1 -- 172.21.15.84:0/3858556350 shutdown_connections 2024-05-22T23:09:25.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.850+0000 7fc5b4c83740 1 -- 172.21.15.84:0/3858556350 wait complete. 2024-05-22T23:09:25.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.851+0000 7fc5b4c83740 1 -- start start 2024-05-22T23:09:25.872 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.851+0000 7fc5b4c83740 1 Processor -- start 2024-05-22T23:09:25.872 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.851+0000 7fc5b4c83740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559259e40c50 0x559259e48c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:25.872 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.851+0000 7fc5b4c83740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e41570 0x559259e49160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:25.872 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.851+0000 7fc5b4c83740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559259e4bc40 0x559259e496a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:25.872 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.851+0000 7fc5b4c83740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559259e4ea30 con 0x559259e41570 2024-05-22T23:09:25.872 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.851+0000 7fc5b4c83740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559259d2e5f0 con 0x559259e40c50 2024-05-22T23:09:25.872 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.851+0000 7fc5b4c83740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559259e4ebb0 con 0x559259e4bc40 2024-05-22T23:09:25.872 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.851+0000 7fc5b4c83740 10 client.0 _ll_register_callbacks cb 0x559259d29cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:09:25.872 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.851+0000 7fc5a3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e41570 0x559259e49160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:25.873 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.851+0000 7fc5a3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e41570 0x559259e49160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42060/0 (socket says 172.21.15.84:42060) 2024-05-22T23:09:25.873 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.851+0000 7fc5a3fff640 1 -- 172.21.15.84:0/3683710915 learned_addr learned my addr 172.21.15.84:0/3683710915 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:25.873 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.851+0000 7fc5ab7fe640 1 --2- 172.21.15.84:0/3683710915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559259e40c50 0x559259e48c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:25.873 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.851+0000 7fc5abfff640 1 --2- 172.21.15.84:0/3683710915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559259e4bc40 0x559259e496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:25.873 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.851+0000 7fc5a3fff640 1 -- 172.21.15.84:0/3683710915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559259e4bc40 msgr2=0x559259e496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:25.873 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.851+0000 7fc5a3fff640 1 --2- 172.21.15.84:0/3683710915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559259e4bc40 0x559259e496a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.873 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.851+0000 7fc5a3fff640 1 -- 172.21.15.84:0/3683710915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559259e40c50 msgr2=0x559259e48c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:25.873 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.851+0000 7fc5a3fff640 1 --2- 172.21.15.84:0/3683710915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559259e40c50 0x559259e48c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.874 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.851+0000 7fc5a3fff640 1 -- 172.21.15.84:0/3683710915 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559259b807e0 con 0x559259e41570 2024-05-22T23:09:25.874 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.852+0000 7fc5a3fff640 1 --2- 172.21.15.84:0/3683710915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e41570 0x559259e49160 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc594097d70 tx=0x7fc59409b580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9727400ac7208ac5 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:25.874 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.852+0000 7fc58affd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:09:25.874 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.852+0000 7fc58affd640 1 -- 172.21.15.84:0/3683710915 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59401d070 con 0x559259e41570 2024-05-22T23:09:25.874 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.852+0000 7fc58affd640 1 -- 172.21.15.84:0/3683710915 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7fc594007ca0 con 0x559259e41570 2024-05-22T23:09:25.874 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.852+0000 7fc5b4c83740 1 -- 172.21.15.84:0/3683710915 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc59c015070 con 0x559259e41570 2024-05-22T23:09:25.874 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.852+0000 7fc58affd640 1 -- 172.21.15.84:0/3683710915 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fc5940a44d0 con 0x559259e41570 2024-05-22T23:09:25.874 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.852+0000 7fc5b4c83740 10 client.16614 fetch_fsmap learned FSMap version 18 2024-05-22T23:09:25.875 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.852+0000 7fc5b4c83740 1 -- 172.21.15.84:0/3683710915 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x559259be0f70 con 0x559259e41570 2024-05-22T23:09:25.875 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.852+0000 7fc58affd640 1 -- 172.21.15.84:0/3683710915 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc59c015070 con 0x559259e41570 2024-05-22T23:09:25.875 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.852+0000 7fc58affd640 1 -- 172.21.15.84:0/3683710915 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fc594086100 con 0x559259e41570 2024-05-22T23:09:25.875 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.852+0000 7fc5b4c83740 10 client.16614 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:09:25.875 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.852+0000 7fc5b4c83740 10 client.16614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:09:25.875 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.852+0000 7fc5b4c83740 20 client.16614 populate_metadata read hostname 'smithi084' 2024-05-22T23:09:25.875 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.852+0000 7fc5b4c83740 1 --2- 172.21.15.84:0/3683710915 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x559259e5c030 0x559259e5e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:25.875 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.852+0000 7fc5b4c83740 4 client.16614 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:09:25.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.852+0000 7fc5b4c83740 1 -- 172.21.15.84:0/3683710915 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559259db8460 con 0x559259e5c030 2024-05-22T23:09:25.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.853+0000 7fc5ab7fe640 1 --2- 172.21.15.84:0/3683710915 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x559259e5c030 0x559259e5e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:25.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.853+0000 7fc58affd640 10 client.16614 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:25.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.853+0000 7fc5ab7fe640 1 --2- 172.21.15.84:0/3683710915 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x559259e5c030 0x559259e5e420 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c377c674cb4b414 server_cookie=20fd64ccb3d46dc0 in_seq=0 out_seq=0 2024-05-22T23:09:25.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.854+0000 7fc58affd640 1 -- 172.21.15.84:0/3683710915 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559259db8460 con 0x559259e5c030 2024-05-22T23:09:25.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.854+0000 7fc58affd640 10 client.16614 handle_command_reply: tid=0 2024-05-22T23:09:25.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.854+0000 7fc5b4c83740 1 client.16614 shutdown 2024-05-22T23:09:25.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.854+0000 7fc5b4c83740 1 -- 172.21.15.84:0/3683710915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e41570 msgr2=0x559259e49160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:25.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.854+0000 7fc5b4c83740 1 --2- 172.21.15.84:0/3683710915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e41570 0x559259e49160 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc594097d70 tx=0x7fc59409b580 comp rx=0 tx=0).stop 2024-05-22T23:09:25.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.854+0000 7fc5b4c83740 1 -- 172.21.15.84:0/3683710915 shutdown_connections 2024-05-22T23:09:25.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.854+0000 7fc5b4c83740 1 --2- 172.21.15.84:0/3683710915 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x559259e5c030 0x559259e5e420 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.854+0000 7fc5b4c83740 1 --2- 172.21.15.84:0/3683710915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559259e4bc40 0x559259e496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.854+0000 7fc58affd640 0 client.16614 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:25.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.854+0000 7fc5b4c83740 1 --2- 172.21.15.84:0/3683710915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e41570 0x559259e49160 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.854+0000 7fc5b4c83740 1 --2- 172.21.15.84:0/3683710915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559259e40c50 0x559259e48c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.854+0000 7fc5b4c83740 1 -- 172.21.15.84:0/3683710915 >> 172.21.15.84:0/3683710915 conn(0x559259e3cf70 msgr2=0x559259e42d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:25.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.854+0000 7fc5b4c83740 1 -- 172.21.15.84:0/3683710915 shutdown_connections 2024-05-22T23:09:25.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.854+0000 7fc5b4c83740 1 -- 172.21.15.84:0/3683710915 wait complete. 2024-05-22T23:09:25.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.854+0000 7fc5b4c83740 20 client.16614 trim_cache size 0 max 16384 2024-05-22T23:09:25.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.871+0000 7fc5b4c83740 1 Processor -- start 2024-05-22T23:09:25.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.871+0000 7fc5b4c83740 1 -- start start 2024-05-22T23:09:25.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.871+0000 7fc5b4c83740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559259e40c50 0x559259e578f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:25.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.872+0000 7fc5b4c83740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559259e41570 0x559259e3c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:25.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.872+0000 7fc5b4c83740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e4bc40 0x559259e63ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:25.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.872+0000 7fc5b4c83740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac1ad900 con 0x559259e4bc40 2024-05-22T23:09:25.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.872+0000 7fc5b4c83740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac1b0160 con 0x559259e41570 2024-05-22T23:09:25.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.872+0000 7fc5b4c83740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac1adf50 con 0x559259e40c50 2024-05-22T23:09:25.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.872+0000 7fc5abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e4bc40 0x559259e63ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:25.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.872+0000 7fc5abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e4bc40 0x559259e63ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42062/0 (socket says 172.21.15.84:42062) 2024-05-22T23:09:25.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.872+0000 7fc5abfff640 1 -- 172.21.15.84:0/3248495085 learned_addr learned my addr 172.21.15.84:0/3248495085 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:25.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.872+0000 7fc5a3fff640 1 --2- 172.21.15.84:0/3248495085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559259e41570 0x559259e3c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:25.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.872+0000 7fc5ab7fe640 1 --2- 172.21.15.84:0/3248495085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559259e40c50 0x559259e578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:25.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.872+0000 7fc5abfff640 1 -- 172.21.15.84:0/3248495085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559259e40c50 msgr2=0x559259e578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:25.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.872+0000 7fc5abfff640 1 --2- 172.21.15.84:0/3248495085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559259e40c50 0x559259e578f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.872+0000 7fc5abfff640 1 -- 172.21.15.84:0/3248495085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559259e41570 msgr2=0x559259e3c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:25.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.872+0000 7fc5abfff640 1 --2- 172.21.15.84:0/3248495085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559259e41570 0x559259e3c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.872+0000 7fc5abfff640 1 -- 172.21.15.84:0/3248495085 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559259e413a0 con 0x559259e4bc40 2024-05-22T23:09:25.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.872+0000 7fc5a3fff640 1 --2- 172.21.15.84:0/3248495085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559259e41570 0x559259e3c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:09:25.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.872+0000 7fc5abfff640 1 --2- 172.21.15.84:0/3248495085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e4bc40 0x559259e63ca0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009a30 tx=0x7fc5a403a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f172e1c194bf103c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:25.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.872+0000 7fc5a17fa640 1 -- 172.21.15.84:0/3248495085 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4033040 con 0x559259e4bc40 2024-05-22T23:09:25.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.872+0000 7fc5a17fa640 1 -- 172.21.15.84:0/3248495085 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5a402eea0 con 0x559259e4bc40 2024-05-22T23:09:25.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.872+0000 7fc5a17fa640 1 -- 172.21.15.84:0/3248495085 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a402f640 con 0x559259e4bc40 2024-05-22T23:09:25.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.873+0000 7fc5b4c83740 1 -- 172.21.15.84:0/3248495085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e4bc40 msgr2=0x559259e63ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:25.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.873+0000 7fc5b4c83740 1 --2- 172.21.15.84:0/3248495085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e4bc40 0x559259e63ca0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009a30 tx=0x7fc5a403a040 comp rx=0 tx=0).stop 2024-05-22T23:09:25.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.873+0000 7fc5b4c83740 1 -- 172.21.15.84:0/3248495085 shutdown_connections 2024-05-22T23:09:25.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.873+0000 7fc5b4c83740 1 --2- 172.21.15.84:0/3248495085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e4bc40 0x559259e63ca0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.873+0000 7fc5b4c83740 1 --2- 172.21.15.84:0/3248495085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559259e41570 0x559259e3c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.873+0000 7fc5b4c83740 1 --2- 172.21.15.84:0/3248495085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559259e40c50 0x559259e578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.873+0000 7fc5b4c83740 1 -- 172.21.15.84:0/3248495085 >> 172.21.15.84:0/3248495085 conn(0x559259e3cf70 msgr2=0x559259e3b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:25.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.873+0000 7fc5b4c83740 1 -- 172.21.15.84:0/3248495085 shutdown_connections 2024-05-22T23:09:25.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.873+0000 7fc5b4c83740 1 -- 172.21.15.84:0/3248495085 wait complete. 2024-05-22T23:09:25.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.873+0000 7fc5b4c83740 1 -- start start 2024-05-22T23:09:25.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.874+0000 7fc5b4c83740 1 Processor -- start 2024-05-22T23:09:25.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.874+0000 7fc5b4c83740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559259e40c50 0x559259e53e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:25.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.874+0000 7fc5b4c83740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559259e41570 0x559259e54340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:25.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.874+0000 7fc5b4c83740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e4bc40 0x559259e50dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:25.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.874+0000 7fc5b4c83740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac1ae620 con 0x559259e4bc40 2024-05-22T23:09:25.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.874+0000 7fc5b4c83740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac1afa90 con 0x559259e40c50 2024-05-22T23:09:25.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.874+0000 7fc5b4c83740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac1aecf0 con 0x559259e41570 2024-05-22T23:09:25.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.874+0000 7fc5b4c83740 10 client.0 _ll_register_callbacks cb 0x7fc594004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:09:25.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.874+0000 7fc5abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e4bc40 0x559259e50dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:25.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.874+0000 7fc5abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e4bc40 0x559259e50dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42076/0 (socket says 172.21.15.84:42076) 2024-05-22T23:09:25.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.874+0000 7fc5abfff640 1 -- 172.21.15.84:0/2326993704 learned_addr learned my addr 172.21.15.84:0/2326993704 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:25.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.874+0000 7fc5ab7fe640 1 --2- 172.21.15.84:0/2326993704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559259e40c50 0x559259e53e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:25.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.874+0000 7fc5abfff640 1 -- 172.21.15.84:0/2326993704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559259e41570 msgr2=0x559259e54340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:25.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.874+0000 7fc5abfff640 1 --2- 172.21.15.84:0/2326993704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559259e41570 0x559259e54340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.874+0000 7fc5abfff640 1 -- 172.21.15.84:0/2326993704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559259e40c50 msgr2=0x559259e53e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:25.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.874+0000 7fc5abfff640 1 --2- 172.21.15.84:0/2326993704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559259e40c50 0x559259e53e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.874+0000 7fc5abfff640 1 -- 172.21.15.84:0/2326993704 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559259a78170 con 0x559259e4bc40 2024-05-22T23:09:25.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.874+0000 7fc5abfff640 1 --2- 172.21.15.84:0/2326993704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e4bc40 0x559259e50dd0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4005cc0 tx=0x7fc5a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798be8ebbaa3dd24 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:25.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.874+0000 7fc58affd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:09:25.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.874+0000 7fc58affd640 1 -- 172.21.15.84:0/2326993704 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4056030 con 0x559259e4bc40 2024-05-22T23:09:25.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.874+0000 7fc58affd640 1 -- 172.21.15.84:0/2326993704 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7fc5a404e070 con 0x559259e4bc40 2024-05-22T23:09:25.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.874+0000 7fc5b4c83740 1 -- 172.21.15.84:0/2326993704 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5a402eea0 con 0x559259e4bc40 2024-05-22T23:09:25.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.875+0000 7fc58affd640 1 -- 172.21.15.84:0/2326993704 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fc5a4033040 con 0x559259e4bc40 2024-05-22T23:09:25.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.875+0000 7fc5b4c83740 10 client.16620 fetch_fsmap learned FSMap version 18 2024-05-22T23:09:25.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.875+0000 7fc5b4c83740 1 -- 172.21.15.84:0/2326993704 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x559259d6d930 con 0x559259e4bc40 2024-05-22T23:09:25.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.875+0000 7fc58affd640 1 -- 172.21.15.84:0/2326993704 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5a402eea0 con 0x559259e4bc40 2024-05-22T23:09:25.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.875+0000 7fc58affd640 1 -- 172.21.15.84:0/2326993704 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fc5a4061020 con 0x559259e4bc40 2024-05-22T23:09:25.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.875+0000 7fc5b4c83740 10 client.16620 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:09:25.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.875+0000 7fc5b4c83740 10 client.16620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:09:25.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.875+0000 7fc5b4c83740 20 client.16620 populate_metadata read hostname 'smithi084' 2024-05-22T23:09:25.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.875+0000 7fc5b4c83740 1 --2- 172.21.15.84:0/2326993704 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x559259ec3340 0x559259ec5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:25.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.875+0000 7fc5b4c83740 4 client.16620 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-22T23:09:25.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.875+0000 7fc5b4c83740 1 -- 172.21.15.84:0/2326993704 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559259db2170 con 0x559259ec3340 2024-05-22T23:09:25.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.875+0000 7fc5ab7fe640 1 --2- 172.21.15.84:0/2326993704 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x559259ec3340 0x559259ec5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:25.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.876+0000 7fc5ab7fe640 1 --2- 172.21.15.84:0/2326993704 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x559259ec3340 0x559259ec5760 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f894ad386fa1ccf3 server_cookie=6e58669b2210b33c in_seq=0 out_seq=0 2024-05-22T23:09:25.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.876+0000 7fc58affd640 10 client.16620 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:25.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.877+0000 7fc58affd640 1 -- 172.21.15.84:0/2326993704 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559259db2170 con 0x559259ec3340 2024-05-22T23:09:25.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.877+0000 7fc58affd640 10 client.16620 handle_command_reply: tid=0 2024-05-22T23:09:25.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.877+0000 7fc5b4c83740 1 client.16620 shutdown 2024-05-22T23:09:25.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.877+0000 7fc5b4c83740 1 -- 172.21.15.84:0/2326993704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e4bc40 msgr2=0x559259e50dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:25.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.877+0000 7fc5b4c83740 1 --2- 172.21.15.84:0/2326993704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e4bc40 0x559259e50dd0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4005cc0 tx=0x7fc5a4004a40 comp rx=0 tx=0).stop 2024-05-22T23:09:25.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.877+0000 7fc5b4c83740 1 -- 172.21.15.84:0/2326993704 shutdown_connections 2024-05-22T23:09:25.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.877+0000 7fc5b4c83740 1 --2- 172.21.15.84:0/2326993704 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x559259ec3340 0x559259ec5760 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.877+0000 7fc5b4c83740 1 --2- 172.21.15.84:0/2326993704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559259e4bc40 0x559259e50dd0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.877+0000 7fc5b4c83740 1 --2- 172.21.15.84:0/2326993704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559259e41570 0x559259e54340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.877+0000 7fc5b4c83740 1 --2- 172.21.15.84:0/2326993704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559259e40c50 0x559259e53e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.877+0000 7fc5b4c83740 1 -- 172.21.15.84:0/2326993704 >> 172.21.15.84:0/2326993704 conn(0x559259e3cf70 msgr2=0x559259e3c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:25.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.877+0000 7fc58affd640 0 client.16620 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:25.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.877+0000 7fc5b4c83740 1 -- 172.21.15.84:0/2326993704 shutdown_connections 2024-05-22T23:09:25.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.877+0000 7fc5b4c83740 1 -- 172.21.15.84:0/2326993704 wait complete. 2024-05-22T23:09:25.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.877+0000 7fc5b4c83740 20 client.16620 trim_cache size 0 max 16384 2024-05-22T23:09:25.899 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:09:25.900 INFO:teuthology.orchestra.run.smithi084.stdout: "return_code": 0, 2024-05-22T23:09:25.900 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_tag": "54a967a9-03c4-495e-8af4-4f159efc7611", 2024-05-22T23:09:25.900 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": "asynchronous" 2024-05-22T23:09:25.900 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:09:25.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.877+0000 7fc5b1f70640 1 -- 172.21.15.84:0/387238220 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fc5980a2d30 msgr2=0x7fc5980a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:25.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.877+0000 7fc5b1f70640 1 --2- 172.21.15.84:0/387238220 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fc5980a2d30 0x7fc5980a51f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4005fe0 tx=0x7fc5a4005c50 comp rx=0 tx=0).stop 2024-05-22T23:09:25.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.877+0000 7fc5b1f70640 1 -- 172.21.15.84:0/387238220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5ac103410 msgr2=0x7fc5ac100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:25.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.877+0000 7fc5b1f70640 1 --2- 172.21.15.84:0/387238220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5ac103410 0x7fc5ac100c80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc59400c890 tx=0x7fc59400cd50 comp rx=0 tx=0).stop 2024-05-22T23:09:25.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.878+0000 7fc5b1f70640 1 -- 172.21.15.84:0/387238220 shutdown_connections 2024-05-22T23:09:25.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.878+0000 7fc5b1f70640 1 --2- 172.21.15.84:0/387238220 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fc5980a2d30 0x7fc5980a51f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.878+0000 7fc5b1f70640 1 --2- 172.21.15.84:0/387238220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5ac10be30 0x7fc5ac1011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.878+0000 7fc5b1f70640 1 --2- 172.21.15.84:0/387238220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5ac103410 0x7fc5ac100c80 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.878+0000 7fc5b1f70640 1 --2- 172.21.15.84:0/387238220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5ac102af0 0x7fc5ac100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:25.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.878+0000 7fc5b1f70640 1 -- 172.21.15.84:0/387238220 >> 172.21.15.84:0/387238220 conn(0x7fc5ac077870 msgr2=0x7fc5ac0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:25.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.878+0000 7fc5b1f70640 1 -- 172.21.15.84:0/387238220 shutdown_connections 2024-05-22T23:09:25.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:25.878+0000 7fc5b1f70640 1 -- 172.21.15.84:0/387238220 wait complete. 2024-05-22T23:09:25.903 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:09:25.903 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:09:26.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.019+0000 7f019495a640 1 -- 172.21.15.84:0/3472522553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0190101840 msgr2=0x7f0190101cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:26.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.019+0000 7f019495a640 1 --2- 172.21.15.84:0/3472522553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0190101840 0x7f0190101cc0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0184009930 tx=0x7f018402a420 comp rx=0 tx=0).stop 2024-05-22T23:09:26.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.019+0000 7f019495a640 1 -- 172.21.15.84:0/3472522553 shutdown_connections 2024-05-22T23:09:26.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.019+0000 7f019495a640 1 --2- 172.21.15.84:0/3472522553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0190102200 0x7f019010e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.019+0000 7f019495a640 1 --2- 172.21.15.84:0/3472522553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0190101840 0x7f0190101cc0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.019+0000 7f019495a640 1 --2- 172.21.15.84:0/3472522553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0190107850 0x7f0190107c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.019+0000 7f019495a640 1 -- 172.21.15.84:0/3472522553 >> 172.21.15.84:0/3472522553 conn(0x7f0190078cc0 msgr2=0x7f01900790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:26.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.019+0000 7f019495a640 1 -- 172.21.15.84:0/3472522553 shutdown_connections 2024-05-22T23:09:26.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.019+0000 7f019495a640 1 -- 172.21.15.84:0/3472522553 wait complete. 2024-05-22T23:09:26.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.019+0000 7f019495a640 1 Processor -- start 2024-05-22T23:09:26.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.019+0000 7f019495a640 1 -- start start 2024-05-22T23:09:26.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.020+0000 7f019495a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0190101840 0x7f01901981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.020+0000 7f019495a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0190102200 0x7f0190198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.020+0000 7f019495a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0190107850 0x7f019019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.020+0000 7f019495a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0190111c60 con 0x7f0190102200 2024-05-22T23:09:26.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.020+0000 7f019495a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0190111ae0 con 0x7f0190101840 2024-05-22T23:09:26.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.020+0000 7f019495a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0190111de0 con 0x7f0190107850 2024-05-22T23:09:26.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.020+0000 7f018dd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0190102200 0x7f0190198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.020+0000 7f018dd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0190102200 0x7f0190198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42102/0 (socket says 172.21.15.84:42102) 2024-05-22T23:09:26.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.020+0000 7f018dd9b640 1 -- 172.21.15.84:0/954001946 learned_addr learned my addr 172.21.15.84:0/954001946 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:26.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.020+0000 7f018e59c640 1 --2- 172.21.15.84:0/954001946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0190101840 0x7f01901981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.020+0000 7f018ed9d640 1 --2- 172.21.15.84:0/954001946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0190107850 0x7f019019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.020+0000 7f018dd9b640 1 -- 172.21.15.84:0/954001946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0190107850 msgr2=0x7f019019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:26.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.020+0000 7f018dd9b640 1 --2- 172.21.15.84:0/954001946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0190107850 0x7f019019ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.020+0000 7f018dd9b640 1 -- 172.21.15.84:0/954001946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0190101840 msgr2=0x7f01901981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:26.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.020+0000 7f018dd9b640 1 --2- 172.21.15.84:0/954001946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0190101840 0x7f01901981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.020+0000 7f018dd9b640 1 -- 172.21.15.84:0/954001946 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019019d1b0 con 0x7f0190102200 2024-05-22T23:09:26.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.020+0000 7f018dd9b640 1 --2- 172.21.15.84:0/954001946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0190102200 0x7f0190198700 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0184009230 tx=0x7f0184004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:26.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.020+0000 7f017f7fe640 1 -- 172.21.15.84:0/954001946 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018403f030 con 0x7f0190102200 2024-05-22T23:09:26.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.020+0000 7f019495a640 1 -- 172.21.15.84:0/954001946 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f019019d440 con 0x7f0190102200 2024-05-22T23:09:26.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.020+0000 7f017f7fe640 1 -- 172.21.15.84:0/954001946 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0184004440 con 0x7f0190102200 2024-05-22T23:09:26.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.020+0000 7f017f7fe640 1 -- 172.21.15.84:0/954001946 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0184035070 con 0x7f0190102200 2024-05-22T23:09:26.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.021+0000 7f019495a640 1 -- 172.21.15.84:0/954001946 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01901a4d30 con 0x7f0190102200 2024-05-22T23:09:26.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.023+0000 7f017f7fe640 1 -- 172.21.15.84:0/954001946 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0184042030 con 0x7f0190102200 2024-05-22T23:09:26.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.023+0000 7f0197664740 1 Processor -- start 2024-05-22T23:09:26.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.024+0000 7f0197664740 1 -- start start 2024-05-22T23:09:26.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.024+0000 7f0197664740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b6384a0 0x55873b6367e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.024+0000 7f0197664740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873b636db0 0x55873b637230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.024+0000 7f0197664740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b641b30 0x55873b643f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.024+0000 7f0197664740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55873b59adb0 con 0x55873b6384a0 2024-05-22T23:09:26.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.024+0000 7f0197664740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55873b539360 con 0x55873b636db0 2024-05-22T23:09:26.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.024+0000 7f0197664740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55873b596d70 con 0x55873b641b30 2024-05-22T23:09:26.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.024+0000 7f018ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b6384a0 0x55873b6367e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.024+0000 7f018ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b6384a0 0x55873b6367e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42114/0 (socket says 172.21.15.84:42114) 2024-05-22T23:09:26.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.024+0000 7f018ed9d640 1 -- 172.21.15.84:0/4235410861 learned_addr learned my addr 172.21.15.84:0/4235410861 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:26.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.024+0000 7f018e59c640 1 --2- 172.21.15.84:0/4235410861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873b636db0 0x55873b637230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.024+0000 7f018dd9b640 1 --2- 172.21.15.84:0/4235410861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b641b30 0x55873b643f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.024+0000 7f018ed9d640 1 -- 172.21.15.84:0/4235410861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b641b30 msgr2=0x55873b643f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:26.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.024+0000 7f018ed9d640 1 --2- 172.21.15.84:0/4235410861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b641b30 0x55873b643f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.024+0000 7f018ed9d640 1 -- 172.21.15.84:0/4235410861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873b636db0 msgr2=0x55873b637230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:26.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.024+0000 7f017f7fe640 1 --2- 172.21.15.84:0/954001946 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f01780a2d00 0x7f01780a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.024+0000 7f018ed9d640 1 --2- 172.21.15.84:0/4235410861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873b636db0 0x55873b637230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.024+0000 7f018ed9d640 1 -- 172.21.15.84:0/4235410861 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55873b55ca60 con 0x55873b6384a0 2024-05-22T23:09:26.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.024+0000 7f018e59c640 1 --2- 172.21.15.84:0/4235410861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873b636db0 0x55873b637230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:09:26.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.024+0000 7f018dd9b640 1 --2- 172.21.15.84:0/4235410861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b641b30 0x55873b643f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:09:26.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.025+0000 7f017f7fe640 1 -- 172.21.15.84:0/954001946 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7f01840ef760 con 0x7f0190102200 2024-05-22T23:09:26.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.025+0000 7f018ed9d640 1 --2- 172.21.15.84:0/954001946 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f01780a2d00 0x7f01780a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.025+0000 7f018ed9d640 1 --2- 172.21.15.84:0/4235410861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b6384a0 0x55873b6367e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0188009fd0 tx=0x7f01880118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf0a28220b830f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:26.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.025+0000 7f017cff9640 1 -- 172.21.15.84:0/4235410861 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0188004250 con 0x55873b6384a0 2024-05-22T23:09:26.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.025+0000 7f017cff9640 1 -- 172.21.15.84:0/4235410861 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0188003db0 con 0x55873b6384a0 2024-05-22T23:09:26.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.025+0000 7f017cff9640 1 -- 172.21.15.84:0/4235410861 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018801bc20 con 0x55873b6384a0 2024-05-22T23:09:26.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.026+0000 7f0197664740 1 -- 172.21.15.84:0/4235410861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b6384a0 msgr2=0x55873b6367e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:26.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.026+0000 7f018ed9d640 1 --2- 172.21.15.84:0/954001946 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f01780a2d00 0x7f01780a51c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0188020000 tx=0x7f0188021000 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:26.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.026+0000 7f0197664740 1 --2- 172.21.15.84:0/4235410861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b6384a0 0x55873b6367e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0188009fd0 tx=0x7f01880118d0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.026+0000 7f0197664740 1 -- 172.21.15.84:0/4235410861 shutdown_connections 2024-05-22T23:09:26.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.026+0000 7f0197664740 1 --2- 172.21.15.84:0/4235410861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b641b30 0x55873b643f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.026+0000 7f0197664740 1 --2- 172.21.15.84:0/4235410861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873b636db0 0x55873b637230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.026+0000 7f0197664740 1 --2- 172.21.15.84:0/4235410861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b6384a0 0x55873b6367e0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.026+0000 7f0197664740 1 -- 172.21.15.84:0/4235410861 >> 172.21.15.84:0/4235410861 conn(0x55873b5ff8e0 msgr2=0x55873b633310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:26.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.026+0000 7f0197664740 1 -- 172.21.15.84:0/4235410861 shutdown_connections 2024-05-22T23:09:26.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.026+0000 7f0197664740 1 -- 172.21.15.84:0/4235410861 wait complete. 2024-05-22T23:09:26.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.026+0000 7f0197664740 1 -- start start 2024-05-22T23:09:26.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.026+0000 7f0197664740 1 Processor -- start 2024-05-22T23:09:26.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.026+0000 7f0197664740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873b636db0 0x55873b63e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.026+0000 7f0197664740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b6384a0 0x55873b63ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.026+0000 7f0197664740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b641b30 0x55873b63f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.027+0000 7f0197664740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55873b6449c0 con 0x55873b641b30 2024-05-22T23:09:26.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.027+0000 7f0197664740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55873b5550f0 con 0x55873b636db0 2024-05-22T23:09:26.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.027+0000 7f0197664740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55873b644b40 con 0x55873b6384a0 2024-05-22T23:09:26.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.027+0000 7f0197664740 10 client.0 _ll_register_callbacks cb 0x55873b559f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:09:26.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.027+0000 7f018ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b641b30 0x55873b63f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.027+0000 7f018e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873b636db0 0x55873b63e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.027+0000 7f018ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b641b30 0x55873b63f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42128/0 (socket says 172.21.15.84:42128) 2024-05-22T23:09:26.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.027+0000 7f018ed9d640 1 -- 172.21.15.84:0/2128363301 learned_addr learned my addr 172.21.15.84:0/2128363301 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:26.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.027+0000 7f018e59c640 1 --2- 172.21.15.84:0/2128363301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873b636db0 0x55873b63e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:49858/0 (socket says 172.21.15.84:49858) 2024-05-22T23:09:26.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.027+0000 7f018dd9b640 1 --2- 172.21.15.84:0/2128363301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b6384a0 0x55873b63ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.027+0000 7f018e59c640 1 -- 172.21.15.84:0/2128363301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b6384a0 msgr2=0x55873b63ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:26.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.027+0000 7f018e59c640 1 --2- 172.21.15.84:0/2128363301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b6384a0 0x55873b63ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.027+0000 7f018e59c640 1 -- 172.21.15.84:0/2128363301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b641b30 msgr2=0x55873b63f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:26.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.027+0000 7f018e59c640 1 --2- 172.21.15.84:0/2128363301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b641b30 0x55873b63f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.027+0000 7f018e59c640 1 -- 172.21.15.84:0/2128363301 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55873b3787d0 con 0x55873b636db0 2024-05-22T23:09:26.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.027+0000 7f018dd9b640 1 --2- 172.21.15.84:0/2128363301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b6384a0 0x55873b63ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:09:26.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.027+0000 7f018e59c640 1 --2- 172.21.15.84:0/2128363301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873b636db0 0x55873b63e9e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f01800126b0 tx=0x7f0180012ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57a9df68c46eafdf server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:26.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.027+0000 7f0175ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:09:26.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.027+0000 7f0175ffb640 1 -- 172.21.15.84:0/2128363301 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0180004030 con 0x55873b636db0 2024-05-22T23:09:26.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.027+0000 7f0175ffb640 1 -- 172.21.15.84:0/2128363301 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7f01800041d0 con 0x55873b636db0 2024-05-22T23:09:26.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.027+0000 7f0197664740 1 -- 172.21.15.84:0/2128363301 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0188003db0 con 0x55873b636db0 2024-05-22T23:09:26.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.027+0000 7f0175ffb640 1 -- 172.21.15.84:0/2128363301 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f01800043d0 con 0x55873b636db0 2024-05-22T23:09:26.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.027+0000 7f0197664740 10 client.26186 fetch_fsmap learned FSMap version 18 2024-05-22T23:09:26.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.027+0000 7f0197664740 1 -- 172.21.15.84:0/2128363301 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55873b2ae3f0 con 0x55873b636db0 2024-05-22T23:09:26.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.028+0000 7f0175ffb640 1 -- 172.21.15.84:0/2128363301 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0188003db0 con 0x55873b636db0 2024-05-22T23:09:26.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.028+0000 7f0175ffb640 1 -- 172.21.15.84:0/2128363301 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f0180003930 con 0x55873b636db0 2024-05-22T23:09:26.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.028+0000 7f0197664740 10 client.26186 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:09:26.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.028+0000 7f0197664740 10 client.26186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:09:26.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.028+0000 7f0197664740 20 client.26186 populate_metadata read hostname 'smithi084' 2024-05-22T23:09:26.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.028+0000 7f0197664740 1 --2- 172.21.15.84:0/2128363301 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55873b651f10 0x55873b6543a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.028+0000 7f0197664740 4 client.26186 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:09:26.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.028+0000 7f0197664740 1 -- 172.21.15.84:0/2128363301 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55873b5a84e0 con 0x55873b651f10 2024-05-22T23:09:26.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.028+0000 7f018dd9b640 1 --2- 172.21.15.84:0/2128363301 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55873b651f10 0x55873b6543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.028+0000 7f0175ffb640 10 client.26186 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:26.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.028+0000 7f018dd9b640 1 --2- 172.21.15.84:0/2128363301 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55873b651f10 0x55873b6543a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d732f1b146720b9 server_cookie=6bd7f1d9ad2fec3c in_seq=0 out_seq=0 2024-05-22T23:09:26.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.029+0000 7f0175ffb640 1 -- 172.21.15.84:0/2128363301 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55873b5a84e0 con 0x55873b651f10 2024-05-22T23:09:26.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.029+0000 7f0175ffb640 10 client.26186 handle_command_reply: tid=0 2024-05-22T23:09:26.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.029+0000 7f0197664740 1 client.26186 shutdown 2024-05-22T23:09:26.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.029+0000 7f0197664740 1 -- 172.21.15.84:0/2128363301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873b636db0 msgr2=0x55873b63e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:26.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.029+0000 7f0197664740 1 --2- 172.21.15.84:0/2128363301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873b636db0 0x55873b63e9e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f01800126b0 tx=0x7f0180012ab0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.029+0000 7f0197664740 1 -- 172.21.15.84:0/2128363301 shutdown_connections 2024-05-22T23:09:26.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.029+0000 7f0197664740 1 --2- 172.21.15.84:0/2128363301 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55873b651f10 0x55873b6543a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.029+0000 7f0197664740 1 --2- 172.21.15.84:0/2128363301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b641b30 0x55873b63f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.029+0000 7f0197664740 1 --2- 172.21.15.84:0/2128363301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b6384a0 0x55873b63ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.029+0000 7f0175ffb640 0 client.26186 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:26.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.029+0000 7f0197664740 1 --2- 172.21.15.84:0/2128363301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873b636db0 0x55873b63e9e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.029+0000 7f0197664740 1 -- 172.21.15.84:0/2128363301 >> 172.21.15.84:0/2128363301 conn(0x55873b5ff8e0 msgr2=0x55873b639180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:26.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.029+0000 7f0197664740 1 -- 172.21.15.84:0/2128363301 shutdown_connections 2024-05-22T23:09:26.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.029+0000 7f0197664740 1 -- 172.21.15.84:0/2128363301 wait complete. 2024-05-22T23:09:26.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.029+0000 7f0197664740 20 client.26186 trim_cache size 0 max 16384 2024-05-22T23:09:26.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.047+0000 7f0197664740 1 Processor -- start 2024-05-22T23:09:26.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.047+0000 7f0197664740 1 -- start start 2024-05-22T23:09:26.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.047+0000 7f0197664740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b636db0 0x55873b64d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.047+0000 7f0197664740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873b6384a0 0x55873b659c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.047+0000 7f0197664740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b641b30 0x55873b65a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.047+0000 7f0197664740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f01901ab750 con 0x55873b636db0 2024-05-22T23:09:26.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.047+0000 7f0197664740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f01901adfb0 con 0x55873b6384a0 2024-05-22T23:09:26.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.047+0000 7f0197664740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f01901abda0 con 0x55873b641b30 2024-05-22T23:09:26.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.047+0000 7f018e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b636db0 0x55873b64d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.047+0000 7f018e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b636db0 0x55873b64d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42134/0 (socket says 172.21.15.84:42134) 2024-05-22T23:09:26.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.047+0000 7f018e59c640 1 -- 172.21.15.84:0/3488531045 learned_addr learned my addr 172.21.15.84:0/3488531045 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:26.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.047+0000 7f018e59c640 1 -- 172.21.15.84:0/3488531045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b641b30 msgr2=0x55873b65a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:26.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.047+0000 7f018ed9d640 1 --2- 172.21.15.84:0/3488531045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b641b30 0x55873b65a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.047+0000 7f018e59c640 1 --2- 172.21.15.84:0/3488531045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b641b30 0x55873b65a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.047+0000 7f018e59c640 1 -- 172.21.15.84:0/3488531045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873b6384a0 msgr2=0x55873b659c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:26.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.047+0000 7f018e59c640 1 --2- 172.21.15.84:0/3488531045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873b6384a0 0x55873b659c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.047+0000 7f018e59c640 1 -- 172.21.15.84:0/3488531045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55873b26e0f0 con 0x55873b636db0 2024-05-22T23:09:26.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.047+0000 7f018ed9d640 1 --2- 172.21.15.84:0/3488531045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b641b30 0x55873b65a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:09:26.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.047+0000 7f018e59c640 1 --2- 172.21.15.84:0/3488531045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b636db0 0x55873b64d8c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f01800059f0 tx=0x7f0180005600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ab844106433166 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:26.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.048+0000 7f0177fff640 1 -- 172.21.15.84:0/3488531045 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0180028070 con 0x55873b636db0 2024-05-22T23:09:26.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.048+0000 7f0177fff640 1 -- 172.21.15.84:0/3488531045 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0180003ad0 con 0x55873b636db0 2024-05-22T23:09:26.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.048+0000 7f0177fff640 1 -- 172.21.15.84:0/3488531045 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018001ca60 con 0x55873b636db0 2024-05-22T23:09:26.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.048+0000 7f0197664740 1 -- 172.21.15.84:0/3488531045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b636db0 msgr2=0x55873b64d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:26.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.048+0000 7f0197664740 1 --2- 172.21.15.84:0/3488531045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b636db0 0x55873b64d8c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f01800059f0 tx=0x7f0180005600 comp rx=0 tx=0).stop 2024-05-22T23:09:26.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.048+0000 7f0197664740 1 -- 172.21.15.84:0/3488531045 shutdown_connections 2024-05-22T23:09:26.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.048+0000 7f0197664740 1 --2- 172.21.15.84:0/3488531045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b641b30 0x55873b65a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.048+0000 7f0197664740 1 --2- 172.21.15.84:0/3488531045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873b6384a0 0x55873b659c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.048+0000 7f0197664740 1 --2- 172.21.15.84:0/3488531045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b636db0 0x55873b64d8c0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.048+0000 7f0197664740 1 -- 172.21.15.84:0/3488531045 >> 172.21.15.84:0/3488531045 conn(0x55873b5ff8e0 msgr2=0x55873b63d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:26.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.048+0000 7f0197664740 1 -- 172.21.15.84:0/3488531045 shutdown_connections 2024-05-22T23:09:26.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.048+0000 7f0197664740 1 -- 172.21.15.84:0/3488531045 wait complete. 2024-05-22T23:09:26.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.049+0000 7f0197664740 1 -- start start 2024-05-22T23:09:26.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.049+0000 7f0197664740 1 Processor -- start 2024-05-22T23:09:26.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.049+0000 7f0197664740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b636db0 0x55873b64a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.049+0000 7f0197664740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873b6384a0 0x55873b63b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.049+0000 7f0197664740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b641b30 0x55873b63b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.049+0000 7f0197664740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f01901ac470 con 0x55873b636db0 2024-05-22T23:09:26.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.049+0000 7f0197664740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f01901ad8e0 con 0x55873b6384a0 2024-05-22T23:09:26.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.049+0000 7f0197664740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f01901acb40 con 0x55873b641b30 2024-05-22T23:09:26.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.049+0000 7f0197664740 10 client.0 _ll_register_callbacks cb 0x7f0180009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:09:26.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.049+0000 7f018e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b636db0 0x55873b64a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.049+0000 7f018e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b636db0 0x55873b64a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42146/0 (socket says 172.21.15.84:42146) 2024-05-22T23:09:26.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.049+0000 7f018e59c640 1 -- 172.21.15.84:0/2624610868 learned_addr learned my addr 172.21.15.84:0/2624610868 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:26.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.049+0000 7f018ed9d640 1 --2- 172.21.15.84:0/2624610868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b641b30 0x55873b63b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.049+0000 7f018e59c640 1 -- 172.21.15.84:0/2624610868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b641b30 msgr2=0x55873b63b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:26.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.049+0000 7f018e59c640 1 --2- 172.21.15.84:0/2624610868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b641b30 0x55873b63b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.049+0000 7f018e59c640 1 -- 172.21.15.84:0/2624610868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873b6384a0 msgr2=0x55873b63b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:26.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.049+0000 7f018dd9b640 1 --2- 172.21.15.84:0/2624610868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873b6384a0 0x55873b63b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.049+0000 7f018e59c640 1 --2- 172.21.15.84:0/2624610868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873b6384a0 0x55873b63b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.049+0000 7f018e59c640 1 -- 172.21.15.84:0/2624610868 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55873b559fe0 con 0x55873b636db0 2024-05-22T23:09:26.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.049+0000 7f018ed9d640 1 --2- 172.21.15.84:0/2624610868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b641b30 0x55873b63b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:09:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.049+0000 7f018dd9b640 1 --2- 172.21.15.84:0/2624610868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873b6384a0 0x55873b63b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:09:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.049+0000 7f018e59c640 1 --2- 172.21.15.84:0/2624610868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b636db0 0x55873b64a030 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f0180005840 tx=0x7f018001c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73b6a85beb1d3d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.049+0000 7f0175ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:09:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.049+0000 7f0175ffb640 1 -- 172.21.15.84:0/2624610868 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0180028040 con 0x55873b636db0 2024-05-22T23:09:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.050+0000 7f0175ffb640 1 -- 172.21.15.84:0/2624610868 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7f0180003800 con 0x55873b636db0 2024-05-22T23:09:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.050+0000 7f0197664740 1 -- 172.21.15.84:0/2624610868 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0180003ad0 con 0x55873b636db0 2024-05-22T23:09:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.050+0000 7f0175ffb640 1 -- 172.21.15.84:0/2624610868 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f018003b050 con 0x55873b636db0 2024-05-22T23:09:26.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.050+0000 7f0197664740 10 client.16638 fetch_fsmap learned FSMap version 18 2024-05-22T23:09:26.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.050+0000 7f0197664740 1 -- 172.21.15.84:0/2624610868 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55873b52b1c0 con 0x55873b636db0 2024-05-22T23:09:26.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.050+0000 7f0175ffb640 1 -- 172.21.15.84:0/2624610868 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0180003ad0 con 0x55873b636db0 2024-05-22T23:09:26.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.050+0000 7f0175ffb640 1 -- 172.21.15.84:0/2624610868 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f018000fbd0 con 0x55873b636db0 2024-05-22T23:09:26.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.050+0000 7f0197664740 10 client.16638 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:09:26.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.050+0000 7f0197664740 10 client.16638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:09:26.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.050+0000 7f0197664740 20 client.16638 populate_metadata read hostname 'smithi084' 2024-05-22T23:09:26.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.050+0000 7f0197664740 1 --2- 172.21.15.84:0/2624610868 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55873b6b85a0 0x55873b6ba990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.050+0000 7f0197664740 4 client.16638 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:09:26.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.050+0000 7f0197664740 1 -- 172.21.15.84:0/2624610868 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55873b5a2830 con 0x55873b6b85a0 2024-05-22T23:09:26.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.050+0000 7f018dd9b640 1 --2- 172.21.15.84:0/2624610868 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55873b6b85a0 0x55873b6ba990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.051+0000 7f0175ffb640 10 client.16638 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:26.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.051+0000 7f018dd9b640 1 --2- 172.21.15.84:0/2624610868 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55873b6b85a0 0x55873b6ba990 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39a55a1cc6e9a1d0 server_cookie=f64c8936dc5a9dad in_seq=0 out_seq=0 2024-05-22T23:09:26.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.051+0000 7f0175ffb640 1 -- 172.21.15.84:0/2624610868 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55873b5a2830 con 0x55873b6b85a0 2024-05-22T23:09:26.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.051+0000 7f0175ffb640 10 client.16638 handle_command_reply: tid=0 2024-05-22T23:09:26.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.051+0000 7f0197664740 1 client.16638 shutdown 2024-05-22T23:09:26.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.051+0000 7f0197664740 1 -- 172.21.15.84:0/2624610868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b636db0 msgr2=0x55873b64a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:26.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.051+0000 7f0197664740 1 --2- 172.21.15.84:0/2624610868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b636db0 0x55873b64a030 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f0180005840 tx=0x7f018001c710 comp rx=0 tx=0).stop 2024-05-22T23:09:26.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.051+0000 7f0197664740 1 -- 172.21.15.84:0/2624610868 shutdown_connections 2024-05-22T23:09:26.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.051+0000 7f0197664740 1 --2- 172.21.15.84:0/2624610868 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55873b6b85a0 0x55873b6ba990 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.051+0000 7f0197664740 1 --2- 172.21.15.84:0/2624610868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55873b641b30 0x55873b63b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.051+0000 7f0175ffb640 0 client.16638 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:26.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.051+0000 7f0197664740 1 --2- 172.21.15.84:0/2624610868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873b6384a0 0x55873b63b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.051+0000 7f0197664740 1 --2- 172.21.15.84:0/2624610868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55873b636db0 0x55873b64a030 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.051+0000 7f0197664740 1 -- 172.21.15.84:0/2624610868 >> 172.21.15.84:0/2624610868 conn(0x55873b5ff8e0 msgr2=0x55873b639110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:26.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.051+0000 7f0197664740 1 -- 172.21.15.84:0/2624610868 shutdown_connections 2024-05-22T23:09:26.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.051+0000 7f0197664740 1 -- 172.21.15.84:0/2624610868 wait complete. 2024-05-22T23:09:26.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.051+0000 7f0197664740 20 client.16638 trim_cache size 0 max 16384 2024-05-22T23:09:26.085 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:09:26.085 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "no active scrubs running", 2024-05-22T23:09:26.085 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": {} 2024-05-22T23:09:26.085 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:09:26.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.052+0000 7f019495a640 1 -- 172.21.15.84:0/954001946 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f01780a2d00 msgr2=0x7f01780a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:26.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.052+0000 7f019495a640 1 --2- 172.21.15.84:0/954001946 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f01780a2d00 0x7f01780a51c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0188020000 tx=0x7f0188021000 comp rx=0 tx=0).stop 2024-05-22T23:09:26.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.052+0000 7f019495a640 1 -- 172.21.15.84:0/954001946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0190102200 msgr2=0x7f0190198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:26.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.052+0000 7f019495a640 1 --2- 172.21.15.84:0/954001946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0190102200 0x7f0190198700 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0184009230 tx=0x7f0184004040 comp rx=0 tx=0).stop 2024-05-22T23:09:26.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.052+0000 7f019495a640 1 -- 172.21.15.84:0/954001946 shutdown_connections 2024-05-22T23:09:26.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.052+0000 7f019495a640 1 --2- 172.21.15.84:0/954001946 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f01780a2d00 0x7f01780a51c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.052+0000 7f019495a640 1 --2- 172.21.15.84:0/954001946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0190107850 0x7f019019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.052+0000 7f019495a640 1 --2- 172.21.15.84:0/954001946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0190102200 0x7f0190198700 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.052+0000 7f019495a640 1 --2- 172.21.15.84:0/954001946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0190101840 0x7f01901981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.052+0000 7f019495a640 1 -- 172.21.15.84:0/954001946 >> 172.21.15.84:0/954001946 conn(0x7f0190078cc0 msgr2=0x7f01900ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:26.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.052+0000 7f019495a640 1 -- 172.21.15.84:0/954001946 shutdown_connections 2024-05-22T23:09:26.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.052+0000 7f019495a640 1 -- 172.21.15.84:0/954001946 wait complete. 2024-05-22T23:09:26.089 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-22T23:09:26.089 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:09:26.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.184+0000 7f6efa77d640 1 -- 172.21.15.84:0/2630724622 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ef406a8b0 msgr2=0x7f6ef410e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:26.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.184+0000 7f6efa77d640 1 --2- 172.21.15.84:0/2630724622 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ef406a8b0 0x7f6ef410e710 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f6eec009cd0 tx=0x7f6eec02a8e0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.184+0000 7f6efa77d640 1 -- 172.21.15.84:0/2630724622 shutdown_connections 2024-05-22T23:09:26.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.184+0000 7f6efa77d640 1 --2- 172.21.15.84:0/2630724622 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ef406a8b0 0x7f6ef410e710 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.184+0000 7f6efa77d640 1 --2- 172.21.15.84:0/2630724622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ef4069ef0 0x7f6ef406a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.184+0000 7f6efa77d640 1 --2- 172.21.15.84:0/2630724622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef4107850 0x7f6ef4107c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.184+0000 7f6efa77d640 1 -- 172.21.15.84:0/2630724622 >> 172.21.15.84:0/2630724622 conn(0x7f6ef4076570 msgr2=0x7f6ef4076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:26.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.184+0000 7f6efa77d640 1 -- 172.21.15.84:0/2630724622 shutdown_connections 2024-05-22T23:09:26.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.184+0000 7f6efa77d640 1 -- 172.21.15.84:0/2630724622 wait complete. 2024-05-22T23:09:26.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.185+0000 7f6efa77d640 1 Processor -- start 2024-05-22T23:09:26.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.185+0000 7f6efa77d640 1 -- start start 2024-05-22T23:09:26.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.185+0000 7f6efa77d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ef4069ef0 0x7f6ef4100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.185+0000 7f6efa77d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef406a8b0 0x7f6ef41010d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.185+0000 7f6efa77d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ef4107850 0x7f6ef4101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.185+0000 7f6efa77d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef4111c80 con 0x7f6ef4069ef0 2024-05-22T23:09:26.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.185+0000 7f6efa77d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef4111b00 con 0x7f6ef4107850 2024-05-22T23:09:26.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.185+0000 7f6efa77d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef4111e00 con 0x7f6ef406a8b0 2024-05-22T23:09:26.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.185+0000 7f6ef3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ef4069ef0 0x7f6ef4100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.185+0000 7f6ef3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ef4069ef0 0x7f6ef4100b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42168/0 (socket says 172.21.15.84:42168) 2024-05-22T23:09:26.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.185+0000 7f6ef3fff640 1 -- 172.21.15.84:0/2273132287 learned_addr learned my addr 172.21.15.84:0/2273132287 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:26.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.185+0000 7f6ef8d1a640 1 --2- 172.21.15.84:0/2273132287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ef4107850 0x7f6ef4101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.185+0000 7f6ef37fe640 1 --2- 172.21.15.84:0/2273132287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef406a8b0 0x7f6ef41010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.185+0000 7f6ef3fff640 1 -- 172.21.15.84:0/2273132287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef406a8b0 msgr2=0x7f6ef41010d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:26.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.185+0000 7f6ef3fff640 1 --2- 172.21.15.84:0/2273132287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef406a8b0 0x7f6ef41010d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.185+0000 7f6ef3fff640 1 -- 172.21.15.84:0/2273132287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ef4107850 msgr2=0x7f6ef4101610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:26.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.185+0000 7f6ef3fff640 1 --2- 172.21.15.84:0/2273132287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ef4107850 0x7f6ef4101610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.185+0000 7f6ef3fff640 1 -- 172.21.15.84:0/2273132287 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef41a6660 con 0x7f6ef4069ef0 2024-05-22T23:09:26.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.186+0000 7f6ef3fff640 1 --2- 172.21.15.84:0/2273132287 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ef4069ef0 0x7f6ef4100b90 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400d770 tx=0x7f6ee400db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:26.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.186+0000 7f6ef17fa640 1 -- 172.21.15.84:0/2273132287 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4017070 con 0x7f6ef4069ef0 2024-05-22T23:09:26.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.186+0000 7f6efa77d640 1 -- 172.21.15.84:0/2273132287 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef41a6950 con 0x7f6ef4069ef0 2024-05-22T23:09:26.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.186+0000 7f6ef17fa640 1 -- 172.21.15.84:0/2273132287 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ee4009820 con 0x7f6ef4069ef0 2024-05-22T23:09:26.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.186+0000 7f6ef17fa640 1 -- 172.21.15.84:0/2273132287 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4004cd0 con 0x7f6ef4069ef0 2024-05-22T23:09:26.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.186+0000 7f6efa77d640 1 -- 172.21.15.84:0/2273132287 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef41a6ea0 con 0x7f6ef4069ef0 2024-05-22T23:09:26.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.187+0000 7f6efa77d640 1 -- 172.21.15.84:0/2273132287 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef4102fa0 con 0x7f6ef4069ef0 2024-05-22T23:09:26.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.188+0000 7f6ef17fa640 1 -- 172.21.15.84:0/2273132287 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6ee40099d0 con 0x7f6ef4069ef0 2024-05-22T23:09:26.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.188+0000 7f6ef17fa640 1 --2- 172.21.15.84:0/2273132287 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6edc0a2d00 0x7f6edc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.189+0000 7f6ef37fe640 1 --2- 172.21.15.84:0/2273132287 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6edc0a2d00 0x7f6edc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.189+0000 7f6ef17fa640 1 -- 172.21.15.84:0/2273132287 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7f6ee40cee10 con 0x7f6ef4069ef0 2024-05-22T23:09:26.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.189+0000 7f6ef37fe640 1 --2- 172.21.15.84:0/2273132287 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6edc0a2d00 0x7f6edc0a51c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4102620 tx=0x7f6ee8009110 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:26.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.193+0000 7f6ef17fa640 1 -- 172.21.15.84:0/2273132287 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6ee409ba00 con 0x7f6ef4069ef0 2024-05-22T23:09:26.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:26 smithi084 ceph-mon[32186]: scrub queued for path: / 2024-05-22T23:09:26.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:26 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:09:26.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:26 smithi084 ceph-mon[32186]: scrub summary: active paths [/] 2024-05-22T23:09:26.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:26 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:09:26.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:26 smithi084 ceph-mon[32186]: scrub summary: idle 2024-05-22T23:09:26.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.407+0000 7f6efa77d640 1 -- 172.21.15.84:0/2273132287 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ef4109dd0 con 0x7f6ef4069ef0 2024-05-22T23:09:26.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.407+0000 7f6ef17fa640 1 -- 172.21.15.84:0/2273132287 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7f6ee4014e90 con 0x7f6ef4069ef0 2024-05-22T23:09:26.414 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:09:26.414 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:09:26.414 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:09:26.415 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:26 smithi204 ceph-mon[36211]: scrub queued for path: / 2024-05-22T23:09:26.415 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:26 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:09:26.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:26 smithi204 ceph-mon[36211]: scrub summary: active paths [/] 2024-05-22T23:09:26.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:26 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:09:26.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:26 smithi204 ceph-mon[36211]: scrub summary: idle 2024-05-22T23:09:26.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.410+0000 7f6efa77d640 1 -- 172.21.15.84:0/2273132287 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6edc0a2d00 msgr2=0x7f6edc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:26.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.410+0000 7f6efa77d640 1 --2- 172.21.15.84:0/2273132287 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6edc0a2d00 0x7f6edc0a51c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4102620 tx=0x7f6ee8009110 comp rx=0 tx=0).stop 2024-05-22T23:09:26.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.410+0000 7f6efa77d640 1 -- 172.21.15.84:0/2273132287 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ef4069ef0 msgr2=0x7f6ef4100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:26.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.410+0000 7f6efa77d640 1 --2- 172.21.15.84:0/2273132287 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ef4069ef0 0x7f6ef4100b90 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400d770 tx=0x7f6ee400db70 comp rx=0 tx=0).stop 2024-05-22T23:09:26.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.410+0000 7f6efa77d640 1 -- 172.21.15.84:0/2273132287 shutdown_connections 2024-05-22T23:09:26.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.410+0000 7f6efa77d640 1 --2- 172.21.15.84:0/2273132287 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6edc0a2d00 0x7f6edc0a51c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.410+0000 7f6efa77d640 1 --2- 172.21.15.84:0/2273132287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ef4107850 0x7f6ef4101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.410+0000 7f6efa77d640 1 --2- 172.21.15.84:0/2273132287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef406a8b0 0x7f6ef41010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.410+0000 7f6efa77d640 1 --2- 172.21.15.84:0/2273132287 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ef4069ef0 0x7f6ef4100b90 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.410+0000 7f6efa77d640 1 -- 172.21.15.84:0/2273132287 >> 172.21.15.84:0/2273132287 conn(0x7f6ef4076570 msgr2=0x7f6ef40fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:26.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.410+0000 7f6efa77d640 1 -- 172.21.15.84:0/2273132287 shutdown_connections 2024-05-22T23:09:26.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.410+0000 7f6efa77d640 1 -- 172.21.15.84:0/2273132287 wait complete. 2024-05-22T23:09:26.428 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-22T23:09:26.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.525+0000 7efcce658640 1 -- 172.21.15.84:0/1988334806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efcc8100410 msgr2=0x7efcc81007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:26.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.525+0000 7efcce658640 1 --2- 172.21.15.84:0/1988334806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efcc8100410 0x7efcc81007f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7efcb8009900 tx=0x7efcb802a410 comp rx=0 tx=0).stop 2024-05-22T23:09:26.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.525+0000 7efcce658640 1 -- 172.21.15.84:0/1988334806 shutdown_connections 2024-05-22T23:09:26.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.525+0000 7efcce658640 1 --2- 172.21.15.84:0/1988334806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcc8106bd0 0x7efcc810e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.525+0000 7efcce658640 1 --2- 172.21.15.84:0/1988334806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcc8100dc0 0x7efcc81064f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.525+0000 7efcce658640 1 --2- 172.21.15.84:0/1988334806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efcc8100410 0x7efcc81007f0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.525+0000 7efcce658640 1 -- 172.21.15.84:0/1988334806 >> 172.21.15.84:0/1988334806 conn(0x7efcc8075a00 msgr2=0x7efcc8075e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:26.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.525+0000 7efcce658640 1 -- 172.21.15.84:0/1988334806 shutdown_connections 2024-05-22T23:09:26.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.525+0000 7efcce658640 1 -- 172.21.15.84:0/1988334806 wait complete. 2024-05-22T23:09:26.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.526+0000 7efcce658640 1 Processor -- start 2024-05-22T23:09:26.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.526+0000 7efcce658640 1 -- start start 2024-05-22T23:09:26.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.526+0000 7efcce658640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efcc8100410 0x7efcc81982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.526+0000 7efcce658640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcc8106bd0 0x7efcc8198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.526+0000 7efcce658640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcc819cb80 0x7efcc819d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.526+0000 7efcce658640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efcc8111cc0 con 0x7efcc8100410 2024-05-22T23:09:26.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.526+0000 7efcce658640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efcc8111b40 con 0x7efcc819cb80 2024-05-22T23:09:26.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.526+0000 7efcce658640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efcc8111e40 con 0x7efcc8106bd0 2024-05-22T23:09:26.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.527+0000 7efcc7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efcc8100410 0x7efcc81982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.527+0000 7efcc7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efcc8100410 0x7efcc81982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42184/0 (socket says 172.21.15.84:42184) 2024-05-22T23:09:26.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.527+0000 7efcc7fff640 1 -- 172.21.15.84:0/824050829 learned_addr learned my addr 172.21.15.84:0/824050829 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:26.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.527+0000 7efcc77fe640 1 --2- 172.21.15.84:0/824050829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcc8106bd0 0x7efcc8198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.527+0000 7efcccbf5640 1 --2- 172.21.15.84:0/824050829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcc819cb80 0x7efcc819d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.527+0000 7efcc7fff640 1 -- 172.21.15.84:0/824050829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcc8106bd0 msgr2=0x7efcc8198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:26.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.527+0000 7efcc7fff640 1 --2- 172.21.15.84:0/824050829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcc8106bd0 0x7efcc8198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.527+0000 7efcc7fff640 1 -- 172.21.15.84:0/824050829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcc819cb80 msgr2=0x7efcc819d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:26.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.527+0000 7efcc7fff640 1 --2- 172.21.15.84:0/824050829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcc819cb80 0x7efcc819d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.527+0000 7efcc7fff640 1 -- 172.21.15.84:0/824050829 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc819d670 con 0x7efcc8100410 2024-05-22T23:09:26.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.527+0000 7efcc7fff640 1 --2- 172.21.15.84:0/824050829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efcc8100410 0x7efcc81982c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7efcb8006010 tx=0x7efcb80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:26.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.527+0000 7efcc57fa640 1 -- 172.21.15.84:0/824050829 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb803f030 con 0x7efcc8100410 2024-05-22T23:09:26.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.527+0000 7efcce658640 1 -- 172.21.15.84:0/824050829 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc819d900 con 0x7efcc8100410 2024-05-22T23:09:26.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.527+0000 7efcc57fa640 1 -- 172.21.15.84:0/824050829 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb8003ce0 con 0x7efcc8100410 2024-05-22T23:09:26.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.527+0000 7efcc57fa640 1 -- 172.21.15.84:0/824050829 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8035070 con 0x7efcc8100410 2024-05-22T23:09:26.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.527+0000 7efcce658640 1 -- 172.21.15.84:0/824050829 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc81a5120 con 0x7efcc8100410 2024-05-22T23:09:26.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.530+0000 7efcc57fa640 1 -- 172.21.15.84:0/824050829 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7efcb8042030 con 0x7efcc8100410 2024-05-22T23:09:26.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.530+0000 7efcd1365740 1 Processor -- start 2024-05-22T23:09:26.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.530+0000 7efcd1365740 1 -- start start 2024-05-22T23:09:26.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.530+0000 7efcd1365740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3d910f80 0x55cf3d911360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.530+0000 7efcd1365740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d9118a0 0x55cf3d91ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.530+0000 7efcd1365740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3d91bf70 0x55cf3d91e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.530+0000 7efcd1365740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55cf3d875db0 con 0x55cf3d9118a0 2024-05-22T23:09:26.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.530+0000 7efcd1365740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cf3d814400 con 0x55cf3d910f80 2024-05-22T23:09:26.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.530+0000 7efcd1365740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cf3d871d70 con 0x55cf3d91bf70 2024-05-22T23:09:26.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.530+0000 7efcc7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d9118a0 0x55cf3d91ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.530+0000 7efcc7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d9118a0 0x55cf3d91ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42196/0 (socket says 172.21.15.84:42196) 2024-05-22T23:09:26.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.530+0000 7efcc7fff640 1 -- 172.21.15.84:0/3883794365 learned_addr learned my addr 172.21.15.84:0/3883794365 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:26.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.530+0000 7efcc57fa640 1 --2- 172.21.15.84:0/824050829 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7efcb00a2d00 0x7efcb00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.530+0000 7efcc77fe640 1 --2- 172.21.15.84:0/3883794365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3d91bf70 0x55cf3d91e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.530+0000 7efcccbf5640 1 --2- 172.21.15.84:0/3883794365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3d910f80 0x55cf3d911360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.530+0000 7efcccbf5640 1 --2- 172.21.15.84:0/824050829 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7efcb00a2d00 0x7efcb00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.531+0000 7efcc57fa640 1 -- 172.21.15.84:0/824050829 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7efcb80ef620 con 0x7efcc8100410 2024-05-22T23:09:26.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.531+0000 7efcc7fff640 1 -- 172.21.15.84:0/3883794365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3d91bf70 msgr2=0x55cf3d91e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:26.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.531+0000 7efcc7fff640 1 --2- 172.21.15.84:0/3883794365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3d91bf70 0x55cf3d91e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.531+0000 7efcc7fff640 1 -- 172.21.15.84:0/3883794365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3d910f80 msgr2=0x55cf3d911360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:26.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.531+0000 7efcc7fff640 1 --2- 172.21.15.84:0/3883794365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3d910f80 0x55cf3d911360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.531+0000 7efcc7fff640 1 -- 172.21.15.84:0/3883794365 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf3d8062f0 con 0x55cf3d9118a0 2024-05-22T23:09:26.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.531+0000 7efcc7fff640 1 --2- 172.21.15.84:0/3883794365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d9118a0 0x55cf3d91ba30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7efcb80e72a0 tx=0x7efcb802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84dcdf376e14405f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:26.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.531+0000 7efcccbf5640 1 --2- 172.21.15.84:0/824050829 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7efcb00a2d00 0x7efcb00a51c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009370 tx=0x7efcc0009240 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:26.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.531+0000 7efcae7fc640 1 -- 172.21.15.84:0/3883794365 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8049030 con 0x55cf3d9118a0 2024-05-22T23:09:26.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.531+0000 7efcae7fc640 1 -- 172.21.15.84:0/3883794365 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb80bc270 con 0x55cf3d9118a0 2024-05-22T23:09:26.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.531+0000 7efcae7fc640 1 -- 172.21.15.84:0/3883794365 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb80bbdd0 con 0x55cf3d9118a0 2024-05-22T23:09:26.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.532+0000 7efcd1365740 1 -- 172.21.15.84:0/3883794365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d9118a0 msgr2=0x55cf3d91ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:26.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.532+0000 7efcd1365740 1 --2- 172.21.15.84:0/3883794365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d9118a0 0x55cf3d91ba30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7efcb80e72a0 tx=0x7efcb802a410 comp rx=0 tx=0).stop 2024-05-22T23:09:26.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.532+0000 7efcd1365740 1 -- 172.21.15.84:0/3883794365 shutdown_connections 2024-05-22T23:09:26.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.532+0000 7efcd1365740 1 --2- 172.21.15.84:0/3883794365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3d91bf70 0x55cf3d91e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.532+0000 7efcd1365740 1 --2- 172.21.15.84:0/3883794365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d9118a0 0x55cf3d91ba30 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.532+0000 7efcd1365740 1 --2- 172.21.15.84:0/3883794365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3d910f80 0x55cf3d911360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.532+0000 7efcd1365740 1 -- 172.21.15.84:0/3883794365 >> 172.21.15.84:0/3883794365 conn(0x55cf3d90d2a0 msgr2=0x55cf3d90d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:26.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.532+0000 7efcd1365740 1 -- 172.21.15.84:0/3883794365 shutdown_connections 2024-05-22T23:09:26.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.532+0000 7efcd1365740 1 -- 172.21.15.84:0/3883794365 wait complete. 2024-05-22T23:09:26.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.532+0000 7efcd1365740 1 -- start start 2024-05-22T23:09:26.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.532+0000 7efcd1365740 1 Processor -- start 2024-05-22T23:09:26.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.532+0000 7efcd1365740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3d910f80 0x55cf3d918fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.532+0000 7efcd1365740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3d9118a0 0x55cf3d919500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.532+0000 7efcd1365740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d91bf70 0x55cf3d919a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.532+0000 7efcd1365740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55cf3d91eb20 con 0x55cf3d91bf70 2024-05-22T23:09:26.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.532+0000 7efcd1365740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cf3d8302e0 con 0x55cf3d910f80 2024-05-22T23:09:26.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.532+0000 7efcd1365740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cf3d91eca0 con 0x55cf3d9118a0 2024-05-22T23:09:26.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.532+0000 7efcd1365740 10 client.0 _ll_register_callbacks cb 0x55cf3d839a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:09:26.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.533+0000 7efcccbf5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d91bf70 0x55cf3d919a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.533+0000 7efcccbf5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d91bf70 0x55cf3d919a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42212/0 (socket says 172.21.15.84:42212) 2024-05-22T23:09:26.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.533+0000 7efcccbf5640 1 -- 172.21.15.84:0/2415474912 learned_addr learned my addr 172.21.15.84:0/2415474912 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:26.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.533+0000 7efcc77fe640 1 --2- 172.21.15.84:0/2415474912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3d9118a0 0x55cf3d919500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.533+0000 7efcc7fff640 1 --2- 172.21.15.84:0/2415474912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3d910f80 0x55cf3d918fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.533+0000 7efcccbf5640 1 -- 172.21.15.84:0/2415474912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3d9118a0 msgr2=0x55cf3d919500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:26.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.533+0000 7efcccbf5640 1 --2- 172.21.15.84:0/2415474912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3d9118a0 0x55cf3d919500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.533+0000 7efcccbf5640 1 -- 172.21.15.84:0/2415474912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3d910f80 msgr2=0x55cf3d918fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:26.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.533+0000 7efcccbf5640 1 --2- 172.21.15.84:0/2415474912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3d910f80 0x55cf3d918fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.533+0000 7efcccbf5640 1 -- 172.21.15.84:0/2415474912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf3d5893f0 con 0x55cf3d91bf70 2024-05-22T23:09:26.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.533+0000 7efcc7fff640 1 --2- 172.21.15.84:0/2415474912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3d910f80 0x55cf3d918fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:09:26.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.533+0000 7efcccbf5640 1 --2- 172.21.15.84:0/2415474912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d91bf70 0x55cf3d919a40 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7efcc00092e0 tx=0x7efcc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a20fdb22f7fcf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:26.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.533+0000 7efca77fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:09:26.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.533+0000 7efca77fe640 1 -- 172.21.15.84:0/2415474912 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc0026020 con 0x55cf3d91bf70 2024-05-22T23:09:26.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.533+0000 7efca77fe640 1 -- 172.21.15.84:0/2415474912 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7efcc0022870 con 0x55cf3d91bf70 2024-05-22T23:09:26.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.533+0000 7efcd1365740 1 -- 172.21.15.84:0/2415474912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcb80bc270 con 0x55cf3d91bf70 2024-05-22T23:09:26.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.533+0000 7efca77fe640 1 -- 172.21.15.84:0/2415474912 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7efcc0031050 con 0x55cf3d91bf70 2024-05-22T23:09:26.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.533+0000 7efcd1365740 10 client.16656 fetch_fsmap learned FSMap version 18 2024-05-22T23:09:26.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.533+0000 7efcd1365740 1 -- 172.21.15.84:0/2415474912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55cf3d6537d0 con 0x55cf3d91bf70 2024-05-22T23:09:26.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.533+0000 7efca77fe640 1 -- 172.21.15.84:0/2415474912 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb80bc270 con 0x55cf3d91bf70 2024-05-22T23:09:26.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.534+0000 7efca77fe640 1 -- 172.21.15.84:0/2415474912 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7efcc000fa20 con 0x55cf3d91bf70 2024-05-22T23:09:26.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.534+0000 7efcd1365740 10 client.16656 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:09:26.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.534+0000 7efcd1365740 10 client.16656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:09:26.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.534+0000 7efcd1365740 20 client.16656 populate_metadata read hostname 'smithi084' 2024-05-22T23:09:26.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.534+0000 7efcd1365740 1 --2- 172.21.15.84:0/2415474912 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55cf3d92c0c0 0x55cf3d92e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.534+0000 7efcd1365740 4 client.16656 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:09:26.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.534+0000 7efcd1365740 1 -- 172.21.15.84:0/2415474912 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf3d8834e0 con 0x55cf3d92c0c0 2024-05-22T23:09:26.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.534+0000 7efcc7fff640 1 --2- 172.21.15.84:0/2415474912 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55cf3d92c0c0 0x55cf3d92e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.534+0000 7efca77fe640 10 client.16656 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:26.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.534+0000 7efcc7fff640 1 --2- 172.21.15.84:0/2415474912 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55cf3d92c0c0 0x55cf3d92e540 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48d07a93bafe3a6c server_cookie=776f4f387bb89296 in_seq=0 out_seq=0 2024-05-22T23:09:26.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.535+0000 7efca77fe640 1 -- 172.21.15.84:0/2415474912 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf3d8834e0 con 0x55cf3d92c0c0 2024-05-22T23:09:26.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.535+0000 7efca77fe640 10 client.16656 handle_command_reply: tid=0 2024-05-22T23:09:26.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.535+0000 7efcd1365740 1 client.16656 shutdown 2024-05-22T23:09:26.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.535+0000 7efcd1365740 1 -- 172.21.15.84:0/2415474912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d91bf70 msgr2=0x55cf3d919a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:26.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.535+0000 7efcd1365740 1 --2- 172.21.15.84:0/2415474912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d91bf70 0x55cf3d919a40 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7efcc00092e0 tx=0x7efcc0004a40 comp rx=0 tx=0).stop 2024-05-22T23:09:26.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.535+0000 7efcd1365740 1 -- 172.21.15.84:0/2415474912 shutdown_connections 2024-05-22T23:09:26.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.535+0000 7efcd1365740 1 --2- 172.21.15.84:0/2415474912 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55cf3d92c0c0 0x55cf3d92e540 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.535+0000 7efca77fe640 0 client.16656 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:26.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.535+0000 7efcd1365740 1 --2- 172.21.15.84:0/2415474912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d91bf70 0x55cf3d919a40 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.535+0000 7efcd1365740 1 --2- 172.21.15.84:0/2415474912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3d9118a0 0x55cf3d919500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.535+0000 7efcd1365740 1 --2- 172.21.15.84:0/2415474912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3d910f80 0x55cf3d918fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.535+0000 7efcd1365740 1 -- 172.21.15.84:0/2415474912 >> 172.21.15.84:0/2415474912 conn(0x55cf3d90d2a0 msgr2=0x55cf3d9130a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:26.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.535+0000 7efcd1365740 1 -- 172.21.15.84:0/2415474912 shutdown_connections 2024-05-22T23:09:26.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.535+0000 7efcd1365740 1 -- 172.21.15.84:0/2415474912 wait complete. 2024-05-22T23:09:26.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.536+0000 7efcd1365740 20 client.16656 trim_cache size 0 max 16384 2024-05-22T23:09:26.566 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:26 smithi096 ceph-mon[36392]: scrub queued for path: / 2024-05-22T23:09:26.566 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:26 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:09:26.567 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:26 smithi096 ceph-mon[36392]: scrub summary: active paths [/] 2024-05-22T23:09:26.567 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:26 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:09:26.567 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:26 smithi096 ceph-mon[36392]: scrub summary: idle 2024-05-22T23:09:26.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.549+0000 7efcd1365740 1 Processor -- start 2024-05-22T23:09:26.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.549+0000 7efcd1365740 1 -- start start 2024-05-22T23:09:26.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.549+0000 7efcd1365740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d910f80 0x55cf3d91b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.549+0000 7efcd1365740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3d9118a0 0x55cf3d90cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.549+0000 7efcd1365740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3d91bf70 0x55cf3d9137a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.549+0000 7efcd1365740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efcc81ab790 con 0x55cf3d910f80 2024-05-22T23:09:26.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.549+0000 7efcd1365740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efcc81adff0 con 0x55cf3d9118a0 2024-05-22T23:09:26.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.549+0000 7efcd1365740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efcc81abde0 con 0x55cf3d91bf70 2024-05-22T23:09:26.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.549+0000 7efcc7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d910f80 0x55cf3d91b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.549+0000 7efcc7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d910f80 0x55cf3d91b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42220/0 (socket says 172.21.15.84:42220) 2024-05-22T23:09:26.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.549+0000 7efcc7fff640 1 -- 172.21.15.84:0/3917270004 learned_addr learned my addr 172.21.15.84:0/3917270004 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:26.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.549+0000 7efcc77fe640 1 --2- 172.21.15.84:0/3917270004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3d9118a0 0x55cf3d90cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.549+0000 7efcc7fff640 1 -- 172.21.15.84:0/3917270004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3d91bf70 msgr2=0x55cf3d9137a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:26.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.549+0000 7efcc7fff640 1 --2- 172.21.15.84:0/3917270004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3d91bf70 0x55cf3d9137a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.549+0000 7efcc7fff640 1 -- 172.21.15.84:0/3917270004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3d9118a0 msgr2=0x55cf3d90cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:26.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.549+0000 7efcc7fff640 1 --2- 172.21.15.84:0/3917270004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3d9118a0 0x55cf3d90cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.549+0000 7efcc7fff640 1 -- 172.21.15.84:0/3917270004 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf3d90dc20 con 0x55cf3d910f80 2024-05-22T23:09:26.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.549+0000 7efcc7fff640 1 --2- 172.21.15.84:0/3917270004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d910f80 0x55cf3d91b6f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7efcb80bbb00 tx=0x7efcb8003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b594d427a7413d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:26.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.549+0000 7efcadffb640 1 -- 172.21.15.84:0/3917270004 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8049070 con 0x55cf3d910f80 2024-05-22T23:09:26.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.549+0000 7efcadffb640 1 -- 172.21.15.84:0/3917270004 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb80bc270 con 0x55cf3d910f80 2024-05-22T23:09:26.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.549+0000 7efcadffb640 1 -- 172.21.15.84:0/3917270004 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb80bbdd0 con 0x55cf3d910f80 2024-05-22T23:09:26.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.550+0000 7efcd1365740 1 -- 172.21.15.84:0/3917270004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d910f80 msgr2=0x55cf3d91b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:26.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.550+0000 7efcd1365740 1 --2- 172.21.15.84:0/3917270004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d910f80 0x55cf3d91b6f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7efcb80bbb00 tx=0x7efcb8003f50 comp rx=0 tx=0).stop 2024-05-22T23:09:26.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.550+0000 7efcd1365740 1 -- 172.21.15.84:0/3917270004 shutdown_connections 2024-05-22T23:09:26.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.550+0000 7efcd1365740 1 --2- 172.21.15.84:0/3917270004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3d91bf70 0x55cf3d9137a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.550+0000 7efcd1365740 1 --2- 172.21.15.84:0/3917270004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3d9118a0 0x55cf3d90cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.550+0000 7efcd1365740 1 --2- 172.21.15.84:0/3917270004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d910f80 0x55cf3d91b6f0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.550+0000 7efcd1365740 1 -- 172.21.15.84:0/3917270004 >> 172.21.15.84:0/3917270004 conn(0x55cf3d90d2a0 msgr2=0x55cf3d919810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:26.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.550+0000 7efcd1365740 1 -- 172.21.15.84:0/3917270004 shutdown_connections 2024-05-22T23:09:26.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.550+0000 7efcd1365740 1 -- 172.21.15.84:0/3917270004 wait complete. 2024-05-22T23:09:26.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.550+0000 7efcd1365740 1 -- start start 2024-05-22T23:09:26.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.551+0000 7efcd1365740 1 Processor -- start 2024-05-22T23:09:26.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.551+0000 7efcd1365740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3d910f80 0x55cf3d916000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.551+0000 7efcd1365740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3d9118a0 0x55cf3d916540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.551+0000 7efcd1365740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d91bf70 0x55cf3d995a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.551+0000 7efcd1365740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efcc81ac4b0 con 0x55cf3d91bf70 2024-05-22T23:09:26.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.551+0000 7efcd1365740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efcc81ad920 con 0x55cf3d910f80 2024-05-22T23:09:26.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.551+0000 7efcd1365740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efcc81acb80 con 0x55cf3d9118a0 2024-05-22T23:09:26.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.551+0000 7efcd1365740 10 client.0 _ll_register_callbacks cb 0x7efcc0014790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:09:26.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.551+0000 7efcccbf5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d91bf70 0x55cf3d995a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.551+0000 7efcccbf5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d91bf70 0x55cf3d995a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42236/0 (socket says 172.21.15.84:42236) 2024-05-22T23:09:26.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.551+0000 7efcc7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3d910f80 0x55cf3d916000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.551+0000 7efcccbf5640 1 -- 172.21.15.84:0/4120773288 learned_addr learned my addr 172.21.15.84:0/4120773288 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:26.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.551+0000 7efcccbf5640 1 -- 172.21.15.84:0/4120773288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3d9118a0 msgr2=0x55cf3d916540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:26.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.551+0000 7efcc77fe640 1 --2- 172.21.15.84:0/4120773288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3d9118a0 0x55cf3d916540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.551+0000 7efcccbf5640 1 --2- 172.21.15.84:0/4120773288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3d9118a0 0x55cf3d916540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.551+0000 7efcccbf5640 1 -- 172.21.15.84:0/4120773288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3d910f80 msgr2=0x55cf3d916000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:26.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.551+0000 7efcccbf5640 1 --2- 172.21.15.84:0/4120773288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3d910f80 0x55cf3d916000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.551+0000 7efcccbf5640 1 -- 172.21.15.84:0/4120773288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf3d995fd0 con 0x55cf3d91bf70 2024-05-22T23:09:26.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.551+0000 7efcc7fff640 1 --2- 172.21.15.84:0/4120773288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3d910f80 0x55cf3d916000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:09:26.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.551+0000 7efcccbf5640 1 --2- 172.21.15.84:0/4120773288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d91bf70 0x55cf3d995a90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009820 tx=0x7efcc00095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0995b518633f7b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:26.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.551+0000 7efca77fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:09:26.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.551+0000 7efca77fe640 1 -- 172.21.15.84:0/4120773288 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc0026040 con 0x55cf3d91bf70 2024-05-22T23:09:26.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.552+0000 7efca77fe640 1 -- 172.21.15.84:0/4120773288 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7efcc0004ac0 con 0x55cf3d91bf70 2024-05-22T23:09:26.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.552+0000 7efcd1365740 1 -- 172.21.15.84:0/4120773288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcb80bc270 con 0x55cf3d91bf70 2024-05-22T23:09:26.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.552+0000 7efca77fe640 1 -- 172.21.15.84:0/4120773288 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7efcc0043050 con 0x55cf3d91bf70 2024-05-22T23:09:26.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.552+0000 7efcd1365740 10 client.16662 fetch_fsmap learned FSMap version 18 2024-05-22T23:09:26.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.552+0000 7efcd1365740 1 -- 172.21.15.84:0/4120773288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55cf3d997110 con 0x55cf3d91bf70 2024-05-22T23:09:26.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.552+0000 7efca77fe640 1 -- 172.21.15.84:0/4120773288 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb80bc270 con 0x55cf3d91bf70 2024-05-22T23:09:26.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.552+0000 7efca77fe640 1 -- 172.21.15.84:0/4120773288 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7efcc000f440 con 0x55cf3d91bf70 2024-05-22T23:09:26.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.552+0000 7efcd1365740 10 client.16662 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:09:26.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.552+0000 7efcd1365740 10 client.16662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:09:26.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.552+0000 7efcd1365740 20 client.16662 populate_metadata read hostname 'smithi084' 2024-05-22T23:09:26.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.552+0000 7efcd1365740 1 --2- 172.21.15.84:0/4120773288 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55cf3d92c0c0 0x55cf3d9997e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:26.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.552+0000 7efcd1365740 4 client.16662 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-22T23:09:26.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.552+0000 7efcd1365740 1 -- 172.21.15.84:0/4120773288 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf3d87d830 con 0x55cf3d92c0c0 2024-05-22T23:09:26.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.552+0000 7efcc7fff640 1 --2- 172.21.15.84:0/4120773288 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55cf3d92c0c0 0x55cf3d9997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:26.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.553+0000 7efca77fe640 10 client.16662 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:26.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.553+0000 7efcc7fff640 1 --2- 172.21.15.84:0/4120773288 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55cf3d92c0c0 0x55cf3d9997e0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b2fe920e025a414 server_cookie=a401aad642e99947 in_seq=0 out_seq=0 2024-05-22T23:09:26.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.553+0000 7efca77fe640 1 -- 172.21.15.84:0/4120773288 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf3d87d830 con 0x55cf3d92c0c0 2024-05-22T23:09:26.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.553+0000 7efca77fe640 10 client.16662 handle_command_reply: tid=0 2024-05-22T23:09:26.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.553+0000 7efcd1365740 1 client.16662 shutdown 2024-05-22T23:09:26.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.553+0000 7efcd1365740 1 -- 172.21.15.84:0/4120773288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d91bf70 msgr2=0x55cf3d995a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:26.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.553+0000 7efcd1365740 1 --2- 172.21.15.84:0/4120773288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d91bf70 0x55cf3d995a90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009820 tx=0x7efcc00095e0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.553+0000 7efcd1365740 1 -- 172.21.15.84:0/4120773288 shutdown_connections 2024-05-22T23:09:26.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.553+0000 7efcd1365740 1 --2- 172.21.15.84:0/4120773288 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55cf3d92c0c0 0x55cf3d9997e0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.553+0000 7efcd1365740 1 --2- 172.21.15.84:0/4120773288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf3d91bf70 0x55cf3d995a90 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.553+0000 7efca77fe640 0 client.16662 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:26.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.553+0000 7efcd1365740 1 --2- 172.21.15.84:0/4120773288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf3d9118a0 0x55cf3d916540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.553+0000 7efcd1365740 1 --2- 172.21.15.84:0/4120773288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf3d910f80 0x55cf3d916000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.553+0000 7efcd1365740 1 -- 172.21.15.84:0/4120773288 >> 172.21.15.84:0/4120773288 conn(0x55cf3d90d2a0 msgr2=0x55cf3d90bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:26.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.553+0000 7efcd1365740 1 -- 172.21.15.84:0/4120773288 shutdown_connections 2024-05-22T23:09:26.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.553+0000 7efcd1365740 1 -- 172.21.15.84:0/4120773288 wait complete. 2024-05-22T23:09:26.585 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.553+0000 7efcd1365740 20 client.16662 trim_cache size 0 max 16384 2024-05-22T23:09:26.586 INFO:teuthology.orchestra.run.smithi084.stdout:[] 2024-05-22T23:09:26.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.553+0000 7efcce658640 1 -- 172.21.15.84:0/824050829 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7efcb00a2d00 msgr2=0x7efcb00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:26.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.554+0000 7efcce658640 1 --2- 172.21.15.84:0/824050829 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7efcb00a2d00 0x7efcb00a51c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009370 tx=0x7efcc0009240 comp rx=0 tx=0).stop 2024-05-22T23:09:26.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.554+0000 7efcce658640 1 -- 172.21.15.84:0/824050829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efcc8100410 msgr2=0x7efcc81982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:26.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.554+0000 7efcce658640 1 --2- 172.21.15.84:0/824050829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efcc8100410 0x7efcc81982c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7efcb8006010 tx=0x7efcb80042f0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.554+0000 7efcce658640 1 -- 172.21.15.84:0/824050829 shutdown_connections 2024-05-22T23:09:26.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.554+0000 7efcce658640 1 --2- 172.21.15.84:0/824050829 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7efcb00a2d00 0x7efcb00a51c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.554+0000 7efcce658640 1 --2- 172.21.15.84:0/824050829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcc819cb80 0x7efcc819d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.554+0000 7efcce658640 1 --2- 172.21.15.84:0/824050829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcc8106bd0 0x7efcc8198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.554+0000 7efcce658640 1 --2- 172.21.15.84:0/824050829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efcc8100410 0x7efcc81982c0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:26.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.554+0000 7efcce658640 1 -- 172.21.15.84:0/824050829 >> 172.21.15.84:0/824050829 conn(0x7efcc8075a00 msgr2=0x7efcc8107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:26.589 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.554+0000 7efcce658640 1 -- 172.21.15.84:0/824050829 shutdown_connections 2024-05-22T23:09:26.589 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:26.554+0000 7efcce658640 1 -- 172.21.15.84:0/824050829 wait complete. 2024-05-22T23:09:27.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:27 smithi084 ceph-mon[32186]: pgmap v235: 97 pgs: 97 active+clean; 467 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 2.3 MiB/s wr, 85 op/s 2024-05-22T23:09:27.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:27 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2273132287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:09:27.415 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:27 smithi204 ceph-mon[36211]: pgmap v235: 97 pgs: 97 active+clean; 467 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 2.3 MiB/s wr, 85 op/s 2024-05-22T23:09:27.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:27 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2273132287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:09:27.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:27 smithi096 ceph-mon[36392]: pgmap v235: 97 pgs: 97 active+clean; 467 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 2.3 MiB/s wr, 85 op/s 2024-05-22T23:09:27.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:27 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2273132287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:09:27.589 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-22T23:09:27.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:09:27.590 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-22T23:09:27.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.702+0000 7f6d2015a640 1 -- 172.21.15.84:0/2555758421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d18107860 msgr2=0x7f6d18107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:27.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.702+0000 7f6d2015a640 1 --2- 172.21.15.84:0/2555758421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d18107860 0x7f6d18107c40 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6d14009900 tx=0x7f6d1402a480 comp rx=0 tx=0).stop 2024-05-22T23:09:27.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.702+0000 7f6d2015a640 1 -- 172.21.15.84:0/2555758421 shutdown_connections 2024-05-22T23:09:27.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.702+0000 7f6d2015a640 1 --2- 172.21.15.84:0/2555758421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d1806a5c0 0x7f6d1810e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.702+0000 7f6d2015a640 1 --2- 172.21.15.84:0/2555758421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d18108180 0x7f6d18069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.702+0000 7f6d2015a640 1 --2- 172.21.15.84:0/2555758421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d18107860 0x7f6d18107c40 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.702+0000 7f6d2015a640 1 -- 172.21.15.84:0/2555758421 >> 172.21.15.84:0/2555758421 conn(0x7f6d18076570 msgr2=0x7f6d18076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:27.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.702+0000 7f6d2015a640 1 -- 172.21.15.84:0/2555758421 shutdown_connections 2024-05-22T23:09:27.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.702+0000 7f6d2015a640 1 -- 172.21.15.84:0/2555758421 wait complete. 2024-05-22T23:09:27.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.703+0000 7f6d2015a640 1 Processor -- start 2024-05-22T23:09:27.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.703+0000 7f6d2015a640 1 -- start start 2024-05-22T23:09:27.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.703+0000 7f6d2015a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d1806a5c0 0x7f6d1819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.703+0000 7f6d2015a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d18107860 0x7f6d1819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.703+0000 7f6d2015a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d18108180 0x7f6d1819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.703+0000 7f6d2015a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6d18111c30 con 0x7f6d18108180 2024-05-22T23:09:27.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.703+0000 7f6d2015a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6d18111ab0 con 0x7f6d1806a5c0 2024-05-22T23:09:27.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.703+0000 7f6d2015a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6d18111db0 con 0x7f6d18107860 2024-05-22T23:09:27.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.703+0000 7f6d1e6f7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d18108180 0x7f6d1819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.703+0000 7f6d1e6f7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d18108180 0x7f6d1819ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42244/0 (socket says 172.21.15.84:42244) 2024-05-22T23:09:27.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.703+0000 7f6d1e6f7640 1 -- 172.21.15.84:0/1757274153 learned_addr learned my addr 172.21.15.84:0/1757274153 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:27.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.703+0000 7f6d1def6640 1 --2- 172.21.15.84:0/1757274153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d1806a5c0 0x7f6d1819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.703+0000 7f6d1d6f5640 1 --2- 172.21.15.84:0/1757274153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d18107860 0x7f6d1819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.703+0000 7f6d1e6f7640 1 -- 172.21.15.84:0/1757274153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d18107860 msgr2=0x7f6d1819a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:27.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.703+0000 7f6d1e6f7640 1 --2- 172.21.15.84:0/1757274153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d18107860 0x7f6d1819a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.703+0000 7f6d1e6f7640 1 -- 172.21.15.84:0/1757274153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d1806a5c0 msgr2=0x7f6d1819a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:27.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.703+0000 7f6d1e6f7640 1 --2- 172.21.15.84:0/1757274153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d1806a5c0 0x7f6d1819a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.703+0000 7f6d1e6f7640 1 -- 172.21.15.84:0/1757274153 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d1819f380 con 0x7f6d18108180 2024-05-22T23:09:27.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.703+0000 7f6d1d6f5640 1 --2- 172.21.15.84:0/1757274153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d18107860 0x7f6d1819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:09:27.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.704+0000 7f6d1e6f7640 1 --2- 172.21.15.84:0/1757274153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d18108180 0x7f6d1819ec50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000cad0 tx=0x7f6d1000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:27.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.704+0000 7f6d0effd640 1 -- 172.21.15.84:0/1757274153 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1000ae20 con 0x7f6d18108180 2024-05-22T23:09:27.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.704+0000 7f6d2015a640 1 -- 172.21.15.84:0/1757274153 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d1819f670 con 0x7f6d18108180 2024-05-22T23:09:27.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.704+0000 7f6d0effd640 1 -- 172.21.15.84:0/1757274153 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d10005020 con 0x7f6d18108180 2024-05-22T23:09:27.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.704+0000 7f6d0effd640 1 -- 172.21.15.84:0/1757274153 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1000f6f0 con 0x7f6d18108180 2024-05-22T23:09:27.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.704+0000 7f6d2015a640 1 -- 172.21.15.84:0/1757274153 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d181a6f00 con 0x7f6d18108180 2024-05-22T23:09:27.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.706+0000 7f6d0effd640 1 -- 172.21.15.84:0/1757274153 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6d100051d0 con 0x7f6d18108180 2024-05-22T23:09:27.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.706+0000 7f6d22e64740 1 Processor -- start 2024-05-22T23:09:27.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.707+0000 7f6d22e64740 1 -- start start 2024-05-22T23:09:27.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.707+0000 7f6d22e64740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea057c50 0x560fea058030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.707+0000 7f6d22e64740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea058570 0x560fea062700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.707+0000 7f6d22e64740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fea062c40 0x560fea065030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.707+0000 7f6d22e64740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560fe9fbba40 con 0x560fea057c50 2024-05-22T23:09:27.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.707+0000 7f6d22e64740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560fe9f5a440 con 0x560fea062c40 2024-05-22T23:09:27.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.707+0000 7f6d22e64740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560fe9fb7a00 con 0x560fea058570 2024-05-22T23:09:27.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.707+0000 7f6d1d6f5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea057c50 0x560fea058030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.707+0000 7f6d0effd640 1 --2- 172.21.15.84:0/1757274153 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6d040a2d00 0x7f6d040a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.707+0000 7f6d1d6f5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea057c50 0x560fea058030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42250/0 (socket says 172.21.15.84:42250) 2024-05-22T23:09:27.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.707+0000 7f6d1d6f5640 1 -- 172.21.15.84:0/1782262145 learned_addr learned my addr 172.21.15.84:0/1782262145 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:27.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.707+0000 7f6d1def6640 1 --2- 172.21.15.84:0/1782262145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea058570 0x560fea062700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.707+0000 7f6d1e6f7640 1 --2- 172.21.15.84:0/1757274153 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6d040a2d00 0x7f6d040a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.707+0000 7f6d1d6f5640 1 --2- 172.21.15.84:0/1782262145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fea062c40 0x560fea065030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.707+0000 7f6d0effd640 1 -- 172.21.15.84:0/1757274153 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7f6d100cf540 con 0x7f6d18108180 2024-05-22T23:09:27.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.707+0000 7f6d1d6f5640 1 -- 172.21.15.84:0/1782262145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea058570 msgr2=0x560fea062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:27.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.707+0000 7f6d1d6f5640 1 --2- 172.21.15.84:0/1782262145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea058570 0x560fea062700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.707+0000 7f6d1d6f5640 1 -- 172.21.15.84:0/1782262145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fea062c40 msgr2=0x560fea065030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:27.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.707+0000 7f6d1d6f5640 1 --2- 172.21.15.84:0/1782262145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fea062c40 0x560fea065030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.707+0000 7f6d1d6f5640 1 -- 172.21.15.84:0/1782262145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fe9f642d0 con 0x560fea057c50 2024-05-22T23:09:27.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.708+0000 7f6d1d6f5640 1 --2- 172.21.15.84:0/1782262145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea057c50 0x560fea058030 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800bd10 tx=0x7f6d0800b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84bbe5be04e2f69e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:27.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.708+0000 7f6d1e6f7640 1 --2- 172.21.15.84:0/1757274153 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6d040a2d00 0x7f6d040a51c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6d1009b730 tx=0x7f6d10086040 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:27.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.708+0000 7f6d03fff640 1 -- 172.21.15.84:0/1782262145 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08012650 con 0x560fea057c50 2024-05-22T23:09:27.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.708+0000 7f6d03fff640 1 -- 172.21.15.84:0/1782262145 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d08012cd0 con 0x560fea057c50 2024-05-22T23:09:27.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.708+0000 7f6d03fff640 1 -- 172.21.15.84:0/1782262145 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08011c00 con 0x560fea057c50 2024-05-22T23:09:27.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.708+0000 7f6d22e64740 1 -- 172.21.15.84:0/1782262145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea057c50 msgr2=0x560fea058030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:27.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.708+0000 7f6d22e64740 1 --2- 172.21.15.84:0/1782262145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea057c50 0x560fea058030 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800bd10 tx=0x7f6d0800b490 comp rx=0 tx=0).stop 2024-05-22T23:09:27.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.709+0000 7f6d22e64740 1 -- 172.21.15.84:0/1782262145 shutdown_connections 2024-05-22T23:09:27.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.709+0000 7f6d22e64740 1 --2- 172.21.15.84:0/1782262145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fea062c40 0x560fea065030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.709+0000 7f6d22e64740 1 --2- 172.21.15.84:0/1782262145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea058570 0x560fea062700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.709+0000 7f6d22e64740 1 --2- 172.21.15.84:0/1782262145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea057c50 0x560fea058030 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.709+0000 7f6d22e64740 1 -- 172.21.15.84:0/1782262145 >> 172.21.15.84:0/1782262145 conn(0x560fea053f70 msgr2=0x560fea054350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:27.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.709+0000 7f6d22e64740 1 -- 172.21.15.84:0/1782262145 shutdown_connections 2024-05-22T23:09:27.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.709+0000 7f6d22e64740 1 -- 172.21.15.84:0/1782262145 wait complete. 2024-05-22T23:09:27.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.709+0000 7f6d22e64740 1 -- start start 2024-05-22T23:09:27.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.709+0000 7f6d22e64740 1 Processor -- start 2024-05-22T23:09:27.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.709+0000 7f6d22e64740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fea057c50 0x560fea05fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.709+0000 7f6d22e64740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea058570 0x560fea060190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.709+0000 7f6d22e64740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea062c40 0x560fea0606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.709+0000 7f6d22e64740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560fea065a30 con 0x560fea062c40 2024-05-22T23:09:27.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.709+0000 7f6d22e64740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560fe9f455f0 con 0x560fea057c50 2024-05-22T23:09:27.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.709+0000 7f6d22e64740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560fea065bb0 con 0x560fea058570 2024-05-22T23:09:27.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.710+0000 7f6d22e64740 10 client.0 _ll_register_callbacks cb 0x560fe9f40cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:09:27.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.710+0000 7f6d1d6f5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea062c40 0x560fea0606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.710+0000 7f6d1d6f5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea062c40 0x560fea0606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42264/0 (socket says 172.21.15.84:42264) 2024-05-22T23:09:27.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.710+0000 7f6d1d6f5640 1 -- 172.21.15.84:0/1616374019 learned_addr learned my addr 172.21.15.84:0/1616374019 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:27.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.710+0000 7f6d1def6640 1 --2- 172.21.15.84:0/1616374019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea058570 0x560fea060190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.710+0000 7f6d1d6f5640 1 --2- 172.21.15.84:0/1616374019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fea057c50 0x560fea05fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.710+0000 7f6d1d6f5640 1 -- 172.21.15.84:0/1616374019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea058570 msgr2=0x560fea060190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:27.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.710+0000 7f6d1d6f5640 1 --2- 172.21.15.84:0/1616374019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea058570 0x560fea060190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.710+0000 7f6d1d6f5640 1 -- 172.21.15.84:0/1616374019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fea057c50 msgr2=0x560fea05fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:27.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.710+0000 7f6d1d6f5640 1 --2- 172.21.15.84:0/1616374019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fea057c50 0x560fea05fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.710+0000 7f6d1d6f5640 1 -- 172.21.15.84:0/1616374019 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fe9d977e0 con 0x560fea062c40 2024-05-22T23:09:27.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.710+0000 7f6d1d6f5640 1 --2- 172.21.15.84:0/1616374019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea062c40 0x560fea0606d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6d08005bf0 tx=0x7f6d08011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0b7c178fd49499 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:27.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.710+0000 7f6d017fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:09:27.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.710+0000 7f6d017fa640 1 -- 172.21.15.84:0/1616374019 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08012450 con 0x560fea062c40 2024-05-22T23:09:27.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.710+0000 7f6d017fa640 1 -- 172.21.15.84:0/1616374019 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7f6d080125f0 con 0x560fea062c40 2024-05-22T23:09:27.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.710+0000 7f6d22e64740 1 -- 172.21.15.84:0/1616374019 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d08012cd0 con 0x560fea062c40 2024-05-22T23:09:27.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.710+0000 7f6d017fa640 1 -- 172.21.15.84:0/1616374019 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f6d080127f0 con 0x560fea062c40 2024-05-22T23:09:27.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.710+0000 7f6d22e64740 10 client.16674 fetch_fsmap learned FSMap version 18 2024-05-22T23:09:27.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.710+0000 7f6d22e64740 1 -- 172.21.15.84:0/1616374019 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x560fe9df7f70 con 0x560fea062c40 2024-05-22T23:09:27.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.711+0000 7f6d017fa640 1 -- 172.21.15.84:0/1616374019 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d08012cd0 con 0x560fea062c40 2024-05-22T23:09:27.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.711+0000 7f6d017fa640 1 -- 172.21.15.84:0/1616374019 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f6d0801acb0 con 0x560fea062c40 2024-05-22T23:09:27.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.711+0000 7f6d22e64740 10 client.16674 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:09:27.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.711+0000 7f6d22e64740 10 client.16674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:09:27.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.711+0000 7f6d22e64740 20 client.16674 populate_metadata read hostname 'smithi084' 2024-05-22T23:09:27.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.711+0000 7f6d22e64740 1 --2- 172.21.15.84:0/1616374019 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x560fea072f20 0x560fea075310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.711+0000 7f6d22e64740 4 client.16674 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:09:27.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.711+0000 7f6d22e64740 1 -- 172.21.15.84:0/1616374019 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fe9fcf460 con 0x560fea072f20 2024-05-22T23:09:27.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.711+0000 7f6d1def6640 1 --2- 172.21.15.84:0/1616374019 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x560fea072f20 0x560fea075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.712+0000 7f6d1def6640 1 --2- 172.21.15.84:0/1616374019 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x560fea072f20 0x560fea075310 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1df2c843e4aadba server_cookie=41f6e2c6e76a6b23 in_seq=0 out_seq=0 2024-05-22T23:09:27.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.712+0000 7f6d017fa640 10 client.16674 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:27.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.712+0000 7f6d017fa640 1 -- 172.21.15.84:0/1616374019 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560fe9fcf460 con 0x560fea072f20 2024-05-22T23:09:27.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.712+0000 7f6d017fa640 10 client.16674 handle_command_reply: tid=0 2024-05-22T23:09:27.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.712+0000 7f6d22e64740 1 client.16674 shutdown 2024-05-22T23:09:27.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.712+0000 7f6d22e64740 1 -- 172.21.15.84:0/1616374019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea062c40 msgr2=0x560fea0606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:27.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.712+0000 7f6d22e64740 1 --2- 172.21.15.84:0/1616374019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea062c40 0x560fea0606d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6d08005bf0 tx=0x7f6d08011900 comp rx=0 tx=0).stop 2024-05-22T23:09:27.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.713+0000 7f6d22e64740 1 -- 172.21.15.84:0/1616374019 shutdown_connections 2024-05-22T23:09:27.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.713+0000 7f6d22e64740 1 --2- 172.21.15.84:0/1616374019 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x560fea072f20 0x560fea075310 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.713+0000 7f6d22e64740 1 --2- 172.21.15.84:0/1616374019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea062c40 0x560fea0606d0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.713+0000 7f6d017fa640 0 client.16674 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:27.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.713+0000 7f6d22e64740 1 --2- 172.21.15.84:0/1616374019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea058570 0x560fea060190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.713+0000 7f6d22e64740 1 --2- 172.21.15.84:0/1616374019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fea057c50 0x560fea05fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.713+0000 7f6d22e64740 1 -- 172.21.15.84:0/1616374019 >> 172.21.15.84:0/1616374019 conn(0x560fea053f70 msgr2=0x560fea059d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:27.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.713+0000 7f6d22e64740 1 -- 172.21.15.84:0/1616374019 shutdown_connections 2024-05-22T23:09:27.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.713+0000 7f6d22e64740 1 -- 172.21.15.84:0/1616374019 wait complete. 2024-05-22T23:09:27.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.713+0000 7f6d22e64740 20 client.16674 trim_cache size 0 max 16384 2024-05-22T23:09:27.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.730+0000 7f6d22e64740 1 Processor -- start 2024-05-22T23:09:27.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.730+0000 7f6d22e64740 1 -- start start 2024-05-22T23:09:27.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.731+0000 7f6d22e64740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea057c50 0x560fea06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.731+0000 7f6d22e64740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea058570 0x560fea05f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.731+0000 7f6d22e64740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fea062c40 0x560fea07aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.731+0000 7f6d22e64740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6d181ad8d0 con 0x560fea057c50 2024-05-22T23:09:27.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.731+0000 7f6d22e64740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6d181b0130 con 0x560fea062c40 2024-05-22T23:09:27.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.731+0000 7f6d22e64740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6d181adf20 con 0x560fea058570 2024-05-22T23:09:27.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.731+0000 7f6d1d6f5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea057c50 0x560fea06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.731+0000 7f6d1d6f5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea057c50 0x560fea06e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42268/0 (socket says 172.21.15.84:42268) 2024-05-22T23:09:27.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.731+0000 7f6d1d6f5640 1 -- 172.21.15.84:0/4086586477 learned_addr learned my addr 172.21.15.84:0/4086586477 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:27.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.731+0000 7f6d1d6f5640 1 -- 172.21.15.84:0/4086586477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea058570 msgr2=0x560fea05f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:27.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.731+0000 7f6d1d6f5640 1 --2- 172.21.15.84:0/4086586477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea058570 0x560fea05f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.731+0000 7f6d1d6f5640 1 -- 172.21.15.84:0/4086586477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fea062c40 msgr2=0x560fea07aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:27.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.731+0000 7f6d1d6f5640 1 --2- 172.21.15.84:0/4086586477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fea062c40 0x560fea07aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.731+0000 7f6d1d6f5640 1 -- 172.21.15.84:0/4086586477 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fe9c8f170 con 0x560fea057c50 2024-05-22T23:09:27.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.731+0000 7f6d1d6f5640 1 --2- 172.21.15.84:0/4086586477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea057c50 0x560fea06e8f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800fc40 tx=0x7f6d08005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de85a49eb60547f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:27.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.731+0000 7f6d037fe640 1 -- 172.21.15.84:0/4086586477 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08035020 con 0x560fea057c50 2024-05-22T23:09:27.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.731+0000 7f6d037fe640 1 -- 172.21.15.84:0/4086586477 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d0800ba70 con 0x560fea057c50 2024-05-22T23:09:27.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.731+0000 7f6d037fe640 1 -- 172.21.15.84:0/4086586477 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0802bbe0 con 0x560fea057c50 2024-05-22T23:09:27.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.732+0000 7f6d22e64740 1 -- 172.21.15.84:0/4086586477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea057c50 msgr2=0x560fea06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:27.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.732+0000 7f6d22e64740 1 --2- 172.21.15.84:0/4086586477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea057c50 0x560fea06e8f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800fc40 tx=0x7f6d08005b40 comp rx=0 tx=0).stop 2024-05-22T23:09:27.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.732+0000 7f6d22e64740 1 -- 172.21.15.84:0/4086586477 shutdown_connections 2024-05-22T23:09:27.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.732+0000 7f6d22e64740 1 --2- 172.21.15.84:0/4086586477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fea062c40 0x560fea07aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.732+0000 7f6d22e64740 1 --2- 172.21.15.84:0/4086586477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea058570 0x560fea05f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.732+0000 7f6d22e64740 1 --2- 172.21.15.84:0/4086586477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea057c50 0x560fea06e8f0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.732+0000 7f6d22e64740 1 -- 172.21.15.84:0/4086586477 >> 172.21.15.84:0/4086586477 conn(0x560fea053f70 msgr2=0x560fea059cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:27.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.732+0000 7f6d22e64740 1 -- 172.21.15.84:0/4086586477 shutdown_connections 2024-05-22T23:09:27.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.732+0000 7f6d22e64740 1 -- 172.21.15.84:0/4086586477 wait complete. 2024-05-22T23:09:27.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.732+0000 7f6d22e64740 1 -- start start 2024-05-22T23:09:27.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.732+0000 7f6d22e64740 1 Processor -- start 2024-05-22T23:09:27.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.733+0000 7f6d22e64740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea057c50 0x560fea06b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.733+0000 7f6d22e64740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea058570 0x560fea066690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.733+0000 7f6d22e64740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fea062c40 0x560fea066dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.733+0000 7f6d22e64740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6d181ae5f0 con 0x560fea057c50 2024-05-22T23:09:27.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.733+0000 7f6d22e64740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6d181afa60 con 0x560fea062c40 2024-05-22T23:09:27.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.733+0000 7f6d22e64740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6d181aecc0 con 0x560fea058570 2024-05-22T23:09:27.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.733+0000 7f6d22e64740 10 client.0 _ll_register_callbacks cb 0x7f6d14005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:09:27.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.733+0000 7f6d1d6f5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea057c50 0x560fea06b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.733+0000 7f6d1d6f5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea057c50 0x560fea06b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42276/0 (socket says 172.21.15.84:42276) 2024-05-22T23:09:27.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.733+0000 7f6d1d6f5640 1 -- 172.21.15.84:0/1211582101 learned_addr learned my addr 172.21.15.84:0/1211582101 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:27.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.733+0000 7f6d1def6640 1 --2- 172.21.15.84:0/1211582101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea058570 0x560fea066690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.733+0000 7f6d1d6f5640 1 -- 172.21.15.84:0/1211582101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea058570 msgr2=0x560fea066690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:27.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.733+0000 7f6d1d6f5640 1 --2- 172.21.15.84:0/1211582101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea058570 0x560fea066690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.733+0000 7f6d1d6f5640 1 -- 172.21.15.84:0/1211582101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fea062c40 msgr2=0x560fea066dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:27.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.733+0000 7f6d1d6f5640 1 --2- 172.21.15.84:0/1211582101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fea062c40 0x560fea066dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.733+0000 7f6d1d6f5640 1 -- 172.21.15.84:0/1211582101 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fe9f84930 con 0x560fea057c50 2024-05-22T23:09:27.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.733+0000 7f6d1def6640 1 --2- 172.21.15.84:0/1211582101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea058570 0x560fea066690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:09:27.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.733+0000 7f6d1d6f5640 1 --2- 172.21.15.84:0/1211582101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea057c50 0x560fea06b320 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f6d0801d040 tx=0x7f6d080058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=714ff8bbad67952c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:27.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.733+0000 7f6d017fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:09:27.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.733+0000 7f6d017fa640 1 -- 172.21.15.84:0/1211582101 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08035070 con 0x560fea057c50 2024-05-22T23:09:27.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.733+0000 7f6d017fa640 1 -- 172.21.15.84:0/1211582101 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7f6d0802b960 con 0x560fea057c50 2024-05-22T23:09:27.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.733+0000 7f6d22e64740 1 -- 172.21.15.84:0/1211582101 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d0800ba70 con 0x560fea057c50 2024-05-22T23:09:27.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.733+0000 7f6d017fa640 1 -- 172.21.15.84:0/1211582101 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f6d0802b0b0 con 0x560fea057c50 2024-05-22T23:09:27.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.734+0000 7f6d22e64740 10 client.16680 fetch_fsmap learned FSMap version 18 2024-05-22T23:09:27.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.734+0000 7f6d22e64740 1 -- 172.21.15.84:0/1211582101 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x560fea0d92c0 con 0x560fea057c50 2024-05-22T23:09:27.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.734+0000 7f6d017fa640 1 -- 172.21.15.84:0/1211582101 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d0800ba70 con 0x560fea057c50 2024-05-22T23:09:27.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.734+0000 7f6d017fa640 1 -- 172.21.15.84:0/1211582101 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f6d08031930 con 0x560fea057c50 2024-05-22T23:09:27.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.734+0000 7f6d22e64740 10 client.16680 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:09:27.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.734+0000 7f6d22e64740 10 client.16680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:09:27.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.734+0000 7f6d22e64740 20 client.16680 populate_metadata read hostname 'smithi084' 2024-05-22T23:09:27.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.734+0000 7f6d22e64740 1 --2- 172.21.15.84:0/1211582101 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x560fea0d9870 0x560fea0dbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.734+0000 7f6d22e64740 4 client.16680 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-22T23:09:27.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.734+0000 7f6d22e64740 1 -- 172.21.15.84:0/1211582101 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560fe9fc9170 con 0x560fea0d9870 2024-05-22T23:09:27.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.734+0000 7f6d1def6640 1 --2- 172.21.15.84:0/1211582101 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x560fea0d9870 0x560fea0dbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.735+0000 7f6d017fa640 10 client.16680 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:27.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.735+0000 7f6d1def6640 1 --2- 172.21.15.84:0/1211582101 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x560fea0d9870 0x560fea0dbc90 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c375349d79e462b server_cookie=c9d04410690b7dc5 in_seq=0 out_seq=0 2024-05-22T23:09:27.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.735+0000 7f6d017fa640 1 -- 172.21.15.84:0/1211582101 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560fe9fc9170 con 0x560fea0d9870 2024-05-22T23:09:27.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.736+0000 7f6d017fa640 10 client.16680 handle_command_reply: tid=0 2024-05-22T23:09:27.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.736+0000 7f6d22e64740 1 client.16680 shutdown 2024-05-22T23:09:27.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.736+0000 7f6d22e64740 1 -- 172.21.15.84:0/1211582101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea057c50 msgr2=0x560fea06b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:27.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.736+0000 7f6d22e64740 1 --2- 172.21.15.84:0/1211582101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea057c50 0x560fea06b320 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f6d0801d040 tx=0x7f6d080058a0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.736+0000 7f6d22e64740 1 -- 172.21.15.84:0/1211582101 shutdown_connections 2024-05-22T23:09:27.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.736+0000 7f6d22e64740 1 --2- 172.21.15.84:0/1211582101 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x560fea0d9870 0x560fea0dbc90 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.736+0000 7f6d22e64740 1 --2- 172.21.15.84:0/1211582101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fea062c40 0x560fea066dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.736+0000 7f6d22e64740 1 --2- 172.21.15.84:0/1211582101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea058570 0x560fea066690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.736+0000 7f6d22e64740 1 --2- 172.21.15.84:0/1211582101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fea057c50 0x560fea06b320 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.736+0000 7f6d22e64740 1 -- 172.21.15.84:0/1211582101 >> 172.21.15.84:0/1211582101 conn(0x560fea053f70 msgr2=0x560fea052920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:27.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.736+0000 7f6d017fa640 0 client.16680 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:27.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.736+0000 7f6d22e64740 1 -- 172.21.15.84:0/1211582101 shutdown_connections 2024-05-22T23:09:27.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.736+0000 7f6d22e64740 1 -- 172.21.15.84:0/1211582101 wait complete. 2024-05-22T23:09:27.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.736+0000 7f6d22e64740 20 client.16680 trim_cache size 0 max 16384 2024-05-22T23:09:27.756 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:09:27.756 INFO:teuthology.orchestra.run.smithi084.stdout: "return_code": 0, 2024-05-22T23:09:27.757 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_tag": "cc0188e3-f056-4693-b2f5-a565e3efd3ed", 2024-05-22T23:09:27.757 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": "asynchronous" 2024-05-22T23:09:27.757 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:09:27.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.736+0000 7f6d2015a640 1 -- 172.21.15.84:0/1757274153 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6d040a2d00 msgr2=0x7f6d040a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:27.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.736+0000 7f6d2015a640 1 --2- 172.21.15.84:0/1757274153 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6d040a2d00 0x7f6d040a51c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6d1009b730 tx=0x7f6d10086040 comp rx=0 tx=0).stop 2024-05-22T23:09:27.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.736+0000 7f6d2015a640 1 -- 172.21.15.84:0/1757274153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d18108180 msgr2=0x7f6d1819ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:27.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.736+0000 7f6d2015a640 1 --2- 172.21.15.84:0/1757274153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d18108180 0x7f6d1819ec50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000cad0 tx=0x7f6d1000cf90 comp rx=0 tx=0).stop 2024-05-22T23:09:27.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.736+0000 7f6d2015a640 1 -- 172.21.15.84:0/1757274153 shutdown_connections 2024-05-22T23:09:27.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.736+0000 7f6d2015a640 1 --2- 172.21.15.84:0/1757274153 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6d040a2d00 0x7f6d040a51c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.736+0000 7f6d2015a640 1 --2- 172.21.15.84:0/1757274153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d18108180 0x7f6d1819ec50 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.736+0000 7f6d2015a640 1 --2- 172.21.15.84:0/1757274153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d18107860 0x7f6d1819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.736+0000 7f6d2015a640 1 --2- 172.21.15.84:0/1757274153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d1806a5c0 0x7f6d1819a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.736+0000 7f6d2015a640 1 -- 172.21.15.84:0/1757274153 >> 172.21.15.84:0/1757274153 conn(0x7f6d18076570 msgr2=0x7f6d1810ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:27.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.737+0000 7f6d2015a640 1 -- 172.21.15.84:0/1757274153 shutdown_connections 2024-05-22T23:09:27.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.737+0000 7f6d2015a640 1 -- 172.21.15.84:0/1757274153 wait complete. 2024-05-22T23:09:27.760 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:09:27.760 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:09:27.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.857+0000 7f06b2729640 1 -- 172.21.15.84:0/3076614384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06ac101460 msgr2=0x7f06ac10d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:27.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.857+0000 7f06b2729640 1 --2- 172.21.15.84:0/3076614384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06ac101460 0x7f06ac10d590 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f06a4009ca0 tx=0x7f06a402a8c0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.857+0000 7f06b2729640 1 -- 172.21.15.84:0/3076614384 shutdown_connections 2024-05-22T23:09:27.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.857+0000 7f06b2729640 1 --2- 172.21.15.84:0/3076614384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06ac101460 0x7f06ac10d590 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.857+0000 7f06b2729640 1 --2- 172.21.15.84:0/3076614384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06ac100ac0 0x7f06ac100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.857+0000 7f06b2729640 1 --2- 172.21.15.84:0/3076614384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06ac1067c0 0x7f06ac106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.857+0000 7f06b2729640 1 -- 172.21.15.84:0/3076614384 >> 172.21.15.84:0/3076614384 conn(0x7f06ac076570 msgr2=0x7f06ac076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:27.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.857+0000 7f06b2729640 1 -- 172.21.15.84:0/3076614384 shutdown_connections 2024-05-22T23:09:27.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.857+0000 7f06b2729640 1 -- 172.21.15.84:0/3076614384 wait complete. 2024-05-22T23:09:27.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.858+0000 7f06b2729640 1 Processor -- start 2024-05-22T23:09:27.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.858+0000 7f06b2729640 1 -- start start 2024-05-22T23:09:27.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.858+0000 7f06b2729640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06ac100ac0 0x7f06ac198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.858+0000 7f06b2729640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06ac101460 0x7f06ac199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.858+0000 7f06b2729640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06ac1067c0 0x7f06ac1a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.858+0000 7f06b2729640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac110b10 con 0x7f06ac1067c0 2024-05-22T23:09:27.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.858+0000 7f06b2729640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac110990 con 0x7f06ac100ac0 2024-05-22T23:09:27.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.858+0000 7f06b2729640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac110c90 con 0x7f06ac101460 2024-05-22T23:09:27.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.858+0000 7f06b0cc6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06ac1067c0 0x7f06ac1a3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.858+0000 7f06b0cc6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06ac1067c0 0x7f06ac1a3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42296/0 (socket says 172.21.15.84:42296) 2024-05-22T23:09:27.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.858+0000 7f06b0cc6640 1 -- 172.21.15.84:0/1771523292 learned_addr learned my addr 172.21.15.84:0/1771523292 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:27.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.858+0000 7f06a3fff640 1 --2- 172.21.15.84:0/1771523292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06ac101460 0x7f06ac199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.858+0000 7f06abfff640 1 --2- 172.21.15.84:0/1771523292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06ac100ac0 0x7f06ac198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.858+0000 7f06b0cc6640 1 -- 172.21.15.84:0/1771523292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06ac101460 msgr2=0x7f06ac199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:27.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.858+0000 7f06b0cc6640 1 --2- 172.21.15.84:0/1771523292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06ac101460 0x7f06ac199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.858+0000 7f06b0cc6640 1 -- 172.21.15.84:0/1771523292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06ac100ac0 msgr2=0x7f06ac198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:27.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.858+0000 7f06b0cc6640 1 --2- 172.21.15.84:0/1771523292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06ac100ac0 0x7f06ac198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.858+0000 7f06b0cc6640 1 -- 172.21.15.84:0/1771523292 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ac1a43d0 con 0x7f06ac1067c0 2024-05-22T23:09:27.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.858+0000 7f06a3fff640 1 --2- 172.21.15.84:0/1771523292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06ac101460 0x7f06ac199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:09:27.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.859+0000 7f06b0cc6640 1 --2- 172.21.15.84:0/1771523292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06ac1067c0 0x7f06ac1a3e90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f06a4031a20 tx=0x7f06a4031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:27.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.859+0000 7f06a9ffb640 1 -- 172.21.15.84:0/1771523292 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a4003c30 con 0x7f06ac1067c0 2024-05-22T23:09:27.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.859+0000 7f06a9ffb640 1 -- 172.21.15.84:0/1771523292 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06a4003dd0 con 0x7f06ac1067c0 2024-05-22T23:09:27.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.859+0000 7f06b2729640 1 -- 172.21.15.84:0/1771523292 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06ac1a45a0 con 0x7f06ac1067c0 2024-05-22T23:09:27.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.859+0000 7f06a9ffb640 1 -- 172.21.15.84:0/1771523292 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a402fcf0 con 0x7f06ac1067c0 2024-05-22T23:09:27.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.859+0000 7f06b2729640 1 -- 172.21.15.84:0/1771523292 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06ac1a4a90 con 0x7f06ac1067c0 2024-05-22T23:09:27.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.861+0000 7f06a9ffb640 1 -- 172.21.15.84:0/1771523292 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f06a4004050 con 0x7f06ac1067c0 2024-05-22T23:09:27.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.861+0000 7f06b5437740 1 Processor -- start 2024-05-22T23:09:27.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.861+0000 7f06b5437740 1 -- start start 2024-05-22T23:09:27.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.862+0000 7f06b5437740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ee44a0 0x559639ee27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.862+0000 7f06b5437740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559639ee2db0 0x559639ee3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.862+0000 7f06b5437740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559639eedb30 0x559639eeff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.862+0000 7f06b5437740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559639e46db0 con 0x559639ee44a0 2024-05-22T23:09:27.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.862+0000 7f06b5437740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559639de5360 con 0x559639ee2db0 2024-05-22T23:09:27.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.862+0000 7f06b5437740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559639e42d70 con 0x559639eedb30 2024-05-22T23:09:27.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.862+0000 7f06a9ffb640 1 --2- 172.21.15.84:0/1771523292 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f06980a2d00 0x7f06980a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.862+0000 7f06a3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ee44a0 0x559639ee27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.862+0000 7f06a3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ee44a0 0x559639ee27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42308/0 (socket says 172.21.15.84:42308) 2024-05-22T23:09:27.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.862+0000 7f06a3fff640 1 -- 172.21.15.84:0/4190741702 learned_addr learned my addr 172.21.15.84:0/4190741702 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:27.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.862+0000 7f06b0cc6640 1 --2- 172.21.15.84:0/1771523292 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f06980a2d00 0x7f06980a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.862+0000 7f06a9ffb640 1 -- 172.21.15.84:0/1771523292 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7f06a40f07c0 con 0x7f06ac1067c0 2024-05-22T23:09:27.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.862+0000 7f06abfff640 1 --2- 172.21.15.84:0/4190741702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559639ee2db0 0x559639ee3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.862+0000 7f06a3fff640 1 -- 172.21.15.84:0/4190741702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559639eedb30 msgr2=0x559639eeff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:27.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.862+0000 7f06a3fff640 1 --2- 172.21.15.84:0/4190741702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559639eedb30 0x559639eeff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.862+0000 7f06a3fff640 1 -- 172.21.15.84:0/4190741702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559639ee2db0 msgr2=0x559639ee3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:27.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.862+0000 7f06a3fff640 1 --2- 172.21.15.84:0/4190741702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559639ee2db0 0x559639ee3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.862+0000 7f06a3fff640 1 -- 172.21.15.84:0/4190741702 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559639e08a60 con 0x559639ee44a0 2024-05-22T23:09:27.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.862+0000 7f06a3fff640 1 --2- 172.21.15.84:0/4190741702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ee44a0 0x559639ee27e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0694008cf0 tx=0x7f0694010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af123d54f0b09ea server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:27.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.862+0000 7f06b0cc6640 1 --2- 172.21.15.84:0/1771523292 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f06980a2d00 0x7f06980a51c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f06a40ee0a0 tx=0x7f06a408ee10 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:27.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.862+0000 7f06a27fc640 1 -- 172.21.15.84:0/4190741702 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694016070 con 0x559639ee44a0 2024-05-22T23:09:27.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.862+0000 7f06a27fc640 1 -- 172.21.15.84:0/4190741702 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0694005700 con 0x559639ee44a0 2024-05-22T23:09:27.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.862+0000 7f06a27fc640 1 -- 172.21.15.84:0/4190741702 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694012720 con 0x559639ee44a0 2024-05-22T23:09:27.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.863+0000 7f06b5437740 1 -- 172.21.15.84:0/4190741702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ee44a0 msgr2=0x559639ee27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:27.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.863+0000 7f06b5437740 1 --2- 172.21.15.84:0/4190741702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ee44a0 0x559639ee27e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0694008cf0 tx=0x7f0694010740 comp rx=0 tx=0).stop 2024-05-22T23:09:27.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.863+0000 7f06b5437740 1 -- 172.21.15.84:0/4190741702 shutdown_connections 2024-05-22T23:09:27.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.863+0000 7f06b5437740 1 --2- 172.21.15.84:0/4190741702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559639eedb30 0x559639eeff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.863+0000 7f06b5437740 1 --2- 172.21.15.84:0/4190741702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559639ee2db0 0x559639ee3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.863+0000 7f06b5437740 1 --2- 172.21.15.84:0/4190741702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ee44a0 0x559639ee27e0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.863+0000 7f06b5437740 1 -- 172.21.15.84:0/4190741702 >> 172.21.15.84:0/4190741702 conn(0x559639eab8e0 msgr2=0x559639edf310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:27.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.863+0000 7f06b5437740 1 -- 172.21.15.84:0/4190741702 shutdown_connections 2024-05-22T23:09:27.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.863+0000 7f06b5437740 1 -- 172.21.15.84:0/4190741702 wait complete. 2024-05-22T23:09:27.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.863+0000 7f06b5437740 1 -- start start 2024-05-22T23:09:27.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.864+0000 7f06b5437740 1 Processor -- start 2024-05-22T23:09:27.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.864+0000 7f06b5437740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559639ee2db0 0x559639eea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.864+0000 7f06b5437740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559639ee44a0 0x559639eeaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.864+0000 7f06b5437740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639eedb30 0x559639eeb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.864+0000 7f06b5437740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559639ef09c0 con 0x559639eedb30 2024-05-22T23:09:27.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.864+0000 7f06b5437740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559639e010f0 con 0x559639ee2db0 2024-05-22T23:09:27.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.864+0000 7f06b5437740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559639ef0b40 con 0x559639ee44a0 2024-05-22T23:09:27.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.864+0000 7f06b5437740 10 client.0 _ll_register_callbacks cb 0x559639e05f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:09:27.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.864+0000 7f06a3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639eedb30 0x559639eeb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.864+0000 7f06a3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639eedb30 0x559639eeb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42320/0 (socket says 172.21.15.84:42320) 2024-05-22T23:09:27.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.864+0000 7f06a3fff640 1 -- 172.21.15.84:0/4146853956 learned_addr learned my addr 172.21.15.84:0/4146853956 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:27.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.864+0000 7f06abfff640 1 --2- 172.21.15.84:0/4146853956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559639ee44a0 0x559639eeaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.864+0000 7f06a3fff640 1 --2- 172.21.15.84:0/4146853956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559639ee2db0 0x559639eea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.864+0000 7f06a3fff640 1 -- 172.21.15.84:0/4146853956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559639ee44a0 msgr2=0x559639eeaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:27.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.864+0000 7f06a3fff640 1 --2- 172.21.15.84:0/4146853956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559639ee44a0 0x559639eeaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.864+0000 7f06a3fff640 1 -- 172.21.15.84:0/4146853956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559639ee2db0 msgr2=0x559639eea9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:27.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.864+0000 7f06a3fff640 1 --2- 172.21.15.84:0/4146853956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559639ee2db0 0x559639eea9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.864+0000 7f06a3fff640 1 -- 172.21.15.84:0/4146853956 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559639c247d0 con 0x559639eedb30 2024-05-22T23:09:27.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.864+0000 7f06a3fff640 1 --2- 172.21.15.84:0/4146853956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639eedb30 0x559639eeb420 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f06940050d0 tx=0x7f0694004e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6525515f557227f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:27.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.864+0000 7f068b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:09:27.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.865+0000 7f068b7fe640 1 -- 172.21.15.84:0/4146853956 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069401f070 con 0x559639eedb30 2024-05-22T23:09:27.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.865+0000 7f06b5437740 1 -- 172.21.15.84:0/4146853956 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0694005700 con 0x559639eedb30 2024-05-22T23:09:27.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.865+0000 7f068b7fe640 1 -- 172.21.15.84:0/4146853956 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7f0694005440 con 0x559639eedb30 2024-05-22T23:09:27.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.865+0000 7f068b7fe640 1 -- 172.21.15.84:0/4146853956 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f0694016040 con 0x559639eedb30 2024-05-22T23:09:27.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.865+0000 7f06b5437740 10 client.16692 fetch_fsmap learned FSMap version 18 2024-05-22T23:09:27.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.865+0000 7f06b5437740 1 -- 172.21.15.84:0/4146853956 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x559639b5a3f0 con 0x559639eedb30 2024-05-22T23:09:27.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.865+0000 7f068b7fe640 1 -- 172.21.15.84:0/4146853956 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0694005700 con 0x559639eedb30 2024-05-22T23:09:27.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.865+0000 7f068b7fe640 1 -- 172.21.15.84:0/4146853956 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f069401a9e0 con 0x559639eedb30 2024-05-22T23:09:27.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.865+0000 7f06b5437740 10 client.16692 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:09:27.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.865+0000 7f06b5437740 10 client.16692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:09:27.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.865+0000 7f06b5437740 20 client.16692 populate_metadata read hostname 'smithi084' 2024-05-22T23:09:27.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.865+0000 7f06b5437740 1 --2- 172.21.15.84:0/4146853956 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x559639efdea0 0x559639f00290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.865+0000 7f06b5437740 4 client.16692 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:09:27.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.865+0000 7f06b5437740 1 -- 172.21.15.84:0/4146853956 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559639e544e0 con 0x559639efdea0 2024-05-22T23:09:27.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.865+0000 7f06abfff640 1 --2- 172.21.15.84:0/4146853956 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x559639efdea0 0x559639f00290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.866+0000 7f068b7fe640 10 client.16692 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:27.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.866+0000 7f06abfff640 1 --2- 172.21.15.84:0/4146853956 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x559639efdea0 0x559639f00290 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=368098780f3235e4 server_cookie=13247391afad31d8 in_seq=0 out_seq=0 2024-05-22T23:09:27.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.866+0000 7f068b7fe640 1 -- 172.21.15.84:0/4146853956 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559639e544e0 con 0x559639efdea0 2024-05-22T23:09:27.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.866+0000 7f068b7fe640 10 client.16692 handle_command_reply: tid=0 2024-05-22T23:09:27.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.866+0000 7f06b5437740 1 client.16692 shutdown 2024-05-22T23:09:27.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.867+0000 7f06b5437740 1 -- 172.21.15.84:0/4146853956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639eedb30 msgr2=0x559639eeb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:27.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.867+0000 7f06b5437740 1 --2- 172.21.15.84:0/4146853956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639eedb30 0x559639eeb420 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f06940050d0 tx=0x7f0694004e70 comp rx=0 tx=0).stop 2024-05-22T23:09:27.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.867+0000 7f06b5437740 1 -- 172.21.15.84:0/4146853956 shutdown_connections 2024-05-22T23:09:27.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.867+0000 7f06b5437740 1 --2- 172.21.15.84:0/4146853956 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x559639efdea0 0x559639f00290 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.867+0000 7f06b5437740 1 --2- 172.21.15.84:0/4146853956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639eedb30 0x559639eeb420 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.867+0000 7f06b5437740 1 --2- 172.21.15.84:0/4146853956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559639ee44a0 0x559639eeaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.867+0000 7f068b7fe640 0 client.16692 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:27.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.867+0000 7f06b5437740 1 --2- 172.21.15.84:0/4146853956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559639ee2db0 0x559639eea9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.867+0000 7f06b5437740 1 -- 172.21.15.84:0/4146853956 >> 172.21.15.84:0/4146853956 conn(0x559639eab8e0 msgr2=0x559639ee5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:27.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.867+0000 7f06b5437740 1 -- 172.21.15.84:0/4146853956 shutdown_connections 2024-05-22T23:09:27.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.867+0000 7f06b5437740 1 -- 172.21.15.84:0/4146853956 wait complete. 2024-05-22T23:09:27.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.867+0000 7f06b5437740 20 client.16692 trim_cache size 0 max 16384 2024-05-22T23:09:27.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.884+0000 7f06b5437740 1 Processor -- start 2024-05-22T23:09:27.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.884+0000 7f06b5437740 1 -- start start 2024-05-22T23:09:27.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.884+0000 7f06b5437740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ee2db0 0x559639ef9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.884+0000 7f06b5437740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559639ee44a0 0x559639edee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.884+0000 7f06b5437740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559639eedb30 0x559639eec8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.884+0000 7f06b5437740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac1ab5d0 con 0x559639ee2db0 2024-05-22T23:09:27.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.884+0000 7f06b5437740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac1ade30 con 0x559639eedb30 2024-05-22T23:09:27.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.884+0000 7f06b5437740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac1abc20 con 0x559639ee44a0 2024-05-22T23:09:27.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.884+0000 7f06a3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ee2db0 0x559639ef9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.884+0000 7f06a3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ee2db0 0x559639ef9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42332/0 (socket says 172.21.15.84:42332) 2024-05-22T23:09:27.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.884+0000 7f06a3fff640 1 -- 172.21.15.84:0/3069705445 learned_addr learned my addr 172.21.15.84:0/3069705445 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:27.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.884+0000 7f06a3fff640 1 -- 172.21.15.84:0/3069705445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559639ee44a0 msgr2=0x559639edee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:27.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.884+0000 7f06a3fff640 1 --2- 172.21.15.84:0/3069705445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559639ee44a0 0x559639edee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.884+0000 7f06a3fff640 1 -- 172.21.15.84:0/3069705445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559639eedb30 msgr2=0x559639eec8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:27.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.884+0000 7f06a3fff640 1 --2- 172.21.15.84:0/3069705445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559639eedb30 0x559639eec8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.884+0000 7f06a3fff640 1 -- 172.21.15.84:0/3069705445 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559639ee9950 con 0x559639ee2db0 2024-05-22T23:09:27.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.884+0000 7f06a3fff640 1 --2- 172.21.15.84:0/3069705445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ee2db0 0x559639ef9920 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f0694008cf0 tx=0x7f0694008230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b0fbd8d809e116 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:27.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.885+0000 7f06a1ffb640 1 -- 172.21.15.84:0/3069705445 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069403a070 con 0x559639ee2db0 2024-05-22T23:09:27.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.885+0000 7f06a1ffb640 1 -- 172.21.15.84:0/3069705445 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0694005700 con 0x559639ee2db0 2024-05-22T23:09:27.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.885+0000 7f06a1ffb640 1 -- 172.21.15.84:0/3069705445 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069401f030 con 0x559639ee2db0 2024-05-22T23:09:27.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.885+0000 7f06b5437740 1 -- 172.21.15.84:0/3069705445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ee2db0 msgr2=0x559639ef9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:27.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.885+0000 7f06b5437740 1 --2- 172.21.15.84:0/3069705445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ee2db0 0x559639ef9920 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f0694008cf0 tx=0x7f0694008230 comp rx=0 tx=0).stop 2024-05-22T23:09:27.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.885+0000 7f06b5437740 1 -- 172.21.15.84:0/3069705445 shutdown_connections 2024-05-22T23:09:27.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.885+0000 7f06b5437740 1 --2- 172.21.15.84:0/3069705445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559639eedb30 0x559639eec8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.885+0000 7f06b5437740 1 --2- 172.21.15.84:0/3069705445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559639ee44a0 0x559639edee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.885+0000 7f06b5437740 1 --2- 172.21.15.84:0/3069705445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ee2db0 0x559639ef9920 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.885+0000 7f06b5437740 1 -- 172.21.15.84:0/3069705445 >> 172.21.15.84:0/3069705445 conn(0x559639eab8e0 msgr2=0x559639eec1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:27.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.885+0000 7f06b5437740 1 -- 172.21.15.84:0/3069705445 shutdown_connections 2024-05-22T23:09:27.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.885+0000 7f06b5437740 1 -- 172.21.15.84:0/3069705445 wait complete. 2024-05-22T23:09:27.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.885+0000 7f06b5437740 1 -- start start 2024-05-22T23:09:27.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.886+0000 7f06b5437740 1 Processor -- start 2024-05-22T23:09:27.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.886+0000 7f06b5437740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559639ee2db0 0x559639ef6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.886+0000 7f06b5437740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ee44a0 0x559639ef15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.886+0000 7f06b5437740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559639eedb30 0x559639ef1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.886+0000 7f06b5437740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac1ac2f0 con 0x559639ee44a0 2024-05-22T23:09:27.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.886+0000 7f06b5437740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac1ad760 con 0x559639eedb30 2024-05-22T23:09:27.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.886+0000 7f06b5437740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac1ac9c0 con 0x559639ee2db0 2024-05-22T23:09:27.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.886+0000 7f06b5437740 10 client.0 _ll_register_callbacks cb 0x7f0694008bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:09:27.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.886+0000 7f06abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ee44a0 0x559639ef15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.886+0000 7f06abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ee44a0 0x559639ef15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42348/0 (socket says 172.21.15.84:42348) 2024-05-22T23:09:27.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.886+0000 7f06abfff640 1 -- 172.21.15.84:0/1124928375 learned_addr learned my addr 172.21.15.84:0/1124928375 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:27.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.886+0000 7f06a3fff640 1 --2- 172.21.15.84:0/1124928375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559639eedb30 0x559639ef1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.886+0000 7f06abfff640 1 -- 172.21.15.84:0/1124928375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559639ee2db0 msgr2=0x559639ef6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:27.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.886+0000 7f06abfff640 1 --2- 172.21.15.84:0/1124928375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559639ee2db0 0x559639ef6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.886+0000 7f06abfff640 1 -- 172.21.15.84:0/1124928375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559639eedb30 msgr2=0x559639ef1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:27.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.886+0000 7f06abfff640 1 --2- 172.21.15.84:0/1124928375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559639eedb30 0x559639ef1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.886+0000 7f06abfff640 1 -- 172.21.15.84:0/1124928375 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559639b1a0f0 con 0x559639ee44a0 2024-05-22T23:09:27.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.886+0000 7f06a3fff640 1 --2- 172.21.15.84:0/1124928375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559639eedb30 0x559639ef1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:09:27.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.886+0000 7f06abfff640 1 --2- 172.21.15.84:0/1124928375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ee44a0 0x559639ef15f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x559639ee59f0 tx=0x7f069c005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883075cd2a4c4b7f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:27.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.887+0000 7f068b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:09:27.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.887+0000 7f068b7fe640 1 -- 172.21.15.84:0/1124928375 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c01f030 con 0x559639ee44a0 2024-05-22T23:09:27.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.887+0000 7f068b7fe640 1 -- 172.21.15.84:0/1124928375 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7f069c01abc0 con 0x559639ee44a0 2024-05-22T23:09:27.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.887+0000 7f06b5437740 1 -- 172.21.15.84:0/1124928375 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0694005700 con 0x559639ee44a0 2024-05-22T23:09:27.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.887+0000 7f068b7fe640 1 -- 172.21.15.84:0/1124928375 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f069c029050 con 0x559639ee44a0 2024-05-22T23:09:27.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.887+0000 7f06b5437740 10 client.16698 fetch_fsmap learned FSMap version 18 2024-05-22T23:09:27.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.887+0000 7f06b5437740 1 -- 172.21.15.84:0/1124928375 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x559639e05fe0 con 0x559639ee44a0 2024-05-22T23:09:27.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.887+0000 7f068b7fe640 1 -- 172.21.15.84:0/1124928375 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0694005700 con 0x559639ee44a0 2024-05-22T23:09:27.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.887+0000 7f068b7fe640 1 -- 172.21.15.84:0/1124928375 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f069c00f040 con 0x559639ee44a0 2024-05-22T23:09:27.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.887+0000 7f06b5437740 10 client.16698 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:09:27.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.887+0000 7f06b5437740 10 client.16698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:09:27.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.887+0000 7f06b5437740 20 client.16698 populate_metadata read hostname 'smithi084' 2024-05-22T23:09:27.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.887+0000 7f06b5437740 1 --2- 172.21.15.84:0/1124928375 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x559639f64820 0x559639f66c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:27.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.887+0000 7f06b5437740 4 client.16698 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:09:27.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.887+0000 7f06b5437740 1 -- 172.21.15.84:0/1124928375 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559639e4e830 con 0x559639f64820 2024-05-22T23:09:27.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.887+0000 7f06a3fff640 1 --2- 172.21.15.84:0/1124928375 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x559639f64820 0x559639f66c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:27.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.888+0000 7f068b7fe640 10 client.16698 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:27.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.888+0000 7f06a3fff640 1 --2- 172.21.15.84:0/1124928375 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x559639f64820 0x559639f66c60 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cd23a2ce88c4469 server_cookie=2360dd654b4b55c8 in_seq=0 out_seq=0 2024-05-22T23:09:27.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.888+0000 7f068b7fe640 1 -- 172.21.15.84:0/1124928375 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559639e4e830 con 0x559639f64820 2024-05-22T23:09:27.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.888+0000 7f068b7fe640 10 client.16698 handle_command_reply: tid=0 2024-05-22T23:09:27.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.888+0000 7f06b5437740 1 client.16698 shutdown 2024-05-22T23:09:27.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.888+0000 7f06b5437740 1 -- 172.21.15.84:0/1124928375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ee44a0 msgr2=0x559639ef15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:27.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.888+0000 7f06b5437740 1 --2- 172.21.15.84:0/1124928375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ee44a0 0x559639ef15f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x559639ee59f0 tx=0x7f069c005410 comp rx=0 tx=0).stop 2024-05-22T23:09:27.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.888+0000 7f06b5437740 1 -- 172.21.15.84:0/1124928375 shutdown_connections 2024-05-22T23:09:27.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.888+0000 7f06b5437740 1 --2- 172.21.15.84:0/1124928375 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x559639f64820 0x559639f66c60 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.888+0000 7f06b5437740 1 --2- 172.21.15.84:0/1124928375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559639eedb30 0x559639ef1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.888+0000 7f06b5437740 1 --2- 172.21.15.84:0/1124928375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ee44a0 0x559639ef15f0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.888+0000 7f068b7fe640 0 client.16698 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:27.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.888+0000 7f06b5437740 1 --2- 172.21.15.84:0/1124928375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559639ee2db0 0x559639ef6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.888+0000 7f06b5437740 1 -- 172.21.15.84:0/1124928375 >> 172.21.15.84:0/1124928375 conn(0x559639eab8e0 msgr2=0x559639ee5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:27.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.888+0000 7f06b5437740 1 -- 172.21.15.84:0/1124928375 shutdown_connections 2024-05-22T23:09:27.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.888+0000 7f06b5437740 1 -- 172.21.15.84:0/1124928375 wait complete. 2024-05-22T23:09:27.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.888+0000 7f06b5437740 20 client.16698 trim_cache size 0 max 16384 2024-05-22T23:09:27.906 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:09:27.906 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-22T23:09:27.906 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": { 2024-05-22T23:09:27.906 INFO:teuthology.orchestra.run.smithi084.stdout: "cc0188e3-f056-4693-b2f5-a565e3efd3ed": { 2024-05-22T23:09:27.906 INFO:teuthology.orchestra.run.smithi084.stdout: "path": "/", 2024-05-22T23:09:27.907 INFO:teuthology.orchestra.run.smithi084.stdout: "tag": "cc0188e3-f056-4693-b2f5-a565e3efd3ed", 2024-05-22T23:09:27.907 INFO:teuthology.orchestra.run.smithi084.stdout: "options": "recursive,force" 2024-05-22T23:09:27.907 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:09:27.907 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:09:27.907 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:09:27.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.889+0000 7f06b2729640 1 -- 172.21.15.84:0/1771523292 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f06980a2d00 msgr2=0x7f06980a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:27.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.889+0000 7f06b2729640 1 --2- 172.21.15.84:0/1771523292 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f06980a2d00 0x7f06980a51c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f06a40ee0a0 tx=0x7f06a408ee10 comp rx=0 tx=0).stop 2024-05-22T23:09:27.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.889+0000 7f06b2729640 1 -- 172.21.15.84:0/1771523292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06ac1067c0 msgr2=0x7f06ac1a3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:27.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.889+0000 7f06b2729640 1 --2- 172.21.15.84:0/1771523292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06ac1067c0 0x7f06ac1a3e90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f06a4031a20 tx=0x7f06a4031a50 comp rx=0 tx=0).stop 2024-05-22T23:09:27.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.889+0000 7f06b2729640 1 -- 172.21.15.84:0/1771523292 shutdown_connections 2024-05-22T23:09:27.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.889+0000 7f06b2729640 1 --2- 172.21.15.84:0/1771523292 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f06980a2d00 0x7f06980a51c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.889+0000 7f06b2729640 1 --2- 172.21.15.84:0/1771523292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06ac1067c0 0x7f06ac1a3e90 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.889+0000 7f06b2729640 1 --2- 172.21.15.84:0/1771523292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06ac101460 0x7f06ac199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.889+0000 7f06b2729640 1 --2- 172.21.15.84:0/1771523292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06ac100ac0 0x7f06ac198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:27.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.889+0000 7f06b2729640 1 -- 172.21.15.84:0/1771523292 >> 172.21.15.84:0/1771523292 conn(0x7f06ac076570 msgr2=0x7f06ac0ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:27.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.889+0000 7f06b2729640 1 -- 172.21.15.84:0/1771523292 shutdown_connections 2024-05-22T23:09:27.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:27.889+0000 7f06b2729640 1 -- 172.21.15.84:0/1771523292 wait complete. 2024-05-22T23:09:27.911 INFO:tasks.cephfs.filesystem:scrub status for tag:cc0188e3-f056-4693-b2f5-a565e3efd3ed - {'path': '/', 'tag': 'cc0188e3-f056-4693-b2f5-a565e3efd3ed', 'options': 'recursive,force'} 2024-05-22T23:09:28.379 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:28 smithi084 ceph-mon[32186]: scrub queued for path: / 2024-05-22T23:09:28.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:28 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:09:28.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:28 smithi084 ceph-mon[32186]: scrub summary: active paths [/] 2024-05-22T23:09:28.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:28 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:09:28.415 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:28 smithi204 ceph-mon[36211]: scrub queued for path: / 2024-05-22T23:09:28.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:28 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:09:28.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:28 smithi204 ceph-mon[36211]: scrub summary: active paths [/] 2024-05-22T23:09:28.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:28 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:09:28.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:28 smithi096 ceph-mon[36392]: scrub queued for path: / 2024-05-22T23:09:28.632 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:28 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:09:28.632 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:28 smithi096 ceph-mon[36392]: scrub summary: active paths [/] 2024-05-22T23:09:28.632 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:28 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:09:29.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:29 smithi084 ceph-mon[32186]: pgmap v236: 97 pgs: 97 active+clean; 467 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 2.1 MiB/s wr, 78 op/s 2024-05-22T23:09:29.415 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:29 smithi204 ceph-mon[36211]: pgmap v236: 97 pgs: 97 active+clean; 467 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 2.1 MiB/s wr, 78 op/s 2024-05-22T23:09:29.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:29 smithi096 ceph-mon[36392]: pgmap v236: 97 pgs: 97 active+clean; 467 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 2.1 MiB/s wr, 78 op/s 2024-05-22T23:09:31.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:31 smithi084 ceph-mon[32186]: pgmap v237: 97 pgs: 97 active+clean; 467 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.9 MiB/s wr, 71 op/s 2024-05-22T23:09:31.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:31 smithi084 ceph-mon[32186]: scrub summary: idle 2024-05-22T23:09:31.415 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:31 smithi204 ceph-mon[36211]: pgmap v237: 97 pgs: 97 active+clean; 467 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.9 MiB/s wr, 71 op/s 2024-05-22T23:09:31.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:31 smithi204 ceph-mon[36211]: scrub summary: idle 2024-05-22T23:09:31.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:31 smithi096 ceph-mon[36392]: pgmap v237: 97 pgs: 97 active+clean; 467 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.9 MiB/s wr, 71 op/s 2024-05-22T23:09:31.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:31 smithi096 ceph-mon[36392]: scrub summary: idle 2024-05-22T23:09:33.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:33 smithi084 ceph-mon[32186]: pgmap v238: 97 pgs: 97 active+clean; 465 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 2.7 MiB/s wr, 100 op/s 2024-05-22T23:09:33.415 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:33 smithi204 ceph-mon[36211]: pgmap v238: 97 pgs: 97 active+clean; 465 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 2.7 MiB/s wr, 100 op/s 2024-05-22T23:09:33.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:33 smithi096 ceph-mon[36392]: pgmap v238: 97 pgs: 97 active+clean; 465 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 2.7 MiB/s wr, 100 op/s 2024-05-22T23:09:35.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:35 smithi084 ceph-mon[32186]: pgmap v239: 97 pgs: 97 active+clean; 465 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.8 MiB/s wr, 68 op/s 2024-05-22T23:09:35.415 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:35 smithi204 ceph-mon[36211]: pgmap v239: 97 pgs: 97 active+clean; 465 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.8 MiB/s wr, 68 op/s 2024-05-22T23:09:35.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:35 smithi096 ceph-mon[36392]: pgmap v239: 97 pgs: 97 active+clean; 465 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.8 MiB/s wr, 68 op/s 2024-05-22T23:09:37.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:37 smithi084 ceph-mon[32186]: pgmap v240: 97 pgs: 97 active+clean; 461 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 2.4 MiB/s wr, 94 op/s 2024-05-22T23:09:37.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:37 smithi204 ceph-mon[36211]: pgmap v240: 97 pgs: 97 active+clean; 461 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 2.4 MiB/s wr, 94 op/s 2024-05-22T23:09:37.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:37 smithi096 ceph-mon[36392]: pgmap v240: 97 pgs: 97 active+clean; 461 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 2.4 MiB/s wr, 94 op/s 2024-05-22T23:09:39.379 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:39 smithi084 ceph-mon[32186]: pgmap v241: 97 pgs: 97 active+clean; 461 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 1.4 MiB/s wr, 56 op/s 2024-05-22T23:09:39.415 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:39 smithi204 ceph-mon[36211]: pgmap v241: 97 pgs: 97 active+clean; 461 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 1.4 MiB/s wr, 56 op/s 2024-05-22T23:09:39.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:39 smithi096 ceph-mon[36392]: pgmap v241: 97 pgs: 97 active+clean; 461 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 1.4 MiB/s wr, 56 op/s 2024-05-22T23:09:41.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:41 smithi084 ceph-mon[32186]: pgmap v242: 97 pgs: 97 active+clean; 461 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 1.4 MiB/s wr, 56 op/s 2024-05-22T23:09:41.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:41 smithi204 ceph-mon[36211]: pgmap v242: 97 pgs: 97 active+clean; 461 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 1.4 MiB/s wr, 56 op/s 2024-05-22T23:09:41.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:41 smithi096 ceph-mon[36392]: pgmap v242: 97 pgs: 97 active+clean; 461 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 1.4 MiB/s wr, 56 op/s 2024-05-22T23:09:43.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:43 smithi084 ceph-mon[32186]: pgmap v243: 97 pgs: 97 active+clean; 455 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 1.4 MiB/s wr, 67 op/s 2024-05-22T23:09:43.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:43 smithi204 ceph-mon[36211]: pgmap v243: 97 pgs: 97 active+clean; 455 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 1.4 MiB/s wr, 67 op/s 2024-05-22T23:09:43.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:43 smithi096 ceph-mon[36392]: pgmap v243: 97 pgs: 97 active+clean; 455 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 1.4 MiB/s wr, 67 op/s 2024-05-22T23:09:45.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:45 smithi084 ceph-mon[32186]: pgmap v244: 97 pgs: 97 active+clean; 455 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 696 KiB/s wr, 37 op/s 2024-05-22T23:09:45.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:45 smithi204 ceph-mon[36211]: pgmap v244: 97 pgs: 97 active+clean; 455 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 696 KiB/s wr, 37 op/s 2024-05-22T23:09:45.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:45 smithi096 ceph-mon[36392]: pgmap v244: 97 pgs: 97 active+clean; 455 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 696 KiB/s wr, 37 op/s 2024-05-22T23:09:47.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:47 smithi084 ceph-mon[32186]: pgmap v245: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 711 KiB/s wr, 39 op/s 2024-05-22T23:09:47.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:47 smithi204 ceph-mon[36211]: pgmap v245: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 711 KiB/s wr, 39 op/s 2024-05-22T23:09:47.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:47 smithi096 ceph-mon[36392]: pgmap v245: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 711 KiB/s wr, 39 op/s 2024-05-22T23:09:49.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:49 smithi084 ceph-mon[32186]: pgmap v246: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 82 KiB/s wr, 13 op/s 2024-05-22T23:09:49.415 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:49 smithi204 ceph-mon[36211]: pgmap v246: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 82 KiB/s wr, 13 op/s 2024-05-22T23:09:49.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:49 smithi096 ceph-mon[36392]: pgmap v246: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 82 KiB/s wr, 13 op/s 2024-05-22T23:09:51.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:51 smithi084 ceph-mon[32186]: pgmap v247: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 82 KiB/s wr, 13 op/s 2024-05-22T23:09:51.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:51 smithi204 ceph-mon[36211]: pgmap v247: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 82 KiB/s wr, 13 op/s 2024-05-22T23:09:51.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:51 smithi096 ceph-mon[36392]: pgmap v247: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 82 KiB/s wr, 13 op/s 2024-05-22T23:09:53.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:53 smithi204 ceph-mon[36211]: pgmap v248: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 89 KiB/s wr, 14 op/s 2024-05-22T23:09:53.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:53 smithi096 ceph-mon[36392]: pgmap v248: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 89 KiB/s wr, 14 op/s 2024-05-22T23:09:53.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:53 smithi084 ceph-mon[32186]: pgmap v248: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 89 KiB/s wr, 14 op/s 2024-05-22T23:09:55.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:55 smithi204 ceph-mon[36211]: pgmap v249: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 KiB/s wr, 3 op/s 2024-05-22T23:09:55.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:55 smithi096 ceph-mon[36392]: pgmap v249: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 KiB/s wr, 3 op/s 2024-05-22T23:09:55.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:55 smithi084 ceph-mon[32186]: pgmap v249: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 KiB/s wr, 3 op/s 2024-05-22T23:09:57.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:57 smithi204 ceph-mon[36211]: pgmap v250: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 4 op/s 2024-05-22T23:09:57.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:57 smithi096 ceph-mon[36392]: pgmap v250: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 4 op/s 2024-05-22T23:09:57.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:57 smithi084 ceph-mon[32186]: pgmap v250: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 4 op/s 2024-05-22T23:09:57.912 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:09:57.912 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:09:58.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.021+0000 7fbe78f68640 1 -- 172.21.15.84:0/3044419473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe74100130 msgr2=0x7fbe74100510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:58.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.021+0000 7fbe78f68640 1 --2- 172.21.15.84:0/3044419473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe74100130 0x7fbe74100510 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009900 tx=0x7fbe6402a410 comp rx=0 tx=0).stop 2024-05-22T23:09:58.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.021+0000 7fbe78f68640 1 -- 172.21.15.84:0/3044419473 shutdown_connections 2024-05-22T23:09:58.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.021+0000 7fbe78f68640 1 --2- 172.21.15.84:0/3044419473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe74105200 0x7fbe7410cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.021+0000 7fbe78f68640 1 --2- 172.21.15.84:0/3044419473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe74100ae0 0x7fbe74104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.021+0000 7fbe78f68640 1 --2- 172.21.15.84:0/3044419473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe74100130 0x7fbe74100510 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.021+0000 7fbe78f68640 1 -- 172.21.15.84:0/3044419473 >> 172.21.15.84:0/3044419473 conn(0x7fbe74076d30 msgr2=0x7fbe740fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:58.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.021+0000 7fbe78f68640 1 -- 172.21.15.84:0/3044419473 shutdown_connections 2024-05-22T23:09:58.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.021+0000 7fbe78f68640 1 -- 172.21.15.84:0/3044419473 wait complete. 2024-05-22T23:09:58.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.022+0000 7fbe78f68640 1 Processor -- start 2024-05-22T23:09:58.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.022+0000 7fbe78f68640 1 -- start start 2024-05-22T23:09:58.031 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.022+0000 7fbe78f68640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe74100130 0x7fbe74198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.032 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.022+0000 7fbe78f68640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe74105200 0x7fbe74198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.032 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.022+0000 7fbe78f68640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe7419caf0 0x7fbe7419cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.032 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.022+0000 7fbe78f68640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbe74111cd0 con 0x7fbe74105200 2024-05-22T23:09:58.032 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.022+0000 7fbe78f68640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbe74111b50 con 0x7fbe7419caf0 2024-05-22T23:09:58.032 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.022+0000 7fbe78f68640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbe74111e50 con 0x7fbe74100130 2024-05-22T23:09:58.032 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.022+0000 7fbe6bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe74105200 0x7fbe74198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.032 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.022+0000 7fbe6bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe74105200 0x7fbe74198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37260/0 (socket says 172.21.15.84:37260) 2024-05-22T23:09:58.033 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.022+0000 7fbe6bfff640 1 -- 172.21.15.84:0/1033669179 learned_addr learned my addr 172.21.15.84:0/1033669179 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:58.033 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.022+0000 7fbe7259c640 1 --2- 172.21.15.84:0/1033669179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe74100130 0x7fbe74198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.033 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.023+0000 7fbe72d9d640 1 --2- 172.21.15.84:0/1033669179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe7419caf0 0x7fbe7419cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.033 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.023+0000 7fbe6bfff640 1 -- 172.21.15.84:0/1033669179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe74100130 msgr2=0x7fbe74198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:58.033 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.023+0000 7fbe6bfff640 1 --2- 172.21.15.84:0/1033669179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe74100130 0x7fbe74198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.034 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.023+0000 7fbe6bfff640 1 -- 172.21.15.84:0/1033669179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe7419caf0 msgr2=0x7fbe7419cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:58.034 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.023+0000 7fbe6bfff640 1 --2- 172.21.15.84:0/1033669179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe7419caf0 0x7fbe7419cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.034 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.023+0000 7fbe6bfff640 1 -- 172.21.15.84:0/1033669179 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7419d5e0 con 0x7fbe74105200 2024-05-22T23:09:58.034 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.023+0000 7fbe6bfff640 1 --2- 172.21.15.84:0/1033669179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe74105200 0x7fbe74198770 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00d740 tx=0x7fbe5c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:58.034 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.023+0000 7fbe6b7fe640 1 -- 172.21.15.84:0/1033669179 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c017070 con 0x7fbe74105200 2024-05-22T23:09:58.034 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.023+0000 7fbe78f68640 1 -- 172.21.15.84:0/1033669179 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe7419d8d0 con 0x7fbe74105200 2024-05-22T23:09:58.034 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.023+0000 7fbe6b7fe640 1 -- 172.21.15.84:0/1033669179 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe5c00de50 con 0x7fbe74105200 2024-05-22T23:09:58.034 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.023+0000 7fbe6b7fe640 1 -- 172.21.15.84:0/1033669179 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c004e00 con 0x7fbe74105200 2024-05-22T23:09:58.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.023+0000 7fbe78f68640 1 -- 172.21.15.84:0/1033669179 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe741a5070 con 0x7fbe74105200 2024-05-22T23:09:58.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.025+0000 7fbe6b7fe640 1 -- 172.21.15.84:0/1033669179 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbe5c004fa0 con 0x7fbe74105200 2024-05-22T23:09:58.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.026+0000 7fbe7bc72740 1 Processor -- start 2024-05-22T23:09:58.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.026+0000 7fbe7bc72740 1 -- start start 2024-05-22T23:09:58.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.026+0000 7fbe7bc72740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55658a54a4a0 0x55658a5487e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.026+0000 7fbe7bc72740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55658a548db0 0x55658a549230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.026+0000 7fbe7bc72740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a553b30 0x55658a555f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.026+0000 7fbe7bc72740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55658a4acdb0 con 0x55658a553b30 2024-05-22T23:09:58.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.026+0000 7fbe7bc72740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55658a44b360 con 0x55658a54a4a0 2024-05-22T23:09:58.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.026+0000 7fbe7bc72740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55658a4a8d70 con 0x55658a548db0 2024-05-22T23:09:58.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.026+0000 7fbe6b7fe640 1 --2- 172.21.15.84:0/1033669179 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fbe600a2d00 0x7fbe600a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.026+0000 7fbe6bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a553b30 0x55658a555f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.026+0000 7fbe6bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a553b30 0x55658a555f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37264/0 (socket says 172.21.15.84:37264) 2024-05-22T23:09:58.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.026+0000 7fbe6bfff640 1 -- 172.21.15.84:0/3643056817 learned_addr learned my addr 172.21.15.84:0/3643056817 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:58.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.027+0000 7fbe7259c640 1 --2- 172.21.15.84:0/3643056817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55658a548db0 0x55658a549230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.027+0000 7fbe72d9d640 1 --2- 172.21.15.84:0/3643056817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55658a54a4a0 0x55658a5487e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.027+0000 7fbe72d9d640 1 --2- 172.21.15.84:0/1033669179 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fbe600a2d00 0x7fbe600a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.027+0000 7fbe6b7fe640 1 -- 172.21.15.84:0/1033669179 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7fbe5c0ceee0 con 0x7fbe74105200 2024-05-22T23:09:58.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.027+0000 7fbe6bfff640 1 -- 172.21.15.84:0/3643056817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55658a548db0 msgr2=0x55658a549230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:58.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.027+0000 7fbe6bfff640 1 --2- 172.21.15.84:0/3643056817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55658a548db0 0x55658a549230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.027+0000 7fbe6bfff640 1 -- 172.21.15.84:0/3643056817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55658a54a4a0 msgr2=0x55658a5487e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:58.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.027+0000 7fbe6bfff640 1 --2- 172.21.15.84:0/3643056817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55658a54a4a0 0x55658a5487e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.027+0000 7fbe6bfff640 1 -- 172.21.15.84:0/3643056817 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55658a46ea60 con 0x55658a553b30 2024-05-22T23:09:58.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.027+0000 7fbe72d9d640 1 --2- 172.21.15.84:0/1033669179 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fbe600a2d00 0x7fbe600a51c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c0070b0 tx=0x7fbe6c0093d0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:58.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.027+0000 7fbe6bfff640 1 --2- 172.21.15.84:0/3643056817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a553b30 0x55658a555f20 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0cc510 tx=0x7fbe5c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9248871d664e41e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:58.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.027+0000 7fbe68ff9640 1 -- 172.21.15.84:0/3643056817 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c020030 con 0x55658a553b30 2024-05-22T23:09:58.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.027+0000 7fbe68ff9640 1 -- 172.21.15.84:0/3643056817 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe5c009dc0 con 0x55658a553b30 2024-05-22T23:09:58.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.027+0000 7fbe68ff9640 1 -- 172.21.15.84:0/3643056817 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c097b30 con 0x55658a553b30 2024-05-22T23:09:58.043 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.028+0000 7fbe7bc72740 1 -- 172.21.15.84:0/3643056817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a553b30 msgr2=0x55658a555f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:58.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.028+0000 7fbe7bc72740 1 --2- 172.21.15.84:0/3643056817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a553b30 0x55658a555f20 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0cc510 tx=0x7fbe5c01e000 comp rx=0 tx=0).stop 2024-05-22T23:09:58.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.028+0000 7fbe7bc72740 1 -- 172.21.15.84:0/3643056817 shutdown_connections 2024-05-22T23:09:58.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.028+0000 7fbe7bc72740 1 --2- 172.21.15.84:0/3643056817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a553b30 0x55658a555f20 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.028+0000 7fbe7bc72740 1 --2- 172.21.15.84:0/3643056817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55658a548db0 0x55658a549230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.028+0000 7fbe7bc72740 1 --2- 172.21.15.84:0/3643056817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55658a54a4a0 0x55658a5487e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.028+0000 7fbe7bc72740 1 -- 172.21.15.84:0/3643056817 >> 172.21.15.84:0/3643056817 conn(0x55658a5118e0 msgr2=0x55658a545310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:58.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.028+0000 7fbe7bc72740 1 -- 172.21.15.84:0/3643056817 shutdown_connections 2024-05-22T23:09:58.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.028+0000 7fbe7bc72740 1 -- 172.21.15.84:0/3643056817 wait complete. 2024-05-22T23:09:58.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.028+0000 7fbe7bc72740 1 -- start start 2024-05-22T23:09:58.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.028+0000 7fbe7bc72740 1 Processor -- start 2024-05-22T23:09:58.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.029+0000 7fbe7bc72740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a548db0 0x55658a550b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.029+0000 7fbe7bc72740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55658a54a4a0 0x55658a551060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.029+0000 7fbe7bc72740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55658a553b30 0x55658a5515a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.029+0000 7fbe7bc72740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55658a556960 con 0x55658a548db0 2024-05-22T23:09:58.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.029+0000 7fbe7bc72740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55658a4670f0 con 0x55658a54a4a0 2024-05-22T23:09:58.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.029+0000 7fbe7bc72740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55658a556ae0 con 0x55658a553b30 2024-05-22T23:09:58.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.029+0000 7fbe7bc72740 10 client.0 _ll_register_callbacks cb 0x55658a46bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:09:58.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.029+0000 7fbe7259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a548db0 0x55658a550b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.029+0000 7fbe7259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a548db0 0x55658a550b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37274/0 (socket says 172.21.15.84:37274) 2024-05-22T23:09:58.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.029+0000 7fbe7259c640 1 -- 172.21.15.84:0/4068218898 learned_addr learned my addr 172.21.15.84:0/4068218898 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:58.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.029+0000 7fbe72d9d640 1 --2- 172.21.15.84:0/4068218898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55658a553b30 0x55658a5515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.029+0000 7fbe6bfff640 1 --2- 172.21.15.84:0/4068218898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55658a54a4a0 0x55658a551060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.029+0000 7fbe7259c640 1 -- 172.21.15.84:0/4068218898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55658a553b30 msgr2=0x55658a5515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:58.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.029+0000 7fbe7259c640 1 --2- 172.21.15.84:0/4068218898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55658a553b30 0x55658a5515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.029+0000 7fbe7259c640 1 -- 172.21.15.84:0/4068218898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55658a54a4a0 msgr2=0x55658a551060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:58.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.029+0000 7fbe7259c640 1 --2- 172.21.15.84:0/4068218898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55658a54a4a0 0x55658a551060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.029+0000 7fbe7259c640 1 -- 172.21.15.84:0/4068218898 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55658a28a7d0 con 0x55658a548db0 2024-05-22T23:09:58.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.029+0000 7fbe7259c640 1 --2- 172.21.15.84:0/4068218898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a548db0 0x55658a550b20 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009900 tx=0x7fbe640319b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2008dce5cbcbf08 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:58.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.029+0000 7fbe59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:09:58.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.029+0000 7fbe59ffb640 1 -- 172.21.15.84:0/4068218898 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe64035040 con 0x55658a548db0 2024-05-22T23:09:58.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.029+0000 7fbe59ffb640 1 -- 172.21.15.84:0/4068218898 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7fbe64004300 con 0x55658a548db0 2024-05-22T23:09:58.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.029+0000 7fbe7bc72740 1 -- 172.21.15.84:0/4068218898 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe5c009dc0 con 0x55658a548db0 2024-05-22T23:09:58.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.030+0000 7fbe59ffb640 1 -- 172.21.15.84:0/4068218898 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fbe64005020 con 0x55658a548db0 2024-05-22T23:09:58.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.030+0000 7fbe7bc72740 10 client.16710 fetch_fsmap learned FSMap version 18 2024-05-22T23:09:58.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.030+0000 7fbe7bc72740 1 -- 172.21.15.84:0/4068218898 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55658a1c03f0 con 0x55658a548db0 2024-05-22T23:09:58.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.030+0000 7fbe59ffb640 1 -- 172.21.15.84:0/4068218898 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe5c009dc0 con 0x55658a548db0 2024-05-22T23:09:58.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.030+0000 7fbe59ffb640 1 -- 172.21.15.84:0/4068218898 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fbe6404d070 con 0x55658a548db0 2024-05-22T23:09:58.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.030+0000 7fbe7bc72740 10 client.16710 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:09:58.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.030+0000 7fbe7bc72740 10 client.16710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:09:58.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.030+0000 7fbe7bc72740 20 client.16710 populate_metadata read hostname 'smithi084' 2024-05-22T23:09:58.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.030+0000 7fbe7bc72740 1 --2- 172.21.15.84:0/4068218898 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55658a564070 0x55658a566500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.030+0000 7fbe7bc72740 4 client.16710 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:09:58.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.030+0000 7fbe7bc72740 1 -- 172.21.15.84:0/4068218898 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55658a4ba4e0 con 0x55658a564070 2024-05-22T23:09:58.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.030+0000 7fbe6bfff640 1 --2- 172.21.15.84:0/4068218898 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55658a564070 0x55658a566500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.031+0000 7fbe59ffb640 10 client.16710 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:58.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.031+0000 7fbe6bfff640 1 --2- 172.21.15.84:0/4068218898 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55658a564070 0x55658a566500 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5f20a81135a67d2 server_cookie=40d38b34fd63a56a in_seq=0 out_seq=0 2024-05-22T23:09:58.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.031+0000 7fbe59ffb640 1 -- 172.21.15.84:0/4068218898 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55658a4ba4e0 con 0x55658a564070 2024-05-22T23:09:58.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.031+0000 7fbe59ffb640 10 client.16710 handle_command_reply: tid=0 2024-05-22T23:09:58.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.031+0000 7fbe7bc72740 1 client.16710 shutdown 2024-05-22T23:09:58.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.031+0000 7fbe7bc72740 1 -- 172.21.15.84:0/4068218898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a548db0 msgr2=0x55658a550b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:58.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.031+0000 7fbe7bc72740 1 --2- 172.21.15.84:0/4068218898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a548db0 0x55658a550b20 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009900 tx=0x7fbe640319b0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.031+0000 7fbe7bc72740 1 -- 172.21.15.84:0/4068218898 shutdown_connections 2024-05-22T23:09:58.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.031+0000 7fbe7bc72740 1 --2- 172.21.15.84:0/4068218898 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55658a564070 0x55658a566500 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.031+0000 7fbe7bc72740 1 --2- 172.21.15.84:0/4068218898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55658a553b30 0x55658a5515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.031+0000 7fbe7bc72740 1 --2- 172.21.15.84:0/4068218898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55658a54a4a0 0x55658a551060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.031+0000 7fbe59ffb640 0 client.16710 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:58.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.031+0000 7fbe7bc72740 1 --2- 172.21.15.84:0/4068218898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a548db0 0x55658a550b20 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.031+0000 7fbe7bc72740 1 -- 172.21.15.84:0/4068218898 >> 172.21.15.84:0/4068218898 conn(0x55658a5118e0 msgr2=0x55658a54b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:58.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.032+0000 7fbe7bc72740 1 -- 172.21.15.84:0/4068218898 shutdown_connections 2024-05-22T23:09:58.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.032+0000 7fbe7bc72740 1 -- 172.21.15.84:0/4068218898 wait complete. 2024-05-22T23:09:58.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.032+0000 7fbe7bc72740 20 client.16710 trim_cache size 0 max 16384 2024-05-22T23:09:58.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.049+0000 7fbe7bc72740 1 Processor -- start 2024-05-22T23:09:58.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.049+0000 7fbe7bc72740 1 -- start start 2024-05-22T23:09:58.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.049+0000 7fbe7bc72740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55658a548db0 0x55658a5454e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.049+0000 7fbe7bc72740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a54a4a0 0x55658a55f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.049+0000 7fbe7bc72740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55658a553b30 0x55658a56bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.049+0000 7fbe7bc72740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbe741ab720 con 0x55658a54a4a0 2024-05-22T23:09:58.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.049+0000 7fbe7bc72740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbe741adf80 con 0x55658a548db0 2024-05-22T23:09:58.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.049+0000 7fbe7bc72740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbe741abd70 con 0x55658a553b30 2024-05-22T23:09:58.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.050+0000 7fbe6bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a54a4a0 0x55658a55f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.050+0000 7fbe6bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a54a4a0 0x55658a55f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37286/0 (socket says 172.21.15.84:37286) 2024-05-22T23:09:58.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.050+0000 7fbe6bfff640 1 -- 172.21.15.84:0/4067281991 learned_addr learned my addr 172.21.15.84:0/4067281991 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:58.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.050+0000 7fbe6bfff640 1 -- 172.21.15.84:0/4067281991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55658a553b30 msgr2=0x55658a56bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:58.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.050+0000 7fbe6bfff640 1 --2- 172.21.15.84:0/4067281991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55658a553b30 0x55658a56bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.050+0000 7fbe6bfff640 1 -- 172.21.15.84:0/4067281991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55658a548db0 msgr2=0x55658a5454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:58.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.050+0000 7fbe7259c640 1 --2- 172.21.15.84:0/4067281991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55658a548db0 0x55658a5454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.050+0000 7fbe6bfff640 1 --2- 172.21.15.84:0/4067281991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55658a548db0 0x55658a5454e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.050+0000 7fbe6bfff640 1 -- 172.21.15.84:0/4067281991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55658a1800f0 con 0x55658a54a4a0 2024-05-22T23:09:58.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.050+0000 7fbe6bfff640 1 --2- 172.21.15.84:0/4067281991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a54a4a0 0x55658a55f860 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c097d40 tx=0x7fbe5c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d3ecdb63771eea server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:58.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.050+0000 7fbe5bfff640 1 -- 172.21.15.84:0/4067281991 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c020040 con 0x55658a54a4a0 2024-05-22T23:09:58.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.050+0000 7fbe5bfff640 1 -- 172.21.15.84:0/4067281991 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe5c0a3fe0 con 0x55658a54a4a0 2024-05-22T23:09:58.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.050+0000 7fbe5bfff640 1 -- 172.21.15.84:0/4067281991 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c09a7c0 con 0x55658a54a4a0 2024-05-22T23:09:58.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.051+0000 7fbe7bc72740 1 -- 172.21.15.84:0/4067281991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a54a4a0 msgr2=0x55658a55f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:58.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.051+0000 7fbe7bc72740 1 --2- 172.21.15.84:0/4067281991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a54a4a0 0x55658a55f860 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c097d40 tx=0x7fbe5c009c30 comp rx=0 tx=0).stop 2024-05-22T23:09:58.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.051+0000 7fbe7bc72740 1 -- 172.21.15.84:0/4067281991 shutdown_connections 2024-05-22T23:09:58.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.051+0000 7fbe7bc72740 1 --2- 172.21.15.84:0/4067281991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55658a553b30 0x55658a56bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.051+0000 7fbe7bc72740 1 --2- 172.21.15.84:0/4067281991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a54a4a0 0x55658a55f860 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.051+0000 7fbe7bc72740 1 --2- 172.21.15.84:0/4067281991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55658a548db0 0x55658a5454e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.051+0000 7fbe7bc72740 1 -- 172.21.15.84:0/4067281991 >> 172.21.15.84:0/4067281991 conn(0x55658a5118e0 msgr2=0x55658a544dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:58.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.051+0000 7fbe7bc72740 1 -- 172.21.15.84:0/4067281991 shutdown_connections 2024-05-22T23:09:58.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.051+0000 7fbe7bc72740 1 -- 172.21.15.84:0/4067281991 wait complete. 2024-05-22T23:09:58.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.051+0000 7fbe7bc72740 1 -- start start 2024-05-22T23:09:58.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.051+0000 7fbe7bc72740 1 Processor -- start 2024-05-22T23:09:58.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.051+0000 7fbe7bc72740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55658a548db0 0x55658a55c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.051+0000 7fbe7bc72740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a54a4a0 0x55658a5574f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.051+0000 7fbe7bc72740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55658a553b30 0x55658a557a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.051+0000 7fbe7bc72740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbe741ac440 con 0x55658a54a4a0 2024-05-22T23:09:58.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.051+0000 7fbe7bc72740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbe741ad8b0 con 0x55658a553b30 2024-05-22T23:09:58.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.051+0000 7fbe7bc72740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbe741acb10 con 0x55658a548db0 2024-05-22T23:09:58.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.051+0000 7fbe7bc72740 10 client.0 _ll_register_callbacks cb 0x7fbe64031640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:09:58.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.051+0000 7fbe6bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a54a4a0 0x55658a5574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.051+0000 7fbe6bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a54a4a0 0x55658a5574f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37288/0 (socket says 172.21.15.84:37288) 2024-05-22T23:09:58.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.051+0000 7fbe6bfff640 1 -- 172.21.15.84:0/382093359 learned_addr learned my addr 172.21.15.84:0/382093359 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:58.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.052+0000 7fbe72d9d640 1 --2- 172.21.15.84:0/382093359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55658a553b30 0x55658a557a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.052+0000 7fbe6bfff640 1 -- 172.21.15.84:0/382093359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55658a548db0 msgr2=0x55658a55c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:58.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.052+0000 7fbe7259c640 1 --2- 172.21.15.84:0/382093359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55658a548db0 0x55658a55c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.052+0000 7fbe6bfff640 1 --2- 172.21.15.84:0/382093359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55658a548db0 0x55658a55c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.052+0000 7fbe6bfff640 1 -- 172.21.15.84:0/382093359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55658a553b30 msgr2=0x55658a557a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:58.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.052+0000 7fbe6bfff640 1 --2- 172.21.15.84:0/382093359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55658a553b30 0x55658a557a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.052+0000 7fbe6bfff640 1 -- 172.21.15.84:0/382093359 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55658a46bfe0 con 0x55658a54a4a0 2024-05-22T23:09:58.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.052+0000 7fbe72d9d640 1 --2- 172.21.15.84:0/382093359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55658a553b30 0x55658a557a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:09:58.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.052+0000 7fbe6bfff640 1 --2- 172.21.15.84:0/382093359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a54a4a0 0x55658a5574f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c09ae40 tx=0x7fbe5c0cf1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bfad48551441a38 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:58.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.052+0000 7fbe59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:09:58.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.052+0000 7fbe59ffb640 1 -- 172.21.15.84:0/382093359 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c020070 con 0x55658a54a4a0 2024-05-22T23:09:58.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.052+0000 7fbe59ffb640 1 -- 172.21.15.84:0/382093359 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7fbe5c097a60 con 0x55658a54a4a0 2024-05-22T23:09:58.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.052+0000 7fbe7bc72740 1 -- 172.21.15.84:0/382093359 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe5c0a3fe0 con 0x55658a54a4a0 2024-05-22T23:09:58.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.052+0000 7fbe59ffb640 1 -- 172.21.15.84:0/382093359 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fbe5c09b370 con 0x55658a54a4a0 2024-05-22T23:09:58.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.052+0000 7fbe7bc72740 10 client.16716 fetch_fsmap learned FSMap version 18 2024-05-22T23:09:58.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.052+0000 7fbe7bc72740 1 -- 172.21.15.84:0/382093359 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55658a43d1c0 con 0x55658a54a4a0 2024-05-22T23:09:58.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.052+0000 7fbe59ffb640 1 -- 172.21.15.84:0/382093359 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe5c0a3fe0 con 0x55658a54a4a0 2024-05-22T23:09:58.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.052+0000 7fbe59ffb640 1 -- 172.21.15.84:0/382093359 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fbe5c09a350 con 0x55658a54a4a0 2024-05-22T23:09:58.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.052+0000 7fbe7bc72740 10 client.16716 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:09:58.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.052+0000 7fbe7bc72740 10 client.16716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:09:58.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.052+0000 7fbe7bc72740 20 client.16716 populate_metadata read hostname 'smithi084' 2024-05-22T23:09:58.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.053+0000 7fbe7bc72740 1 --2- 172.21.15.84:0/382093359 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55658a5ca580 0x55658a5cc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.053+0000 7fbe7bc72740 4 client.16716 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:09:58.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.053+0000 7fbe7bc72740 1 -- 172.21.15.84:0/382093359 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55658a4b4830 con 0x55658a5ca580 2024-05-22T23:09:58.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.053+0000 7fbe7259c640 1 --2- 172.21.15.84:0/382093359 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55658a5ca580 0x55658a5cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.053+0000 7fbe59ffb640 10 client.16716 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:58.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.053+0000 7fbe7259c640 1 --2- 172.21.15.84:0/382093359 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55658a5ca580 0x55658a5cc970 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a62dc8f977167c45 server_cookie=6a26a40075195ec6 in_seq=0 out_seq=0 2024-05-22T23:09:58.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.053+0000 7fbe59ffb640 1 -- 172.21.15.84:0/382093359 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55658a4b4830 con 0x55658a5ca580 2024-05-22T23:09:58.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.053+0000 7fbe59ffb640 10 client.16716 handle_command_reply: tid=0 2024-05-22T23:09:58.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe7bc72740 1 client.16716 shutdown 2024-05-22T23:09:58.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe7bc72740 1 -- 172.21.15.84:0/382093359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a54a4a0 msgr2=0x55658a5574f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:58.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe7bc72740 1 --2- 172.21.15.84:0/382093359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a54a4a0 0x55658a5574f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c09ae40 tx=0x7fbe5c0cf1e0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe7bc72740 1 -- 172.21.15.84:0/382093359 shutdown_connections 2024-05-22T23:09:58.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe7bc72740 1 --2- 172.21.15.84:0/382093359 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55658a5ca580 0x55658a5cc970 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe7bc72740 1 --2- 172.21.15.84:0/382093359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55658a553b30 0x55658a557a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe59ffb640 0 client.16716 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:58.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe7bc72740 1 --2- 172.21.15.84:0/382093359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55658a54a4a0 0x55658a5574f0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe7bc72740 1 --2- 172.21.15.84:0/382093359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55658a548db0 0x55658a55c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe7bc72740 1 -- 172.21.15.84:0/382093359 >> 172.21.15.84:0/382093359 conn(0x55658a5118e0 msgr2=0x55658a552350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:58.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe7bc72740 1 -- 172.21.15.84:0/382093359 shutdown_connections 2024-05-22T23:09:58.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe7bc72740 1 -- 172.21.15.84:0/382093359 wait complete. 2024-05-22T23:09:58.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe7bc72740 20 client.16716 trim_cache size 0 max 16384 2024-05-22T23:09:58.072 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:09:58.073 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "no active scrubs running", 2024-05-22T23:09:58.073 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": {} 2024-05-22T23:09:58.073 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:09:58.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe78f68640 1 -- 172.21.15.84:0/1033669179 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fbe600a2d00 msgr2=0x7fbe600a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:58.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe78f68640 1 --2- 172.21.15.84:0/1033669179 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fbe600a2d00 0x7fbe600a51c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c0070b0 tx=0x7fbe6c0093d0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe78f68640 1 -- 172.21.15.84:0/1033669179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe74105200 msgr2=0x7fbe74198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:58.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe78f68640 1 --2- 172.21.15.84:0/1033669179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe74105200 0x7fbe74198770 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00d740 tx=0x7fbe5c00dc00 comp rx=0 tx=0).stop 2024-05-22T23:09:58.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe78f68640 1 -- 172.21.15.84:0/1033669179 shutdown_connections 2024-05-22T23:09:58.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe78f68640 1 --2- 172.21.15.84:0/1033669179 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fbe600a2d00 0x7fbe600a51c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe78f68640 1 --2- 172.21.15.84:0/1033669179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe7419caf0 0x7fbe7419cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe78f68640 1 --2- 172.21.15.84:0/1033669179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe74105200 0x7fbe74198770 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe78f68640 1 --2- 172.21.15.84:0/1033669179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe74100130 0x7fbe74198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe78f68640 1 -- 172.21.15.84:0/1033669179 >> 172.21.15.84:0/1033669179 conn(0x7fbe74076d30 msgr2=0x7fbe740fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:58.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe78f68640 1 -- 172.21.15.84:0/1033669179 shutdown_connections 2024-05-22T23:09:58.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.054+0000 7fbe78f68640 1 -- 172.21.15.84:0/1033669179 wait complete. 2024-05-22T23:09:58.076 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-22T23:09:58.076 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:09:58.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.170+0000 7ff408aa5640 1 -- 172.21.15.84:0/35510975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff404107880 msgr2=0x7ff404107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:58.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.170+0000 7ff408aa5640 1 --2- 172.21.15.84:0/35510975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff404107880 0x7ff404107c60 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009900 tx=0x7ff3f402a480 comp rx=0 tx=0).stop 2024-05-22T23:09:58.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.170+0000 7ff408aa5640 1 -- 172.21.15.84:0/35510975 shutdown_connections 2024-05-22T23:09:58.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.170+0000 7ff408aa5640 1 --2- 172.21.15.84:0/35510975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff404102290 0x7ff40410e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.170+0000 7ff408aa5640 1 --2- 172.21.15.84:0/35510975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4041018f0 0x7ff404101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.170+0000 7ff408aa5640 1 --2- 172.21.15.84:0/35510975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff404107880 0x7ff404107c60 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.170+0000 7ff408aa5640 1 -- 172.21.15.84:0/35510975 >> 172.21.15.84:0/35510975 conn(0x7ff404078a00 msgr2=0x7ff404078e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:58.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.170+0000 7ff408aa5640 1 -- 172.21.15.84:0/35510975 shutdown_connections 2024-05-22T23:09:58.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.170+0000 7ff408aa5640 1 -- 172.21.15.84:0/35510975 wait complete. 2024-05-22T23:09:58.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.171+0000 7ff408aa5640 1 Processor -- start 2024-05-22T23:09:58.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.171+0000 7ff408aa5640 1 -- start start 2024-05-22T23:09:58.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.171+0000 7ff408aa5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4041018f0 0x7ff404198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.171+0000 7ff408aa5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff404102290 0x7ff4041986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.171+0000 7ff408aa5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff404107880 0x7ff40419c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.171+0000 7ff408aa5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff404111c00 con 0x7ff404102290 2024-05-22T23:09:58.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.171+0000 7ff408aa5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff404111a80 con 0x7ff4041018f0 2024-05-22T23:09:58.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.171+0000 7ff408aa5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff404111d80 con 0x7ff404107880 2024-05-22T23:09:58.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.171+0000 7ff401d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff404102290 0x7ff4041986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.171+0000 7ff401d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff404102290 0x7ff4041986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37310/0 (socket says 172.21.15.84:37310) 2024-05-22T23:09:58.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.171+0000 7ff401d9b640 1 -- 172.21.15.84:0/2422061478 learned_addr learned my addr 172.21.15.84:0/2422061478 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:58.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.171+0000 7ff402d9d640 1 --2- 172.21.15.84:0/2422061478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff404107880 0x7ff40419c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.171+0000 7ff40259c640 1 --2- 172.21.15.84:0/2422061478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4041018f0 0x7ff404198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.172+0000 7ff401d9b640 1 -- 172.21.15.84:0/2422061478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff404107880 msgr2=0x7ff40419c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:58.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.172+0000 7ff401d9b640 1 --2- 172.21.15.84:0/2422061478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff404107880 0x7ff40419c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.172+0000 7ff401d9b640 1 -- 172.21.15.84:0/2422061478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4041018f0 msgr2=0x7ff404198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:58.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.172+0000 7ff401d9b640 1 --2- 172.21.15.84:0/2422061478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4041018f0 0x7ff404198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.172+0000 7ff401d9b640 1 -- 172.21.15.84:0/2422061478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40419d120 con 0x7ff404102290 2024-05-22T23:09:58.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.172+0000 7ff401d9b640 1 --2- 172.21.15.84:0/2422061478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff404102290 0x7ff4041986c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800d950 tx=0x7ff3f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:58.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.172+0000 7ff3f37fe640 1 -- 172.21.15.84:0/2422061478 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8017070 con 0x7ff404102290 2024-05-22T23:09:58.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.172+0000 7ff3f37fe640 1 -- 172.21.15.84:0/2422061478 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f8009c30 con 0x7ff404102290 2024-05-22T23:09:58.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.172+0000 7ff408aa5640 1 -- 172.21.15.84:0/2422061478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff40419d410 con 0x7ff404102290 2024-05-22T23:09:58.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.172+0000 7ff3f37fe640 1 -- 172.21.15.84:0/2422061478 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f800f6b0 con 0x7ff404102290 2024-05-22T23:09:58.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.172+0000 7ff408aa5640 1 -- 172.21.15.84:0/2422061478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4041a4ca0 con 0x7ff404102290 2024-05-22T23:09:58.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.173+0000 7ff408aa5640 1 -- 172.21.15.84:0/2422061478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff404068ae0 con 0x7ff404102290 2024-05-22T23:09:58.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.174+0000 7ff3f37fe640 1 -- 172.21.15.84:0/2422061478 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff3f80109d0 con 0x7ff404102290 2024-05-22T23:09:58.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.175+0000 7ff3f37fe640 1 --2- 172.21.15.84:0/2422061478 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff3ec0a2d00 0x7ff3ec0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.175+0000 7ff40259c640 1 --2- 172.21.15.84:0/2422061478 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff3ec0a2d00 0x7ff3ec0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.175+0000 7ff3f37fe640 1 -- 172.21.15.84:0/2422061478 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7ff3f80cf090 con 0x7ff404102290 2024-05-22T23:09:58.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.175+0000 7ff40259c640 1 --2- 172.21.15.84:0/2422061478 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff3ec0a2d00 0x7ff3ec0a51c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4005ab0 tx=0x7ff3f4005a20 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:58.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.179+0000 7ff3f37fe640 1 -- 172.21.15.84:0/2422061478 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff3f8009de0 con 0x7ff404102290 2024-05-22T23:09:58.396 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.389+0000 7ff408aa5640 1 -- 172.21.15.84:0/2422061478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff404199480 con 0x7ff404102290 2024-05-22T23:09:58.396 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.390+0000 7ff3f37fe640 1 -- 172.21.15.84:0/2422061478 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7ff404199480 con 0x7ff404102290 2024-05-22T23:09:58.397 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:09:58.397 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:09:58.397 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:09:58.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.392+0000 7ff408aa5640 1 -- 172.21.15.84:0/2422061478 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff3ec0a2d00 msgr2=0x7ff3ec0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:58.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.392+0000 7ff408aa5640 1 --2- 172.21.15.84:0/2422061478 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff3ec0a2d00 0x7ff3ec0a51c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4005ab0 tx=0x7ff3f4005a20 comp rx=0 tx=0).stop 2024-05-22T23:09:58.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.392+0000 7ff408aa5640 1 -- 172.21.15.84:0/2422061478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff404102290 msgr2=0x7ff4041986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:58.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.392+0000 7ff408aa5640 1 --2- 172.21.15.84:0/2422061478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff404102290 0x7ff4041986c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800d950 tx=0x7ff3f800de10 comp rx=0 tx=0).stop 2024-05-22T23:09:58.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.392+0000 7ff408aa5640 1 -- 172.21.15.84:0/2422061478 shutdown_connections 2024-05-22T23:09:58.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.392+0000 7ff408aa5640 1 --2- 172.21.15.84:0/2422061478 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff3ec0a2d00 0x7ff3ec0a51c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.392+0000 7ff408aa5640 1 --2- 172.21.15.84:0/2422061478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff404107880 0x7ff40419c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.392+0000 7ff408aa5640 1 --2- 172.21.15.84:0/2422061478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff404102290 0x7ff4041986c0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.392+0000 7ff408aa5640 1 --2- 172.21.15.84:0/2422061478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4041018f0 0x7ff404198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.392+0000 7ff408aa5640 1 -- 172.21.15.84:0/2422061478 >> 172.21.15.84:0/2422061478 conn(0x7ff404078a00 msgr2=0x7ff404100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:58.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.392+0000 7ff408aa5640 1 -- 172.21.15.84:0/2422061478 shutdown_connections 2024-05-22T23:09:58.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.392+0000 7ff408aa5640 1 -- 172.21.15.84:0/2422061478 wait complete. 2024-05-22T23:09:58.410 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-22T23:09:58.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.507+0000 7ff386ef5640 1 -- 172.21.15.84:0/3403373015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff380106750 msgr2=0x7ff380106b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:58.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.507+0000 7ff386ef5640 1 --2- 172.21.15.84:0/3403373015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff380106750 0x7ff380106b30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff378009930 tx=0x7ff37802a420 comp rx=0 tx=0).stop 2024-05-22T23:09:58.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.507+0000 7ff386ef5640 1 -- 172.21.15.84:0/3403373015 shutdown_connections 2024-05-22T23:09:58.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.507+0000 7ff386ef5640 1 --2- 172.21.15.84:0/3403373015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff380101100 0x7ff38010d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.507+0000 7ff386ef5640 1 --2- 172.21.15.84:0/3403373015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff380100740 0x7ff380100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.507+0000 7ff386ef5640 1 --2- 172.21.15.84:0/3403373015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff380106750 0x7ff380106b30 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.507+0000 7ff386ef5640 1 -- 172.21.15.84:0/3403373015 >> 172.21.15.84:0/3403373015 conn(0x7ff380077870 msgr2=0x7ff380077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:58.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.507+0000 7ff386ef5640 1 -- 172.21.15.84:0/3403373015 shutdown_connections 2024-05-22T23:09:58.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.507+0000 7ff386ef5640 1 -- 172.21.15.84:0/3403373015 wait complete. 2024-05-22T23:09:58.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.507+0000 7ff386ef5640 1 Processor -- start 2024-05-22T23:09:58.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.507+0000 7ff386ef5640 1 -- start start 2024-05-22T23:09:58.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.508+0000 7ff386ef5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff380100740 0x7ff380198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.508+0000 7ff386ef5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff380101100 0x7ff380198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.508+0000 7ff386ef5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff380106750 0x7ff38019c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.508+0000 7ff386ef5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff380111c60 con 0x7ff380100740 2024-05-22T23:09:58.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.508+0000 7ff386ef5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff380111ae0 con 0x7ff380106750 2024-05-22T23:09:58.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.508+0000 7ff386ef5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff380111de0 con 0x7ff380101100 2024-05-22T23:09:58.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.508+0000 7ff384c91640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff380100740 0x7ff380198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.508+0000 7ff384c91640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff380100740 0x7ff380198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37338/0 (socket says 172.21.15.84:37338) 2024-05-22T23:09:58.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.508+0000 7ff384c91640 1 -- 172.21.15.84:0/4231578188 learned_addr learned my addr 172.21.15.84:0/4231578188 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:58.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.508+0000 7ff385492640 1 --2- 172.21.15.84:0/4231578188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff380106750 0x7ff38019c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.508+0000 7ff384c91640 1 -- 172.21.15.84:0/4231578188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff380101100 msgr2=0x7ff380198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:58.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.508+0000 7ff37ffff640 1 --2- 172.21.15.84:0/4231578188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff380101100 0x7ff380198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.508+0000 7ff384c91640 1 --2- 172.21.15.84:0/4231578188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff380101100 0x7ff380198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.508+0000 7ff384c91640 1 -- 172.21.15.84:0/4231578188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff380106750 msgr2=0x7ff38019c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:58.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.508+0000 7ff384c91640 1 --2- 172.21.15.84:0/4231578188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff380106750 0x7ff38019c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.508+0000 7ff384c91640 1 -- 172.21.15.84:0/4231578188 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38019d0c0 con 0x7ff380100740 2024-05-22T23:09:58.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.508+0000 7ff385492640 1 --2- 172.21.15.84:0/4231578188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff380106750 0x7ff38019c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:09:58.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.508+0000 7ff384c91640 1 --2- 172.21.15.84:0/4231578188 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff380100740 0x7ff380198120 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff378009a50 tx=0x7ff378004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:58.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.508+0000 7ff37dffb640 1 -- 172.21.15.84:0/4231578188 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37803f030 con 0x7ff380100740 2024-05-22T23:09:58.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.508+0000 7ff37dffb640 1 -- 172.21.15.84:0/4231578188 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff378003ce0 con 0x7ff380100740 2024-05-22T23:09:58.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.508+0000 7ff386ef5640 1 -- 172.21.15.84:0/4231578188 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38019d350 con 0x7ff380100740 2024-05-22T23:09:58.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.508+0000 7ff37dffb640 1 -- 172.21.15.84:0/4231578188 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378035070 con 0x7ff380100740 2024-05-22T23:09:58.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.509+0000 7ff386ef5640 1 -- 172.21.15.84:0/4231578188 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3801a4c40 con 0x7ff380100740 2024-05-22T23:09:58.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.511+0000 7ff37dffb640 1 -- 172.21.15.84:0/4231578188 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff378042030 con 0x7ff380100740 2024-05-22T23:09:58.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.511+0000 7ff389c00740 1 Processor -- start 2024-05-22T23:09:58.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.511+0000 7ff389c00740 1 -- start start 2024-05-22T23:09:58.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff389c00740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559693230f80 0x559693231360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff389c00740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596932318a0 0x55969323ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff389c00740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55969323bf70 0x55969323e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff389c00740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559693195db0 con 0x55969323bf70 2024-05-22T23:09:58.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff389c00740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559693134400 con 0x559693230f80 2024-05-22T23:09:58.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff389c00740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559693191d70 con 0x5596932318a0 2024-05-22T23:09:58.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff37dffb640 1 --2- 172.21.15.84:0/4231578188 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff36c0a2d00 0x7ff36c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff37ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55969323bf70 0x55969323e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff37ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55969323bf70 0x55969323e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37340/0 (socket says 172.21.15.84:37340) 2024-05-22T23:09:58.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff37ffff640 1 -- 172.21.15.84:0/1547814049 learned_addr learned my addr 172.21.15.84:0/1547814049 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:58.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff385492640 1 --2- 172.21.15.84:0/4231578188 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff36c0a2d00 0x7ff36c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff385492640 1 --2- 172.21.15.84:0/1547814049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559693230f80 0x559693231360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff37dffb640 1 -- 172.21.15.84:0/4231578188 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7ff3780efb10 con 0x7ff380100740 2024-05-22T23:09:58.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff384c91640 1 --2- 172.21.15.84:0/1547814049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596932318a0 0x55969323ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff37ffff640 1 -- 172.21.15.84:0/1547814049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596932318a0 msgr2=0x55969323ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:58.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff37ffff640 1 --2- 172.21.15.84:0/1547814049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596932318a0 0x55969323ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff37ffff640 1 -- 172.21.15.84:0/1547814049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559693230f80 msgr2=0x559693231360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:58.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff37ffff640 1 --2- 172.21.15.84:0/1547814049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559693230f80 0x559693231360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff37ffff640 1 -- 172.21.15.84:0/1547814049 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596931262f0 con 0x55969323bf70 2024-05-22T23:09:58.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff384c91640 1 --2- 172.21.15.84:0/1547814049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596932318a0 0x55969323ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:09:58.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff37ffff640 1 --2- 172.21.15.84:0/1547814049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55969323bf70 0x55969323e360 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff370008d60 tx=0x7ff37000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0bcf0b325e2f33 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:58.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff385492640 1 --2- 172.21.15.84:0/4231578188 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff36c0a2d00 0x7ff36c0a51c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff36800df00 tx=0x7ff36800d5b0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:58.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff376ffd640 1 -- 172.21.15.84:0/1547814049 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37000f830 con 0x55969323bf70 2024-05-22T23:09:58.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff376ffd640 1 -- 172.21.15.84:0/1547814049 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff37000de20 con 0x55969323bf70 2024-05-22T23:09:58.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.512+0000 7ff376ffd640 1 -- 172.21.15.84:0/1547814049 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37000e3b0 con 0x55969323bf70 2024-05-22T23:09:58.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.513+0000 7ff389c00740 1 -- 172.21.15.84:0/1547814049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55969323bf70 msgr2=0x55969323e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:58.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.513+0000 7ff389c00740 1 --2- 172.21.15.84:0/1547814049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55969323bf70 0x55969323e360 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff370008d60 tx=0x7ff37000d710 comp rx=0 tx=0).stop 2024-05-22T23:09:58.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.513+0000 7ff389c00740 1 -- 172.21.15.84:0/1547814049 shutdown_connections 2024-05-22T23:09:58.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.513+0000 7ff389c00740 1 --2- 172.21.15.84:0/1547814049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55969323bf70 0x55969323e360 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.513+0000 7ff389c00740 1 --2- 172.21.15.84:0/1547814049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596932318a0 0x55969323ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.513+0000 7ff389c00740 1 --2- 172.21.15.84:0/1547814049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559693230f80 0x559693231360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.513+0000 7ff389c00740 1 -- 172.21.15.84:0/1547814049 >> 172.21.15.84:0/1547814049 conn(0x55969322d2a0 msgr2=0x55969322d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:58.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.513+0000 7ff389c00740 1 -- 172.21.15.84:0/1547814049 shutdown_connections 2024-05-22T23:09:58.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.513+0000 7ff389c00740 1 -- 172.21.15.84:0/1547814049 wait complete. 2024-05-22T23:09:58.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.513+0000 7ff389c00740 1 -- start start 2024-05-22T23:09:58.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.513+0000 7ff389c00740 1 Processor -- start 2024-05-22T23:09:58.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.514+0000 7ff389c00740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559693230f80 0x559693238eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.514+0000 7ff389c00740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596932318a0 0x5596932393f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.514+0000 7ff389c00740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55969323bf70 0x559693239930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.514+0000 7ff389c00740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55969323ed60 con 0x5596932318a0 2024-05-22T23:09:58.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.514+0000 7ff389c00740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5596931502e0 con 0x55969323bf70 2024-05-22T23:09:58.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.514+0000 7ff389c00740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55969323eee0 con 0x559693230f80 2024-05-22T23:09:58.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.514+0000 7ff389c00740 10 client.0 _ll_register_callbacks cb 0x559693159a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:09:58.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.514+0000 7ff37ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596932318a0 0x5596932393f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.514+0000 7ff37ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596932318a0 0x5596932393f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37342/0 (socket says 172.21.15.84:37342) 2024-05-22T23:09:58.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.514+0000 7ff37ffff640 1 -- 172.21.15.84:0/1318918824 learned_addr learned my addr 172.21.15.84:0/1318918824 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:58.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.514+0000 7ff385492640 1 --2- 172.21.15.84:0/1318918824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55969323bf70 0x559693239930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.514+0000 7ff384c91640 1 --2- 172.21.15.84:0/1318918824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559693230f80 0x559693238eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.514+0000 7ff37ffff640 1 -- 172.21.15.84:0/1318918824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559693230f80 msgr2=0x559693238eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:58.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.514+0000 7ff37ffff640 1 --2- 172.21.15.84:0/1318918824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559693230f80 0x559693238eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.514+0000 7ff37ffff640 1 -- 172.21.15.84:0/1318918824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55969323bf70 msgr2=0x559693239930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:58.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.514+0000 7ff37ffff640 1 --2- 172.21.15.84:0/1318918824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55969323bf70 0x559693239930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.514+0000 7ff37ffff640 1 -- 172.21.15.84:0/1318918824 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559692ea93f0 con 0x5596932318a0 2024-05-22T23:09:58.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.514+0000 7ff385492640 1 --2- 172.21.15.84:0/1318918824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55969323bf70 0x559693239930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:09:58.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.514+0000 7ff37ffff640 1 --2- 172.21.15.84:0/1318918824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596932318a0 0x5596932393f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff370008d30 tx=0x7ff3700186b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea101962e5a8c1d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:58.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.514+0000 7ff35ffff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:09:58.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.515+0000 7ff35ffff640 1 -- 172.21.15.84:0/1318918824 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff370028060 con 0x5596932318a0 2024-05-22T23:09:58.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.515+0000 7ff35ffff640 1 -- 172.21.15.84:0/1318918824 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7ff37000f630 con 0x5596932318a0 2024-05-22T23:09:58.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.515+0000 7ff389c00740 1 -- 172.21.15.84:0/1318918824 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff37000de20 con 0x5596932318a0 2024-05-22T23:09:58.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.515+0000 7ff35ffff640 1 -- 172.21.15.84:0/1318918824 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7ff370004e50 con 0x5596932318a0 2024-05-22T23:09:58.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.515+0000 7ff389c00740 10 client.16734 fetch_fsmap learned FSMap version 18 2024-05-22T23:09:58.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.515+0000 7ff389c00740 1 -- 172.21.15.84:0/1318918824 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x559692f737d0 con 0x5596932318a0 2024-05-22T23:09:58.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.515+0000 7ff35ffff640 1 -- 172.21.15.84:0/1318918824 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff37000de20 con 0x5596932318a0 2024-05-22T23:09:58.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.515+0000 7ff35ffff640 1 -- 172.21.15.84:0/1318918824 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7ff370005560 con 0x5596932318a0 2024-05-22T23:09:58.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.515+0000 7ff389c00740 10 client.16734 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:09:58.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.515+0000 7ff389c00740 10 client.16734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:09:58.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.515+0000 7ff389c00740 20 client.16734 populate_metadata read hostname 'smithi084' 2024-05-22T23:09:58.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.515+0000 7ff389c00740 1 --2- 172.21.15.84:0/1318918824 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55969324c150 0x55969324e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.515+0000 7ff389c00740 4 client.16734 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:09:58.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.515+0000 7ff389c00740 1 -- 172.21.15.84:0/1318918824 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596931a34e0 con 0x55969324c150 2024-05-22T23:09:58.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.516+0000 7ff384c91640 1 --2- 172.21.15.84:0/1318918824 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55969324c150 0x55969324e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.516+0000 7ff35ffff640 10 client.16734 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:58.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.516+0000 7ff384c91640 1 --2- 172.21.15.84:0/1318918824 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55969324c150 0x55969324e540 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4ee5cc72664637c server_cookie=1f4d582320760686 in_seq=0 out_seq=0 2024-05-22T23:09:58.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.517+0000 7ff35ffff640 1 -- 172.21.15.84:0/1318918824 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5596931a34e0 con 0x55969324c150 2024-05-22T23:09:58.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.517+0000 7ff35ffff640 10 client.16734 handle_command_reply: tid=0 2024-05-22T23:09:58.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.517+0000 7ff389c00740 1 client.16734 shutdown 2024-05-22T23:09:58.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.517+0000 7ff389c00740 1 -- 172.21.15.84:0/1318918824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596932318a0 msgr2=0x5596932393f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:58.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.517+0000 7ff389c00740 1 --2- 172.21.15.84:0/1318918824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596932318a0 0x5596932393f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff370008d30 tx=0x7ff3700186b0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.517+0000 7ff389c00740 1 -- 172.21.15.84:0/1318918824 shutdown_connections 2024-05-22T23:09:58.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.517+0000 7ff389c00740 1 --2- 172.21.15.84:0/1318918824 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55969324c150 0x55969324e540 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.517+0000 7ff389c00740 1 --2- 172.21.15.84:0/1318918824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55969323bf70 0x559693239930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.517+0000 7ff389c00740 1 --2- 172.21.15.84:0/1318918824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596932318a0 0x5596932393f0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.517+0000 7ff35ffff640 0 client.16734 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:58.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.517+0000 7ff389c00740 1 --2- 172.21.15.84:0/1318918824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559693230f80 0x559693238eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.517+0000 7ff389c00740 1 -- 172.21.15.84:0/1318918824 >> 172.21.15.84:0/1318918824 conn(0x55969322d2a0 msgr2=0x5596932330a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:58.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.517+0000 7ff389c00740 1 -- 172.21.15.84:0/1318918824 shutdown_connections 2024-05-22T23:09:58.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.517+0000 7ff389c00740 1 -- 172.21.15.84:0/1318918824 wait complete. 2024-05-22T23:09:58.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.517+0000 7ff389c00740 20 client.16734 trim_cache size 0 max 16384 2024-05-22T23:09:58.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.531+0000 7ff389c00740 1 Processor -- start 2024-05-22T23:09:58.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.531+0000 7ff389c00740 1 -- start start 2024-05-22T23:09:58.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.531+0000 7ff389c00740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559693230f80 0x559693247c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.531+0000 7ff389c00740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596932318a0 0x559693233720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.531+0000 7ff389c00740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55969323bf70 0x559693253b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.531+0000 7ff389c00740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801ab690 con 0x55969323bf70 2024-05-22T23:09:58.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.531+0000 7ff389c00740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801adef0 con 0x5596932318a0 2024-05-22T23:09:58.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.531+0000 7ff389c00740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801abce0 con 0x559693230f80 2024-05-22T23:09:58.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.531+0000 7ff385492640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55969323bf70 0x559693253b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.531+0000 7ff385492640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55969323bf70 0x559693253b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37356/0 (socket says 172.21.15.84:37356) 2024-05-22T23:09:58.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.531+0000 7ff385492640 1 -- 172.21.15.84:0/3630961826 learned_addr learned my addr 172.21.15.84:0/3630961826 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:58.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.531+0000 7ff37ffff640 1 --2- 172.21.15.84:0/3630961826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596932318a0 0x559693233720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.531+0000 7ff384c91640 1 --2- 172.21.15.84:0/3630961826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559693230f80 0x559693247c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.531+0000 7ff385492640 1 -- 172.21.15.84:0/3630961826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559693230f80 msgr2=0x559693247c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:58.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.531+0000 7ff385492640 1 --2- 172.21.15.84:0/3630961826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559693230f80 0x559693247c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.531+0000 7ff385492640 1 -- 172.21.15.84:0/3630961826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596932318a0 msgr2=0x559693233720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:58.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.531+0000 7ff385492640 1 --2- 172.21.15.84:0/3630961826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596932318a0 0x559693233720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.531+0000 7ff385492640 1 -- 172.21.15.84:0/3630961826 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596932316d0 con 0x55969323bf70 2024-05-22T23:09:58.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.531+0000 7ff385492640 1 --2- 172.21.15.84:0/3630961826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55969323bf70 0x559693253b40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff36800da70 tx=0x7ff36801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617b535fc951a457 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:58.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.531+0000 7ff3767fc640 1 -- 172.21.15.84:0/3630961826 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff368020480 con 0x55969323bf70 2024-05-22T23:09:58.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.531+0000 7ff3767fc640 1 -- 172.21.15.84:0/3630961826 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff368020b00 con 0x55969323bf70 2024-05-22T23:09:58.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.531+0000 7ff3767fc640 1 -- 172.21.15.84:0/3630961826 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36801fac0 con 0x55969323bf70 2024-05-22T23:09:58.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.532+0000 7ff389c00740 1 -- 172.21.15.84:0/3630961826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55969323bf70 msgr2=0x559693253b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:58.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.532+0000 7ff389c00740 1 --2- 172.21.15.84:0/3630961826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55969323bf70 0x559693253b40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff36800da70 tx=0x7ff36801b000 comp rx=0 tx=0).stop 2024-05-22T23:09:58.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.532+0000 7ff389c00740 1 -- 172.21.15.84:0/3630961826 shutdown_connections 2024-05-22T23:09:58.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.532+0000 7ff389c00740 1 --2- 172.21.15.84:0/3630961826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55969323bf70 0x559693253b40 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.532+0000 7ff389c00740 1 --2- 172.21.15.84:0/3630961826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596932318a0 0x559693233720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.532+0000 7ff389c00740 1 --2- 172.21.15.84:0/3630961826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559693230f80 0x559693247c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.532+0000 7ff389c00740 1 -- 172.21.15.84:0/3630961826 >> 172.21.15.84:0/3630961826 conn(0x55969322d2a0 msgr2=0x559693238e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:58.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.532+0000 7ff389c00740 1 -- 172.21.15.84:0/3630961826 shutdown_connections 2024-05-22T23:09:58.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.532+0000 7ff389c00740 1 -- 172.21.15.84:0/3630961826 wait complete. 2024-05-22T23:09:58.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.532+0000 7ff389c00740 1 -- start start 2024-05-22T23:09:58.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.532+0000 7ff389c00740 1 Processor -- start 2024-05-22T23:09:58.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff389c00740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559693230f80 0x559693235bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff389c00740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596932318a0 0x559693236100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff389c00740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55969323bf70 0x559693236640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff389c00740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801ac3b0 con 0x559693230f80 2024-05-22T23:09:58.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff389c00740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801ad820 con 0x55969323bf70 2024-05-22T23:09:58.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff389c00740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801aca80 con 0x5596932318a0 2024-05-22T23:09:58.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff389c00740 10 client.0 _ll_register_callbacks cb 0x7ff370008bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:09:58.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff384c91640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559693230f80 0x559693235bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff384c91640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559693230f80 0x559693235bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37366/0 (socket says 172.21.15.84:37366) 2024-05-22T23:09:58.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff384c91640 1 -- 172.21.15.84:0/2267665351 learned_addr learned my addr 172.21.15.84:0/2267665351 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:58.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff385492640 1 --2- 172.21.15.84:0/2267665351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55969323bf70 0x559693236640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff384c91640 1 -- 172.21.15.84:0/2267665351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596932318a0 msgr2=0x559693236100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:58.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff384c91640 1 --2- 172.21.15.84:0/2267665351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596932318a0 0x559693236100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff384c91640 1 -- 172.21.15.84:0/2267665351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55969323bf70 msgr2=0x559693236640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:58.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff384c91640 1 --2- 172.21.15.84:0/2267665351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55969323bf70 0x559693236640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff384c91640 1 -- 172.21.15.84:0/2267665351 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596932b13b0 con 0x559693230f80 2024-05-22T23:09:58.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff385492640 1 --2- 172.21.15.84:0/2267665351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55969323bf70 0x559693236640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:09:58.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff384c91640 1 --2- 172.21.15.84:0/2267665351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559693230f80 0x559693235bc0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff378031550 tx=0x7ff3780314b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171b8e11c7aaa08f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:58.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff35ffff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:09:58.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff35ffff640 1 -- 172.21.15.84:0/2267665351 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378049040 con 0x559693230f80 2024-05-22T23:09:58.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff35ffff640 1 -- 172.21.15.84:0/2267665351 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7ff3780b8610 con 0x559693230f80 2024-05-22T23:09:58.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff389c00740 1 -- 172.21.15.84:0/2267665351 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff368020b00 con 0x559693230f80 2024-05-22T23:09:58.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff35ffff640 1 -- 172.21.15.84:0/2267665351 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7ff3780c4c10 con 0x559693230f80 2024-05-22T23:09:58.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff389c00740 10 client.16740 fetch_fsmap learned FSMap version 18 2024-05-22T23:09:58.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.533+0000 7ff389c00740 1 -- 172.21.15.84:0/2267665351 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5596932b26f0 con 0x559693230f80 2024-05-22T23:09:58.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.534+0000 7ff35ffff640 1 -- 172.21.15.84:0/2267665351 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff368020b00 con 0x559693230f80 2024-05-22T23:09:58.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.534+0000 7ff35ffff640 1 -- 172.21.15.84:0/2267665351 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7ff3780bc200 con 0x559693230f80 2024-05-22T23:09:58.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.534+0000 7ff389c00740 10 client.16740 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:09:58.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.534+0000 7ff389c00740 10 client.16740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:09:58.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.534+0000 7ff389c00740 20 client.16740 populate_metadata read hostname 'smithi084' 2024-05-22T23:09:58.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.534+0000 7ff389c00740 1 --2- 172.21.15.84:0/2267665351 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5596932b2e20 0x5596932b5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:58.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.534+0000 7ff389c00740 4 client.16740 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-22T23:09:58.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.534+0000 7ff389c00740 1 -- 172.21.15.84:0/2267665351 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55969319d830 con 0x5596932b2e20 2024-05-22T23:09:58.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.534+0000 7ff37ffff640 1 --2- 172.21.15.84:0/2267665351 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5596932b2e20 0x5596932b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:58.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.535+0000 7ff35ffff640 10 client.16740 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:58.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.535+0000 7ff37ffff640 1 --2- 172.21.15.84:0/2267665351 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5596932b2e20 0x5596932b5240 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c13e064f192b968b server_cookie=d92ad3713097a28f in_seq=0 out_seq=0 2024-05-22T23:09:58.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.535+0000 7ff35ffff640 1 -- 172.21.15.84:0/2267665351 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55969319d830 con 0x5596932b2e20 2024-05-22T23:09:58.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.535+0000 7ff35ffff640 10 client.16740 handle_command_reply: tid=0 2024-05-22T23:09:58.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.535+0000 7ff389c00740 1 client.16740 shutdown 2024-05-22T23:09:58.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.535+0000 7ff389c00740 1 -- 172.21.15.84:0/2267665351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559693230f80 msgr2=0x559693235bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:58.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.535+0000 7ff389c00740 1 --2- 172.21.15.84:0/2267665351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559693230f80 0x559693235bc0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff378031550 tx=0x7ff3780314b0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.535+0000 7ff389c00740 1 -- 172.21.15.84:0/2267665351 shutdown_connections 2024-05-22T23:09:58.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.535+0000 7ff389c00740 1 --2- 172.21.15.84:0/2267665351 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5596932b2e20 0x5596932b5240 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.535+0000 7ff389c00740 1 --2- 172.21.15.84:0/2267665351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55969323bf70 0x559693236640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.535+0000 7ff35ffff640 0 client.16740 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:58.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.535+0000 7ff389c00740 1 --2- 172.21.15.84:0/2267665351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596932318a0 0x559693236100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.535+0000 7ff389c00740 1 --2- 172.21.15.84:0/2267665351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559693230f80 0x559693235bc0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.535+0000 7ff389c00740 1 -- 172.21.15.84:0/2267665351 >> 172.21.15.84:0/2267665351 conn(0x55969322d2a0 msgr2=0x5596932396e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:58.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.535+0000 7ff389c00740 1 -- 172.21.15.84:0/2267665351 shutdown_connections 2024-05-22T23:09:58.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.535+0000 7ff389c00740 1 -- 172.21.15.84:0/2267665351 wait complete. 2024-05-22T23:09:58.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.535+0000 7ff389c00740 20 client.16740 trim_cache size 0 max 16384 2024-05-22T23:09:58.553 INFO:teuthology.orchestra.run.smithi084.stdout:[] 2024-05-22T23:09:58.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.536+0000 7ff386ef5640 1 -- 172.21.15.84:0/4231578188 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff36c0a2d00 msgr2=0x7ff36c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:58.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.536+0000 7ff386ef5640 1 --2- 172.21.15.84:0/4231578188 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff36c0a2d00 0x7ff36c0a51c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff36800df00 tx=0x7ff36800d5b0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.536+0000 7ff386ef5640 1 -- 172.21.15.84:0/4231578188 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff380100740 msgr2=0x7ff380198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:58.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.536+0000 7ff386ef5640 1 --2- 172.21.15.84:0/4231578188 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff380100740 0x7ff380198120 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff378009a50 tx=0x7ff378004270 comp rx=0 tx=0).stop 2024-05-22T23:09:58.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.536+0000 7ff386ef5640 1 -- 172.21.15.84:0/4231578188 shutdown_connections 2024-05-22T23:09:58.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.536+0000 7ff386ef5640 1 --2- 172.21.15.84:0/4231578188 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff36c0a2d00 0x7ff36c0a51c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.536+0000 7ff386ef5640 1 --2- 172.21.15.84:0/4231578188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff380106750 0x7ff38019c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.536+0000 7ff386ef5640 1 --2- 172.21.15.84:0/4231578188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff380101100 0x7ff380198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.536+0000 7ff386ef5640 1 --2- 172.21.15.84:0/4231578188 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff380100740 0x7ff380198120 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:58.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.536+0000 7ff386ef5640 1 -- 172.21.15.84:0/4231578188 >> 172.21.15.84:0/4231578188 conn(0x7ff380077870 msgr2=0x7ff380102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:58.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.536+0000 7ff386ef5640 1 -- 172.21.15.84:0/4231578188 shutdown_connections 2024-05-22T23:09:58.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:58.536+0000 7ff386ef5640 1 -- 172.21.15.84:0/4231578188 wait complete. 2024-05-22T23:09:59.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:59 smithi204 ceph-mon[36211]: pgmap v251: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-05-22T23:09:59.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:09:59 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2422061478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:09:59.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:59 smithi096 ceph-mon[36392]: pgmap v251: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-05-22T23:09:59.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:09:59 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2422061478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:09:59.557 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-22T23:09:59.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:09:59.557 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-22T23:09:59.583 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:59 smithi084 ceph-mon[32186]: pgmap v251: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-05-22T23:09:59.584 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:09:59 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2422061478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:09:59.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.669+0000 7f4371cda640 1 -- 172.21.15.84:0/2184601914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f436c103420 msgr2=0x7f436c10b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:59.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.669+0000 7f4371cda640 1 --2- 172.21.15.84:0/2184601914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f436c103420 0x7f436c10b900 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4354009ca0 tx=0x7f435402a9e0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.669+0000 7f4371cda640 1 -- 172.21.15.84:0/2184601914 shutdown_connections 2024-05-22T23:09:59.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.669+0000 7f4371cda640 1 --2- 172.21.15.84:0/2184601914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f436c10be40 0x7f436c10e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.669+0000 7f4371cda640 1 --2- 172.21.15.84:0/2184601914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f436c103420 0x7f436c10b900 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.669+0000 7f4371cda640 1 --2- 172.21.15.84:0/2184601914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f436c102b00 0x7f436c102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.669+0000 7f4371cda640 1 -- 172.21.15.84:0/2184601914 >> 172.21.15.84:0/2184601914 conn(0x7f436c076af0 msgr2=0x7f436c076f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:59.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.669+0000 7f4371cda640 1 -- 172.21.15.84:0/2184601914 shutdown_connections 2024-05-22T23:09:59.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.669+0000 7f4371cda640 1 -- 172.21.15.84:0/2184601914 wait complete. 2024-05-22T23:09:59.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.669+0000 7f4371cda640 1 Processor -- start 2024-05-22T23:09:59.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.670+0000 7f4371cda640 1 -- start start 2024-05-22T23:09:59.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.670+0000 7f4371cda640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f436c102b00 0x7f436c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.670+0000 7f4371cda640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f436c103420 0x7f436c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.670+0000 7f4371cda640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f436c10be40 0x7f436c19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.670+0000 7f4371cda640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f436c111c80 con 0x7f436c103420 2024-05-22T23:09:59.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.670+0000 7f4371cda640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f436c111b00 con 0x7f436c102b00 2024-05-22T23:09:59.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.670+0000 7f4371cda640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f436c111e00 con 0x7f436c10be40 2024-05-22T23:09:59.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.670+0000 7f436affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f436c103420 0x7f436c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.670+0000 7f436affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f436c103420 0x7f436c1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37390/0 (socket says 172.21.15.84:37390) 2024-05-22T23:09:59.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.670+0000 7f436affd640 1 -- 172.21.15.84:0/3354851219 learned_addr learned my addr 172.21.15.84:0/3354851219 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:59.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.670+0000 7f436b7fe640 1 --2- 172.21.15.84:0/3354851219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f436c102b00 0x7f436c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.670+0000 7f436affd640 1 -- 172.21.15.84:0/3354851219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f436c10be40 msgr2=0x7f436c19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:59.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.670+0000 7f436affd640 1 --2- 172.21.15.84:0/3354851219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f436c10be40 0x7f436c19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.670+0000 7f436affd640 1 -- 172.21.15.84:0/3354851219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f436c102b00 msgr2=0x7f436c1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:59.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.670+0000 7f436affd640 1 --2- 172.21.15.84:0/3354851219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f436c102b00 0x7f436c1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.670+0000 7f436affd640 1 -- 172.21.15.84:0/3354851219 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436c19d190 con 0x7f436c103420 2024-05-22T23:09:59.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.670+0000 7f436b7fe640 1 --2- 172.21.15.84:0/3354851219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f436c102b00 0x7f436c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:09:59.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.670+0000 7f436affd640 1 --2- 172.21.15.84:0/3354851219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f436c103420 0x7f436c1986e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4354004af0 tx=0x7f4354003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:59.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.671+0000 7f4368ff9640 1 -- 172.21.15.84:0/3354851219 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354047070 con 0x7f436c103420 2024-05-22T23:09:59.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.671+0000 7f4368ff9640 1 -- 172.21.15.84:0/3354851219 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f435402e870 con 0x7f436c103420 2024-05-22T23:09:59.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.671+0000 7f4371cda640 1 -- 172.21.15.84:0/3354851219 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f436c19d420 con 0x7f436c103420 2024-05-22T23:09:59.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.671+0000 7f4368ff9640 1 -- 172.21.15.84:0/3354851219 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354033050 con 0x7f436c103420 2024-05-22T23:09:59.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.671+0000 7f4371cda640 1 -- 172.21.15.84:0/3354851219 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f436c1a4d10 con 0x7f436c103420 2024-05-22T23:09:59.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.673+0000 7f4368ff9640 1 -- 172.21.15.84:0/3354851219 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f435404a070 con 0x7f436c103420 2024-05-22T23:09:59.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.673+0000 7f43749ed740 1 Processor -- start 2024-05-22T23:09:59.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.673+0000 7f43749ed740 1 -- start start 2024-05-22T23:09:59.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.673+0000 7f43749ed740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556369b9fc50 0x556369ba0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.673+0000 7f43749ed740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556369ba0570 0x556369baa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.673+0000 7f43749ed740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369baac40 0x556369bad030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.674+0000 7f43749ed740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556369b03a40 con 0x556369baac40 2024-05-22T23:09:59.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.674+0000 7f43749ed740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556369aa2440 con 0x556369ba0570 2024-05-22T23:09:59.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.674+0000 7f43749ed740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556369affa00 con 0x556369b9fc50 2024-05-22T23:09:59.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.674+0000 7f4368ff9640 1 --2- 172.21.15.84:0/3354851219 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f43580a2d00 0x7f43580a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.674+0000 7f436affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369baac40 0x556369bad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.674+0000 7f436affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369baac40 0x556369bad030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37396/0 (socket says 172.21.15.84:37396) 2024-05-22T23:09:59.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.674+0000 7f436affd640 1 -- 172.21.15.84:0/442180066 learned_addr learned my addr 172.21.15.84:0/442180066 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:59.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.674+0000 7f436bfff640 1 --2- 172.21.15.84:0/442180066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556369b9fc50 0x556369ba0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.674+0000 7f436bfff640 1 --2- 172.21.15.84:0/3354851219 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f43580a2d00 0x7f43580a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.674+0000 7f4368ff9640 1 -- 172.21.15.84:0/3354851219 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7f43540f0230 con 0x7f436c103420 2024-05-22T23:09:59.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.674+0000 7f436b7fe640 1 --2- 172.21.15.84:0/442180066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556369ba0570 0x556369baa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.674+0000 7f436affd640 1 -- 172.21.15.84:0/442180066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556369b9fc50 msgr2=0x556369ba0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:59.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.674+0000 7f436affd640 1 --2- 172.21.15.84:0/442180066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556369b9fc50 0x556369ba0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.674+0000 7f436affd640 1 -- 172.21.15.84:0/442180066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556369ba0570 msgr2=0x556369baa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:59.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.674+0000 7f436affd640 1 --2- 172.21.15.84:0/442180066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556369ba0570 0x556369baa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.674+0000 7f436affd640 1 -- 172.21.15.84:0/442180066 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556369aac2d0 con 0x556369baac40 2024-05-22T23:09:59.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.674+0000 7f436b7fe640 1 --2- 172.21.15.84:0/442180066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556369ba0570 0x556369baa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:09:59.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.674+0000 7f436affd640 1 --2- 172.21.15.84:0/442180066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369baac40 0x556369bad030 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f43540ed8c0 tx=0x7f43540093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b71d2c28dc2ce4d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:59.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.674+0000 7f436bfff640 1 --2- 172.21.15.84:0/3354851219 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f43580a2d00 0x7f43580a51c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f43640058c0 tx=0x7f4364005850 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:59.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.674+0000 7f4361ffb640 1 -- 172.21.15.84:0/442180066 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43540bce20 con 0x556369baac40 2024-05-22T23:09:59.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.674+0000 7f4361ffb640 1 -- 172.21.15.84:0/442180066 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43540c0ad0 con 0x556369baac40 2024-05-22T23:09:59.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.674+0000 7f4361ffb640 1 -- 172.21.15.84:0/442180066 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43540b8ef0 con 0x556369baac40 2024-05-22T23:09:59.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.675+0000 7f43749ed740 1 -- 172.21.15.84:0/442180066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369baac40 msgr2=0x556369bad030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:59.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.675+0000 7f43749ed740 1 --2- 172.21.15.84:0/442180066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369baac40 0x556369bad030 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f43540ed8c0 tx=0x7f43540093d0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.675+0000 7f43749ed740 1 -- 172.21.15.84:0/442180066 shutdown_connections 2024-05-22T23:09:59.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.675+0000 7f43749ed740 1 --2- 172.21.15.84:0/442180066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369baac40 0x556369bad030 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.675+0000 7f43749ed740 1 --2- 172.21.15.84:0/442180066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556369ba0570 0x556369baa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.675+0000 7f43749ed740 1 --2- 172.21.15.84:0/442180066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556369b9fc50 0x556369ba0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.675+0000 7f43749ed740 1 -- 172.21.15.84:0/442180066 >> 172.21.15.84:0/442180066 conn(0x556369b9bf70 msgr2=0x556369b9c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:59.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.675+0000 7f43749ed740 1 -- 172.21.15.84:0/442180066 shutdown_connections 2024-05-22T23:09:59.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.675+0000 7f43749ed740 1 -- 172.21.15.84:0/442180066 wait complete. 2024-05-22T23:09:59.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.675+0000 7f43749ed740 1 -- start start 2024-05-22T23:09:59.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.676+0000 7f43749ed740 1 Processor -- start 2024-05-22T23:09:59.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.676+0000 7f43749ed740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556369b9fc50 0x556369ba7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.676+0000 7f43749ed740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556369ba0570 0x556369ba8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.676+0000 7f43749ed740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369baac40 0x556369ba8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.676+0000 7f43749ed740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556369bad7f0 con 0x556369baac40 2024-05-22T23:09:59.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.676+0000 7f43749ed740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556369a8d5f0 con 0x556369b9fc50 2024-05-22T23:09:59.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.676+0000 7f43749ed740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556369bad970 con 0x556369ba0570 2024-05-22T23:09:59.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.676+0000 7f43749ed740 10 client.0 _ll_register_callbacks cb 0x556369a88cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:09:59.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.676+0000 7f436bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369baac40 0x556369ba8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.676+0000 7f436bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369baac40 0x556369ba8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37402/0 (socket says 172.21.15.84:37402) 2024-05-22T23:09:59.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.676+0000 7f436bfff640 1 -- 172.21.15.84:0/1154733037 learned_addr learned my addr 172.21.15.84:0/1154733037 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:59.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.676+0000 7f436affd640 1 --2- 172.21.15.84:0/1154733037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556369ba0570 0x556369ba8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.676+0000 7f436b7fe640 1 --2- 172.21.15.84:0/1154733037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556369b9fc50 0x556369ba7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.676+0000 7f436bfff640 1 -- 172.21.15.84:0/1154733037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556369ba0570 msgr2=0x556369ba8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:59.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.676+0000 7f436bfff640 1 --2- 172.21.15.84:0/1154733037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556369ba0570 0x556369ba8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.676+0000 7f436bfff640 1 -- 172.21.15.84:0/1154733037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556369b9fc50 msgr2=0x556369ba7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:59.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.676+0000 7f436bfff640 1 --2- 172.21.15.84:0/1154733037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556369b9fc50 0x556369ba7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.676+0000 7f436bfff640 1 -- 172.21.15.84:0/1154733037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563698df7e0 con 0x556369baac40 2024-05-22T23:09:59.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.676+0000 7f436affd640 1 --2- 172.21.15.84:0/1154733037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556369ba0570 0x556369ba8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:09:59.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.677+0000 7f436bfff640 1 --2- 172.21.15.84:0/1154733037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369baac40 0x556369ba8680 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4364015040 tx=0x7f43640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5ecfa22dac0074 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:59.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.677+0000 7f434affd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:09:59.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.677+0000 7f434affd640 1 -- 172.21.15.84:0/1154733037 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364024070 con 0x556369baac40 2024-05-22T23:09:59.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.677+0000 7f434affd640 1 -- 172.21.15.84:0/1154733037 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7f4364005020 con 0x556369baac40 2024-05-22T23:09:59.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.677+0000 7f43749ed740 1 -- 172.21.15.84:0/1154733037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43540c0ad0 con 0x556369baac40 2024-05-22T23:09:59.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.677+0000 7f434affd640 1 -- 172.21.15.84:0/1154733037 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f436402f050 con 0x556369baac40 2024-05-22T23:09:59.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.677+0000 7f43749ed740 10 client.16752 fetch_fsmap learned FSMap version 18 2024-05-22T23:09:59.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.677+0000 7f43749ed740 1 -- 172.21.15.84:0/1154733037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55636993ff70 con 0x556369baac40 2024-05-22T23:09:59.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.677+0000 7f434affd640 1 -- 172.21.15.84:0/1154733037 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43540c0ad0 con 0x556369baac40 2024-05-22T23:09:59.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.677+0000 7f434affd640 1 -- 172.21.15.84:0/1154733037 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f436401b050 con 0x556369baac40 2024-05-22T23:09:59.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.677+0000 7f43749ed740 10 client.16752 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:09:59.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.677+0000 7f43749ed740 10 client.16752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:09:59.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.677+0000 7f43749ed740 20 client.16752 populate_metadata read hostname 'smithi084' 2024-05-22T23:09:59.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.677+0000 7f43749ed740 1 --2- 172.21.15.84:0/1154733037 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556369bbabf0 0x556369bbd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.677+0000 7f43749ed740 4 client.16752 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:09:59.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.677+0000 7f43749ed740 1 -- 172.21.15.84:0/1154733037 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556369b17460 con 0x556369bbabf0 2024-05-22T23:09:59.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.677+0000 7f436b7fe640 1 --2- 172.21.15.84:0/1154733037 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556369bbabf0 0x556369bbd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.678+0000 7f436b7fe640 1 --2- 172.21.15.84:0/1154733037 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556369bbabf0 0x556369bbd020 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bef1a94b5b7f6360 server_cookie=d92e1c81dfee32d7 in_seq=0 out_seq=0 2024-05-22T23:09:59.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.678+0000 7f434affd640 10 client.16752 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:59.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.679+0000 7f434affd640 1 -- 172.21.15.84:0/1154733037 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556369b17460 con 0x556369bbabf0 2024-05-22T23:09:59.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.679+0000 7f434affd640 10 client.16752 handle_command_reply: tid=0 2024-05-22T23:09:59.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.679+0000 7f43749ed740 1 client.16752 shutdown 2024-05-22T23:09:59.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.679+0000 7f43749ed740 1 -- 172.21.15.84:0/1154733037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369baac40 msgr2=0x556369ba8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:59.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.679+0000 7f43749ed740 1 --2- 172.21.15.84:0/1154733037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369baac40 0x556369ba8680 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4364015040 tx=0x7f43640046c0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.679+0000 7f43749ed740 1 -- 172.21.15.84:0/1154733037 shutdown_connections 2024-05-22T23:09:59.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.679+0000 7f43749ed740 1 --2- 172.21.15.84:0/1154733037 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556369bbabf0 0x556369bbd020 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.679+0000 7f43749ed740 1 --2- 172.21.15.84:0/1154733037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369baac40 0x556369ba8680 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.679+0000 7f43749ed740 1 --2- 172.21.15.84:0/1154733037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556369ba0570 0x556369ba8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.679+0000 7f434affd640 0 client.16752 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:59.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.679+0000 7f43749ed740 1 --2- 172.21.15.84:0/1154733037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556369b9fc50 0x556369ba7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.679+0000 7f43749ed740 1 -- 172.21.15.84:0/1154733037 >> 172.21.15.84:0/1154733037 conn(0x556369b9bf70 msgr2=0x556369ba1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:59.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.679+0000 7f43749ed740 1 -- 172.21.15.84:0/1154733037 shutdown_connections 2024-05-22T23:09:59.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.679+0000 7f43749ed740 1 -- 172.21.15.84:0/1154733037 wait complete. 2024-05-22T23:09:59.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.679+0000 7f43749ed740 20 client.16752 trim_cache size 0 max 16384 2024-05-22T23:09:59.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.697+0000 7f43749ed740 1 Processor -- start 2024-05-22T23:09:59.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.697+0000 7f43749ed740 1 -- start start 2024-05-22T23:09:59.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.697+0000 7f43749ed740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556369b9fc50 0x556369ba24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.697+0000 7f43749ed740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369ba0570 0x556369bc28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.697+0000 7f43749ed740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556369baac40 0x556369bc2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.697+0000 7f43749ed740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f436c1ab6d0 con 0x556369ba0570 2024-05-22T23:09:59.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.697+0000 7f43749ed740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f436c1adf30 con 0x556369baac40 2024-05-22T23:09:59.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.697+0000 7f43749ed740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f436c1abd20 con 0x556369b9fc50 2024-05-22T23:09:59.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.697+0000 7f436affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369ba0570 0x556369bc28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.697+0000 7f436affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369ba0570 0x556369bc28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37406/0 (socket says 172.21.15.84:37406) 2024-05-22T23:09:59.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.697+0000 7f436affd640 1 -- 172.21.15.84:0/1596400289 learned_addr learned my addr 172.21.15.84:0/1596400289 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:59.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.697+0000 7f436affd640 1 -- 172.21.15.84:0/1596400289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556369b9fc50 msgr2=0x556369ba24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:59.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.697+0000 7f436affd640 1 --2- 172.21.15.84:0/1596400289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556369b9fc50 0x556369ba24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.697+0000 7f436affd640 1 -- 172.21.15.84:0/1596400289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556369baac40 msgr2=0x556369bc2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:59.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.697+0000 7f436affd640 1 --2- 172.21.15.84:0/1596400289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556369baac40 0x556369bc2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.697+0000 7f436affd640 1 -- 172.21.15.84:0/1596400289 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563697d7170 con 0x556369ba0570 2024-05-22T23:09:59.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.697+0000 7f436affd640 1 --2- 172.21.15.84:0/1596400289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369ba0570 0x556369bc28f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4354031040 tx=0x7f43540095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492014950dcf83ad server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:59.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.697+0000 7f43617fa640 1 -- 172.21.15.84:0/1596400289 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43540d4030 con 0x556369ba0570 2024-05-22T23:09:59.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.697+0000 7f43617fa640 1 -- 172.21.15.84:0/1596400289 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43540bce20 con 0x556369ba0570 2024-05-22T23:09:59.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.697+0000 7f43617fa640 1 -- 172.21.15.84:0/1596400289 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43540edbd0 con 0x556369ba0570 2024-05-22T23:09:59.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.698+0000 7f43749ed740 1 -- 172.21.15.84:0/1596400289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369ba0570 msgr2=0x556369bc28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:59.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.698+0000 7f43749ed740 1 --2- 172.21.15.84:0/1596400289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369ba0570 0x556369bc28f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4354031040 tx=0x7f43540095c0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.698+0000 7f43749ed740 1 -- 172.21.15.84:0/1596400289 shutdown_connections 2024-05-22T23:09:59.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.698+0000 7f43749ed740 1 --2- 172.21.15.84:0/1596400289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556369baac40 0x556369bc2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.698+0000 7f43749ed740 1 --2- 172.21.15.84:0/1596400289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369ba0570 0x556369bc28f0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.698+0000 7f43749ed740 1 --2- 172.21.15.84:0/1596400289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556369b9fc50 0x556369ba24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.698+0000 7f43749ed740 1 -- 172.21.15.84:0/1596400289 >> 172.21.15.84:0/1596400289 conn(0x556369b9bf70 msgr2=0x556369b9b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:59.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.698+0000 7f43749ed740 1 -- 172.21.15.84:0/1596400289 shutdown_connections 2024-05-22T23:09:59.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.698+0000 7f43749ed740 1 -- 172.21.15.84:0/1596400289 wait complete. 2024-05-22T23:09:59.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.698+0000 7f43749ed740 1 -- start start 2024-05-22T23:09:59.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.699+0000 7f43749ed740 1 Processor -- start 2024-05-22T23:09:59.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.699+0000 7f43749ed740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556369b9fc50 0x556369bb2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.699+0000 7f43749ed740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556369ba0570 0x556369bafb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.699+0000 7f43749ed740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369baac40 0x556369bb0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.699+0000 7f43749ed740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f436c1ac3f0 con 0x556369baac40 2024-05-22T23:09:59.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.699+0000 7f43749ed740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f436c1ad860 con 0x556369b9fc50 2024-05-22T23:09:59.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.699+0000 7f43749ed740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f436c1acac0 con 0x556369ba0570 2024-05-22T23:09:59.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.699+0000 7f43749ed740 10 client.0 _ll_register_callbacks cb 0x7f4364012540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:09:59.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.699+0000 7f436bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369baac40 0x556369bb0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.699+0000 7f436bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369baac40 0x556369bb0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37418/0 (socket says 172.21.15.84:37418) 2024-05-22T23:09:59.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.699+0000 7f436bfff640 1 -- 172.21.15.84:0/3077026032 learned_addr learned my addr 172.21.15.84:0/3077026032 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:59.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.699+0000 7f436bfff640 1 -- 172.21.15.84:0/3077026032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556369ba0570 msgr2=0x556369bafb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:59.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.699+0000 7f436bfff640 1 --2- 172.21.15.84:0/3077026032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556369ba0570 0x556369bafb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.699+0000 7f436bfff640 1 -- 172.21.15.84:0/3077026032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556369b9fc50 msgr2=0x556369bb2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:59.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.699+0000 7f436bfff640 1 --2- 172.21.15.84:0/3077026032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556369b9fc50 0x556369bb2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.699+0000 7f436bfff640 1 -- 172.21.15.84:0/3077026032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556369acc930 con 0x556369baac40 2024-05-22T23:09:59.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.699+0000 7f436bfff640 1 --2- 172.21.15.84:0/3077026032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369baac40 0x556369bb0040 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f4364004a20 tx=0x7f4364004cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e17ec2a30d885c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:59.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.699+0000 7f434affd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:09:59.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.699+0000 7f434affd640 1 -- 172.21.15.84:0/3077026032 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436402c070 con 0x556369baac40 2024-05-22T23:09:59.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.699+0000 7f434affd640 1 -- 172.21.15.84:0/3077026032 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7f436400ca30 con 0x556369baac40 2024-05-22T23:09:59.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.699+0000 7f43749ed740 1 -- 172.21.15.84:0/3077026032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43540bce20 con 0x556369baac40 2024-05-22T23:09:59.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.700+0000 7f434affd640 1 -- 172.21.15.84:0/3077026032 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f4364024040 con 0x556369baac40 2024-05-22T23:09:59.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.700+0000 7f43749ed740 10 client.16758 fetch_fsmap learned FSMap version 18 2024-05-22T23:09:59.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.700+0000 7f43749ed740 1 -- 172.21.15.84:0/3077026032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x556369c21cd0 con 0x556369baac40 2024-05-22T23:09:59.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.700+0000 7f434affd640 1 -- 172.21.15.84:0/3077026032 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43540bce20 con 0x556369baac40 2024-05-22T23:09:59.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.700+0000 7f434affd640 1 -- 172.21.15.84:0/3077026032 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f436401b040 con 0x556369baac40 2024-05-22T23:09:59.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.700+0000 7f43749ed740 10 client.16758 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:09:59.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.700+0000 7f43749ed740 10 client.16758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:09:59.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.700+0000 7f43749ed740 20 client.16758 populate_metadata read hostname 'smithi084' 2024-05-22T23:09:59.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.700+0000 7f43749ed740 1 --2- 172.21.15.84:0/3077026032 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556369c223c0 0x556369c24800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.700+0000 7f43749ed740 4 client.16758 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-22T23:09:59.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.700+0000 7f43749ed740 1 -- 172.21.15.84:0/3077026032 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556369b11170 con 0x556369c223c0 2024-05-22T23:09:59.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.700+0000 7f436b7fe640 1 --2- 172.21.15.84:0/3077026032 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556369c223c0 0x556369c24800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.701+0000 7f436b7fe640 1 --2- 172.21.15.84:0/3077026032 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556369c223c0 0x556369c24800 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0b5acb3ebe6786c server_cookie=3bdc1d06f07bb9b9 in_seq=0 out_seq=0 2024-05-22T23:09:59.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.701+0000 7f434affd640 10 client.16758 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:59.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.702+0000 7f434affd640 1 -- 172.21.15.84:0/3077026032 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556369b11170 con 0x556369c223c0 2024-05-22T23:09:59.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.702+0000 7f434affd640 10 client.16758 handle_command_reply: tid=0 2024-05-22T23:09:59.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.702+0000 7f43749ed740 1 client.16758 shutdown 2024-05-22T23:09:59.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.702+0000 7f43749ed740 1 -- 172.21.15.84:0/3077026032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369baac40 msgr2=0x556369bb0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:59.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.702+0000 7f43749ed740 1 --2- 172.21.15.84:0/3077026032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369baac40 0x556369bb0040 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f4364004a20 tx=0x7f4364004cd0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.702+0000 7f43749ed740 1 -- 172.21.15.84:0/3077026032 shutdown_connections 2024-05-22T23:09:59.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.702+0000 7f43749ed740 1 --2- 172.21.15.84:0/3077026032 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x556369c223c0 0x556369c24800 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.702+0000 7f43749ed740 1 --2- 172.21.15.84:0/3077026032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556369baac40 0x556369bb0040 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.702+0000 7f43749ed740 1 --2- 172.21.15.84:0/3077026032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556369ba0570 0x556369bafb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.702+0000 7f434affd640 0 client.16758 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:59.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.702+0000 7f43749ed740 1 --2- 172.21.15.84:0/3077026032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556369b9fc50 0x556369bb2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.702+0000 7f43749ed740 1 -- 172.21.15.84:0/3077026032 >> 172.21.15.84:0/3077026032 conn(0x556369b9bf70 msgr2=0x556369ba9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:59.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.702+0000 7f43749ed740 1 -- 172.21.15.84:0/3077026032 shutdown_connections 2024-05-22T23:09:59.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.702+0000 7f43749ed740 1 -- 172.21.15.84:0/3077026032 wait complete. 2024-05-22T23:09:59.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.702+0000 7f43749ed740 20 client.16758 trim_cache size 0 max 16384 2024-05-22T23:09:59.717 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:09:59.717 INFO:teuthology.orchestra.run.smithi084.stdout: "return_code": 0, 2024-05-22T23:09:59.717 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_tag": "009fd2d4-9f08-4c32-bc39-6dec6b85d10b", 2024-05-22T23:09:59.718 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": "asynchronous" 2024-05-22T23:09:59.718 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:09:59.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.702+0000 7f4371cda640 1 -- 172.21.15.84:0/3354851219 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f43580a2d00 msgr2=0x7f43580a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:59.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.702+0000 7f4371cda640 1 --2- 172.21.15.84:0/3354851219 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f43580a2d00 0x7f43580a51c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f43640058c0 tx=0x7f4364005850 comp rx=0 tx=0).stop 2024-05-22T23:09:59.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.702+0000 7f4371cda640 1 -- 172.21.15.84:0/3354851219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f436c103420 msgr2=0x7f436c1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:59.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.702+0000 7f4371cda640 1 --2- 172.21.15.84:0/3354851219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f436c103420 0x7f436c1986e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4354004af0 tx=0x7f4354003ce0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.702+0000 7f4371cda640 1 -- 172.21.15.84:0/3354851219 shutdown_connections 2024-05-22T23:09:59.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.703+0000 7f4371cda640 1 --2- 172.21.15.84:0/3354851219 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f43580a2d00 0x7f43580a51c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.703+0000 7f4371cda640 1 --2- 172.21.15.84:0/3354851219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f436c10be40 0x7f436c19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.703+0000 7f4371cda640 1 --2- 172.21.15.84:0/3354851219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f436c103420 0x7f436c1986e0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.703+0000 7f4371cda640 1 --2- 172.21.15.84:0/3354851219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f436c102b00 0x7f436c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.703+0000 7f4371cda640 1 -- 172.21.15.84:0/3354851219 >> 172.21.15.84:0/3354851219 conn(0x7f436c076af0 msgr2=0x7f436c0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:59.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.703+0000 7f4371cda640 1 -- 172.21.15.84:0/3354851219 shutdown_connections 2024-05-22T23:09:59.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.703+0000 7f4371cda640 1 -- 172.21.15.84:0/3354851219 wait complete. 2024-05-22T23:09:59.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:09:59.721 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:09:59.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.859+0000 7fc6782a5640 1 -- 172.21.15.84:0/3780580800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc670101100 msgr2=0x7fc67010d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:59.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.859+0000 7fc6782a5640 1 --2- 172.21.15.84:0/3780580800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc670101100 0x7fc67010d610 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fc668009cd0 tx=0x7fc66802a8e0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.859+0000 7fc6782a5640 1 -- 172.21.15.84:0/3780580800 shutdown_connections 2024-05-22T23:09:59.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.859+0000 7fc6782a5640 1 --2- 172.21.15.84:0/3780580800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc670101100 0x7fc67010d610 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.859+0000 7fc6782a5640 1 --2- 172.21.15.84:0/3780580800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc670100740 0x7fc670100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.859+0000 7fc6782a5640 1 --2- 172.21.15.84:0/3780580800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc670106750 0x7fc670106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.859+0000 7fc6782a5640 1 -- 172.21.15.84:0/3780580800 >> 172.21.15.84:0/3780580800 conn(0x7fc670077870 msgr2=0x7fc670077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:59.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.859+0000 7fc6782a5640 1 -- 172.21.15.84:0/3780580800 shutdown_connections 2024-05-22T23:09:59.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.859+0000 7fc6782a5640 1 -- 172.21.15.84:0/3780580800 wait complete. 2024-05-22T23:09:59.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.859+0000 7fc6782a5640 1 Processor -- start 2024-05-22T23:09:59.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.859+0000 7fc6782a5640 1 -- start start 2024-05-22T23:09:59.874 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.859+0000 7fc6782a5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc670100740 0x7fc670198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.875 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.859+0000 7fc6782a5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc670101100 0x7fc670198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.875 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.859+0000 7fc6782a5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc670106750 0x7fc67019cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.875 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.859+0000 7fc6782a5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc670110be0 con 0x7fc670106750 2024-05-22T23:09:59.875 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.859+0000 7fc6782a5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc670110a60 con 0x7fc670101100 2024-05-22T23:09:59.875 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.859+0000 7fc6782a5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc670110d60 con 0x7fc670100740 2024-05-22T23:09:59.875 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.860+0000 7fc676842640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc670106750 0x7fc67019cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.875 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.860+0000 7fc676842640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc670106750 0x7fc67019cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37438/0 (socket says 172.21.15.84:37438) 2024-05-22T23:09:59.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.860+0000 7fc676842640 1 -- 172.21.15.84:0/929613935 learned_addr learned my addr 172.21.15.84:0/929613935 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:59.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.860+0000 7fc676041640 1 --2- 172.21.15.84:0/929613935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc670100740 0x7fc670198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.860+0000 7fc675840640 1 --2- 172.21.15.84:0/929613935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc670101100 0x7fc670198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.860+0000 7fc676842640 1 -- 172.21.15.84:0/929613935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc670100740 msgr2=0x7fc670198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:59.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.860+0000 7fc676842640 1 --2- 172.21.15.84:0/929613935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc670100740 0x7fc670198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.860+0000 7fc676842640 1 -- 172.21.15.84:0/929613935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc670101100 msgr2=0x7fc670198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:59.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.860+0000 7fc676842640 1 --2- 172.21.15.84:0/929613935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc670101100 0x7fc670198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.860+0000 7fc676842640 1 -- 172.21.15.84:0/929613935 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc67019d240 con 0x7fc670106750 2024-05-22T23:09:59.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.860+0000 7fc676041640 1 --2- 172.21.15.84:0/929613935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc670100740 0x7fc670198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:09:59.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.860+0000 7fc676842640 1 --2- 172.21.15.84:0/929613935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc670106750 0x7fc67019cb10 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc668006010 tx=0x7fc668003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:59.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.860+0000 7fc6677fe640 1 -- 172.21.15.84:0/929613935 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc668004450 con 0x7fc670106750 2024-05-22T23:09:59.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.860+0000 7fc6677fe640 1 -- 172.21.15.84:0/929613935 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc668003e10 con 0x7fc670106750 2024-05-22T23:09:59.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.860+0000 7fc6782a5640 1 -- 172.21.15.84:0/929613935 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc67019d4d0 con 0x7fc670106750 2024-05-22T23:09:59.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.860+0000 7fc6677fe640 1 -- 172.21.15.84:0/929613935 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6680466e0 con 0x7fc670106750 2024-05-22T23:09:59.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.860+0000 7fc6782a5640 1 -- 172.21.15.84:0/929613935 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6701a4dc0 con 0x7fc670106750 2024-05-22T23:09:59.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.863+0000 7fc6677fe640 1 -- 172.21.15.84:0/929613935 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc6680305a0 con 0x7fc670106750 2024-05-22T23:09:59.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.863+0000 7fc67afb9740 1 Processor -- start 2024-05-22T23:09:59.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.863+0000 7fc67afb9740 1 -- start start 2024-05-22T23:09:59.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.863+0000 7fc67afb9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561adae544a0 0x561adae527e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.863+0000 7fc67afb9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561adae52db0 0x561adae53230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.863+0000 7fc67afb9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae5db30 0x561adae5ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.863+0000 7fc67afb9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561adadb6db0 con 0x561adae5db30 2024-05-22T23:09:59.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.863+0000 7fc67afb9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561adad55360 con 0x561adae52db0 2024-05-22T23:09:59.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.863+0000 7fc67afb9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561adadb2d70 con 0x561adae544a0 2024-05-22T23:09:59.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.863+0000 7fc675840640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae5db30 0x561adae5ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.863+0000 7fc675840640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae5db30 0x561adae5ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37454/0 (socket says 172.21.15.84:37454) 2024-05-22T23:09:59.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.864+0000 7fc675840640 1 -- 172.21.15.84:0/1352044715 learned_addr learned my addr 172.21.15.84:0/1352044715 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:59.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.864+0000 7fc675840640 1 --2- 172.21.15.84:0/1352044715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561adae544a0 0x561adae527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.864+0000 7fc676041640 1 --2- 172.21.15.84:0/1352044715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561adae52db0 0x561adae53230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.864+0000 7fc6677fe640 1 --2- 172.21.15.84:0/929613935 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fc65c0a2d00 0x7fc65c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.864+0000 7fc675840640 1 -- 172.21.15.84:0/1352044715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561adae544a0 msgr2=0x561adae527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:59.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.864+0000 7fc675840640 1 --2- 172.21.15.84:0/1352044715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561adae544a0 0x561adae527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.864+0000 7fc675840640 1 -- 172.21.15.84:0/1352044715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561adae52db0 msgr2=0x561adae53230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:59.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.864+0000 7fc675840640 1 --2- 172.21.15.84:0/1352044715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561adae52db0 0x561adae53230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.864+0000 7fc675840640 1 -- 172.21.15.84:0/1352044715 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561adad78a60 con 0x561adae5db30 2024-05-22T23:09:59.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.864+0000 7fc676842640 1 --2- 172.21.15.84:0/929613935 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fc65c0a2d00 0x7fc65c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.864+0000 7fc6677fe640 1 -- 172.21.15.84:0/929613935 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7fc66802e030 con 0x7fc670106750 2024-05-22T23:09:59.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.864+0000 7fc675840640 1 --2- 172.21.15.84:0/1352044715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae5db30 0x561adae5ff20 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00cf60 tx=0x7fc66c00c170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decaf4b0c1e4c32f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:59.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.864+0000 7fc664bfe640 1 -- 172.21.15.84:0/1352044715 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c01d070 con 0x561adae5db30 2024-05-22T23:09:59.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.864+0000 7fc664bfe640 1 -- 172.21.15.84:0/1352044715 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc66c018b50 con 0x561adae5db30 2024-05-22T23:09:59.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.864+0000 7fc664bfe640 1 -- 172.21.15.84:0/1352044715 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c00f040 con 0x561adae5db30 2024-05-22T23:09:59.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.864+0000 7fc676842640 1 --2- 172.21.15.84:0/929613935 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fc65c0a2d00 0x7fc65c0a51c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc6680eeae0 tx=0x7fc66802a8e0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:59.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.865+0000 7fc67afb9740 1 -- 172.21.15.84:0/1352044715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae5db30 msgr2=0x561adae5ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:59.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.865+0000 7fc67afb9740 1 --2- 172.21.15.84:0/1352044715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae5db30 0x561adae5ff20 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00cf60 tx=0x7fc66c00c170 comp rx=0 tx=0).stop 2024-05-22T23:09:59.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.865+0000 7fc67afb9740 1 -- 172.21.15.84:0/1352044715 shutdown_connections 2024-05-22T23:09:59.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.865+0000 7fc67afb9740 1 --2- 172.21.15.84:0/1352044715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae5db30 0x561adae5ff20 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.865+0000 7fc67afb9740 1 --2- 172.21.15.84:0/1352044715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561adae52db0 0x561adae53230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.865+0000 7fc67afb9740 1 --2- 172.21.15.84:0/1352044715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561adae544a0 0x561adae527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.865+0000 7fc67afb9740 1 -- 172.21.15.84:0/1352044715 >> 172.21.15.84:0/1352044715 conn(0x561adae1b8e0 msgr2=0x561adae4f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:59.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.865+0000 7fc67afb9740 1 -- 172.21.15.84:0/1352044715 shutdown_connections 2024-05-22T23:09:59.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.865+0000 7fc67afb9740 1 -- 172.21.15.84:0/1352044715 wait complete. 2024-05-22T23:09:59.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.865+0000 7fc67afb9740 1 -- start start 2024-05-22T23:09:59.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.865+0000 7fc67afb9740 1 Processor -- start 2024-05-22T23:09:59.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.866+0000 7fc67afb9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561adae52db0 0x561adae5aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.866+0000 7fc67afb9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae544a0 0x561adae5afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.866+0000 7fc67afb9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561adae5db30 0x561adae5b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.866+0000 7fc67afb9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561adae609c0 con 0x561adae544a0 2024-05-22T23:09:59.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.866+0000 7fc67afb9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561adad710f0 con 0x561adae5db30 2024-05-22T23:09:59.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.866+0000 7fc67afb9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561adae60b40 con 0x561adae52db0 2024-05-22T23:09:59.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.866+0000 7fc67afb9740 10 client.0 _ll_register_callbacks cb 0x561adad75f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:09:59.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.866+0000 7fc676041640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae544a0 0x561adae5afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.866+0000 7fc676041640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae544a0 0x561adae5afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37462/0 (socket says 172.21.15.84:37462) 2024-05-22T23:09:59.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.866+0000 7fc676041640 1 -- 172.21.15.84:0/403744827 learned_addr learned my addr 172.21.15.84:0/403744827 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:59.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.866+0000 7fc675840640 1 --2- 172.21.15.84:0/403744827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561adae52db0 0x561adae5aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.866+0000 7fc676041640 1 -- 172.21.15.84:0/403744827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561adae52db0 msgr2=0x561adae5aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:59.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.866+0000 7fc676041640 1 --2- 172.21.15.84:0/403744827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561adae52db0 0x561adae5aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.866+0000 7fc675840640 1 --2- 172.21.15.84:0/403744827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561adae5db30 0x561adae5b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.866+0000 7fc676041640 1 -- 172.21.15.84:0/403744827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561adae5db30 msgr2=0x561adae5b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:59.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.866+0000 7fc676041640 1 --2- 172.21.15.84:0/403744827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561adae5db30 0x561adae5b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.866+0000 7fc676041640 1 -- 172.21.15.84:0/403744827 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561adab947d0 con 0x561adae544a0 2024-05-22T23:09:59.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.866+0000 7fc676041640 1 --2- 172.21.15.84:0/403744827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae544a0 0x561adae5afc0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc66000f530 tx=0x7fc66000f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c10131fd6704c48 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:59.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.866+0000 7fc659ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:09:59.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.866+0000 7fc659ffb640 1 -- 172.21.15.84:0/403744827 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc660013850 con 0x561adae544a0 2024-05-22T23:09:59.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.866+0000 7fc659ffb640 1 -- 172.21.15.84:0/403744827 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7fc6600139f0 con 0x561adae544a0 2024-05-22T23:09:59.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.866+0000 7fc67afb9740 1 -- 172.21.15.84:0/403744827 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc66c018b50 con 0x561adae544a0 2024-05-22T23:09:59.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.867+0000 7fc659ffb640 1 -- 172.21.15.84:0/403744827 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fc660013bf0 con 0x561adae544a0 2024-05-22T23:09:59.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.867+0000 7fc67afb9740 10 client.16770 fetch_fsmap learned FSMap version 18 2024-05-22T23:09:59.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.867+0000 7fc67afb9740 1 -- 172.21.15.84:0/403744827 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x561adaaca3f0 con 0x561adae544a0 2024-05-22T23:09:59.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.867+0000 7fc659ffb640 1 -- 172.21.15.84:0/403744827 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc66c018b50 con 0x561adae544a0 2024-05-22T23:09:59.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.867+0000 7fc659ffb640 1 -- 172.21.15.84:0/403744827 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fc660022070 con 0x561adae544a0 2024-05-22T23:09:59.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.867+0000 7fc67afb9740 10 client.16770 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:09:59.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.867+0000 7fc67afb9740 10 client.16770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:09:59.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.867+0000 7fc67afb9740 20 client.16770 populate_metadata read hostname 'smithi084' 2024-05-22T23:09:59.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.867+0000 7fc67afb9740 1 --2- 172.21.15.84:0/403744827 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x561adae6e110 0x561adae70500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.867+0000 7fc67afb9740 4 client.16770 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:09:59.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.867+0000 7fc67afb9740 1 -- 172.21.15.84:0/403744827 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561adadc44e0 con 0x561adae6e110 2024-05-22T23:09:59.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.867+0000 7fc675840640 1 --2- 172.21.15.84:0/403744827 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x561adae6e110 0x561adae70500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.868+0000 7fc659ffb640 10 client.16770 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:59.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.868+0000 7fc675840640 1 --2- 172.21.15.84:0/403744827 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x561adae6e110 0x561adae70500 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce5a0d3bbad0684 server_cookie=f08f7146f073a80 in_seq=0 out_seq=0 2024-05-22T23:09:59.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.868+0000 7fc659ffb640 1 -- 172.21.15.84:0/403744827 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561adadc44e0 con 0x561adae6e110 2024-05-22T23:09:59.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.868+0000 7fc659ffb640 10 client.16770 handle_command_reply: tid=0 2024-05-22T23:09:59.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.868+0000 7fc67afb9740 1 client.16770 shutdown 2024-05-22T23:09:59.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.868+0000 7fc67afb9740 1 -- 172.21.15.84:0/403744827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae544a0 msgr2=0x561adae5afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:59.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.868+0000 7fc67afb9740 1 --2- 172.21.15.84:0/403744827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae544a0 0x561adae5afc0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc66000f530 tx=0x7fc66000f9f0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.868+0000 7fc67afb9740 1 -- 172.21.15.84:0/403744827 shutdown_connections 2024-05-22T23:09:59.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.868+0000 7fc67afb9740 1 --2- 172.21.15.84:0/403744827 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x561adae6e110 0x561adae70500 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.868+0000 7fc67afb9740 1 --2- 172.21.15.84:0/403744827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561adae5db30 0x561adae5b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.868+0000 7fc659ffb640 0 client.16770 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:59.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.868+0000 7fc67afb9740 1 --2- 172.21.15.84:0/403744827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae544a0 0x561adae5afc0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.868+0000 7fc67afb9740 1 --2- 172.21.15.84:0/403744827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561adae52db0 0x561adae5aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.868+0000 7fc67afb9740 1 -- 172.21.15.84:0/403744827 >> 172.21.15.84:0/403744827 conn(0x561adae1b8e0 msgr2=0x561adae55180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:59.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.869+0000 7fc67afb9740 1 -- 172.21.15.84:0/403744827 shutdown_connections 2024-05-22T23:09:59.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.869+0000 7fc67afb9740 1 -- 172.21.15.84:0/403744827 wait complete. 2024-05-22T23:09:59.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.869+0000 7fc67afb9740 20 client.16770 trim_cache size 0 max 16384 2024-05-22T23:09:59.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.886+0000 7fc67afb9740 1 Processor -- start 2024-05-22T23:09:59.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.886+0000 7fc67afb9740 1 -- start start 2024-05-22T23:09:59.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.887+0000 7fc67afb9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae52db0 0x561adae698f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.887+0000 7fc67afb9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561adae544a0 0x561adae661e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.887+0000 7fc67afb9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561adae5db30 0x561adae5b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.887+0000 7fc67afb9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc6701ab630 con 0x561adae52db0 2024-05-22T23:09:59.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.887+0000 7fc67afb9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc6701ade90 con 0x561adae544a0 2024-05-22T23:09:59.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.887+0000 7fc67afb9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc6701abc80 con 0x561adae5db30 2024-05-22T23:09:59.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.887+0000 7fc675840640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae52db0 0x561adae698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.887+0000 7fc675840640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae52db0 0x561adae698f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37470/0 (socket says 172.21.15.84:37470) 2024-05-22T23:09:59.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.887+0000 7fc675840640 1 -- 172.21.15.84:0/573472125 learned_addr learned my addr 172.21.15.84:0/573472125 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:59.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.887+0000 7fc676041640 1 --2- 172.21.15.84:0/573472125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561adae544a0 0x561adae661e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.887+0000 7fc675840640 1 -- 172.21.15.84:0/573472125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561adae5db30 msgr2=0x561adae5b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:59.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.887+0000 7fc675840640 1 --2- 172.21.15.84:0/573472125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561adae5db30 0x561adae5b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.887+0000 7fc675840640 1 -- 172.21.15.84:0/573472125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561adae544a0 msgr2=0x561adae661e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:59.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.887+0000 7fc675840640 1 --2- 172.21.15.84:0/573472125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561adae544a0 0x561adae661e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.887+0000 7fc675840640 1 -- 172.21.15.84:0/573472125 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561adae70870 con 0x561adae52db0 2024-05-22T23:09:59.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.887+0000 7fc675840640 1 --2- 172.21.15.84:0/573472125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae52db0 0x561adae698f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00c140 tx=0x7fc66c016590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5625199a62d4d51f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.887+0000 7fc65bfff640 1 -- 172.21.15.84:0/573472125 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c02d070 con 0x561adae52db0 2024-05-22T23:09:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.887+0000 7fc65bfff640 1 -- 172.21.15.84:0/573472125 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc66c028e50 con 0x561adae52db0 2024-05-22T23:09:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.887+0000 7fc65bfff640 1 -- 172.21.15.84:0/573472125 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c01d040 con 0x561adae52db0 2024-05-22T23:09:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.888+0000 7fc67afb9740 1 -- 172.21.15.84:0/573472125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae52db0 msgr2=0x561adae698f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.888+0000 7fc67afb9740 1 --2- 172.21.15.84:0/573472125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae52db0 0x561adae698f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00c140 tx=0x7fc66c016590 comp rx=0 tx=0).stop 2024-05-22T23:09:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.888+0000 7fc67afb9740 1 -- 172.21.15.84:0/573472125 shutdown_connections 2024-05-22T23:09:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.888+0000 7fc67afb9740 1 --2- 172.21.15.84:0/573472125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561adae5db30 0x561adae5b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.888+0000 7fc67afb9740 1 --2- 172.21.15.84:0/573472125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561adae544a0 0x561adae661e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.888+0000 7fc67afb9740 1 --2- 172.21.15.84:0/573472125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae52db0 0x561adae698f0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.888+0000 7fc67afb9740 1 -- 172.21.15.84:0/573472125 >> 172.21.15.84:0/573472125 conn(0x561adae1b8e0 msgr2=0x561adae4dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:59.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.888+0000 7fc67afb9740 1 -- 172.21.15.84:0/573472125 shutdown_connections 2024-05-22T23:09:59.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.888+0000 7fc67afb9740 1 -- 172.21.15.84:0/573472125 wait complete. 2024-05-22T23:09:59.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.888+0000 7fc67afb9740 1 -- start start 2024-05-22T23:09:59.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.888+0000 7fc67afb9740 1 Processor -- start 2024-05-22T23:09:59.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.889+0000 7fc67afb9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae52db0 0x561adae57420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.889+0000 7fc67afb9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561adae544a0 0x561adae57960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.889+0000 7fc67afb9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561adae5db30 0x561adae57ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.889+0000 7fc67afb9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc6701ac350 con 0x561adae52db0 2024-05-22T23:09:59.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.889+0000 7fc67afb9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc6701ad7c0 con 0x561adae544a0 2024-05-22T23:09:59.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.889+0000 7fc67afb9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc6701aca20 con 0x561adae5db30 2024-05-22T23:09:59.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.889+0000 7fc67afb9740 10 client.0 _ll_register_callbacks cb 0x7fc66000a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:09:59.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.889+0000 7fc675840640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae52db0 0x561adae57420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.889+0000 7fc675840640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae52db0 0x561adae57420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37474/0 (socket says 172.21.15.84:37474) 2024-05-22T23:09:59.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.889+0000 7fc675840640 1 -- 172.21.15.84:0/2056486570 learned_addr learned my addr 172.21.15.84:0/2056486570 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:09:59.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.889+0000 7fc675840640 1 -- 172.21.15.84:0/2056486570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561adae5db30 msgr2=0x561adae57ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:59.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.889+0000 7fc675840640 1 --2- 172.21.15.84:0/2056486570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561adae5db30 0x561adae57ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.889+0000 7fc675840640 1 -- 172.21.15.84:0/2056486570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561adae544a0 msgr2=0x561adae57960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:09:59.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.889+0000 7fc675840640 1 --2- 172.21.15.84:0/2056486570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561adae544a0 0x561adae57960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.889+0000 7fc675840640 1 -- 172.21.15.84:0/2056486570 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561adaa8a0f0 con 0x561adae52db0 2024-05-22T23:09:59.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.889+0000 7fc675840640 1 --2- 172.21.15.84:0/2056486570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae52db0 0x561adae57420 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc66c010040 tx=0x7fc66c00c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd2f7c0e7c391d88 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:09:59.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.889+0000 7fc659ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:09:59.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.889+0000 7fc659ffb640 1 -- 172.21.15.84:0/2056486570 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c036060 con 0x561adae52db0 2024-05-22T23:09:59.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.889+0000 7fc659ffb640 1 -- 172.21.15.84:0/2056486570 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8571+0+0 (secure 0 0 0) 0x7fc66c0189f0 con 0x561adae52db0 2024-05-22T23:09:59.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.889+0000 7fc67afb9740 1 -- 172.21.15.84:0/2056486570 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc66c028e50 con 0x561adae52db0 2024-05-22T23:09:59.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.889+0000 7fc659ffb640 1 -- 172.21.15.84:0/2056486570 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fc66c02d050 con 0x561adae52db0 2024-05-22T23:09:59.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.889+0000 7fc67afb9740 10 client.16776 fetch_fsmap learned FSMap version 18 2024-05-22T23:09:59.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.889+0000 7fc67afb9740 1 -- 172.21.15.84:0/2056486570 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x561adad75fe0 con 0x561adae52db0 2024-05-22T23:09:59.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.890+0000 7fc659ffb640 1 -- 172.21.15.84:0/2056486570 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc66c028e50 con 0x561adae52db0 2024-05-22T23:09:59.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.890+0000 7fc659ffb640 1 -- 172.21.15.84:0/2056486570 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fc66c015040 con 0x561adae52db0 2024-05-22T23:09:59.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.890+0000 7fc67afb9740 10 client.16776 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:09:59.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.890+0000 7fc67afb9740 10 client.16776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:09:59.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.890+0000 7fc67afb9740 20 client.16776 populate_metadata read hostname 'smithi084' 2024-05-22T23:09:59.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.890+0000 7fc67afb9740 1 --2- 172.21.15.84:0/2056486570 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x561adaed8b00 0x561adaedaf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:09:59.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.890+0000 7fc67afb9740 4 client.16776 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:09:59.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.890+0000 7fc67afb9740 1 -- 172.21.15.84:0/2056486570 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561adadbe830 con 0x561adaed8b00 2024-05-22T23:09:59.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.890+0000 7fc676041640 1 --2- 172.21.15.84:0/2056486570 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x561adaed8b00 0x561adaedaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:09:59.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.890+0000 7fc659ffb640 10 client.16776 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:59.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.890+0000 7fc676041640 1 --2- 172.21.15.84:0/2056486570 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x561adaed8b00 0x561adaedaf10 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2065519136076b71 server_cookie=4ffad938fbac4e1d in_seq=0 out_seq=0 2024-05-22T23:09:59.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc659ffb640 1 -- 172.21.15.84:0/2056486570 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561adadbe830 con 0x561adaed8b00 2024-05-22T23:09:59.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc659ffb640 10 client.16776 handle_command_reply: tid=0 2024-05-22T23:09:59.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc67afb9740 1 client.16776 shutdown 2024-05-22T23:09:59.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc67afb9740 1 -- 172.21.15.84:0/2056486570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae52db0 msgr2=0x561adae57420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:59.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc67afb9740 1 --2- 172.21.15.84:0/2056486570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae52db0 0x561adae57420 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc66c010040 tx=0x7fc66c00c3e0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc67afb9740 1 -- 172.21.15.84:0/2056486570 shutdown_connections 2024-05-22T23:09:59.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc67afb9740 1 --2- 172.21.15.84:0/2056486570 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x561adaed8b00 0x561adaedaf10 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc67afb9740 1 --2- 172.21.15.84:0/2056486570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561adae5db30 0x561adae57ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc67afb9740 1 --2- 172.21.15.84:0/2056486570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561adae544a0 0x561adae57960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc67afb9740 1 --2- 172.21.15.84:0/2056486570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561adae52db0 0x561adae57420 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc659ffb640 0 client.16776 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:09:59.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc67afb9740 1 -- 172.21.15.84:0/2056486570 >> 172.21.15.84:0/2056486570 conn(0x561adae1b8e0 msgr2=0x561adae54880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:59.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc67afb9740 1 -- 172.21.15.84:0/2056486570 shutdown_connections 2024-05-22T23:09:59.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc67afb9740 1 -- 172.21.15.84:0/2056486570 wait complete. 2024-05-22T23:09:59.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc67afb9740 20 client.16776 trim_cache size 0 max 16384 2024-05-22T23:09:59.910 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:09:59.910 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-22T23:09:59.911 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": { 2024-05-22T23:09:59.911 INFO:teuthology.orchestra.run.smithi084.stdout: "009fd2d4-9f08-4c32-bc39-6dec6b85d10b": { 2024-05-22T23:09:59.911 INFO:teuthology.orchestra.run.smithi084.stdout: "path": "/", 2024-05-22T23:09:59.911 INFO:teuthology.orchestra.run.smithi084.stdout: "tag": "009fd2d4-9f08-4c32-bc39-6dec6b85d10b", 2024-05-22T23:09:59.911 INFO:teuthology.orchestra.run.smithi084.stdout: "options": "recursive,force" 2024-05-22T23:09:59.911 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:09:59.911 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:09:59.912 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:09:59.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc6782a5640 1 -- 172.21.15.84:0/929613935 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fc65c0a2d00 msgr2=0x7fc65c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:59.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc6782a5640 1 --2- 172.21.15.84:0/929613935 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fc65c0a2d00 0x7fc65c0a51c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc6680eeae0 tx=0x7fc66802a8e0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc6782a5640 1 -- 172.21.15.84:0/929613935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc670106750 msgr2=0x7fc67019cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:09:59.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc6782a5640 1 --2- 172.21.15.84:0/929613935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc670106750 0x7fc67019cb10 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc668006010 tx=0x7fc668003c20 comp rx=0 tx=0).stop 2024-05-22T23:09:59.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc6782a5640 1 -- 172.21.15.84:0/929613935 shutdown_connections 2024-05-22T23:09:59.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc6782a5640 1 --2- 172.21.15.84:0/929613935 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fc65c0a2d00 0x7fc65c0a51c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc6782a5640 1 --2- 172.21.15.84:0/929613935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc670106750 0x7fc67019cb10 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc6782a5640 1 --2- 172.21.15.84:0/929613935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc670101100 0x7fc670198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc6782a5640 1 --2- 172.21.15.84:0/929613935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc670100740 0x7fc670198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:09:59.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc6782a5640 1 -- 172.21.15.84:0/929613935 >> 172.21.15.84:0/929613935 conn(0x7fc670077870 msgr2=0x7fc670068d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:09:59.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc6782a5640 1 -- 172.21.15.84:0/929613935 shutdown_connections 2024-05-22T23:09:59.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:09:59.891+0000 7fc6782a5640 1 -- 172.21.15.84:0/929613935 wait complete. 2024-05-22T23:09:59.915 INFO:tasks.cephfs.filesystem:scrub status for tag:009fd2d4-9f08-4c32-bc39-6dec6b85d10b - {'path': '/', 'tag': '009fd2d4-9f08-4c32-bc39-6dec6b85d10b', 'options': 'recursive,force'} 2024-05-22T23:10:00.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:00 smithi084 ceph-mon[32186]: scrub queued for path: / 2024-05-22T23:10:00.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:00 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:10:00.380 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:00 smithi084 ceph-mon[32186]: scrub summary: active paths [/] 2024-05-22T23:10:00.381 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:00 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:10:00.381 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:00 smithi084 ceph-mon[32186]: overall HEALTH_OK 2024-05-22T23:10:00.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:00 smithi204 ceph-mon[36211]: scrub queued for path: / 2024-05-22T23:10:00.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:00 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:10:00.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:00 smithi204 ceph-mon[36211]: scrub summary: active paths [/] 2024-05-22T23:10:00.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:00 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:10:00.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:00 smithi204 ceph-mon[36211]: overall HEALTH_OK 2024-05-22T23:10:00.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:00 smithi096 ceph-mon[36392]: scrub queued for path: / 2024-05-22T23:10:00.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:00 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:10:00.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:00 smithi096 ceph-mon[36392]: scrub summary: active paths [/] 2024-05-22T23:10:00.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:00 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:10:00.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:00 smithi096 ceph-mon[36392]: overall HEALTH_OK 2024-05-22T23:10:01.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:01 smithi204 ceph-mon[36211]: pgmap v252: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-05-22T23:10:01.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:01 smithi204 ceph-mon[36211]: scrub summary: idle 2024-05-22T23:10:01.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:01 smithi096 ceph-mon[36392]: pgmap v252: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-05-22T23:10:01.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:01 smithi096 ceph-mon[36392]: scrub summary: idle 2024-05-22T23:10:01.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:01 smithi084 ceph-mon[32186]: pgmap v252: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-05-22T23:10:01.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:01 smithi084 ceph-mon[32186]: scrub summary: idle 2024-05-22T23:10:03.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:03 smithi204 ceph-mon[36211]: pgmap v253: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 62 KiB/s wr, 22 op/s 2024-05-22T23:10:03.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:03 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:10:03.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:03 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:10:03.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:03 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:10:03.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:03 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:10:03.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:03 smithi096 ceph-mon[36392]: pgmap v253: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 62 KiB/s wr, 22 op/s 2024-05-22T23:10:03.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:03 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:10:03.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:03 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:10:03.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:03 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:10:03.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:03 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:10:03.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:03 smithi084 ceph-mon[32186]: pgmap v253: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 62 KiB/s wr, 22 op/s 2024-05-22T23:10:03.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:03 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:10:03.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:03 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:10:03.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:03 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:10:03.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:03 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:10:05.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:05 smithi204 ceph-mon[36211]: pgmap v254: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-05-22T23:10:05.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:05 smithi096 ceph-mon[36392]: pgmap v254: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-05-22T23:10:05.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:05 smithi084 ceph-mon[32186]: pgmap v254: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-05-22T23:10:07.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:07 smithi204 ceph-mon[36211]: pgmap v255: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-05-22T23:10:07.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:07 smithi204 ceph-mon[36211]: osdmap e94: 12 total, 12 up, 12 in 2024-05-22T23:10:07.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:07 smithi096 ceph-mon[36392]: pgmap v255: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-05-22T23:10:07.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:07 smithi096 ceph-mon[36392]: osdmap e94: 12 total, 12 up, 12 in 2024-05-22T23:10:07.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:07 smithi084 ceph-mon[32186]: pgmap v255: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-05-22T23:10:07.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:07 smithi084 ceph-mon[32186]: osdmap e94: 12 total, 12 up, 12 in 2024-05-22T23:10:09.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:09 smithi204 ceph-mon[36211]: pgmap v257: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 61 KiB/s wr, 25 op/s 2024-05-22T23:10:09.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:09 smithi096 ceph-mon[36392]: pgmap v257: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 61 KiB/s wr, 25 op/s 2024-05-22T23:10:09.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:09 smithi084 ceph-mon[32186]: pgmap v257: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 61 KiB/s wr, 25 op/s 2024-05-22T23:10:11.416 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:11 smithi204 ceph-mon[36211]: pgmap v258: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 61 KiB/s wr, 25 op/s 2024-05-22T23:10:11.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:11 smithi096 ceph-mon[36392]: pgmap v258: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 61 KiB/s wr, 25 op/s 2024-05-22T23:10:11.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:11 smithi084 ceph-mon[32186]: pgmap v258: 97 pgs: 97 active+clean; 456 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 61 KiB/s wr, 25 op/s 2024-05-22T23:10:13.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:13 smithi096 ceph-mon[36392]: pgmap v259: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.3 KiB/s wr, 2 op/s 2024-05-22T23:10:13.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:13 smithi084 ceph-mon[32186]: pgmap v259: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.3 KiB/s wr, 2 op/s 2024-05-22T23:10:13.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:13 smithi204 ceph-mon[36211]: pgmap v259: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.3 KiB/s wr, 2 op/s 2024-05-22T23:10:15.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:15 smithi096 ceph-mon[36392]: pgmap v260: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.3 KiB/s wr, 2 op/s 2024-05-22T23:10:15.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:15 smithi084 ceph-mon[32186]: pgmap v260: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.3 KiB/s wr, 2 op/s 2024-05-22T23:10:15.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:15 smithi204 ceph-mon[36211]: pgmap v260: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.3 KiB/s wr, 2 op/s 2024-05-22T23:10:17.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:17 smithi096 ceph-mon[36392]: pgmap v261: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.1 KiB/s wr, 2 op/s 2024-05-22T23:10:17.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:17 smithi096 ceph-mon[36392]: osdmap e95: 12 total, 12 up, 12 in 2024-05-22T23:10:17.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:17 smithi084 ceph-mon[32186]: pgmap v261: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.1 KiB/s wr, 2 op/s 2024-05-22T23:10:17.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:17 smithi084 ceph-mon[32186]: osdmap e95: 12 total, 12 up, 12 in 2024-05-22T23:10:17.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:17 smithi204 ceph-mon[36211]: pgmap v261: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.1 KiB/s wr, 2 op/s 2024-05-22T23:10:17.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:17 smithi204 ceph-mon[36211]: osdmap e95: 12 total, 12 up, 12 in 2024-05-22T23:10:19.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:19 smithi096 ceph-mon[36392]: pgmap v263: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.1 KiB/s wr, 2 op/s 2024-05-22T23:10:19.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:19 smithi084 ceph-mon[32186]: pgmap v263: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.1 KiB/s wr, 2 op/s 2024-05-22T23:10:19.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:19 smithi204 ceph-mon[36211]: pgmap v263: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.1 KiB/s wr, 2 op/s 2024-05-22T23:10:21.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:21 smithi096 ceph-mon[36392]: pgmap v264: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.1 KiB/s wr, 2 op/s 2024-05-22T23:10:21.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:21 smithi084 ceph-mon[32186]: pgmap v264: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.1 KiB/s wr, 2 op/s 2024-05-22T23:10:21.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:21 smithi204 ceph-mon[36211]: pgmap v264: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.1 KiB/s wr, 2 op/s 2024-05-22T23:10:23.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:23 smithi096 ceph-mon[36392]: pgmap v265: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.7 KiB/s wr, 2 op/s 2024-05-22T23:10:23.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:23 smithi084 ceph-mon[32186]: pgmap v265: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.7 KiB/s wr, 2 op/s 2024-05-22T23:10:23.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:23 smithi204 ceph-mon[36211]: pgmap v265: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.7 KiB/s wr, 2 op/s 2024-05-22T23:10:25.468 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:25 smithi204 ceph-mon[36211]: pgmap v266: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.7 KiB/s wr, 2 op/s 2024-05-22T23:10:25.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:25 smithi096 ceph-mon[36392]: pgmap v266: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.7 KiB/s wr, 2 op/s 2024-05-22T23:10:25.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:25 smithi084 ceph-mon[32186]: pgmap v266: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.7 KiB/s wr, 2 op/s 2024-05-22T23:10:27.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:27 smithi096 ceph-mon[36392]: pgmap v267: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.7 KiB/s wr, 2 op/s 2024-05-22T23:10:27.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:27 smithi084 ceph-mon[32186]: pgmap v267: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.7 KiB/s wr, 2 op/s 2024-05-22T23:10:27.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:27 smithi204 ceph-mon[36211]: pgmap v267: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.7 KiB/s wr, 2 op/s 2024-05-22T23:10:29.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:29 smithi096 ceph-mon[36392]: pgmap v268: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.0 KiB/s wr, 2 op/s 2024-05-22T23:10:29.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:29 smithi084 ceph-mon[32186]: pgmap v268: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.0 KiB/s wr, 2 op/s 2024-05-22T23:10:29.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:29 smithi204 ceph-mon[36211]: pgmap v268: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.0 KiB/s wr, 2 op/s 2024-05-22T23:10:29.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:10:29.917 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:10:30.036 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.027+0000 7fb134641640 1 -- 172.21.15.84:0/2233506125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb12c102200 msgr2=0x7fb12c10e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:30.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.027+0000 7fb134641640 1 --2- 172.21.15.84:0/2233506125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb12c102200 0x7fb12c10e710 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fb124009930 tx=0x7fb12402a420 comp rx=0 tx=0).stop 2024-05-22T23:10:30.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.027+0000 7fb134641640 1 -- 172.21.15.84:0/2233506125 shutdown_connections 2024-05-22T23:10:30.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.027+0000 7fb134641640 1 --2- 172.21.15.84:0/2233506125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb12c102200 0x7fb12c10e710 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.027+0000 7fb134641640 1 --2- 172.21.15.84:0/2233506125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb12c101840 0x7fb12c101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.027+0000 7fb134641640 1 --2- 172.21.15.84:0/2233506125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb12c107850 0x7fb12c107c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.027+0000 7fb134641640 1 -- 172.21.15.84:0/2233506125 >> 172.21.15.84:0/2233506125 conn(0x7fb12c0fd2b0 msgr2=0x7fb12c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:30.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.027+0000 7fb134641640 1 -- 172.21.15.84:0/2233506125 shutdown_connections 2024-05-22T23:10:30.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.027+0000 7fb134641640 1 -- 172.21.15.84:0/2233506125 wait complete. 2024-05-22T23:10:30.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.028+0000 7fb134641640 1 Processor -- start 2024-05-22T23:10:30.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.028+0000 7fb134641640 1 -- start start 2024-05-22T23:10:30.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.028+0000 7fb134641640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb12c101840 0x7fb12c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.028+0000 7fb134641640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb12c102200 0x7fb12c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.028+0000 7fb134641640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb12c107850 0x7fb12c19c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.028+0000 7fb134641640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c111c20 con 0x7fb12c102200 2024-05-22T23:10:30.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.028+0000 7fb134641640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c111aa0 con 0x7fb12c107850 2024-05-22T23:10:30.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.028+0000 7fb134641640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c111da0 con 0x7fb12c101840 2024-05-22T23:10:30.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.028+0000 7fb131bdc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb12c102200 0x7fb12c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.028+0000 7fb131bdc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb12c102200 0x7fb12c1985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48450/0 (socket says 172.21.15.84:48450) 2024-05-22T23:10:30.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.028+0000 7fb131bdc640 1 -- 172.21.15.84:0/2015175452 learned_addr learned my addr 172.21.15.84:0/2015175452 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:10:30.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.028+0000 7fb132bde640 1 --2- 172.21.15.84:0/2015175452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb12c107850 0x7fb12c19c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.028+0000 7fb1323dd640 1 --2- 172.21.15.84:0/2015175452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb12c101840 0x7fb12c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.028+0000 7fb131bdc640 1 -- 172.21.15.84:0/2015175452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb12c101840 msgr2=0x7fb12c198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:30.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.028+0000 7fb131bdc640 1 --2- 172.21.15.84:0/2015175452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb12c101840 0x7fb12c198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.028+0000 7fb131bdc640 1 -- 172.21.15.84:0/2015175452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb12c107850 msgr2=0x7fb12c19c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:30.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.028+0000 7fb131bdc640 1 --2- 172.21.15.84:0/2015175452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb12c107850 0x7fb12c19c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.028+0000 7fb131bdc640 1 -- 172.21.15.84:0/2015175452 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb12c19d050 con 0x7fb12c102200 2024-05-22T23:10:30.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.028+0000 7fb132bde640 1 --2- 172.21.15.84:0/2015175452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb12c107850 0x7fb12c19c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:10:30.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.029+0000 7fb131bdc640 1 --2- 172.21.15.84:0/2015175452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb12c102200 0x7fb12c1985a0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fb11c00d950 tx=0x7fb11c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:30.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.029+0000 7fb1237fe640 1 -- 172.21.15.84:0/2015175452 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11c017070 con 0x7fb12c102200 2024-05-22T23:10:30.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.029+0000 7fb1237fe640 1 -- 172.21.15.84:0/2015175452 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb11c009c30 con 0x7fb12c102200 2024-05-22T23:10:30.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.029+0000 7fb134641640 1 -- 172.21.15.84:0/2015175452 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb12c19d340 con 0x7fb12c102200 2024-05-22T23:10:30.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.029+0000 7fb1237fe640 1 -- 172.21.15.84:0/2015175452 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11c00f6b0 con 0x7fb12c102200 2024-05-22T23:10:30.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.029+0000 7fb134641640 1 -- 172.21.15.84:0/2015175452 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb12c1a4bd0 con 0x7fb12c102200 2024-05-22T23:10:30.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.031+0000 7fb1237fe640 1 -- 172.21.15.84:0/2015175452 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb11c0109d0 con 0x7fb12c102200 2024-05-22T23:10:30.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.032+0000 7fb137420740 1 Processor -- start 2024-05-22T23:10:30.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.032+0000 7fb137420740 1 -- start start 2024-05-22T23:10:30.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.032+0000 7fb137420740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f1a8bf4a0 0x558f1a8bd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.032+0000 7fb137420740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8bddb0 0x558f1a8be230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.032+0000 7fb137420740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f1a8c8b30 0x558f1a8caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.032+0000 7fb137420740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558f1a821db0 con 0x558f1a8bddb0 2024-05-22T23:10:30.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.032+0000 7fb137420740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558f1a7c0360 con 0x558f1a8c8b30 2024-05-22T23:10:30.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.032+0000 7fb137420740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558f1a81dd70 con 0x558f1a8bf4a0 2024-05-22T23:10:30.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.032+0000 7fb1237fe640 1 --2- 172.21.15.84:0/2015175452 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fb1180a2d00 0x7fb1180a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.032+0000 7fb1323dd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8bddb0 0x558f1a8be230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.032+0000 7fb1323dd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8bddb0 0x558f1a8be230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48466/0 (socket says 172.21.15.84:48466) 2024-05-22T23:10:30.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.032+0000 7fb1323dd640 1 -- 172.21.15.84:0/1003604893 learned_addr learned my addr 172.21.15.84:0/1003604893 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:10:30.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.032+0000 7fb132bde640 1 --2- 172.21.15.84:0/1003604893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f1a8bf4a0 0x558f1a8bd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.033+0000 7fb132bde640 1 --2- 172.21.15.84:0/2015175452 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fb1180a2d00 0x7fb1180a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.033+0000 7fb131bdc640 1 --2- 172.21.15.84:0/1003604893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f1a8c8b30 0x558f1a8caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.033+0000 7fb1237fe640 1 -- 172.21.15.84:0/2015175452 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7fb11c0cf180 con 0x7fb12c102200 2024-05-22T23:10:30.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.033+0000 7fb1323dd640 1 -- 172.21.15.84:0/1003604893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f1a8bf4a0 msgr2=0x558f1a8bd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:30.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.033+0000 7fb1323dd640 1 --2- 172.21.15.84:0/1003604893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f1a8bf4a0 0x558f1a8bd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.033+0000 7fb1323dd640 1 -- 172.21.15.84:0/1003604893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f1a8c8b30 msgr2=0x558f1a8caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:30.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.033+0000 7fb1323dd640 1 --2- 172.21.15.84:0/1003604893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f1a8c8b30 0x558f1a8caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.033+0000 7fb1323dd640 1 -- 172.21.15.84:0/1003604893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f1a7e3a60 con 0x558f1a8bddb0 2024-05-22T23:10:30.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.033+0000 7fb131bdc640 1 --2- 172.21.15.84:0/1003604893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f1a8c8b30 0x558f1a8caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:10:30.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.033+0000 7fb132bde640 1 --2- 172.21.15.84:0/2015175452 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fb1180a2d00 0x7fb1180a51c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb124031550 tx=0x7fb124009260 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:30.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.033+0000 7fb1323dd640 1 --2- 172.21.15.84:0/1003604893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8bddb0 0x558f1a8be230 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fb128008db0 tx=0x7fb12800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f90d302bc0bb6f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:30.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.033+0000 7fb120bfe640 1 -- 172.21.15.84:0/1003604893 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12800f830 con 0x558f1a8bddb0 2024-05-22T23:10:30.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.033+0000 7fb120bfe640 1 -- 172.21.15.84:0/1003604893 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb12800de20 con 0x558f1a8bddb0 2024-05-22T23:10:30.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.033+0000 7fb120bfe640 1 -- 172.21.15.84:0/1003604893 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12800e3b0 con 0x558f1a8bddb0 2024-05-22T23:10:30.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.034+0000 7fb137420740 1 -- 172.21.15.84:0/1003604893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8bddb0 msgr2=0x558f1a8be230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:30.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.034+0000 7fb137420740 1 --2- 172.21.15.84:0/1003604893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8bddb0 0x558f1a8be230 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fb128008db0 tx=0x7fb12800d710 comp rx=0 tx=0).stop 2024-05-22T23:10:30.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.034+0000 7fb137420740 1 -- 172.21.15.84:0/1003604893 shutdown_connections 2024-05-22T23:10:30.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.034+0000 7fb137420740 1 --2- 172.21.15.84:0/1003604893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f1a8c8b30 0x558f1a8caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.034+0000 7fb137420740 1 --2- 172.21.15.84:0/1003604893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8bddb0 0x558f1a8be230 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.034+0000 7fb137420740 1 --2- 172.21.15.84:0/1003604893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f1a8bf4a0 0x558f1a8bd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.034+0000 7fb137420740 1 -- 172.21.15.84:0/1003604893 >> 172.21.15.84:0/1003604893 conn(0x558f1a8868e0 msgr2=0x558f1a8ba310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:30.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.034+0000 7fb137420740 1 -- 172.21.15.84:0/1003604893 shutdown_connections 2024-05-22T23:10:30.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.034+0000 7fb137420740 1 -- 172.21.15.84:0/1003604893 wait complete. 2024-05-22T23:10:30.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.034+0000 7fb137420740 1 -- start start 2024-05-22T23:10:30.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.034+0000 7fb137420740 1 Processor -- start 2024-05-22T23:10:30.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.035+0000 7fb137420740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8bddb0 0x558f1a8c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.035+0000 7fb137420740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f1a8bf4a0 0x558f1a8c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.035+0000 7fb137420740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f1a8c8b30 0x558f1a8c6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.035+0000 7fb137420740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558f1a8cb9c0 con 0x558f1a8bddb0 2024-05-22T23:10:30.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.035+0000 7fb137420740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558f1a7dc0f0 con 0x558f1a8bf4a0 2024-05-22T23:10:30.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.035+0000 7fb137420740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558f1a8cbb40 con 0x558f1a8c8b30 2024-05-22T23:10:30.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.035+0000 7fb137420740 10 client.0 _ll_register_callbacks cb 0x558f1a7e0f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:10:30.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.035+0000 7fb1323dd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8bddb0 0x558f1a8c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.035+0000 7fb1323dd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8bddb0 0x558f1a8c5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48474/0 (socket says 172.21.15.84:48474) 2024-05-22T23:10:30.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.035+0000 7fb1323dd640 1 -- 172.21.15.84:0/3084873313 learned_addr learned my addr 172.21.15.84:0/3084873313 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:10:30.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.035+0000 7fb131bdc640 1 --2- 172.21.15.84:0/3084873313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f1a8bf4a0 0x558f1a8c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.035+0000 7fb132bde640 1 --2- 172.21.15.84:0/3084873313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f1a8c8b30 0x558f1a8c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.035+0000 7fb1323dd640 1 -- 172.21.15.84:0/3084873313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f1a8c8b30 msgr2=0x558f1a8c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:30.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.035+0000 7fb1323dd640 1 --2- 172.21.15.84:0/3084873313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f1a8c8b30 0x558f1a8c6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.035+0000 7fb1323dd640 1 -- 172.21.15.84:0/3084873313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f1a8bf4a0 msgr2=0x558f1a8c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:30.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.035+0000 7fb1323dd640 1 --2- 172.21.15.84:0/3084873313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f1a8bf4a0 0x558f1a8c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.035+0000 7fb1323dd640 1 -- 172.21.15.84:0/3084873313 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f1a5ff7d0 con 0x558f1a8bddb0 2024-05-22T23:10:30.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.035+0000 7fb132bde640 1 --2- 172.21.15.84:0/3084873313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f1a8c8b30 0x558f1a8c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:10:30.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.035+0000 7fb1323dd640 1 --2- 172.21.15.84:0/3084873313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8bddb0 0x558f1a8c5a00 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb128018af0 tx=0x7fb1280084f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46688857d7fd7e9c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:30.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.035+0000 7fb115ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:10:30.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.035+0000 7fb115ffb640 1 -- 172.21.15.84:0/3084873313 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb128028060 con 0x558f1a8bddb0 2024-05-22T23:10:30.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.035+0000 7fb115ffb640 1 -- 172.21.15.84:0/3084873313 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7fb128004c50 con 0x558f1a8bddb0 2024-05-22T23:10:30.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.035+0000 7fb137420740 1 -- 172.21.15.84:0/3084873313 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb12800de20 con 0x558f1a8bddb0 2024-05-22T23:10:30.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.036+0000 7fb115ffb640 1 -- 172.21.15.84:0/3084873313 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fb128004e50 con 0x558f1a8bddb0 2024-05-22T23:10:30.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.036+0000 7fb137420740 10 client.16788 fetch_fsmap learned FSMap version 18 2024-05-22T23:10:30.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.036+0000 7fb137420740 1 -- 172.21.15.84:0/3084873313 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x558f1a5353f0 con 0x558f1a8bddb0 2024-05-22T23:10:30.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.036+0000 7fb115ffb640 1 -- 172.21.15.84:0/3084873313 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb12800de20 con 0x558f1a8bddb0 2024-05-22T23:10:30.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.036+0000 7fb115ffb640 1 -- 172.21.15.84:0/3084873313 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fb1280054b0 con 0x558f1a8bddb0 2024-05-22T23:10:30.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.036+0000 7fb137420740 10 client.16788 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:10:30.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.036+0000 7fb137420740 10 client.16788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:10:30.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.036+0000 7fb137420740 20 client.16788 populate_metadata read hostname 'smithi084' 2024-05-22T23:10:30.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.036+0000 7fb137420740 1 --2- 172.21.15.84:0/3084873313 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558f1a8d8ef0 0x558f1a8db2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.036+0000 7fb137420740 4 client.16788 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:10:30.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.036+0000 7fb137420740 1 -- 172.21.15.84:0/3084873313 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f1a82f4e0 con 0x558f1a8d8ef0 2024-05-22T23:10:30.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.036+0000 7fb131bdc640 1 --2- 172.21.15.84:0/3084873313 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558f1a8d8ef0 0x558f1a8db2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.037+0000 7fb131bdc640 1 --2- 172.21.15.84:0/3084873313 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558f1a8d8ef0 0x558f1a8db2e0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=258286c422b1ea6e server_cookie=fb31ae8b88b1aa9f in_seq=0 out_seq=0 2024-05-22T23:10:30.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.037+0000 7fb115ffb640 10 client.16788 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:10:30.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.037+0000 7fb115ffb640 1 -- 172.21.15.84:0/3084873313 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558f1a82f4e0 con 0x558f1a8d8ef0 2024-05-22T23:10:30.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.037+0000 7fb115ffb640 10 client.16788 handle_command_reply: tid=0 2024-05-22T23:10:30.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.037+0000 7fb137420740 1 client.16788 shutdown 2024-05-22T23:10:30.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.037+0000 7fb137420740 1 -- 172.21.15.84:0/3084873313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8bddb0 msgr2=0x558f1a8c5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:30.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.037+0000 7fb137420740 1 --2- 172.21.15.84:0/3084873313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8bddb0 0x558f1a8c5a00 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb128018af0 tx=0x7fb1280084f0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.037+0000 7fb137420740 1 -- 172.21.15.84:0/3084873313 shutdown_connections 2024-05-22T23:10:30.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.037+0000 7fb137420740 1 --2- 172.21.15.84:0/3084873313 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558f1a8d8ef0 0x558f1a8db2e0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.037+0000 7fb137420740 1 --2- 172.21.15.84:0/3084873313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f1a8c8b30 0x558f1a8c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.037+0000 7fb115ffb640 0 client.16788 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:10:30.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.037+0000 7fb137420740 1 --2- 172.21.15.84:0/3084873313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f1a8bf4a0 0x558f1a8c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.037+0000 7fb137420740 1 --2- 172.21.15.84:0/3084873313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8bddb0 0x558f1a8c5a00 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.037+0000 7fb137420740 1 -- 172.21.15.84:0/3084873313 >> 172.21.15.84:0/3084873313 conn(0x558f1a8868e0 msgr2=0x558f1a8c0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:30.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.038+0000 7fb137420740 1 -- 172.21.15.84:0/3084873313 shutdown_connections 2024-05-22T23:10:30.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.038+0000 7fb137420740 1 -- 172.21.15.84:0/3084873313 wait complete. 2024-05-22T23:10:30.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.038+0000 7fb137420740 20 client.16788 trim_cache size 0 max 16384 2024-05-22T23:10:30.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.055+0000 7fb137420740 1 Processor -- start 2024-05-22T23:10:30.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.055+0000 7fb137420740 1 -- start start 2024-05-22T23:10:30.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.055+0000 7fb137420740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f1a8bddb0 0x558f1a8d48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.055+0000 7fb137420740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f1a8bf4a0 0x558f1a8c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.055+0000 7fb137420740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8c8b30 0x558f1a8caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.055+0000 7fb137420740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c1ab680 con 0x558f1a8c8b30 2024-05-22T23:10:30.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.055+0000 7fb137420740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c1adee0 con 0x558f1a8bddb0 2024-05-22T23:10:30.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.055+0000 7fb137420740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c1abcd0 con 0x558f1a8bf4a0 2024-05-22T23:10:30.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.055+0000 7fb132bde640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8c8b30 0x558f1a8caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.055+0000 7fb132bde640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8c8b30 0x558f1a8caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48484/0 (socket says 172.21.15.84:48484) 2024-05-22T23:10:30.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.055+0000 7fb132bde640 1 -- 172.21.15.84:0/2290490290 learned_addr learned my addr 172.21.15.84:0/2290490290 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:10:30.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.055+0000 7fb132bde640 1 -- 172.21.15.84:0/2290490290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f1a8bf4a0 msgr2=0x558f1a8c52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:30.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.055+0000 7fb132bde640 1 --2- 172.21.15.84:0/2290490290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f1a8bf4a0 0x558f1a8c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.055+0000 7fb132bde640 1 -- 172.21.15.84:0/2290490290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f1a8bddb0 msgr2=0x558f1a8d48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:30.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.055+0000 7fb132bde640 1 --2- 172.21.15.84:0/2290490290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f1a8bddb0 0x558f1a8d48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.055+0000 7fb132bde640 1 -- 172.21.15.84:0/2290490290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f1a4f50f0 con 0x558f1a8c8b30 2024-05-22T23:10:30.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.056+0000 7fb132bde640 1 --2- 172.21.15.84:0/2290490290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8c8b30 0x558f1a8caba0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb124009930 tx=0x7fb124040070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a36e4b7ab24e52 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:30.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.056+0000 7fb117fff640 1 -- 172.21.15.84:0/2290490290 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124035040 con 0x558f1a8c8b30 2024-05-22T23:10:30.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.056+0000 7fb117fff640 1 -- 172.21.15.84:0/2290490290 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb124050070 con 0x558f1a8c8b30 2024-05-22T23:10:30.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.056+0000 7fb117fff640 1 -- 172.21.15.84:0/2290490290 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12404aa40 con 0x558f1a8c8b30 2024-05-22T23:10:30.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.056+0000 7fb137420740 1 -- 172.21.15.84:0/2290490290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8c8b30 msgr2=0x558f1a8caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:30.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.056+0000 7fb137420740 1 --2- 172.21.15.84:0/2290490290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8c8b30 0x558f1a8caba0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb124009930 tx=0x7fb124040070 comp rx=0 tx=0).stop 2024-05-22T23:10:30.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.056+0000 7fb137420740 1 -- 172.21.15.84:0/2290490290 shutdown_connections 2024-05-22T23:10:30.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.056+0000 7fb137420740 1 --2- 172.21.15.84:0/2290490290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8c8b30 0x558f1a8caba0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.056+0000 7fb137420740 1 --2- 172.21.15.84:0/2290490290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f1a8bf4a0 0x558f1a8c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.056+0000 7fb137420740 1 --2- 172.21.15.84:0/2290490290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f1a8bddb0 0x558f1a8d48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.056+0000 7fb137420740 1 -- 172.21.15.84:0/2290490290 >> 172.21.15.84:0/2290490290 conn(0x558f1a8868e0 msgr2=0x558f1a8b8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:30.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.056+0000 7fb137420740 1 -- 172.21.15.84:0/2290490290 shutdown_connections 2024-05-22T23:10:30.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.056+0000 7fb137420740 1 -- 172.21.15.84:0/2290490290 wait complete. 2024-05-22T23:10:30.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.057+0000 7fb137420740 1 -- start start 2024-05-22T23:10:30.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.057+0000 7fb137420740 1 Processor -- start 2024-05-22T23:10:30.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.057+0000 7fb137420740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f1a8bddb0 0x558f1a8d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.057+0000 7fb137420740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8bf4a0 0x558f1a8c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.057+0000 7fb137420740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f1a8c8b30 0x558f1a8ca210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.057+0000 7fb137420740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c1ac3a0 con 0x558f1a8bf4a0 2024-05-22T23:10:30.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.057+0000 7fb137420740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c1ad810 con 0x558f1a8c8b30 2024-05-22T23:10:30.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.057+0000 7fb137420740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c1aca70 con 0x558f1a8bddb0 2024-05-22T23:10:30.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.057+0000 7fb137420740 10 client.0 _ll_register_callbacks cb 0x7fb124009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:10:30.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.057+0000 7fb131bdc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8bf4a0 0x558f1a8c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.057+0000 7fb131bdc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8bf4a0 0x558f1a8c9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48498/0 (socket says 172.21.15.84:48498) 2024-05-22T23:10:30.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.057+0000 7fb131bdc640 1 -- 172.21.15.84:0/281727879 learned_addr learned my addr 172.21.15.84:0/281727879 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:10:30.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.057+0000 7fb1323dd640 1 --2- 172.21.15.84:0/281727879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f1a8bddb0 0x558f1a8d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.057+0000 7fb132bde640 1 --2- 172.21.15.84:0/281727879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f1a8c8b30 0x558f1a8ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.057+0000 7fb131bdc640 1 -- 172.21.15.84:0/281727879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f1a8bddb0 msgr2=0x558f1a8d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:30.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.057+0000 7fb131bdc640 1 --2- 172.21.15.84:0/281727879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f1a8bddb0 0x558f1a8d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.057+0000 7fb131bdc640 1 -- 172.21.15.84:0/281727879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f1a8c8b30 msgr2=0x558f1a8ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:30.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.057+0000 7fb131bdc640 1 --2- 172.21.15.84:0/281727879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f1a8c8b30 0x558f1a8ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.057+0000 7fb131bdc640 1 -- 172.21.15.84:0/281727879 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f1a7e0fe0 con 0x558f1a8bf4a0 2024-05-22T23:10:30.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.057+0000 7fb132bde640 1 --2- 172.21.15.84:0/281727879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f1a8c8b30 0x558f1a8ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:10:30.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.057+0000 7fb131bdc640 1 --2- 172.21.15.84:0/281727879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8bf4a0 0x558f1a8c9af0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb11c098040 tx=0x7fb11c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71919b9ad6b9088b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:30.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.057+0000 7fb115ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:10:30.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.058+0000 7fb115ffb640 1 -- 172.21.15.84:0/281727879 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11c0b3070 con 0x558f1a8bf4a0 2024-05-22T23:10:30.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.058+0000 7fb115ffb640 1 -- 172.21.15.84:0/281727879 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7fb11c0ccb20 con 0x558f1a8bf4a0 2024-05-22T23:10:30.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.058+0000 7fb137420740 1 -- 172.21.15.84:0/281727879 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb124050070 con 0x558f1a8bf4a0 2024-05-22T23:10:30.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.058+0000 7fb115ffb640 1 -- 172.21.15.84:0/281727879 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fb11c020040 con 0x558f1a8bf4a0 2024-05-22T23:10:30.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.058+0000 7fb137420740 10 client.16794 fetch_fsmap learned FSMap version 18 2024-05-22T23:10:30.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.058+0000 7fb137420740 1 -- 172.21.15.84:0/281727879 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x558f1a7b21c0 con 0x558f1a8bf4a0 2024-05-22T23:10:30.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.058+0000 7fb115ffb640 1 -- 172.21.15.84:0/281727879 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb124050070 con 0x558f1a8bf4a0 2024-05-22T23:10:30.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.058+0000 7fb115ffb640 1 -- 172.21.15.84:0/281727879 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fb11c097ce0 con 0x558f1a8bf4a0 2024-05-22T23:10:30.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.058+0000 7fb137420740 10 client.16794 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:10:30.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.058+0000 7fb137420740 10 client.16794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:10:30.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.058+0000 7fb137420740 20 client.16794 populate_metadata read hostname 'smithi084' 2024-05-22T23:10:30.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.058+0000 7fb137420740 1 --2- 172.21.15.84:0/281727879 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558f1a93f620 0x558f1a941a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.058+0000 7fb137420740 4 client.16794 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:10:30.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.058+0000 7fb137420740 1 -- 172.21.15.84:0/281727879 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558f1a829830 con 0x558f1a93f620 2024-05-22T23:10:30.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.058+0000 7fb1323dd640 1 --2- 172.21.15.84:0/281727879 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558f1a93f620 0x558f1a941a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.059+0000 7fb115ffb640 10 client.16794 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:10:30.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.059+0000 7fb1323dd640 1 --2- 172.21.15.84:0/281727879 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558f1a93f620 0x558f1a941a10 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6765a4a466a3775b server_cookie=75db1c0fb0b40dc3 in_seq=0 out_seq=0 2024-05-22T23:10:30.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.059+0000 7fb115ffb640 1 -- 172.21.15.84:0/281727879 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558f1a829830 con 0x558f1a93f620 2024-05-22T23:10:30.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.059+0000 7fb115ffb640 10 client.16794 handle_command_reply: tid=0 2024-05-22T23:10:30.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.059+0000 7fb137420740 1 client.16794 shutdown 2024-05-22T23:10:30.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.059+0000 7fb137420740 1 -- 172.21.15.84:0/281727879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8bf4a0 msgr2=0x558f1a8c9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:30.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.059+0000 7fb137420740 1 --2- 172.21.15.84:0/281727879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8bf4a0 0x558f1a8c9af0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb11c098040 tx=0x7fb11c01e000 comp rx=0 tx=0).stop 2024-05-22T23:10:30.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.059+0000 7fb137420740 1 -- 172.21.15.84:0/281727879 shutdown_connections 2024-05-22T23:10:30.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.059+0000 7fb137420740 1 --2- 172.21.15.84:0/281727879 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558f1a93f620 0x558f1a941a10 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.059+0000 7fb137420740 1 --2- 172.21.15.84:0/281727879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f1a8c8b30 0x558f1a8ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.059+0000 7fb137420740 1 --2- 172.21.15.84:0/281727879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f1a8bf4a0 0x558f1a8c9af0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.059+0000 7fb115ffb640 0 client.16794 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:10:30.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.059+0000 7fb137420740 1 --2- 172.21.15.84:0/281727879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f1a8bddb0 0x558f1a8d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.059+0000 7fb137420740 1 -- 172.21.15.84:0/281727879 >> 172.21.15.84:0/281727879 conn(0x558f1a8868e0 msgr2=0x558f1a8b9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:30.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.059+0000 7fb137420740 1 -- 172.21.15.84:0/281727879 shutdown_connections 2024-05-22T23:10:30.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.059+0000 7fb137420740 1 -- 172.21.15.84:0/281727879 wait complete. 2024-05-22T23:10:30.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.059+0000 7fb137420740 20 client.16794 trim_cache size 0 max 16384 2024-05-22T23:10:30.077 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:10:30.077 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "no active scrubs running", 2024-05-22T23:10:30.077 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": {} 2024-05-22T23:10:30.077 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:10:30.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.060+0000 7fb134641640 1 -- 172.21.15.84:0/2015175452 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fb1180a2d00 msgr2=0x7fb1180a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:30.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.060+0000 7fb134641640 1 --2- 172.21.15.84:0/2015175452 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fb1180a2d00 0x7fb1180a51c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb124031550 tx=0x7fb124009260 comp rx=0 tx=0).stop 2024-05-22T23:10:30.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.060+0000 7fb134641640 1 -- 172.21.15.84:0/2015175452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb12c102200 msgr2=0x7fb12c1985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:30.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.060+0000 7fb134641640 1 --2- 172.21.15.84:0/2015175452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb12c102200 0x7fb12c1985a0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fb11c00d950 tx=0x7fb11c00de10 comp rx=0 tx=0).stop 2024-05-22T23:10:30.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.060+0000 7fb134641640 1 -- 172.21.15.84:0/2015175452 shutdown_connections 2024-05-22T23:10:30.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.060+0000 7fb134641640 1 --2- 172.21.15.84:0/2015175452 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fb1180a2d00 0x7fb1180a51c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.060+0000 7fb134641640 1 --2- 172.21.15.84:0/2015175452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb12c107850 0x7fb12c19c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.060+0000 7fb134641640 1 --2- 172.21.15.84:0/2015175452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb12c102200 0x7fb12c1985a0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.060+0000 7fb134641640 1 --2- 172.21.15.84:0/2015175452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb12c101840 0x7fb12c198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.060+0000 7fb134641640 1 -- 172.21.15.84:0/2015175452 >> 172.21.15.84:0/2015175452 conn(0x7fb12c0fd2b0 msgr2=0x7fb12c0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:30.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.060+0000 7fb134641640 1 -- 172.21.15.84:0/2015175452 shutdown_connections 2024-05-22T23:10:30.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.060+0000 7fb134641640 1 -- 172.21.15.84:0/2015175452 wait complete. 2024-05-22T23:10:30.081 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-22T23:10:30.081 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:10:30.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.175+0000 7fb1a1c95640 1 -- 172.21.15.84:0/3333545033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb19c100750 msgr2=0x7fb19c100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:30.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.175+0000 7fb1a1c95640 1 --2- 172.21.15.84:0/3333545033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb19c100750 0x7fb19c100bd0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb184009930 tx=0x7fb18402a420 comp rx=0 tx=0).stop 2024-05-22T23:10:30.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.175+0000 7fb1a1c95640 1 -- 172.21.15.84:0/3333545033 shutdown_connections 2024-05-22T23:10:30.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.175+0000 7fb1a1c95640 1 --2- 172.21.15.84:0/3333545033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb19c101110 0x7fb19c10d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.175+0000 7fb1a1c95640 1 --2- 172.21.15.84:0/3333545033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb19c100750 0x7fb19c100bd0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.175+0000 7fb1a1c95640 1 --2- 172.21.15.84:0/3333545033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb19c106760 0x7fb19c106b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.175+0000 7fb1a1c95640 1 -- 172.21.15.84:0/3333545033 >> 172.21.15.84:0/3333545033 conn(0x7fb19c077870 msgr2=0x7fb19c077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:30.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.175+0000 7fb1a1c95640 1 -- 172.21.15.84:0/3333545033 shutdown_connections 2024-05-22T23:10:30.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.175+0000 7fb1a1c95640 1 -- 172.21.15.84:0/3333545033 wait complete. 2024-05-22T23:10:30.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.175+0000 7fb1a1c95640 1 Processor -- start 2024-05-22T23:10:30.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.175+0000 7fb1a1c95640 1 -- start start 2024-05-22T23:10:30.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.176+0000 7fb1a1c95640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb19c100750 0x7fb19c197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.176+0000 7fb1a1c95640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb19c101110 0x7fb19c197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.176+0000 7fb1a1c95640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb19c106760 0x7fb19c19b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.176+0000 7fb1a1c95640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c110b70 con 0x7fb19c101110 2024-05-22T23:10:30.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.176+0000 7fb1a1c95640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c1109f0 con 0x7fb19c106760 2024-05-22T23:10:30.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.176+0000 7fb1a1c95640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c110cf0 con 0x7fb19c100750 2024-05-22T23:10:30.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.176+0000 7fb19affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb19c101110 0x7fb19c197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.176+0000 7fb19affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb19c101110 0x7fb19c197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48528/0 (socket says 172.21.15.84:48528) 2024-05-22T23:10:30.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.176+0000 7fb19affd640 1 -- 172.21.15.84:0/2958239997 learned_addr learned my addr 172.21.15.84:0/2958239997 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:10:30.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.176+0000 7fb19b7fe640 1 --2- 172.21.15.84:0/2958239997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb19c100750 0x7fb19c197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.176+0000 7fb19affd640 1 -- 172.21.15.84:0/2958239997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb19c100750 msgr2=0x7fb19c197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:30.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.176+0000 7fb19affd640 1 --2- 172.21.15.84:0/2958239997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb19c100750 0x7fb19c197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.176+0000 7fb19affd640 1 -- 172.21.15.84:0/2958239997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb19c106760 msgr2=0x7fb19c19b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:30.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.176+0000 7fb19affd640 1 --2- 172.21.15.84:0/2958239997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb19c106760 0x7fb19c19b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.176+0000 7fb19affd640 1 -- 172.21.15.84:0/2958239997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb19c19c020 con 0x7fb19c101110 2024-05-22T23:10:30.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.176+0000 7fb19b7fe640 1 --2- 172.21.15.84:0/2958239997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb19c100750 0x7fb19c197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:10:30.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.176+0000 7fb19affd640 1 --2- 172.21.15.84:0/2958239997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb19c101110 0x7fb19c197570 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb184009230 tx=0x7fb184004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:30.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.176+0000 7fb198ff9640 1 -- 172.21.15.84:0/2958239997 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18403f030 con 0x7fb19c101110 2024-05-22T23:10:30.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.176+0000 7fb1a1c95640 1 -- 172.21.15.84:0/2958239997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb19c19c2b0 con 0x7fb19c101110 2024-05-22T23:10:30.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.176+0000 7fb198ff9640 1 -- 172.21.15.84:0/2958239997 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb184003ad0 con 0x7fb19c101110 2024-05-22T23:10:30.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.176+0000 7fb198ff9640 1 -- 172.21.15.84:0/2958239997 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184035070 con 0x7fb19c101110 2024-05-22T23:10:30.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.176+0000 7fb1a1c95640 1 -- 172.21.15.84:0/2958239997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb19c1a3ba0 con 0x7fb19c101110 2024-05-22T23:10:30.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.177+0000 7fb1a1c95640 1 -- 172.21.15.84:0/2958239997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb19c068690 con 0x7fb19c101110 2024-05-22T23:10:30.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.178+0000 7fb198ff9640 1 -- 172.21.15.84:0/2958239997 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb184042030 con 0x7fb19c101110 2024-05-22T23:10:30.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.179+0000 7fb198ff9640 1 --2- 172.21.15.84:0/2958239997 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fb1880a2d00 0x7fb1880a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.179+0000 7fb19b7fe640 1 --2- 172.21.15.84:0/2958239997 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fb1880a2d00 0x7fb1880a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.179+0000 7fb198ff9640 1 -- 172.21.15.84:0/2958239997 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7fb1840efdd0 con 0x7fb19c101110 2024-05-22T23:10:30.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.179+0000 7fb19b7fe640 1 --2- 172.21.15.84:0/2958239997 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fb1880a2d00 0x7fb1880a51c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb18c005fd0 tx=0x7fb18c005850 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:30.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.183+0000 7fb198ff9640 1 -- 172.21.15.84:0/2958239997 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb1840bc9c0 con 0x7fb19c101110 2024-05-22T23:10:30.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.399+0000 7fb1a1c95640 1 -- 172.21.15.84:0/2958239997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb19c108750 con 0x7fb19c101110 2024-05-22T23:10:30.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.399+0000 7fb198ff9640 1 -- 172.21.15.84:0/2958239997 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7fb1840bc2c0 con 0x7fb19c101110 2024-05-22T23:10:30.406 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:10:30.407 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:10:30.407 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:10:30.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.401+0000 7fb1a1c95640 1 -- 172.21.15.84:0/2958239997 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fb1880a2d00 msgr2=0x7fb1880a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:30.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.401+0000 7fb1a1c95640 1 --2- 172.21.15.84:0/2958239997 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fb1880a2d00 0x7fb1880a51c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb18c005fd0 tx=0x7fb18c005850 comp rx=0 tx=0).stop 2024-05-22T23:10:30.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.401+0000 7fb1a1c95640 1 -- 172.21.15.84:0/2958239997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb19c101110 msgr2=0x7fb19c197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:30.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.402+0000 7fb1a1c95640 1 --2- 172.21.15.84:0/2958239997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb19c101110 0x7fb19c197570 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb184009230 tx=0x7fb184004270 comp rx=0 tx=0).stop 2024-05-22T23:10:30.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.402+0000 7fb1a1c95640 1 -- 172.21.15.84:0/2958239997 shutdown_connections 2024-05-22T23:10:30.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.402+0000 7fb1a1c95640 1 --2- 172.21.15.84:0/2958239997 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fb1880a2d00 0x7fb1880a51c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.402+0000 7fb1a1c95640 1 --2- 172.21.15.84:0/2958239997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb19c106760 0x7fb19c19b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.402+0000 7fb1a1c95640 1 --2- 172.21.15.84:0/2958239997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb19c101110 0x7fb19c197570 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.402+0000 7fb1a1c95640 1 --2- 172.21.15.84:0/2958239997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb19c100750 0x7fb19c197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.402+0000 7fb1a1c95640 1 -- 172.21.15.84:0/2958239997 >> 172.21.15.84:0/2958239997 conn(0x7fb19c077870 msgr2=0x7fb19c103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:30.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.402+0000 7fb1a1c95640 1 -- 172.21.15.84:0/2958239997 shutdown_connections 2024-05-22T23:10:30.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.402+0000 7fb1a1c95640 1 -- 172.21.15.84:0/2958239997 wait complete. 2024-05-22T23:10:30.420 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-22T23:10:30.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.516+0000 7f3768b32640 1 -- 172.21.15.84:0/830183390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3764107850 msgr2=0x7f3764107c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:30.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.516+0000 7f3768b32640 1 --2- 172.21.15.84:0/830183390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3764107850 0x7f3764107c30 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f375c009930 tx=0x7f375c02a420 comp rx=0 tx=0).stop 2024-05-22T23:10:30.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.516+0000 7f3768b32640 1 -- 172.21.15.84:0/830183390 shutdown_connections 2024-05-22T23:10:30.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.516+0000 7f3768b32640 1 --2- 172.21.15.84:0/830183390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3764102200 0x7f376410e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.516+0000 7f3768b32640 1 --2- 172.21.15.84:0/830183390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3764101840 0x7f3764101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.516+0000 7f3768b32640 1 --2- 172.21.15.84:0/830183390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3764107850 0x7f3764107c30 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.516+0000 7f3768b32640 1 -- 172.21.15.84:0/830183390 >> 172.21.15.84:0/830183390 conn(0x7f3764078cc0 msgr2=0x7f37640790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:30.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.516+0000 7f3768b32640 1 -- 172.21.15.84:0/830183390 shutdown_connections 2024-05-22T23:10:30.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.516+0000 7f3768b32640 1 -- 172.21.15.84:0/830183390 wait complete. 2024-05-22T23:10:30.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.517+0000 7f3768b32640 1 Processor -- start 2024-05-22T23:10:30.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.517+0000 7f3768b32640 1 -- start start 2024-05-22T23:10:30.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.517+0000 7f3768b32640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3764101840 0x7f3764198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.517+0000 7f3768b32640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3764102200 0x7f3764198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.517+0000 7f3768b32640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3764107850 0x7f376419c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.517+0000 7f3768b32640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3764111c20 con 0x7f3764107850 2024-05-22T23:10:30.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.517+0000 7f3768b32640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3764111aa0 con 0x7f3764101840 2024-05-22T23:10:30.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.517+0000 7f3768b32640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3764111da0 con 0x7f3764102200 2024-05-22T23:10:30.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.517+0000 7f3762d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3764107850 0x7f376419c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.517+0000 7f3762d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3764107850 0x7f376419c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48536/0 (socket says 172.21.15.84:48536) 2024-05-22T23:10:30.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.517+0000 7f3762d9d640 1 -- 172.21.15.84:0/699782690 learned_addr learned my addr 172.21.15.84:0/699782690 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:10:30.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.517+0000 7f376259c640 1 --2- 172.21.15.84:0/699782690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3764101840 0x7f3764198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.517+0000 7f3761d9b640 1 --2- 172.21.15.84:0/699782690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3764102200 0x7f3764198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.517+0000 7f3762d9d640 1 -- 172.21.15.84:0/699782690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3764102200 msgr2=0x7f3764198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:30.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.517+0000 7f3762d9d640 1 --2- 172.21.15.84:0/699782690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3764102200 0x7f3764198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.517+0000 7f3762d9d640 1 -- 172.21.15.84:0/699782690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3764101840 msgr2=0x7f3764198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:30.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.517+0000 7f3762d9d640 1 --2- 172.21.15.84:0/699782690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3764101840 0x7f3764198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.517+0000 7f3762d9d640 1 -- 172.21.15.84:0/699782690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376419d110 con 0x7f3764107850 2024-05-22T23:10:30.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.517+0000 7f376259c640 1 --2- 172.21.15.84:0/699782690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3764101840 0x7f3764198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:10:30.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.518+0000 7f3762d9d640 1 --2- 172.21.15.84:0/699782690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3764107850 0x7f376419c9e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f375400c970 tx=0x7f375400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:30.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.518+0000 7f37537fe640 1 -- 172.21.15.84:0/699782690 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754007bb0 con 0x7f3764107850 2024-05-22T23:10:30.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.518+0000 7f37537fe640 1 -- 172.21.15.84:0/699782690 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3754007d50 con 0x7f3764107850 2024-05-22T23:10:30.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.518+0000 7f37537fe640 1 -- 172.21.15.84:0/699782690 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754018800 con 0x7f3764107850 2024-05-22T23:10:30.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.518+0000 7f3768b32640 1 -- 172.21.15.84:0/699782690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376419d400 con 0x7f3764107850 2024-05-22T23:10:30.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.518+0000 7f3768b32640 1 -- 172.21.15.84:0/699782690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37641a4c90 con 0x7f3764107850 2024-05-22T23:10:30.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.520+0000 7f37537fe640 1 -- 172.21.15.84:0/699782690 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f375400fa50 con 0x7f3764107850 2024-05-22T23:10:30.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.520+0000 7f376b83d740 1 Processor -- start 2024-05-22T23:10:30.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.521+0000 7f376b83d740 1 -- start start 2024-05-22T23:10:30.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.521+0000 7f37537fe640 1 --2- 172.21.15.84:0/699782690 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f374c0a2d00 0x7f374c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.521+0000 7f376b83d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561117c9e80 0x5561117ca260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.521+0000 7f376b83d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117ca7a0 0x5561117d8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.521+0000 7f376b83d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561117d9380 0x5561117db770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.521+0000 7f376b83d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55611172edb0 con 0x5561117ca7a0 2024-05-22T23:10:30.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.521+0000 7f376b83d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5561116cd400 con 0x5561117d9380 2024-05-22T23:10:30.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.521+0000 7f376b83d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55611172ad70 con 0x5561117c9e80 2024-05-22T23:10:30.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.521+0000 7f37537fe640 1 -- 172.21.15.84:0/699782690 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7f37540cf510 con 0x7f3764107850 2024-05-22T23:10:30.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.521+0000 7f3761d9b640 1 --2- 172.21.15.84:0/699782690 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f374c0a2d00 0x7f374c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.521+0000 7f3761d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117ca7a0 0x5561117d8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.521+0000 7f3761d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117ca7a0 0x5561117d8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48540/0 (socket says 172.21.15.84:48540) 2024-05-22T23:10:30.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.521+0000 7f3761d9b640 1 -- 172.21.15.84:0/940369612 learned_addr learned my addr 172.21.15.84:0/940369612 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:10:30.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.521+0000 7f3762d9d640 1 --2- 172.21.15.84:0/940369612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561117d9380 0x5561117db770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.521+0000 7f376259c640 1 --2- 172.21.15.84:0/940369612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561117c9e80 0x5561117ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.521+0000 7f3761d9b640 1 -- 172.21.15.84:0/940369612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561117c9e80 msgr2=0x5561117ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:30.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.521+0000 7f3761d9b640 1 --2- 172.21.15.84:0/940369612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561117c9e80 0x5561117ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.521+0000 7f3761d9b640 1 -- 172.21.15.84:0/940369612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561117d9380 msgr2=0x5561117db770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:30.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.521+0000 7f3761d9b640 1 --2- 172.21.15.84:0/940369612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561117d9380 0x5561117db770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.521+0000 7f3761d9b640 1 -- 172.21.15.84:0/940369612 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561116bf2f0 con 0x5561117ca7a0 2024-05-22T23:10:30.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.521+0000 7f3762d9d640 1 --2- 172.21.15.84:0/940369612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561117d9380 0x5561117db770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:10:30.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.521+0000 7f3761d9b640 1 --2- 172.21.15.84:0/699782690 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f374c0a2d00 0x7f374c0a51c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f375800afd0 tx=0x7f375800ad60 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:30.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.521+0000 7f3761d9b640 1 --2- 172.21.15.84:0/940369612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117ca7a0 0x5561117d8e40 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f375800fcc0 tx=0x7f3758019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3727643599ab13d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:30.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.522+0000 7f3750ff9640 1 -- 172.21.15.84:0/940369612 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3758022070 con 0x5561117ca7a0 2024-05-22T23:10:30.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.522+0000 7f3750ff9640 1 -- 172.21.15.84:0/940369612 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f375800dde0 con 0x5561117ca7a0 2024-05-22T23:10:30.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.522+0000 7f3750ff9640 1 -- 172.21.15.84:0/940369612 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3758010610 con 0x5561117ca7a0 2024-05-22T23:10:30.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.522+0000 7f376b83d740 1 -- 172.21.15.84:0/940369612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117ca7a0 msgr2=0x5561117d8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:30.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.522+0000 7f376b83d740 1 --2- 172.21.15.84:0/940369612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117ca7a0 0x5561117d8e40 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f375800fcc0 tx=0x7f3758019000 comp rx=0 tx=0).stop 2024-05-22T23:10:30.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.522+0000 7f376b83d740 1 -- 172.21.15.84:0/940369612 shutdown_connections 2024-05-22T23:10:30.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.522+0000 7f376b83d740 1 --2- 172.21.15.84:0/940369612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561117d9380 0x5561117db770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.522+0000 7f376b83d740 1 --2- 172.21.15.84:0/940369612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117ca7a0 0x5561117d8e40 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.522+0000 7f376b83d740 1 --2- 172.21.15.84:0/940369612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561117c9e80 0x5561117ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.522+0000 7f376b83d740 1 -- 172.21.15.84:0/940369612 >> 172.21.15.84:0/940369612 conn(0x5561117c62a0 msgr2=0x5561117c6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:30.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.522+0000 7f376b83d740 1 -- 172.21.15.84:0/940369612 shutdown_connections 2024-05-22T23:10:30.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.522+0000 7f376b83d740 1 -- 172.21.15.84:0/940369612 wait complete. 2024-05-22T23:10:30.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.523+0000 7f376b83d740 1 -- start start 2024-05-22T23:10:30.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.523+0000 7f376b83d740 1 Processor -- start 2024-05-22T23:10:30.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.523+0000 7f376b83d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117c9e80 0x5561117d62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.523+0000 7f376b83d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561117ca7a0 0x5561117d6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.523+0000 7f376b83d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561117d9380 0x5561117d6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.523+0000 7f376b83d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5561117dc0e0 con 0x5561117c9e80 2024-05-22T23:10:30.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.523+0000 7f376b83d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5561116e92e0 con 0x5561117ca7a0 2024-05-22T23:10:30.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.523+0000 7f376b83d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5561117dc260 con 0x5561117d9380 2024-05-22T23:10:30.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.523+0000 7f376b83d740 10 client.0 _ll_register_callbacks cb 0x5561116f2a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:10:30.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.523+0000 7f376259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117c9e80 0x5561117d62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.523+0000 7f376259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117c9e80 0x5561117d62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48552/0 (socket says 172.21.15.84:48552) 2024-05-22T23:10:30.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.523+0000 7f376259c640 1 -- 172.21.15.84:0/791221310 learned_addr learned my addr 172.21.15.84:0/791221310 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:10:30.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.523+0000 7f3761d9b640 1 --2- 172.21.15.84:0/791221310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561117ca7a0 0x5561117d6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.523+0000 7f376259c640 1 -- 172.21.15.84:0/791221310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561117d9380 msgr2=0x5561117d6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:30.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.523+0000 7f376259c640 1 --2- 172.21.15.84:0/791221310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561117d9380 0x5561117d6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.523+0000 7f376259c640 1 -- 172.21.15.84:0/791221310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561117ca7a0 msgr2=0x5561117d6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:30.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.523+0000 7f376259c640 1 --2- 172.21.15.84:0/791221310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561117ca7a0 0x5561117d6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.523+0000 7f376259c640 1 -- 172.21.15.84:0/791221310 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561114423f0 con 0x5561117c9e80 2024-05-22T23:10:30.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.523+0000 7f376259c640 1 --2- 172.21.15.84:0/791221310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117c9e80 0x5561117d62e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f375c033000 tx=0x7f375c031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d51844b104ba61 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:30.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.524+0000 7f3749ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:10:30.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.524+0000 7f3749ffb640 1 -- 172.21.15.84:0/791221310 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c035030 con 0x5561117c9e80 2024-05-22T23:10:30.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.524+0000 7f3749ffb640 1 -- 172.21.15.84:0/791221310 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7f375c004460 con 0x5561117c9e80 2024-05-22T23:10:30.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.524+0000 7f376b83d740 1 -- 172.21.15.84:0/791221310 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f375800dde0 con 0x5561117c9e80 2024-05-22T23:10:30.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.524+0000 7f3749ffb640 1 -- 172.21.15.84:0/791221310 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f375c005030 con 0x5561117c9e80 2024-05-22T23:10:30.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.524+0000 7f376b83d740 10 client.16812 fetch_fsmap learned FSMap version 18 2024-05-22T23:10:30.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.524+0000 7f376b83d740 1 -- 172.21.15.84:0/791221310 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55611150c7d0 con 0x5561117c9e80 2024-05-22T23:10:30.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.524+0000 7f3749ffb640 1 -- 172.21.15.84:0/791221310 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f375800dde0 con 0x5561117c9e80 2024-05-22T23:10:30.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.524+0000 7f3749ffb640 1 -- 172.21.15.84:0/791221310 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f375c0035f0 con 0x5561117c9e80 2024-05-22T23:10:30.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.524+0000 7f376b83d740 10 client.16812 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:10:30.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.524+0000 7f376b83d740 10 client.16812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:10:30.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.524+0000 7f376b83d740 20 client.16812 populate_metadata read hostname 'smithi084' 2024-05-22T23:10:30.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.524+0000 7f376b83d740 1 --2- 172.21.15.84:0/791221310 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5561117e9680 0x5561117ebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.524+0000 7f376b83d740 4 client.16812 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:10:30.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.524+0000 7f376b83d740 1 -- 172.21.15.84:0/791221310 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55611173c4e0 con 0x5561117e9680 2024-05-22T23:10:30.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.524+0000 7f3761d9b640 1 --2- 172.21.15.84:0/791221310 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5561117e9680 0x5561117ebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.525+0000 7f3761d9b640 1 --2- 172.21.15.84:0/791221310 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5561117e9680 0x5561117ebaf0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=745ac024128fb600 server_cookie=37de2d23d7533503 in_seq=0 out_seq=0 2024-05-22T23:10:30.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.525+0000 7f3749ffb640 10 client.16812 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:10:30.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.526+0000 7f3749ffb640 1 -- 172.21.15.84:0/791221310 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55611173c4e0 con 0x5561117e9680 2024-05-22T23:10:30.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.526+0000 7f3749ffb640 10 client.16812 handle_command_reply: tid=0 2024-05-22T23:10:30.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.526+0000 7f376b83d740 1 client.16812 shutdown 2024-05-22T23:10:30.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.526+0000 7f376b83d740 1 -- 172.21.15.84:0/791221310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117c9e80 msgr2=0x5561117d62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:30.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.526+0000 7f376b83d740 1 --2- 172.21.15.84:0/791221310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117c9e80 0x5561117d62e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f375c033000 tx=0x7f375c031920 comp rx=0 tx=0).stop 2024-05-22T23:10:30.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.526+0000 7f376b83d740 1 -- 172.21.15.84:0/791221310 shutdown_connections 2024-05-22T23:10:30.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.526+0000 7f376b83d740 1 --2- 172.21.15.84:0/791221310 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5561117e9680 0x5561117ebaf0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.526+0000 7f376b83d740 1 --2- 172.21.15.84:0/791221310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561117d9380 0x5561117d6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.526+0000 7f376b83d740 1 --2- 172.21.15.84:0/791221310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561117ca7a0 0x5561117d6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.526+0000 7f3749ffb640 0 client.16812 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:10:30.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.526+0000 7f376b83d740 1 --2- 172.21.15.84:0/791221310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117c9e80 0x5561117d62e0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.526+0000 7f376b83d740 1 -- 172.21.15.84:0/791221310 >> 172.21.15.84:0/791221310 conn(0x5561117c62a0 msgr2=0x5561117cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:30.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.526+0000 7f376b83d740 1 -- 172.21.15.84:0/791221310 shutdown_connections 2024-05-22T23:10:30.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.526+0000 7f376b83d740 1 -- 172.21.15.84:0/791221310 wait complete. 2024-05-22T23:10:30.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.526+0000 7f376b83d740 20 client.16812 trim_cache size 0 max 16384 2024-05-22T23:10:30.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.539+0000 7f376b83d740 1 Processor -- start 2024-05-22T23:10:30.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.539+0000 7f376b83d740 1 -- start start 2024-05-22T23:10:30.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.539+0000 7f376b83d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561117c9e80 0x5561117d8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.539+0000 7f376b83d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561117ca7a0 0x5561117e5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.539+0000 7f376b83d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117d9380 0x5561117c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.539+0000 7f376b83d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f37641a93a0 con 0x5561117d9380 2024-05-22T23:10:30.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.539+0000 7f376b83d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f37641abc00 con 0x5561117c9e80 2024-05-22T23:10:30.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.539+0000 7f376b83d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f37641a99f0 con 0x5561117ca7a0 2024-05-22T23:10:30.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.539+0000 7f3762d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117d9380 0x5561117c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.539+0000 7f3762d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117d9380 0x5561117c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48564/0 (socket says 172.21.15.84:48564) 2024-05-22T23:10:30.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.539+0000 7f3762d9d640 1 -- 172.21.15.84:0/3346783957 learned_addr learned my addr 172.21.15.84:0/3346783957 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:10:30.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.539+0000 7f3762d9d640 1 -- 172.21.15.84:0/3346783957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561117ca7a0 msgr2=0x5561117e5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:30.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.539+0000 7f3762d9d640 1 --2- 172.21.15.84:0/3346783957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561117ca7a0 0x5561117e5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.539+0000 7f376259c640 1 --2- 172.21.15.84:0/3346783957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561117c9e80 0x5561117d8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.539+0000 7f3762d9d640 1 -- 172.21.15.84:0/3346783957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561117c9e80 msgr2=0x5561117d8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:30.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.539+0000 7f3762d9d640 1 --2- 172.21.15.84:0/3346783957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561117c9e80 0x5561117d8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.539+0000 7f3762d9d640 1 -- 172.21.15.84:0/3346783957 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561117c08d0 con 0x5561117d9380 2024-05-22T23:10:30.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.539+0000 7f3762d9d640 1 --2- 172.21.15.84:0/3346783957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117d9380 0x5561117c5340 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f375409b990 tx=0x7f3754097ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b9b6e49e49949c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:30.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.540+0000 7f374bfff640 1 -- 172.21.15.84:0/3346783957 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375409fdb0 con 0x5561117d9380 2024-05-22T23:10:30.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.540+0000 7f374bfff640 1 -- 172.21.15.84:0/3346783957 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37540a4670 con 0x5561117d9380 2024-05-22T23:10:30.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.540+0000 7f374bfff640 1 -- 172.21.15.84:0/3346783957 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37540cf070 con 0x5561117d9380 2024-05-22T23:10:30.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.540+0000 7f376b83d740 1 -- 172.21.15.84:0/3346783957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117d9380 msgr2=0x5561117c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:30.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.540+0000 7f376b83d740 1 --2- 172.21.15.84:0/3346783957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117d9380 0x5561117c5340 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f375409b990 tx=0x7f3754097ca0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.540+0000 7f376b83d740 1 -- 172.21.15.84:0/3346783957 shutdown_connections 2024-05-22T23:10:30.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.540+0000 7f376b83d740 1 --2- 172.21.15.84:0/3346783957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117d9380 0x5561117c5340 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.540+0000 7f376b83d740 1 --2- 172.21.15.84:0/3346783957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561117ca7a0 0x5561117e5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.540+0000 7f376b83d740 1 --2- 172.21.15.84:0/3346783957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561117c9e80 0x5561117d8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.540+0000 7f376b83d740 1 -- 172.21.15.84:0/3346783957 >> 172.21.15.84:0/3346783957 conn(0x5561117c62a0 msgr2=0x5561117c4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:30.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.540+0000 7f376b83d740 1 -- 172.21.15.84:0/3346783957 shutdown_connections 2024-05-22T23:10:30.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.540+0000 7f376b83d740 1 -- 172.21.15.84:0/3346783957 wait complete. 2024-05-22T23:10:30.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.540+0000 7f376b83d740 1 -- start start 2024-05-22T23:10:30.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.541+0000 7f376b83d740 1 Processor -- start 2024-05-22T23:10:30.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.541+0000 7f376b83d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117c9e80 0x5561117dd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.541+0000 7f376b83d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561117ca7a0 0x5561117dd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.541+0000 7f376b83d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561117d9380 0x5561117ddaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.541+0000 7f376b83d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f37641aa0c0 con 0x5561117c9e80 2024-05-22T23:10:30.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.541+0000 7f376b83d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f37641ab530 con 0x5561117ca7a0 2024-05-22T23:10:30.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.541+0000 7f376b83d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f37641aa790 con 0x5561117d9380 2024-05-22T23:10:30.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.541+0000 7f376b83d740 10 client.0 _ll_register_callbacks cb 0x7f375c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:10:30.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.541+0000 7f376259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117c9e80 0x5561117dd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.541+0000 7f376259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117c9e80 0x5561117dd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48578/0 (socket says 172.21.15.84:48578) 2024-05-22T23:10:30.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.541+0000 7f376259c640 1 -- 172.21.15.84:0/1792063697 learned_addr learned my addr 172.21.15.84:0/1792063697 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:10:30.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.541+0000 7f376259c640 1 -- 172.21.15.84:0/1792063697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561117d9380 msgr2=0x5561117ddaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:30.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.541+0000 7f376259c640 1 --2- 172.21.15.84:0/1792063697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561117d9380 0x5561117ddaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.541+0000 7f376259c640 1 -- 172.21.15.84:0/1792063697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561117ca7a0 msgr2=0x5561117dd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:30.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.541+0000 7f376259c640 1 --2- 172.21.15.84:0/1792063697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561117ca7a0 0x5561117dd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.541+0000 7f376259c640 1 -- 172.21.15.84:0/1792063697 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55611184ed30 con 0x5561117c9e80 2024-05-22T23:10:30.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.541+0000 7f376259c640 1 --2- 172.21.15.84:0/1792063697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117c9e80 0x5561117dd070 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f375c009850 tx=0x7f375c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1751adc389b1acf server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:30.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.541+0000 7f3749ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:10:30.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.541+0000 7f3749ffb640 1 -- 172.21.15.84:0/1792063697 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c058060 con 0x5561117c9e80 2024-05-22T23:10:30.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.541+0000 7f3749ffb640 1 -- 172.21.15.84:0/1792063697 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7f375c004460 con 0x5561117c9e80 2024-05-22T23:10:30.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.541+0000 7f376b83d740 1 -- 172.21.15.84:0/1792063697 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37540a4670 con 0x5561117c9e80 2024-05-22T23:10:30.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.542+0000 7f3749ffb640 1 -- 172.21.15.84:0/1792063697 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f375c04f040 con 0x5561117c9e80 2024-05-22T23:10:30.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.542+0000 7f376b83d740 10 client.16818 fetch_fsmap learned FSMap version 18 2024-05-22T23:10:30.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.542+0000 7f376b83d740 1 -- 172.21.15.84:0/1792063697 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x556111850030 con 0x5561117c9e80 2024-05-22T23:10:30.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.542+0000 7f3749ffb640 1 -- 172.21.15.84:0/1792063697 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37540a4670 con 0x5561117c9e80 2024-05-22T23:10:30.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.542+0000 7f3749ffb640 1 -- 172.21.15.84:0/1792063697 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f375c004c90 con 0x5561117c9e80 2024-05-22T23:10:30.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.542+0000 7f376b83d740 10 client.16818 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:10:30.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.542+0000 7f376b83d740 10 client.16818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:10:30.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.542+0000 7f376b83d740 20 client.16818 populate_metadata read hostname 'smithi084' 2024-05-22T23:10:30.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.542+0000 7f376b83d740 1 --2- 172.21.15.84:0/1792063697 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5561117e9680 0x556111852810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:30.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.542+0000 7f376b83d740 4 client.16818 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-22T23:10:30.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.542+0000 7f376b83d740 1 -- 172.21.15.84:0/1792063697 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556111736830 con 0x5561117e9680 2024-05-22T23:10:30.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.542+0000 7f3761d9b640 1 --2- 172.21.15.84:0/1792063697 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5561117e9680 0x556111852810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:30.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.542+0000 7f3749ffb640 10 client.16818 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:10:30.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.542+0000 7f3761d9b640 1 --2- 172.21.15.84:0/1792063697 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5561117e9680 0x556111852810 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd7745964d19f722 server_cookie=3152ed4db167f6cb in_seq=0 out_seq=0 2024-05-22T23:10:30.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.543+0000 7f3749ffb640 1 -- 172.21.15.84:0/1792063697 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556111736830 con 0x5561117e9680 2024-05-22T23:10:30.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.543+0000 7f3749ffb640 10 client.16818 handle_command_reply: tid=0 2024-05-22T23:10:30.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.543+0000 7f376b83d740 1 client.16818 shutdown 2024-05-22T23:10:30.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.543+0000 7f376b83d740 1 -- 172.21.15.84:0/1792063697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117c9e80 msgr2=0x5561117dd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:30.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.543+0000 7f376b83d740 1 --2- 172.21.15.84:0/1792063697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117c9e80 0x5561117dd070 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f375c009850 tx=0x7f375c005fd0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.543+0000 7f376b83d740 1 -- 172.21.15.84:0/1792063697 shutdown_connections 2024-05-22T23:10:30.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.543+0000 7f376b83d740 1 --2- 172.21.15.84:0/1792063697 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5561117e9680 0x556111852810 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.543+0000 7f376b83d740 1 --2- 172.21.15.84:0/1792063697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561117d9380 0x5561117ddaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.543+0000 7f3749ffb640 0 client.16818 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:10:30.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.543+0000 7f376b83d740 1 --2- 172.21.15.84:0/1792063697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561117ca7a0 0x5561117dd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.543+0000 7f376b83d740 1 --2- 172.21.15.84:0/1792063697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561117c9e80 0x5561117dd070 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.543+0000 7f376b83d740 1 -- 172.21.15.84:0/1792063697 >> 172.21.15.84:0/1792063697 conn(0x5561117c62a0 msgr2=0x5561117d6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:30.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.543+0000 7f376b83d740 1 -- 172.21.15.84:0/1792063697 shutdown_connections 2024-05-22T23:10:30.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.543+0000 7f376b83d740 1 -- 172.21.15.84:0/1792063697 wait complete. 2024-05-22T23:10:30.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.543+0000 7f376b83d740 20 client.16818 trim_cache size 0 max 16384 2024-05-22T23:10:30.563 INFO:teuthology.orchestra.run.smithi084.stdout:[] 2024-05-22T23:10:30.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.543+0000 7f3768b32640 1 -- 172.21.15.84:0/699782690 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f374c0a2d00 msgr2=0x7f374c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:30.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.543+0000 7f3768b32640 1 --2- 172.21.15.84:0/699782690 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f374c0a2d00 0x7f374c0a51c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f375800afd0 tx=0x7f375800ad60 comp rx=0 tx=0).stop 2024-05-22T23:10:30.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.543+0000 7f3768b32640 1 -- 172.21.15.84:0/699782690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3764107850 msgr2=0x7f376419c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:30.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.543+0000 7f3768b32640 1 --2- 172.21.15.84:0/699782690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3764107850 0x7f376419c9e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f375400c970 tx=0x7f375400ce30 comp rx=0 tx=0).stop 2024-05-22T23:10:30.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.543+0000 7f3768b32640 1 -- 172.21.15.84:0/699782690 shutdown_connections 2024-05-22T23:10:30.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.543+0000 7f3768b32640 1 --2- 172.21.15.84:0/699782690 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f374c0a2d00 0x7f374c0a51c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.543+0000 7f3768b32640 1 --2- 172.21.15.84:0/699782690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3764107850 0x7f376419c9e0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.543+0000 7f3768b32640 1 --2- 172.21.15.84:0/699782690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3764102200 0x7f3764198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.543+0000 7f3768b32640 1 --2- 172.21.15.84:0/699782690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3764101840 0x7f3764198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:30.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.544+0000 7f3768b32640 1 -- 172.21.15.84:0/699782690 >> 172.21.15.84:0/699782690 conn(0x7f3764078cc0 msgr2=0x7f37640ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:30.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.544+0000 7f3768b32640 1 -- 172.21.15.84:0/699782690 shutdown_connections 2024-05-22T23:10:30.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:30.544+0000 7f3768b32640 1 -- 172.21.15.84:0/699782690 wait complete. 2024-05-22T23:10:31.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:31 smithi096 ceph-mon[36392]: pgmap v269: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.4 KiB/s wr, 2 op/s 2024-05-22T23:10:31.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:31 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2958239997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:10:31.567 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-22T23:10:31.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:10:31.568 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-22T23:10:31.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:31 smithi084 ceph-mon[32186]: pgmap v269: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.4 KiB/s wr, 2 op/s 2024-05-22T23:10:31.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:31 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2958239997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:10:31.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:31 smithi204 ceph-mon[36211]: pgmap v269: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.4 KiB/s wr, 2 op/s 2024-05-22T23:10:31.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:31 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2958239997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:10:31.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.680+0000 7f7800c38640 1 -- 172.21.15.84:0/3434987853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77fc107860 msgr2=0x7f77fc107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:31.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.680+0000 7f7800c38640 1 --2- 172.21.15.84:0/3434987853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77fc107860 0x7f77fc107c40 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009930 tx=0x7f77ec02a420 comp rx=0 tx=0).stop 2024-05-22T23:10:31.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.680+0000 7f7800c38640 1 -- 172.21.15.84:0/3434987853 shutdown_connections 2024-05-22T23:10:31.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.680+0000 7f7800c38640 1 --2- 172.21.15.84:0/3434987853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77fc06a8b0 0x7f77fc10e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.680+0000 7f7800c38640 1 --2- 172.21.15.84:0/3434987853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77fc069ef0 0x7f77fc06a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.680+0000 7f7800c38640 1 --2- 172.21.15.84:0/3434987853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77fc107860 0x7f77fc107c40 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.680+0000 7f7800c38640 1 -- 172.21.15.84:0/3434987853 >> 172.21.15.84:0/3434987853 conn(0x7f77fc0777d0 msgr2=0x7f77fc077be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:31.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.680+0000 7f7800c38640 1 -- 172.21.15.84:0/3434987853 shutdown_connections 2024-05-22T23:10:31.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.680+0000 7f7800c38640 1 -- 172.21.15.84:0/3434987853 wait complete. 2024-05-22T23:10:31.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.681+0000 7f7800c38640 1 Processor -- start 2024-05-22T23:10:31.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.681+0000 7f7800c38640 1 -- start start 2024-05-22T23:10:31.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.681+0000 7f7800c38640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77fc069ef0 0x7f77fc1007a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.681+0000 7f7800c38640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77fc06a8b0 0x7f77fc100ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.681+0000 7f7800c38640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77fc107860 0x7f77fc101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.681+0000 7f7800c38640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc111c70 con 0x7f77fc069ef0 2024-05-22T23:10:31.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.681+0000 7f7800c38640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc111af0 con 0x7f77fc107860 2024-05-22T23:10:31.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.681+0000 7f7800c38640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc111df0 con 0x7f77fc06a8b0 2024-05-22T23:10:31.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.681+0000 7f77fa59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77fc069ef0 0x7f77fc1007a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.681+0000 7f77fa59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77fc069ef0 0x7f77fc1007a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54894/0 (socket says 172.21.15.84:54894) 2024-05-22T23:10:31.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.681+0000 7f77fa59c640 1 -- 172.21.15.84:0/670885231 learned_addr learned my addr 172.21.15.84:0/670885231 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:10:31.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.681+0000 7f77fad9d640 1 --2- 172.21.15.84:0/670885231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77fc107860 0x7f77fc101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.682+0000 7f77f1d9b640 1 --2- 172.21.15.84:0/670885231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77fc06a8b0 0x7f77fc100ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.682+0000 7f77fa59c640 1 -- 172.21.15.84:0/670885231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77fc06a8b0 msgr2=0x7f77fc100ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:31.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.682+0000 7f77fa59c640 1 --2- 172.21.15.84:0/670885231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77fc06a8b0 0x7f77fc100ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.682+0000 7f77fa59c640 1 -- 172.21.15.84:0/670885231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77fc107860 msgr2=0x7f77fc101220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:31.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.682+0000 7f77fa59c640 1 --2- 172.21.15.84:0/670885231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77fc107860 0x7f77fc101220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.682+0000 7f77fa59c640 1 -- 172.21.15.84:0/670885231 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77fc1a6610 con 0x7f77fc069ef0 2024-05-22T23:10:31.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.682+0000 7f77fa59c640 1 --2- 172.21.15.84:0/670885231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77fc069ef0 0x7f77fc1007a0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009230 tx=0x7f77ec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:31.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.682+0000 7f77f3fff640 1 -- 172.21.15.84:0/670885231 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec03f030 con 0x7f77fc069ef0 2024-05-22T23:10:31.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.682+0000 7f7800c38640 1 -- 172.21.15.84:0/670885231 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77fc1a68a0 con 0x7f77fc069ef0 2024-05-22T23:10:31.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.682+0000 7f77f3fff640 1 -- 172.21.15.84:0/670885231 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77ec003ad0 con 0x7f77fc069ef0 2024-05-22T23:10:31.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.682+0000 7f7800c38640 1 -- 172.21.15.84:0/670885231 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77fc1a6d90 con 0x7f77fc069ef0 2024-05-22T23:10:31.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.682+0000 7f77f3fff640 1 -- 172.21.15.84:0/670885231 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec049060 con 0x7f77fc069ef0 2024-05-22T23:10:31.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.684+0000 7f77f3fff640 1 -- 172.21.15.84:0/670885231 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f77ec042030 con 0x7f77fc069ef0 2024-05-22T23:10:31.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.685+0000 7f7803a24740 1 Processor -- start 2024-05-22T23:10:31.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.685+0000 7f7803a24740 1 -- start start 2024-05-22T23:10:31.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.685+0000 7f7803a24740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563df0e27c50 0x563df0e28030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.685+0000 7f7803a24740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e28570 0x563df0e32700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.685+0000 7f7803a24740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563df0e32c40 0x563df0e35030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.685+0000 7f7803a24740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563df0d8ba40 con 0x563df0e28570 2024-05-22T23:10:31.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.685+0000 7f7803a24740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563df0d2a440 con 0x563df0e32c40 2024-05-22T23:10:31.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.685+0000 7f7803a24740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563df0d87a00 con 0x563df0e27c50 2024-05-22T23:10:31.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.685+0000 7f77fa59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e28570 0x563df0e32700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.685+0000 7f77fa59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e28570 0x563df0e32700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54902/0 (socket says 172.21.15.84:54902) 2024-05-22T23:10:31.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.685+0000 7f77fa59c640 1 -- 172.21.15.84:0/1188669557 learned_addr learned my addr 172.21.15.84:0/1188669557 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:10:31.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.685+0000 7f77f1d9b640 1 --2- 172.21.15.84:0/1188669557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563df0e32c40 0x563df0e35030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.685+0000 7f77fad9d640 1 --2- 172.21.15.84:0/1188669557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563df0e27c50 0x563df0e28030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.685+0000 7f77f3fff640 1 --2- 172.21.15.84:0/670885231 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f77e80a2d30 0x7f77e80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.685+0000 7f77fa59c640 1 -- 172.21.15.84:0/1188669557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563df0e27c50 msgr2=0x563df0e28030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:31.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.686+0000 7f77fa59c640 1 --2- 172.21.15.84:0/1188669557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563df0e27c50 0x563df0e28030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.686+0000 7f77fa59c640 1 -- 172.21.15.84:0/1188669557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563df0e32c40 msgr2=0x563df0e35030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:31.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.686+0000 7f77fa59c640 1 --2- 172.21.15.84:0/1188669557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563df0e32c40 0x563df0e35030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.686+0000 7f77fa59c640 1 -- 172.21.15.84:0/1188669557 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563df0d342d0 con 0x563df0e28570 2024-05-22T23:10:31.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.686+0000 7f77f1d9b640 1 --2- 172.21.15.84:0/1188669557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563df0e32c40 0x563df0e35030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:10:31.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.686+0000 7f77fad9d640 1 --2- 172.21.15.84:0/670885231 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f77e80a2d30 0x7f77e80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.686+0000 7f77f3fff640 1 -- 172.21.15.84:0/670885231 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7f77ec0efcb0 con 0x7f77fc069ef0 2024-05-22T23:10:31.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.686+0000 7f77fa59c640 1 --2- 172.21.15.84:0/1188669557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e28570 0x563df0e32700 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0ed340 tx=0x7f77ec08c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4adecbe9a6c0c164 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:31.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.686+0000 7f77f0d99640 1 -- 172.21.15.84:0/1188669557 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec0bbb00 con 0x563df0e28570 2024-05-22T23:10:31.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.686+0000 7f77f0d99640 1 -- 172.21.15.84:0/1188669557 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77ec0bc8a0 con 0x563df0e28570 2024-05-22T23:10:31.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.686+0000 7f77f0d99640 1 -- 172.21.15.84:0/1188669557 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec0d3a10 con 0x563df0e28570 2024-05-22T23:10:31.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.687+0000 7f7803a24740 1 -- 172.21.15.84:0/1188669557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e28570 msgr2=0x563df0e32700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:31.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.687+0000 7f77fad9d640 1 --2- 172.21.15.84:0/670885231 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f77e80a2d30 0x7f77e80a51f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f77f4009fd0 tx=0x7f77f4004b50 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:31.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.687+0000 7f7803a24740 1 --2- 172.21.15.84:0/1188669557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e28570 0x563df0e32700 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0ed340 tx=0x7f77ec08c000 comp rx=0 tx=0).stop 2024-05-22T23:10:31.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.687+0000 7f7803a24740 1 -- 172.21.15.84:0/1188669557 shutdown_connections 2024-05-22T23:10:31.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.687+0000 7f7803a24740 1 --2- 172.21.15.84:0/1188669557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563df0e32c40 0x563df0e35030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.687+0000 7f7803a24740 1 --2- 172.21.15.84:0/1188669557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e28570 0x563df0e32700 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.687+0000 7f7803a24740 1 --2- 172.21.15.84:0/1188669557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563df0e27c50 0x563df0e28030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.687+0000 7f7803a24740 1 -- 172.21.15.84:0/1188669557 >> 172.21.15.84:0/1188669557 conn(0x563df0e23f70 msgr2=0x563df0e24350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:31.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.687+0000 7f7803a24740 1 -- 172.21.15.84:0/1188669557 shutdown_connections 2024-05-22T23:10:31.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.687+0000 7f7803a24740 1 -- 172.21.15.84:0/1188669557 wait complete. 2024-05-22T23:10:31.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.687+0000 7f7803a24740 1 -- start start 2024-05-22T23:10:31.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.687+0000 7f7803a24740 1 Processor -- start 2024-05-22T23:10:31.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.687+0000 7f7803a24740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e27c50 0x563df0e2fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.687+0000 7f7803a24740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563df0e28570 0x563df0e300d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.688+0000 7f7803a24740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563df0e32c40 0x563df0e30610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.688+0000 7f7803a24740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563df0e357f0 con 0x563df0e27c50 2024-05-22T23:10:31.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.688+0000 7f7803a24740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563df0d155f0 con 0x563df0e28570 2024-05-22T23:10:31.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.688+0000 7f7803a24740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563df0e35970 con 0x563df0e32c40 2024-05-22T23:10:31.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.688+0000 7f7803a24740 10 client.0 _ll_register_callbacks cb 0x563df0d10cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:10:31.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.688+0000 7f77fa59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e27c50 0x563df0e2fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.688+0000 7f77fa59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e27c50 0x563df0e2fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54914/0 (socket says 172.21.15.84:54914) 2024-05-22T23:10:31.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.688+0000 7f77fa59c640 1 -- 172.21.15.84:0/2767577867 learned_addr learned my addr 172.21.15.84:0/2767577867 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:10:31.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.688+0000 7f77fad9d640 1 --2- 172.21.15.84:0/2767577867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563df0e32c40 0x563df0e30610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.688+0000 7f77fa59c640 1 -- 172.21.15.84:0/2767577867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563df0e32c40 msgr2=0x563df0e30610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:31.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.688+0000 7f77fa59c640 1 --2- 172.21.15.84:0/2767577867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563df0e32c40 0x563df0e30610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.688+0000 7f77fa59c640 1 -- 172.21.15.84:0/2767577867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563df0e28570 msgr2=0x563df0e300d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:31.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.688+0000 7f77fa59c640 1 --2- 172.21.15.84:0/2767577867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563df0e28570 0x563df0e300d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.688+0000 7f77fa59c640 1 -- 172.21.15.84:0/2767577867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563df0b677e0 con 0x563df0e27c50 2024-05-22T23:10:31.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.688+0000 7f77fa59c640 1 --2- 172.21.15.84:0/2767577867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e27c50 0x563df0e2fb90 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0b8600 tx=0x7f77ec0bbde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119f9a00c3f15b05 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:31.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.688+0000 7f77e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:10:31.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.688+0000 7f77e1ffb640 1 -- 172.21.15.84:0/2767577867 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec035070 con 0x563df0e27c50 2024-05-22T23:10:31.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.688+0000 7f77e1ffb640 1 -- 172.21.15.84:0/2767577867 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7f77ec0ba4a0 con 0x563df0e27c50 2024-05-22T23:10:31.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.688+0000 7f7803a24740 1 -- 172.21.15.84:0/2767577867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77ec0bc8a0 con 0x563df0e27c50 2024-05-22T23:10:31.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.688+0000 7f77e1ffb640 1 -- 172.21.15.84:0/2767577867 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f77ec0bbb00 con 0x563df0e27c50 2024-05-22T23:10:31.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.688+0000 7f7803a24740 10 client.16830 fetch_fsmap learned FSMap version 18 2024-05-22T23:10:31.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.688+0000 7f7803a24740 1 -- 172.21.15.84:0/2767577867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x563df0bc7f70 con 0x563df0e27c50 2024-05-22T23:10:31.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.688+0000 7f77e1ffb640 1 -- 172.21.15.84:0/2767577867 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77ec0bc8a0 con 0x563df0e27c50 2024-05-22T23:10:31.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.689+0000 7f77e1ffb640 1 -- 172.21.15.84:0/2767577867 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f77ec0ed650 con 0x563df0e27c50 2024-05-22T23:10:31.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.689+0000 7f7803a24740 10 client.16830 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:10:31.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.689+0000 7f7803a24740 10 client.16830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:10:31.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.689+0000 7f7803a24740 20 client.16830 populate_metadata read hostname 'smithi084' 2024-05-22T23:10:31.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.689+0000 7f7803a24740 1 --2- 172.21.15.84:0/2767577867 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563df0e42ab0 0x563df0e44ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.689+0000 7f7803a24740 4 client.16830 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:10:31.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.689+0000 7f7803a24740 1 -- 172.21.15.84:0/2767577867 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563df0d9f460 con 0x563df0e42ab0 2024-05-22T23:10:31.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.689+0000 7f77f1d9b640 1 --2- 172.21.15.84:0/2767577867 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563df0e42ab0 0x563df0e44ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.689+0000 7f77e1ffb640 10 client.16830 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:10:31.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.689+0000 7f77f1d9b640 1 --2- 172.21.15.84:0/2767577867 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563df0e42ab0 0x563df0e44ea0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5aa7e0979c761787 server_cookie=3f1e5eae493b6dfc in_seq=0 out_seq=0 2024-05-22T23:10:31.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.690+0000 7f77e1ffb640 1 -- 172.21.15.84:0/2767577867 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563df0d9f460 con 0x563df0e42ab0 2024-05-22T23:10:31.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.690+0000 7f77e1ffb640 10 client.16830 handle_command_reply: tid=0 2024-05-22T23:10:31.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.690+0000 7f7803a24740 1 client.16830 shutdown 2024-05-22T23:10:31.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.690+0000 7f7803a24740 1 -- 172.21.15.84:0/2767577867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e27c50 msgr2=0x563df0e2fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:31.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.690+0000 7f7803a24740 1 --2- 172.21.15.84:0/2767577867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e27c50 0x563df0e2fb90 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0b8600 tx=0x7f77ec0bbde0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.690+0000 7f7803a24740 1 -- 172.21.15.84:0/2767577867 shutdown_connections 2024-05-22T23:10:31.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.690+0000 7f7803a24740 1 --2- 172.21.15.84:0/2767577867 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563df0e42ab0 0x563df0e44ea0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.690+0000 7f7803a24740 1 --2- 172.21.15.84:0/2767577867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563df0e32c40 0x563df0e30610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.690+0000 7f7803a24740 1 --2- 172.21.15.84:0/2767577867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563df0e28570 0x563df0e300d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.690+0000 7f7803a24740 1 --2- 172.21.15.84:0/2767577867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e27c50 0x563df0e2fb90 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.690+0000 7f77e1ffb640 0 client.16830 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:10:31.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.690+0000 7f7803a24740 1 -- 172.21.15.84:0/2767577867 >> 172.21.15.84:0/2767577867 conn(0x563df0e23f70 msgr2=0x563df0e29d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:31.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.690+0000 7f7803a24740 1 -- 172.21.15.84:0/2767577867 shutdown_connections 2024-05-22T23:10:31.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.690+0000 7f7803a24740 1 -- 172.21.15.84:0/2767577867 wait complete. 2024-05-22T23:10:31.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.690+0000 7f7803a24740 20 client.16830 trim_cache size 0 max 16384 2024-05-22T23:10:31.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.708+0000 7f7803a24740 1 Processor -- start 2024-05-22T23:10:31.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.708+0000 7f7803a24740 1 -- start start 2024-05-22T23:10:31.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.708+0000 7f7803a24740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563df0e27c50 0x563df0e3e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.708+0000 7f7803a24740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563df0e28570 0x563df0e30430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.708+0000 7f7803a24740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e32c40 0x563df0e4a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.708+0000 7f7803a24740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc1ad790 con 0x563df0e32c40 2024-05-22T23:10:31.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.708+0000 7f7803a24740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc1afff0 con 0x563df0e28570 2024-05-22T23:10:31.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.708+0000 7f7803a24740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc1adde0 con 0x563df0e27c50 2024-05-22T23:10:31.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.708+0000 7f77fad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e32c40 0x563df0e4a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.708+0000 7f77fad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e32c40 0x563df0e4a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54926/0 (socket says 172.21.15.84:54926) 2024-05-22T23:10:31.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.708+0000 7f77fad9d640 1 -- 172.21.15.84:0/1299529462 learned_addr learned my addr 172.21.15.84:0/1299529462 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:10:31.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.708+0000 7f77fa59c640 1 --2- 172.21.15.84:0/1299529462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563df0e27c50 0x563df0e3e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.708+0000 7f77fad9d640 1 -- 172.21.15.84:0/1299529462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563df0e27c50 msgr2=0x563df0e3e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:31.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.709+0000 7f77f1d9b640 1 --2- 172.21.15.84:0/1299529462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563df0e28570 0x563df0e30430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.709+0000 7f77fad9d640 1 --2- 172.21.15.84:0/1299529462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563df0e27c50 0x563df0e3e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.709+0000 7f77fad9d640 1 -- 172.21.15.84:0/1299529462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563df0e28570 msgr2=0x563df0e30430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:31.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.709+0000 7f77fad9d640 1 --2- 172.21.15.84:0/1299529462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563df0e28570 0x563df0e30430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.709+0000 7f77fad9d640 1 -- 172.21.15.84:0/1299529462 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563df0e45210 con 0x563df0e32c40 2024-05-22T23:10:31.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.709+0000 7f77f1d9b640 1 --2- 172.21.15.84:0/1299529462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563df0e28570 0x563df0e30430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:10:31.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.709+0000 7f77fad9d640 1 --2- 172.21.15.84:0/1299529462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e32c40 0x563df0e4a730 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f77f4016740 tx=0x7f77f4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a364519390d16f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:31.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.709+0000 7f77e3fff640 1 -- 172.21.15.84:0/1299529462 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f4007a20 con 0x563df0e32c40 2024-05-22T23:10:31.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.709+0000 7f77e3fff640 1 -- 172.21.15.84:0/1299529462 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77f4028070 con 0x563df0e32c40 2024-05-22T23:10:31.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.709+0000 7f77e3fff640 1 -- 172.21.15.84:0/1299529462 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f40155b0 con 0x563df0e32c40 2024-05-22T23:10:31.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.709+0000 7f7803a24740 1 -- 172.21.15.84:0/1299529462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e32c40 msgr2=0x563df0e4a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:31.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.709+0000 7f7803a24740 1 --2- 172.21.15.84:0/1299529462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e32c40 0x563df0e4a730 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f77f4016740 tx=0x7f77f4014000 comp rx=0 tx=0).stop 2024-05-22T23:10:31.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.709+0000 7f7803a24740 1 -- 172.21.15.84:0/1299529462 shutdown_connections 2024-05-22T23:10:31.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.709+0000 7f7803a24740 1 --2- 172.21.15.84:0/1299529462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e32c40 0x563df0e4a730 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.709+0000 7f7803a24740 1 --2- 172.21.15.84:0/1299529462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563df0e28570 0x563df0e30430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.709+0000 7f7803a24740 1 --2- 172.21.15.84:0/1299529462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563df0e27c50 0x563df0e3e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.709+0000 7f7803a24740 1 -- 172.21.15.84:0/1299529462 >> 172.21.15.84:0/1299529462 conn(0x563df0e23f70 msgr2=0x563df0e313c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:31.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.709+0000 7f7803a24740 1 -- 172.21.15.84:0/1299529462 shutdown_connections 2024-05-22T23:10:31.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.710+0000 7f7803a24740 1 -- 172.21.15.84:0/1299529462 wait complete. 2024-05-22T23:10:31.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.710+0000 7f7803a24740 1 -- start start 2024-05-22T23:10:31.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.710+0000 7f7803a24740 1 Processor -- start 2024-05-22T23:10:31.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.710+0000 7f7803a24740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563df0e27c50 0x563df0e2cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.710+0000 7f7803a24740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563df0e28570 0x563df0e2d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.710+0000 7f7803a24740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e32c40 0x563df0ea7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.710+0000 7f7803a24740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc1ae4b0 con 0x563df0e32c40 2024-05-22T23:10:31.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.710+0000 7f7803a24740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc1af920 con 0x563df0e28570 2024-05-22T23:10:31.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.710+0000 7f7803a24740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc1aeb80 con 0x563df0e27c50 2024-05-22T23:10:31.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.710+0000 7f7803a24740 10 client.0 _ll_register_callbacks cb 0x7f77e400b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:10:31.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.710+0000 7f77fad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e32c40 0x563df0ea7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.710+0000 7f77fad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e32c40 0x563df0ea7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54934/0 (socket says 172.21.15.84:54934) 2024-05-22T23:10:31.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.710+0000 7f77fad9d640 1 -- 172.21.15.84:0/2845925969 learned_addr learned my addr 172.21.15.84:0/2845925969 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:10:31.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.710+0000 7f77f1d9b640 1 --2- 172.21.15.84:0/2845925969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563df0e28570 0x563df0e2d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.710+0000 7f77fa59c640 1 --2- 172.21.15.84:0/2845925969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563df0e27c50 0x563df0e2cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.710+0000 7f77fad9d640 1 -- 172.21.15.84:0/2845925969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563df0e27c50 msgr2=0x563df0e2cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:31.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.710+0000 7f77fad9d640 1 --2- 172.21.15.84:0/2845925969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563df0e27c50 0x563df0e2cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.710+0000 7f77fad9d640 1 -- 172.21.15.84:0/2845925969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563df0e28570 msgr2=0x563df0e2d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:31.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.710+0000 7f77fad9d640 1 --2- 172.21.15.84:0/2845925969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563df0e28570 0x563df0e2d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.710+0000 7f77fad9d640 1 -- 172.21.15.84:0/2845925969 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563df0a5f170 con 0x563df0e32c40 2024-05-22T23:10:31.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.711+0000 7f77fad9d640 1 --2- 172.21.15.84:0/2845925969 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e32c40 0x563df0ea7fa0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f77f4009a00 tx=0x7f77f4009a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1420d3991dc4cb86 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:31.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.711+0000 7f77e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:10:31.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.711+0000 7f77e1ffb640 1 -- 172.21.15.84:0/2845925969 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f4007c30 con 0x563df0e32c40 2024-05-22T23:10:31.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.711+0000 7f77e1ffb640 1 -- 172.21.15.84:0/2845925969 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7f77f4007dd0 con 0x563df0e32c40 2024-05-22T23:10:31.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.711+0000 7f7803a24740 1 -- 172.21.15.84:0/2845925969 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77f4028070 con 0x563df0e32c40 2024-05-22T23:10:31.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.711+0000 7f77e1ffb640 1 -- 172.21.15.84:0/2845925969 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f77f40238e0 con 0x563df0e32c40 2024-05-22T23:10:31.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.711+0000 7f7803a24740 10 client.16836 fetch_fsmap learned FSMap version 18 2024-05-22T23:10:31.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.711+0000 7f7803a24740 1 -- 172.21.15.84:0/2845925969 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x563df0d54930 con 0x563df0e32c40 2024-05-22T23:10:31.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.711+0000 7f77e1ffb640 1 -- 172.21.15.84:0/2845925969 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77f4028070 con 0x563df0e32c40 2024-05-22T23:10:31.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.711+0000 7f77e1ffb640 1 -- 172.21.15.84:0/2845925969 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f77f403e070 con 0x563df0e32c40 2024-05-22T23:10:31.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.711+0000 7f7803a24740 10 client.16836 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:10:31.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.711+0000 7f7803a24740 10 client.16836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:10:31.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.711+0000 7f7803a24740 20 client.16836 populate_metadata read hostname 'smithi084' 2024-05-22T23:10:31.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.711+0000 7f7803a24740 1 --2- 172.21.15.84:0/2845925969 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563df0ea9a90 0x563df0eabeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.711+0000 7f7803a24740 4 client.16836 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-22T23:10:31.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.711+0000 7f7803a24740 1 -- 172.21.15.84:0/2845925969 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563df0d99170 con 0x563df0ea9a90 2024-05-22T23:10:31.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.711+0000 7f77fa59c640 1 --2- 172.21.15.84:0/2845925969 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563df0ea9a90 0x563df0eabeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.712+0000 7f77e1ffb640 10 client.16836 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:10:31.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.712+0000 7f77fa59c640 1 --2- 172.21.15.84:0/2845925969 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563df0ea9a90 0x563df0eabeb0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60260abe427fffca server_cookie=6253b0073f462420 in_seq=0 out_seq=0 2024-05-22T23:10:31.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.713+0000 7f77e1ffb640 1 -- 172.21.15.84:0/2845925969 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563df0d99170 con 0x563df0ea9a90 2024-05-22T23:10:31.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.713+0000 7f77e1ffb640 10 client.16836 handle_command_reply: tid=0 2024-05-22T23:10:31.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.713+0000 7f7803a24740 1 client.16836 shutdown 2024-05-22T23:10:31.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.713+0000 7f7803a24740 1 -- 172.21.15.84:0/2845925969 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e32c40 msgr2=0x563df0ea7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:31.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.713+0000 7f7803a24740 1 --2- 172.21.15.84:0/2845925969 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e32c40 0x563df0ea7fa0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f77f4009a00 tx=0x7f77f4009a30 comp rx=0 tx=0).stop 2024-05-22T23:10:31.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.713+0000 7f7803a24740 1 -- 172.21.15.84:0/2845925969 shutdown_connections 2024-05-22T23:10:31.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.713+0000 7f7803a24740 1 --2- 172.21.15.84:0/2845925969 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x563df0ea9a90 0x563df0eabeb0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.713+0000 7f7803a24740 1 --2- 172.21.15.84:0/2845925969 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563df0e32c40 0x563df0ea7fa0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.713+0000 7f77e1ffb640 0 client.16836 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:10:31.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.713+0000 7f7803a24740 1 --2- 172.21.15.84:0/2845925969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563df0e28570 0x563df0e2d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.713+0000 7f7803a24740 1 --2- 172.21.15.84:0/2845925969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563df0e27c50 0x563df0e2cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.713+0000 7f7803a24740 1 -- 172.21.15.84:0/2845925969 >> 172.21.15.84:0/2845925969 conn(0x563df0e23f70 msgr2=0x563df0e31c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:31.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.713+0000 7f7803a24740 1 -- 172.21.15.84:0/2845925969 shutdown_connections 2024-05-22T23:10:31.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.713+0000 7f7803a24740 1 -- 172.21.15.84:0/2845925969 wait complete. 2024-05-22T23:10:31.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.713+0000 7f7803a24740 20 client.16836 trim_cache size 0 max 16384 2024-05-22T23:10:31.731 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:10:31.731 INFO:teuthology.orchestra.run.smithi084.stdout: "return_code": 0, 2024-05-22T23:10:31.732 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_tag": "f18eef90-8438-4e87-a879-cf3417aa50c4", 2024-05-22T23:10:31.732 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": "asynchronous" 2024-05-22T23:10:31.732 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:10:31.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.714+0000 7f7800c38640 1 -- 172.21.15.84:0/670885231 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f77e80a2d30 msgr2=0x7f77e80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:31.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.714+0000 7f7800c38640 1 --2- 172.21.15.84:0/670885231 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f77e80a2d30 0x7f77e80a51f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f77f4009fd0 tx=0x7f77f4004b50 comp rx=0 tx=0).stop 2024-05-22T23:10:31.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.714+0000 7f7800c38640 1 -- 172.21.15.84:0/670885231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77fc069ef0 msgr2=0x7f77fc1007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:31.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.714+0000 7f7800c38640 1 --2- 172.21.15.84:0/670885231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77fc069ef0 0x7f77fc1007a0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009230 tx=0x7f77ec004270 comp rx=0 tx=0).stop 2024-05-22T23:10:31.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.714+0000 7f7800c38640 1 -- 172.21.15.84:0/670885231 shutdown_connections 2024-05-22T23:10:31.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.714+0000 7f7800c38640 1 --2- 172.21.15.84:0/670885231 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f77e80a2d30 0x7f77e80a51f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.714+0000 7f7800c38640 1 --2- 172.21.15.84:0/670885231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77fc107860 0x7f77fc101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.714+0000 7f7800c38640 1 --2- 172.21.15.84:0/670885231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77fc06a8b0 0x7f77fc100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.714+0000 7f7800c38640 1 --2- 172.21.15.84:0/670885231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77fc069ef0 0x7f77fc1007a0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.714+0000 7f7800c38640 1 -- 172.21.15.84:0/670885231 >> 172.21.15.84:0/670885231 conn(0x7f77fc0777d0 msgr2=0x7f77fc0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:31.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.714+0000 7f7800c38640 1 -- 172.21.15.84:0/670885231 shutdown_connections 2024-05-22T23:10:31.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.714+0000 7f7800c38640 1 -- 172.21.15.84:0/670885231 wait complete. 2024-05-22T23:10:31.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:10:31.735 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:10:31.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.869+0000 7fd01f038640 1 -- 172.21.15.84:0/3283501136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd018103420 msgr2=0x7fd01810b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:31.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.869+0000 7fd01f038640 1 --2- 172.21.15.84:0/3283501136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd018103420 0x7fd01810b900 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009900 tx=0x7fd00c02a510 comp rx=0 tx=0).stop 2024-05-22T23:10:31.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.869+0000 7fd01f038640 1 -- 172.21.15.84:0/3283501136 shutdown_connections 2024-05-22T23:10:31.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.869+0000 7fd01f038640 1 --2- 172.21.15.84:0/3283501136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd01810be40 0x7fd01810e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.869+0000 7fd01f038640 1 --2- 172.21.15.84:0/3283501136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd018103420 0x7fd01810b900 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.869+0000 7fd01f038640 1 --2- 172.21.15.84:0/3283501136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd018102b00 0x7fd018102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.869+0000 7fd01f038640 1 -- 172.21.15.84:0/3283501136 >> 172.21.15.84:0/3283501136 conn(0x7fd018076af0 msgr2=0x7fd018076f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:31.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.869+0000 7fd01f038640 1 -- 172.21.15.84:0/3283501136 shutdown_connections 2024-05-22T23:10:31.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.869+0000 7fd01f038640 1 -- 172.21.15.84:0/3283501136 wait complete. 2024-05-22T23:10:31.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.870+0000 7fd01f038640 1 Processor -- start 2024-05-22T23:10:31.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.870+0000 7fd01f038640 1 -- start start 2024-05-22T23:10:31.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.870+0000 7fd01f038640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd018102b00 0x7fd0181981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.870+0000 7fd01f038640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd018103420 0x7fd018198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.870+0000 7fd01f038640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd01810be40 0x7fd01819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.870+0000 7fd01f038640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd018111c40 con 0x7fd018102b00 2024-05-22T23:10:31.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.870+0000 7fd01f038640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd018111ac0 con 0x7fd018103420 2024-05-22T23:10:31.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.870+0000 7fd01f038640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd018111dc0 con 0x7fd01810be40 2024-05-22T23:10:31.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.870+0000 7fd01cdd4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd018102b00 0x7fd0181981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.870+0000 7fd01cdd4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd018102b00 0x7fd0181981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54946/0 (socket says 172.21.15.84:54946) 2024-05-22T23:10:31.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.870+0000 7fd01cdd4640 1 -- 172.21.15.84:0/2578923566 learned_addr learned my addr 172.21.15.84:0/2578923566 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:10:31.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.870+0000 7fd017fff640 1 --2- 172.21.15.84:0/2578923566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd018103420 0x7fd018198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.871+0000 7fd01d5d5640 1 --2- 172.21.15.84:0/2578923566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd01810be40 0x7fd01819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.871+0000 7fd01cdd4640 1 -- 172.21.15.84:0/2578923566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd01810be40 msgr2=0x7fd01819caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:31.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.871+0000 7fd01cdd4640 1 --2- 172.21.15.84:0/2578923566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd01810be40 0x7fd01819caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.871+0000 7fd01cdd4640 1 -- 172.21.15.84:0/2578923566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd018103420 msgr2=0x7fd018198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:31.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.871+0000 7fd01cdd4640 1 --2- 172.21.15.84:0/2578923566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd018103420 0x7fd018198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.871+0000 7fd01cdd4640 1 -- 172.21.15.84:0/2578923566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01819d1d0 con 0x7fd018102b00 2024-05-22T23:10:31.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.871+0000 7fd01d5d5640 1 --2- 172.21.15.84:0/2578923566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd01810be40 0x7fd01819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:10:31.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.871+0000 7fd01cdd4640 1 --2- 172.21.15.84:0/2578923566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd018102b00 0x7fd0181981e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fd00800c970 tx=0x7fd00800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:31.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.871+0000 7fd015ffb640 1 -- 172.21.15.84:0/2578923566 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008007bb0 con 0x7fd018102b00 2024-05-22T23:10:31.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.871+0000 7fd015ffb640 1 -- 172.21.15.84:0/2578923566 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd008007d50 con 0x7fd018102b00 2024-05-22T23:10:31.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.871+0000 7fd01f038640 1 -- 172.21.15.84:0/2578923566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd01819d4c0 con 0x7fd018102b00 2024-05-22T23:10:31.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.871+0000 7fd015ffb640 1 -- 172.21.15.84:0/2578923566 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00800f450 con 0x7fd018102b00 2024-05-22T23:10:31.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.871+0000 7fd01f038640 1 -- 172.21.15.84:0/2578923566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0181a4d50 con 0x7fd018102b00 2024-05-22T23:10:31.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.873+0000 7fd015ffb640 1 -- 172.21.15.84:0/2578923566 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd008005020 con 0x7fd018102b00 2024-05-22T23:10:31.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.874+0000 7fd021e22740 1 Processor -- start 2024-05-22T23:10:31.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.874+0000 7fd021e22740 1 -- start start 2024-05-22T23:10:31.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.874+0000 7fd015ffb640 1 --2- 172.21.15.84:0/2578923566 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd0000a2d00 0x7fd0000a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.874+0000 7fd021e22740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 0x558b199b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.874+0000 7fd021e22740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b199b1db0 0x558b199b2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.874+0000 7fd021e22740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b199c0f40 0x558b199c3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.874+0000 7fd021e22740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558b19915db0 con 0x558b199b34a0 2024-05-22T23:10:31.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.874+0000 7fd021e22740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558b198b4360 con 0x558b199b1db0 2024-05-22T23:10:31.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.874+0000 7fd01d5d5640 1 --2- 172.21.15.84:0/2578923566 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd0000a2d00 0x7fd0000a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.874+0000 7fd021e22740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558b19911d70 con 0x558b199c0f40 2024-05-22T23:10:31.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.874+0000 7fd015ffb640 1 -- 172.21.15.84:0/2578923566 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7fd0080cf2d0 con 0x7fd018102b00 2024-05-22T23:10:31.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.874+0000 7fd01cdd4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 0x558b199b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.874+0000 7fd01cdd4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 0x558b199b17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54954/0 (socket says 172.21.15.84:54954) 2024-05-22T23:10:31.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.874+0000 7fd01cdd4640 1 -- 172.21.15.84:0/3180800350 learned_addr learned my addr 172.21.15.84:0/3180800350 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:10:31.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.874+0000 7fd017fff640 1 --2- 172.21.15.84:0/3180800350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b199b1db0 0x558b199b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.874+0000 7fd01cdd4640 1 -- 172.21.15.84:0/3180800350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b199c0f40 msgr2=0x558b199c3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:31.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.874+0000 7fd01cdd4640 1 --2- 172.21.15.84:0/3180800350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b199c0f40 0x558b199c3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.874+0000 7fd01cdd4640 1 -- 172.21.15.84:0/3180800350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b199b1db0 msgr2=0x558b199b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:31.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.874+0000 7fd01cdd4640 1 --2- 172.21.15.84:0/3180800350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b199b1db0 0x558b199b2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.874+0000 7fd01cdd4640 1 -- 172.21.15.84:0/3180800350 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b198d7a60 con 0x558b199b34a0 2024-05-22T23:10:31.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.874+0000 7fd01d5d5640 1 --2- 172.21.15.84:0/2578923566 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd0000a2d00 0x7fd0000a51c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd010008c60 tx=0x7fd01000d710 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:31.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.874+0000 7fd01cdd4640 1 --2- 172.21.15.84:0/3180800350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 0x558b199b17e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fd0080a3ea0 tx=0x7fd00809b180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f56b5f36e2ea1df server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:31.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.875+0000 7fcffeffd640 1 -- 172.21.15.84:0/3180800350 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00801d040 con 0x558b199b34a0 2024-05-22T23:10:31.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.875+0000 7fcffeffd640 1 -- 172.21.15.84:0/3180800350 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0080cf050 con 0x558b199b34a0 2024-05-22T23:10:31.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.875+0000 7fcffeffd640 1 -- 172.21.15.84:0/3180800350 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008097dd0 con 0x558b199b34a0 2024-05-22T23:10:31.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.875+0000 7fd021e22740 1 -- 172.21.15.84:0/3180800350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 msgr2=0x558b199b17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:31.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.875+0000 7fd021e22740 1 --2- 172.21.15.84:0/3180800350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 0x558b199b17e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fd0080a3ea0 tx=0x7fd00809b180 comp rx=0 tx=0).stop 2024-05-22T23:10:31.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.875+0000 7fd021e22740 1 -- 172.21.15.84:0/3180800350 shutdown_connections 2024-05-22T23:10:31.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.875+0000 7fd021e22740 1 --2- 172.21.15.84:0/3180800350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b199c0f40 0x558b199c3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.875+0000 7fd021e22740 1 --2- 172.21.15.84:0/3180800350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b199b1db0 0x558b199b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.875+0000 7fd021e22740 1 --2- 172.21.15.84:0/3180800350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 0x558b199b17e0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.875+0000 7fd021e22740 1 -- 172.21.15.84:0/3180800350 >> 172.21.15.84:0/3180800350 conn(0x558b1997a8e0 msgr2=0x558b199ae310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:31.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.876+0000 7fd021e22740 1 -- 172.21.15.84:0/3180800350 shutdown_connections 2024-05-22T23:10:31.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.876+0000 7fd021e22740 1 -- 172.21.15.84:0/3180800350 wait complete. 2024-05-22T23:10:31.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.876+0000 7fd021e22740 1 -- start start 2024-05-22T23:10:31.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.876+0000 7fd021e22740 1 Processor -- start 2024-05-22T23:10:31.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fd021e22740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b199b1db0 0x558b199bde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.893 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fd021e22740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 0x558b199be3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fd021e22740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b199c0f40 0x558b199be8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fd021e22740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558b199c3dd0 con 0x558b199b34a0 2024-05-22T23:10:31.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fd021e22740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558b198d00f0 con 0x558b199c0f40 2024-05-22T23:10:31.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fd021e22740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558b199c3f50 con 0x558b199b1db0 2024-05-22T23:10:31.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fd021e22740 10 client.0 _ll_register_callbacks cb 0x558b198d4f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:10:31.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fd017fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 0x558b199be3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fd017fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 0x558b199be3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54964/0 (socket says 172.21.15.84:54964) 2024-05-22T23:10:31.894 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fd017fff640 1 -- 172.21.15.84:0/2073920798 learned_addr learned my addr 172.21.15.84:0/2073920798 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:10:31.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fd01cdd4640 1 --2- 172.21.15.84:0/2073920798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b199b1db0 0x558b199bde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fd01d5d5640 1 --2- 172.21.15.84:0/2073920798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b199c0f40 0x558b199be8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fd017fff640 1 -- 172.21.15.84:0/2073920798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b199b1db0 msgr2=0x558b199bde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:31.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fd017fff640 1 --2- 172.21.15.84:0/2073920798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b199b1db0 0x558b199bde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fd017fff640 1 -- 172.21.15.84:0/2073920798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b199c0f40 msgr2=0x558b199be8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:31.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fd017fff640 1 --2- 172.21.15.84:0/2073920798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b199c0f40 0x558b199be8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fd017fff640 1 -- 172.21.15.84:0/2073920798 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b196f37d0 con 0x558b199b34a0 2024-05-22T23:10:31.895 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fd01d5d5640 1 --2- 172.21.15.84:0/2073920798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b199c0f40 0x558b199be8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:10:31.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fd017fff640 1 --2- 172.21.15.84:0/2073920798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 0x558b199be3a0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fd00c033000 tx=0x7fd00c031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e6d861a9772a19 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:31.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fcff7fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:10:31.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fcff7fff640 1 -- 172.21.15.84:0/2073920798 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c040070 con 0x558b199b34a0 2024-05-22T23:10:31.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fcff7fff640 1 -- 172.21.15.84:0/2073920798 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7fd00c004460 con 0x558b199b34a0 2024-05-22T23:10:31.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fd021e22740 1 -- 172.21.15.84:0/2073920798 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0080cf050 con 0x558b199b34a0 2024-05-22T23:10:31.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fcff7fff640 1 -- 172.21.15.84:0/2073920798 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fd00c04c050 con 0x558b199b34a0 2024-05-22T23:10:31.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fd021e22740 10 client.16848 fetch_fsmap learned FSMap version 18 2024-05-22T23:10:31.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.877+0000 7fd021e22740 1 -- 172.21.15.84:0/2073920798 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x558b196293f0 con 0x558b199b34a0 2024-05-22T23:10:31.896 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.878+0000 7fcff7fff640 1 -- 172.21.15.84:0/2073920798 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0080cf050 con 0x558b199b34a0 2024-05-22T23:10:31.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.878+0000 7fcff7fff640 1 -- 172.21.15.84:0/2073920798 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fd00c0035f0 con 0x558b199b34a0 2024-05-22T23:10:31.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.878+0000 7fd021e22740 10 client.16848 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:10:31.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.878+0000 7fd021e22740 10 client.16848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:10:31.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.878+0000 7fd021e22740 20 client.16848 populate_metadata read hostname 'smithi084' 2024-05-22T23:10:31.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.878+0000 7fd021e22740 1 --2- 172.21.15.84:0/2073920798 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558b199d13b0 0x558b199d3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.878+0000 7fd021e22740 4 client.16848 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:10:31.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.878+0000 7fd021e22740 1 -- 172.21.15.84:0/2073920798 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b199234e0 con 0x558b199d13b0 2024-05-22T23:10:31.897 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.878+0000 7fd01cdd4640 1 --2- 172.21.15.84:0/2073920798 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558b199d13b0 0x558b199d3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.879+0000 7fcff7fff640 10 client.16848 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:10:31.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.879+0000 7fd01cdd4640 1 --2- 172.21.15.84:0/2073920798 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558b199d13b0 0x558b199d3810 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eb8dffa3c0eb18c server_cookie=444cf67fa063cc03 in_seq=0 out_seq=0 2024-05-22T23:10:31.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.879+0000 7fcff7fff640 1 -- 172.21.15.84:0/2073920798 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b199234e0 con 0x558b199d13b0 2024-05-22T23:10:31.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.879+0000 7fcff7fff640 10 client.16848 handle_command_reply: tid=0 2024-05-22T23:10:31.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.879+0000 7fd021e22740 1 client.16848 shutdown 2024-05-22T23:10:31.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.879+0000 7fd021e22740 1 -- 172.21.15.84:0/2073920798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 msgr2=0x558b199be3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:31.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.879+0000 7fd021e22740 1 --2- 172.21.15.84:0/2073920798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 0x558b199be3a0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fd00c033000 tx=0x7fd00c031850 comp rx=0 tx=0).stop 2024-05-22T23:10:31.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.879+0000 7fd021e22740 1 -- 172.21.15.84:0/2073920798 shutdown_connections 2024-05-22T23:10:31.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.879+0000 7fd021e22740 1 --2- 172.21.15.84:0/2073920798 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558b199d13b0 0x558b199d3810 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.879+0000 7fd021e22740 1 --2- 172.21.15.84:0/2073920798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b199c0f40 0x558b199be8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.879+0000 7fcff7fff640 0 client.16848 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:10:31.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.879+0000 7fd021e22740 1 --2- 172.21.15.84:0/2073920798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 0x558b199be3a0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.879+0000 7fd021e22740 1 --2- 172.21.15.84:0/2073920798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b199b1db0 0x558b199bde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.879+0000 7fd021e22740 1 -- 172.21.15.84:0/2073920798 >> 172.21.15.84:0/2073920798 conn(0x558b1997a8e0 msgr2=0x558b199b4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:31.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.880+0000 7fd021e22740 1 -- 172.21.15.84:0/2073920798 shutdown_connections 2024-05-22T23:10:31.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.880+0000 7fd021e22740 1 -- 172.21.15.84:0/2073920798 wait complete. 2024-05-22T23:10:31.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.880+0000 7fd021e22740 20 client.16848 trim_cache size 0 max 16384 2024-05-22T23:10:31.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.897+0000 7fd021e22740 1 Processor -- start 2024-05-22T23:10:31.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.897+0000 7fd021e22740 1 -- start start 2024-05-22T23:10:31.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.897+0000 7fd021e22740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b199b1db0 0x558b199cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.897+0000 7fd021e22740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 0x558b199d90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.897+0000 7fd021e22740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b199c0f40 0x558b199d9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.897+0000 7fd021e22740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd0181a9590 con 0x558b199b34a0 2024-05-22T23:10:31.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.897+0000 7fd021e22740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd0181abdf0 con 0x558b199c0f40 2024-05-22T23:10:31.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.897+0000 7fd021e22740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd0181a9be0 con 0x558b199b1db0 2024-05-22T23:10:31.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.897+0000 7fd017fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 0x558b199d90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.897+0000 7fd017fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 0x558b199d90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54970/0 (socket says 172.21.15.84:54970) 2024-05-22T23:10:31.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.897+0000 7fd017fff640 1 -- 172.21.15.84:0/480429030 learned_addr learned my addr 172.21.15.84:0/480429030 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:10:31.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.897+0000 7fd017fff640 1 -- 172.21.15.84:0/480429030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b199b1db0 msgr2=0x558b199cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:31.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.897+0000 7fd017fff640 1 --2- 172.21.15.84:0/480429030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b199b1db0 0x558b199cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.897+0000 7fd017fff640 1 -- 172.21.15.84:0/480429030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b199c0f40 msgr2=0x558b199d9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:31.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.897+0000 7fd017fff640 1 --2- 172.21.15.84:0/480429030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b199c0f40 0x558b199d9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.897+0000 7fd017fff640 1 -- 172.21.15.84:0/480429030 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b195e90f0 con 0x558b199b34a0 2024-05-22T23:10:31.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.897+0000 7fd017fff640 1 --2- 172.21.15.84:0/480429030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 0x558b199d90e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fd00c006010 tx=0x7fd00c02a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c3fb6f6c51109b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:31.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.898+0000 7fcffe7fc640 1 -- 172.21.15.84:0/480429030 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c000e60 con 0x558b199b34a0 2024-05-22T23:10:31.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.898+0000 7fcffe7fc640 1 -- 172.21.15.84:0/480429030 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd00c005550 con 0x558b199b34a0 2024-05-22T23:10:31.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.898+0000 7fcffe7fc640 1 -- 172.21.15.84:0/480429030 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c040070 con 0x558b199b34a0 2024-05-22T23:10:31.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.898+0000 7fd021e22740 1 -- 172.21.15.84:0/480429030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 msgr2=0x558b199d90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:31.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.898+0000 7fd021e22740 1 --2- 172.21.15.84:0/480429030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 0x558b199d90e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fd00c006010 tx=0x7fd00c02a510 comp rx=0 tx=0).stop 2024-05-22T23:10:31.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.898+0000 7fd021e22740 1 -- 172.21.15.84:0/480429030 shutdown_connections 2024-05-22T23:10:31.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.898+0000 7fd021e22740 1 --2- 172.21.15.84:0/480429030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b199c0f40 0x558b199d9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.898+0000 7fd021e22740 1 --2- 172.21.15.84:0/480429030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 0x558b199d90e0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.898+0000 7fd021e22740 1 --2- 172.21.15.84:0/480429030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b199b1db0 0x558b199cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.898+0000 7fd021e22740 1 -- 172.21.15.84:0/480429030 >> 172.21.15.84:0/480429030 conn(0x558b1997a8e0 msgr2=0x558b199be6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:31.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.898+0000 7fd021e22740 1 -- 172.21.15.84:0/480429030 shutdown_connections 2024-05-22T23:10:31.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.898+0000 7fd021e22740 1 -- 172.21.15.84:0/480429030 wait complete. 2024-05-22T23:10:31.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.898+0000 7fd021e22740 1 -- start start 2024-05-22T23:10:31.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.899+0000 7fd021e22740 1 Processor -- start 2024-05-22T23:10:31.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.899+0000 7fd021e22740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b199b1db0 0x558b199bab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.899+0000 7fd021e22740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 0x558b199bb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.899+0000 7fd021e22740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b199c0f40 0x558b199bb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.899+0000 7fd021e22740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd0181aa2b0 con 0x558b199b34a0 2024-05-22T23:10:31.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.899+0000 7fd021e22740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd0181ab720 con 0x558b199c0f40 2024-05-22T23:10:31.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.899+0000 7fd021e22740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd0181aa980 con 0x558b199b1db0 2024-05-22T23:10:31.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.899+0000 7fd021e22740 10 client.0 _ll_register_callbacks cb 0x7fd00c0314d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:10:31.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.899+0000 7fd017fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 0x558b199bb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.899+0000 7fd017fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 0x558b199bb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54986/0 (socket says 172.21.15.84:54986) 2024-05-22T23:10:31.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.899+0000 7fd017fff640 1 -- 172.21.15.84:0/2683048818 learned_addr learned my addr 172.21.15.84:0/2683048818 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:10:31.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.899+0000 7fd01d5d5640 1 --2- 172.21.15.84:0/2683048818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b199c0f40 0x558b199bb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.899+0000 7fd017fff640 1 -- 172.21.15.84:0/2683048818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b199b1db0 msgr2=0x558b199bab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:31.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.899+0000 7fd017fff640 1 --2- 172.21.15.84:0/2683048818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b199b1db0 0x558b199bab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.899+0000 7fd017fff640 1 -- 172.21.15.84:0/2683048818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b199c0f40 msgr2=0x558b199bb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:10:31.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.899+0000 7fd017fff640 1 --2- 172.21.15.84:0/2683048818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b199c0f40 0x558b199bb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.899+0000 7fd017fff640 1 -- 172.21.15.84:0/2683048818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b198d4fe0 con 0x558b199b34a0 2024-05-22T23:10:31.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.899+0000 7fd01d5d5640 1 --2- 172.21.15.84:0/2683048818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b199c0f40 0x558b199bb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:10:31.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.899+0000 7fd017fff640 1 --2- 172.21.15.84:0/2683048818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 0x558b199bb070 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0039f0 tx=0x7fd00c005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dbda20cc2bdef54 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:10:31.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.899+0000 7fcff7fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:10:31.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.900+0000 7fcff7fff640 1 -- 172.21.15.84:0/2683048818 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c040040 con 0x558b199b34a0 2024-05-22T23:10:31.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.900+0000 7fcff7fff640 1 -- 172.21.15.84:0/2683048818 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7fd00c03c440 con 0x558b199b34a0 2024-05-22T23:10:31.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.900+0000 7fd021e22740 1 -- 172.21.15.84:0/2683048818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd00c005550 con 0x558b199b34a0 2024-05-22T23:10:31.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.900+0000 7fcff7fff640 1 -- 172.21.15.84:0/2683048818 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fd00c066050 con 0x558b199b34a0 2024-05-22T23:10:31.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.900+0000 7fd021e22740 10 client.16854 fetch_fsmap learned FSMap version 18 2024-05-22T23:10:31.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.900+0000 7fd021e22740 1 -- 172.21.15.84:0/2683048818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x558b198a61c0 con 0x558b199b34a0 2024-05-22T23:10:31.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.900+0000 7fcff7fff640 1 -- 172.21.15.84:0/2683048818 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd00c005550 con 0x558b199b34a0 2024-05-22T23:10:31.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.900+0000 7fcff7fff640 1 -- 172.21.15.84:0/2683048818 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fd00c061030 con 0x558b199b34a0 2024-05-22T23:10:31.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.900+0000 7fd021e22740 10 client.16854 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:10:31.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.900+0000 7fd021e22740 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:10:31.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.900+0000 7fd021e22740 20 client.16854 populate_metadata read hostname 'smithi084' 2024-05-22T23:10:31.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.900+0000 7fd021e22740 1 --2- 172.21.15.84:0/2683048818 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558b19a38060 0x558b19a3a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:10:31.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.900+0000 7fd021e22740 4 client.16854 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:10:31.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.900+0000 7fd021e22740 1 -- 172.21.15.84:0/2683048818 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558b1991d830 con 0x558b19a38060 2024-05-22T23:10:31.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.900+0000 7fd01cdd4640 1 --2- 172.21.15.84:0/2683048818 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558b19a38060 0x558b19a3a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:10:31.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.901+0000 7fcff7fff640 10 client.16854 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:10:31.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.901+0000 7fd01cdd4640 1 --2- 172.21.15.84:0/2683048818 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558b19a38060 0x558b19a3a480 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd5fff27dd550f99 server_cookie=d50a30e3cbfa38ec in_seq=0 out_seq=0 2024-05-22T23:10:31.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.901+0000 7fcff7fff640 1 -- 172.21.15.84:0/2683048818 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558b1991d830 con 0x558b19a38060 2024-05-22T23:10:31.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.901+0000 7fcff7fff640 10 client.16854 handle_command_reply: tid=0 2024-05-22T23:10:31.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.901+0000 7fd021e22740 1 client.16854 shutdown 2024-05-22T23:10:31.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.901+0000 7fd021e22740 1 -- 172.21.15.84:0/2683048818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 msgr2=0x558b199bb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:31.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.901+0000 7fd021e22740 1 --2- 172.21.15.84:0/2683048818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 0x558b199bb070 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0039f0 tx=0x7fd00c005960 comp rx=0 tx=0).stop 2024-05-22T23:10:31.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.901+0000 7fd021e22740 1 -- 172.21.15.84:0/2683048818 shutdown_connections 2024-05-22T23:10:31.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.901+0000 7fd021e22740 1 --2- 172.21.15.84:0/2683048818 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558b19a38060 0x558b19a3a480 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.901+0000 7fd021e22740 1 --2- 172.21.15.84:0/2683048818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b199c0f40 0x558b199bb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.901+0000 7fcff7fff640 0 client.16854 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:10:31.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.901+0000 7fd021e22740 1 --2- 172.21.15.84:0/2683048818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b199b34a0 0x558b199bb070 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.901+0000 7fd021e22740 1 --2- 172.21.15.84:0/2683048818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b199b1db0 0x558b199bab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.901+0000 7fd021e22740 1 -- 172.21.15.84:0/2683048818 >> 172.21.15.84:0/2683048818 conn(0x558b1997a8e0 msgr2=0x558b199b3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:31.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.901+0000 7fd021e22740 1 -- 172.21.15.84:0/2683048818 shutdown_connections 2024-05-22T23:10:31.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.901+0000 7fd021e22740 1 -- 172.21.15.84:0/2683048818 wait complete. 2024-05-22T23:10:31.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.901+0000 7fd021e22740 20 client.16854 trim_cache size 0 max 16384 2024-05-22T23:10:31.920 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:10:31.920 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-22T23:10:31.920 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": { 2024-05-22T23:10:31.920 INFO:teuthology.orchestra.run.smithi084.stdout: "f18eef90-8438-4e87-a879-cf3417aa50c4": { 2024-05-22T23:10:31.920 INFO:teuthology.orchestra.run.smithi084.stdout: "path": "/", 2024-05-22T23:10:31.920 INFO:teuthology.orchestra.run.smithi084.stdout: "tag": "f18eef90-8438-4e87-a879-cf3417aa50c4", 2024-05-22T23:10:31.920 INFO:teuthology.orchestra.run.smithi084.stdout: "options": "recursive,force" 2024-05-22T23:10:31.920 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:10:31.921 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:10:31.921 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:10:31.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.902+0000 7fd01f038640 1 -- 172.21.15.84:0/2578923566 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd0000a2d00 msgr2=0x7fd0000a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:31.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.902+0000 7fd01f038640 1 --2- 172.21.15.84:0/2578923566 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd0000a2d00 0x7fd0000a51c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd010008c60 tx=0x7fd01000d710 comp rx=0 tx=0).stop 2024-05-22T23:10:31.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.902+0000 7fd01f038640 1 -- 172.21.15.84:0/2578923566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd018102b00 msgr2=0x7fd0181981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:10:31.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.902+0000 7fd01f038640 1 --2- 172.21.15.84:0/2578923566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd018102b00 0x7fd0181981e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fd00800c970 tx=0x7fd00800ce30 comp rx=0 tx=0).stop 2024-05-22T23:10:31.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.902+0000 7fd01f038640 1 -- 172.21.15.84:0/2578923566 shutdown_connections 2024-05-22T23:10:31.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.902+0000 7fd01f038640 1 --2- 172.21.15.84:0/2578923566 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd0000a2d00 0x7fd0000a51c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.902+0000 7fd01f038640 1 --2- 172.21.15.84:0/2578923566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd01810be40 0x7fd01819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.902+0000 7fd01f038640 1 --2- 172.21.15.84:0/2578923566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd018103420 0x7fd018198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.902+0000 7fd01f038640 1 --2- 172.21.15.84:0/2578923566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd018102b00 0x7fd0181981e0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:10:31.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.902+0000 7fd01f038640 1 -- 172.21.15.84:0/2578923566 >> 172.21.15.84:0/2578923566 conn(0x7fd018076af0 msgr2=0x7fd0180fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:10:31.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.902+0000 7fd01f038640 1 -- 172.21.15.84:0/2578923566 shutdown_connections 2024-05-22T23:10:31.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:10:31.902+0000 7fd01f038640 1 -- 172.21.15.84:0/2578923566 wait complete. 2024-05-22T23:10:31.924 INFO:tasks.cephfs.filesystem:scrub status for tag:f18eef90-8438-4e87-a879-cf3417aa50c4 - {'path': '/', 'tag': 'f18eef90-8438-4e87-a879-cf3417aa50c4', 'options': 'recursive,force'} 2024-05-22T23:10:32.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:32 smithi096 ceph-mon[36392]: scrub queued for path: / 2024-05-22T23:10:32.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:32 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:10:32.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:32 smithi096 ceph-mon[36392]: scrub summary: active paths [/] 2024-05-22T23:10:32.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:32 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:10:32.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:32 smithi084 ceph-mon[32186]: scrub queued for path: / 2024-05-22T23:10:32.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:32 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:10:32.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:32 smithi084 ceph-mon[32186]: scrub summary: active paths [/] 2024-05-22T23:10:32.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:32 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:10:32.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:32 smithi204 ceph-mon[36211]: scrub queued for path: / 2024-05-22T23:10:32.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:32 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:10:32.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:32 smithi204 ceph-mon[36211]: scrub summary: active paths [/] 2024-05-22T23:10:32.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:32 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:10:33.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:33 smithi096 ceph-mon[36392]: pgmap v270: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 KiB/s wr, 3 op/s 2024-05-22T23:10:33.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:33 smithi084 ceph-mon[32186]: pgmap v270: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 KiB/s wr, 3 op/s 2024-05-22T23:10:33.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:33 smithi204 ceph-mon[36211]: pgmap v270: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 KiB/s wr, 3 op/s 2024-05-22T23:10:35.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:35 smithi096 ceph-mon[36392]: pgmap v271: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.4 KiB/s wr, 1 op/s 2024-05-22T23:10:35.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:35 smithi084 ceph-mon[32186]: pgmap v271: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.4 KiB/s wr, 1 op/s 2024-05-22T23:10:35.667 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:35 smithi204 ceph-mon[36211]: pgmap v271: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.4 KiB/s wr, 1 op/s 2024-05-22T23:10:36.305 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:36 smithi096 ceph-mon[36392]: scrub summary: idle 2024-05-22T23:10:36.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:36 smithi084 ceph-mon[32186]: scrub summary: idle 2024-05-22T23:10:36.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:36 smithi204 ceph-mon[36211]: scrub summary: idle 2024-05-22T23:10:37.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:37 smithi096 ceph-mon[36392]: pgmap v272: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.4 KiB/s wr, 4 op/s 2024-05-22T23:10:37.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:37 smithi084 ceph-mon[32186]: pgmap v272: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.4 KiB/s wr, 4 op/s 2024-05-22T23:10:37.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:37 smithi204 ceph-mon[36211]: pgmap v272: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.4 KiB/s wr, 4 op/s 2024-05-22T23:10:39.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:39 smithi096 ceph-mon[36392]: pgmap v273: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.4 KiB/s wr, 3 op/s 2024-05-22T23:10:39.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:39 smithi084 ceph-mon[32186]: pgmap v273: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.4 KiB/s wr, 3 op/s 2024-05-22T23:10:39.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:39 smithi204 ceph-mon[36211]: pgmap v273: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.4 KiB/s wr, 3 op/s 2024-05-22T23:10:41.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:41 smithi096 ceph-mon[36392]: pgmap v274: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.4 KiB/s wr, 3 op/s 2024-05-22T23:10:41.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:41 smithi084 ceph-mon[32186]: pgmap v274: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.4 KiB/s wr, 3 op/s 2024-05-22T23:10:41.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:41 smithi204 ceph-mon[36211]: pgmap v274: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.4 KiB/s wr, 3 op/s 2024-05-22T23:10:43.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:43 smithi096 ceph-mon[36392]: pgmap v275: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.4 KiB/s wr, 4 op/s 2024-05-22T23:10:43.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:43 smithi084 ceph-mon[32186]: pgmap v275: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.4 KiB/s wr, 4 op/s 2024-05-22T23:10:43.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:43 smithi204 ceph-mon[36211]: pgmap v275: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.4 KiB/s wr, 4 op/s 2024-05-22T23:10:45.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:45 smithi096 ceph-mon[36392]: pgmap v276: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3 op/s 2024-05-22T23:10:45.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:45 smithi084 ceph-mon[32186]: pgmap v276: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3 op/s 2024-05-22T23:10:45.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:45 smithi204 ceph-mon[36211]: pgmap v276: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3 op/s 2024-05-22T23:10:47.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:47 smithi096 ceph-mon[36392]: pgmap v277: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4 op/s 2024-05-22T23:10:47.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:47 smithi084 ceph-mon[32186]: pgmap v277: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4 op/s 2024-05-22T23:10:47.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:47 smithi204 ceph-mon[36211]: pgmap v277: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4 op/s 2024-05-22T23:10:49.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:49 smithi096 ceph-mon[36392]: pgmap v278: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:10:49.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:49 smithi084 ceph-mon[32186]: pgmap v278: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:10:49.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:49 smithi204 ceph-mon[36211]: pgmap v278: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:10:51.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:51 smithi096 ceph-mon[36392]: pgmap v279: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:10:51.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:51 smithi084 ceph-mon[32186]: pgmap v279: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:10:51.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:51 smithi204 ceph-mon[36211]: pgmap v279: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:10:53.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:53 smithi096 ceph-mon[36392]: pgmap v280: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:10:53.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:53 smithi084 ceph-mon[32186]: pgmap v280: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:10:53.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:53 smithi204 ceph-mon[36211]: pgmap v280: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:10:55.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:55 smithi096 ceph-mon[36392]: pgmap v281: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:10:55.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:55 smithi084 ceph-mon[32186]: pgmap v281: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:10:55.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:55 smithi204 ceph-mon[36211]: pgmap v281: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:10:57.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:57 smithi096 ceph-mon[36392]: pgmap v282: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:10:57.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:57 smithi084 ceph-mon[32186]: pgmap v282: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:10:57.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:57 smithi204 ceph-mon[36211]: pgmap v282: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:10:59.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:10:59 smithi096 ceph-mon[36392]: pgmap v283: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:10:59.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:10:59 smithi084 ceph-mon[32186]: pgmap v283: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:10:59.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:10:59 smithi204 ceph-mon[36211]: pgmap v283: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:11:01.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:01 smithi096 ceph-mon[36392]: pgmap v284: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:11:01.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:01 smithi084 ceph-mon[32186]: pgmap v284: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:11:01.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:01 smithi204 ceph-mon[36211]: pgmap v284: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:11:01.924 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:11:01.925 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:11:02.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.033+0000 7f615e75c640 1 -- 172.21.15.84:0/2346042101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6158106970 msgr2=0x7f6158106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:02.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.033+0000 7f615e75c640 1 --2- 172.21.15.84:0/2346042101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6158106970 0x7f6158106d50 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f6148009930 tx=0x7f614802a420 comp rx=0 tx=0).stop 2024-05-22T23:11:02.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.033+0000 7f615e75c640 1 -- 172.21.15.84:0/2346042101 shutdown_connections 2024-05-22T23:11:02.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.033+0000 7f615e75c640 1 --2- 172.21.15.84:0/2346042101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6158101840 0x7f615810d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.033+0000 7f615e75c640 1 --2- 172.21.15.84:0/2346042101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6158100e80 0x7f6158101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.033+0000 7f615e75c640 1 --2- 172.21.15.84:0/2346042101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6158106970 0x7f6158106d50 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.033+0000 7f615e75c640 1 -- 172.21.15.84:0/2346042101 >> 172.21.15.84:0/2346042101 conn(0x7f6158076d30 msgr2=0x7f61580fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:02.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.033+0000 7f615e75c640 1 -- 172.21.15.84:0/2346042101 shutdown_connections 2024-05-22T23:11:02.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.033+0000 7f615e75c640 1 -- 172.21.15.84:0/2346042101 wait complete. 2024-05-22T23:11:02.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.034+0000 7f615e75c640 1 Processor -- start 2024-05-22T23:11:02.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.034+0000 7f615e75c640 1 -- start start 2024-05-22T23:11:02.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.034+0000 7f615e75c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6158100e80 0x7f615819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.034+0000 7f615e75c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6158101840 0x7f615819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.034+0000 7f615e75c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6158106970 0x7f615819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.034+0000 7f615e75c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6158110d50 con 0x7f6158101840 2024-05-22T23:11:02.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.034+0000 7f615e75c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6158110bd0 con 0x7f6158100e80 2024-05-22T23:11:02.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.034+0000 7f615e75c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6158110ed0 con 0x7f6158106970 2024-05-22T23:11:02.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.035+0000 7f61577fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6158101840 0x7f615819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.035+0000 7f61577fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6158101840 0x7f615819aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34384/0 (socket says 172.21.15.84:34384) 2024-05-22T23:11:02.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.035+0000 7f61577fe640 1 -- 172.21.15.84:0/2170897075 learned_addr learned my addr 172.21.15.84:0/2170897075 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:02.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.035+0000 7f6157fff640 1 --2- 172.21.15.84:0/2170897075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6158100e80 0x7f615819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.035+0000 7f615ccf9640 1 --2- 172.21.15.84:0/2170897075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6158106970 0x7f615819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.035+0000 7f61577fe640 1 -- 172.21.15.84:0/2170897075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6158106970 msgr2=0x7f615819ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:02.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.035+0000 7f61577fe640 1 --2- 172.21.15.84:0/2170897075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6158106970 0x7f615819ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.035+0000 7f61577fe640 1 -- 172.21.15.84:0/2170897075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6158100e80 msgr2=0x7f615819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:02.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.035+0000 7f61577fe640 1 --2- 172.21.15.84:0/2170897075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6158100e80 0x7f615819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.035+0000 7f61577fe640 1 -- 172.21.15.84:0/2170897075 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615819f4c0 con 0x7f6158101840 2024-05-22T23:11:02.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.035+0000 7f615ccf9640 1 --2- 172.21.15.84:0/2170897075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6158106970 0x7f615819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:11:02.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.035+0000 7f6157fff640 1 --2- 172.21.15.84:0/2170897075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6158100e80 0x7f615819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:11:02.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.035+0000 7f61577fe640 1 --2- 172.21.15.84:0/2170897075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6158101840 0x7f615819aa10 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f614c00c920 tx=0x7f614c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:02.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.035+0000 7f61557fa640 1 -- 172.21.15.84:0/2170897075 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c007c70 con 0x7f6158101840 2024-05-22T23:11:02.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.035+0000 7f61557fa640 1 -- 172.21.15.84:0/2170897075 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f614c007e10 con 0x7f6158101840 2024-05-22T23:11:02.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.035+0000 7f615e75c640 1 -- 172.21.15.84:0/2170897075 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f615819f7b0 con 0x7f6158101840 2024-05-22T23:11:02.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.035+0000 7f61557fa640 1 -- 172.21.15.84:0/2170897075 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c018890 con 0x7f6158101840 2024-05-22T23:11:02.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.035+0000 7f615e75c640 1 -- 172.21.15.84:0/2170897075 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61581a7040 con 0x7f6158101840 2024-05-22T23:11:02.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.037+0000 7f61557fa640 1 -- 172.21.15.84:0/2170897075 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f614c018a70 con 0x7f6158101840 2024-05-22T23:11:02.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.038+0000 7f6161460740 1 Processor -- start 2024-05-22T23:11:02.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.038+0000 7f6161460740 1 -- start start 2024-05-22T23:11:02.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.038+0000 7f6161460740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb0081a4a0 0x55bb008187e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.038+0000 7f6161460740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb00818db0 0x55bb00819230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.038+0000 7f6161460740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb00823b30 0x55bb00825f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.038+0000 7f6161460740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55bb0077cdb0 con 0x55bb00818db0 2024-05-22T23:11:02.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.038+0000 7f6161460740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bb0071b360 con 0x55bb00823b30 2024-05-22T23:11:02.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.038+0000 7f6161460740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bb00778d70 con 0x55bb0081a4a0 2024-05-22T23:11:02.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.038+0000 7f6157fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb00818db0 0x55bb00819230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.038+0000 7f61557fa640 1 --2- 172.21.15.84:0/2170897075 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f61400a2d80 0x7f61400a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.038+0000 7f6157fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb00818db0 0x55bb00819230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34392/0 (socket says 172.21.15.84:34392) 2024-05-22T23:11:02.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.038+0000 7f6157fff640 1 -- 172.21.15.84:0/1877232118 learned_addr learned my addr 172.21.15.84:0/1877232118 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:02.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.038+0000 7f61577fe640 1 --2- 172.21.15.84:0/1877232118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb00823b30 0x55bb00825f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.038+0000 7f615ccf9640 1 --2- 172.21.15.84:0/2170897075 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f61400a2d80 0x7f61400a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.038+0000 7f615ccf9640 1 --2- 172.21.15.84:0/1877232118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb0081a4a0 0x55bb008187e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.038+0000 7f6157fff640 1 -- 172.21.15.84:0/1877232118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb0081a4a0 msgr2=0x55bb008187e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:02.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.038+0000 7f6157fff640 1 --2- 172.21.15.84:0/1877232118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb0081a4a0 0x55bb008187e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.038+0000 7f6157fff640 1 -- 172.21.15.84:0/1877232118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb00823b30 msgr2=0x55bb00825f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:02.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.038+0000 7f6157fff640 1 --2- 172.21.15.84:0/1877232118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb00823b30 0x55bb00825f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.038+0000 7f61557fa640 1 -- 172.21.15.84:0/2170897075 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7f614c0cf5e0 con 0x7f6158101840 2024-05-22T23:11:02.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.038+0000 7f6157fff640 1 -- 172.21.15.84:0/1877232118 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb0073ea60 con 0x55bb00818db0 2024-05-22T23:11:02.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.038+0000 7f615ccf9640 1 --2- 172.21.15.84:0/1877232118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb0081a4a0 0x55bb008187e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:11:02.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.039+0000 7f6157fff640 1 --2- 172.21.15.84:0/1877232118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb00818db0 0x55bb00819230 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6148005ab0 tx=0x7f6148009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfa5cd867e45fbb server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:02.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.039+0000 7f615ccf9640 1 --2- 172.21.15.84:0/2170897075 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f61400a2d80 0x7f61400a5240 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f615000af30 tx=0x7f615000a360 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:02.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.039+0000 7f613e7fc640 1 -- 172.21.15.84:0/1877232118 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614803f070 con 0x55bb00818db0 2024-05-22T23:11:02.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.039+0000 7f613e7fc640 1 -- 172.21.15.84:0/1877232118 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61480317b0 con 0x55bb00818db0 2024-05-22T23:11:02.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.039+0000 7f613e7fc640 1 -- 172.21.15.84:0/1877232118 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148035070 con 0x55bb00818db0 2024-05-22T23:11:02.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.039+0000 7f6161460740 1 -- 172.21.15.84:0/1877232118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb00818db0 msgr2=0x55bb00819230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:02.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.039+0000 7f6161460740 1 --2- 172.21.15.84:0/1877232118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb00818db0 0x55bb00819230 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6148005ab0 tx=0x7f6148009680 comp rx=0 tx=0).stop 2024-05-22T23:11:02.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.040+0000 7f6161460740 1 -- 172.21.15.84:0/1877232118 shutdown_connections 2024-05-22T23:11:02.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.040+0000 7f6161460740 1 --2- 172.21.15.84:0/1877232118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb00823b30 0x55bb00825f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.040+0000 7f6161460740 1 --2- 172.21.15.84:0/1877232118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb00818db0 0x55bb00819230 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.040+0000 7f6161460740 1 --2- 172.21.15.84:0/1877232118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb0081a4a0 0x55bb008187e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.040+0000 7f6161460740 1 -- 172.21.15.84:0/1877232118 >> 172.21.15.84:0/1877232118 conn(0x55bb007e18e0 msgr2=0x55bb00815310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:02.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.040+0000 7f6161460740 1 -- 172.21.15.84:0/1877232118 shutdown_connections 2024-05-22T23:11:02.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.040+0000 7f6161460740 1 -- 172.21.15.84:0/1877232118 wait complete. 2024-05-22T23:11:02.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.040+0000 7f6161460740 1 -- start start 2024-05-22T23:11:02.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.040+0000 7f6161460740 1 Processor -- start 2024-05-22T23:11:02.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.040+0000 7f6161460740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb00818db0 0x55bb00820980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.040+0000 7f6161460740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb0081a4a0 0x55bb00820ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.040+0000 7f6161460740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb00823b30 0x55bb00821400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.040+0000 7f6161460740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55bb00826960 con 0x55bb00818db0 2024-05-22T23:11:02.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.040+0000 7f6161460740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bb007370f0 con 0x55bb0081a4a0 2024-05-22T23:11:02.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.040+0000 7f6161460740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bb00826ae0 con 0x55bb00823b30 2024-05-22T23:11:02.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.040+0000 7f6161460740 10 client.0 _ll_register_callbacks cb 0x55bb0073bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:11:02.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.041+0000 7f6157fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb00818db0 0x55bb00820980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.041+0000 7f6157fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb00818db0 0x55bb00820980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34398/0 (socket says 172.21.15.84:34398) 2024-05-22T23:11:02.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.041+0000 7f6157fff640 1 -- 172.21.15.84:0/192068083 learned_addr learned my addr 172.21.15.84:0/192068083 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:02.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.041+0000 7f61577fe640 1 --2- 172.21.15.84:0/192068083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb0081a4a0 0x55bb00820ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.041+0000 7f615ccf9640 1 --2- 172.21.15.84:0/192068083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb00823b30 0x55bb00821400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.041+0000 7f6157fff640 1 -- 172.21.15.84:0/192068083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb00823b30 msgr2=0x55bb00821400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:02.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.041+0000 7f6157fff640 1 --2- 172.21.15.84:0/192068083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb00823b30 0x55bb00821400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.041+0000 7f6157fff640 1 -- 172.21.15.84:0/192068083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb0081a4a0 msgr2=0x55bb00820ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:02.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.041+0000 7f6157fff640 1 --2- 172.21.15.84:0/192068083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb0081a4a0 0x55bb00820ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.041+0000 7f6157fff640 1 -- 172.21.15.84:0/192068083 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb0055a7d0 con 0x55bb00818db0 2024-05-22T23:11:02.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.041+0000 7f615ccf9640 1 --2- 172.21.15.84:0/192068083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb00823b30 0x55bb00821400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:11:02.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.041+0000 7f61577fe640 1 --2- 172.21.15.84:0/192068083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb0081a4a0 0x55bb00820ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:11:02.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.041+0000 7f6157fff640 1 --2- 172.21.15.84:0/192068083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb00818db0 0x55bb00820980 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f6148031550 tx=0x7f6148005340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef66ae9cbf4bf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:02.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.041+0000 7f61377fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:11:02.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.041+0000 7f61377fe640 1 -- 172.21.15.84:0/192068083 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614803f050 con 0x55bb00818db0 2024-05-22T23:11:02.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.041+0000 7f61377fe640 1 -- 172.21.15.84:0/192068083 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7f6148004240 con 0x55bb00818db0 2024-05-22T23:11:02.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.041+0000 7f6161460740 1 -- 172.21.15.84:0/192068083 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61480317b0 con 0x55bb00818db0 2024-05-22T23:11:02.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.041+0000 7f61377fe640 1 -- 172.21.15.84:0/192068083 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f6148035040 con 0x55bb00818db0 2024-05-22T23:11:02.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.041+0000 7f6161460740 10 client.16866 fetch_fsmap learned FSMap version 18 2024-05-22T23:11:02.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.041+0000 7f6161460740 1 -- 172.21.15.84:0/192068083 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55bb004903f0 con 0x55bb00818db0 2024-05-22T23:11:02.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.041+0000 7f61377fe640 1 -- 172.21.15.84:0/192068083 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61480317b0 con 0x55bb00818db0 2024-05-22T23:11:02.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.042+0000 7f61377fe640 1 -- 172.21.15.84:0/192068083 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f6148054070 con 0x55bb00818db0 2024-05-22T23:11:02.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.042+0000 7f6161460740 10 client.16866 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:11:02.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.042+0000 7f6161460740 10 client.16866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:11:02.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.042+0000 7f6161460740 20 client.16866 populate_metadata read hostname 'smithi084' 2024-05-22T23:11:02.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.042+0000 7f6161460740 1 --2- 172.21.15.84:0/192068083 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55bb00833cd0 0x55bb008360c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.042+0000 7f6161460740 4 client.16866 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:11:02.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.042+0000 7f6161460740 1 -- 172.21.15.84:0/192068083 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb0078a4e0 con 0x55bb00833cd0 2024-05-22T23:11:02.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.042+0000 7f61577fe640 1 --2- 172.21.15.84:0/192068083 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55bb00833cd0 0x55bb008360c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.043+0000 7f61377fe640 10 client.16866 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:02.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.043+0000 7f61577fe640 1 --2- 172.21.15.84:0/192068083 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55bb00833cd0 0x55bb008360c0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27c3ad74a0536d43 server_cookie=e6085a6ac4b26060 in_seq=0 out_seq=0 2024-05-22T23:11:02.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.043+0000 7f61377fe640 1 -- 172.21.15.84:0/192068083 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bb0078a4e0 con 0x55bb00833cd0 2024-05-22T23:11:02.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.043+0000 7f61377fe640 10 client.16866 handle_command_reply: tid=0 2024-05-22T23:11:02.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.043+0000 7f6161460740 1 client.16866 shutdown 2024-05-22T23:11:02.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.043+0000 7f6161460740 1 -- 172.21.15.84:0/192068083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb00818db0 msgr2=0x55bb00820980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:02.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.043+0000 7f6161460740 1 --2- 172.21.15.84:0/192068083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb00818db0 0x55bb00820980 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f6148031550 tx=0x7f6148005340 comp rx=0 tx=0).stop 2024-05-22T23:11:02.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.043+0000 7f6161460740 1 -- 172.21.15.84:0/192068083 shutdown_connections 2024-05-22T23:11:02.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.043+0000 7f6161460740 1 --2- 172.21.15.84:0/192068083 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55bb00833cd0 0x55bb008360c0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.043+0000 7f6161460740 1 --2- 172.21.15.84:0/192068083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb00823b30 0x55bb00821400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.043+0000 7f61377fe640 0 client.16866 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:02.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.043+0000 7f6161460740 1 --2- 172.21.15.84:0/192068083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb0081a4a0 0x55bb00820ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.043+0000 7f6161460740 1 --2- 172.21.15.84:0/192068083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb00818db0 0x55bb00820980 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.043+0000 7f6161460740 1 -- 172.21.15.84:0/192068083 >> 172.21.15.84:0/192068083 conn(0x55bb007e18e0 msgr2=0x55bb0081b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:02.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.044+0000 7f6161460740 1 -- 172.21.15.84:0/192068083 shutdown_connections 2024-05-22T23:11:02.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.044+0000 7f6161460740 1 -- 172.21.15.84:0/192068083 wait complete. 2024-05-22T23:11:02.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.044+0000 7f6161460740 20 client.16866 trim_cache size 0 max 16384 2024-05-22T23:11:02.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.061+0000 7f6161460740 1 Processor -- start 2024-05-22T23:11:02.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.061+0000 7f6161460740 1 -- start start 2024-05-22T23:11:02.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.061+0000 7f6161460740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb00818db0 0x55bb0082f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.061+0000 7f6161460740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb0081a4a0 0x55bb0081f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.061+0000 7f6161460740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb00823b30 0x55bb0081f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.061+0000 7f6161460740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f61581ad760 con 0x55bb0081a4a0 2024-05-22T23:11:02.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.061+0000 7f6161460740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f61581affc0 con 0x55bb00818db0 2024-05-22T23:11:02.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.061+0000 7f6161460740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f61581addb0 con 0x55bb00823b30 2024-05-22T23:11:02.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.061+0000 7f61577fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb0081a4a0 0x55bb0081f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.061+0000 7f61577fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb0081a4a0 0x55bb0081f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34408/0 (socket says 172.21.15.84:34408) 2024-05-22T23:11:02.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.061+0000 7f61577fe640 1 -- 172.21.15.84:0/964993534 learned_addr learned my addr 172.21.15.84:0/964993534 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:02.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.061+0000 7f6157fff640 1 --2- 172.21.15.84:0/964993534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb00818db0 0x55bb0082f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.061+0000 7f61577fe640 1 -- 172.21.15.84:0/964993534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb00823b30 msgr2=0x55bb0081f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:02.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.061+0000 7f61577fe640 1 --2- 172.21.15.84:0/964993534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb00823b30 0x55bb0081f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.061+0000 7f61577fe640 1 -- 172.21.15.84:0/964993534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb00818db0 msgr2=0x55bb0082f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:02.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.061+0000 7f61577fe640 1 --2- 172.21.15.84:0/964993534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb00818db0 0x55bb0082f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.061+0000 7f61577fe640 1 -- 172.21.15.84:0/964993534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb004500f0 con 0x55bb0081a4a0 2024-05-22T23:11:02.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.061+0000 7f61577fe640 1 --2- 172.21.15.84:0/964993534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb0081a4a0 0x55bb0081f140 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f614c0ccd60 tx=0x7f614c06b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660b81de62c9d516 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:02.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.062+0000 7f613dffb640 1 -- 172.21.15.84:0/964993534 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c09bad0 con 0x55bb0081a4a0 2024-05-22T23:11:02.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.062+0000 7f613dffb640 1 -- 172.21.15.84:0/964993534 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f614c00de00 con 0x55bb0081a4a0 2024-05-22T23:11:02.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.062+0000 7f613dffb640 1 -- 172.21.15.84:0/964993534 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c0ccf80 con 0x55bb0081a4a0 2024-05-22T23:11:02.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.062+0000 7f6161460740 1 -- 172.21.15.84:0/964993534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb0081a4a0 msgr2=0x55bb0081f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:02.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.062+0000 7f6161460740 1 --2- 172.21.15.84:0/964993534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb0081a4a0 0x55bb0081f140 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f614c0ccd60 tx=0x7f614c06b930 comp rx=0 tx=0).stop 2024-05-22T23:11:02.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.062+0000 7f6161460740 1 -- 172.21.15.84:0/964993534 shutdown_connections 2024-05-22T23:11:02.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.062+0000 7f6161460740 1 --2- 172.21.15.84:0/964993534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb00823b30 0x55bb0081f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.062+0000 7f6161460740 1 --2- 172.21.15.84:0/964993534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb0081a4a0 0x55bb0081f140 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.062+0000 7f6161460740 1 --2- 172.21.15.84:0/964993534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb00818db0 0x55bb0082f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.062+0000 7f6161460740 1 -- 172.21.15.84:0/964993534 >> 172.21.15.84:0/964993534 conn(0x55bb007e18e0 msgr2=0x55bb00825b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.062+0000 7f6161460740 1 -- 172.21.15.84:0/964993534 shutdown_connections 2024-05-22T23:11:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.062+0000 7f6161460740 1 -- 172.21.15.84:0/964993534 wait complete. 2024-05-22T23:11:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.063+0000 7f6161460740 1 -- start start 2024-05-22T23:11:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.063+0000 7f6161460740 1 Processor -- start 2024-05-22T23:11:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.063+0000 7f6161460740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb00818db0 0x55bb0081ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.063+0000 7f6161460740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb0081a4a0 0x55bb0081d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.063+0000 7f6161460740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb00823b30 0x55bb0081d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.063+0000 7f6161460740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f61581ae480 con 0x55bb0081a4a0 2024-05-22T23:11:02.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.063+0000 7f6161460740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f61581af8f0 con 0x55bb00823b30 2024-05-22T23:11:02.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.063+0000 7f6161460740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f61581aeb50 con 0x55bb00818db0 2024-05-22T23:11:02.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.063+0000 7f6161460740 10 client.0 _ll_register_callbacks cb 0x7f6148032cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:11:02.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.063+0000 7f61577fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb0081a4a0 0x55bb0081d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.063+0000 7f61577fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb0081a4a0 0x55bb0081d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34410/0 (socket says 172.21.15.84:34410) 2024-05-22T23:11:02.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.063+0000 7f61577fe640 1 -- 172.21.15.84:0/1375596038 learned_addr learned my addr 172.21.15.84:0/1375596038 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:02.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.063+0000 7f615ccf9640 1 --2- 172.21.15.84:0/1375596038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb00823b30 0x55bb0081d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.063+0000 7f6157fff640 1 --2- 172.21.15.84:0/1375596038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb00818db0 0x55bb0081ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.063+0000 7f61577fe640 1 -- 172.21.15.84:0/1375596038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb00818db0 msgr2=0x55bb0081ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:02.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.063+0000 7f61577fe640 1 --2- 172.21.15.84:0/1375596038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb00818db0 0x55bb0081ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.063+0000 7f61577fe640 1 -- 172.21.15.84:0/1375596038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb00823b30 msgr2=0x55bb0081d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:02.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.063+0000 7f61577fe640 1 --2- 172.21.15.84:0/1375596038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb00823b30 0x55bb0081d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.063+0000 7f61577fe640 1 -- 172.21.15.84:0/1375596038 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb0073bfe0 con 0x55bb0081a4a0 2024-05-22T23:11:02.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.063+0000 7f6157fff640 1 --2- 172.21.15.84:0/1375596038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb00818db0 0x55bb0081ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:11:02.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.064+0000 7f61577fe640 1 --2- 172.21.15.84:0/1375596038 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb0081a4a0 0x55bb0081d350 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f614c09ba60 tx=0x7f614c0a48c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35319fc0c518aaa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:02.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.064+0000 7f61377fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:11:02.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.064+0000 7f61377fe640 1 -- 172.21.15.84:0/1375596038 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c018d60 con 0x55bb0081a4a0 2024-05-22T23:11:02.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.064+0000 7f61377fe640 1 -- 172.21.15.84:0/1375596038 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7f614c09ce00 con 0x55bb0081a4a0 2024-05-22T23:11:02.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.064+0000 7f6161460740 1 -- 172.21.15.84:0/1375596038 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f614c00de00 con 0x55bb0081a4a0 2024-05-22T23:11:02.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.064+0000 7f61377fe640 1 -- 172.21.15.84:0/1375596038 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f614c00fb30 con 0x55bb0081a4a0 2024-05-22T23:11:02.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.064+0000 7f6161460740 10 client.16872 fetch_fsmap learned FSMap version 18 2024-05-22T23:11:02.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.064+0000 7f6161460740 1 -- 172.21.15.84:0/1375596038 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55bb0070d1c0 con 0x55bb0081a4a0 2024-05-22T23:11:02.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.064+0000 7f61377fe640 1 -- 172.21.15.84:0/1375596038 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f614c00de00 con 0x55bb0081a4a0 2024-05-22T23:11:02.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.064+0000 7f61377fe640 1 -- 172.21.15.84:0/1375596038 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f614c0ba370 con 0x55bb0081a4a0 2024-05-22T23:11:02.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.064+0000 7f6161460740 10 client.16872 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:11:02.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.064+0000 7f6161460740 10 client.16872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:11:02.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.064+0000 7f6161460740 20 client.16872 populate_metadata read hostname 'smithi084' 2024-05-22T23:11:02.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.064+0000 7f6161460740 1 --2- 172.21.15.84:0/1375596038 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55bb00899f60 0x55bb0089c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.064+0000 7f6161460740 4 client.16872 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:11:02.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.064+0000 7f6161460740 1 -- 172.21.15.84:0/1375596038 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bb00784830 con 0x55bb00899f60 2024-05-22T23:11:02.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.064+0000 7f6157fff640 1 --2- 172.21.15.84:0/1375596038 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55bb00899f60 0x55bb0089c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.065+0000 7f6157fff640 1 --2- 172.21.15.84:0/1375596038 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55bb00899f60 0x55bb0089c350 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e0f09228205fe04 server_cookie=edb8435081b96cbf in_seq=0 out_seq=0 2024-05-22T23:11:02.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.065+0000 7f61377fe640 10 client.16872 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:02.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.065+0000 7f61377fe640 1 -- 172.21.15.84:0/1375596038 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bb00784830 con 0x55bb00899f60 2024-05-22T23:11:02.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.065+0000 7f61377fe640 10 client.16872 handle_command_reply: tid=0 2024-05-22T23:11:02.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.065+0000 7f6161460740 1 client.16872 shutdown 2024-05-22T23:11:02.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.065+0000 7f6161460740 1 -- 172.21.15.84:0/1375596038 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb0081a4a0 msgr2=0x55bb0081d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:02.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.065+0000 7f6161460740 1 --2- 172.21.15.84:0/1375596038 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb0081a4a0 0x55bb0081d350 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f614c09ba60 tx=0x7f614c0a48c0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.065+0000 7f6161460740 1 -- 172.21.15.84:0/1375596038 shutdown_connections 2024-05-22T23:11:02.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.065+0000 7f6161460740 1 --2- 172.21.15.84:0/1375596038 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55bb00899f60 0x55bb0089c350 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.065+0000 7f6161460740 1 --2- 172.21.15.84:0/1375596038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb00823b30 0x55bb0081d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.065+0000 7f6161460740 1 --2- 172.21.15.84:0/1375596038 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bb0081a4a0 0x55bb0081d350 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.065+0000 7f6161460740 1 --2- 172.21.15.84:0/1375596038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb00818db0 0x55bb0081ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.065+0000 7f61377fe640 0 client.16872 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:02.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.065+0000 7f6161460740 1 -- 172.21.15.84:0/1375596038 >> 172.21.15.84:0/1375596038 conn(0x55bb007e18e0 msgr2=0x55bb00822a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:02.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.065+0000 7f6161460740 1 -- 172.21.15.84:0/1375596038 shutdown_connections 2024-05-22T23:11:02.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.065+0000 7f6161460740 1 -- 172.21.15.84:0/1375596038 wait complete. 2024-05-22T23:11:02.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.065+0000 7f6161460740 20 client.16872 trim_cache size 0 max 16384 2024-05-22T23:11:02.085 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:11:02.085 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "no active scrubs running", 2024-05-22T23:11:02.085 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": {} 2024-05-22T23:11:02.085 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:11:02.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.066+0000 7f615e75c640 1 -- 172.21.15.84:0/2170897075 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f61400a2d80 msgr2=0x7f61400a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:02.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.066+0000 7f615e75c640 1 --2- 172.21.15.84:0/2170897075 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f61400a2d80 0x7f61400a5240 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f615000af30 tx=0x7f615000a360 comp rx=0 tx=0).stop 2024-05-22T23:11:02.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.066+0000 7f615e75c640 1 -- 172.21.15.84:0/2170897075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6158101840 msgr2=0x7f615819aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:02.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.066+0000 7f615e75c640 1 --2- 172.21.15.84:0/2170897075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6158101840 0x7f615819aa10 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f614c00c920 tx=0x7f614c00cde0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.066+0000 7f615e75c640 1 -- 172.21.15.84:0/2170897075 shutdown_connections 2024-05-22T23:11:02.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.066+0000 7f615e75c640 1 --2- 172.21.15.84:0/2170897075 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f61400a2d80 0x7f61400a5240 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.066+0000 7f615e75c640 1 --2- 172.21.15.84:0/2170897075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6158106970 0x7f615819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.066+0000 7f615e75c640 1 --2- 172.21.15.84:0/2170897075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6158101840 0x7f615819aa10 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.066+0000 7f615e75c640 1 --2- 172.21.15.84:0/2170897075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6158100e80 0x7f615819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.066+0000 7f615e75c640 1 -- 172.21.15.84:0/2170897075 >> 172.21.15.84:0/2170897075 conn(0x7f6158076d30 msgr2=0x7f61580ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:02.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.066+0000 7f615e75c640 1 -- 172.21.15.84:0/2170897075 shutdown_connections 2024-05-22T23:11:02.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.066+0000 7f615e75c640 1 -- 172.21.15.84:0/2170897075 wait complete. 2024-05-22T23:11:02.088 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-22T23:11:02.089 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:11:02.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.184+0000 7fd065a1c640 1 -- 172.21.15.84:0/2386111013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd06010e280 msgr2=0x7fd060110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:02.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.184+0000 7fd065a1c640 1 --2- 172.21.15.84:0/2386111013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd06010e280 0x7fd060110670 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fd058009ca0 tx=0x7fd05802a8c0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.184+0000 7fd065a1c640 1 -- 172.21.15.84:0/2386111013 shutdown_connections 2024-05-22T23:11:02.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.184+0000 7fd065a1c640 1 --2- 172.21.15.84:0/2386111013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd06010e280 0x7fd060110670 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.184+0000 7fd065a1c640 1 --2- 172.21.15.84:0/2386111013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd060100ff0 0x7fd06010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.184+0000 7fd065a1c640 1 --2- 172.21.15.84:0/2386111013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0601006d0 0x7fd060100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.184+0000 7fd065a1c640 1 -- 172.21.15.84:0/2386111013 >> 172.21.15.84:0/2386111013 conn(0x7fd0600778d0 msgr2=0x7fd060077ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:02.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.184+0000 7fd065a1c640 1 -- 172.21.15.84:0/2386111013 shutdown_connections 2024-05-22T23:11:02.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.184+0000 7fd065a1c640 1 -- 172.21.15.84:0/2386111013 wait complete. 2024-05-22T23:11:02.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.184+0000 7fd065a1c640 1 Processor -- start 2024-05-22T23:11:02.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.184+0000 7fd065a1c640 1 -- start start 2024-05-22T23:11:02.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.185+0000 7fd065a1c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0601006d0 0x7fd06019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.185+0000 7fd065a1c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd060100ff0 0x7fd06019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.185+0000 7fd065a1c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd06010e280 0x7fd06019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.185+0000 7fd065a1c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd060113e90 con 0x7fd06010e280 2024-05-22T23:11:02.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.185+0000 7fd065a1c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd060113d10 con 0x7fd060100ff0 2024-05-22T23:11:02.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.185+0000 7fd065a1c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd060114010 con 0x7fd0601006d0 2024-05-22T23:11:02.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.185+0000 7fd05f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd06010e280 0x7fd06019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.185+0000 7fd05f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd06010e280 0x7fd06019ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34420/0 (socket says 172.21.15.84:34420) 2024-05-22T23:11:02.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.185+0000 7fd05f7fe640 1 -- 172.21.15.84:0/3919999333 learned_addr learned my addr 172.21.15.84:0/3919999333 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:02.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.185+0000 7fd05effd640 1 --2- 172.21.15.84:0/3919999333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0601006d0 0x7fd06019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.185+0000 7fd05f7fe640 1 -- 172.21.15.84:0/3919999333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0601006d0 msgr2=0x7fd06019a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:02.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.185+0000 7fd05f7fe640 1 --2- 172.21.15.84:0/3919999333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0601006d0 0x7fd06019a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.185+0000 7fd05f7fe640 1 -- 172.21.15.84:0/3919999333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd060100ff0 msgr2=0x7fd06019a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:02.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.185+0000 7fd0567fc640 1 --2- 172.21.15.84:0/3919999333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd060100ff0 0x7fd06019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.185+0000 7fd05f7fe640 1 --2- 172.21.15.84:0/3919999333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd060100ff0 0x7fd06019a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.185+0000 7fd05f7fe640 1 -- 172.21.15.84:0/3919999333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06019f2e0 con 0x7fd06010e280 2024-05-22T23:11:02.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.185+0000 7fd05f7fe640 1 --2- 172.21.15.84:0/3919999333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd06010e280 0x7fd06019ebb0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fd05802ade0 tx=0x7fd058031940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:02.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.185+0000 7fd064a1a640 1 -- 172.21.15.84:0/3919999333 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058003b70 con 0x7fd06010e280 2024-05-22T23:11:02.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.185+0000 7fd064a1a640 1 -- 172.21.15.84:0/3919999333 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd058003d10 con 0x7fd06010e280 2024-05-22T23:11:02.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.185+0000 7fd065a1c640 1 -- 172.21.15.84:0/3919999333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd06019f570 con 0x7fd06010e280 2024-05-22T23:11:02.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.185+0000 7fd064a1a640 1 -- 172.21.15.84:0/3919999333 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05802fbd0 con 0x7fd06010e280 2024-05-22T23:11:02.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.186+0000 7fd065a1c640 1 -- 172.21.15.84:0/3919999333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0601a6e60 con 0x7fd06010e280 2024-05-22T23:11:02.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.187+0000 7fd065a1c640 1 -- 172.21.15.84:0/3919999333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd060067e00 con 0x7fd06010e280 2024-05-22T23:11:02.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.187+0000 7fd064a1a640 1 -- 172.21.15.84:0/3919999333 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd05802fd70 con 0x7fd06010e280 2024-05-22T23:11:02.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.188+0000 7fd064a1a640 1 --2- 172.21.15.84:0/3919999333 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd04c0a2d00 0x7fd04c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.188+0000 7fd05effd640 1 --2- 172.21.15.84:0/3919999333 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd04c0a2d00 0x7fd04c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.188+0000 7fd064a1a640 1 -- 172.21.15.84:0/3919999333 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7fd0580f04f0 con 0x7fd06010e280 2024-05-22T23:11:02.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.189+0000 7fd05effd640 1 --2- 172.21.15.84:0/3919999333 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd04c0a2d00 0x7fd04c0a51c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd050009b60 tx=0x7fd050005f90 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:02.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.192+0000 7fd064a1a640 1 -- 172.21.15.84:0/3919999333 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd0580bd1f0 con 0x7fd06010e280 2024-05-22T23:11:02.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.408+0000 7fd065a1c640 1 -- 172.21.15.84:0/3919999333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd060109410 con 0x7fd06010e280 2024-05-22T23:11:02.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.409+0000 7fd064a1a640 1 -- 172.21.15.84:0/3919999333 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7fd0580bcaf0 con 0x7fd06010e280 2024-05-22T23:11:02.415 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:11:02.415 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:11:02.415 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:11:02.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.411+0000 7fd065a1c640 1 -- 172.21.15.84:0/3919999333 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd04c0a2d00 msgr2=0x7fd04c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:02.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.411+0000 7fd065a1c640 1 --2- 172.21.15.84:0/3919999333 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd04c0a2d00 0x7fd04c0a51c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd050009b60 tx=0x7fd050005f90 comp rx=0 tx=0).stop 2024-05-22T23:11:02.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.411+0000 7fd065a1c640 1 -- 172.21.15.84:0/3919999333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd06010e280 msgr2=0x7fd06019ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:02.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.411+0000 7fd065a1c640 1 --2- 172.21.15.84:0/3919999333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd06010e280 0x7fd06019ebb0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fd05802ade0 tx=0x7fd058031940 comp rx=0 tx=0).stop 2024-05-22T23:11:02.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.411+0000 7fd065a1c640 1 -- 172.21.15.84:0/3919999333 shutdown_connections 2024-05-22T23:11:02.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.411+0000 7fd065a1c640 1 --2- 172.21.15.84:0/3919999333 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd04c0a2d00 0x7fd04c0a51c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.411+0000 7fd065a1c640 1 --2- 172.21.15.84:0/3919999333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd06010e280 0x7fd06019ebb0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.411+0000 7fd065a1c640 1 --2- 172.21.15.84:0/3919999333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd060100ff0 0x7fd06019a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.411+0000 7fd065a1c640 1 --2- 172.21.15.84:0/3919999333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0601006d0 0x7fd06019a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.411+0000 7fd065a1c640 1 -- 172.21.15.84:0/3919999333 >> 172.21.15.84:0/3919999333 conn(0x7fd0600778d0 msgr2=0x7fd06010f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:02.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.411+0000 7fd065a1c640 1 -- 172.21.15.84:0/3919999333 shutdown_connections 2024-05-22T23:11:02.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.411+0000 7fd065a1c640 1 -- 172.21.15.84:0/3919999333 wait complete. 2024-05-22T23:11:02.428 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-22T23:11:02.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.524+0000 7fa34a26d640 1 -- 172.21.15.84:0/2348047586 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3441067c0 msgr2=0x7fa344106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:02.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.524+0000 7fa34a26d640 1 --2- 172.21.15.84:0/2348047586 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3441067c0 0x7fa344106ba0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fa334009900 tx=0x7fa33402a410 comp rx=0 tx=0).stop 2024-05-22T23:11:02.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.525+0000 7fa34a26d640 1 -- 172.21.15.84:0/2348047586 shutdown_connections 2024-05-22T23:11:02.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.525+0000 7fa34a26d640 1 --2- 172.21.15.84:0/2348047586 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa344101460 0x7fa34410d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.525+0000 7fa34a26d640 1 --2- 172.21.15.84:0/2348047586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa344100ac0 0x7fa344100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.525+0000 7fa34a26d640 1 --2- 172.21.15.84:0/2348047586 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3441067c0 0x7fa344106ba0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.525+0000 7fa34a26d640 1 -- 172.21.15.84:0/2348047586 >> 172.21.15.84:0/2348047586 conn(0x7fa344076570 msgr2=0x7fa344076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:02.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.525+0000 7fa34a26d640 1 -- 172.21.15.84:0/2348047586 shutdown_connections 2024-05-22T23:11:02.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.525+0000 7fa34a26d640 1 -- 172.21.15.84:0/2348047586 wait complete. 2024-05-22T23:11:02.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.525+0000 7fa34a26d640 1 Processor -- start 2024-05-22T23:11:02.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.525+0000 7fa34a26d640 1 -- start start 2024-05-22T23:11:02.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.525+0000 7fa34a26d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa344100ac0 0x7fa344198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.526+0000 7fa34a26d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa344101460 0x7fa3441993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.526+0000 7fa34a26d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3441067c0 0x7fa3441a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.526+0000 7fa34a26d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa344110b10 con 0x7fa3441067c0 2024-05-22T23:11:02.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.526+0000 7fa34a26d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa344110990 con 0x7fa344101460 2024-05-22T23:11:02.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.526+0000 7fa34a26d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa344110c90 con 0x7fa344100ac0 2024-05-22T23:11:02.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.526+0000 7fa34880a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3441067c0 0x7fa3441a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.526+0000 7fa34880a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3441067c0 0x7fa3441a3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34440/0 (socket says 172.21.15.84:34440) 2024-05-22T23:11:02.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.526+0000 7fa34880a640 1 -- 172.21.15.84:0/2895972211 learned_addr learned my addr 172.21.15.84:0/2895972211 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:02.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.526+0000 7fa3437fe640 1 --2- 172.21.15.84:0/2895972211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa344101460 0x7fa3441993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.526+0000 7fa343fff640 1 --2- 172.21.15.84:0/2895972211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa344100ac0 0x7fa344198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.526+0000 7fa34880a640 1 -- 172.21.15.84:0/2895972211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa344100ac0 msgr2=0x7fa344198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:02.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.526+0000 7fa34880a640 1 --2- 172.21.15.84:0/2895972211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa344100ac0 0x7fa344198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.526+0000 7fa34880a640 1 -- 172.21.15.84:0/2895972211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa344101460 msgr2=0x7fa3441993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:02.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.526+0000 7fa34880a640 1 --2- 172.21.15.84:0/2895972211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa344101460 0x7fa3441993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.526+0000 7fa34880a640 1 -- 172.21.15.84:0/2895972211 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3441a4430 con 0x7fa3441067c0 2024-05-22T23:11:02.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.526+0000 7fa34880a640 1 --2- 172.21.15.84:0/2895972211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3441067c0 0x7fa3441a3ef0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00cce0 tx=0x7fa33c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:02.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.526+0000 7fa3417fa640 1 -- 172.21.15.84:0/2895972211 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c00f450 con 0x7fa3441067c0 2024-05-22T23:11:02.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.526+0000 7fa34a26d640 1 -- 172.21.15.84:0/2895972211 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3441a4600 con 0x7fa3441067c0 2024-05-22T23:11:02.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.526+0000 7fa3417fa640 1 -- 172.21.15.84:0/2895972211 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa33c00fad0 con 0x7fa3441067c0 2024-05-22T23:11:02.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.526+0000 7fa3417fa640 1 -- 172.21.15.84:0/2895972211 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c018c10 con 0x7fa3441067c0 2024-05-22T23:11:02.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.526+0000 7fa34a26d640 1 -- 172.21.15.84:0/2895972211 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3441a4b50 con 0x7fa3441067c0 2024-05-22T23:11:02.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.529+0000 7fa3417fa640 1 -- 172.21.15.84:0/2895972211 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa33c00f5f0 con 0x7fa3441067c0 2024-05-22T23:11:02.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.529+0000 7fa34cf76740 1 Processor -- start 2024-05-22T23:11:02.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.529+0000 7fa34cf76740 1 -- start start 2024-05-22T23:11:02.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.529+0000 7fa34cf76740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3be1af80 0x558f3be1b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.529+0000 7fa34cf76740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f3be1b8a0 0x558f3be25a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.529+0000 7fa34cf76740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be25f70 0x558f3be28360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.529+0000 7fa34cf76740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558f3bd7fdb0 con 0x558f3be25f70 2024-05-22T23:11:02.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.529+0000 7fa34cf76740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558f3bd1e400 con 0x558f3be1b8a0 2024-05-22T23:11:02.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.529+0000 7fa34cf76740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558f3bd7bd70 con 0x558f3be1af80 2024-05-22T23:11:02.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.529+0000 7fa3437fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be25f70 0x558f3be28360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.530+0000 7fa3437fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be25f70 0x558f3be28360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34456/0 (socket says 172.21.15.84:34456) 2024-05-22T23:11:02.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.530+0000 7fa3437fe640 1 -- 172.21.15.84:0/3803719268 learned_addr learned my addr 172.21.15.84:0/3803719268 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:02.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.530+0000 7fa3417fa640 1 --2- 172.21.15.84:0/2895972211 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fa32c0a2d00 0x7fa32c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.530+0000 7fa3437fe640 1 --2- 172.21.15.84:0/3803719268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3be1af80 0x558f3be1b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.530+0000 7fa343fff640 1 --2- 172.21.15.84:0/3803719268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f3be1b8a0 0x558f3be25a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.530+0000 7fa34880a640 1 --2- 172.21.15.84:0/2895972211 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fa32c0a2d00 0x7fa32c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.530+0000 7fa3437fe640 1 -- 172.21.15.84:0/3803719268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3be1af80 msgr2=0x558f3be1b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:02.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.530+0000 7fa3437fe640 1 --2- 172.21.15.84:0/3803719268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3be1af80 0x558f3be1b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.530+0000 7fa3437fe640 1 -- 172.21.15.84:0/3803719268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f3be1b8a0 msgr2=0x558f3be25a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:02.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.530+0000 7fa3437fe640 1 --2- 172.21.15.84:0/3803719268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f3be1b8a0 0x558f3be25a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.530+0000 7fa3437fe640 1 -- 172.21.15.84:0/3803719268 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3bd102f0 con 0x558f3be25f70 2024-05-22T23:11:02.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.530+0000 7fa3417fa640 1 -- 172.21.15.84:0/2895972211 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7fa33c0cfbd0 con 0x7fa3441067c0 2024-05-22T23:11:02.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.530+0000 7fa3437fe640 1 --2- 172.21.15.84:0/3803719268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be25f70 0x558f3be28360 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fa3380149c0 tx=0x7fa33800c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2314a601b5259e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:02.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.530+0000 7fa34880a640 1 --2- 172.21.15.84:0/2895972211 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fa32c0a2d00 0x7fa32c0a51c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa33c09bd60 tx=0x7fa33c06f0d0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:02.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.530+0000 7fa32a7fc640 1 -- 172.21.15.84:0/3803719268 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3380142d0 con 0x558f3be25f70 2024-05-22T23:11:02.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.530+0000 7fa32a7fc640 1 -- 172.21.15.84:0/3803719268 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa338005500 con 0x558f3be25f70 2024-05-22T23:11:02.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.530+0000 7fa32a7fc640 1 -- 172.21.15.84:0/3803719268 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33800f760 con 0x558f3be25f70 2024-05-22T23:11:02.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.531+0000 7fa34cf76740 1 -- 172.21.15.84:0/3803719268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be25f70 msgr2=0x558f3be28360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:02.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.531+0000 7fa34cf76740 1 --2- 172.21.15.84:0/3803719268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be25f70 0x558f3be28360 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fa3380149c0 tx=0x7fa33800c4d0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.531+0000 7fa34cf76740 1 -- 172.21.15.84:0/3803719268 shutdown_connections 2024-05-22T23:11:02.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.531+0000 7fa34cf76740 1 --2- 172.21.15.84:0/3803719268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be25f70 0x558f3be28360 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.531+0000 7fa34cf76740 1 --2- 172.21.15.84:0/3803719268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f3be1b8a0 0x558f3be25a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.531+0000 7fa34cf76740 1 --2- 172.21.15.84:0/3803719268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3be1af80 0x558f3be1b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.531+0000 7fa34cf76740 1 -- 172.21.15.84:0/3803719268 >> 172.21.15.84:0/3803719268 conn(0x558f3be172a0 msgr2=0x558f3be17680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:02.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.531+0000 7fa34cf76740 1 -- 172.21.15.84:0/3803719268 shutdown_connections 2024-05-22T23:11:02.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.531+0000 7fa34cf76740 1 -- 172.21.15.84:0/3803719268 wait complete. 2024-05-22T23:11:02.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.531+0000 7fa34cf76740 1 -- start start 2024-05-22T23:11:02.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.532+0000 7fa34cf76740 1 Processor -- start 2024-05-22T23:11:02.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.532+0000 7fa34cf76740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f3be1af80 0x558f3be22f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.532+0000 7fa34cf76740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3be1b8a0 0x558f3be23460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.532+0000 7fa34cf76740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be25f70 0x558f3be239a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.532+0000 7fa34cf76740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558f3be28d60 con 0x558f3be25f70 2024-05-22T23:11:02.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.532+0000 7fa34cf76740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558f3bd3a2e0 con 0x558f3be1af80 2024-05-22T23:11:02.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.532+0000 7fa34cf76740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558f3be28ee0 con 0x558f3be1b8a0 2024-05-22T23:11:02.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.532+0000 7fa3437fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be25f70 0x558f3be239a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.532+0000 7fa34cf76740 10 client.0 _ll_register_callbacks cb 0x558f3bd43a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:11:02.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.532+0000 7fa3437fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be25f70 0x558f3be239a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34468/0 (socket says 172.21.15.84:34468) 2024-05-22T23:11:02.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.532+0000 7fa3437fe640 1 -- 172.21.15.84:0/1515042876 learned_addr learned my addr 172.21.15.84:0/1515042876 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:02.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.532+0000 7fa3437fe640 1 --2- 172.21.15.84:0/1515042876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f3be1af80 0x558f3be22f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.532+0000 7fa343fff640 1 --2- 172.21.15.84:0/1515042876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3be1b8a0 0x558f3be23460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.532+0000 7fa3437fe640 1 -- 172.21.15.84:0/1515042876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3be1b8a0 msgr2=0x558f3be23460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:02.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.532+0000 7fa3437fe640 1 --2- 172.21.15.84:0/1515042876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3be1b8a0 0x558f3be23460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.532+0000 7fa3437fe640 1 -- 172.21.15.84:0/1515042876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f3be1af80 msgr2=0x558f3be22f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:02.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.532+0000 7fa3437fe640 1 --2- 172.21.15.84:0/1515042876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f3be1af80 0x558f3be22f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.532+0000 7fa3437fe640 1 -- 172.21.15.84:0/1515042876 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f3ba933f0 con 0x558f3be25f70 2024-05-22T23:11:02.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.532+0000 7fa343fff640 1 --2- 172.21.15.84:0/1515042876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3be1b8a0 0x558f3be23460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:11:02.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.532+0000 7fa3437fe640 1 --2- 172.21.15.84:0/1515042876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be25f70 0x558f3be239a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa338004cc0 tx=0x7fa33800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c4737db9e39afa server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:02.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.532+0000 7fa3237fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:11:02.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.533+0000 7fa3237fe640 1 -- 172.21.15.84:0/1515042876 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa338005000 con 0x558f3be25f70 2024-05-22T23:11:02.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.533+0000 7fa3237fe640 1 -- 172.21.15.84:0/1515042876 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7fa3380051a0 con 0x558f3be25f70 2024-05-22T23:11:02.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.533+0000 7fa34cf76740 1 -- 172.21.15.84:0/1515042876 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa338005500 con 0x558f3be25f70 2024-05-22T23:11:02.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.533+0000 7fa3237fe640 1 -- 172.21.15.84:0/1515042876 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fa33800f730 con 0x558f3be25f70 2024-05-22T23:11:02.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.533+0000 7fa34cf76740 10 client.16890 fetch_fsmap learned FSMap version 18 2024-05-22T23:11:02.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.533+0000 7fa34cf76740 1 -- 172.21.15.84:0/1515042876 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x558f3bb5d7d0 con 0x558f3be25f70 2024-05-22T23:11:02.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.533+0000 7fa3237fe640 1 -- 172.21.15.84:0/1515042876 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa338005500 con 0x558f3be25f70 2024-05-22T23:11:02.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.533+0000 7fa3237fe640 1 -- 172.21.15.84:0/1515042876 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fa3380126f0 con 0x558f3be25f70 2024-05-22T23:11:02.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.533+0000 7fa34cf76740 10 client.16890 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:11:02.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.533+0000 7fa34cf76740 10 client.16890 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:11:02.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.533+0000 7fa34cf76740 20 client.16890 populate_metadata read hostname 'smithi084' 2024-05-22T23:11:02.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.533+0000 7fa34cf76740 1 --2- 172.21.15.84:0/1515042876 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558f3be36220 0x558f3be38610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.533+0000 7fa34cf76740 4 client.16890 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:11:02.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.533+0000 7fa34cf76740 1 -- 172.21.15.84:0/1515042876 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f3bd8d4e0 con 0x558f3be36220 2024-05-22T23:11:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.533+0000 7fa343fff640 1 --2- 172.21.15.84:0/1515042876 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558f3be36220 0x558f3be38610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.534+0000 7fa3237fe640 10 client.16890 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.534+0000 7fa343fff640 1 --2- 172.21.15.84:0/1515042876 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558f3be36220 0x558f3be38610 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6df1d0790595e6ff server_cookie=64882cd7be50ec28 in_seq=0 out_seq=0 2024-05-22T23:11:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.534+0000 7fa3237fe640 1 -- 172.21.15.84:0/1515042876 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558f3bd8d4e0 con 0x558f3be36220 2024-05-22T23:11:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.534+0000 7fa3237fe640 10 client.16890 handle_command_reply: tid=0 2024-05-22T23:11:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.534+0000 7fa34cf76740 1 client.16890 shutdown 2024-05-22T23:11:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.535+0000 7fa34cf76740 1 -- 172.21.15.84:0/1515042876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be25f70 msgr2=0x558f3be239a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.535+0000 7fa34cf76740 1 --2- 172.21.15.84:0/1515042876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be25f70 0x558f3be239a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa338004cc0 tx=0x7fa33800cc10 comp rx=0 tx=0).stop 2024-05-22T23:11:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.535+0000 7fa34cf76740 1 -- 172.21.15.84:0/1515042876 shutdown_connections 2024-05-22T23:11:02.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.535+0000 7fa34cf76740 1 --2- 172.21.15.84:0/1515042876 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558f3be36220 0x558f3be38610 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.535+0000 7fa34cf76740 1 --2- 172.21.15.84:0/1515042876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be25f70 0x558f3be239a0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.535+0000 7fa34cf76740 1 --2- 172.21.15.84:0/1515042876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3be1b8a0 0x558f3be23460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.535+0000 7fa3237fe640 0 client.16890 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:02.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.535+0000 7fa34cf76740 1 --2- 172.21.15.84:0/1515042876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f3be1af80 0x558f3be22f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.535+0000 7fa34cf76740 1 -- 172.21.15.84:0/1515042876 >> 172.21.15.84:0/1515042876 conn(0x558f3be172a0 msgr2=0x558f3be1d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:02.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.535+0000 7fa34cf76740 1 -- 172.21.15.84:0/1515042876 shutdown_connections 2024-05-22T23:11:02.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.535+0000 7fa34cf76740 1 -- 172.21.15.84:0/1515042876 wait complete. 2024-05-22T23:11:02.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.535+0000 7fa34cf76740 20 client.16890 trim_cache size 0 max 16384 2024-05-22T23:11:02.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.548+0000 7fa34cf76740 1 Processor -- start 2024-05-22T23:11:02.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.548+0000 7fa34cf76740 1 -- start start 2024-05-22T23:11:02.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.548+0000 7fa34cf76740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3be1af80 0x558f3be31c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.548+0000 7fa34cf76740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be1b8a0 0x558f3be1d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.548+0000 7fa34cf76740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f3be25f70 0x558f3be21dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.548+0000 7fa34cf76740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa3441ab5e0 con 0x558f3be1b8a0 2024-05-22T23:11:02.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.548+0000 7fa34cf76740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa3441ade40 con 0x558f3be25f70 2024-05-22T23:11:02.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.548+0000 7fa34cf76740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa3441abc30 con 0x558f3be1af80 2024-05-22T23:11:02.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.548+0000 7fa343fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be1b8a0 0x558f3be1d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.548+0000 7fa343fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be1b8a0 0x558f3be1d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34484/0 (socket says 172.21.15.84:34484) 2024-05-22T23:11:02.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.548+0000 7fa343fff640 1 -- 172.21.15.84:0/1891493437 learned_addr learned my addr 172.21.15.84:0/1891493437 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:02.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.548+0000 7fa343fff640 1 -- 172.21.15.84:0/1891493437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3be1af80 msgr2=0x558f3be31c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:02.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.548+0000 7fa343fff640 1 --2- 172.21.15.84:0/1891493437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3be1af80 0x558f3be31c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.548+0000 7fa343fff640 1 -- 172.21.15.84:0/1891493437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f3be25f70 msgr2=0x558f3be21dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:02.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.548+0000 7fa343fff640 1 --2- 172.21.15.84:0/1891493437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f3be25f70 0x558f3be21dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.548+0000 7fa343fff640 1 -- 172.21.15.84:0/1891493437 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3be184d0 con 0x558f3be1b8a0 2024-05-22T23:11:02.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.548+0000 7fa343fff640 1 --2- 172.21.15.84:0/1891493437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be1b8a0 0x558f3be1d720 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa334009900 tx=0x7fa334005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6bd4540ed4b8e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:02.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.549+0000 7fa329ffb640 1 -- 172.21.15.84:0/1891493437 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334035040 con 0x558f3be1b8a0 2024-05-22T23:11:02.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.549+0000 7fa329ffb640 1 -- 172.21.15.84:0/1891493437 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa334048070 con 0x558f3be1b8a0 2024-05-22T23:11:02.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.549+0000 7fa329ffb640 1 -- 172.21.15.84:0/1891493437 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334043db0 con 0x558f3be1b8a0 2024-05-22T23:11:02.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.549+0000 7fa34cf76740 1 -- 172.21.15.84:0/1891493437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be1b8a0 msgr2=0x558f3be1d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:02.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.549+0000 7fa34cf76740 1 --2- 172.21.15.84:0/1891493437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be1b8a0 0x558f3be1d720 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa334009900 tx=0x7fa334005de0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.549+0000 7fa34cf76740 1 -- 172.21.15.84:0/1891493437 shutdown_connections 2024-05-22T23:11:02.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.549+0000 7fa34cf76740 1 --2- 172.21.15.84:0/1891493437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f3be25f70 0x558f3be21dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.549+0000 7fa34cf76740 1 --2- 172.21.15.84:0/1891493437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be1b8a0 0x558f3be1d720 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.549+0000 7fa34cf76740 1 --2- 172.21.15.84:0/1891493437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3be1af80 0x558f3be31c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.549+0000 7fa34cf76740 1 -- 172.21.15.84:0/1891493437 >> 172.21.15.84:0/1891493437 conn(0x558f3be172a0 msgr2=0x558f3be216c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:02.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.549+0000 7fa34cf76740 1 -- 172.21.15.84:0/1891493437 shutdown_connections 2024-05-22T23:11:02.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.549+0000 7fa34cf76740 1 -- 172.21.15.84:0/1891493437 wait complete. 2024-05-22T23:11:02.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.549+0000 7fa34cf76740 1 -- start start 2024-05-22T23:11:02.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.550+0000 7fa34cf76740 1 Processor -- start 2024-05-22T23:11:02.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.550+0000 7fa34cf76740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f3be1af80 0x558f3be2e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.550+0000 7fa34cf76740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3be1b8a0 0x558f3be2e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.550+0000 7fa34cf76740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be25f70 0x558f3be1f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.550+0000 7fa34cf76740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa3441ac300 con 0x558f3be25f70 2024-05-22T23:11:02.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.550+0000 7fa34cf76740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa3441ad770 con 0x558f3be1af80 2024-05-22T23:11:02.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.550+0000 7fa34cf76740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa3441ac9d0 con 0x558f3be1b8a0 2024-05-22T23:11:02.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.550+0000 7fa34cf76740 10 client.0 _ll_register_callbacks cb 0x7fa334009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:11:02.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.550+0000 7fa3437fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be25f70 0x558f3be1f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.550+0000 7fa3437fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be25f70 0x558f3be1f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34488/0 (socket says 172.21.15.84:34488) 2024-05-22T23:11:02.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.550+0000 7fa3437fe640 1 -- 172.21.15.84:0/677963329 learned_addr learned my addr 172.21.15.84:0/677963329 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:02.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.550+0000 7fa3437fe640 1 --2- 172.21.15.84:0/677963329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f3be1af80 0x558f3be2e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.550+0000 7fa3437fe640 1 -- 172.21.15.84:0/677963329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3be1b8a0 msgr2=0x558f3be2e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:02.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.550+0000 7fa343fff640 1 --2- 172.21.15.84:0/677963329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3be1b8a0 0x558f3be2e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.550+0000 7fa3437fe640 1 --2- 172.21.15.84:0/677963329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3be1b8a0 0x558f3be2e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.550+0000 7fa3437fe640 1 -- 172.21.15.84:0/677963329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f3be1af80 msgr2=0x558f3be2e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:02.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.550+0000 7fa3437fe640 1 --2- 172.21.15.84:0/677963329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f3be1af80 0x558f3be2e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.550+0000 7fa3437fe640 1 -- 172.21.15.84:0/677963329 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f3be1ff30 con 0x558f3be25f70 2024-05-22T23:11:02.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.550+0000 7fa343fff640 1 --2- 172.21.15.84:0/677963329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3be1b8a0 0x558f3be2e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:11:02.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.551+0000 7fa3437fe640 1 --2- 172.21.15.84:0/677963329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be25f70 0x558f3be1f730 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa33800cc50 tx=0x7fa33800c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3f6232cd953bab server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:02.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.551+0000 7fa3237fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:11:02.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.551+0000 7fa3237fe640 1 -- 172.21.15.84:0/677963329 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa338012650 con 0x558f3be25f70 2024-05-22T23:11:02.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.551+0000 7fa3237fe640 1 -- 172.21.15.84:0/677963329 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7fa3380127f0 con 0x558f3be25f70 2024-05-22T23:11:02.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.551+0000 7fa34cf76740 1 -- 172.21.15.84:0/677963329 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa334048070 con 0x558f3be25f70 2024-05-22T23:11:02.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.551+0000 7fa3237fe640 1 -- 172.21.15.84:0/677963329 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fa33800f8d0 con 0x558f3be25f70 2024-05-22T23:11:02.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.551+0000 7fa34cf76740 10 client.16896 fetch_fsmap learned FSMap version 18 2024-05-22T23:11:02.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.551+0000 7fa34cf76740 1 -- 172.21.15.84:0/677963329 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x558f3be9c1b0 con 0x558f3be25f70 2024-05-22T23:11:02.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.551+0000 7fa3237fe640 1 -- 172.21.15.84:0/677963329 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa334048070 con 0x558f3be25f70 2024-05-22T23:11:02.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.551+0000 7fa3237fe640 1 -- 172.21.15.84:0/677963329 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fa3380052f0 con 0x558f3be25f70 2024-05-22T23:11:02.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.551+0000 7fa34cf76740 10 client.16896 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:11:02.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.551+0000 7fa34cf76740 10 client.16896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:11:02.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.551+0000 7fa34cf76740 20 client.16896 populate_metadata read hostname 'smithi084' 2024-05-22T23:11:02.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.551+0000 7fa34cf76740 1 --2- 172.21.15.84:0/677963329 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558f3be9c840 0x558f3be9ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:02.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.551+0000 7fa34cf76740 4 client.16896 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-22T23:11:02.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.551+0000 7fa34cf76740 1 -- 172.21.15.84:0/677963329 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f3bd87830 con 0x558f3be9c840 2024-05-22T23:11:02.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.551+0000 7fa343fff640 1 --2- 172.21.15.84:0/677963329 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558f3be9c840 0x558f3be9ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:02.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.552+0000 7fa3237fe640 10 client.16896 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:02.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.552+0000 7fa343fff640 1 --2- 172.21.15.84:0/677963329 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558f3be9c840 0x558f3be9ec60 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=987a168b8cd64ec5 server_cookie=d15ce0d90beb70f8 in_seq=0 out_seq=0 2024-05-22T23:11:02.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.552+0000 7fa3237fe640 1 -- 172.21.15.84:0/677963329 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f3bd87830 con 0x558f3be9c840 2024-05-22T23:11:02.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.552+0000 7fa3237fe640 10 client.16896 handle_command_reply: tid=0 2024-05-22T23:11:02.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.552+0000 7fa34cf76740 1 client.16896 shutdown 2024-05-22T23:11:02.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.552+0000 7fa34cf76740 1 -- 172.21.15.84:0/677963329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be25f70 msgr2=0x558f3be1f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:02.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.552+0000 7fa34cf76740 1 --2- 172.21.15.84:0/677963329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be25f70 0x558f3be1f730 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa33800cc50 tx=0x7fa33800c5a0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.552+0000 7fa34cf76740 1 -- 172.21.15.84:0/677963329 shutdown_connections 2024-05-22T23:11:02.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.552+0000 7fa34cf76740 1 --2- 172.21.15.84:0/677963329 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558f3be9c840 0x558f3be9ec60 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.552+0000 7fa34cf76740 1 --2- 172.21.15.84:0/677963329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f3be25f70 0x558f3be1f730 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.552+0000 7fa3237fe640 0 client.16896 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:02.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.552+0000 7fa34cf76740 1 --2- 172.21.15.84:0/677963329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f3be1b8a0 0x558f3be2e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.552+0000 7fa34cf76740 1 --2- 172.21.15.84:0/677963329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f3be1af80 0x558f3be2e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.552+0000 7fa34cf76740 1 -- 172.21.15.84:0/677963329 >> 172.21.15.84:0/677963329 conn(0x558f3be172a0 msgr2=0x558f3be23770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:02.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.552+0000 7fa34cf76740 1 -- 172.21.15.84:0/677963329 shutdown_connections 2024-05-22T23:11:02.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.552+0000 7fa34cf76740 1 -- 172.21.15.84:0/677963329 wait complete. 2024-05-22T23:11:02.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.552+0000 7fa34cf76740 20 client.16896 trim_cache size 0 max 16384 2024-05-22T23:11:02.574 INFO:teuthology.orchestra.run.smithi084.stdout:[] 2024-05-22T23:11:02.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.553+0000 7fa34a26d640 1 -- 172.21.15.84:0/2895972211 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fa32c0a2d00 msgr2=0x7fa32c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:02.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.553+0000 7fa34a26d640 1 --2- 172.21.15.84:0/2895972211 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fa32c0a2d00 0x7fa32c0a51c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa33c09bd60 tx=0x7fa33c06f0d0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.553+0000 7fa34a26d640 1 -- 172.21.15.84:0/2895972211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3441067c0 msgr2=0x7fa3441a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:02.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.553+0000 7fa34a26d640 1 --2- 172.21.15.84:0/2895972211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3441067c0 0x7fa3441a3ef0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00cce0 tx=0x7fa33c00a620 comp rx=0 tx=0).stop 2024-05-22T23:11:02.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.553+0000 7fa34a26d640 1 -- 172.21.15.84:0/2895972211 shutdown_connections 2024-05-22T23:11:02.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.553+0000 7fa34a26d640 1 --2- 172.21.15.84:0/2895972211 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fa32c0a2d00 0x7fa32c0a51c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.553+0000 7fa34a26d640 1 --2- 172.21.15.84:0/2895972211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3441067c0 0x7fa3441a3ef0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.553+0000 7fa34a26d640 1 --2- 172.21.15.84:0/2895972211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa344101460 0x7fa3441993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.553+0000 7fa34a26d640 1 --2- 172.21.15.84:0/2895972211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa344100ac0 0x7fa344198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:02.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.553+0000 7fa34a26d640 1 -- 172.21.15.84:0/2895972211 >> 172.21.15.84:0/2895972211 conn(0x7fa344076570 msgr2=0x7fa3440ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:02.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.553+0000 7fa34a26d640 1 -- 172.21.15.84:0/2895972211 shutdown_connections 2024-05-22T23:11:02.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:02.553+0000 7fa34a26d640 1 -- 172.21.15.84:0/2895972211 wait complete. 2024-05-22T23:11:03.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:03 smithi096 ceph-mon[36392]: pgmap v285: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-05-22T23:11:03.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:03 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3919999333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:11:03.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:03 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:11:03.578 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-22T23:11:03.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:11:03.578 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-22T23:11:03.600 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:03 smithi084 ceph-mon[32186]: pgmap v285: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-05-22T23:11:03.600 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:03 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3919999333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:11:03.600 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:03 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:11:03.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:03 smithi204 ceph-mon[36211]: pgmap v285: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-05-22T23:11:03.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:03 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3919999333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:11:03.667 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:03 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:11:03.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.683+0000 7f26bd1c1640 1 -- 172.21.15.84:0/3969757521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26b81009a0 msgr2=0x7f26b81064d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:03.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.683+0000 7f26bd1c1640 1 --2- 172.21.15.84:0/3969757521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26b81009a0 0x7f26b81064d0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009900 tx=0x7f26ac02a410 comp rx=0 tx=0).stop 2024-05-22T23:11:03.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.683+0000 7f26bd1c1640 1 -- 172.21.15.84:0/3969757521 shutdown_connections 2024-05-22T23:11:03.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.683+0000 7f26bd1c1640 1 --2- 172.21.15.84:0/3969757521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26b8106c00 0x7f26b810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.683+0000 7f26bd1c1640 1 --2- 172.21.15.84:0/3969757521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26b81009a0 0x7f26b81064d0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.683+0000 7f26bd1c1640 1 --2- 172.21.15.84:0/3969757521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26b80ffff0 0x7f26b81003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.683+0000 7f26bd1c1640 1 -- 172.21.15.84:0/3969757521 >> 172.21.15.84:0/3969757521 conn(0x7f26b8076d10 msgr2=0x7f26b80fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:03.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.683+0000 7f26bd1c1640 1 -- 172.21.15.84:0/3969757521 shutdown_connections 2024-05-22T23:11:03.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.683+0000 7f26bd1c1640 1 -- 172.21.15.84:0/3969757521 wait complete. 2024-05-22T23:11:03.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.683+0000 7f26bd1c1640 1 Processor -- start 2024-05-22T23:11:03.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.684+0000 7f26bd1c1640 1 -- start start 2024-05-22T23:11:03.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.684+0000 7f26bd1c1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26b80ffff0 0x7f26b81981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.684+0000 7f26bd1c1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26b8106c00 0x7f26b81986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.684+0000 7f26bd1c1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26b819ca60 0x7f26b819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.684+0000 7f26bd1c1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f26b8111c20 con 0x7f26b819ca60 2024-05-22T23:11:03.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.684+0000 7f26bd1c1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f26b8111aa0 con 0x7f26b80ffff0 2024-05-22T23:11:03.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.684+0000 7f26bd1c1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f26b8111da0 con 0x7f26b8106c00 2024-05-22T23:11:03.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.684+0000 7f26b759e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26b819ca60 0x7f26b819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.684+0000 7f26b6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26b80ffff0 0x7f26b81981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.684+0000 7f26b759e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26b819ca60 0x7f26b819cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34494/0 (socket says 172.21.15.84:34494) 2024-05-22T23:11:03.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.684+0000 7f26b759e640 1 -- 172.21.15.84:0/1388388574 learned_addr learned my addr 172.21.15.84:0/1388388574 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:03.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.684+0000 7f26b659c640 1 --2- 172.21.15.84:0/1388388574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26b8106c00 0x7f26b81986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.684+0000 7f26b759e640 1 -- 172.21.15.84:0/1388388574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26b8106c00 msgr2=0x7f26b81986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:03.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.684+0000 7f26b759e640 1 --2- 172.21.15.84:0/1388388574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26b8106c00 0x7f26b81986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.684+0000 7f26b759e640 1 -- 172.21.15.84:0/1388388574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26b80ffff0 msgr2=0x7f26b81981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:03.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.684+0000 7f26b759e640 1 --2- 172.21.15.84:0/1388388574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26b80ffff0 0x7f26b81981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.684+0000 7f26b759e640 1 -- 172.21.15.84:0/1388388574 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b819d550 con 0x7f26b819ca60 2024-05-22T23:11:03.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.684+0000 7f26b6d9d640 1 --2- 172.21.15.84:0/1388388574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26b80ffff0 0x7f26b81981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:11:03.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.685+0000 7f26b759e640 1 --2- 172.21.15.84:0/1388388574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26b819ca60 0x7f26b819cf10 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009fd0 tx=0x7f26b000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:03.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.685+0000 7f269ffff640 1 -- 172.21.15.84:0/1388388574 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0017070 con 0x7f26b819ca60 2024-05-22T23:11:03.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.685+0000 7f26bd1c1640 1 -- 172.21.15.84:0/1388388574 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b819d840 con 0x7f26b819ca60 2024-05-22T23:11:03.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.685+0000 7f269ffff640 1 -- 172.21.15.84:0/1388388574 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b0003e50 con 0x7f26b819ca60 2024-05-22T23:11:03.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.685+0000 7f269ffff640 1 -- 172.21.15.84:0/1388388574 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0020060 con 0x7f26b819ca60 2024-05-22T23:11:03.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.685+0000 7f26bd1c1640 1 -- 172.21.15.84:0/1388388574 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b81a5000 con 0x7f26b819ca60 2024-05-22T23:11:03.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.687+0000 7f269ffff640 1 -- 172.21.15.84:0/1388388574 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f26b00040b0 con 0x7f26b819ca60 2024-05-22T23:11:03.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.687+0000 7f26bffad740 1 Processor -- start 2024-05-22T23:11:03.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.687+0000 7f26bffad740 1 -- start start 2024-05-22T23:11:03.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.687+0000 7f26bffad740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecc6b2db50 0x55ecc6b2df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.687+0000 7f26bffad740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecc6b2e470 0x55ecc6b3cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.687+0000 7f26bffad740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b3d050 0x55ecc6b3f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.687+0000 7f26bffad740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ecc6a91a40 con 0x55ecc6b3d050 2024-05-22T23:11:03.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.687+0000 7f26bffad740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ecc6a30440 con 0x55ecc6b2db50 2024-05-22T23:11:03.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.687+0000 7f26bffad740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ecc6a8da00 con 0x55ecc6b2e470 2024-05-22T23:11:03.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.688+0000 7f26b659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b3d050 0x55ecc6b3f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.688+0000 7f26b6d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecc6b2e470 0x55ecc6b3cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.688+0000 7f26b6d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecc6b2e470 0x55ecc6b3cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.84:57272/0 (socket says 172.21.15.84:57272) 2024-05-22T23:11:03.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.688+0000 7f26b659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b3d050 0x55ecc6b3f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34500/0 (socket says 172.21.15.84:34500) 2024-05-22T23:11:03.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.688+0000 7f26b6d9d640 1 -- 172.21.15.84:0/3162069126 learned_addr learned my addr 172.21.15.84:0/3162069126 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:03.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.688+0000 7f26b659c640 1 --2- 172.21.15.84:0/3162069126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecc6b2db50 0x55ecc6b2df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.688+0000 7f269ffff640 1 --2- 172.21.15.84:0/1388388574 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f26a00a2d00 0x7f26a00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.688+0000 7f26b659c640 1 -- 172.21.15.84:0/3162069126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecc6b2e470 msgr2=0x55ecc6b3cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:03.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.688+0000 7f26b759e640 1 --2- 172.21.15.84:0/1388388574 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f26a00a2d00 0x7f26a00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.688+0000 7f26b659c640 1 --2- 172.21.15.84:0/3162069126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecc6b2e470 0x55ecc6b3cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.688+0000 7f26b659c640 1 -- 172.21.15.84:0/3162069126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecc6b2db50 msgr2=0x55ecc6b2df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:03.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.688+0000 7f26b659c640 1 --2- 172.21.15.84:0/3162069126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecc6b2db50 0x55ecc6b2df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.688+0000 7f26b659c640 1 -- 172.21.15.84:0/3162069126 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecc6a3a2d0 con 0x55ecc6b3d050 2024-05-22T23:11:03.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.688+0000 7f26b6d9d640 1 --2- 172.21.15.84:0/3162069126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecc6b2e470 0x55ecc6b3cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:11:03.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.688+0000 7f269ffff640 1 -- 172.21.15.84:0/1388388574 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7f26b00cede0 con 0x7f26b819ca60 2024-05-22T23:11:03.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.688+0000 7f26b659c640 1 --2- 172.21.15.84:0/3162069126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b3d050 0x55ecc6b3f440 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f26ac02a930 tx=0x7f26ac005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8d97b34b8fc6d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:03.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.688+0000 7f26b759e640 1 --2- 172.21.15.84:0/1388388574 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f26a00a2d00 0x7f26a00a51c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f26b00cc5b0 tx=0x7f26b001d040 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:03.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.688+0000 7f269d7fa640 1 -- 172.21.15.84:0/3162069126 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac035070 con 0x55ecc6b3d050 2024-05-22T23:11:03.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.689+0000 7f269d7fa640 1 -- 172.21.15.84:0/3162069126 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ac0035f0 con 0x55ecc6b3d050 2024-05-22T23:11:03.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.689+0000 7f269d7fa640 1 -- 172.21.15.84:0/3162069126 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac03d6b0 con 0x55ecc6b3d050 2024-05-22T23:11:03.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.689+0000 7f26bffad740 1 -- 172.21.15.84:0/3162069126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b3d050 msgr2=0x55ecc6b3f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:03.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.689+0000 7f26bffad740 1 --2- 172.21.15.84:0/3162069126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b3d050 0x55ecc6b3f440 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f26ac02a930 tx=0x7f26ac005780 comp rx=0 tx=0).stop 2024-05-22T23:11:03.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.689+0000 7f26bffad740 1 -- 172.21.15.84:0/3162069126 shutdown_connections 2024-05-22T23:11:03.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.689+0000 7f26bffad740 1 --2- 172.21.15.84:0/3162069126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b3d050 0x55ecc6b3f440 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.689+0000 7f26bffad740 1 --2- 172.21.15.84:0/3162069126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecc6b2e470 0x55ecc6b3cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.689+0000 7f26bffad740 1 --2- 172.21.15.84:0/3162069126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecc6b2db50 0x55ecc6b2df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.689+0000 7f26bffad740 1 -- 172.21.15.84:0/3162069126 >> 172.21.15.84:0/3162069126 conn(0x55ecc6b29f70 msgr2=0x55ecc6b2a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:03.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.689+0000 7f26bffad740 1 -- 172.21.15.84:0/3162069126 shutdown_connections 2024-05-22T23:11:03.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.689+0000 7f26bffad740 1 -- 172.21.15.84:0/3162069126 wait complete. 2024-05-22T23:11:03.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.690+0000 7f26bffad740 1 -- start start 2024-05-22T23:11:03.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.690+0000 7f26bffad740 1 Processor -- start 2024-05-22T23:11:03.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.690+0000 7f26bffad740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecc6b2db50 0x55ecc6b39fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.690+0000 7f26bffad740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b2e470 0x55ecc6b3a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.690+0000 7f26bffad740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecc6b3d050 0x55ecc6b3aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.690+0000 7f26bffad740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ecc6b3fdb0 con 0x55ecc6b2e470 2024-05-22T23:11:03.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.690+0000 7f26bffad740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ecc6a1b5f0 con 0x55ecc6b3d050 2024-05-22T23:11:03.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.690+0000 7f26bffad740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ecc6b3ff30 con 0x55ecc6b2db50 2024-05-22T23:11:03.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.690+0000 7f26bffad740 10 client.0 _ll_register_callbacks cb 0x55ecc6a16cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:11:03.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.690+0000 7f26b6d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b2e470 0x55ecc6b3a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.690+0000 7f26b6d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b2e470 0x55ecc6b3a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34504/0 (socket says 172.21.15.84:34504) 2024-05-22T23:11:03.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.690+0000 7f26b6d9d640 1 -- 172.21.15.84:0/1046643063 learned_addr learned my addr 172.21.15.84:0/1046643063 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:03.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.690+0000 7f26b659c640 1 --2- 172.21.15.84:0/1046643063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecc6b3d050 0x55ecc6b3aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.690+0000 7f26b659c640 1 --2- 172.21.15.84:0/1046643063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecc6b2db50 0x55ecc6b39fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.690+0000 7f26b6d9d640 1 -- 172.21.15.84:0/1046643063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecc6b2db50 msgr2=0x55ecc6b39fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:03.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.690+0000 7f26b6d9d640 1 --2- 172.21.15.84:0/1046643063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecc6b2db50 0x55ecc6b39fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.690+0000 7f26b6d9d640 1 -- 172.21.15.84:0/1046643063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecc6b3d050 msgr2=0x55ecc6b3aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:03.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.690+0000 7f26b6d9d640 1 --2- 172.21.15.84:0/1046643063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecc6b3d050 0x55ecc6b3aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.690+0000 7f26b6d9d640 1 -- 172.21.15.84:0/1046643063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecc686d7e0 con 0x55ecc6b2e470 2024-05-22T23:11:03.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.690+0000 7f26b659c640 1 --2- 172.21.15.84:0/1046643063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecc6b2db50 0x55ecc6b39fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:11:03.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.691+0000 7f26b6d9d640 1 --2- 172.21.15.84:0/1046643063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b2e470 0x55ecc6b3a520 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f26a8011810 tx=0x7f26a8011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bcc9cd4ddccd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:03.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.691+0000 7f269aff5640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:11:03.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.691+0000 7f269aff5640 1 -- 172.21.15.84:0/1046643063 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8010c60 con 0x55ecc6b2e470 2024-05-22T23:11:03.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.691+0000 7f269aff5640 1 -- 172.21.15.84:0/1046643063 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7f26a8010e00 con 0x55ecc6b2e470 2024-05-22T23:11:03.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.691+0000 7f26bffad740 1 -- 172.21.15.84:0/1046643063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26ac0035f0 con 0x55ecc6b2e470 2024-05-22T23:11:03.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.691+0000 7f269aff5640 1 -- 172.21.15.84:0/1046643063 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f26a8005020 con 0x55ecc6b2e470 2024-05-22T23:11:03.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.691+0000 7f26bffad740 10 client.16908 fetch_fsmap learned FSMap version 18 2024-05-22T23:11:03.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.691+0000 7f26bffad740 1 -- 172.21.15.84:0/1046643063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55ecc68cdf70 con 0x55ecc6b2e470 2024-05-22T23:11:03.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.691+0000 7f269aff5640 1 -- 172.21.15.84:0/1046643063 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ac0035f0 con 0x55ecc6b2e470 2024-05-22T23:11:03.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.691+0000 7f269aff5640 1 -- 172.21.15.84:0/1046643063 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f26a801a410 con 0x55ecc6b2e470 2024-05-22T23:11:03.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.691+0000 7f26bffad740 10 client.16908 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:11:03.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.691+0000 7f26bffad740 10 client.16908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:11:03.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.691+0000 7f26bffad740 20 client.16908 populate_metadata read hostname 'smithi084' 2024-05-22T23:11:03.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.691+0000 7f26bffad740 1 --2- 172.21.15.84:0/1046643063 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55ecc6b4d4c0 0x55ecc6b4f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.691+0000 7f26bffad740 4 client.16908 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:11:03.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.691+0000 7f26bffad740 1 -- 172.21.15.84:0/1046643063 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecc6aa5460 con 0x55ecc6b4d4c0 2024-05-22T23:11:03.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.691+0000 7f26b659c640 1 --2- 172.21.15.84:0/1046643063 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55ecc6b4d4c0 0x55ecc6b4f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.692+0000 7f26b659c640 1 --2- 172.21.15.84:0/1046643063 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55ecc6b4d4c0 0x55ecc6b4f8d0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8497ac59710bf1ec server_cookie=7bb3da534ab79dc5 in_seq=0 out_seq=0 2024-05-22T23:11:03.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.692+0000 7f269aff5640 10 client.16908 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:03.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.692+0000 7f269aff5640 1 -- 172.21.15.84:0/1046643063 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ecc6aa5460 con 0x55ecc6b4d4c0 2024-05-22T23:11:03.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.692+0000 7f269aff5640 10 client.16908 handle_command_reply: tid=0 2024-05-22T23:11:03.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.692+0000 7f26bffad740 1 client.16908 shutdown 2024-05-22T23:11:03.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.693+0000 7f26bffad740 1 -- 172.21.15.84:0/1046643063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b2e470 msgr2=0x55ecc6b3a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:03.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.693+0000 7f26bffad740 1 --2- 172.21.15.84:0/1046643063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b2e470 0x55ecc6b3a520 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f26a8011810 tx=0x7f26a8011cd0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.693+0000 7f26bffad740 1 -- 172.21.15.84:0/1046643063 shutdown_connections 2024-05-22T23:11:03.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.693+0000 7f26bffad740 1 --2- 172.21.15.84:0/1046643063 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55ecc6b4d4c0 0x55ecc6b4f8d0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.693+0000 7f26bffad740 1 --2- 172.21.15.84:0/1046643063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecc6b3d050 0x55ecc6b3aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.693+0000 7f269aff5640 0 client.16908 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:03.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.693+0000 7f26bffad740 1 --2- 172.21.15.84:0/1046643063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b2e470 0x55ecc6b3a520 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.693+0000 7f26bffad740 1 --2- 172.21.15.84:0/1046643063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecc6b2db50 0x55ecc6b39fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.693+0000 7f26bffad740 1 -- 172.21.15.84:0/1046643063 >> 172.21.15.84:0/1046643063 conn(0x55ecc6b29f70 msgr2=0x55ecc6b2fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:03.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.693+0000 7f26bffad740 1 -- 172.21.15.84:0/1046643063 shutdown_connections 2024-05-22T23:11:03.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.693+0000 7f26bffad740 1 -- 172.21.15.84:0/1046643063 wait complete. 2024-05-22T23:11:03.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.693+0000 7f26bffad740 20 client.16908 trim_cache size 0 max 16384 2024-05-22T23:11:03.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.710+0000 7f26bffad740 1 Processor -- start 2024-05-22T23:11:03.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.710+0000 7f26bffad740 1 -- start start 2024-05-22T23:11:03.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.710+0000 7f26bffad740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecc6b2db50 0x55ecc6b48cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.710+0000 7f26bffad740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecc6b2e470 0x55ecc6b3a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.710+0000 7f26bffad740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b3d050 0x55ecc6b551e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.710+0000 7f26bffad740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f26b81a9420 con 0x55ecc6b3d050 2024-05-22T23:11:03.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.710+0000 7f26bffad740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f26b81abc80 con 0x55ecc6b2db50 2024-05-22T23:11:03.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.710+0000 7f26bffad740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f26b81a9a70 con 0x55ecc6b2e470 2024-05-22T23:11:03.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.711+0000 7f26b659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b3d050 0x55ecc6b551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.711+0000 7f26b659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b3d050 0x55ecc6b551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34516/0 (socket says 172.21.15.84:34516) 2024-05-22T23:11:03.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.711+0000 7f26b659c640 1 -- 172.21.15.84:0/612587416 learned_addr learned my addr 172.21.15.84:0/612587416 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:03.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.711+0000 7f26b659c640 1 --2- 172.21.15.84:0/612587416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecc6b2db50 0x55ecc6b48cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.711+0000 7f26b6d9d640 1 --2- 172.21.15.84:0/612587416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecc6b2e470 0x55ecc6b3a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.711+0000 7f26b659c640 1 -- 172.21.15.84:0/612587416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecc6b2e470 msgr2=0x55ecc6b3a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:03.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.711+0000 7f26b659c640 1 --2- 172.21.15.84:0/612587416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecc6b2e470 0x55ecc6b3a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.711+0000 7f26b659c640 1 -- 172.21.15.84:0/612587416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecc6b2db50 msgr2=0x55ecc6b48cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:03.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.711+0000 7f26b659c640 1 --2- 172.21.15.84:0/612587416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecc6b2db50 0x55ecc6b48cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.711+0000 7f26b659c640 1 -- 172.21.15.84:0/612587416 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecc6b2e2a0 con 0x55ecc6b3d050 2024-05-22T23:11:03.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.711+0000 7f26b659c640 1 --2- 172.21.15.84:0/612587416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b3d050 0x55ecc6b551e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x55ecc6b2b280 tx=0x7f26ac041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983d07523c65bd0d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:03.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.711+0000 7f269cff9640 1 -- 172.21.15.84:0/612587416 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac035040 con 0x55ecc6b3d050 2024-05-22T23:11:03.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.711+0000 7f269cff9640 1 -- 172.21.15.84:0/612587416 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ac005530 con 0x55ecc6b3d050 2024-05-22T23:11:03.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.711+0000 7f269cff9640 1 -- 172.21.15.84:0/612587416 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac031d20 con 0x55ecc6b3d050 2024-05-22T23:11:03.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.712+0000 7f26bffad740 1 -- 172.21.15.84:0/612587416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b3d050 msgr2=0x55ecc6b551e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:03.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.712+0000 7f26bffad740 1 --2- 172.21.15.84:0/612587416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b3d050 0x55ecc6b551e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x55ecc6b2b280 tx=0x7f26ac041040 comp rx=0 tx=0).stop 2024-05-22T23:11:03.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.712+0000 7f26bffad740 1 -- 172.21.15.84:0/612587416 shutdown_connections 2024-05-22T23:11:03.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.712+0000 7f26bffad740 1 --2- 172.21.15.84:0/612587416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b3d050 0x55ecc6b551e0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.712+0000 7f26bffad740 1 --2- 172.21.15.84:0/612587416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecc6b2e470 0x55ecc6b3a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.712+0000 7f26bffad740 1 --2- 172.21.15.84:0/612587416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecc6b2db50 0x55ecc6b48cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.712+0000 7f26bffad740 1 -- 172.21.15.84:0/612587416 >> 172.21.15.84:0/612587416 conn(0x55ecc6b29f70 msgr2=0x55ecc6b291b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:03.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.712+0000 7f26bffad740 1 -- 172.21.15.84:0/612587416 shutdown_connections 2024-05-22T23:11:03.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.712+0000 7f26bffad740 1 -- 172.21.15.84:0/612587416 wait complete. 2024-05-22T23:11:03.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.712+0000 7f26bffad740 1 -- start start 2024-05-22T23:11:03.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.712+0000 7f26bffad740 1 Processor -- start 2024-05-22T23:11:03.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.713+0000 7f26bffad740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecc6b2db50 0x55ecc6b36b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.713+0000 7f26bffad740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b2e470 0x55ecc6b370c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.713+0000 7f26bffad740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecc6b3d050 0x55ecc6b37600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.713+0000 7f26bffad740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f26b81aa140 con 0x55ecc6b2e470 2024-05-22T23:11:03.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.713+0000 7f26bffad740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f26b81ab5b0 con 0x55ecc6b3d050 2024-05-22T23:11:03.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.713+0000 7f26bffad740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f26b81aa810 con 0x55ecc6b2db50 2024-05-22T23:11:03.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.713+0000 7f26bffad740 10 client.0 _ll_register_callbacks cb 0x7f26a800bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:11:03.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.713+0000 7f26b6d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b2e470 0x55ecc6b370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.713+0000 7f26b659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecc6b2db50 0x55ecc6b36b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.713+0000 7f26b6d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b2e470 0x55ecc6b370c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34528/0 (socket says 172.21.15.84:34528) 2024-05-22T23:11:03.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.713+0000 7f26b6d9d640 1 -- 172.21.15.84:0/2707608258 learned_addr learned my addr 172.21.15.84:0/2707608258 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:03.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.713+0000 7f26b659c640 1 --2- 172.21.15.84:0/2707608258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecc6b3d050 0x55ecc6b37600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.713+0000 7f26b6d9d640 1 -- 172.21.15.84:0/2707608258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecc6b2db50 msgr2=0x55ecc6b36b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:03.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.713+0000 7f26b6d9d640 1 --2- 172.21.15.84:0/2707608258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecc6b2db50 0x55ecc6b36b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.713+0000 7f26b6d9d640 1 -- 172.21.15.84:0/2707608258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecc6b3d050 msgr2=0x55ecc6b37600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:03.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.713+0000 7f26b6d9d640 1 --2- 172.21.15.84:0/2707608258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecc6b3d050 0x55ecc6b37600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.713+0000 7f26b6d9d640 1 -- 172.21.15.84:0/2707608258 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecc6765170 con 0x55ecc6b2e470 2024-05-22T23:11:03.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.713+0000 7f26b659c640 1 --2- 172.21.15.84:0/2707608258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecc6b2db50 0x55ecc6b36b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:11:03.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.713+0000 7f26b6d9d640 1 --2- 172.21.15.84:0/2707608258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b2e470 0x55ecc6b370c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f26a8011810 tx=0x7f26a801a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d13cb2848b03bc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:03.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.713+0000 7f269aff5640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:11:03.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.713+0000 7f269aff5640 1 -- 172.21.15.84:0/2707608258 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a801acd0 con 0x55ecc6b2e470 2024-05-22T23:11:03.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.714+0000 7f26bffad740 1 -- 172.21.15.84:0/2707608258 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26ac005530 con 0x55ecc6b2e470 2024-05-22T23:11:03.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.714+0000 7f269aff5640 1 -- 172.21.15.84:0/2707608258 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7f26a80051c0 con 0x55ecc6b2e470 2024-05-22T23:11:03.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.714+0000 7f269aff5640 1 -- 172.21.15.84:0/2707608258 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f26a803a050 con 0x55ecc6b2e470 2024-05-22T23:11:03.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.714+0000 7f26bffad740 10 client.16914 fetch_fsmap learned FSMap version 18 2024-05-22T23:11:03.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.714+0000 7f26bffad740 1 -- 172.21.15.84:0/2707608258 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55ecc6a5a930 con 0x55ecc6b2e470 2024-05-22T23:11:03.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.714+0000 7f269aff5640 1 -- 172.21.15.84:0/2707608258 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ac005530 con 0x55ecc6b2e470 2024-05-22T23:11:03.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.714+0000 7f269aff5640 1 -- 172.21.15.84:0/2707608258 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f26a8004b40 con 0x55ecc6b2e470 2024-05-22T23:11:03.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.714+0000 7f26bffad740 10 client.16914 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:11:03.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.714+0000 7f26bffad740 10 client.16914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:11:03.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.714+0000 7f26bffad740 20 client.16914 populate_metadata read hostname 'smithi084' 2024-05-22T23:11:03.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.714+0000 7f26bffad740 1 --2- 172.21.15.84:0/2707608258 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55ecc6bb4050 0x55ecc6bb6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.714+0000 7f26bffad740 4 client.16914 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-22T23:11:03.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.714+0000 7f26bffad740 1 -- 172.21.15.84:0/2707608258 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ecc6a9f170 con 0x55ecc6bb4050 2024-05-22T23:11:03.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.714+0000 7f26b659c640 1 --2- 172.21.15.84:0/2707608258 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55ecc6bb4050 0x55ecc6bb6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.715+0000 7f269aff5640 10 client.16914 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:03.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.715+0000 7f26b659c640 1 --2- 172.21.15.84:0/2707608258 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55ecc6bb4050 0x55ecc6bb6470 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90a34b7dded7ac54 server_cookie=e31840cd8e2f9efb in_seq=0 out_seq=0 2024-05-22T23:11:03.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.715+0000 7f269aff5640 1 -- 172.21.15.84:0/2707608258 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ecc6a9f170 con 0x55ecc6bb4050 2024-05-22T23:11:03.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.715+0000 7f269aff5640 10 client.16914 handle_command_reply: tid=0 2024-05-22T23:11:03.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.715+0000 7f26bffad740 1 client.16914 shutdown 2024-05-22T23:11:03.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.716+0000 7f26bffad740 1 -- 172.21.15.84:0/2707608258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b2e470 msgr2=0x55ecc6b370c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:03.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.716+0000 7f26bffad740 1 --2- 172.21.15.84:0/2707608258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b2e470 0x55ecc6b370c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f26a8011810 tx=0x7f26a801a610 comp rx=0 tx=0).stop 2024-05-22T23:11:03.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.716+0000 7f26bffad740 1 -- 172.21.15.84:0/2707608258 shutdown_connections 2024-05-22T23:11:03.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.716+0000 7f26bffad740 1 --2- 172.21.15.84:0/2707608258 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x55ecc6bb4050 0x55ecc6bb6470 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.716+0000 7f26bffad740 1 --2- 172.21.15.84:0/2707608258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecc6b3d050 0x55ecc6b37600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.716+0000 7f26bffad740 1 --2- 172.21.15.84:0/2707608258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ecc6b2e470 0x55ecc6b370c0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.716+0000 7f26bffad740 1 --2- 172.21.15.84:0/2707608258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecc6b2db50 0x55ecc6b36b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.716+0000 7f26bffad740 1 -- 172.21.15.84:0/2707608258 >> 172.21.15.84:0/2707608258 conn(0x55ecc6b29f70 msgr2=0x55ecc6b2f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:03.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.716+0000 7f269aff5640 0 client.16914 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:03.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.716+0000 7f26bffad740 1 -- 172.21.15.84:0/2707608258 shutdown_connections 2024-05-22T23:11:03.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.716+0000 7f26bffad740 1 -- 172.21.15.84:0/2707608258 wait complete. 2024-05-22T23:11:03.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.716+0000 7f26bffad740 20 client.16914 trim_cache size 0 max 16384 2024-05-22T23:11:03.734 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:11:03.734 INFO:teuthology.orchestra.run.smithi084.stdout: "return_code": 0, 2024-05-22T23:11:03.734 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_tag": "8f0b8e56-0083-47c7-afaa-4a8555a72a19", 2024-05-22T23:11:03.734 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": "asynchronous" 2024-05-22T23:11:03.734 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:11:03.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.716+0000 7f26bd1c1640 1 -- 172.21.15.84:0/1388388574 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f26a00a2d00 msgr2=0x7f26a00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:03.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.716+0000 7f26bd1c1640 1 --2- 172.21.15.84:0/1388388574 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f26a00a2d00 0x7f26a00a51c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f26b00cc5b0 tx=0x7f26b001d040 comp rx=0 tx=0).stop 2024-05-22T23:11:03.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.716+0000 7f26bd1c1640 1 -- 172.21.15.84:0/1388388574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26b819ca60 msgr2=0x7f26b819cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:03.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.716+0000 7f26bd1c1640 1 --2- 172.21.15.84:0/1388388574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26b819ca60 0x7f26b819cf10 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009fd0 tx=0x7f26b000b260 comp rx=0 tx=0).stop 2024-05-22T23:11:03.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.716+0000 7f26bd1c1640 1 -- 172.21.15.84:0/1388388574 shutdown_connections 2024-05-22T23:11:03.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.716+0000 7f26bd1c1640 1 --2- 172.21.15.84:0/1388388574 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f26a00a2d00 0x7f26a00a51c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.716+0000 7f26bd1c1640 1 --2- 172.21.15.84:0/1388388574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26b819ca60 0x7f26b819cf10 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.716+0000 7f26bd1c1640 1 --2- 172.21.15.84:0/1388388574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26b8106c00 0x7f26b81986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.716+0000 7f26bd1c1640 1 --2- 172.21.15.84:0/1388388574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26b80ffff0 0x7f26b81981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.716+0000 7f26bd1c1640 1 -- 172.21.15.84:0/1388388574 >> 172.21.15.84:0/1388388574 conn(0x7f26b8076d10 msgr2=0x7f26b8104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:03.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.717+0000 7f26bd1c1640 1 -- 172.21.15.84:0/1388388574 shutdown_connections 2024-05-22T23:11:03.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.717+0000 7f26bd1c1640 1 -- 172.21.15.84:0/1388388574 wait complete. 2024-05-22T23:11:03.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:11:03.737 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:11:03.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.879+0000 7ffade3f9640 1 -- 172.21.15.84:0/3620908268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffad8106750 msgr2=0x7ffad8106b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:03.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.879+0000 7ffade3f9640 1 --2- 172.21.15.84:0/3620908268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffad8106750 0x7ffad8106b30 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009930 tx=0x7ffac802a420 comp rx=0 tx=0).stop 2024-05-22T23:11:03.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.879+0000 7ffade3f9640 1 -- 172.21.15.84:0/3620908268 shutdown_connections 2024-05-22T23:11:03.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.879+0000 7ffade3f9640 1 --2- 172.21.15.84:0/3620908268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffad8101100 0x7ffad810d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.879+0000 7ffade3f9640 1 --2- 172.21.15.84:0/3620908268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffad8100740 0x7ffad8100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.879+0000 7ffade3f9640 1 --2- 172.21.15.84:0/3620908268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffad8106750 0x7ffad8106b30 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.879+0000 7ffade3f9640 1 -- 172.21.15.84:0/3620908268 >> 172.21.15.84:0/3620908268 conn(0x7ffad8077870 msgr2=0x7ffad8077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:03.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.879+0000 7ffade3f9640 1 -- 172.21.15.84:0/3620908268 shutdown_connections 2024-05-22T23:11:03.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.879+0000 7ffade3f9640 1 -- 172.21.15.84:0/3620908268 wait complete. 2024-05-22T23:11:03.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.880+0000 7ffade3f9640 1 Processor -- start 2024-05-22T23:11:03.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.880+0000 7ffade3f9640 1 -- start start 2024-05-22T23:11:03.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.880+0000 7ffade3f9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffad8100740 0x7ffad8198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.880+0000 7ffade3f9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffad8101100 0x7ffad81986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.880+0000 7ffade3f9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffad8106750 0x7ffad819ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.880+0000 7ffade3f9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ffad8111c60 con 0x7ffad8101100 2024-05-22T23:11:03.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.880+0000 7ffade3f9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffad8111ae0 con 0x7ffad8106750 2024-05-22T23:11:03.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.880+0000 7ffade3f9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffad8111de0 con 0x7ffad8100740 2024-05-22T23:11:03.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.880+0000 7ffacffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffad8101100 0x7ffad81986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.880+0000 7ffacffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffad8101100 0x7ffad81986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34540/0 (socket says 172.21.15.84:34540) 2024-05-22T23:11:03.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.880+0000 7ffacffff640 1 -- 172.21.15.84:0/2510711573 learned_addr learned my addr 172.21.15.84:0/2510711573 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:03.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.881+0000 7ffad7fff640 1 --2- 172.21.15.84:0/2510711573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffad8100740 0x7ffad8198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.881+0000 7ffadc996640 1 --2- 172.21.15.84:0/2510711573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffad8106750 0x7ffad819ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.881+0000 7ffacffff640 1 -- 172.21.15.84:0/2510711573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffad8100740 msgr2=0x7ffad8198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:03.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.881+0000 7ffacffff640 1 --2- 172.21.15.84:0/2510711573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffad8100740 0x7ffad8198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.881+0000 7ffacffff640 1 -- 172.21.15.84:0/2510711573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffad8106750 msgr2=0x7ffad819ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:03.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.881+0000 7ffacffff640 1 --2- 172.21.15.84:0/2510711573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffad8106750 0x7ffad819ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.881+0000 7ffacffff640 1 -- 172.21.15.84:0/2510711573 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad819d150 con 0x7ffad8101100 2024-05-22T23:11:03.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.881+0000 7ffacffff640 1 --2- 172.21.15.84:0/2510711573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffad8101100 0x7ffad81986a0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ffac000c970 tx=0x7ffac000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:03.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.881+0000 7ffad5ffb640 1 -- 172.21.15.84:0/2510711573 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac0007bb0 con 0x7ffad8101100 2024-05-22T23:11:03.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.881+0000 7ffad5ffb640 1 -- 172.21.15.84:0/2510711573 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffac0007d50 con 0x7ffad8101100 2024-05-22T23:11:03.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.881+0000 7ffade3f9640 1 -- 172.21.15.84:0/2510711573 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad819d440 con 0x7ffad8101100 2024-05-22T23:11:03.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.881+0000 7ffad5ffb640 1 -- 172.21.15.84:0/2510711573 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac000f450 con 0x7ffad8101100 2024-05-22T23:11:03.892 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.881+0000 7ffade3f9640 1 -- 172.21.15.84:0/2510711573 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad81a4cd0 con 0x7ffad8101100 2024-05-22T23:11:03.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.883+0000 7ffad5ffb640 1 -- 172.21.15.84:0/2510711573 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ffac0005020 con 0x7ffad8101100 2024-05-22T23:11:03.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.884+0000 7ffae11e8740 1 Processor -- start 2024-05-22T23:11:03.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.884+0000 7ffae11e8740 1 -- start start 2024-05-22T23:11:03.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.884+0000 7ffae11e8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571ed8d14a0 0x5571ed8cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.884+0000 7ffad5ffb640 1 --2- 172.21.15.84:0/2510711573 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ffac40a2d00 0x7ffac40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.884+0000 7ffae11e8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571ed8cfdb0 0x5571ed8d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.884+0000 7ffae11e8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8def40 0x5571ed8e1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.884+0000 7ffae11e8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5571ed833db0 con 0x5571ed8def40 2024-05-22T23:11:03.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.884+0000 7ffae11e8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5571ed7d2360 con 0x5571ed8cfdb0 2024-05-22T23:11:03.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.884+0000 7ffae11e8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5571ed82fd70 con 0x5571ed8d14a0 2024-05-22T23:11:03.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.884+0000 7ffacffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571ed8cfdb0 0x5571ed8d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.884+0000 7ffadc996640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571ed8d14a0 0x5571ed8cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.884+0000 7ffad7fff640 1 --2- 172.21.15.84:0/2510711573 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ffac40a2d00 0x7ffac40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.884+0000 7ffadc996640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8def40 0x5571ed8e1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.884+0000 7ffacffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571ed8cfdb0 0x5571ed8d0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:33088/0 (socket says 172.21.15.84:33088) 2024-05-22T23:11:03.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.884+0000 7ffacffff640 1 -- 172.21.15.84:0/3369694258 learned_addr learned my addr 172.21.15.84:0/3369694258 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:03.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.884+0000 7ffad5ffb640 1 -- 172.21.15.84:0/2510711573 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7ffac00cf120 con 0x7ffad8101100 2024-05-22T23:11:03.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.885+0000 7ffacffff640 1 -- 172.21.15.84:0/3369694258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571ed8d14a0 msgr2=0x5571ed8cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:03.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.885+0000 7ffacffff640 1 --2- 172.21.15.84:0/3369694258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571ed8d14a0 0x5571ed8cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.885+0000 7ffacffff640 1 -- 172.21.15.84:0/3369694258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8def40 msgr2=0x5571ed8e1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:03.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.885+0000 7ffacffff640 1 --2- 172.21.15.84:0/3369694258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8def40 0x5571ed8e1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.885+0000 7ffacffff640 1 -- 172.21.15.84:0/3369694258 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ed7f5a60 con 0x5571ed8cfdb0 2024-05-22T23:11:03.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.885+0000 7ffadc996640 1 --2- 172.21.15.84:0/3369694258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8def40 0x5571ed8e1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:11:03.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.885+0000 7ffacffff640 1 --2- 172.21.15.84:0/3369694258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571ed8cfdb0 0x5571ed8d0230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffac0097bc0 tx=0x7ffac000a570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91d35dac500aba7d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:03.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.885+0000 7ffad7fff640 1 --2- 172.21.15.84:0/2510711573 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ffac40a2d00 0x7ffac40a51c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009900 tx=0x7ffac80058d0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:03.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.885+0000 7fface7fc640 1 -- 172.21.15.84:0/3369694258 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac001d070 con 0x5571ed8cfdb0 2024-05-22T23:11:03.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.885+0000 7fface7fc640 1 -- 172.21.15.84:0/3369694258 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffac0082070 con 0x5571ed8cfdb0 2024-05-22T23:11:03.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.885+0000 7fface7fc640 1 -- 172.21.15.84:0/3369694258 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac009c9e0 con 0x5571ed8cfdb0 2024-05-22T23:11:03.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.886+0000 7ffae11e8740 1 -- 172.21.15.84:0/3369694258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571ed8cfdb0 msgr2=0x5571ed8d0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:03.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.886+0000 7ffae11e8740 1 --2- 172.21.15.84:0/3369694258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571ed8cfdb0 0x5571ed8d0230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffac0097bc0 tx=0x7ffac000a570 comp rx=0 tx=0).stop 2024-05-22T23:11:03.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.886+0000 7ffae11e8740 1 -- 172.21.15.84:0/3369694258 shutdown_connections 2024-05-22T23:11:03.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.886+0000 7ffae11e8740 1 --2- 172.21.15.84:0/3369694258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8def40 0x5571ed8e1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.886+0000 7ffae11e8740 1 --2- 172.21.15.84:0/3369694258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571ed8cfdb0 0x5571ed8d0230 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.886+0000 7ffae11e8740 1 --2- 172.21.15.84:0/3369694258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571ed8d14a0 0x5571ed8cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.886+0000 7ffae11e8740 1 -- 172.21.15.84:0/3369694258 >> 172.21.15.84:0/3369694258 conn(0x5571ed8988e0 msgr2=0x5571ed8cc310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:03.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.886+0000 7ffae11e8740 1 -- 172.21.15.84:0/3369694258 shutdown_connections 2024-05-22T23:11:03.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.886+0000 7ffae11e8740 1 -- 172.21.15.84:0/3369694258 wait complete. 2024-05-22T23:11:03.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.886+0000 7ffae11e8740 1 -- start start 2024-05-22T23:11:03.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.886+0000 7ffae11e8740 1 Processor -- start 2024-05-22T23:11:03.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.887+0000 7ffae11e8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571ed8cfdb0 0x5571ed8dbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.887+0000 7ffae11e8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8d14a0 0x5571ed8dc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.887+0000 7ffae11e8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571ed8def40 0x5571ed8dc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.887+0000 7ffae11e8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5571ed8e1dd0 con 0x5571ed8d14a0 2024-05-22T23:11:03.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.887+0000 7ffae11e8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5571ed7ee0f0 con 0x5571ed8def40 2024-05-22T23:11:03.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.887+0000 7ffae11e8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5571ed8e1f50 con 0x5571ed8cfdb0 2024-05-22T23:11:03.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.887+0000 7ffae11e8740 10 client.0 _ll_register_callbacks cb 0x5571ed7f2f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:11:03.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.887+0000 7ffacffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8d14a0 0x5571ed8dc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.887+0000 7ffacffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8d14a0 0x5571ed8dc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34568/0 (socket says 172.21.15.84:34568) 2024-05-22T23:11:03.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.887+0000 7ffacffff640 1 -- 172.21.15.84:0/913020439 learned_addr learned my addr 172.21.15.84:0/913020439 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:03.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.887+0000 7ffadc996640 1 --2- 172.21.15.84:0/913020439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571ed8def40 0x5571ed8dc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.887+0000 7ffad7fff640 1 --2- 172.21.15.84:0/913020439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571ed8cfdb0 0x5571ed8dbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.887+0000 7ffacffff640 1 -- 172.21.15.84:0/913020439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571ed8cfdb0 msgr2=0x5571ed8dbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:03.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.887+0000 7ffacffff640 1 --2- 172.21.15.84:0/913020439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571ed8cfdb0 0x5571ed8dbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.887+0000 7ffacffff640 1 -- 172.21.15.84:0/913020439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571ed8def40 msgr2=0x5571ed8dc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:03.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.887+0000 7ffacffff640 1 --2- 172.21.15.84:0/913020439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571ed8def40 0x5571ed8dc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.887+0000 7ffacffff640 1 -- 172.21.15.84:0/913020439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571ed6117d0 con 0x5571ed8d14a0 2024-05-22T23:11:03.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.887+0000 7ffacffff640 1 --2- 172.21.15.84:0/913020439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8d14a0 0x5571ed8dc3e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ffac00a3df0 tx=0x7ffac0005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf71d2113595f35 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:03.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.887+0000 7ffab77fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:11:03.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.887+0000 7ffab77fe640 1 -- 172.21.15.84:0/913020439 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac007e030 con 0x5571ed8d14a0 2024-05-22T23:11:03.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.887+0000 7ffab77fe640 1 -- 172.21.15.84:0/913020439 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7ffac0099a60 con 0x5571ed8d14a0 2024-05-22T23:11:03.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.887+0000 7ffae11e8740 1 -- 172.21.15.84:0/913020439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffac0082070 con 0x5571ed8d14a0 2024-05-22T23:11:03.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.888+0000 7ffab77fe640 1 -- 172.21.15.84:0/913020439 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7ffac001d070 con 0x5571ed8d14a0 2024-05-22T23:11:03.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.888+0000 7ffae11e8740 10 client.16926 fetch_fsmap learned FSMap version 18 2024-05-22T23:11:03.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.888+0000 7ffae11e8740 1 -- 172.21.15.84:0/913020439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5571ed5473f0 con 0x5571ed8d14a0 2024-05-22T23:11:03.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.888+0000 7ffab77fe640 1 -- 172.21.15.84:0/913020439 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffac0082070 con 0x5571ed8d14a0 2024-05-22T23:11:03.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.888+0000 7ffab77fe640 1 -- 172.21.15.84:0/913020439 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7ffac00bb070 con 0x5571ed8d14a0 2024-05-22T23:11:03.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.888+0000 7ffae11e8740 10 client.16926 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:11:03.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.888+0000 7ffae11e8740 10 client.16926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:11:03.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.888+0000 7ffae11e8740 20 client.16926 populate_metadata read hostname 'smithi084' 2024-05-22T23:11:03.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.888+0000 7ffae11e8740 1 --2- 172.21.15.84:0/913020439 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5571ed8ef290 0x5571ed8f1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.888+0000 7ffae11e8740 4 client.16926 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:11:03.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.888+0000 7ffae11e8740 1 -- 172.21.15.84:0/913020439 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571ed8414e0 con 0x5571ed8ef290 2024-05-22T23:11:03.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.888+0000 7ffad7fff640 1 --2- 172.21.15.84:0/913020439 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5571ed8ef290 0x5571ed8f1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.888+0000 7ffad7fff640 1 --2- 172.21.15.84:0/913020439 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5571ed8ef290 0x5571ed8f1680 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ee116403b74e87a server_cookie=2d89c6cfaa90587d in_seq=0 out_seq=0 2024-05-22T23:11:03.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.888+0000 7ffab77fe640 10 client.16926 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:03.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.889+0000 7ffab77fe640 1 -- 172.21.15.84:0/913020439 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5571ed8414e0 con 0x5571ed8ef290 2024-05-22T23:11:03.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.889+0000 7ffab77fe640 10 client.16926 handle_command_reply: tid=0 2024-05-22T23:11:03.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.889+0000 7ffae11e8740 1 client.16926 shutdown 2024-05-22T23:11:03.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.889+0000 7ffae11e8740 1 -- 172.21.15.84:0/913020439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8d14a0 msgr2=0x5571ed8dc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:03.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.889+0000 7ffae11e8740 1 --2- 172.21.15.84:0/913020439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8d14a0 0x5571ed8dc3e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ffac00a3df0 tx=0x7ffac0005700 comp rx=0 tx=0).stop 2024-05-22T23:11:03.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.889+0000 7ffae11e8740 1 -- 172.21.15.84:0/913020439 shutdown_connections 2024-05-22T23:11:03.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.889+0000 7ffae11e8740 1 --2- 172.21.15.84:0/913020439 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5571ed8ef290 0x5571ed8f1680 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.889+0000 7ffae11e8740 1 --2- 172.21.15.84:0/913020439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571ed8def40 0x5571ed8dc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.889+0000 7ffae11e8740 1 --2- 172.21.15.84:0/913020439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8d14a0 0x5571ed8dc3e0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.889+0000 7ffab77fe640 0 client.16926 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:03.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.889+0000 7ffae11e8740 1 --2- 172.21.15.84:0/913020439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571ed8cfdb0 0x5571ed8dbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.889+0000 7ffae11e8740 1 -- 172.21.15.84:0/913020439 >> 172.21.15.84:0/913020439 conn(0x5571ed8988e0 msgr2=0x5571ed8d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:03.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.889+0000 7ffae11e8740 1 -- 172.21.15.84:0/913020439 shutdown_connections 2024-05-22T23:11:03.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.889+0000 7ffae11e8740 1 -- 172.21.15.84:0/913020439 wait complete. 2024-05-22T23:11:03.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.889+0000 7ffae11e8740 20 client.16926 trim_cache size 0 max 16384 2024-05-22T23:11:03.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.907+0000 7ffae11e8740 1 Processor -- start 2024-05-22T23:11:03.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.907+0000 7ffae11e8740 1 -- start start 2024-05-22T23:11:03.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.907+0000 7ffae11e8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8cfdb0 0x5571ed8ddde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.907+0000 7ffae11e8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571ed8d14a0 0x5571ed8f46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.907+0000 7ffae11e8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571ed8def40 0x5571ed8f4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.907+0000 7ffae11e8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ffad81a94b0 con 0x5571ed8cfdb0 2024-05-22T23:11:03.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.907+0000 7ffae11e8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffad81abd10 con 0x5571ed8def40 2024-05-22T23:11:03.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.907+0000 7ffae11e8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffad81a9b00 con 0x5571ed8d14a0 2024-05-22T23:11:03.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.907+0000 7ffad7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8cfdb0 0x5571ed8ddde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.907+0000 7ffad7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8cfdb0 0x5571ed8ddde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34574/0 (socket says 172.21.15.84:34574) 2024-05-22T23:11:03.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.907+0000 7ffad7fff640 1 -- 172.21.15.84:0/3281644165 learned_addr learned my addr 172.21.15.84:0/3281644165 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:03.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.907+0000 7ffacffff640 1 --2- 172.21.15.84:0/3281644165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571ed8d14a0 0x5571ed8f46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.907+0000 7ffadc996640 1 --2- 172.21.15.84:0/3281644165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571ed8def40 0x5571ed8f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.907+0000 7ffad7fff640 1 -- 172.21.15.84:0/3281644165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571ed8d14a0 msgr2=0x5571ed8f46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:03.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.907+0000 7ffad7fff640 1 --2- 172.21.15.84:0/3281644165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571ed8d14a0 0x5571ed8f46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.907+0000 7ffad7fff640 1 -- 172.21.15.84:0/3281644165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571ed8def40 msgr2=0x5571ed8f4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:03.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.907+0000 7ffad7fff640 1 --2- 172.21.15.84:0/3281644165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571ed8def40 0x5571ed8f4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.907+0000 7ffad7fff640 1 -- 172.21.15.84:0/3281644165 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ed5070f0 con 0x5571ed8cfdb0 2024-05-22T23:11:03.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.907+0000 7ffacffff640 1 --2- 172.21.15.84:0/3281644165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571ed8d14a0 0x5571ed8f46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:11:03.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.907+0000 7ffadc996640 1 --2- 172.21.15.84:0/3281644165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571ed8def40 0x5571ed8f4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:11:03.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.908+0000 7ffad7fff640 1 --2- 172.21.15.84:0/3281644165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8cfdb0 0x5571ed8ddde0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ffac8006010 tx=0x7ffac8040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a431df14fe49d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:03.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.908+0000 7ffacdffb640 1 -- 172.21.15.84:0/3281644165 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac8055070 con 0x5571ed8cfdb0 2024-05-22T23:11:03.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.908+0000 7ffacdffb640 1 -- 172.21.15.84:0/3281644165 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffac8050730 con 0x5571ed8cfdb0 2024-05-22T23:11:03.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.908+0000 7ffacdffb640 1 -- 172.21.15.84:0/3281644165 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac8035040 con 0x5571ed8cfdb0 2024-05-22T23:11:03.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.908+0000 7ffae11e8740 1 -- 172.21.15.84:0/3281644165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8cfdb0 msgr2=0x5571ed8ddde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:03.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.908+0000 7ffae11e8740 1 --2- 172.21.15.84:0/3281644165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8cfdb0 0x5571ed8ddde0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ffac8006010 tx=0x7ffac8040000 comp rx=0 tx=0).stop 2024-05-22T23:11:03.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.908+0000 7ffae11e8740 1 -- 172.21.15.84:0/3281644165 shutdown_connections 2024-05-22T23:11:03.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.908+0000 7ffae11e8740 1 --2- 172.21.15.84:0/3281644165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571ed8def40 0x5571ed8f4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.908+0000 7ffae11e8740 1 --2- 172.21.15.84:0/3281644165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571ed8d14a0 0x5571ed8f46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.908+0000 7ffae11e8740 1 --2- 172.21.15.84:0/3281644165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8cfdb0 0x5571ed8ddde0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.908+0000 7ffae11e8740 1 -- 172.21.15.84:0/3281644165 >> 172.21.15.84:0/3281644165 conn(0x5571ed8988e0 msgr2=0x5571ed8dd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:03.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.908+0000 7ffae11e8740 1 -- 172.21.15.84:0/3281644165 shutdown_connections 2024-05-22T23:11:03.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.908+0000 7ffae11e8740 1 -- 172.21.15.84:0/3281644165 wait complete. 2024-05-22T23:11:03.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.909+0000 7ffae11e8740 1 -- start start 2024-05-22T23:11:03.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.909+0000 7ffae11e8740 1 Processor -- start 2024-05-22T23:11:03.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.909+0000 7ffae11e8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571ed8cfdb0 0x5571ed8f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.909+0000 7ffae11e8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571ed8d14a0 0x5571ed8f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.909+0000 7ffae11e8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8def40 0x5571ed8f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.909+0000 7ffae11e8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ffad81aa1d0 con 0x5571ed8def40 2024-05-22T23:11:03.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.909+0000 7ffae11e8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffad81ab640 con 0x5571ed8d14a0 2024-05-22T23:11:03.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.909+0000 7ffae11e8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffad81aa8a0 con 0x5571ed8cfdb0 2024-05-22T23:11:03.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.909+0000 7ffae11e8740 10 client.0 _ll_register_callbacks cb 0x7ffad0009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:11:03.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.909+0000 7ffadc996640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8def40 0x5571ed8f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.909+0000 7ffadc996640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8def40 0x5571ed8f21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34582/0 (socket says 172.21.15.84:34582) 2024-05-22T23:11:03.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.909+0000 7ffadc996640 1 -- 172.21.15.84:0/671340420 learned_addr learned my addr 172.21.15.84:0/671340420 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:03.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.909+0000 7ffacffff640 1 --2- 172.21.15.84:0/671340420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571ed8d14a0 0x5571ed8f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.909+0000 7ffad7fff640 1 --2- 172.21.15.84:0/671340420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571ed8cfdb0 0x5571ed8f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.910+0000 7ffadc996640 1 -- 172.21.15.84:0/671340420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571ed8cfdb0 msgr2=0x5571ed8f46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:03.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.910+0000 7ffadc996640 1 --2- 172.21.15.84:0/671340420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571ed8cfdb0 0x5571ed8f46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.910+0000 7ffadc996640 1 -- 172.21.15.84:0/671340420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571ed8d14a0 msgr2=0x5571ed8f1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:03.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.910+0000 7ffadc996640 1 --2- 172.21.15.84:0/671340420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571ed8d14a0 0x5571ed8f1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.910+0000 7ffadc996640 1 -- 172.21.15.84:0/671340420 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571ed7f2fe0 con 0x5571ed8def40 2024-05-22T23:11:03.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.910+0000 7ffadc996640 1 --2- 172.21.15.84:0/671340420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8def40 0x5571ed8f21c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7ffad0003ee0 tx=0x7ffad0011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517a6a2831b681bc server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:03.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.910+0000 7ffab77fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:11:03.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.910+0000 7ffab77fe640 1 -- 172.21.15.84:0/671340420 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad0019040 con 0x5571ed8def40 2024-05-22T23:11:03.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.910+0000 7ffab77fe640 1 -- 172.21.15.84:0/671340420 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8571+0+0 (secure 0 0 0) 0x7ffad0004b20 con 0x5571ed8def40 2024-05-22T23:11:03.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.910+0000 7ffae11e8740 1 -- 172.21.15.84:0/671340420 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffac8050730 con 0x5571ed8def40 2024-05-22T23:11:03.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.910+0000 7ffab77fe640 1 -- 172.21.15.84:0/671340420 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7ffad0004d40 con 0x5571ed8def40 2024-05-22T23:11:03.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.910+0000 7ffae11e8740 10 client.16932 fetch_fsmap learned FSMap version 18 2024-05-22T23:11:03.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.910+0000 7ffae11e8740 1 -- 172.21.15.84:0/671340420 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5571ed7c41c0 con 0x5571ed8def40 2024-05-22T23:11:03.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.910+0000 7ffab77fe640 1 -- 172.21.15.84:0/671340420 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffac8050730 con 0x5571ed8def40 2024-05-22T23:11:03.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.910+0000 7ffab77fe640 1 -- 172.21.15.84:0/671340420 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7ffad000f560 con 0x5571ed8def40 2024-05-22T23:11:03.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.910+0000 7ffae11e8740 10 client.16932 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:11:03.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.910+0000 7ffae11e8740 10 client.16932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:11:03.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.910+0000 7ffae11e8740 20 client.16932 populate_metadata read hostname 'smithi084' 2024-05-22T23:11:03.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.910+0000 7ffae11e8740 1 --2- 172.21.15.84:0/671340420 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5571ed955590 0x5571ed957980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:03.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.910+0000 7ffae11e8740 4 client.16932 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:11:03.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.910+0000 7ffae11e8740 1 -- 172.21.15.84:0/671340420 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5571ed83b830 con 0x5571ed955590 2024-05-22T23:11:03.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.911+0000 7ffad7fff640 1 --2- 172.21.15.84:0/671340420 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5571ed955590 0x5571ed957980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:03.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.911+0000 7ffab77fe640 10 client.16932 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:03.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.911+0000 7ffad7fff640 1 --2- 172.21.15.84:0/671340420 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5571ed955590 0x5571ed957980 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2a3df43dea1ce9f server_cookie=5cd1ca4eaf2c4da5 in_seq=0 out_seq=0 2024-05-22T23:11:03.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.911+0000 7ffab77fe640 1 -- 172.21.15.84:0/671340420 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5571ed83b830 con 0x5571ed955590 2024-05-22T23:11:03.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.911+0000 7ffab77fe640 10 client.16932 handle_command_reply: tid=0 2024-05-22T23:11:03.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.911+0000 7ffae11e8740 1 client.16932 shutdown 2024-05-22T23:11:03.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.911+0000 7ffae11e8740 1 -- 172.21.15.84:0/671340420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8def40 msgr2=0x5571ed8f21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:03.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.911+0000 7ffae11e8740 1 --2- 172.21.15.84:0/671340420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8def40 0x5571ed8f21c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7ffad0003ee0 tx=0x7ffad0011b30 comp rx=0 tx=0).stop 2024-05-22T23:11:03.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.911+0000 7ffae11e8740 1 -- 172.21.15.84:0/671340420 shutdown_connections 2024-05-22T23:11:03.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.911+0000 7ffae11e8740 1 --2- 172.21.15.84:0/671340420 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5571ed955590 0x5571ed957980 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.911+0000 7ffae11e8740 1 --2- 172.21.15.84:0/671340420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571ed8def40 0x5571ed8f21c0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.911+0000 7ffae11e8740 1 --2- 172.21.15.84:0/671340420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571ed8d14a0 0x5571ed8f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.911+0000 7ffae11e8740 1 --2- 172.21.15.84:0/671340420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571ed8cfdb0 0x5571ed8f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.911+0000 7ffae11e8740 1 -- 172.21.15.84:0/671340420 >> 172.21.15.84:0/671340420 conn(0x5571ed8988e0 msgr2=0x5571ed8dc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:03.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.911+0000 7ffab77fe640 0 client.16932 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:03.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.911+0000 7ffae11e8740 1 -- 172.21.15.84:0/671340420 shutdown_connections 2024-05-22T23:11:03.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.911+0000 7ffae11e8740 1 -- 172.21.15.84:0/671340420 wait complete. 2024-05-22T23:11:03.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.911+0000 7ffae11e8740 20 client.16932 trim_cache size 0 max 16384 2024-05-22T23:11:03.929 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:11:03.930 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-22T23:11:03.930 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": { 2024-05-22T23:11:03.930 INFO:teuthology.orchestra.run.smithi084.stdout: "8f0b8e56-0083-47c7-afaa-4a8555a72a19": { 2024-05-22T23:11:03.930 INFO:teuthology.orchestra.run.smithi084.stdout: "path": "/", 2024-05-22T23:11:03.930 INFO:teuthology.orchestra.run.smithi084.stdout: "tag": "8f0b8e56-0083-47c7-afaa-4a8555a72a19", 2024-05-22T23:11:03.930 INFO:teuthology.orchestra.run.smithi084.stdout: "options": "recursive,force" 2024-05-22T23:11:03.930 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:11:03.930 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:11:03.931 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:11:03.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.912+0000 7ffade3f9640 1 -- 172.21.15.84:0/2510711573 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ffac40a2d00 msgr2=0x7ffac40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:03.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.912+0000 7ffade3f9640 1 --2- 172.21.15.84:0/2510711573 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ffac40a2d00 0x7ffac40a51c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009900 tx=0x7ffac80058d0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.912+0000 7ffade3f9640 1 -- 172.21.15.84:0/2510711573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffad8101100 msgr2=0x7ffad81986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:03.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.912+0000 7ffade3f9640 1 --2- 172.21.15.84:0/2510711573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffad8101100 0x7ffad81986a0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ffac000c970 tx=0x7ffac000ce30 comp rx=0 tx=0).stop 2024-05-22T23:11:03.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.912+0000 7ffade3f9640 1 -- 172.21.15.84:0/2510711573 shutdown_connections 2024-05-22T23:11:03.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.912+0000 7ffade3f9640 1 --2- 172.21.15.84:0/2510711573 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ffac40a2d00 0x7ffac40a51c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.912+0000 7ffade3f9640 1 --2- 172.21.15.84:0/2510711573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffad8106750 0x7ffad819ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.912+0000 7ffade3f9640 1 --2- 172.21.15.84:0/2510711573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffad8101100 0x7ffad81986a0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.912+0000 7ffade3f9640 1 --2- 172.21.15.84:0/2510711573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffad8100740 0x7ffad8198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:03.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.912+0000 7ffade3f9640 1 -- 172.21.15.84:0/2510711573 >> 172.21.15.84:0/2510711573 conn(0x7ffad8077870 msgr2=0x7ffad8103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:03.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.912+0000 7ffade3f9640 1 -- 172.21.15.84:0/2510711573 shutdown_connections 2024-05-22T23:11:03.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:03.912+0000 7ffade3f9640 1 -- 172.21.15.84:0/2510711573 wait complete. 2024-05-22T23:11:03.934 INFO:tasks.cephfs.filesystem:scrub status for tag:8f0b8e56-0083-47c7-afaa-4a8555a72a19 - {'path': '/', 'tag': '8f0b8e56-0083-47c7-afaa-4a8555a72a19', 'options': 'recursive,force'} 2024-05-22T23:11:04.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:04 smithi096 ceph-mon[36392]: scrub queued for path: / 2024-05-22T23:11:04.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:04 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:11:04.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:04 smithi096 ceph-mon[36392]: scrub summary: active paths [/] 2024-05-22T23:11:04.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:04 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:11:04.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:04 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:11:04.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:04 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:11:04.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:04 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:11:04.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:04 smithi084 ceph-mon[32186]: scrub queued for path: / 2024-05-22T23:11:04.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:04 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:11:04.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:04 smithi084 ceph-mon[32186]: scrub summary: active paths [/] 2024-05-22T23:11:04.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:04 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:11:04.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:04 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:11:04.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:04 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:11:04.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:04 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:11:04.667 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:04 smithi204 ceph-mon[36211]: scrub queued for path: / 2024-05-22T23:11:04.668 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:04 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:11:04.668 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:04 smithi204 ceph-mon[36211]: scrub summary: active paths [/] 2024-05-22T23:11:04.668 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:04 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:11:04.668 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:04 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:11:04.668 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:04 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:11:04.668 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:04 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:11:05.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:05 smithi096 ceph-mon[36392]: pgmap v286: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-05-22T23:11:05.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:05 smithi084 ceph-mon[32186]: pgmap v286: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-05-22T23:11:05.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:05 smithi204 ceph-mon[36211]: pgmap v286: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-05-22T23:11:06.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:06 smithi096 ceph-mon[36392]: scrub summary: idle 2024-05-22T23:11:06.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:06 smithi084 ceph-mon[32186]: scrub summary: idle 2024-05-22T23:11:06.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:06 smithi204 ceph-mon[36211]: scrub summary: idle 2024-05-22T23:11:07.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:07 smithi096 ceph-mon[36392]: pgmap v287: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 21 op/s 2024-05-22T23:11:07.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:07 smithi096 ceph-mon[36392]: osdmap e96: 12 total, 12 up, 12 in 2024-05-22T23:11:07.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:07 smithi084 ceph-mon[32186]: pgmap v287: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 21 op/s 2024-05-22T23:11:07.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:07 smithi084 ceph-mon[32186]: osdmap e96: 12 total, 12 up, 12 in 2024-05-22T23:11:07.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:07 smithi204 ceph-mon[36211]: pgmap v287: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 21 op/s 2024-05-22T23:11:07.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:07 smithi204 ceph-mon[36211]: osdmap e96: 12 total, 12 up, 12 in 2024-05-22T23:11:09.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:09 smithi096 ceph-mon[36392]: pgmap v289: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-05-22T23:11:09.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:09 smithi084 ceph-mon[32186]: pgmap v289: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-05-22T23:11:09.667 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:09 smithi204 ceph-mon[36211]: pgmap v289: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-05-22T23:11:11.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:11 smithi096 ceph-mon[36392]: pgmap v290: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-05-22T23:11:11.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:11 smithi084 ceph-mon[32186]: pgmap v290: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-05-22T23:11:11.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:11 smithi204 ceph-mon[36211]: pgmap v290: 97 pgs: 97 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-05-22T23:11:13.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:13 smithi096 ceph-mon[36392]: pgmap v291: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 204 B/s wr, 4 op/s 2024-05-22T23:11:13.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:13 smithi084 ceph-mon[32186]: pgmap v291: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 204 B/s wr, 4 op/s 2024-05-22T23:11:13.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:13 smithi204 ceph-mon[36211]: pgmap v291: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 204 B/s wr, 4 op/s 2024-05-22T23:11:15.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:15 smithi096 ceph-mon[36392]: pgmap v292: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 204 B/s wr, 4 op/s 2024-05-22T23:11:15.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:15 smithi084 ceph-mon[32186]: pgmap v292: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 204 B/s wr, 4 op/s 2024-05-22T23:11:15.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:15 smithi204 ceph-mon[36211]: pgmap v292: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 204 B/s wr, 4 op/s 2024-05-22T23:11:17.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:17 smithi096 ceph-mon[36392]: pgmap v293: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-22T23:11:17.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:17 smithi096 ceph-mon[36392]: osdmap e97: 12 total, 12 up, 12 in 2024-05-22T23:11:17.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:17 smithi084 ceph-mon[32186]: pgmap v293: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-22T23:11:17.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:17 smithi084 ceph-mon[32186]: osdmap e97: 12 total, 12 up, 12 in 2024-05-22T23:11:17.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:17 smithi204 ceph-mon[36211]: pgmap v293: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-22T23:11:17.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:17 smithi204 ceph-mon[36211]: osdmap e97: 12 total, 12 up, 12 in 2024-05-22T23:11:19.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:19 smithi096 ceph-mon[36392]: pgmap v295: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-22T23:11:19.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:19 smithi084 ceph-mon[32186]: pgmap v295: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-22T23:11:19.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:19 smithi204 ceph-mon[36211]: pgmap v295: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-22T23:11:21.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:21 smithi096 ceph-mon[36392]: pgmap v296: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-22T23:11:21.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:21 smithi084 ceph-mon[32186]: pgmap v296: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-22T23:11:21.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:21 smithi204 ceph-mon[36211]: pgmap v296: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-22T23:11:23.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:23 smithi096 ceph-mon[36392]: pgmap v297: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-22T23:11:23.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:23 smithi084 ceph-mon[32186]: pgmap v297: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-22T23:11:23.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:23 smithi204 ceph-mon[36211]: pgmap v297: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-22T23:11:25.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:25 smithi096 ceph-mon[36392]: pgmap v298: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-22T23:11:25.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:25 smithi084 ceph-mon[32186]: pgmap v298: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-22T23:11:25.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:25 smithi204 ceph-mon[36211]: pgmap v298: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-22T23:11:27.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:27 smithi096 ceph-mon[36392]: pgmap v299: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-22T23:11:27.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:27 smithi084 ceph-mon[32186]: pgmap v299: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-22T23:11:27.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:27 smithi204 ceph-mon[36211]: pgmap v299: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-22T23:11:29.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:29 smithi084 ceph-mon[32186]: pgmap v300: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 935 B/s rd, 1 op/s 2024-05-22T23:11:29.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:29 smithi204 ceph-mon[36211]: pgmap v300: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 935 B/s rd, 1 op/s 2024-05-22T23:11:29.805 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:29 smithi096 ceph-mon[36392]: pgmap v300: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 935 B/s rd, 1 op/s 2024-05-22T23:11:31.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:31 smithi084 ceph-mon[32186]: pgmap v301: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:11:31.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:31 smithi204 ceph-mon[36211]: pgmap v301: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:11:31.805 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:31 smithi096 ceph-mon[36392]: pgmap v301: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:11:33.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:33 smithi084 ceph-mon[32186]: pgmap v302: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:11:33.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:33 smithi204 ceph-mon[36211]: pgmap v302: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:11:33.805 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:33 smithi096 ceph-mon[36392]: pgmap v302: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:11:33.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:11:33.935 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:11:34.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.044+0000 7fecfff16640 1 -- 172.21.15.84:0/3056176096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf8101840 msgr2=0x7fecf8101cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:34.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.044+0000 7fecfff16640 1 --2- 172.21.15.84:0/3056176096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf8101840 0x7fecf8101cc0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009930 tx=0x7fecf402a420 comp rx=0 tx=0).stop 2024-05-22T23:11:34.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.044+0000 7fecfff16640 1 -- 172.21.15.84:0/3056176096 shutdown_connections 2024-05-22T23:11:34.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.044+0000 7fecfff16640 1 --2- 172.21.15.84:0/3056176096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecf8102200 0x7fecf810e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.044+0000 7fecfff16640 1 --2- 172.21.15.84:0/3056176096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf8101840 0x7fecf8101cc0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.044+0000 7fecfff16640 1 --2- 172.21.15.84:0/3056176096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecf8107850 0x7fecf8107c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.044+0000 7fecfff16640 1 -- 172.21.15.84:0/3056176096 >> 172.21.15.84:0/3056176096 conn(0x7fecf80fd2b0 msgr2=0x7fecf80ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:34.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.044+0000 7fecfff16640 1 -- 172.21.15.84:0/3056176096 shutdown_connections 2024-05-22T23:11:34.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.044+0000 7fecfff16640 1 -- 172.21.15.84:0/3056176096 wait complete. 2024-05-22T23:11:34.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.045+0000 7fecfff16640 1 Processor -- start 2024-05-22T23:11:34.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.045+0000 7fecfff16640 1 -- start start 2024-05-22T23:11:34.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.045+0000 7fecfff16640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf8101840 0x7fecf81981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.045+0000 7fecfff16640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecf8102200 0x7fecf8198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.045+0000 7fecfff16640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecf8107850 0x7fecf819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.045+0000 7fecfff16640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fecf8111c60 con 0x7fecf8101840 2024-05-22T23:11:34.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.045+0000 7fecfff16640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fecf8111ae0 con 0x7fecf8107850 2024-05-22T23:11:34.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.045+0000 7fecfff16640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fecf8111de0 con 0x7fecf8102200 2024-05-22T23:11:34.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.045+0000 7fecfdcb2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf8101840 0x7fecf81981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.045+0000 7fecfdcb2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf8101840 0x7fecf81981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48448/0 (socket says 172.21.15.84:48448) 2024-05-22T23:11:34.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.045+0000 7fecfdcb2640 1 -- 172.21.15.84:0/2987536857 learned_addr learned my addr 172.21.15.84:0/2987536857 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:34.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.045+0000 7fecfd4b1640 1 --2- 172.21.15.84:0/2987536857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecf8102200 0x7fecf8198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.045+0000 7fecfe4b3640 1 --2- 172.21.15.84:0/2987536857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecf8107850 0x7fecf819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.045+0000 7fecfdcb2640 1 -- 172.21.15.84:0/2987536857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecf8102200 msgr2=0x7fecf8198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:34.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.045+0000 7fecfdcb2640 1 --2- 172.21.15.84:0/2987536857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecf8102200 0x7fecf8198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.045+0000 7fecfdcb2640 1 -- 172.21.15.84:0/2987536857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecf8107850 msgr2=0x7fecf819ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:34.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.045+0000 7fecfdcb2640 1 --2- 172.21.15.84:0/2987536857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecf8107850 0x7fecf819ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.045+0000 7fecfdcb2640 1 -- 172.21.15.84:0/2987536857 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf819d1b0 con 0x7fecf8101840 2024-05-22T23:11:34.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.045+0000 7fecfd4b1640 1 --2- 172.21.15.84:0/2987536857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecf8102200 0x7fecf8198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:11:34.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.045+0000 7fecfe4b3640 1 --2- 172.21.15.84:0/2987536857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecf8107850 0x7fecf819ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:11:34.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.046+0000 7fecfdcb2640 1 --2- 172.21.15.84:0/2987536857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf8101840 0x7fecf81981c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fece800d770 tx=0x7fece800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:34.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.046+0000 7feceeffd640 1 -- 172.21.15.84:0/2987536857 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece8017070 con 0x7fecf8101840 2024-05-22T23:11:34.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.046+0000 7feceeffd640 1 -- 172.21.15.84:0/2987536857 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fece8005500 con 0x7fecf8101840 2024-05-22T23:11:34.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.046+0000 7fecfff16640 1 -- 172.21.15.84:0/2987536857 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf819d4a0 con 0x7fecf8101840 2024-05-22T23:11:34.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.046+0000 7feceeffd640 1 -- 172.21.15.84:0/2987536857 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece8004e00 con 0x7fecf8101840 2024-05-22T23:11:34.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.046+0000 7fecfff16640 1 -- 172.21.15.84:0/2987536857 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf81a4d30 con 0x7fecf8101840 2024-05-22T23:11:34.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.048+0000 7feceeffd640 1 -- 172.21.15.84:0/2987536857 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fece8004fa0 con 0x7fecf8101840 2024-05-22T23:11:34.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.048+0000 7fed02c1b740 1 Processor -- start 2024-05-22T23:11:34.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.049+0000 7fed02c1b740 1 -- start start 2024-05-22T23:11:34.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.049+0000 7fed02c1b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a124a0 0x560077a107e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.049+0000 7fed02c1b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560077a10db0 0x560077a11230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.049+0000 7fed02c1b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560077a1bb30 0x560077a1df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.049+0000 7fed02c1b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560077974db0 con 0x560077a124a0 2024-05-22T23:11:34.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.049+0000 7fed02c1b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560077913360 con 0x560077a10db0 2024-05-22T23:11:34.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.049+0000 7fed02c1b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560077970d70 con 0x560077a1bb30 2024-05-22T23:11:34.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.049+0000 7fecfe4b3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a124a0 0x560077a107e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.049+0000 7fecfe4b3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a124a0 0x560077a107e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48452/0 (socket says 172.21.15.84:48452) 2024-05-22T23:11:34.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.049+0000 7fecfe4b3640 1 -- 172.21.15.84:0/1401092027 learned_addr learned my addr 172.21.15.84:0/1401092027 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:34.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.049+0000 7feceeffd640 1 --2- 172.21.15.84:0/2987536857 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fece40a2d00 0x7fece40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.049+0000 7fecfdcb2640 1 --2- 172.21.15.84:0/1401092027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560077a10db0 0x560077a11230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.049+0000 7fecfe4b3640 1 --2- 172.21.15.84:0/2987536857 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fece40a2d00 0x7fece40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.049+0000 7fecfd4b1640 1 --2- 172.21.15.84:0/1401092027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560077a1bb30 0x560077a1df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.049+0000 7fecfe4b3640 1 -- 172.21.15.84:0/1401092027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560077a1bb30 msgr2=0x560077a1df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:34.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.049+0000 7fecfe4b3640 1 --2- 172.21.15.84:0/1401092027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560077a1bb30 0x560077a1df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.049+0000 7feceeffd640 1 -- 172.21.15.84:0/2987536857 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8571+0+0 (secure 0 0 0) 0x7fece80cef00 con 0x7fecf8101840 2024-05-22T23:11:34.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.049+0000 7fecfe4b3640 1 -- 172.21.15.84:0/1401092027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560077a10db0 msgr2=0x560077a11230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:34.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.049+0000 7fecfe4b3640 1 --2- 172.21.15.84:0/1401092027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560077a10db0 0x560077a11230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.049+0000 7fecfe4b3640 1 -- 172.21.15.84:0/1401092027 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560077936a60 con 0x560077a124a0 2024-05-22T23:11:34.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.049+0000 7fecfd4b1640 1 --2- 172.21.15.84:0/1401092027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560077a1bb30 0x560077a1df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:11:34.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.050+0000 7fecfe4b3640 1 --2- 172.21.15.84:0/1401092027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a124a0 0x560077a107e0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fecf0012a00 tx=0x7fecf0012ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce57e928a1c4f9c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:34.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.050+0000 7fece3fff640 1 -- 172.21.15.84:0/1401092027 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf0022070 con 0x560077a124a0 2024-05-22T23:11:34.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.050+0000 7fece3fff640 1 -- 172.21.15.84:0/1401092027 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecf0004b60 con 0x560077a124a0 2024-05-22T23:11:34.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.050+0000 7fece3fff640 1 -- 172.21.15.84:0/1401092027 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf0005050 con 0x560077a124a0 2024-05-22T23:11:34.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.050+0000 7fed02c1b740 1 -- 172.21.15.84:0/1401092027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a124a0 msgr2=0x560077a107e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:34.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.050+0000 7fecfe4b3640 1 --2- 172.21.15.84:0/2987536857 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fece40a2d00 0x7fece40a51c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fecf001f000 tx=0x7fecf001b000 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:34.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.050+0000 7fed02c1b740 1 --2- 172.21.15.84:0/1401092027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a124a0 0x560077a107e0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fecf0012a00 tx=0x7fecf0012ec0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.050+0000 7fed02c1b740 1 -- 172.21.15.84:0/1401092027 shutdown_connections 2024-05-22T23:11:34.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.050+0000 7fed02c1b740 1 --2- 172.21.15.84:0/1401092027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560077a1bb30 0x560077a1df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.050+0000 7fed02c1b740 1 --2- 172.21.15.84:0/1401092027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560077a10db0 0x560077a11230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.050+0000 7fed02c1b740 1 --2- 172.21.15.84:0/1401092027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a124a0 0x560077a107e0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.050+0000 7fed02c1b740 1 -- 172.21.15.84:0/1401092027 >> 172.21.15.84:0/1401092027 conn(0x5600779d98e0 msgr2=0x560077a0d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:34.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.051+0000 7fed02c1b740 1 -- 172.21.15.84:0/1401092027 shutdown_connections 2024-05-22T23:11:34.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.051+0000 7fed02c1b740 1 -- 172.21.15.84:0/1401092027 wait complete. 2024-05-22T23:11:34.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.051+0000 7fed02c1b740 1 -- start start 2024-05-22T23:11:34.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.051+0000 7fed02c1b740 1 Processor -- start 2024-05-22T23:11:34.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.051+0000 7fed02c1b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560077a10db0 0x560077a18a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.051+0000 7fed02c1b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a124a0 0x560077a18f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.051+0000 7fed02c1b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560077a1bb30 0x560077a19490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.051+0000 7fed02c1b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560077a1e9c0 con 0x560077a124a0 2024-05-22T23:11:34.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.051+0000 7fed02c1b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56007792f0f0 con 0x560077a1bb30 2024-05-22T23:11:34.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.051+0000 7fed02c1b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560077a1eb40 con 0x560077a10db0 2024-05-22T23:11:34.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.051+0000 7fed02c1b740 10 client.0 _ll_register_callbacks cb 0x560077933f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:11:34.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.052+0000 7fecfd4b1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a124a0 0x560077a18f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.052+0000 7fecfd4b1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a124a0 0x560077a18f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48458/0 (socket says 172.21.15.84:48458) 2024-05-22T23:11:34.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.052+0000 7fecfd4b1640 1 -- 172.21.15.84:0/2522366981 learned_addr learned my addr 172.21.15.84:0/2522366981 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:34.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.052+0000 7fecfe4b3640 1 --2- 172.21.15.84:0/2522366981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560077a1bb30 0x560077a19490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.052+0000 7fecfdcb2640 1 --2- 172.21.15.84:0/2522366981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560077a10db0 0x560077a18a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.052+0000 7fecfd4b1640 1 -- 172.21.15.84:0/2522366981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560077a10db0 msgr2=0x560077a18a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:34.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.052+0000 7fecfd4b1640 1 --2- 172.21.15.84:0/2522366981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560077a10db0 0x560077a18a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.052+0000 7fecfd4b1640 1 -- 172.21.15.84:0/2522366981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560077a1bb30 msgr2=0x560077a19490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:34.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.052+0000 7fecfd4b1640 1 --2- 172.21.15.84:0/2522366981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560077a1bb30 0x560077a19490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.052+0000 7fecfd4b1640 1 -- 172.21.15.84:0/2522366981 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600777527d0 con 0x560077a124a0 2024-05-22T23:11:34.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.052+0000 7fecfe4b3640 1 --2- 172.21.15.84:0/2522366981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560077a1bb30 0x560077a19490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:11:34.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.052+0000 7fecfd4b1640 1 --2- 172.21.15.84:0/2522366981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a124a0 0x560077a18f50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fecf4005c20 tx=0x7fecf4031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65619c11c115c1cb server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:34.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.052+0000 7fece17fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:11:34.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.052+0000 7fece17fa640 1 -- 172.21.15.84:0/2522366981 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf403f070 con 0x560077a124a0 2024-05-22T23:11:34.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.052+0000 7fece17fa640 1 -- 172.21.15.84:0/2522366981 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8571+0+0 (secure 0 0 0) 0x7fecf4004240 con 0x560077a124a0 2024-05-22T23:11:34.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.052+0000 7fed02c1b740 1 -- 172.21.15.84:0/2522366981 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecf0004b60 con 0x560077a124a0 2024-05-22T23:11:34.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.052+0000 7fece17fa640 1 -- 172.21.15.84:0/2522366981 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fecf404b050 con 0x560077a124a0 2024-05-22T23:11:34.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.052+0000 7fed02c1b740 10 client.16944 fetch_fsmap learned FSMap version 18 2024-05-22T23:11:34.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.052+0000 7fed02c1b740 1 -- 172.21.15.84:0/2522366981 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5600776883f0 con 0x560077a124a0 2024-05-22T23:11:34.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.053+0000 7fece17fa640 1 -- 172.21.15.84:0/2522366981 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecf0004b60 con 0x560077a124a0 2024-05-22T23:11:34.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.053+0000 7fece17fa640 1 -- 172.21.15.84:0/2522366981 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fecf40049c0 con 0x560077a124a0 2024-05-22T23:11:34.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.053+0000 7fed02c1b740 10 client.16944 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:11:34.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.053+0000 7fed02c1b740 10 client.16944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:11:34.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.053+0000 7fed02c1b740 20 client.16944 populate_metadata read hostname 'smithi084' 2024-05-22T23:11:34.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.053+0000 7fed02c1b740 1 --2- 172.21.15.84:0/2522366981 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x560077a2be60 0x560077a2e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.053+0000 7fed02c1b740 4 client.16944 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:11:34.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.053+0000 7fed02c1b740 1 -- 172.21.15.84:0/2522366981 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600779824e0 con 0x560077a2be60 2024-05-22T23:11:34.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.053+0000 7fecfdcb2640 1 --2- 172.21.15.84:0/2522366981 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x560077a2be60 0x560077a2e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.053+0000 7fece17fa640 10 client.16944 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:34.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.053+0000 7fecfdcb2640 1 --2- 172.21.15.84:0/2522366981 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x560077a2be60 0x560077a2e2a0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bf2c942912c3729 server_cookie=a7cc73a7a04ade3a in_seq=0 out_seq=0 2024-05-22T23:11:34.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.054+0000 7fece17fa640 1 -- 172.21.15.84:0/2522366981 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5600779824e0 con 0x560077a2be60 2024-05-22T23:11:34.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.054+0000 7fece17fa640 10 client.16944 handle_command_reply: tid=0 2024-05-22T23:11:34.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.054+0000 7fed02c1b740 1 client.16944 shutdown 2024-05-22T23:11:34.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.054+0000 7fed02c1b740 1 -- 172.21.15.84:0/2522366981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a124a0 msgr2=0x560077a18f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.054+0000 7fed02c1b740 1 --2- 172.21.15.84:0/2522366981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a124a0 0x560077a18f50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fecf4005c20 tx=0x7fecf4031630 comp rx=0 tx=0).stop 2024-05-22T23:11:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.054+0000 7fed02c1b740 1 -- 172.21.15.84:0/2522366981 shutdown_connections 2024-05-22T23:11:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.054+0000 7fed02c1b740 1 --2- 172.21.15.84:0/2522366981 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x560077a2be60 0x560077a2e2a0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.054+0000 7fed02c1b740 1 --2- 172.21.15.84:0/2522366981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560077a1bb30 0x560077a19490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.054+0000 7fed02c1b740 1 --2- 172.21.15.84:0/2522366981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a124a0 0x560077a18f50 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.054+0000 7fed02c1b740 1 --2- 172.21.15.84:0/2522366981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560077a10db0 0x560077a18a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.054+0000 7fece17fa640 0 client.16944 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.054+0000 7fed02c1b740 1 -- 172.21.15.84:0/2522366981 >> 172.21.15.84:0/2522366981 conn(0x5600779d98e0 msgr2=0x560077a13180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.055+0000 7fed02c1b740 1 -- 172.21.15.84:0/2522366981 shutdown_connections 2024-05-22T23:11:34.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.055+0000 7fed02c1b740 1 -- 172.21.15.84:0/2522366981 wait complete. 2024-05-22T23:11:34.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.055+0000 7fed02c1b740 20 client.16944 trim_cache size 0 max 16384 2024-05-22T23:11:34.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.072+0000 7fed02c1b740 1 Processor -- start 2024-05-22T23:11:34.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.072+0000 7fed02c1b740 1 -- start start 2024-05-22T23:11:34.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.072+0000 7fed02c1b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560077a10db0 0x560077a278c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.072+0000 7fed02c1b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560077a124a0 0x560077a33b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.072+0000 7fed02c1b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a1bb30 0x560077a340b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.072+0000 7fed02c1b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fecf81ab7a0 con 0x560077a1bb30 2024-05-22T23:11:34.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.072+0000 7fed02c1b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fecf81ae000 con 0x560077a10db0 2024-05-22T23:11:34.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.072+0000 7fed02c1b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fecf81abdf0 con 0x560077a124a0 2024-05-22T23:11:34.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.072+0000 7fecfe4b3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a1bb30 0x560077a340b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.072+0000 7fecfe4b3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a1bb30 0x560077a340b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48462/0 (socket says 172.21.15.84:48462) 2024-05-22T23:11:34.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.072+0000 7fecfe4b3640 1 -- 172.21.15.84:0/3852894361 learned_addr learned my addr 172.21.15.84:0/3852894361 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:34.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.072+0000 7fecfe4b3640 1 -- 172.21.15.84:0/3852894361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560077a124a0 msgr2=0x560077a33b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:34.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.072+0000 7fecfe4b3640 1 --2- 172.21.15.84:0/3852894361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560077a124a0 0x560077a33b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.072+0000 7fecfdcb2640 1 --2- 172.21.15.84:0/3852894361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560077a10db0 0x560077a278c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.072+0000 7fecfe4b3640 1 -- 172.21.15.84:0/3852894361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560077a10db0 msgr2=0x560077a278c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:34.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.072+0000 7fecfe4b3640 1 --2- 172.21.15.84:0/3852894361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560077a10db0 0x560077a278c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.073+0000 7fecfe4b3640 1 -- 172.21.15.84:0/3852894361 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600776480f0 con 0x560077a1bb30 2024-05-22T23:11:34.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.073+0000 7fecfe4b3640 1 --2- 172.21.15.84:0/3852894361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a1bb30 0x560077a340b0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fecf0015960 tx=0x7fecf00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27def4c31db33590 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:34.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.073+0000 7fece37fe640 1 -- 172.21.15.84:0/3852894361 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf0022040 con 0x560077a1bb30 2024-05-22T23:11:34.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.073+0000 7fece37fe640 1 -- 172.21.15.84:0/3852894361 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecf0005050 con 0x560077a1bb30 2024-05-22T23:11:34.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.073+0000 7fece37fe640 1 -- 172.21.15.84:0/3852894361 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf000f680 con 0x560077a1bb30 2024-05-22T23:11:34.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.073+0000 7fed02c1b740 1 -- 172.21.15.84:0/3852894361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a1bb30 msgr2=0x560077a340b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:34.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.073+0000 7fed02c1b740 1 --2- 172.21.15.84:0/3852894361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a1bb30 0x560077a340b0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fecf0015960 tx=0x7fecf00095c0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.073+0000 7fed02c1b740 1 -- 172.21.15.84:0/3852894361 shutdown_connections 2024-05-22T23:11:34.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.073+0000 7fed02c1b740 1 --2- 172.21.15.84:0/3852894361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a1bb30 0x560077a340b0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.073+0000 7fed02c1b740 1 --2- 172.21.15.84:0/3852894361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560077a124a0 0x560077a33b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.073+0000 7fed02c1b740 1 --2- 172.21.15.84:0/3852894361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560077a10db0 0x560077a278c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.073+0000 7fed02c1b740 1 -- 172.21.15.84:0/3852894361 >> 172.21.15.84:0/3852894361 conn(0x5600779d98e0 msgr2=0x560077a13160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:34.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.074+0000 7fed02c1b740 1 -- 172.21.15.84:0/3852894361 shutdown_connections 2024-05-22T23:11:34.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.074+0000 7fed02c1b740 1 -- 172.21.15.84:0/3852894361 wait complete. 2024-05-22T23:11:34.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.074+0000 7fed02c1b740 1 -- start start 2024-05-22T23:11:34.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.074+0000 7fed02c1b740 1 Processor -- start 2024-05-22T23:11:34.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.074+0000 7fed02c1b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560077a10db0 0x560077a15590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.074+0000 7fed02c1b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560077a124a0 0x560077a15ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.074+0000 7fed02c1b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a1bb30 0x560077a16010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.074+0000 7fed02c1b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fecf81ac4c0 con 0x560077a1bb30 2024-05-22T23:11:34.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.074+0000 7fed02c1b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fecf81ad930 con 0x560077a124a0 2024-05-22T23:11:34.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.074+0000 7fed02c1b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fecf81acb90 con 0x560077a10db0 2024-05-22T23:11:34.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.075+0000 7fecfe4b3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a1bb30 0x560077a16010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.075+0000 7fed02c1b740 10 client.0 _ll_register_callbacks cb 0x7fecf4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:11:34.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.075+0000 7fecfe4b3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a1bb30 0x560077a16010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48470/0 (socket says 172.21.15.84:48470) 2024-05-22T23:11:34.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.075+0000 7fecfe4b3640 1 -- 172.21.15.84:0/3683306229 learned_addr learned my addr 172.21.15.84:0/3683306229 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:34.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.075+0000 7fecfdcb2640 1 --2- 172.21.15.84:0/3683306229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560077a10db0 0x560077a15590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.075+0000 7fecfd4b1640 1 --2- 172.21.15.84:0/3683306229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560077a124a0 0x560077a15ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.075+0000 7fecfe4b3640 1 -- 172.21.15.84:0/3683306229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560077a10db0 msgr2=0x560077a15590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:34.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.075+0000 7fecfe4b3640 1 --2- 172.21.15.84:0/3683306229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560077a10db0 0x560077a15590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.075+0000 7fecfe4b3640 1 -- 172.21.15.84:0/3683306229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560077a124a0 msgr2=0x560077a15ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:34.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.075+0000 7fecfe4b3640 1 --2- 172.21.15.84:0/3683306229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560077a124a0 0x560077a15ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.075+0000 7fecfe4b3640 1 -- 172.21.15.84:0/3683306229 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560077933fe0 con 0x560077a1bb30 2024-05-22T23:11:34.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.075+0000 7fecfd4b1640 1 --2- 172.21.15.84:0/3683306229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560077a124a0 0x560077a15ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:11:34.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.075+0000 7fecfe4b3640 1 --2- 172.21.15.84:0/3683306229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a1bb30 0x560077a16010 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fecf0004e10 tx=0x7fecf0004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f1093f036ab314 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:34.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.075+0000 7fece17fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:11:34.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.075+0000 7fece17fa640 1 -- 172.21.15.84:0/3683306229 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf003f060 con 0x560077a1bb30 2024-05-22T23:11:34.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.075+0000 7fece17fa640 1 -- 172.21.15.84:0/3683306229 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8571+0+0 (secure 0 0 0) 0x7fecf0031440 con 0x560077a1bb30 2024-05-22T23:11:34.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.075+0000 7fed02c1b740 1 -- 172.21.15.84:0/3683306229 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecf0005050 con 0x560077a1bb30 2024-05-22T23:11:34.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.075+0000 7fece17fa640 1 -- 172.21.15.84:0/3683306229 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fecf0022070 con 0x560077a1bb30 2024-05-22T23:11:34.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.075+0000 7fed02c1b740 10 client.16950 fetch_fsmap learned FSMap version 18 2024-05-22T23:11:34.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.075+0000 7fed02c1b740 1 -- 172.21.15.84:0/3683306229 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5600779051c0 con 0x560077a1bb30 2024-05-22T23:11:34.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.076+0000 7fece17fa640 1 -- 172.21.15.84:0/3683306229 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecf0005050 con 0x560077a1bb30 2024-05-22T23:11:34.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.076+0000 7fece17fa640 1 -- 172.21.15.84:0/3683306229 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fecf0039030 con 0x560077a1bb30 2024-05-22T23:11:34.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.076+0000 7fed02c1b740 10 client.16950 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:11:34.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.076+0000 7fed02c1b740 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:11:34.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.076+0000 7fed02c1b740 20 client.16950 populate_metadata read hostname 'smithi084' 2024-05-22T23:11:34.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.076+0000 7fed02c1b740 1 --2- 172.21.15.84:0/3683306229 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x560077a96b70 0x560077a98f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.076+0000 7fed02c1b740 4 client.16950 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:11:34.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.076+0000 7fed02c1b740 1 -- 172.21.15.84:0/3683306229 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56007797c830 con 0x560077a96b70 2024-05-22T23:11:34.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.076+0000 7fecfdcb2640 1 --2- 172.21.15.84:0/3683306229 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x560077a96b70 0x560077a98f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.076+0000 7fecfdcb2640 1 --2- 172.21.15.84:0/3683306229 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x560077a96b70 0x560077a98f60 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7d79bb65907337 server_cookie=1867e604b8bd6722 in_seq=0 out_seq=0 2024-05-22T23:11:34.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.076+0000 7fece17fa640 10 client.16950 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:34.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.076+0000 7fece17fa640 1 -- 172.21.15.84:0/3683306229 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56007797c830 con 0x560077a96b70 2024-05-22T23:11:34.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.076+0000 7fece17fa640 10 client.16950 handle_command_reply: tid=0 2024-05-22T23:11:34.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.076+0000 7fed02c1b740 1 client.16950 shutdown 2024-05-22T23:11:34.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.077+0000 7fed02c1b740 1 -- 172.21.15.84:0/3683306229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a1bb30 msgr2=0x560077a16010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:34.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.077+0000 7fed02c1b740 1 --2- 172.21.15.84:0/3683306229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a1bb30 0x560077a16010 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fecf0004e10 tx=0x7fecf0004e40 comp rx=0 tx=0).stop 2024-05-22T23:11:34.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.077+0000 7fed02c1b740 1 -- 172.21.15.84:0/3683306229 shutdown_connections 2024-05-22T23:11:34.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.077+0000 7fed02c1b740 1 --2- 172.21.15.84:0/3683306229 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x560077a96b70 0x560077a98f60 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.077+0000 7fed02c1b740 1 --2- 172.21.15.84:0/3683306229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560077a1bb30 0x560077a16010 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.077+0000 7fed02c1b740 1 --2- 172.21.15.84:0/3683306229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560077a124a0 0x560077a15ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.077+0000 7fed02c1b740 1 --2- 172.21.15.84:0/3683306229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560077a10db0 0x560077a15590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.077+0000 7fece17fa640 0 client.16950 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:34.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.077+0000 7fed02c1b740 1 -- 172.21.15.84:0/3683306229 >> 172.21.15.84:0/3683306229 conn(0x5600779d98e0 msgr2=0x560077a1a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:34.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.077+0000 7fed02c1b740 1 -- 172.21.15.84:0/3683306229 shutdown_connections 2024-05-22T23:11:34.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.077+0000 7fed02c1b740 1 -- 172.21.15.84:0/3683306229 wait complete. 2024-05-22T23:11:34.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.077+0000 7fed02c1b740 20 client.16950 trim_cache size 0 max 16384 2024-05-22T23:11:34.094 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:11:34.095 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "no active scrubs running", 2024-05-22T23:11:34.095 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": {} 2024-05-22T23:11:34.095 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:11:34.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.077+0000 7fecfff16640 1 -- 172.21.15.84:0/2987536857 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fece40a2d00 msgr2=0x7fece40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:34.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.077+0000 7fecfff16640 1 --2- 172.21.15.84:0/2987536857 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fece40a2d00 0x7fece40a51c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fecf001f000 tx=0x7fecf001b000 comp rx=0 tx=0).stop 2024-05-22T23:11:34.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.077+0000 7fecfff16640 1 -- 172.21.15.84:0/2987536857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf8101840 msgr2=0x7fecf81981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:34.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.077+0000 7fecfff16640 1 --2- 172.21.15.84:0/2987536857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf8101840 0x7fecf81981c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fece800d770 tx=0x7fece800dc30 comp rx=0 tx=0).stop 2024-05-22T23:11:34.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.077+0000 7fecfff16640 1 -- 172.21.15.84:0/2987536857 shutdown_connections 2024-05-22T23:11:34.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.077+0000 7fecfff16640 1 --2- 172.21.15.84:0/2987536857 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fece40a2d00 0x7fece40a51c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.077+0000 7fecfff16640 1 --2- 172.21.15.84:0/2987536857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecf8107850 0x7fecf819ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.077+0000 7fecfff16640 1 --2- 172.21.15.84:0/2987536857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecf8102200 0x7fecf8198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.077+0000 7fecfff16640 1 --2- 172.21.15.84:0/2987536857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf8101840 0x7fecf81981c0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.077+0000 7fecfff16640 1 -- 172.21.15.84:0/2987536857 >> 172.21.15.84:0/2987536857 conn(0x7fecf80fd2b0 msgr2=0x7fecf80fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:34.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.077+0000 7fecfff16640 1 -- 172.21.15.84:0/2987536857 shutdown_connections 2024-05-22T23:11:34.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.077+0000 7fecfff16640 1 -- 172.21.15.84:0/2987536857 wait complete. 2024-05-22T23:11:34.098 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-22T23:11:34.098 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:11:34.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.192+0000 7f757567b640 1 -- 172.21.15.84:0/4060806454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75701007c0 msgr2=0x7f7570100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:34.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.192+0000 7f757567b640 1 --2- 172.21.15.84:0/4060806454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75701007c0 0x7f7570100c40 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7558009930 tx=0x7f755802a420 comp rx=0 tx=0).stop 2024-05-22T23:11:34.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.192+0000 7f757567b640 1 -- 172.21.15.84:0/4060806454 shutdown_connections 2024-05-22T23:11:34.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.192+0000 7f757567b640 1 --2- 172.21.15.84:0/4060806454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7570101180 0x7f757010d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.192+0000 7f757567b640 1 --2- 172.21.15.84:0/4060806454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75701007c0 0x7f7570100c40 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.192+0000 7f757567b640 1 --2- 172.21.15.84:0/4060806454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7570106780 0x7f7570106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.192+0000 7f757567b640 1 -- 172.21.15.84:0/4060806454 >> 172.21.15.84:0/4060806454 conn(0x7f75700778d0 msgr2=0x7f7570077ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:34.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.192+0000 7f757567b640 1 -- 172.21.15.84:0/4060806454 shutdown_connections 2024-05-22T23:11:34.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.192+0000 7f757567b640 1 -- 172.21.15.84:0/4060806454 wait complete. 2024-05-22T23:11:34.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.193+0000 7f757567b640 1 Processor -- start 2024-05-22T23:11:34.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.193+0000 7f757567b640 1 -- start start 2024-05-22T23:11:34.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.193+0000 7f757567b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75701007c0 0x7f75701970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.193+0000 7f757567b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7570101180 0x7f7570197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.193+0000 7f757567b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7570106780 0x7f757019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.193+0000 7f757567b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7570110b50 con 0x7f7570101180 2024-05-22T23:11:34.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.193+0000 7f757567b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f75701109d0 con 0x7f75701007c0 2024-05-22T23:11:34.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.193+0000 7f757567b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7570110cd0 con 0x7f7570106780 2024-05-22T23:11:34.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.193+0000 7f7567fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7570101180 0x7f7570197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.193+0000 7f7567fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7570101180 0x7f7570197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48494/0 (socket says 172.21.15.84:48494) 2024-05-22T23:11:34.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.193+0000 7f7567fff640 1 -- 172.21.15.84:0/2198849506 learned_addr learned my addr 172.21.15.84:0/2198849506 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:34.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.193+0000 7f756effd640 1 --2- 172.21.15.84:0/2198849506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75701007c0 0x7f75701970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.193+0000 7f756f7fe640 1 --2- 172.21.15.84:0/2198849506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7570106780 0x7f757019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.193+0000 7f7567fff640 1 -- 172.21.15.84:0/2198849506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7570106780 msgr2=0x7f757019b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.193+0000 7f7567fff640 1 --2- 172.21.15.84:0/2198849506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7570106780 0x7f757019b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.193+0000 7f7567fff640 1 -- 172.21.15.84:0/2198849506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75701007c0 msgr2=0x7f75701970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.193+0000 7f7567fff640 1 --2- 172.21.15.84:0/2198849506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75701007c0 0x7f75701970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.193+0000 7f7567fff640 1 -- 172.21.15.84:0/2198849506 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757019c0d0 con 0x7f7570101180 2024-05-22T23:11:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.193+0000 7f756f7fe640 1 --2- 172.21.15.84:0/2198849506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7570106780 0x7f757019b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:11:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.194+0000 7f7567fff640 1 --2- 172.21.15.84:0/2198849506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7570101180 0x7f7570197620 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f7558009230 tx=0x7f7558004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.194+0000 7f756cff9640 1 -- 172.21.15.84:0/2198849506 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755803f030 con 0x7f7570101180 2024-05-22T23:11:34.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.194+0000 7f756cff9640 1 -- 172.21.15.84:0/2198849506 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7558003ad0 con 0x7f7570101180 2024-05-22T23:11:34.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.194+0000 7f757567b640 1 -- 172.21.15.84:0/2198849506 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f757019c360 con 0x7f7570101180 2024-05-22T23:11:34.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.194+0000 7f756cff9640 1 -- 172.21.15.84:0/2198849506 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558035070 con 0x7f7570101180 2024-05-22T23:11:34.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.194+0000 7f757567b640 1 -- 172.21.15.84:0/2198849506 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75701a3c50 con 0x7f7570101180 2024-05-22T23:11:34.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.195+0000 7f757567b640 1 -- 172.21.15.84:0/2198849506 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7570067e00 con 0x7f7570101180 2024-05-22T23:11:34.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.196+0000 7f756cff9640 1 -- 172.21.15.84:0/2198849506 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7558042030 con 0x7f7570101180 2024-05-22T23:11:34.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.196+0000 7f756cff9640 1 --2- 172.21.15.84:0/2198849506 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f755c0a2d00 0x7f755c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.197+0000 7f756effd640 1 --2- 172.21.15.84:0/2198849506 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f755c0a2d00 0x7f755c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.197+0000 7f756cff9640 1 -- 172.21.15.84:0/2198849506 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8571+0+0 (secure 0 0 0) 0x7f75580ef9d0 con 0x7f7570101180 2024-05-22T23:11:34.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.197+0000 7f756effd640 1 --2- 172.21.15.84:0/2198849506 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f755c0a2d00 0x7f755c0a51c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7560005d00 tx=0x7f756000b410 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:34.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.201+0000 7f756cff9640 1 -- 172.21.15.84:0/2198849506 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f75580bc5c0 con 0x7f7570101180 2024-05-22T23:11:34.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.416+0000 7f757567b640 1 -- 172.21.15.84:0/2198849506 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7570108d00 con 0x7f7570101180 2024-05-22T23:11:34.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.417+0000 7f756cff9640 1 -- 172.21.15.84:0/2198849506 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7f75580bbec0 con 0x7f7570101180 2024-05-22T23:11:34.424 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:11:34.425 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:11:34.425 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:11:34.426 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.419+0000 7f757567b640 1 -- 172.21.15.84:0/2198849506 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f755c0a2d00 msgr2=0x7f755c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:34.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.419+0000 7f757567b640 1 --2- 172.21.15.84:0/2198849506 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f755c0a2d00 0x7f755c0a51c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7560005d00 tx=0x7f756000b410 comp rx=0 tx=0).stop 2024-05-22T23:11:34.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.419+0000 7f757567b640 1 -- 172.21.15.84:0/2198849506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7570101180 msgr2=0x7f7570197620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:34.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.419+0000 7f757567b640 1 --2- 172.21.15.84:0/2198849506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7570101180 0x7f7570197620 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f7558009230 tx=0x7f7558004270 comp rx=0 tx=0).stop 2024-05-22T23:11:34.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.419+0000 7f757567b640 1 -- 172.21.15.84:0/2198849506 shutdown_connections 2024-05-22T23:11:34.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.419+0000 7f757567b640 1 --2- 172.21.15.84:0/2198849506 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f755c0a2d00 0x7f755c0a51c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.419+0000 7f757567b640 1 --2- 172.21.15.84:0/2198849506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7570106780 0x7f757019b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.419+0000 7f757567b640 1 --2- 172.21.15.84:0/2198849506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7570101180 0x7f7570197620 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.427 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.419+0000 7f757567b640 1 --2- 172.21.15.84:0/2198849506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75701007c0 0x7f75701970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.419+0000 7f757567b640 1 -- 172.21.15.84:0/2198849506 >> 172.21.15.84:0/2198849506 conn(0x7f75700778d0 msgr2=0x7f7570103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:34.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.419+0000 7f757567b640 1 -- 172.21.15.84:0/2198849506 shutdown_connections 2024-05-22T23:11:34.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.419+0000 7f757567b640 1 -- 172.21.15.84:0/2198849506 wait complete. 2024-05-22T23:11:34.438 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-22T23:11:34.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.533+0000 7f06d4f9d640 1 -- 172.21.15.84:0/3057628694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06d0106760 msgr2=0x7f06d0106b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:34.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.533+0000 7f06d4f9d640 1 --2- 172.21.15.84:0/3057628694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06d0106760 0x7f06d0106b40 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009930 tx=0x7f06c002a420 comp rx=0 tx=0).stop 2024-05-22T23:11:34.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.533+0000 7f06d4f9d640 1 -- 172.21.15.84:0/3057628694 shutdown_connections 2024-05-22T23:11:34.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.533+0000 7f06d4f9d640 1 --2- 172.21.15.84:0/3057628694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06d0101110 0x7f06d010d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.533+0000 7f06d4f9d640 1 --2- 172.21.15.84:0/3057628694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06d0100750 0x7f06d0100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.533+0000 7f06d4f9d640 1 --2- 172.21.15.84:0/3057628694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06d0106760 0x7f06d0106b40 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.533+0000 7f06d4f9d640 1 -- 172.21.15.84:0/3057628694 >> 172.21.15.84:0/3057628694 conn(0x7f06d0076af0 msgr2=0x7f06d0076f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:34.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.533+0000 7f06d4f9d640 1 -- 172.21.15.84:0/3057628694 shutdown_connections 2024-05-22T23:11:34.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.533+0000 7f06d4f9d640 1 -- 172.21.15.84:0/3057628694 wait complete. 2024-05-22T23:11:34.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.534+0000 7f06d4f9d640 1 Processor -- start 2024-05-22T23:11:34.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.534+0000 7f06d4f9d640 1 -- start start 2024-05-22T23:11:34.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.534+0000 7f06d4f9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06d0100750 0x7f06d0197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.534+0000 7f06d4f9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06d0101110 0x7f06d0197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.534+0000 7f06d4f9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06d0106760 0x7f06d019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.534+0000 7f06d4f9d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f06d0110b90 con 0x7f06d0101110 2024-05-22T23:11:34.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.534+0000 7f06d4f9d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f06d0110a10 con 0x7f06d0100750 2024-05-22T23:11:34.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.534+0000 7f06d4f9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f06d0110d10 con 0x7f06d0106760 2024-05-22T23:11:34.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.534+0000 7f06cdd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06d0101110 0x7f06d0197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.534+0000 7f06cdd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06d0101110 0x7f06d0197670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48514/0 (socket says 172.21.15.84:48514) 2024-05-22T23:11:34.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.534+0000 7f06cdd9b640 1 -- 172.21.15.84:0/2610980838 learned_addr learned my addr 172.21.15.84:0/2610980838 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:34.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.534+0000 7f06ce59c640 1 --2- 172.21.15.84:0/2610980838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06d0100750 0x7f06d0197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.534+0000 7f06ced9d640 1 --2- 172.21.15.84:0/2610980838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06d0106760 0x7f06d019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.534+0000 7f06cdd9b640 1 -- 172.21.15.84:0/2610980838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06d0106760 msgr2=0x7f06d019b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:34.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.534+0000 7f06cdd9b640 1 --2- 172.21.15.84:0/2610980838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06d0106760 0x7f06d019b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.534+0000 7f06cdd9b640 1 -- 172.21.15.84:0/2610980838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06d0100750 msgr2=0x7f06d0197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:34.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.534+0000 7f06cdd9b640 1 --2- 172.21.15.84:0/2610980838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06d0100750 0x7f06d0197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.534+0000 7f06cdd9b640 1 -- 172.21.15.84:0/2610980838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d019c0d0 con 0x7f06d0101110 2024-05-22T23:11:34.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.534+0000 7f06ced9d640 1 --2- 172.21.15.84:0/2610980838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06d0106760 0x7f06d019b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:11:34.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.535+0000 7f06cdd9b640 1 --2- 172.21.15.84:0/2610980838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06d0101110 0x7f06d0197670 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f06b800d990 tx=0x7f06b800de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:34.547 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.535+0000 7f06c77fe640 1 -- 172.21.15.84:0/2610980838 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b801b070 con 0x7f06d0101110 2024-05-22T23:11:34.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.535+0000 7f06d4f9d640 1 -- 172.21.15.84:0/2610980838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d019c3c0 con 0x7f06d0101110 2024-05-22T23:11:34.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.535+0000 7f06c77fe640 1 -- 172.21.15.84:0/2610980838 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06b800edf0 con 0x7f06d0101110 2024-05-22T23:11:34.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.535+0000 7f06c77fe640 1 -- 172.21.15.84:0/2610980838 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8014870 con 0x7f06d0101110 2024-05-22T23:11:34.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.535+0000 7f06d4f9d640 1 -- 172.21.15.84:0/2610980838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d0069eb0 con 0x7f06d0101110 2024-05-22T23:11:34.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.538+0000 7f06c77fe640 1 -- 172.21.15.84:0/2610980838 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f06b801e070 con 0x7f06d0101110 2024-05-22T23:11:34.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.538+0000 7f06d7d81740 1 Processor -- start 2024-05-22T23:11:34.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.538+0000 7f06d7d81740 1 -- start start 2024-05-22T23:11:34.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.538+0000 7f06d7d81740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce4ed7e80 0x558ce4ed8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.538+0000 7f06d7d81740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ed87a0 0x558ce4ee6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.538+0000 7f06d7d81740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ce4ee7380 0x558ce4ee9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.538+0000 7f06d7d81740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558ce4e3cdb0 con 0x558ce4ed87a0 2024-05-22T23:11:34.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.538+0000 7f06d7d81740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558ce4ddb400 con 0x558ce4ee7380 2024-05-22T23:11:34.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.538+0000 7f06d7d81740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558ce4e38d70 con 0x558ce4ed7e80 2024-05-22T23:11:34.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.538+0000 7f06ce59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ed87a0 0x558ce4ee6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.538+0000 7f06ce59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ed87a0 0x558ce4ee6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48518/0 (socket says 172.21.15.84:48518) 2024-05-22T23:11:34.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.538+0000 7f06ce59c640 1 -- 172.21.15.84:0/670937063 learned_addr learned my addr 172.21.15.84:0/670937063 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:34.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.538+0000 7f06ced9d640 1 --2- 172.21.15.84:0/670937063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce4ed7e80 0x558ce4ed8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.538+0000 7f06cdd9b640 1 --2- 172.21.15.84:0/670937063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ce4ee7380 0x558ce4ee9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.539+0000 7f06c77fe640 1 --2- 172.21.15.84:0/2610980838 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f06bc0a2d00 0x7f06bc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.539+0000 7f06ce59c640 1 -- 172.21.15.84:0/670937063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce4ed7e80 msgr2=0x558ce4ed8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:34.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.539+0000 7f06ce59c640 1 --2- 172.21.15.84:0/670937063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce4ed7e80 0x558ce4ed8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.539+0000 7f06ce59c640 1 -- 172.21.15.84:0/670937063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ce4ee7380 msgr2=0x558ce4ee9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:34.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.539+0000 7f06ce59c640 1 --2- 172.21.15.84:0/670937063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ce4ee7380 0x558ce4ee9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.539+0000 7f06ce59c640 1 -- 172.21.15.84:0/670937063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ce4dcd2f0 con 0x558ce4ed87a0 2024-05-22T23:11:34.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.539+0000 7f06cdd9b640 1 --2- 172.21.15.84:0/670937063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ce4ee7380 0x558ce4ee9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:11:34.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.539+0000 7f06ced9d640 1 --2- 172.21.15.84:0/2610980838 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f06bc0a2d00 0x7f06bc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.539+0000 7f06c77fe640 1 -- 172.21.15.84:0/2610980838 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8571+0+0 (secure 0 0 0) 0x7f06b80d0a40 con 0x7f06d0101110 2024-05-22T23:11:34.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.539+0000 7f06ce59c640 1 --2- 172.21.15.84:0/670937063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ed87a0 0x558ce4ee6e40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009930 tx=0x7f06c0031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46de31022f83d03 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:34.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.539+0000 7f06c4ff9640 1 -- 172.21.15.84:0/670937063 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0040070 con 0x558ce4ed87a0 2024-05-22T23:11:34.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.539+0000 7f06c4ff9640 1 -- 172.21.15.84:0/670937063 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c0005020 con 0x558ce4ed87a0 2024-05-22T23:11:34.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.539+0000 7f06c4ff9640 1 -- 172.21.15.84:0/670937063 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0035040 con 0x558ce4ed87a0 2024-05-22T23:11:34.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.540+0000 7f06d7d81740 1 -- 172.21.15.84:0/670937063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ed87a0 msgr2=0x558ce4ee6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:34.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.540+0000 7f06ced9d640 1 --2- 172.21.15.84:0/2610980838 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f06bc0a2d00 0x7f06bc0a51c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f06c8005fd0 tx=0x7f06c8004a40 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:34.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.540+0000 7f06d7d81740 1 --2- 172.21.15.84:0/670937063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ed87a0 0x558ce4ee6e40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009930 tx=0x7f06c0031630 comp rx=0 tx=0).stop 2024-05-22T23:11:34.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.540+0000 7f06d7d81740 1 -- 172.21.15.84:0/670937063 shutdown_connections 2024-05-22T23:11:34.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.540+0000 7f06d7d81740 1 --2- 172.21.15.84:0/670937063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ce4ee7380 0x558ce4ee9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.540+0000 7f06d7d81740 1 --2- 172.21.15.84:0/670937063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ed87a0 0x558ce4ee6e40 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.540+0000 7f06d7d81740 1 --2- 172.21.15.84:0/670937063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce4ed7e80 0x558ce4ed8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.540+0000 7f06d7d81740 1 -- 172.21.15.84:0/670937063 >> 172.21.15.84:0/670937063 conn(0x558ce4ed42a0 msgr2=0x558ce4ed4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:34.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.540+0000 7f06d7d81740 1 -- 172.21.15.84:0/670937063 shutdown_connections 2024-05-22T23:11:34.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.540+0000 7f06d7d81740 1 -- 172.21.15.84:0/670937063 wait complete. 2024-05-22T23:11:34.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.540+0000 7f06d7d81740 1 -- start start 2024-05-22T23:11:34.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.540+0000 7f06d7d81740 1 Processor -- start 2024-05-22T23:11:34.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.540+0000 7f06d7d81740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce4ed7e80 0x558ce4ee4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.540+0000 7f06d7d81740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ce4ed87a0 0x558ce4ee47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.540+0000 7f06d7d81740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ee7380 0x558ce4ee4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.540+0000 7f06d7d81740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558ce4eea0e0 con 0x558ce4ee7380 2024-05-22T23:11:34.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.540+0000 7f06d7d81740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558ce4df72e0 con 0x558ce4ed87a0 2024-05-22T23:11:34.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.540+0000 7f06d7d81740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558ce4eea260 con 0x558ce4ed7e80 2024-05-22T23:11:34.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.540+0000 7f06d7d81740 10 client.0 _ll_register_callbacks cb 0x558ce4e00a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:11:34.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.541+0000 7f06ced9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ee7380 0x558ce4ee4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.541+0000 7f06ced9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ee7380 0x558ce4ee4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48530/0 (socket says 172.21.15.84:48530) 2024-05-22T23:11:34.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.541+0000 7f06ced9d640 1 -- 172.21.15.84:0/2420369501 learned_addr learned my addr 172.21.15.84:0/2420369501 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:34.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.541+0000 7f06ce59c640 1 --2- 172.21.15.84:0/2420369501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce4ed7e80 0x558ce4ee4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.541+0000 7f06ced9d640 1 -- 172.21.15.84:0/2420369501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce4ed7e80 msgr2=0x558ce4ee4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:34.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.541+0000 7f06ced9d640 1 --2- 172.21.15.84:0/2420369501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce4ed7e80 0x558ce4ee4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.541+0000 7f06ced9d640 1 -- 172.21.15.84:0/2420369501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ce4ed87a0 msgr2=0x558ce4ee47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:34.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.541+0000 7f06cdd9b640 1 --2- 172.21.15.84:0/2420369501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ce4ed87a0 0x558ce4ee47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.541+0000 7f06ced9d640 1 --2- 172.21.15.84:0/2420369501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ce4ed87a0 0x558ce4ee47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.541+0000 7f06ced9d640 1 -- 172.21.15.84:0/2420369501 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ce4b503f0 con 0x558ce4ee7380 2024-05-22T23:11:34.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.541+0000 7f06ce59c640 1 --2- 172.21.15.84:0/2420369501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce4ed7e80 0x558ce4ee4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:11:34.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.541+0000 7f06ced9d640 1 --2- 172.21.15.84:0/2420369501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ee7380 0x558ce4ee4cf0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f06c80138b0 tx=0x7f06c8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353e0f3c25bcd541 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:34.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.541+0000 7f06b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:11:34.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.541+0000 7f06b5ffb640 1 -- 172.21.15.84:0/2420369501 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c801ecf0 con 0x558ce4ee7380 2024-05-22T23:11:34.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.541+0000 7f06b5ffb640 1 -- 172.21.15.84:0/2420369501 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8571+0+0 (secure 0 0 0) 0x7f06c801f450 con 0x558ce4ee7380 2024-05-22T23:11:34.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.541+0000 7f06d7d81740 1 -- 172.21.15.84:0/2420369501 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06c0005020 con 0x558ce4ee7380 2024-05-22T23:11:34.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.541+0000 7f06b5ffb640 1 -- 172.21.15.84:0/2420369501 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f06c801f650 con 0x558ce4ee7380 2024-05-22T23:11:34.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.541+0000 7f06d7d81740 10 client.16968 fetch_fsmap learned FSMap version 18 2024-05-22T23:11:34.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.541+0000 7f06d7d81740 1 -- 172.21.15.84:0/2420369501 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x558ce4c1a7d0 con 0x558ce4ee7380 2024-05-22T23:11:34.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.542+0000 7f06b5ffb640 1 -- 172.21.15.84:0/2420369501 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c0005020 con 0x558ce4ee7380 2024-05-22T23:11:34.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.542+0000 7f06b5ffb640 1 -- 172.21.15.84:0/2420369501 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f06c800f040 con 0x558ce4ee7380 2024-05-22T23:11:34.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.542+0000 7f06d7d81740 10 client.16968 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:11:34.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.542+0000 7f06d7d81740 10 client.16968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:11:34.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.542+0000 7f06d7d81740 20 client.16968 populate_metadata read hostname 'smithi084' 2024-05-22T23:11:34.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.542+0000 7f06d7d81740 1 --2- 172.21.15.84:0/2420369501 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558ce4ef7480 0x558ce4ef98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.542+0000 7f06d7d81740 4 client.16968 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:11:34.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.542+0000 7f06d7d81740 1 -- 172.21.15.84:0/2420369501 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ce4e4a4e0 con 0x558ce4ef7480 2024-05-22T23:11:34.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.542+0000 7f06ce59c640 1 --2- 172.21.15.84:0/2420369501 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558ce4ef7480 0x558ce4ef98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.542+0000 7f06b5ffb640 10 client.16968 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:34.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.542+0000 7f06ce59c640 1 --2- 172.21.15.84:0/2420369501 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558ce4ef7480 0x558ce4ef98d0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba3dd70efaefa390 server_cookie=cefd1ea6dbddb615 in_seq=0 out_seq=0 2024-05-22T23:11:34.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.543+0000 7f06b5ffb640 1 -- 172.21.15.84:0/2420369501 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ce4e4a4e0 con 0x558ce4ef7480 2024-05-22T23:11:34.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.543+0000 7f06b5ffb640 10 client.16968 handle_command_reply: tid=0 2024-05-22T23:11:34.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.543+0000 7f06d7d81740 1 client.16968 shutdown 2024-05-22T23:11:34.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.543+0000 7f06d7d81740 1 -- 172.21.15.84:0/2420369501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ee7380 msgr2=0x558ce4ee4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:34.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.543+0000 7f06d7d81740 1 --2- 172.21.15.84:0/2420369501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ee7380 0x558ce4ee4cf0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f06c80138b0 tx=0x7f06c8004ab0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.543+0000 7f06d7d81740 1 -- 172.21.15.84:0/2420369501 shutdown_connections 2024-05-22T23:11:34.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.543+0000 7f06d7d81740 1 --2- 172.21.15.84:0/2420369501 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558ce4ef7480 0x558ce4ef98d0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.543+0000 7f06d7d81740 1 --2- 172.21.15.84:0/2420369501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ee7380 0x558ce4ee4cf0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.543+0000 7f06d7d81740 1 --2- 172.21.15.84:0/2420369501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ce4ed87a0 0x558ce4ee47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.543+0000 7f06b5ffb640 0 client.16968 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:34.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.543+0000 7f06d7d81740 1 --2- 172.21.15.84:0/2420369501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce4ed7e80 0x558ce4ee4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.543+0000 7f06d7d81740 1 -- 172.21.15.84:0/2420369501 >> 172.21.15.84:0/2420369501 conn(0x558ce4ed42a0 msgr2=0x558ce4eda0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:34.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.543+0000 7f06d7d81740 1 -- 172.21.15.84:0/2420369501 shutdown_connections 2024-05-22T23:11:34.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.543+0000 7f06d7d81740 1 -- 172.21.15.84:0/2420369501 wait complete. 2024-05-22T23:11:34.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.543+0000 7f06d7d81740 20 client.16968 trim_cache size 0 max 16384 2024-05-22T23:11:34.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.556+0000 7f06d7d81740 1 Processor -- start 2024-05-22T23:11:34.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.556+0000 7f06d7d81740 1 -- start start 2024-05-22T23:11:34.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.556+0000 7f06d7d81740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ce4ed7e80 0x558ce4ee69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.557+0000 7f06d7d81740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ed87a0 0x558ce4ef2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.557+0000 7f06d7d81740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce4ee7380 0x558ce4ed3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.557+0000 7f06d7d81740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f06d01a9520 con 0x558ce4ed87a0 2024-05-22T23:11:34.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.557+0000 7f06d7d81740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f06d01abd80 con 0x558ce4ed7e80 2024-05-22T23:11:34.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.557+0000 7f06d7d81740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f06d01a9b70 con 0x558ce4ee7380 2024-05-22T23:11:34.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.557+0000 7f06cdd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ed87a0 0x558ce4ef2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.557+0000 7f06cdd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ed87a0 0x558ce4ef2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48542/0 (socket says 172.21.15.84:48542) 2024-05-22T23:11:34.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.557+0000 7f06cdd9b640 1 -- 172.21.15.84:0/2315166849 learned_addr learned my addr 172.21.15.84:0/2315166849 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:34.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.557+0000 7f06ce59c640 1 --2- 172.21.15.84:0/2315166849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ce4ed7e80 0x558ce4ee69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.557+0000 7f06cdd9b640 1 -- 172.21.15.84:0/2315166849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce4ee7380 msgr2=0x558ce4ed3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:34.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.557+0000 7f06cdd9b640 1 --2- 172.21.15.84:0/2315166849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce4ee7380 0x558ce4ed3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.557+0000 7f06cdd9b640 1 -- 172.21.15.84:0/2315166849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ce4ed7e80 msgr2=0x558ce4ee69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:34.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.557+0000 7f06cdd9b640 1 --2- 172.21.15.84:0/2315166849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ce4ed7e80 0x558ce4ee69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.557+0000 7f06cdd9b640 1 -- 172.21.15.84:0/2315166849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ce4ed40d0 con 0x558ce4ed87a0 2024-05-22T23:11:34.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.557+0000 7f06cdd9b640 1 --2- 172.21.15.84:0/2315166849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ed87a0 0x558ce4ef2fe0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f06b800e9b0 tx=0x7f06b800b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7317874dcf0bc4d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:34.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.557+0000 7f06b7fff640 1 -- 172.21.15.84:0/2315166849 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b80a5cb0 con 0x558ce4ed87a0 2024-05-22T23:11:34.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.557+0000 7f06b7fff640 1 -- 172.21.15.84:0/2315166849 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06b809b3f0 con 0x558ce4ed87a0 2024-05-22T23:11:34.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.557+0000 7f06b7fff640 1 -- 172.21.15.84:0/2315166849 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b809d350 con 0x558ce4ed87a0 2024-05-22T23:11:34.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.558+0000 7f06d7d81740 1 -- 172.21.15.84:0/2315166849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ed87a0 msgr2=0x558ce4ef2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:34.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.558+0000 7f06d7d81740 1 --2- 172.21.15.84:0/2315166849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ed87a0 0x558ce4ef2fe0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f06b800e9b0 tx=0x7f06b800b5f0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.558+0000 7f06d7d81740 1 -- 172.21.15.84:0/2315166849 shutdown_connections 2024-05-22T23:11:34.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.558+0000 7f06d7d81740 1 --2- 172.21.15.84:0/2315166849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce4ee7380 0x558ce4ed3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.558+0000 7f06d7d81740 1 --2- 172.21.15.84:0/2315166849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ed87a0 0x558ce4ef2fe0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.558+0000 7f06d7d81740 1 --2- 172.21.15.84:0/2315166849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ce4ed7e80 0x558ce4ee69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.558+0000 7f06d7d81740 1 -- 172.21.15.84:0/2315166849 >> 172.21.15.84:0/2315166849 conn(0x558ce4ed42a0 msgr2=0x558ce4ed2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:34.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.558+0000 7f06d7d81740 1 -- 172.21.15.84:0/2315166849 shutdown_connections 2024-05-22T23:11:34.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.558+0000 7f06d7d81740 1 -- 172.21.15.84:0/2315166849 wait complete. 2024-05-22T23:11:34.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.558+0000 7f06d7d81740 1 -- start start 2024-05-22T23:11:34.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.558+0000 7f06d7d81740 1 Processor -- start 2024-05-22T23:11:34.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.558+0000 7f06d7d81740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce4ed7e80 0x558ce4eeaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.558+0000 7f06d7d81740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ed87a0 0x558ce4eeb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.558+0000 7f06d7d81740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ce4ee7380 0x558ce4eeb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.558+0000 7f06d7d81740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f06d01aa240 con 0x558ce4ed87a0 2024-05-22T23:11:34.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.558+0000 7f06d7d81740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f06d01ab6b0 con 0x558ce4ee7380 2024-05-22T23:11:34.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.558+0000 7f06d7d81740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f06d01aa910 con 0x558ce4ed7e80 2024-05-22T23:11:34.579 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.558+0000 7f06d7d81740 10 client.0 _ll_register_callbacks cb 0x7f06c8013490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:11:34.579 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.558+0000 7f06cdd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ed87a0 0x558ce4eeb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.579 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06cdd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ed87a0 0x558ce4eeb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48550/0 (socket says 172.21.15.84:48550) 2024-05-22T23:11:34.579 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06cdd9b640 1 -- 172.21.15.84:0/2926363504 learned_addr learned my addr 172.21.15.84:0/2926363504 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:34.579 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06ce59c640 1 --2- 172.21.15.84:0/2926363504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce4ed7e80 0x558ce4eeaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.579 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06cdd9b640 1 -- 172.21.15.84:0/2926363504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce4ed7e80 msgr2=0x558ce4eeaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:34.579 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06ced9d640 1 --2- 172.21.15.84:0/2926363504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ce4ee7380 0x558ce4eeb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.579 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06cdd9b640 1 --2- 172.21.15.84:0/2926363504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce4ed7e80 0x558ce4eeaf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.580 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06cdd9b640 1 -- 172.21.15.84:0/2926363504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ce4ee7380 msgr2=0x558ce4eeb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:34.580 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06cdd9b640 1 --2- 172.21.15.84:0/2926363504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ce4ee7380 0x558ce4eeb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.580 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06cdd9b640 1 -- 172.21.15.84:0/2926363504 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ce4f5ca60 con 0x558ce4ed87a0 2024-05-22T23:11:34.580 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06ce59c640 1 --2- 172.21.15.84:0/2926363504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce4ed7e80 0x558ce4eeaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:11:34.580 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06ced9d640 1 --2- 172.21.15.84:0/2926363504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ce4ee7380 0x558ce4eeb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:11:34.580 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06cdd9b640 1 --2- 172.21.15.84:0/2926363504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ed87a0 0x558ce4eeb4a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f06b80ce320 tx=0x7f06b8099a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2934fd1008c00 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:34.580 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:11:34.580 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06b5ffb640 1 -- 172.21.15.84:0/2926363504 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b809d350 con 0x558ce4ed87a0 2024-05-22T23:11:34.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06b5ffb640 1 -- 172.21.15.84:0/2926363504 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8571+0+0 (secure 0 0 0) 0x7f06b809d4f0 con 0x558ce4ed87a0 2024-05-22T23:11:34.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06d7d81740 1 -- 172.21.15.84:0/2926363504 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06b809b3f0 con 0x558ce4ed87a0 2024-05-22T23:11:34.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06b5ffb640 1 -- 172.21.15.84:0/2926363504 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f06b80a5820 con 0x558ce4ed87a0 2024-05-22T23:11:34.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06d7d81740 10 client.16974 fetch_fsmap learned FSMap version 18 2024-05-22T23:11:34.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06d7d81740 1 -- 172.21.15.84:0/2926363504 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x558ce4f5dd60 con 0x558ce4ed87a0 2024-05-22T23:11:34.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06b5ffb640 1 -- 172.21.15.84:0/2926363504 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06b809b3f0 con 0x558ce4ed87a0 2024-05-22T23:11:34.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06b5ffb640 1 -- 172.21.15.84:0/2926363504 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f06b800e040 con 0x558ce4ed87a0 2024-05-22T23:11:34.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06d7d81740 10 client.16974 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:11:34.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06d7d81740 10 client.16974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:11:34.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06d7d81740 20 client.16974 populate_metadata read hostname 'smithi084' 2024-05-22T23:11:34.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.559+0000 7f06d7d81740 1 --2- 172.21.15.84:0/2926363504 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558ce4f5e3c0 0x558ce4f607e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:34.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.560+0000 7f06d7d81740 4 client.16974 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-22T23:11:34.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.560+0000 7f06d7d81740 1 -- 172.21.15.84:0/2926363504 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ce4e44830 con 0x558ce4f5e3c0 2024-05-22T23:11:34.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.560+0000 7f06ce59c640 1 --2- 172.21.15.84:0/2926363504 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558ce4f5e3c0 0x558ce4f607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:34.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.560+0000 7f06b5ffb640 10 client.16974 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:34.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.560+0000 7f06ce59c640 1 --2- 172.21.15.84:0/2926363504 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558ce4f5e3c0 0x558ce4f607e0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d63ab44bf73e6e34 server_cookie=4aa96547c3cc3059 in_seq=0 out_seq=0 2024-05-22T23:11:34.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.560+0000 7f06b5ffb640 1 -- 172.21.15.84:0/2926363504 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ce4e44830 con 0x558ce4f5e3c0 2024-05-22T23:11:34.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.560+0000 7f06b5ffb640 10 client.16974 handle_command_reply: tid=0 2024-05-22T23:11:34.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.560+0000 7f06d7d81740 1 client.16974 shutdown 2024-05-22T23:11:34.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.560+0000 7f06d7d81740 1 -- 172.21.15.84:0/2926363504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ed87a0 msgr2=0x558ce4eeb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:34.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.560+0000 7f06d7d81740 1 --2- 172.21.15.84:0/2926363504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ed87a0 0x558ce4eeb4a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f06b80ce320 tx=0x7f06b8099a00 comp rx=0 tx=0).stop 2024-05-22T23:11:34.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.560+0000 7f06d7d81740 1 -- 172.21.15.84:0/2926363504 shutdown_connections 2024-05-22T23:11:34.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.560+0000 7f06d7d81740 1 --2- 172.21.15.84:0/2926363504 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x558ce4f5e3c0 0x558ce4f607e0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.560+0000 7f06d7d81740 1 --2- 172.21.15.84:0/2926363504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ce4ee7380 0x558ce4eeb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.560+0000 7f06d7d81740 1 --2- 172.21.15.84:0/2926363504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4ed87a0 0x558ce4eeb4a0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.560+0000 7f06b5ffb640 0 client.16974 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:34.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.560+0000 7f06d7d81740 1 --2- 172.21.15.84:0/2926363504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce4ed7e80 0x558ce4eeaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.560+0000 7f06d7d81740 1 -- 172.21.15.84:0/2926363504 >> 172.21.15.84:0/2926363504 conn(0x558ce4ed42a0 msgr2=0x558ce4ee2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:34.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.560+0000 7f06d7d81740 1 -- 172.21.15.84:0/2926363504 shutdown_connections 2024-05-22T23:11:34.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.560+0000 7f06d7d81740 1 -- 172.21.15.84:0/2926363504 wait complete. 2024-05-22T23:11:34.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.560+0000 7f06d7d81740 20 client.16974 trim_cache size 0 max 16384 2024-05-22T23:11:34.585 INFO:teuthology.orchestra.run.smithi084.stdout:[] 2024-05-22T23:11:34.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.561+0000 7f06d4f9d640 1 -- 172.21.15.84:0/2610980838 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f06bc0a2d00 msgr2=0x7f06bc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:34.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.561+0000 7f06d4f9d640 1 --2- 172.21.15.84:0/2610980838 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f06bc0a2d00 0x7f06bc0a51c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f06c8005fd0 tx=0x7f06c8004a40 comp rx=0 tx=0).stop 2024-05-22T23:11:34.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.561+0000 7f06d4f9d640 1 -- 172.21.15.84:0/2610980838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06d0101110 msgr2=0x7f06d0197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:34.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.561+0000 7f06d4f9d640 1 --2- 172.21.15.84:0/2610980838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06d0101110 0x7f06d0197670 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f06b800d990 tx=0x7f06b800de50 comp rx=0 tx=0).stop 2024-05-22T23:11:34.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.561+0000 7f06d4f9d640 1 -- 172.21.15.84:0/2610980838 shutdown_connections 2024-05-22T23:11:34.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.561+0000 7f06d4f9d640 1 --2- 172.21.15.84:0/2610980838 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f06bc0a2d00 0x7f06bc0a51c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.561+0000 7f06d4f9d640 1 --2- 172.21.15.84:0/2610980838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06d0106760 0x7f06d019b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.561+0000 7f06d4f9d640 1 --2- 172.21.15.84:0/2610980838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06d0101110 0x7f06d0197670 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.561+0000 7f06d4f9d640 1 --2- 172.21.15.84:0/2610980838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06d0100750 0x7f06d0197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:34.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.561+0000 7f06d4f9d640 1 -- 172.21.15.84:0/2610980838 >> 172.21.15.84:0/2610980838 conn(0x7f06d0076af0 msgr2=0x7f06d00fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:34.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.561+0000 7f06d4f9d640 1 -- 172.21.15.84:0/2610980838 shutdown_connections 2024-05-22T23:11:34.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:34.561+0000 7f06d4f9d640 1 -- 172.21.15.84:0/2610980838 wait complete. 2024-05-22T23:11:35.589 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-22T23:11:35.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:11:35.590 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-22T23:11:35.617 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:35 smithi084 ceph-mon[32186]: pgmap v303: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:11:35.618 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:35 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2198849506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:11:35.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:35 smithi204 ceph-mon[36211]: pgmap v303: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:11:35.667 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:35 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2198849506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:11:35.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.700+0000 7f6c069b7640 1 -- 172.21.15.84:0/2870071261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c00101840 msgr2=0x7f6c00101cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:35.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.700+0000 7f6c069b7640 1 --2- 172.21.15.84:0/2870071261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c00101840 0x7f6c00101cc0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f6be8009930 tx=0x7f6be802a420 comp rx=0 tx=0).stop 2024-05-22T23:11:35.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.700+0000 7f6c069b7640 1 -- 172.21.15.84:0/2870071261 shutdown_connections 2024-05-22T23:11:35.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.700+0000 7f6c069b7640 1 --2- 172.21.15.84:0/2870071261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c00102200 0x7f6c0010e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.700+0000 7f6c069b7640 1 --2- 172.21.15.84:0/2870071261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c00101840 0x7f6c00101cc0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.700+0000 7f6c069b7640 1 --2- 172.21.15.84:0/2870071261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c00107850 0x7f6c00107c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.700+0000 7f6c069b7640 1 -- 172.21.15.84:0/2870071261 >> 172.21.15.84:0/2870071261 conn(0x7f6c00078cc0 msgr2=0x7f6c000790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:35.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.700+0000 7f6c069b7640 1 -- 172.21.15.84:0/2870071261 shutdown_connections 2024-05-22T23:11:35.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.700+0000 7f6c069b7640 1 -- 172.21.15.84:0/2870071261 wait complete. 2024-05-22T23:11:35.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.700+0000 7f6c069b7640 1 Processor -- start 2024-05-22T23:11:35.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.701+0000 7f6c069b7640 1 -- start start 2024-05-22T23:11:35.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.701+0000 7f6c069b7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c00101840 0x7f6c00198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.701+0000 7f6c069b7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c00102200 0x7f6c001986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.701+0000 7f6c069b7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c00107850 0x7f6c0019ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.701+0000 7f6c069b7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6c00111c20 con 0x7f6c00102200 2024-05-22T23:11:35.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.701+0000 7f6c069b7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6c00111aa0 con 0x7f6c00107850 2024-05-22T23:11:35.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.701+0000 7f6c069b7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6c00111da0 con 0x7f6c00101840 2024-05-22T23:11:35.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.701+0000 7f6bf7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c00102200 0x7f6c001986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.701+0000 7f6bf7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c00102200 0x7f6c001986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48578/0 (socket says 172.21.15.84:48578) 2024-05-22T23:11:35.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.701+0000 7f6bf7fff640 1 -- 172.21.15.84:0/364342579 learned_addr learned my addr 172.21.15.84:0/364342579 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:35.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.701+0000 7f6c04f54640 1 --2- 172.21.15.84:0/364342579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c00107850 0x7f6c0019ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.701+0000 7f6bfffff640 1 --2- 172.21.15.84:0/364342579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c00101840 0x7f6c00198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.701+0000 7f6bf7fff640 1 -- 172.21.15.84:0/364342579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c00101840 msgr2=0x7f6c00198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:35.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.701+0000 7f6bf7fff640 1 --2- 172.21.15.84:0/364342579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c00101840 0x7f6c00198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.701+0000 7f6bf7fff640 1 -- 172.21.15.84:0/364342579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c00107850 msgr2=0x7f6c0019ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:35.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.701+0000 7f6bf7fff640 1 --2- 172.21.15.84:0/364342579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c00107850 0x7f6c0019ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.701+0000 7f6bf7fff640 1 -- 172.21.15.84:0/364342579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c0019d150 con 0x7f6c00102200 2024-05-22T23:11:35.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.701+0000 7f6c04f54640 1 --2- 172.21.15.84:0/364342579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c00107850 0x7f6c0019ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:11:35.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.701+0000 7f6bfffff640 1 --2- 172.21.15.84:0/364342579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c00101840 0x7f6c00198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:11:35.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.701+0000 7f6bf7fff640 1 --2- 172.21.15.84:0/364342579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c00102200 0x7f6c001986a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f6be8009230 tx=0x7f6be8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:35.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.701+0000 7f6bfdffb640 1 -- 172.21.15.84:0/364342579 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be803f030 con 0x7f6c00102200 2024-05-22T23:11:35.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.702+0000 7f6bfdffb640 1 -- 172.21.15.84:0/364342579 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6be8003ad0 con 0x7f6c00102200 2024-05-22T23:11:35.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.702+0000 7f6c069b7640 1 -- 172.21.15.84:0/364342579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c0019d3e0 con 0x7f6c00102200 2024-05-22T23:11:35.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.702+0000 7f6bfdffb640 1 -- 172.21.15.84:0/364342579 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be8035070 con 0x7f6c00102200 2024-05-22T23:11:35.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.702+0000 7f6c069b7640 1 -- 172.21.15.84:0/364342579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c001a4cd0 con 0x7f6c00102200 2024-05-22T23:11:35.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.704+0000 7f6bfdffb640 1 -- 172.21.15.84:0/364342579 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6be8042030 con 0x7f6c00102200 2024-05-22T23:11:35.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.704+0000 7f6c097a4740 1 Processor -- start 2024-05-22T23:11:35.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.704+0000 7f6c097a4740 1 -- start start 2024-05-22T23:11:35.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.704+0000 7f6c097a4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56015cd1ac50 0x56015cd1b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.704+0000 7f6c097a4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd1b570 0x56015cd25700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.704+0000 7f6c097a4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56015cd25c40 0x56015cd28030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.704+0000 7f6c097a4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56015cc7ea40 con 0x56015cd1b570 2024-05-22T23:11:35.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.704+0000 7f6c097a4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56015cc1d440 con 0x56015cd1ac50 2024-05-22T23:11:35.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.704+0000 7f6c097a4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56015cc7aa00 con 0x56015cd25c40 2024-05-22T23:11:35.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.704+0000 7f6bfffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd1b570 0x56015cd25700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.704+0000 7f6bfffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd1b570 0x56015cd25700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48586/0 (socket says 172.21.15.84:48586) 2024-05-22T23:11:35.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.704+0000 7f6bfffff640 1 -- 172.21.15.84:0/808416264 learned_addr learned my addr 172.21.15.84:0/808416264 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:35.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.705+0000 7f6c04f54640 1 --2- 172.21.15.84:0/808416264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56015cd1ac50 0x56015cd1b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.705+0000 7f6bf7fff640 1 --2- 172.21.15.84:0/808416264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56015cd25c40 0x56015cd28030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.705+0000 7f6bfdffb640 1 --2- 172.21.15.84:0/364342579 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6bec0a2d00 0x7f6bec0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.705+0000 7f6bfffff640 1 -- 172.21.15.84:0/808416264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56015cd25c40 msgr2=0x56015cd28030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:35.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.705+0000 7f6bfffff640 1 --2- 172.21.15.84:0/808416264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56015cd25c40 0x56015cd28030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.705+0000 7f6bfffff640 1 -- 172.21.15.84:0/808416264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56015cd1ac50 msgr2=0x56015cd1b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:35.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.705+0000 7f6bfffff640 1 --2- 172.21.15.84:0/808416264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56015cd1ac50 0x56015cd1b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.705+0000 7f6bfffff640 1 -- 172.21.15.84:0/808416264 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56015cc272d0 con 0x56015cd1b570 2024-05-22T23:11:35.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.705+0000 7f6c04f54640 1 --2- 172.21.15.84:0/364342579 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6bec0a2d00 0x7f6bec0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.705+0000 7f6bfdffb640 1 -- 172.21.15.84:0/364342579 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8571+0+0 (secure 0 0 0) 0x7f6be80ef6c0 con 0x7f6c00102200 2024-05-22T23:11:35.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.705+0000 7f6bfffff640 1 --2- 172.21.15.84:0/808416264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd1b570 0x56015cd25700 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0009bd0 tx=0x7f6bf000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f43f96a127be7e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:35.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.705+0000 7f6bf67fc640 1 -- 172.21.15.84:0/808416264 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0016070 con 0x56015cd1b570 2024-05-22T23:11:35.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.705+0000 7f6bf67fc640 1 -- 172.21.15.84:0/808416264 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bf0005500 con 0x56015cd1b570 2024-05-22T23:11:35.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.705+0000 7f6bf67fc640 1 -- 172.21.15.84:0/808416264 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf00116d0 con 0x56015cd1b570 2024-05-22T23:11:35.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.706+0000 7f6c097a4740 1 -- 172.21.15.84:0/808416264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd1b570 msgr2=0x56015cd25700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:35.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.706+0000 7f6c04f54640 1 --2- 172.21.15.84:0/364342579 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6bec0a2d00 0x7f6bec0a51c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8005fd0 tx=0x7f6bf8004a40 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:35.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.706+0000 7f6c097a4740 1 --2- 172.21.15.84:0/808416264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd1b570 0x56015cd25700 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0009bd0 tx=0x7f6bf000e6c0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.706+0000 7f6c097a4740 1 -- 172.21.15.84:0/808416264 shutdown_connections 2024-05-22T23:11:35.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.706+0000 7f6c097a4740 1 --2- 172.21.15.84:0/808416264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56015cd25c40 0x56015cd28030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.706+0000 7f6c097a4740 1 --2- 172.21.15.84:0/808416264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd1b570 0x56015cd25700 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.706+0000 7f6c097a4740 1 --2- 172.21.15.84:0/808416264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56015cd1ac50 0x56015cd1b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.706+0000 7f6c097a4740 1 -- 172.21.15.84:0/808416264 >> 172.21.15.84:0/808416264 conn(0x56015cd16f70 msgr2=0x56015cd17350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:35.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.706+0000 7f6c097a4740 1 -- 172.21.15.84:0/808416264 shutdown_connections 2024-05-22T23:11:35.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.706+0000 7f6c097a4740 1 -- 172.21.15.84:0/808416264 wait complete. 2024-05-22T23:11:35.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.706+0000 7f6c097a4740 1 -- start start 2024-05-22T23:11:35.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.706+0000 7f6c097a4740 1 Processor -- start 2024-05-22T23:11:35.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.707+0000 7f6c097a4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56015cd1ac50 0x56015cd22b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.707+0000 7f6c097a4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56015cd1b570 0x56015cd23080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.707+0000 7f6c097a4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd25c40 0x56015cd235c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.707+0000 7f6c097a4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56015cd28a30 con 0x56015cd25c40 2024-05-22T23:11:35.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.707+0000 7f6c097a4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56015cc085f0 con 0x56015cd1ac50 2024-05-22T23:11:35.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.707+0000 7f6c097a4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56015cd28bb0 con 0x56015cd1b570 2024-05-22T23:11:35.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.707+0000 7f6c097a4740 10 client.0 _ll_register_callbacks cb 0x56015cc03cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:11:35.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.707+0000 7f6c04f54640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd25c40 0x56015cd235c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.707+0000 7f6c04f54640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd25c40 0x56015cd235c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48602/0 (socket says 172.21.15.84:48602) 2024-05-22T23:11:35.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.707+0000 7f6c04f54640 1 -- 172.21.15.84:0/2825587049 learned_addr learned my addr 172.21.15.84:0/2825587049 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:35.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.707+0000 7f6bfffff640 1 --2- 172.21.15.84:0/2825587049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56015cd1ac50 0x56015cd22b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.707+0000 7f6bf7fff640 1 --2- 172.21.15.84:0/2825587049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56015cd1b570 0x56015cd23080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.707+0000 7f6c04f54640 1 -- 172.21.15.84:0/2825587049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56015cd1b570 msgr2=0x56015cd23080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:35.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.707+0000 7f6c04f54640 1 --2- 172.21.15.84:0/2825587049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56015cd1b570 0x56015cd23080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.707+0000 7f6c04f54640 1 -- 172.21.15.84:0/2825587049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56015cd1ac50 msgr2=0x56015cd22b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:35.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.707+0000 7f6c04f54640 1 --2- 172.21.15.84:0/2825587049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56015cd1ac50 0x56015cd22b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.707+0000 7f6c04f54640 1 -- 172.21.15.84:0/2825587049 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56015ca5a7e0 con 0x56015cd25c40 2024-05-22T23:11:35.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.707+0000 7f6c04f54640 1 --2- 172.21.15.84:0/2825587049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd25c40 0x56015cd235c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80095b0 tx=0x7f6bf8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebbbcd2cbe4ad84d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:35.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.707+0000 7f6bdf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:11:35.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.707+0000 7f6bdf7fe640 1 -- 172.21.15.84:0/2825587049 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf802b060 con 0x56015cd25c40 2024-05-22T23:11:35.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.708+0000 7f6bdf7fe640 1 -- 172.21.15.84:0/2825587049 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8571+0+0 (secure 0 0 0) 0x7f6bf8018070 con 0x56015cd25c40 2024-05-22T23:11:35.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.708+0000 7f6c097a4740 1 -- 172.21.15.84:0/2825587049 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bf0005500 con 0x56015cd25c40 2024-05-22T23:11:35.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.708+0000 7f6bdf7fe640 1 -- 172.21.15.84:0/2825587049 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f6bf801e830 con 0x56015cd25c40 2024-05-22T23:11:35.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.708+0000 7f6c097a4740 10 client.16986 fetch_fsmap learned FSMap version 18 2024-05-22T23:11:35.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.708+0000 7f6c097a4740 1 -- 172.21.15.84:0/2825587049 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x56015cabaf70 con 0x56015cd25c40 2024-05-22T23:11:35.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.708+0000 7f6bdf7fe640 1 -- 172.21.15.84:0/2825587049 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bf0005500 con 0x56015cd25c40 2024-05-22T23:11:35.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.708+0000 7f6bdf7fe640 1 -- 172.21.15.84:0/2825587049 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f6bf800f040 con 0x56015cd25c40 2024-05-22T23:11:35.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.708+0000 7f6c097a4740 10 client.16986 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:11:35.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.708+0000 7f6c097a4740 10 client.16986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:11:35.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.708+0000 7f6c097a4740 20 client.16986 populate_metadata read hostname 'smithi084' 2024-05-22T23:11:35.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.708+0000 7f6c097a4740 1 --2- 172.21.15.84:0/2825587049 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56015cd35f80 0x56015cd383d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.708+0000 7f6c097a4740 4 client.16986 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:11:35.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.708+0000 7f6c097a4740 1 -- 172.21.15.84:0/2825587049 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56015cc92460 con 0x56015cd35f80 2024-05-22T23:11:35.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.708+0000 7f6bfffff640 1 --2- 172.21.15.84:0/2825587049 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56015cd35f80 0x56015cd383d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.709+0000 7f6bdf7fe640 10 client.16986 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:35.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.709+0000 7f6bfffff640 1 --2- 172.21.15.84:0/2825587049 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56015cd35f80 0x56015cd383d0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbba7046ab76c7 server_cookie=52457782543dae36 in_seq=0 out_seq=0 2024-05-22T23:11:35.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.709+0000 7f6bdf7fe640 1 -- 172.21.15.84:0/2825587049 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56015cc92460 con 0x56015cd35f80 2024-05-22T23:11:35.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.709+0000 7f6bdf7fe640 10 client.16986 handle_command_reply: tid=0 2024-05-22T23:11:35.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.710+0000 7f6c097a4740 1 client.16986 shutdown 2024-05-22T23:11:35.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.710+0000 7f6c097a4740 1 -- 172.21.15.84:0/2825587049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd25c40 msgr2=0x56015cd235c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:35.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.710+0000 7f6c097a4740 1 --2- 172.21.15.84:0/2825587049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd25c40 0x56015cd235c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80095b0 tx=0x7f6bf8004ab0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.710+0000 7f6c097a4740 1 -- 172.21.15.84:0/2825587049 shutdown_connections 2024-05-22T23:11:35.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.710+0000 7f6c097a4740 1 --2- 172.21.15.84:0/2825587049 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56015cd35f80 0x56015cd383d0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.710+0000 7f6c097a4740 1 --2- 172.21.15.84:0/2825587049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd25c40 0x56015cd235c0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.710+0000 7f6c097a4740 1 --2- 172.21.15.84:0/2825587049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56015cd1b570 0x56015cd23080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.710+0000 7f6bdf7fe640 0 client.16986 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:35.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.710+0000 7f6c097a4740 1 --2- 172.21.15.84:0/2825587049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56015cd1ac50 0x56015cd22b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.710+0000 7f6c097a4740 1 -- 172.21.15.84:0/2825587049 >> 172.21.15.84:0/2825587049 conn(0x56015cd16f70 msgr2=0x56015cd161b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:35.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.710+0000 7f6c097a4740 1 -- 172.21.15.84:0/2825587049 shutdown_connections 2024-05-22T23:11:35.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.710+0000 7f6c097a4740 1 -- 172.21.15.84:0/2825587049 wait complete. 2024-05-22T23:11:35.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.710+0000 7f6c097a4740 20 client.16986 trim_cache size 0 max 16384 2024-05-22T23:11:35.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.727+0000 7f6c097a4740 1 Processor -- start 2024-05-22T23:11:35.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.727+0000 7f6c097a4740 1 -- start start 2024-05-22T23:11:35.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.727+0000 7f6c097a4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd1ac50 0x56015cd25310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.727+0000 7f6c097a4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56015cd1b570 0x56015cd318f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.727+0000 7f6c097a4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56015cd25c40 0x56015cd1d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.727+0000 7f6c097a4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6c001ab7b0 con 0x56015cd1ac50 2024-05-22T23:11:35.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.727+0000 7f6c097a4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6c001ae010 con 0x56015cd1b570 2024-05-22T23:11:35.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.727+0000 7f6c097a4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6c001abe00 con 0x56015cd25c40 2024-05-22T23:11:35.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.728+0000 7f6bfffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd1ac50 0x56015cd25310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.728+0000 7f6bfffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd1ac50 0x56015cd25310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48616/0 (socket says 172.21.15.84:48616) 2024-05-22T23:11:35.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.728+0000 7f6bfffff640 1 -- 172.21.15.84:0/2085815714 learned_addr learned my addr 172.21.15.84:0/2085815714 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:35.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.728+0000 7f6c04f54640 1 --2- 172.21.15.84:0/2085815714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56015cd25c40 0x56015cd1d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.728+0000 7f6bf7fff640 1 --2- 172.21.15.84:0/2085815714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56015cd1b570 0x56015cd318f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.728+0000 7f6bfffff640 1 -- 172.21.15.84:0/2085815714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56015cd25c40 msgr2=0x56015cd1d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:35.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.728+0000 7f6bfffff640 1 --2- 172.21.15.84:0/2085815714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56015cd25c40 0x56015cd1d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.728+0000 7f6bfffff640 1 -- 172.21.15.84:0/2085815714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56015cd1b570 msgr2=0x56015cd318f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:35.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.728+0000 7f6bfffff640 1 --2- 172.21.15.84:0/2085815714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56015cd1b570 0x56015cd318f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.728+0000 7f6bfffff640 1 -- 172.21.15.84:0/2085815714 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56015c952170 con 0x56015cd1ac50 2024-05-22T23:11:35.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.728+0000 7f6c04f54640 1 --2- 172.21.15.84:0/2085815714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56015cd25c40 0x56015cd1d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:11:35.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.728+0000 7f6bf7fff640 1 --2- 172.21.15.84:0/2085815714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56015cd1b570 0x56015cd318f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:11:35.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.728+0000 7f6bfffff640 1 --2- 172.21.15.84:0/2085815714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd1ac50 0x56015cd25310 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0009bd0 tx=0x7f6bf0009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c398fb51d2fd6c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:35.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.728+0000 7f6bf5ffb640 1 -- 172.21.15.84:0/2085815714 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0030070 con 0x56015cd1ac50 2024-05-22T23:11:35.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.728+0000 7f6bf5ffb640 1 -- 172.21.15.84:0/2085815714 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bf00097a0 con 0x56015cd1ac50 2024-05-22T23:11:35.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.728+0000 7f6bf5ffb640 1 -- 172.21.15.84:0/2085815714 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0016040 con 0x56015cd1ac50 2024-05-22T23:11:35.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.729+0000 7f6c097a4740 1 -- 172.21.15.84:0/2085815714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd1ac50 msgr2=0x56015cd25310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:35.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.729+0000 7f6c097a4740 1 --2- 172.21.15.84:0/2085815714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd1ac50 0x56015cd25310 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0009bd0 tx=0x7f6bf0009ad0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.729+0000 7f6c097a4740 1 -- 172.21.15.84:0/2085815714 shutdown_connections 2024-05-22T23:11:35.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.729+0000 7f6c097a4740 1 --2- 172.21.15.84:0/2085815714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56015cd25c40 0x56015cd1d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.729+0000 7f6c097a4740 1 --2- 172.21.15.84:0/2085815714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56015cd1b570 0x56015cd318f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.729+0000 7f6c097a4740 1 --2- 172.21.15.84:0/2085815714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd1ac50 0x56015cd25310 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.742 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.729+0000 7f6c097a4740 1 -- 172.21.15.84:0/2085815714 >> 172.21.15.84:0/2085815714 conn(0x56015cd16f70 msgr2=0x56015cd24c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:35.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.729+0000 7f6c097a4740 1 -- 172.21.15.84:0/2085815714 shutdown_connections 2024-05-22T23:11:35.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.729+0000 7f6c097a4740 1 -- 172.21.15.84:0/2085815714 wait complete. 2024-05-22T23:11:35.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.729+0000 7f6c097a4740 1 -- start start 2024-05-22T23:11:35.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.729+0000 7f6c097a4740 1 Processor -- start 2024-05-22T23:11:35.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6c097a4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56015cd1ac50 0x56015cd2dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6c097a4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd1b570 0x56015cd2add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6c097a4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56015cd25c40 0x56015cd2b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.743 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6c097a4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6c001ac4d0 con 0x56015cd1b570 2024-05-22T23:11:35.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6c097a4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6c001ad940 con 0x56015cd1ac50 2024-05-22T23:11:35.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6c097a4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6c001acba0 con 0x56015cd25c40 2024-05-22T23:11:35.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6c097a4740 10 client.0 _ll_register_callbacks cb 0x7f6bf8009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:11:35.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6bf7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd1b570 0x56015cd2add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6bf7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd1b570 0x56015cd2add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48626/0 (socket says 172.21.15.84:48626) 2024-05-22T23:11:35.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6bf7fff640 1 -- 172.21.15.84:0/2325298195 learned_addr learned my addr 172.21.15.84:0/2325298195 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:35.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6bfffff640 1 --2- 172.21.15.84:0/2325298195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56015cd1ac50 0x56015cd2dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.744 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6bf7fff640 1 -- 172.21.15.84:0/2325298195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56015cd25c40 msgr2=0x56015cd2b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:35.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6c04f54640 1 --2- 172.21.15.84:0/2325298195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56015cd25c40 0x56015cd2b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6bf7fff640 1 --2- 172.21.15.84:0/2325298195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56015cd25c40 0x56015cd2b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6bf7fff640 1 -- 172.21.15.84:0/2325298195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56015cd1ac50 msgr2=0x56015cd2dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:35.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6bf7fff640 1 --2- 172.21.15.84:0/2325298195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56015cd1ac50 0x56015cd2dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6bf7fff640 1 -- 172.21.15.84:0/2325298195 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56015cc47930 con 0x56015cd1b570 2024-05-22T23:11:35.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6bfffff640 1 --2- 172.21.15.84:0/2325298195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56015cd1ac50 0x56015cd2dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:11:35.745 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6bf7fff640 1 --2- 172.21.15.84:0/2325298195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd1b570 0x56015cd2add0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6be80ece40 tx=0x7f6be8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3393e8ea98f20b7d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:35.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6bdf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:11:35.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6bdf7fe640 1 -- 172.21.15.84:0/2325298195 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be8049040 con 0x56015cd1b570 2024-05-22T23:11:35.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6bdf7fe640 1 -- 172.21.15.84:0/2325298195 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8571+0+0 (secure 0 0 0) 0x7f6be8003c80 con 0x56015cd1b570 2024-05-22T23:11:35.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6c097a4740 1 -- 172.21.15.84:0/2325298195 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bf00097a0 con 0x56015cd1b570 2024-05-22T23:11:35.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6bdf7fe640 1 -- 172.21.15.84:0/2325298195 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f6be80c4820 con 0x56015cd1b570 2024-05-22T23:11:35.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6c097a4740 10 client.16992 fetch_fsmap learned FSMap version 18 2024-05-22T23:11:35.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.730+0000 7f6c097a4740 1 -- 172.21.15.84:0/2325298195 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x56015cd9d050 con 0x56015cd1b570 2024-05-22T23:11:35.746 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.731+0000 7f6bdf7fe640 1 -- 172.21.15.84:0/2325298195 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bf00097a0 con 0x56015cd1b570 2024-05-22T23:11:35.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.731+0000 7f6bdf7fe640 1 -- 172.21.15.84:0/2325298195 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f6be8031c60 con 0x56015cd1b570 2024-05-22T23:11:35.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.731+0000 7f6c097a4740 10 client.16992 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:11:35.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.731+0000 7f6c097a4740 10 client.16992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:11:35.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.731+0000 7f6c097a4740 20 client.16992 populate_metadata read hostname 'smithi084' 2024-05-22T23:11:35.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.731+0000 7f6c097a4740 1 --2- 172.21.15.84:0/2325298195 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56015cd9d7d0 0x56015cd9fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.731+0000 7f6c097a4740 4 client.16992 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-22T23:11:35.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.731+0000 7f6c097a4740 1 -- 172.21.15.84:0/2325298195 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56015cc8c170 con 0x56015cd9d7d0 2024-05-22T23:11:35.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.731+0000 7f6bfffff640 1 --2- 172.21.15.84:0/2325298195 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56015cd9d7d0 0x56015cd9fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.731+0000 7f6bdf7fe640 10 client.16992 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:35.748 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.731+0000 7f6bfffff640 1 --2- 172.21.15.84:0/2325298195 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56015cd9d7d0 0x56015cd9fbf0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aca578d914aeb802 server_cookie=c710fd57bc905e79 in_seq=0 out_seq=0 2024-05-22T23:11:35.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.733+0000 7f6bdf7fe640 1 -- 172.21.15.84:0/2325298195 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56015cc8c170 con 0x56015cd9d7d0 2024-05-22T23:11:35.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.733+0000 7f6bdf7fe640 10 client.16992 handle_command_reply: tid=0 2024-05-22T23:11:35.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.733+0000 7f6c097a4740 1 client.16992 shutdown 2024-05-22T23:11:35.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.733+0000 7f6c097a4740 1 -- 172.21.15.84:0/2325298195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd1b570 msgr2=0x56015cd2add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:35.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.733+0000 7f6c097a4740 1 --2- 172.21.15.84:0/2325298195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd1b570 0x56015cd2add0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6be80ece40 tx=0x7f6be8009260 comp rx=0 tx=0).stop 2024-05-22T23:11:35.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.733+0000 7f6c097a4740 1 -- 172.21.15.84:0/2325298195 shutdown_connections 2024-05-22T23:11:35.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.733+0000 7f6c097a4740 1 --2- 172.21.15.84:0/2325298195 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56015cd9d7d0 0x56015cd9fbf0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.733+0000 7f6c097a4740 1 --2- 172.21.15.84:0/2325298195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56015cd25c40 0x56015cd2b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.733+0000 7f6c097a4740 1 --2- 172.21.15.84:0/2325298195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56015cd1b570 0x56015cd2add0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.733+0000 7f6c097a4740 1 --2- 172.21.15.84:0/2325298195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56015cd1ac50 0x56015cd2dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.733+0000 7f6c097a4740 1 -- 172.21.15.84:0/2325298195 >> 172.21.15.84:0/2325298195 conn(0x56015cd16f70 msgr2=0x56015cd308e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:35.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.733+0000 7f6bdf7fe640 0 client.16992 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:35.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.733+0000 7f6c097a4740 1 -- 172.21.15.84:0/2325298195 shutdown_connections 2024-05-22T23:11:35.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.733+0000 7f6c097a4740 1 -- 172.21.15.84:0/2325298195 wait complete. 2024-05-22T23:11:35.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.733+0000 7f6c097a4740 20 client.16992 trim_cache size 0 max 16384 2024-05-22T23:11:35.751 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:11:35.751 INFO:teuthology.orchestra.run.smithi084.stdout: "return_code": 0, 2024-05-22T23:11:35.751 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_tag": "c5a4cea4-dbcc-4f69-bbba-ca301a39133c", 2024-05-22T23:11:35.751 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": "asynchronous" 2024-05-22T23:11:35.752 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:11:35.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.734+0000 7f6c069b7640 1 -- 172.21.15.84:0/364342579 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6bec0a2d00 msgr2=0x7f6bec0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:35.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.734+0000 7f6c069b7640 1 --2- 172.21.15.84:0/364342579 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6bec0a2d00 0x7f6bec0a51c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8005fd0 tx=0x7f6bf8004a40 comp rx=0 tx=0).stop 2024-05-22T23:11:35.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.734+0000 7f6c069b7640 1 -- 172.21.15.84:0/364342579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c00102200 msgr2=0x7f6c001986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:35.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.734+0000 7f6c069b7640 1 --2- 172.21.15.84:0/364342579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c00102200 0x7f6c001986a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f6be8009230 tx=0x7f6be8004270 comp rx=0 tx=0).stop 2024-05-22T23:11:35.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.734+0000 7f6c069b7640 1 -- 172.21.15.84:0/364342579 shutdown_connections 2024-05-22T23:11:35.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.734+0000 7f6c069b7640 1 --2- 172.21.15.84:0/364342579 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f6bec0a2d00 0x7f6bec0a51c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.734+0000 7f6c069b7640 1 --2- 172.21.15.84:0/364342579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c00107850 0x7f6c0019ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.734+0000 7f6c069b7640 1 --2- 172.21.15.84:0/364342579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c00102200 0x7f6c001986a0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.734+0000 7f6c069b7640 1 --2- 172.21.15.84:0/364342579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c00101840 0x7f6c00198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.734+0000 7f6c069b7640 1 -- 172.21.15.84:0/364342579 >> 172.21.15.84:0/364342579 conn(0x7f6c00078cc0 msgr2=0x7f6c000ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:35.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.734+0000 7f6c069b7640 1 -- 172.21.15.84:0/364342579 shutdown_connections 2024-05-22T23:11:35.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.734+0000 7f6c069b7640 1 -- 172.21.15.84:0/364342579 wait complete. 2024-05-22T23:11:35.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:11:35.755 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:11:35.805 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:35 smithi096 ceph-mon[36392]: pgmap v303: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:11:35.805 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:35 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2198849506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:11:35.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.888+0000 7f7df7790640 1 -- 172.21.15.84:0/2794470894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7df0100dc0 msgr2=0x7f7df0104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:35.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.888+0000 7f7df7790640 1 --2- 172.21.15.84:0/2794470894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7df0100dc0 0x7f7df0104b60 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009900 tx=0x7f7de002a410 comp rx=0 tx=0).stop 2024-05-22T23:11:35.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.888+0000 7f7df7790640 1 -- 172.21.15.84:0/2794470894 shutdown_connections 2024-05-22T23:11:35.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.888+0000 7f7df7790640 1 --2- 172.21.15.84:0/2794470894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7df0105290 0x7f7df010ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.888+0000 7f7df7790640 1 --2- 172.21.15.84:0/2794470894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7df0100dc0 0x7f7df0104b60 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.888+0000 7f7df7790640 1 --2- 172.21.15.84:0/2794470894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7df0100410 0x7f7df01007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.888+0000 7f7df7790640 1 -- 172.21.15.84:0/2794470894 >> 172.21.15.84:0/2794470894 conn(0x7f7df0075a00 msgr2=0x7f7df0075e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:35.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.888+0000 7f7df7790640 1 -- 172.21.15.84:0/2794470894 shutdown_connections 2024-05-22T23:11:35.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.888+0000 7f7df7790640 1 -- 172.21.15.84:0/2794470894 wait complete. 2024-05-22T23:11:35.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.889+0000 7f7df7790640 1 Processor -- start 2024-05-22T23:11:35.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.889+0000 7f7df7790640 1 -- start start 2024-05-22T23:11:35.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.889+0000 7f7df7790640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7df0100410 0x7f7df0198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.889+0000 7f7df7790640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7df0105290 0x7f7df01994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.889+0000 7f7df7790640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7df0199a70 0x7f7df01a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.889+0000 7f7df7790640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7df01102b0 con 0x7f7df0199a70 2024-05-22T23:11:35.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.889+0000 7f7df7790640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7df0110130 con 0x7f7df0105290 2024-05-22T23:11:35.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.889+0000 7f7df7790640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7df0110430 con 0x7f7df0100410 2024-05-22T23:11:35.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.889+0000 7f7df5d2d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7df0199a70 0x7f7df01a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.889+0000 7f7df5d2d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7df0199a70 0x7f7df01a4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48648/0 (socket says 172.21.15.84:48648) 2024-05-22T23:11:35.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.889+0000 7f7df5d2d640 1 -- 172.21.15.84:0/3681435312 learned_addr learned my addr 172.21.15.84:0/3681435312 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:35.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.889+0000 7f7df4d2b640 1 --2- 172.21.15.84:0/3681435312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7df0105290 0x7f7df01994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.889+0000 7f7df552c640 1 --2- 172.21.15.84:0/3681435312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7df0100410 0x7f7df0198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.889+0000 7f7df5d2d640 1 -- 172.21.15.84:0/3681435312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7df0100410 msgr2=0x7f7df0198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:35.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.889+0000 7f7df5d2d640 1 --2- 172.21.15.84:0/3681435312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7df0100410 0x7f7df0198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.889+0000 7f7df5d2d640 1 -- 172.21.15.84:0/3681435312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7df0105290 msgr2=0x7f7df01994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:35.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.889+0000 7f7df5d2d640 1 --2- 172.21.15.84:0/3681435312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7df0105290 0x7f7df01994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.890+0000 7f7df5d2d640 1 -- 172.21.15.84:0/3681435312 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df01a47f0 con 0x7f7df0199a70 2024-05-22T23:11:35.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.890+0000 7f7df552c640 1 --2- 172.21.15.84:0/3681435312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7df0100410 0x7f7df0198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:11:35.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.890+0000 7f7df5d2d640 1 --2- 172.21.15.84:0/3681435312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7df0199a70 0x7f7df01a4020 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7de800d750 tx=0x7f7de800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:35.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.890+0000 7f7de67fc640 1 -- 172.21.15.84:0/3681435312 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de8017070 con 0x7f7df0199a70 2024-05-22T23:11:35.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.890+0000 7f7df7790640 1 -- 172.21.15.84:0/3681435312 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df01a4ae0 con 0x7f7df0199a70 2024-05-22T23:11:35.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.890+0000 7f7de67fc640 1 -- 172.21.15.84:0/3681435312 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de8005500 con 0x7f7df0199a70 2024-05-22T23:11:35.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.890+0000 7f7de67fc640 1 -- 172.21.15.84:0/3681435312 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de8004d90 con 0x7f7df0199a70 2024-05-22T23:11:35.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.890+0000 7f7df7790640 1 -- 172.21.15.84:0/3681435312 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df01a5000 con 0x7f7df0199a70 2024-05-22T23:11:35.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.892+0000 7f7de67fc640 1 -- 172.21.15.84:0/3681435312 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7de8004f30 con 0x7f7df0199a70 2024-05-22T23:11:35.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.893+0000 7f7dfa579740 1 Processor -- start 2024-05-22T23:11:35.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.893+0000 7f7dfa579740 1 -- start start 2024-05-22T23:11:35.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.893+0000 7f7de67fc640 1 --2- 172.21.15.84:0/3681435312 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f7ddc0a2d00 0x7f7ddc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.893+0000 7f7dfa579740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fa69884a0 0x557fa69867e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.893+0000 7f7dfa579740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fa6986db0 0x557fa6987230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.893+0000 7f7dfa579740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa6991b30 0x557fa6993f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.893+0000 7f7dfa579740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557fa68eadb0 con 0x557fa6991b30 2024-05-22T23:11:35.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.893+0000 7f7dfa579740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557fa6889360 con 0x557fa6986db0 2024-05-22T23:11:35.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.893+0000 7f7dfa579740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557fa68e6d70 con 0x557fa69884a0 2024-05-22T23:11:35.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.893+0000 7f7df4d2b640 1 --2- 172.21.15.84:0/3681435312 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f7ddc0a2d00 0x7f7ddc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.893+0000 7f7df5d2d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa6991b30 0x557fa6993f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.893+0000 7f7df5d2d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa6991b30 0x557fa6993f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48662/0 (socket says 172.21.15.84:48662) 2024-05-22T23:11:35.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.893+0000 7f7df5d2d640 1 -- 172.21.15.84:0/2204982214 learned_addr learned my addr 172.21.15.84:0/2204982214 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:35.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.893+0000 7f7de67fc640 1 -- 172.21.15.84:0/3681435312 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8571+0+0 (secure 0 0 0) 0x7f7de80ced80 con 0x7f7df0199a70 2024-05-22T23:11:35.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.893+0000 7f7df4d2b640 1 --2- 172.21.15.84:0/2204982214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fa6986db0 0x557fa6987230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.893+0000 7f7df552c640 1 --2- 172.21.15.84:0/2204982214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fa69884a0 0x557fa69867e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.893+0000 7f7df5d2d640 1 -- 172.21.15.84:0/2204982214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fa69884a0 msgr2=0x557fa69867e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:35.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.893+0000 7f7df5d2d640 1 --2- 172.21.15.84:0/2204982214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fa69884a0 0x557fa69867e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.893+0000 7f7df5d2d640 1 -- 172.21.15.84:0/2204982214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fa6986db0 msgr2=0x557fa6987230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:35.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.893+0000 7f7df5d2d640 1 --2- 172.21.15.84:0/2204982214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fa6986db0 0x557fa6987230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.893+0000 7f7df5d2d640 1 -- 172.21.15.84:0/2204982214 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fa68aca60 con 0x557fa6991b30 2024-05-22T23:11:35.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.893+0000 7f7df4d2b640 1 --2- 172.21.15.84:0/3681435312 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f7ddc0a2d00 0x7f7ddc0a51c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7de0006010 tx=0x7f7de0032d10 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:35.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.893+0000 7f7df5d2d640 1 --2- 172.21.15.84:0/2204982214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa6991b30 0x557fa6993f20 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f7de809b200 tx=0x7f7de801e310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4161ad55e368db server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:35.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.894+0000 7f7ddb7fe640 1 -- 172.21.15.84:0/2204982214 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de8020070 con 0x557fa6991b30 2024-05-22T23:11:35.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.894+0000 7f7ddb7fe640 1 -- 172.21.15.84:0/2204982214 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de809b970 con 0x557fa6991b30 2024-05-22T23:11:35.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.894+0000 7f7ddb7fe640 1 -- 172.21.15.84:0/2204982214 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de80a3ee0 con 0x557fa6991b30 2024-05-22T23:11:35.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.894+0000 7f7dfa579740 1 -- 172.21.15.84:0/2204982214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa6991b30 msgr2=0x557fa6993f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:35.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.894+0000 7f7dfa579740 1 --2- 172.21.15.84:0/2204982214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa6991b30 0x557fa6993f20 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f7de809b200 tx=0x7f7de801e310 comp rx=0 tx=0).stop 2024-05-22T23:11:35.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.894+0000 7f7dfa579740 1 -- 172.21.15.84:0/2204982214 shutdown_connections 2024-05-22T23:11:35.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.894+0000 7f7dfa579740 1 --2- 172.21.15.84:0/2204982214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa6991b30 0x557fa6993f20 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.894+0000 7f7dfa579740 1 --2- 172.21.15.84:0/2204982214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fa6986db0 0x557fa6987230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.894+0000 7f7dfa579740 1 --2- 172.21.15.84:0/2204982214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fa69884a0 0x557fa69867e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.894+0000 7f7dfa579740 1 -- 172.21.15.84:0/2204982214 >> 172.21.15.84:0/2204982214 conn(0x557fa694f8e0 msgr2=0x557fa6983310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:35.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.894+0000 7f7dfa579740 1 -- 172.21.15.84:0/2204982214 shutdown_connections 2024-05-22T23:11:35.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.894+0000 7f7dfa579740 1 -- 172.21.15.84:0/2204982214 wait complete. 2024-05-22T23:11:35.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.895+0000 7f7dfa579740 1 -- start start 2024-05-22T23:11:35.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.895+0000 7f7dfa579740 1 Processor -- start 2024-05-22T23:11:35.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.895+0000 7f7dfa579740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fa6986db0 0x557fa698eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.895+0000 7f7dfa579740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa69884a0 0x557fa698efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.895+0000 7f7dfa579740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fa6991b30 0x557fa698f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.895+0000 7f7dfa579740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557fa6994960 con 0x557fa69884a0 2024-05-22T23:11:35.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.895+0000 7f7dfa579740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557fa68a50f0 con 0x557fa6986db0 2024-05-22T23:11:35.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.895+0000 7f7dfa579740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557fa6994ae0 con 0x557fa6991b30 2024-05-22T23:11:35.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.895+0000 7f7dfa579740 10 client.0 _ll_register_callbacks cb 0x557fa68a9f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:11:35.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.895+0000 7f7df4d2b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa69884a0 0x557fa698efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.895+0000 7f7df4d2b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa69884a0 0x557fa698efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48678/0 (socket says 172.21.15.84:48678) 2024-05-22T23:11:35.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.895+0000 7f7df4d2b640 1 -- 172.21.15.84:0/2001732144 learned_addr learned my addr 172.21.15.84:0/2001732144 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:35.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.895+0000 7f7df5d2d640 1 --2- 172.21.15.84:0/2001732144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fa6991b30 0x557fa698f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.895+0000 7f7df552c640 1 --2- 172.21.15.84:0/2001732144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fa6986db0 0x557fa698eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.896+0000 7f7df4d2b640 1 -- 172.21.15.84:0/2001732144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fa6991b30 msgr2=0x557fa698f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:35.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.896+0000 7f7df4d2b640 1 --2- 172.21.15.84:0/2001732144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fa6991b30 0x557fa698f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.896+0000 7f7df4d2b640 1 -- 172.21.15.84:0/2001732144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fa6986db0 msgr2=0x557fa698eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:35.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.896+0000 7f7df4d2b640 1 --2- 172.21.15.84:0/2001732144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fa6986db0 0x557fa698eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.896+0000 7f7df4d2b640 1 -- 172.21.15.84:0/2001732144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fa66c87d0 con 0x557fa69884a0 2024-05-22T23:11:35.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.896+0000 7f7df5d2d640 1 --2- 172.21.15.84:0/2001732144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fa6991b30 0x557fa698f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:11:35.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.896+0000 7f7df4d2b640 1 --2- 172.21.15.84:0/2001732144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa69884a0 0x557fa698efe0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f7de0005ec0 tx=0x7f7de002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98941d6be618d30 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:35.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.896+0000 7f7dd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:11:35.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.896+0000 7f7dd8ff9640 1 -- 172.21.15.84:0/2001732144 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de004d020 con 0x557fa69884a0 2024-05-22T23:11:35.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.896+0000 7f7dd8ff9640 1 -- 172.21.15.84:0/2001732144 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8571+0+0 (secure 0 0 0) 0x7f7de0005040 con 0x557fa69884a0 2024-05-22T23:11:35.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.896+0000 7f7dfa579740 1 -- 172.21.15.84:0/2001732144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7de809b970 con 0x557fa69884a0 2024-05-22T23:11:35.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.896+0000 7f7dd8ff9640 1 -- 172.21.15.84:0/2001732144 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f7de0058050 con 0x557fa69884a0 2024-05-22T23:11:35.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.896+0000 7f7dfa579740 10 client.17004 fetch_fsmap learned FSMap version 18 2024-05-22T23:11:35.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.896+0000 7f7dfa579740 1 -- 172.21.15.84:0/2001732144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x557fa65fe3f0 con 0x557fa69884a0 2024-05-22T23:11:35.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.896+0000 7f7dd8ff9640 1 -- 172.21.15.84:0/2001732144 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de809b970 con 0x557fa69884a0 2024-05-22T23:11:35.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.896+0000 7f7dd8ff9640 1 -- 172.21.15.84:0/2001732144 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f7de0048070 con 0x557fa69884a0 2024-05-22T23:11:35.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.896+0000 7f7dfa579740 10 client.17004 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:11:35.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.896+0000 7f7dfa579740 10 client.17004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:11:35.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.896+0000 7f7dfa579740 20 client.17004 populate_metadata read hostname 'smithi084' 2024-05-22T23:11:35.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.896+0000 7f7dfa579740 1 --2- 172.21.15.84:0/2001732144 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x557fa69a2030 0x557fa69a4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.896+0000 7f7dfa579740 4 client.17004 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:11:35.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.896+0000 7f7dfa579740 1 -- 172.21.15.84:0/2001732144 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fa68f84e0 con 0x557fa69a2030 2024-05-22T23:11:35.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.897+0000 7f7df552c640 1 --2- 172.21.15.84:0/2001732144 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x557fa69a2030 0x557fa69a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.897+0000 7f7dd8ff9640 10 client.17004 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:35.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.897+0000 7f7df552c640 1 --2- 172.21.15.84:0/2001732144 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x557fa69a2030 0x557fa69a4490 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eb52593f298f01d server_cookie=cb2ace8322101bc1 in_seq=0 out_seq=0 2024-05-22T23:11:35.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.898+0000 7f7dd8ff9640 1 -- 172.21.15.84:0/2001732144 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557fa68f84e0 con 0x557fa69a2030 2024-05-22T23:11:35.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.898+0000 7f7dd8ff9640 10 client.17004 handle_command_reply: tid=0 2024-05-22T23:11:35.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.898+0000 7f7dfa579740 1 client.17004 shutdown 2024-05-22T23:11:35.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.898+0000 7f7dfa579740 1 -- 172.21.15.84:0/2001732144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa69884a0 msgr2=0x557fa698efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:35.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.898+0000 7f7dfa579740 1 --2- 172.21.15.84:0/2001732144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa69884a0 0x557fa698efe0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f7de0005ec0 tx=0x7f7de002a410 comp rx=0 tx=0).stop 2024-05-22T23:11:35.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.898+0000 7f7dfa579740 1 -- 172.21.15.84:0/2001732144 shutdown_connections 2024-05-22T23:11:35.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.898+0000 7f7dfa579740 1 --2- 172.21.15.84:0/2001732144 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x557fa69a2030 0x557fa69a4490 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.898+0000 7f7dfa579740 1 --2- 172.21.15.84:0/2001732144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fa6991b30 0x557fa698f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.898+0000 7f7dd8ff9640 0 client.17004 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:35.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.898+0000 7f7dfa579740 1 --2- 172.21.15.84:0/2001732144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa69884a0 0x557fa698efe0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.898+0000 7f7dfa579740 1 --2- 172.21.15.84:0/2001732144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fa6986db0 0x557fa698eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.898+0000 7f7dfa579740 1 -- 172.21.15.84:0/2001732144 >> 172.21.15.84:0/2001732144 conn(0x557fa694f8e0 msgr2=0x557fa6989180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:35.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.898+0000 7f7dfa579740 1 -- 172.21.15.84:0/2001732144 shutdown_connections 2024-05-22T23:11:35.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.898+0000 7f7dfa579740 1 -- 172.21.15.84:0/2001732144 wait complete. 2024-05-22T23:11:35.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.898+0000 7f7dfa579740 20 client.17004 trim_cache size 0 max 16384 2024-05-22T23:11:35.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.915+0000 7f7dfa579740 1 Processor -- start 2024-05-22T23:11:35.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.915+0000 7f7dfa579740 1 -- start start 2024-05-22T23:11:35.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.915+0000 7f7dfa579740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fa6986db0 0x557fa699d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.915+0000 7f7dfa579740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa69884a0 0x557fa699a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.915+0000 7f7dfa579740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fa6991b30 0x557fa69a9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.915+0000 7f7dfa579740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7df01ab740 con 0x557fa69884a0 2024-05-22T23:11:35.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.915+0000 7f7dfa579740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7df01adfa0 con 0x557fa6991b30 2024-05-22T23:11:35.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.915+0000 7f7dfa579740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7df01abd90 con 0x557fa6986db0 2024-05-22T23:11:35.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.915+0000 7f7df4d2b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa69884a0 0x557fa699a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.916+0000 7f7df4d2b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa69884a0 0x557fa699a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48686/0 (socket says 172.21.15.84:48686) 2024-05-22T23:11:35.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.916+0000 7f7df4d2b640 1 -- 172.21.15.84:0/2017082398 learned_addr learned my addr 172.21.15.84:0/2017082398 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:35.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.916+0000 7f7df552c640 1 --2- 172.21.15.84:0/2017082398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fa6986db0 0x557fa699d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.916+0000 7f7df4d2b640 1 -- 172.21.15.84:0/2017082398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fa6986db0 msgr2=0x557fa699d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:35.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.916+0000 7f7df4d2b640 1 --2- 172.21.15.84:0/2017082398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fa6986db0 0x557fa699d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.916+0000 7f7df4d2b640 1 -- 172.21.15.84:0/2017082398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fa6991b30 msgr2=0x557fa69a9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:35.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.916+0000 7f7df4d2b640 1 --2- 172.21.15.84:0/2017082398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fa6991b30 0x557fa69a9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.916+0000 7f7df4d2b640 1 -- 172.21.15.84:0/2017082398 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fa65be0f0 con 0x557fa69884a0 2024-05-22T23:11:35.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.916+0000 7f7df552c640 1 --2- 172.21.15.84:0/2017082398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fa6986db0 0x557fa699d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:11:35.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.916+0000 7f7df4d2b640 1 --2- 172.21.15.84:0/2017082398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa69884a0 0x557fa699a150 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f7de002a930 tx=0x7f7de0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d62e4c519e5a05 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:35.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.916+0000 7f7ddaffd640 1 -- 172.21.15.84:0/2017082398 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0061070 con 0x557fa69884a0 2024-05-22T23:11:35.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.916+0000 7f7ddaffd640 1 -- 172.21.15.84:0/2017082398 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de00315b0 con 0x557fa69884a0 2024-05-22T23:11:35.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.916+0000 7f7ddaffd640 1 -- 172.21.15.84:0/2017082398 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de004d040 con 0x557fa69884a0 2024-05-22T23:11:35.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.917+0000 7f7dfa579740 1 -- 172.21.15.84:0/2017082398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa69884a0 msgr2=0x557fa699a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:35.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.917+0000 7f7dfa579740 1 --2- 172.21.15.84:0/2017082398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa69884a0 0x557fa699a150 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f7de002a930 tx=0x7f7de0005b50 comp rx=0 tx=0).stop 2024-05-22T23:11:35.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.917+0000 7f7dfa579740 1 -- 172.21.15.84:0/2017082398 shutdown_connections 2024-05-22T23:11:35.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.917+0000 7f7dfa579740 1 --2- 172.21.15.84:0/2017082398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fa6991b30 0x557fa69a9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.917+0000 7f7dfa579740 1 --2- 172.21.15.84:0/2017082398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa69884a0 0x557fa699a150 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.917+0000 7f7dfa579740 1 --2- 172.21.15.84:0/2017082398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fa6986db0 0x557fa699d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.917+0000 7f7dfa579740 1 -- 172.21.15.84:0/2017082398 >> 172.21.15.84:0/2017082398 conn(0x557fa694f8e0 msgr2=0x557fa698d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:35.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.917+0000 7f7dfa579740 1 -- 172.21.15.84:0/2017082398 shutdown_connections 2024-05-22T23:11:35.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.917+0000 7f7dfa579740 1 -- 172.21.15.84:0/2017082398 wait complete. 2024-05-22T23:11:35.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.917+0000 7f7dfa579740 1 -- start start 2024-05-22T23:11:35.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.917+0000 7f7dfa579740 1 Processor -- start 2024-05-22T23:11:35.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.917+0000 7f7dfa579740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fa6986db0 0x557fa6a00590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.917+0000 7f7dfa579740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa69884a0 0x557fa6a00ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.917+0000 7f7dfa579740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fa6991b30 0x557fa6a0b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.917+0000 7f7dfa579740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7df01ac460 con 0x557fa69884a0 2024-05-22T23:11:35.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.917+0000 7f7dfa579740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7df01ad8d0 con 0x557fa6986db0 2024-05-22T23:11:35.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.917+0000 7f7dfa579740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7df01acb30 con 0x557fa6991b30 2024-05-22T23:11:35.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.918+0000 7f7dfa579740 10 client.0 _ll_register_callbacks cb 0x7f7de0005e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:11:35.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.918+0000 7f7df4d2b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa69884a0 0x557fa6a00ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.918+0000 7f7df4d2b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa69884a0 0x557fa6a00ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48698/0 (socket says 172.21.15.84:48698) 2024-05-22T23:11:35.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.918+0000 7f7df4d2b640 1 -- 172.21.15.84:0/3604909994 learned_addr learned my addr 172.21.15.84:0/3604909994 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:35.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.918+0000 7f7df552c640 1 --2- 172.21.15.84:0/3604909994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fa6986db0 0x557fa6a00590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.918+0000 7f7df5d2d640 1 --2- 172.21.15.84:0/3604909994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fa6991b30 0x557fa6a0b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.918+0000 7f7df4d2b640 1 -- 172.21.15.84:0/3604909994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fa6991b30 msgr2=0x557fa6a0b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:35.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.918+0000 7f7df4d2b640 1 --2- 172.21.15.84:0/3604909994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fa6991b30 0x557fa6a0b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.918+0000 7f7df4d2b640 1 -- 172.21.15.84:0/3604909994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fa6986db0 msgr2=0x557fa6a00590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:11:35.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.918+0000 7f7df4d2b640 1 --2- 172.21.15.84:0/3604909994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fa6986db0 0x557fa6a00590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.918+0000 7f7df4d2b640 1 -- 172.21.15.84:0/3604909994 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fa68a9fe0 con 0x557fa69884a0 2024-05-22T23:11:35.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.918+0000 7f7df552c640 1 --2- 172.21.15.84:0/3604909994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fa6986db0 0x557fa6a00590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:11:35.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.918+0000 7f7df4d2b640 1 --2- 172.21.15.84:0/3604909994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa69884a0 0x557fa6a00ad0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009900 tx=0x7f7de0009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97663f3f15edec0d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:35.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.918+0000 7f7dd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:11:35.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.918+0000 7f7dd8ff9640 1 -- 172.21.15.84:0/3604909994 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0061030 con 0x557fa69884a0 2024-05-22T23:11:35.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.918+0000 7f7dd8ff9640 1 -- 172.21.15.84:0/3604909994 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8571+0+0 (secure 0 0 0) 0x7f7de0003b20 con 0x557fa69884a0 2024-05-22T23:11:35.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.918+0000 7f7dfa579740 1 -- 172.21.15.84:0/3604909994 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557fa68f2830 con 0x557fa69884a0 2024-05-22T23:11:35.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.918+0000 7f7dd8ff9640 1 -- 172.21.15.84:0/3604909994 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f7de004d040 con 0x557fa69884a0 2024-05-22T23:11:35.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.918+0000 7f7dfa579740 10 client.17010 fetch_fsmap learned FSMap version 18 2024-05-22T23:11:35.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.918+0000 7f7dfa579740 1 -- 172.21.15.84:0/3604909994 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x557fa687b1c0 con 0x557fa69884a0 2024-05-22T23:11:35.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.919+0000 7f7dd8ff9640 1 -- 172.21.15.84:0/3604909994 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557fa68f2830 con 0x557fa69884a0 2024-05-22T23:11:35.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.919+0000 7f7dd8ff9640 1 -- 172.21.15.84:0/3604909994 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f7de0048040 con 0x557fa69884a0 2024-05-22T23:11:35.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.919+0000 7f7dfa579740 10 client.17010 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:11:35.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.919+0000 7f7dfa579740 10 client.17010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:11:35.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.919+0000 7f7dfa579740 20 client.17010 populate_metadata read hostname 'smithi084' 2024-05-22T23:11:35.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.919+0000 7f7dfa579740 1 --2- 172.21.15.84:0/3604909994 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x557fa6a0ceb0 0x557fa6a0f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:35.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.919+0000 7f7dfa579740 4 client.17010 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:11:35.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.919+0000 7f7dfa579740 1 -- 172.21.15.84:0/3604909994 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557fa686f6f0 con 0x557fa6a0ceb0 2024-05-22T23:11:35.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.919+0000 7f7df552c640 1 --2- 172.21.15.84:0/3604909994 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x557fa6a0ceb0 0x557fa6a0f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:35.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.919+0000 7f7dd8ff9640 10 client.17010 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:35.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.919+0000 7f7df552c640 1 --2- 172.21.15.84:0/3604909994 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x557fa6a0ceb0 0x557fa6a0f2d0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9801e4e0e8d679a server_cookie=19a5c070bb63cfa0 in_seq=0 out_seq=0 2024-05-22T23:11:35.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.920+0000 7f7dd8ff9640 1 -- 172.21.15.84:0/3604909994 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557fa686f6f0 con 0x557fa6a0ceb0 2024-05-22T23:11:35.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.920+0000 7f7dd8ff9640 10 client.17010 handle_command_reply: tid=0 2024-05-22T23:11:35.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.920+0000 7f7dfa579740 1 client.17010 shutdown 2024-05-22T23:11:35.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.920+0000 7f7dfa579740 1 -- 172.21.15.84:0/3604909994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa69884a0 msgr2=0x557fa6a00ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:35.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.920+0000 7f7dfa579740 1 --2- 172.21.15.84:0/3604909994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa69884a0 0x557fa6a00ad0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009900 tx=0x7f7de0009680 comp rx=0 tx=0).stop 2024-05-22T23:11:35.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.920+0000 7f7dfa579740 1 -- 172.21.15.84:0/3604909994 shutdown_connections 2024-05-22T23:11:35.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.920+0000 7f7dfa579740 1 --2- 172.21.15.84:0/3604909994 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x557fa6a0ceb0 0x557fa6a0f2d0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.920+0000 7f7dfa579740 1 --2- 172.21.15.84:0/3604909994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fa6991b30 0x557fa6a0b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.920+0000 7f7dd8ff9640 0 client.17010 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:11:35.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.920+0000 7f7dfa579740 1 --2- 172.21.15.84:0/3604909994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fa69884a0 0x557fa6a00ad0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.920+0000 7f7dfa579740 1 --2- 172.21.15.84:0/3604909994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fa6986db0 0x557fa6a00590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.920+0000 7f7dfa579740 1 -- 172.21.15.84:0/3604909994 >> 172.21.15.84:0/3604909994 conn(0x557fa694f8e0 msgr2=0x557fa698ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:35.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.920+0000 7f7dfa579740 1 -- 172.21.15.84:0/3604909994 shutdown_connections 2024-05-22T23:11:35.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.920+0000 7f7dfa579740 1 -- 172.21.15.84:0/3604909994 wait complete. 2024-05-22T23:11:35.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.920+0000 7f7dfa579740 20 client.17010 trim_cache size 0 max 16384 2024-05-22T23:11:35.940 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:11:35.941 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-22T23:11:35.941 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": { 2024-05-22T23:11:35.941 INFO:teuthology.orchestra.run.smithi084.stdout: "c5a4cea4-dbcc-4f69-bbba-ca301a39133c": { 2024-05-22T23:11:35.941 INFO:teuthology.orchestra.run.smithi084.stdout: "path": "/", 2024-05-22T23:11:35.941 INFO:teuthology.orchestra.run.smithi084.stdout: "tag": "c5a4cea4-dbcc-4f69-bbba-ca301a39133c", 2024-05-22T23:11:35.941 INFO:teuthology.orchestra.run.smithi084.stdout: "options": "recursive,force" 2024-05-22T23:11:35.941 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:11:35.942 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:11:35.942 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:11:35.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.920+0000 7f7df7790640 1 -- 172.21.15.84:0/3681435312 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f7ddc0a2d00 msgr2=0x7f7ddc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:35.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.920+0000 7f7df7790640 1 --2- 172.21.15.84:0/3681435312 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f7ddc0a2d00 0x7f7ddc0a51c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7de0006010 tx=0x7f7de0032d10 comp rx=0 tx=0).stop 2024-05-22T23:11:35.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.920+0000 7f7df7790640 1 -- 172.21.15.84:0/3681435312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7df0199a70 msgr2=0x7f7df01a4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:35.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.920+0000 7f7df7790640 1 --2- 172.21.15.84:0/3681435312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7df0199a70 0x7f7df01a4020 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7de800d750 tx=0x7f7de800dc10 comp rx=0 tx=0).stop 2024-05-22T23:11:35.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.920+0000 7f7df7790640 1 -- 172.21.15.84:0/3681435312 shutdown_connections 2024-05-22T23:11:35.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.920+0000 7f7df7790640 1 --2- 172.21.15.84:0/3681435312 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f7ddc0a2d00 0x7f7ddc0a51c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.921+0000 7f7df7790640 1 --2- 172.21.15.84:0/3681435312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7df0199a70 0x7f7df01a4020 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.921+0000 7f7df7790640 1 --2- 172.21.15.84:0/3681435312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7df0105290 0x7f7df01994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.921+0000 7f7df7790640 1 --2- 172.21.15.84:0/3681435312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7df0100410 0x7f7df0198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:35.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.921+0000 7f7df7790640 1 -- 172.21.15.84:0/3681435312 >> 172.21.15.84:0/3681435312 conn(0x7f7df0075a00 msgr2=0x7f7df00ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:35.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.921+0000 7f7df7790640 1 -- 172.21.15.84:0/3681435312 shutdown_connections 2024-05-22T23:11:35.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:35.921+0000 7f7df7790640 1 -- 172.21.15.84:0/3681435312 wait complete. 2024-05-22T23:11:35.945 INFO:tasks.cephfs.filesystem:scrub status for tag:c5a4cea4-dbcc-4f69-bbba-ca301a39133c - {'path': '/', 'tag': 'c5a4cea4-dbcc-4f69-bbba-ca301a39133c', 'options': 'recursive,force'} 2024-05-22T23:11:36.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:36 smithi084 ceph-mon[32186]: scrub queued for path: / 2024-05-22T23:11:36.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:36 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:11:36.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:36 smithi084 ceph-mon[32186]: scrub summary: active paths [/] 2024-05-22T23:11:36.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:36 smithi084 ceph-mon[32186]: scrub summary: idle+waiting paths [/] 2024-05-22T23:11:36.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:36 smithi084 ceph-mon[32186]: scrub summary: idle 2024-05-22T23:11:36.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:36 smithi204 ceph-mon[36211]: scrub queued for path: / 2024-05-22T23:11:36.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:36 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:11:36.667 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:36 smithi204 ceph-mon[36211]: scrub summary: active paths [/] 2024-05-22T23:11:36.667 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:36 smithi204 ceph-mon[36211]: scrub summary: idle+waiting paths [/] 2024-05-22T23:11:36.667 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:36 smithi204 ceph-mon[36211]: scrub summary: idle 2024-05-22T23:11:36.805 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:36 smithi096 ceph-mon[36392]: scrub queued for path: / 2024-05-22T23:11:36.805 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:36 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:11:36.805 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:36 smithi096 ceph-mon[36392]: scrub summary: active paths [/] 2024-05-22T23:11:36.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:36 smithi096 ceph-mon[36392]: scrub summary: idle+waiting paths [/] 2024-05-22T23:11:36.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:36 smithi096 ceph-mon[36392]: scrub summary: idle 2024-05-22T23:11:37.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:37 smithi084 ceph-mon[32186]: pgmap v304: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4 op/s 2024-05-22T23:11:37.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:37 smithi204 ceph-mon[36211]: pgmap v304: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4 op/s 2024-05-22T23:11:37.805 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:37 smithi096 ceph-mon[36392]: pgmap v304: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4 op/s 2024-05-22T23:11:39.642 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:39 smithi084 ceph-mon[32186]: pgmap v305: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3 op/s 2024-05-22T23:11:39.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:39 smithi204 ceph-mon[36211]: pgmap v305: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3 op/s 2024-05-22T23:11:39.805 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:39 smithi096 ceph-mon[36392]: pgmap v305: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3 op/s 2024-05-22T23:11:41.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:41 smithi084 ceph-mon[32186]: pgmap v306: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3 op/s 2024-05-22T23:11:41.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:41 smithi204 ceph-mon[36211]: pgmap v306: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3 op/s 2024-05-22T23:11:41.805 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:41 smithi096 ceph-mon[36392]: pgmap v306: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3 op/s 2024-05-22T23:11:43.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:43 smithi084 ceph-mon[32186]: pgmap v307: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4 op/s 2024-05-22T23:11:43.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:43 smithi204 ceph-mon[36211]: pgmap v307: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4 op/s 2024-05-22T23:11:43.805 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:43 smithi096 ceph-mon[36392]: pgmap v307: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4 op/s 2024-05-22T23:11:45.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:45 smithi084 ceph-mon[32186]: pgmap v308: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3 op/s 2024-05-22T23:11:45.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:45 smithi204 ceph-mon[36211]: pgmap v308: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3 op/s 2024-05-22T23:11:45.805 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:45 smithi096 ceph-mon[36392]: pgmap v308: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3 op/s 2024-05-22T23:11:47.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:47 smithi084 ceph-mon[32186]: pgmap v309: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4 op/s 2024-05-22T23:11:47.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:47 smithi204 ceph-mon[36211]: pgmap v309: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4 op/s 2024-05-22T23:11:47.805 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:47 smithi096 ceph-mon[36392]: pgmap v309: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4 op/s 2024-05-22T23:11:49.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:49 smithi084 ceph-mon[32186]: pgmap v310: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:11:49.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:49 smithi204 ceph-mon[36211]: pgmap v310: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:11:49.805 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:49 smithi096 ceph-mon[36392]: pgmap v310: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:11:51.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:51 smithi084 ceph-mon[32186]: pgmap v311: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:11:51.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:51 smithi204 ceph-mon[36211]: pgmap v311: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:11:51.805 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:51 smithi096 ceph-mon[36392]: pgmap v311: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:11:53.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:53 smithi084 ceph-mon[32186]: pgmap v312: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:11:53.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:53 smithi204 ceph-mon[36211]: pgmap v312: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:11:53.805 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:53 smithi096 ceph-mon[36392]: pgmap v312: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:11:55.607 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:55 smithi096 ceph-mon[36392]: pgmap v313: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:11:55.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:55 smithi084 ceph-mon[32186]: pgmap v313: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:11:55.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:55 smithi204 ceph-mon[36211]: pgmap v313: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:11:56.511 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-22T23:11:56.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.615+0000 7f477d0a7640 1 -- 172.21.15.84:0/2798545166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f477806aa60 msgr2=0x7f477810e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:56.624 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.615+0000 7f477d0a7640 1 --2- 172.21.15.84:0/2798545166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f477806aa60 0x7f477810e710 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f4770009900 tx=0x7f477002a480 comp rx=0 tx=0).stop 2024-05-22T23:11:56.624 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.615+0000 7f477d0a7640 1 -- 172.21.15.84:0/2798545166 shutdown_connections 2024-05-22T23:11:56.624 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.615+0000 7f477d0a7640 1 --2- 172.21.15.84:0/2798545166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f477806aa60 0x7f477810e710 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:56.624 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.615+0000 7f477d0a7640 1 --2- 172.21.15.84:0/2798545166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f477806a0a0 0x7f477806a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:56.624 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.615+0000 7f477d0a7640 1 --2- 172.21.15.84:0/2798545166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4778107a70 0x7f4778107e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:56.624 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.615+0000 7f477d0a7640 1 -- 172.21.15.84:0/2798545166 >> 172.21.15.84:0/2798545166 conn(0x7f4778076570 msgr2=0x7f4778076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:56.624 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.616+0000 7f477d0a7640 1 -- 172.21.15.84:0/2798545166 shutdown_connections 2024-05-22T23:11:56.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.616+0000 7f477d0a7640 1 -- 172.21.15.84:0/2798545166 wait complete. 2024-05-22T23:11:56.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.616+0000 7f477d0a7640 1 Processor -- start 2024-05-22T23:11:56.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.616+0000 7f477d0a7640 1 -- start start 2024-05-22T23:11:56.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.617+0000 7f477d0a7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f477806a0a0 0x7f4778100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:56.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.617+0000 7f477d0a7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f477806aa60 0x7f47781010d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:56.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.617+0000 7f477d0a7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4778107a70 0x7f4778101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:56.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.617+0000 7f477d0a7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4778111c90 con 0x7f4778107a70 2024-05-22T23:11:56.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.617+0000 7f477d0a7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4778111b10 con 0x7f477806aa60 2024-05-22T23:11:56.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.617+0000 7f477d0a7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4778111e10 con 0x7f477806a0a0 2024-05-22T23:11:56.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.617+0000 7f477759e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4778107a70 0x7f4778101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:56.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.617+0000 7f477759e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4778107a70 0x7f4778101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40234/0 (socket says 172.21.15.84:40234) 2024-05-22T23:11:56.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.617+0000 7f477759e640 1 -- 172.21.15.84:0/1740237994 learned_addr learned my addr 172.21.15.84:0/1740237994 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:56.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.617+0000 7f477659c640 1 --2- 172.21.15.84:0/1740237994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f477806aa60 0x7f47781010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:56.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.617+0000 7f477759e640 1 -- 172.21.15.84:0/1740237994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f477806a0a0 msgr2=0x7f4778100b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:56.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.617+0000 7f477759e640 1 --2- 172.21.15.84:0/1740237994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f477806a0a0 0x7f4778100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:56.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.617+0000 7f477759e640 1 -- 172.21.15.84:0/1740237994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f477806aa60 msgr2=0x7f47781010d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:56.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.617+0000 7f477759e640 1 --2- 172.21.15.84:0/1740237994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f477806aa60 0x7f47781010d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:56.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.617+0000 7f477759e640 1 -- 172.21.15.84:0/1740237994 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47781a6510 con 0x7f4778107a70 2024-05-22T23:11:56.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.617+0000 7f477759e640 1 --2- 172.21.15.84:0/1740237994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4778107a70 0x7f4778101610 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f4770005ab0 tx=0x7f4770003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:56.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.617+0000 7f475ffff640 1 -- 172.21.15.84:0/1740237994 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477003f070 con 0x7f4778107a70 2024-05-22T23:11:56.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.617+0000 7f477d0a7640 1 -- 172.21.15.84:0/1740237994 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47781a67a0 con 0x7f4778107a70 2024-05-22T23:11:56.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.617+0000 7f475ffff640 1 -- 172.21.15.84:0/1740237994 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4770031e30 con 0x7f4778107a70 2024-05-22T23:11:56.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.617+0000 7f475ffff640 1 -- 172.21.15.84:0/1740237994 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770035030 con 0x7f4778107a70 2024-05-22T23:11:56.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.618+0000 7f477d0a7640 1 -- 172.21.15.84:0/1740237994 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47781a6c90 con 0x7f4778107a70 2024-05-22T23:11:56.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.619+0000 7f477d0a7640 1 -- 172.21.15.84:0/1740237994 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4778102f80 con 0x7f4778107a70 2024-05-22T23:11:56.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.620+0000 7f475ffff640 1 -- 172.21.15.84:0/1740237994 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4770042070 con 0x7f4778107a70 2024-05-22T23:11:56.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.620+0000 7f475ffff640 1 --2- 172.21.15.84:0/1740237994 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f47600a2da0 0x7f47600a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:56.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.620+0000 7f475ffff640 1 -- 172.21.15.84:0/1740237994 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8571+0+0 (secure 0 0 0) 0x7f47700efdc0 con 0x7f4778107a70 2024-05-22T23:11:56.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.620+0000 7f4776d9d640 1 --2- 172.21.15.84:0/1740237994 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f47600a2da0 0x7f47600a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:56.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.621+0000 7f4776d9d640 1 --2- 172.21.15.84:0/1740237994 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f47600a2da0 0x7f47600a5260 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4768007be0 tx=0x7f4768005f70 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:56.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.625+0000 7f475ffff640 1 -- 172.21.15.84:0/1740237994 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4770003e20 con 0x7f4778107a70 2024-05-22T23:11:56.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.822+0000 7f477d0a7640 1 -- 172.21.15.84:0/1740237994 --> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f47780ffb90 con 0x7f47600a2da0 2024-05-22T23:11:56.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.831+0000 7f475ffff640 1 -- 172.21.15.84:0/1740237994 <== mgr.24851 v2:172.21.15.84:6800/3358259585 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f47780ffb90 con 0x7f47600a2da0 2024-05-22T23:11:56.839 INFO:teuthology.orchestra.run.smithi084.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-22T00:00:00", "created": "2024-05-22T23:01:55", "first": "2024-05-22T23:02:00", "last": "2024-05-22T23:11:00", "last_pruned": "2024-05-22T23:11:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-05-22T23:11:56.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.833+0000 7f477d0a7640 1 -- 172.21.15.84:0/1740237994 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f47600a2da0 msgr2=0x7f47600a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:56.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.833+0000 7f477d0a7640 1 --2- 172.21.15.84:0/1740237994 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f47600a2da0 0x7f47600a5260 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4768007be0 tx=0x7f4768005f70 comp rx=0 tx=0).stop 2024-05-22T23:11:56.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.833+0000 7f477d0a7640 1 -- 172.21.15.84:0/1740237994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4778107a70 msgr2=0x7f4778101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:56.842 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.833+0000 7f477d0a7640 1 --2- 172.21.15.84:0/1740237994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4778107a70 0x7f4778101610 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f4770005ab0 tx=0x7f4770003c30 comp rx=0 tx=0).stop 2024-05-22T23:11:56.842 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.833+0000 7f477d0a7640 1 -- 172.21.15.84:0/1740237994 shutdown_connections 2024-05-22T23:11:56.842 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.833+0000 7f477d0a7640 1 --2- 172.21.15.84:0/1740237994 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f47600a2da0 0x7f47600a5260 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:56.842 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.833+0000 7f477d0a7640 1 --2- 172.21.15.84:0/1740237994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4778107a70 0x7f4778101610 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:56.842 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.833+0000 7f477d0a7640 1 --2- 172.21.15.84:0/1740237994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f477806aa60 0x7f47781010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:56.842 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.833+0000 7f477d0a7640 1 --2- 172.21.15.84:0/1740237994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f477806a0a0 0x7f4778100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:56.842 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.833+0000 7f477d0a7640 1 -- 172.21.15.84:0/1740237994 >> 172.21.15.84:0/1740237994 conn(0x7f4778076570 msgr2=0x7f47780ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:56.842 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.833+0000 7f477d0a7640 1 -- 172.21.15.84:0/1740237994 shutdown_connections 2024-05-22T23:11:56.843 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.833+0000 7f477d0a7640 1 -- 172.21.15.84:0/1740237994 wait complete. 2024-05-22T23:11:56.852 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-22T23:11:56.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.947+0000 7fd1dc076640 1 -- 172.21.15.84:0/1121268208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1d4103460 msgr2=0x7fd1d410b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:56.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.947+0000 7fd1dc076640 1 --2- 172.21.15.84:0/1121268208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1d4103460 0x7fd1d410b940 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009900 tx=0x7fd1c802a480 comp rx=0 tx=0).stop 2024-05-22T23:11:56.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.948+0000 7fd1dc076640 1 -- 172.21.15.84:0/1121268208 shutdown_connections 2024-05-22T23:11:56.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.948+0000 7fd1dc076640 1 --2- 172.21.15.84:0/1121268208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d410be80 0x7fd1d410e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:56.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.948+0000 7fd1dc076640 1 --2- 172.21.15.84:0/1121268208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1d4103460 0x7fd1d410b940 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:56.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.948+0000 7fd1dc076640 1 --2- 172.21.15.84:0/1121268208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1d4102b40 0x7fd1d4102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:56.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.948+0000 7fd1dc076640 1 -- 172.21.15.84:0/1121268208 >> 172.21.15.84:0/1121268208 conn(0x7fd1d40778f0 msgr2=0x7fd1d4077d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:56.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.948+0000 7fd1dc076640 1 -- 172.21.15.84:0/1121268208 shutdown_connections 2024-05-22T23:11:56.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.948+0000 7fd1dc076640 1 -- 172.21.15.84:0/1121268208 wait complete. 2024-05-22T23:11:56.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.948+0000 7fd1dc076640 1 Processor -- start 2024-05-22T23:11:56.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.948+0000 7fd1dc076640 1 -- start start 2024-05-22T23:11:56.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.948+0000 7fd1dc076640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d4102b40 0x7fd1d4198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:56.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.948+0000 7fd1dc076640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1d4103460 0x7fd1d41986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:56.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.948+0000 7fd1dc076640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1d410be80 0x7fd1d419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:56.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.948+0000 7fd1dc076640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d4111c30 con 0x7fd1d4103460 2024-05-22T23:11:56.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.948+0000 7fd1dc076640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d4111ab0 con 0x7fd1d4102b40 2024-05-22T23:11:56.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.948+0000 7fd1dc076640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d4111db0 con 0x7fd1d410be80 2024-05-22T23:11:56.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.949+0000 7fd1d9611640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1d4103460 0x7fd1d41986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:56.959 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.949+0000 7fd1d9611640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1d4103460 0x7fd1d41986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40258/0 (socket says 172.21.15.84:40258) 2024-05-22T23:11:56.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.949+0000 7fd1d9611640 1 -- 172.21.15.84:0/3902583834 learned_addr learned my addr 172.21.15.84:0/3902583834 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:11:56.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.949+0000 7fd1d9e12640 1 --2- 172.21.15.84:0/3902583834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d4102b40 0x7fd1d4198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:56.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.949+0000 7fd1da613640 1 --2- 172.21.15.84:0/3902583834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1d410be80 0x7fd1d419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:56.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.949+0000 7fd1d9611640 1 -- 172.21.15.84:0/3902583834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1d410be80 msgr2=0x7fd1d419ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:56.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.949+0000 7fd1d9611640 1 --2- 172.21.15.84:0/3902583834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1d410be80 0x7fd1d419ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:56.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.949+0000 7fd1d9611640 1 -- 172.21.15.84:0/3902583834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d4102b40 msgr2=0x7fd1d4198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:56.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.949+0000 7fd1d9611640 1 --2- 172.21.15.84:0/3902583834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d4102b40 0x7fd1d4198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:56.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.949+0000 7fd1d9611640 1 -- 172.21.15.84:0/3902583834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d419d160 con 0x7fd1d4103460 2024-05-22T23:11:56.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.949+0000 7fd1da613640 1 --2- 172.21.15.84:0/3902583834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1d410be80 0x7fd1d419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:11:56.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.949+0000 7fd1d9611640 1 --2- 172.21.15.84:0/3902583834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1d4103460 0x7fd1d41986b0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8006010 tx=0x7fd1c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:56.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.949+0000 7fd1d2ffd640 1 -- 172.21.15.84:0/3902583834 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c803f030 con 0x7fd1d4103460 2024-05-22T23:11:56.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.949+0000 7fd1d2ffd640 1 -- 172.21.15.84:0/3902583834 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1c8003ce0 con 0x7fd1d4103460 2024-05-22T23:11:56.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.949+0000 7fd1d2ffd640 1 -- 172.21.15.84:0/3902583834 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8035070 con 0x7fd1d4103460 2024-05-22T23:11:56.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.949+0000 7fd1dc076640 1 -- 172.21.15.84:0/3902583834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d419d3f0 con 0x7fd1d4103460 2024-05-22T23:11:56.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.949+0000 7fd1dc076640 1 -- 172.21.15.84:0/3902583834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d41a4ce0 con 0x7fd1d4103460 2024-05-22T23:11:56.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.950+0000 7fd1dc076640 1 -- 172.21.15.84:0/3902583834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1d404f630 con 0x7fd1d4103460 2024-05-22T23:11:56.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.951+0000 7fd1d2ffd640 1 -- 172.21.15.84:0/3902583834 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd1c8042030 con 0x7fd1d4103460 2024-05-22T23:11:56.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.952+0000 7fd1d2ffd640 1 --2- 172.21.15.84:0/3902583834 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd1bc0a2e20 0x7fd1bc0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:11:56.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.952+0000 7fd1d9e12640 1 --2- 172.21.15.84:0/3902583834 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd1bc0a2e20 0x7fd1bc0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:11:56.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.952+0000 7fd1d2ffd640 1 -- 172.21.15.84:0/3902583834 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8571+0+0 (secure 0 0 0) 0x7fd1c80ef9c0 con 0x7fd1d4103460 2024-05-22T23:11:56.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.953+0000 7fd1d9e12640 1 --2- 172.21.15.84:0/3902583834 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd1bc0a2e20 0x7fd1bc0a52e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4005fd0 tx=0x7fd1c4009680 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:11:56.964 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:56.957+0000 7fd1d2ffd640 1 -- 172.21.15.84:0/3902583834 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd1c80f1e20 con 0x7fd1d4103460 2024-05-22T23:11:57.163 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:57.155+0000 7fd1dc076640 1 -- 172.21.15.84:0/3902583834 --> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fd1d4061cb0 con 0x7fd1bc0a2e20 2024-05-22T23:11:57.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:57.163+0000 7fd1d2ffd640 1 -- 172.21.15.84:0/3902583834 <== mgr.24851 v2:172.21.15.84:6800/3358259585 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fd1d4061cb0 con 0x7fd1bc0a2e20 2024-05-22T23:11:57.171 INFO:teuthology.orchestra.run.smithi084.stdout:/ 1m 6m3h 2024-05-22T23:11:57.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:57.165+0000 7fd1dc076640 1 -- 172.21.15.84:0/3902583834 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd1bc0a2e20 msgr2=0x7fd1bc0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:57.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:57.165+0000 7fd1dc076640 1 --2- 172.21.15.84:0/3902583834 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd1bc0a2e20 0x7fd1bc0a52e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4005fd0 tx=0x7fd1c4009680 comp rx=0 tx=0).stop 2024-05-22T23:11:57.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:57.165+0000 7fd1dc076640 1 -- 172.21.15.84:0/3902583834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1d4103460 msgr2=0x7fd1d41986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:11:57.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:57.165+0000 7fd1dc076640 1 --2- 172.21.15.84:0/3902583834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1d4103460 0x7fd1d41986b0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8006010 tx=0x7fd1c80042f0 comp rx=0 tx=0).stop 2024-05-22T23:11:57.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:57.165+0000 7fd1dc076640 1 -- 172.21.15.84:0/3902583834 shutdown_connections 2024-05-22T23:11:57.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:57.165+0000 7fd1dc076640 1 --2- 172.21.15.84:0/3902583834 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd1bc0a2e20 0x7fd1bc0a52e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:57.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:57.165+0000 7fd1dc076640 1 --2- 172.21.15.84:0/3902583834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1d410be80 0x7fd1d419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:57.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:57.165+0000 7fd1dc076640 1 --2- 172.21.15.84:0/3902583834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1d4103460 0x7fd1d41986b0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:57.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:57.165+0000 7fd1dc076640 1 --2- 172.21.15.84:0/3902583834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d4102b40 0x7fd1d4198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:11:57.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:57.165+0000 7fd1dc076640 1 -- 172.21.15.84:0/3902583834 >> 172.21.15.84:0/3902583834 conn(0x7fd1d40778f0 msgr2=0x7fd1d40fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:11:57.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:57.165+0000 7fd1dc076640 1 -- 172.21.15.84:0/3902583834 shutdown_connections 2024-05-22T23:11:57.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:11:57.165+0000 7fd1dc076640 1 -- 172.21.15.84:0/3902583834 wait complete. 2024-05-22T23:11:57.184 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-22T23:11:57.194 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-22T23:11:57.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:57 smithi084 ceph-mon[32186]: pgmap v314: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:11:57.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:57 smithi204 ceph-mon[36211]: pgmap v314: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:11:57.805 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:57 smithi096 ceph-mon[36392]: pgmap v314: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:11:58.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:58 smithi084 ceph-mon[32186]: from='client.17016 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:11:58.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:58 smithi084 ceph-mon[32186]: from='client.17022 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:11:58.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:58 smithi204 ceph-mon[36211]: from='client.17016 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:11:58.667 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:58 smithi204 ceph-mon[36211]: from='client.17022 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:11:58.805 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:58 smithi096 ceph-mon[36392]: from='client.17016 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:11:58.805 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:58 smithi096 ceph-mon[36392]: from='client.17022 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-22T23:11:59.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:11:59 smithi084 ceph-mon[32186]: pgmap v315: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:11:59.667 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:11:59 smithi204 ceph-mon[36211]: pgmap v315: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:11:59.805 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:11:59 smithi096 ceph-mon[36392]: pgmap v315: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:12:01.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:01 smithi084 ceph-mon[32186]: pgmap v316: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:12:01.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:01 smithi204 ceph-mon[36211]: pgmap v316: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:12:01.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:01 smithi096 ceph-mon[36392]: pgmap v316: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:12:03.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:03 smithi084 ceph-mon[32186]: pgmap v317: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-22T23:12:03.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:03 smithi204 ceph-mon[36211]: pgmap v317: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-22T23:12:03.805 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:03 smithi096 ceph-mon[36392]: pgmap v317: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-22T23:12:04.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:04 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:12:04.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:04 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:12:04.805 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:04 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-22T23:12:05.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:05 smithi096 ceph-mon[36392]: pgmap v318: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 21 op/s 2024-05-22T23:12:05.630 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:05 smithi084 ceph-mon[32186]: pgmap v318: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 21 op/s 2024-05-22T23:12:05.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:05 smithi204 ceph-mon[36211]: pgmap v318: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 21 op/s 2024-05-22T23:12:05.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-22T23:12:05.946 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-22T23:12:06.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.048+0000 7fdca7e64640 1 -- 172.21.15.84:0/4220331319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdca0107850 msgr2=0x7fdca0107c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:06.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.048+0000 7fdca7e64640 1 --2- 172.21.15.84:0/4220331319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdca0107850 0x7fdca0107c30 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009930 tx=0x7fdc9c02a420 comp rx=0 tx=0).stop 2024-05-22T23:12:06.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.048+0000 7fdca7e64640 1 -- 172.21.15.84:0/4220331319 shutdown_connections 2024-05-22T23:12:06.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.048+0000 7fdca7e64640 1 --2- 172.21.15.84:0/4220331319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdca0102200 0x7fdca010e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.048+0000 7fdca7e64640 1 --2- 172.21.15.84:0/4220331319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdca0101840 0x7fdca0101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.048+0000 7fdca7e64640 1 --2- 172.21.15.84:0/4220331319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdca0107850 0x7fdca0107c30 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.048+0000 7fdca7e64640 1 -- 172.21.15.84:0/4220331319 >> 172.21.15.84:0/4220331319 conn(0x7fdca0078cc0 msgr2=0x7fdca00790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:06.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.048+0000 7fdca7e64640 1 -- 172.21.15.84:0/4220331319 shutdown_connections 2024-05-22T23:12:06.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.048+0000 7fdca7e64640 1 -- 172.21.15.84:0/4220331319 wait complete. 2024-05-22T23:12:06.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.049+0000 7fdca7e64640 1 Processor -- start 2024-05-22T23:12:06.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.049+0000 7fdca7e64640 1 -- start start 2024-05-22T23:12:06.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.049+0000 7fdca7e64640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdca0101840 0x7fdca01981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.049+0000 7fdca7e64640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdca0102200 0x7fdca01986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.049+0000 7fdca7e64640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdca0107850 0x7fdca019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.049+0000 7fdca7e64640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdca0111c60 con 0x7fdca0107850 2024-05-22T23:12:06.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.049+0000 7fdca7e64640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdca0111ae0 con 0x7fdca0101840 2024-05-22T23:12:06.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.049+0000 7fdca7e64640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdca0111de0 con 0x7fdca0102200 2024-05-22T23:12:06.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.049+0000 7fdca6401640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdca0107850 0x7fdca019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.049+0000 7fdca6401640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdca0107850 0x7fdca019ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49118/0 (socket says 172.21.15.84:49118) 2024-05-22T23:12:06.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.049+0000 7fdca6401640 1 -- 172.21.15.84:0/1128606544 learned_addr learned my addr 172.21.15.84:0/1128606544 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:06.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.049+0000 7fdca53ff640 1 --2- 172.21.15.84:0/1128606544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdca0102200 0x7fdca01986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.049+0000 7fdca5c00640 1 --2- 172.21.15.84:0/1128606544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdca0101840 0x7fdca01981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.049+0000 7fdca6401640 1 -- 172.21.15.84:0/1128606544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdca0102200 msgr2=0x7fdca01986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:06.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.049+0000 7fdca6401640 1 --2- 172.21.15.84:0/1128606544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdca0102200 0x7fdca01986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.049+0000 7fdca6401640 1 -- 172.21.15.84:0/1128606544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdca0101840 msgr2=0x7fdca01981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:06.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.049+0000 7fdca6401640 1 --2- 172.21.15.84:0/1128606544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdca0101840 0x7fdca01981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.049+0000 7fdca6401640 1 -- 172.21.15.84:0/1128606544 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca019d190 con 0x7fdca0107850 2024-05-22T23:12:06.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.049+0000 7fdca5c00640 1 --2- 172.21.15.84:0/1128606544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdca0101840 0x7fdca01981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:12:06.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.050+0000 7fdca6401640 1 --2- 172.21.15.84:0/1128606544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdca0107850 0x7fdca019ca60 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800eaa0 tx=0x7fdc9800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:06.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.050+0000 7fdc96ffd640 1 -- 172.21.15.84:0/1128606544 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9800cdb0 con 0x7fdca0107850 2024-05-22T23:12:06.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.050+0000 7fdca7e64640 1 -- 172.21.15.84:0/1128606544 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca019d480 con 0x7fdca0107850 2024-05-22T23:12:06.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.050+0000 7fdc96ffd640 1 -- 172.21.15.84:0/1128606544 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc98005020 con 0x7fdca0107850 2024-05-22T23:12:06.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.050+0000 7fdc96ffd640 1 -- 172.21.15.84:0/1128606544 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc980116b0 con 0x7fdca0107850 2024-05-22T23:12:06.061 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.050+0000 7fdca7e64640 1 -- 172.21.15.84:0/1128606544 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca01a4d10 con 0x7fdca0107850 2024-05-22T23:12:06.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.052+0000 7fdc96ffd640 1 -- 172.21.15.84:0/1128606544 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdc98005560 con 0x7fdca0107850 2024-05-22T23:12:06.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.052+0000 7fdcaab73740 1 Processor -- start 2024-05-22T23:12:06.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.053+0000 7fdcaab73740 1 -- start start 2024-05-22T23:12:06.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.053+0000 7fdcaab73740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290848f4a0 0x56290848d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.053+0000 7fdcaab73740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56290848ddb0 0x56290848e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.053+0000 7fdcaab73740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562908498b30 0x56290849af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.053+0000 7fdcaab73740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5629083f1db0 con 0x56290848f4a0 2024-05-22T23:12:06.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.053+0000 7fdcaab73740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562908390360 con 0x56290848ddb0 2024-05-22T23:12:06.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.053+0000 7fdcaab73740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5629083edd70 con 0x562908498b30 2024-05-22T23:12:06.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.053+0000 7fdca53ff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290848f4a0 0x56290848d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.053+0000 7fdca53ff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290848f4a0 0x56290848d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49130/0 (socket says 172.21.15.84:49130) 2024-05-22T23:12:06.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.053+0000 7fdca5c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56290848ddb0 0x56290848e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.053+0000 7fdca53ff640 1 -- 172.21.15.84:0/231821159 learned_addr learned my addr 172.21.15.84:0/231821159 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:06.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.053+0000 7fdca53ff640 1 --2- 172.21.15.84:0/231821159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562908498b30 0x56290849af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.053+0000 7fdca5c00640 1 -- 172.21.15.84:0/231821159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562908498b30 msgr2=0x56290849af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:12:06.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.053+0000 7fdca5c00640 1 --2- 172.21.15.84:0/231821159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562908498b30 0x56290849af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.053+0000 7fdca5c00640 1 -- 172.21.15.84:0/231821159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290848f4a0 msgr2=0x56290848d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:12:06.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.053+0000 7fdca5c00640 1 --2- 172.21.15.84:0/231821159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290848f4a0 0x56290848d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.053+0000 7fdca5c00640 1 -- 172.21.15.84:0/231821159 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629083b3a60 con 0x56290848ddb0 2024-05-22T23:12:06.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.053+0000 7fdc96ffd640 1 --2- 172.21.15.84:0/1128606544 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fdc8c0a2d00 0x7fdc8c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.054+0000 7fdca53ff640 1 --2- 172.21.15.84:0/1128606544 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fdc8c0a2d00 0x7fdc8c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.054+0000 7fdca5c00640 1 --2- 172.21.15.84:0/231821159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56290848ddb0 0x56290848e230 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c005940 tx=0x7fdc9c005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b246ec57d398045 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:06.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.054+0000 7fdc96ffd640 1 -- 172.21.15.84:0/1128606544 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8571+0+0 (secure 0 0 0) 0x7fdc980cf5e0 con 0x7fdca0107850 2024-05-22T23:12:06.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.054+0000 7fdc8bfff640 1 -- 172.21.15.84:0/231821159 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c035040 con 0x56290848ddb0 2024-05-22T23:12:06.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.054+0000 7fdc8bfff640 1 -- 172.21.15.84:0/231821159 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc9c0318f0 con 0x56290848ddb0 2024-05-22T23:12:06.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.054+0000 7fdc8bfff640 1 -- 172.21.15.84:0/231821159 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c004a00 con 0x56290848ddb0 2024-05-22T23:12:06.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.055+0000 7fdcaab73740 1 -- 172.21.15.84:0/231821159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56290848ddb0 msgr2=0x56290848e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:06.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.055+0000 7fdca53ff640 1 --2- 172.21.15.84:0/1128606544 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fdc8c0a2d00 0x7fdc8c0a51c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x56290848b4a0 tx=0x7fdc900097a0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:06.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.055+0000 7fdcaab73740 1 --2- 172.21.15.84:0/231821159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56290848ddb0 0x56290848e230 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c005940 tx=0x7fdc9c005e20 comp rx=0 tx=0).stop 2024-05-22T23:12:06.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.055+0000 7fdcaab73740 1 -- 172.21.15.84:0/231821159 shutdown_connections 2024-05-22T23:12:06.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.055+0000 7fdcaab73740 1 --2- 172.21.15.84:0/231821159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562908498b30 0x56290849af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.055+0000 7fdcaab73740 1 --2- 172.21.15.84:0/231821159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56290848ddb0 0x56290848e230 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.055+0000 7fdcaab73740 1 --2- 172.21.15.84:0/231821159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290848f4a0 0x56290848d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.055+0000 7fdcaab73740 1 -- 172.21.15.84:0/231821159 >> 172.21.15.84:0/231821159 conn(0x5629084568e0 msgr2=0x56290848a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:06.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.055+0000 7fdcaab73740 1 -- 172.21.15.84:0/231821159 shutdown_connections 2024-05-22T23:12:06.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.055+0000 7fdcaab73740 1 -- 172.21.15.84:0/231821159 wait complete. 2024-05-22T23:12:06.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.055+0000 7fdcaab73740 1 -- start start 2024-05-22T23:12:06.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.055+0000 7fdcaab73740 1 Processor -- start 2024-05-22T23:12:06.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.055+0000 7fdcaab73740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56290848ddb0 0x5629084959e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.055+0000 7fdcaab73740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290848f4a0 0x562908495f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.055+0000 7fdcaab73740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562908498b30 0x562908496460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.055+0000 7fdcaab73740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56290849b960 con 0x56290848f4a0 2024-05-22T23:12:06.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.056+0000 7fdcaab73740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5629083ac0f0 con 0x56290848ddb0 2024-05-22T23:12:06.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.056+0000 7fdcaab73740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56290849bae0 con 0x562908498b30 2024-05-22T23:12:06.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.056+0000 7fdcaab73740 10 client.0 _ll_register_callbacks cb 0x5629083b0f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:12:06.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.056+0000 7fdca53ff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290848f4a0 0x562908495f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.056+0000 7fdca5c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56290848ddb0 0x5629084959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.056+0000 7fdca5c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56290848ddb0 0x5629084959e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:42496/0 (socket says 172.21.15.84:42496) 2024-05-22T23:12:06.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.056+0000 7fdca5c00640 1 -- 172.21.15.84:0/1134178773 learned_addr learned my addr 172.21.15.84:0/1134178773 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:06.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.056+0000 7fdca6401640 1 --2- 172.21.15.84:0/1134178773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562908498b30 0x562908496460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.056+0000 7fdca5c00640 1 -- 172.21.15.84:0/1134178773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562908498b30 msgr2=0x562908496460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:12:06.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.056+0000 7fdca5c00640 1 --2- 172.21.15.84:0/1134178773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562908498b30 0x562908496460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.056+0000 7fdca5c00640 1 -- 172.21.15.84:0/1134178773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290848f4a0 msgr2=0x562908495f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:12:06.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.056+0000 7fdca5c00640 1 --2- 172.21.15.84:0/1134178773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290848f4a0 0x562908495f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.056+0000 7fdca5c00640 1 -- 172.21.15.84:0/1134178773 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629081cf7d0 con 0x56290848ddb0 2024-05-22T23:12:06.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.056+0000 7fdca5c00640 1 --2- 172.21.15.84:0/1134178773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56290848ddb0 0x5629084959e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c031b50 tx=0x7fdc9c031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f2a1226c973f85d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:06.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.056+0000 7fdc897fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-22T23:12:06.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.056+0000 7fdc897fa640 1 -- 172.21.15.84:0/1134178773 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c035040 con 0x56290848ddb0 2024-05-22T23:12:06.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.056+0000 7fdc897fa640 1 -- 172.21.15.84:0/1134178773 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8571+0+0 (secure 0 0 0) 0x7fdc9c004250 con 0x56290848ddb0 2024-05-22T23:12:06.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.056+0000 7fdcaab73740 1 -- 172.21.15.84:0/1134178773 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc9c0318f0 con 0x56290848ddb0 2024-05-22T23:12:06.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.056+0000 7fdc897fa640 1 -- 172.21.15.84:0/1134178773 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fdc9c059050 con 0x56290848ddb0 2024-05-22T23:12:06.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.056+0000 7fdcaab73740 10 client.26420 fetch_fsmap learned FSMap version 18 2024-05-22T23:12:06.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.056+0000 7fdcaab73740 1 -- 172.21.15.84:0/1134178773 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5629081053f0 con 0x56290848ddb0 2024-05-22T23:12:06.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.057+0000 7fdc897fa640 1 -- 172.21.15.84:0/1134178773 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc9c0318f0 con 0x56290848ddb0 2024-05-22T23:12:06.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.057+0000 7fdc897fa640 1 -- 172.21.15.84:0/1134178773 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fdc9c046610 con 0x56290848ddb0 2024-05-22T23:12:06.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.057+0000 7fdcaab73740 10 client.26420 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:12:06.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.057+0000 7fdcaab73740 10 client.26420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:12:06.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.057+0000 7fdcaab73740 20 client.26420 populate_metadata read hostname 'smithi084' 2024-05-22T23:12:06.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.057+0000 7fdcaab73740 1 --2- 172.21.15.84:0/1134178773 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5629084a8d70 0x5629084ab160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.057+0000 7fdcaab73740 4 client.26420 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:12:06.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.057+0000 7fdcaab73740 1 -- 172.21.15.84:0/1134178773 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629083ff4e0 con 0x5629084a8d70 2024-05-22T23:12:06.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.057+0000 7fdca53ff640 1 --2- 172.21.15.84:0/1134178773 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5629084a8d70 0x5629084ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.057+0000 7fdc897fa640 10 client.26420 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:12:06.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.057+0000 7fdca53ff640 1 --2- 172.21.15.84:0/1134178773 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5629084a8d70 0x5629084ab160 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f9868d27fb4632f server_cookie=cf50001f5d338cbf in_seq=0 out_seq=0 2024-05-22T23:12:06.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.058+0000 7fdc897fa640 1 -- 172.21.15.84:0/1134178773 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629083ff4e0 con 0x5629084a8d70 2024-05-22T23:12:06.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.058+0000 7fdc897fa640 10 client.26420 handle_command_reply: tid=0 2024-05-22T23:12:06.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.058+0000 7fdcaab73740 1 client.26420 shutdown 2024-05-22T23:12:06.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.058+0000 7fdcaab73740 1 -- 172.21.15.84:0/1134178773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56290848ddb0 msgr2=0x5629084959e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:06.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.058+0000 7fdcaab73740 1 --2- 172.21.15.84:0/1134178773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56290848ddb0 0x5629084959e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c031b50 tx=0x7fdc9c031400 comp rx=0 tx=0).stop 2024-05-22T23:12:06.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.058+0000 7fdcaab73740 1 -- 172.21.15.84:0/1134178773 shutdown_connections 2024-05-22T23:12:06.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.058+0000 7fdcaab73740 1 --2- 172.21.15.84:0/1134178773 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x5629084a8d70 0x5629084ab160 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.058+0000 7fdcaab73740 1 --2- 172.21.15.84:0/1134178773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562908498b30 0x562908496460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.058+0000 7fdc897fa640 0 client.26420 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:12:06.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.058+0000 7fdcaab73740 1 --2- 172.21.15.84:0/1134178773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290848f4a0 0x562908495f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.058+0000 7fdcaab73740 1 --2- 172.21.15.84:0/1134178773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56290848ddb0 0x5629084959e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.058+0000 7fdcaab73740 1 -- 172.21.15.84:0/1134178773 >> 172.21.15.84:0/1134178773 conn(0x5629084568e0 msgr2=0x562908490110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:06.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.058+0000 7fdcaab73740 1 -- 172.21.15.84:0/1134178773 shutdown_connections 2024-05-22T23:12:06.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.058+0000 7fdcaab73740 1 -- 172.21.15.84:0/1134178773 wait complete. 2024-05-22T23:12:06.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.058+0000 7fdcaab73740 20 client.26420 trim_cache size 0 max 16384 2024-05-22T23:12:06.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.076+0000 7fdcaab73740 1 Processor -- start 2024-05-22T23:12:06.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.076+0000 7fdcaab73740 1 -- start start 2024-05-22T23:12:06.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.076+0000 7fdcaab73740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56290848ddb0 0x56290848a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.076+0000 7fdcaab73740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56290848f4a0 0x5629084a4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.076+0000 7fdcaab73740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562908498b30 0x5629084b0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.076+0000 7fdcaab73740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdca01ab670 con 0x562908498b30 2024-05-22T23:12:06.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.076+0000 7fdcaab73740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdca01aded0 con 0x56290848f4a0 2024-05-22T23:12:06.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.076+0000 7fdcaab73740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdca01abcc0 con 0x56290848ddb0 2024-05-22T23:12:06.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.076+0000 7fdca6401640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562908498b30 0x5629084b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.076+0000 7fdca6401640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562908498b30 0x5629084b0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49138/0 (socket says 172.21.15.84:49138) 2024-05-22T23:12:06.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.076+0000 7fdca6401640 1 -- 172.21.15.84:0/3529130033 learned_addr learned my addr 172.21.15.84:0/3529130033 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:06.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.076+0000 7fdca5c00640 1 --2- 172.21.15.84:0/3529130033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56290848ddb0 0x56290848a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.076+0000 7fdca53ff640 1 --2- 172.21.15.84:0/3529130033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56290848f4a0 0x5629084a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.076+0000 7fdca6401640 1 -- 172.21.15.84:0/3529130033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56290848ddb0 msgr2=0x56290848a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:12:06.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.076+0000 7fdca6401640 1 --2- 172.21.15.84:0/3529130033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56290848ddb0 0x56290848a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.076+0000 7fdca6401640 1 -- 172.21.15.84:0/3529130033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56290848f4a0 msgr2=0x5629084a4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:12:06.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.076+0000 7fdca6401640 1 --2- 172.21.15.84:0/3529130033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56290848f4a0 0x5629084a4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.076+0000 7fdca6401640 1 -- 172.21.15.84:0/3529130033 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629080c50f0 con 0x562908498b30 2024-05-22T23:12:06.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.077+0000 7fdca6401640 1 --2- 172.21.15.84:0/3529130033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562908498b30 0x5629084b0840 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fdc980ccd60 tx=0x7fdc9809bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af277aae553382cc server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:06.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.077+0000 7fdc8b7fe640 1 -- 172.21.15.84:0/3529130033 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9801f050 con 0x562908498b30 2024-05-22T23:12:06.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.077+0000 7fdc8b7fe640 1 -- 172.21.15.84:0/3529130033 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc980a4740 con 0x562908498b30 2024-05-22T23:12:06.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.077+0000 7fdc8b7fe640 1 -- 172.21.15.84:0/3529130033 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc980cf110 con 0x562908498b30 2024-05-22T23:12:06.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.077+0000 7fdcaab73740 1 -- 172.21.15.84:0/3529130033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562908498b30 msgr2=0x5629084b0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:06.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.077+0000 7fdcaab73740 1 --2- 172.21.15.84:0/3529130033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562908498b30 0x5629084b0840 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fdc980ccd60 tx=0x7fdc9809bb80 comp rx=0 tx=0).stop 2024-05-22T23:12:06.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.077+0000 7fdcaab73740 1 -- 172.21.15.84:0/3529130033 shutdown_connections 2024-05-22T23:12:06.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.077+0000 7fdcaab73740 1 --2- 172.21.15.84:0/3529130033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562908498b30 0x5629084b0840 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.077+0000 7fdcaab73740 1 --2- 172.21.15.84:0/3529130033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56290848f4a0 0x5629084a4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.077+0000 7fdcaab73740 1 --2- 172.21.15.84:0/3529130033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56290848ddb0 0x56290848a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.077+0000 7fdcaab73740 1 -- 172.21.15.84:0/3529130033 >> 172.21.15.84:0/3529130033 conn(0x5629084568e0 msgr2=0x562908489da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:06.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.078+0000 7fdcaab73740 1 -- 172.21.15.84:0/3529130033 shutdown_connections 2024-05-22T23:12:06.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.078+0000 7fdcaab73740 1 -- 172.21.15.84:0/3529130033 wait complete. 2024-05-22T23:12:06.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.078+0000 7fdcaab73740 1 -- start start 2024-05-22T23:12:06.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.078+0000 7fdcaab73740 1 Processor -- start 2024-05-22T23:12:06.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.078+0000 7fdcaab73740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290848ddb0 0x5629084a0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.078+0000 7fdcaab73740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56290848f4a0 0x562908492250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.078+0000 7fdcaab73740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562908498b30 0x562908492790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.078+0000 7fdcaab73740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdca01ac390 con 0x56290848ddb0 2024-05-22T23:12:06.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.078+0000 7fdcaab73740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdca01ad800 con 0x562908498b30 2024-05-22T23:12:06.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.078+0000 7fdcaab73740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdca01aca60 con 0x56290848f4a0 2024-05-22T23:12:06.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.078+0000 7fdcaab73740 10 client.0 _ll_register_callbacks cb 0x7fdc9c040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:12:06.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.078+0000 7fdca5c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290848ddb0 0x5629084a0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.078+0000 7fdca5c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290848ddb0 0x5629084a0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49154/0 (socket says 172.21.15.84:49154) 2024-05-22T23:12:06.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.078+0000 7fdca5c00640 1 -- 172.21.15.84:0/2355961425 learned_addr learned my addr 172.21.15.84:0/2355961425 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:06.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.078+0000 7fdca53ff640 1 --2- 172.21.15.84:0/2355961425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56290848f4a0 0x562908492250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.079+0000 7fdca6401640 1 --2- 172.21.15.84:0/2355961425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562908498b30 0x562908492790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.079+0000 7fdca5c00640 1 -- 172.21.15.84:0/2355961425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56290848f4a0 msgr2=0x562908492250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:12:06.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.079+0000 7fdca5c00640 1 --2- 172.21.15.84:0/2355961425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56290848f4a0 0x562908492250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.079+0000 7fdca5c00640 1 -- 172.21.15.84:0/2355961425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562908498b30 msgr2=0x562908492790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:12:06.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.079+0000 7fdca5c00640 1 --2- 172.21.15.84:0/2355961425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562908498b30 0x562908492790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.079+0000 7fdca5c00640 1 -- 172.21.15.84:0/2355961425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629083b0fe0 con 0x56290848ddb0 2024-05-22T23:12:06.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.079+0000 7fdca53ff640 1 --2- 172.21.15.84:0/2355961425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56290848f4a0 0x562908492250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T23:12:06.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.079+0000 7fdca5c00640 1 --2- 172.21.15.84:0/2355961425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290848ddb0 0x5629084a0f60 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c046a30 tx=0x7fdc9c004480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6bc936f521a974 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:06.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.079+0000 7fdc897fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:12:06.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.079+0000 7fdc897fa640 1 -- 172.21.15.84:0/2355961425 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c062060 con 0x56290848ddb0 2024-05-22T23:12:06.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.079+0000 7fdc897fa640 1 -- 172.21.15.84:0/2355961425 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8571+0+0 (secure 0 0 0) 0x7fdc9c031b80 con 0x56290848ddb0 2024-05-22T23:12:06.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.079+0000 7fdcaab73740 1 -- 172.21.15.84:0/2355961425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc980a4740 con 0x56290848ddb0 2024-05-22T23:12:06.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.079+0000 7fdc897fa640 1 -- 172.21.15.84:0/2355961425 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fdc9c035040 con 0x56290848ddb0 2024-05-22T23:12:06.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.079+0000 7fdcaab73740 10 client.17040 fetch_fsmap learned FSMap version 18 2024-05-22T23:12:06.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.079+0000 7fdcaab73740 1 -- 172.21.15.84:0/2355961425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5629083821c0 con 0x56290848ddb0 2024-05-22T23:12:06.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.079+0000 7fdc897fa640 1 -- 172.21.15.84:0/2355961425 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc980a4740 con 0x56290848ddb0 2024-05-22T23:12:06.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.080+0000 7fdc897fa640 1 -- 172.21.15.84:0/2355961425 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fdc9c0037b0 con 0x56290848ddb0 2024-05-22T23:12:06.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.080+0000 7fdcaab73740 10 client.17040 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:12:06.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.080+0000 7fdcaab73740 10 client.17040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:12:06.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.080+0000 7fdcaab73740 20 client.17040 populate_metadata read hostname 'smithi084' 2024-05-22T23:12:06.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.080+0000 7fdcaab73740 1 --2- 172.21.15.84:0/2355961425 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56290850f170 0x562908511590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.080+0000 7fdcaab73740 4 client.17040 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-22T23:12:06.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.080+0000 7fdcaab73740 1 -- 172.21.15.84:0/2355961425 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5629083f9830 con 0x56290850f170 2024-05-22T23:12:06.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.080+0000 7fdca53ff640 1 --2- 172.21.15.84:0/2355961425 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56290850f170 0x562908511590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.080+0000 7fdc897fa640 10 client.17040 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:12:06.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.080+0000 7fdca53ff640 1 --2- 172.21.15.84:0/2355961425 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56290850f170 0x562908511590 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98cff0ca145560ae server_cookie=21a5a15efb74e846 in_seq=0 out_seq=0 2024-05-22T23:12:06.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.080+0000 7fdc897fa640 1 -- 172.21.15.84:0/2355961425 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5629083f9830 con 0x56290850f170 2024-05-22T23:12:06.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.080+0000 7fdc897fa640 10 client.17040 handle_command_reply: tid=0 2024-05-22T23:12:06.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.080+0000 7fdcaab73740 1 client.17040 shutdown 2024-05-22T23:12:06.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.080+0000 7fdcaab73740 1 -- 172.21.15.84:0/2355961425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290848ddb0 msgr2=0x5629084a0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:06.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.080+0000 7fdcaab73740 1 --2- 172.21.15.84:0/2355961425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290848ddb0 0x5629084a0f60 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c046a30 tx=0x7fdc9c004480 comp rx=0 tx=0).stop 2024-05-22T23:12:06.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.081+0000 7fdcaab73740 1 -- 172.21.15.84:0/2355961425 shutdown_connections 2024-05-22T23:12:06.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.081+0000 7fdcaab73740 1 --2- 172.21.15.84:0/2355961425 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x56290850f170 0x562908511590 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.081+0000 7fdcaab73740 1 --2- 172.21.15.84:0/2355961425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562908498b30 0x562908492790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.081+0000 7fdcaab73740 1 --2- 172.21.15.84:0/2355961425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56290848f4a0 0x562908492250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.081+0000 7fdcaab73740 1 --2- 172.21.15.84:0/2355961425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290848ddb0 0x5629084a0f60 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.081+0000 7fdc897fa640 0 client.17040 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:12:06.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.081+0000 7fdcaab73740 1 -- 172.21.15.84:0/2355961425 >> 172.21.15.84:0/2355961425 conn(0x5629084568e0 msgr2=0x56290848f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:06.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.081+0000 7fdcaab73740 1 -- 172.21.15.84:0/2355961425 shutdown_connections 2024-05-22T23:12:06.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.081+0000 7fdcaab73740 1 -- 172.21.15.84:0/2355961425 wait complete. 2024-05-22T23:12:06.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.081+0000 7fdcaab73740 20 client.17040 trim_cache size 0 max 16384 2024-05-22T23:12:06.098 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:12:06.098 INFO:teuthology.orchestra.run.smithi084.stdout: "status": "no active scrubs running", 2024-05-22T23:12:06.098 INFO:teuthology.orchestra.run.smithi084.stdout: "scrubs": {} 2024-05-22T23:12:06.099 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:12:06.100 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.081+0000 7fdca7e64640 1 -- 172.21.15.84:0/1128606544 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fdc8c0a2d00 msgr2=0x7fdc8c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:06.100 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.081+0000 7fdca7e64640 1 --2- 172.21.15.84:0/1128606544 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fdc8c0a2d00 0x7fdc8c0a51c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x56290848b4a0 tx=0x7fdc900097a0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.100 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.081+0000 7fdca7e64640 1 -- 172.21.15.84:0/1128606544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdca0107850 msgr2=0x7fdca019ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:06.100 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.081+0000 7fdca7e64640 1 --2- 172.21.15.84:0/1128606544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdca0107850 0x7fdca019ca60 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800eaa0 tx=0x7fdc9800ef60 comp rx=0 tx=0).stop 2024-05-22T23:12:06.100 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.081+0000 7fdca7e64640 1 -- 172.21.15.84:0/1128606544 shutdown_connections 2024-05-22T23:12:06.101 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.081+0000 7fdca7e64640 1 --2- 172.21.15.84:0/1128606544 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fdc8c0a2d00 0x7fdc8c0a51c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.101 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.081+0000 7fdca7e64640 1 --2- 172.21.15.84:0/1128606544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdca0107850 0x7fdca019ca60 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.101 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.081+0000 7fdca7e64640 1 --2- 172.21.15.84:0/1128606544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdca0102200 0x7fdca01986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.101 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.081+0000 7fdca7e64640 1 --2- 172.21.15.84:0/1128606544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdca0101840 0x7fdca01981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.101 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.081+0000 7fdca7e64640 1 -- 172.21.15.84:0/1128606544 >> 172.21.15.84:0/1128606544 conn(0x7fdca0078cc0 msgr2=0x7fdca00ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:06.101 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.081+0000 7fdca7e64640 1 -- 172.21.15.84:0/1128606544 shutdown_connections 2024-05-22T23:12:06.101 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.081+0000 7fdca7e64640 1 -- 172.21.15.84:0/1128606544 wait complete. 2024-05-22T23:12:06.102 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-22T23:12:06.102 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:12:06.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.199+0000 7f1f77231640 1 -- 172.21.15.84:0/3952335334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f70100750 msgr2=0x7f1f70100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:06.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.199+0000 7f1f77231640 1 --2- 172.21.15.84:0/3952335334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f70100750 0x7f1f70100bd0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009930 tx=0x7f1f5c02a420 comp rx=0 tx=0).stop 2024-05-22T23:12:06.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.199+0000 7f1f77231640 1 -- 172.21.15.84:0/3952335334 shutdown_connections 2024-05-22T23:12:06.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.199+0000 7f1f77231640 1 --2- 172.21.15.84:0/3952335334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f70101110 0x7f1f7010d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.199+0000 7f1f77231640 1 --2- 172.21.15.84:0/3952335334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f70100750 0x7f1f70100bd0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.199+0000 7f1f77231640 1 --2- 172.21.15.84:0/3952335334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f70106760 0x7f1f70106b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.199+0000 7f1f77231640 1 -- 172.21.15.84:0/3952335334 >> 172.21.15.84:0/3952335334 conn(0x7f1f70077870 msgr2=0x7f1f70077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:06.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.199+0000 7f1f77231640 1 -- 172.21.15.84:0/3952335334 shutdown_connections 2024-05-22T23:12:06.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.199+0000 7f1f77231640 1 -- 172.21.15.84:0/3952335334 wait complete. 2024-05-22T23:12:06.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.200+0000 7f1f77231640 1 Processor -- start 2024-05-22T23:12:06.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.200+0000 7f1f77231640 1 -- start start 2024-05-22T23:12:06.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.200+0000 7f1f77231640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f70100750 0x7f1f70199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.200+0000 7f1f77231640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f70101110 0x7f1f70199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.201+0000 7f1f77231640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f70106760 0x7f1f701a40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.201+0000 7f1f77231640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1f70110b70 con 0x7f1f70100750 2024-05-22T23:12:06.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.201+0000 7f1f77231640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1f701109f0 con 0x7f1f70106760 2024-05-22T23:12:06.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.201+0000 7f1f77231640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1f70110cf0 con 0x7f1f70101110 2024-05-22T23:12:06.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.201+0000 7f1f74fcd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f70100750 0x7f1f70199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.201+0000 7f1f74fcd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f70100750 0x7f1f70199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49160/0 (socket says 172.21.15.84:49160) 2024-05-22T23:12:06.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.201+0000 7f1f74fcd640 1 -- 172.21.15.84:0/2923248207 learned_addr learned my addr 172.21.15.84:0/2923248207 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:06.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.201+0000 7f1f6ffff640 1 --2- 172.21.15.84:0/2923248207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f70101110 0x7f1f70199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.201+0000 7f1f757ce640 1 --2- 172.21.15.84:0/2923248207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f70106760 0x7f1f701a40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.201+0000 7f1f74fcd640 1 -- 172.21.15.84:0/2923248207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f70101110 msgr2=0x7f1f70199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:06.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.201+0000 7f1f74fcd640 1 --2- 172.21.15.84:0/2923248207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f70101110 0x7f1f70199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.201+0000 7f1f74fcd640 1 -- 172.21.15.84:0/2923248207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f70106760 msgr2=0x7f1f701a40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:06.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.201+0000 7f1f74fcd640 1 --2- 172.21.15.84:0/2923248207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f70106760 0x7f1f701a40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.201+0000 7f1f74fcd640 1 -- 172.21.15.84:0/2923248207 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f701a45f0 con 0x7f1f70100750 2024-05-22T23:12:06.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.201+0000 7f1f757ce640 1 --2- 172.21.15.84:0/2923248207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f70106760 0x7f1f701a40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:12:06.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.201+0000 7f1f74fcd640 1 --2- 172.21.15.84:0/2923248207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f70100750 0x7f1f70199030 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400e9f0 tx=0x7f1f6400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:06.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.201+0000 7f1f6dffb640 1 -- 172.21.15.84:0/2923248207 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6400cd80 con 0x7f1f70100750 2024-05-22T23:12:06.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.201+0000 7f1f6dffb640 1 -- 172.21.15.84:0/2923248207 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f64005500 con 0x7f1f70100750 2024-05-22T23:12:06.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.201+0000 7f1f77231640 1 -- 172.21.15.84:0/2923248207 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f701a47c0 con 0x7f1f70100750 2024-05-22T23:12:06.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.201+0000 7f1f6dffb640 1 -- 172.21.15.84:0/2923248207 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f64010690 con 0x7f1f70100750 2024-05-22T23:12:06.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.201+0000 7f1f77231640 1 -- 172.21.15.84:0/2923248207 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f701a4ce0 con 0x7f1f70100750 2024-05-22T23:12:06.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.203+0000 7f1f77231640 1 -- 172.21.15.84:0/2923248207 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f70068690 con 0x7f1f70100750 2024-05-22T23:12:06.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.203+0000 7f1f6dffb640 1 -- 172.21.15.84:0/2923248207 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1f640046c0 con 0x7f1f70100750 2024-05-22T23:12:06.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.204+0000 7f1f6dffb640 1 --2- 172.21.15.84:0/2923248207 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f1f600a2d80 0x7f1f600a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.205+0000 7f1f6ffff640 1 --2- 172.21.15.84:0/2923248207 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f1f600a2d80 0x7f1f600a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.205+0000 7f1f6dffb640 1 -- 172.21.15.84:0/2923248207 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8571+0+0 (secure 0 0 0) 0x7f1f640cf880 con 0x7f1f70100750 2024-05-22T23:12:06.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.205+0000 7f1f6ffff640 1 --2- 172.21.15.84:0/2923248207 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f1f600a2d80 0x7f1f600a5240 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009900 tx=0x7f1f5c0058d0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:06.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.209+0000 7f1f6dffb640 1 -- 172.21.15.84:0/2923248207 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1f6409c410 con 0x7f1f70100750 2024-05-22T23:12:06.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.506+0000 7f1f77231640 1 -- 172.21.15.84:0/2923248207 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f70068ff0 con 0x7f1f70100750 2024-05-22T23:12:06.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.507+0000 7f1f6dffb640 1 -- 172.21.15.84:0/2923248207 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7f1f6409bd10 con 0x7f1f70100750 2024-05-22T23:12:06.513 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:12:06.514 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:12:06.514 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:12:06.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.509+0000 7f1f77231640 1 -- 172.21.15.84:0/2923248207 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f1f600a2d80 msgr2=0x7f1f600a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:06.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.509+0000 7f1f77231640 1 --2- 172.21.15.84:0/2923248207 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f1f600a2d80 0x7f1f600a5240 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009900 tx=0x7f1f5c0058d0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.509+0000 7f1f77231640 1 -- 172.21.15.84:0/2923248207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f70100750 msgr2=0x7f1f70199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:06.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.509+0000 7f1f77231640 1 --2- 172.21.15.84:0/2923248207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f70100750 0x7f1f70199030 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400e9f0 tx=0x7f1f6400edf0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.509+0000 7f1f77231640 1 -- 172.21.15.84:0/2923248207 shutdown_connections 2024-05-22T23:12:06.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.509+0000 7f1f77231640 1 --2- 172.21.15.84:0/2923248207 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f1f600a2d80 0x7f1f600a5240 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.509+0000 7f1f77231640 1 --2- 172.21.15.84:0/2923248207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f70106760 0x7f1f701a40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.509+0000 7f1f77231640 1 --2- 172.21.15.84:0/2923248207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f70101110 0x7f1f70199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.509+0000 7f1f77231640 1 --2- 172.21.15.84:0/2923248207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f70100750 0x7f1f70199030 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.509+0000 7f1f77231640 1 -- 172.21.15.84:0/2923248207 >> 172.21.15.84:0/2923248207 conn(0x7f1f70077870 msgr2=0x7f1f701030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:06.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.509+0000 7f1f77231640 1 -- 172.21.15.84:0/2923248207 shutdown_connections 2024-05-22T23:12:06.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.509+0000 7f1f77231640 1 -- 172.21.15.84:0/2923248207 wait complete. 2024-05-22T23:12:06.528 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-22T23:12:06.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.633+0000 7f8ed8729640 1 -- 172.21.15.84:0/1471019377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ed0101840 msgr2=0x7f8ed0101cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:06.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.633+0000 7f8ed8729640 1 --2- 172.21.15.84:0/1471019377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ed0101840 0x7f8ed0101cc0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009cd0 tx=0x7f8ec402a8e0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.633+0000 7f8ed8729640 1 -- 172.21.15.84:0/1471019377 shutdown_connections 2024-05-22T23:12:06.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.633+0000 7f8ed8729640 1 --2- 172.21.15.84:0/1471019377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed0102200 0x7f8ed010e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.633+0000 7f8ed8729640 1 --2- 172.21.15.84:0/1471019377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ed0101840 0x7f8ed0101cc0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.633+0000 7f8ed8729640 1 --2- 172.21.15.84:0/1471019377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ed0107850 0x7f8ed0107c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.633+0000 7f8ed8729640 1 -- 172.21.15.84:0/1471019377 >> 172.21.15.84:0/1471019377 conn(0x7f8ed00fd2b0 msgr2=0x7f8ed00ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:06.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.633+0000 7f8ed8729640 1 -- 172.21.15.84:0/1471019377 shutdown_connections 2024-05-22T23:12:06.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.633+0000 7f8ed8729640 1 -- 172.21.15.84:0/1471019377 wait complete. 2024-05-22T23:12:06.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.634+0000 7f8ed8729640 1 Processor -- start 2024-05-22T23:12:06.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.634+0000 7f8ed8729640 1 -- start start 2024-05-22T23:12:06.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.634+0000 7f8ed8729640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed0101840 0x7f8ed01981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.634+0000 7f8ed8729640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ed0102200 0x7f8ed0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.634+0000 7f8ed8729640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ed0107850 0x7f8ed019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.634+0000 7f8ed8729640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed0111c60 con 0x7f8ed0102200 2024-05-22T23:12:06.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.634+0000 7f8ed8729640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed0111ae0 con 0x7f8ed0101840 2024-05-22T23:12:06.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.634+0000 7f8ed8729640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed0111de0 con 0x7f8ed0107850 2024-05-22T23:12:06.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.634+0000 7f8ed5cc4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ed0102200 0x7f8ed0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.634+0000 7f8ed5cc4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ed0102200 0x7f8ed0198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49172/0 (socket says 172.21.15.84:49172) 2024-05-22T23:12:06.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.634+0000 7f8ed5cc4640 1 -- 172.21.15.84:0/3357618 learned_addr learned my addr 172.21.15.84:0/3357618 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:06.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.634+0000 7f8ed64c5640 1 --2- 172.21.15.84:0/3357618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed0101840 0x7f8ed01981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.634+0000 7f8ed6cc6640 1 --2- 172.21.15.84:0/3357618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ed0107850 0x7f8ed019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.634+0000 7f8ed5cc4640 1 -- 172.21.15.84:0/3357618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ed0107850 msgr2=0x7f8ed019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:06.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.634+0000 7f8ed5cc4640 1 --2- 172.21.15.84:0/3357618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ed0107850 0x7f8ed019caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.634+0000 7f8ed5cc4640 1 -- 172.21.15.84:0/3357618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed0101840 msgr2=0x7f8ed01981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:06.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.634+0000 7f8ed5cc4640 1 --2- 172.21.15.84:0/3357618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed0101840 0x7f8ed01981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.634+0000 7f8ed5cc4640 1 -- 172.21.15.84:0/3357618 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed019d1d0 con 0x7f8ed0102200 2024-05-22T23:12:06.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.635+0000 7f8ed5cc4640 1 --2- 172.21.15.84:0/3357618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ed0102200 0x7f8ed0198720 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8ec402ae00 tx=0x7f8ec4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:06.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.635+0000 7f8ecf7fe640 1 -- 172.21.15.84:0/3357618 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4004040 con 0x7f8ed0102200 2024-05-22T23:12:06.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.635+0000 7f8ed8729640 1 -- 172.21.15.84:0/3357618 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed019d460 con 0x7f8ed0102200 2024-05-22T23:12:06.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.635+0000 7f8ecf7fe640 1 -- 172.21.15.84:0/3357618 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec40046c0 con 0x7f8ed0102200 2024-05-22T23:12:06.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.635+0000 7f8ecf7fe640 1 -- 172.21.15.84:0/3357618 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4046480 con 0x7f8ed0102200 2024-05-22T23:12:06.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.635+0000 7f8ed8729640 1 -- 172.21.15.84:0/3357618 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed01a4d50 con 0x7f8ed0102200 2024-05-22T23:12:06.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.638+0000 7f8edb436740 1 Processor -- start 2024-05-22T23:12:06.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.638+0000 7f8ecf7fe640 1 -- 172.21.15.84:0/3357618 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8ec40305a0 con 0x7f8ed0102200 2024-05-22T23:12:06.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.638+0000 7f8edb436740 1 -- start start 2024-05-22T23:12:06.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.638+0000 7f8edb436740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a963f80 0x562a0a964360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.652 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.638+0000 7f8edb436740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a0a9648a0 0x562a0a96ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.638+0000 7f8edb436740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0a96ef70 0x562a0a971360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.638+0000 7f8edb436740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562a0a8c8db0 con 0x562a0a963f80 2024-05-22T23:12:06.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.638+0000 7f8edb436740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562a0a867400 con 0x562a0a96ef70 2024-05-22T23:12:06.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.638+0000 7f8edb436740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562a0a8c4d70 con 0x562a0a9648a0 2024-05-22T23:12:06.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.638+0000 7f8ed6cc6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a963f80 0x562a0a964360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.638+0000 7f8ed6cc6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a963f80 0x562a0a964360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49188/0 (socket says 172.21.15.84:49188) 2024-05-22T23:12:06.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.638+0000 7f8ed6cc6640 1 -- 172.21.15.84:0/623805519 learned_addr learned my addr 172.21.15.84:0/623805519 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:06.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.638+0000 7f8ed64c5640 1 --2- 172.21.15.84:0/623805519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a0a9648a0 0x562a0a96ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.638+0000 7f8ed5cc4640 1 --2- 172.21.15.84:0/623805519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0a96ef70 0x562a0a971360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.638+0000 7f8ed6cc6640 1 -- 172.21.15.84:0/623805519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a0a9648a0 msgr2=0x562a0a96ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:12:06.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.638+0000 7f8ed6cc6640 1 --2- 172.21.15.84:0/623805519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a0a9648a0 0x562a0a96ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.638+0000 7f8ed6cc6640 1 -- 172.21.15.84:0/623805519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0a96ef70 msgr2=0x562a0a971360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:12:06.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.638+0000 7f8ed6cc6640 1 --2- 172.21.15.84:0/623805519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0a96ef70 0x562a0a971360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.638+0000 7f8ed6cc6640 1 -- 172.21.15.84:0/623805519 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a0a8592f0 con 0x562a0a963f80 2024-05-22T23:12:06.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.638+0000 7f8ed5cc4640 1 --2- 172.21.15.84:0/623805519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0a96ef70 0x562a0a971360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:12:06.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.639+0000 7f8ecf7fe640 1 --2- 172.21.15.84:0/3357618 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f8eb80a2d00 0x7f8eb80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.639+0000 7f8ed6cc6640 1 --2- 172.21.15.84:0/623805519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a963f80 0x562a0a964360 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009ac0 tx=0x7f8ec8009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efe4cd4f6d592ca server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:06.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.639+0000 7f8ed64c5640 1 --2- 172.21.15.84:0/3357618 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f8eb80a2d00 0x7f8eb80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.639+0000 7f8ecf7fe640 1 -- 172.21.15.84:0/3357618 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8571+0+0 (secure 0 0 0) 0x7f8ec402e030 con 0x7f8ed0102200 2024-05-22T23:12:06.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.639+0000 7f8eccff9640 1 -- 172.21.15.84:0/623805519 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8016070 con 0x562a0a963f80 2024-05-22T23:12:06.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.639+0000 7f8eccff9640 1 -- 172.21.15.84:0/623805519 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec8005500 con 0x562a0a963f80 2024-05-22T23:12:06.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.639+0000 7f8eccff9640 1 -- 172.21.15.84:0/623805519 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8011470 con 0x562a0a963f80 2024-05-22T23:12:06.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.640+0000 7f8edb436740 1 -- 172.21.15.84:0/623805519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a963f80 msgr2=0x562a0a964360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:06.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.640+0000 7f8ed64c5640 1 --2- 172.21.15.84:0/3357618 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f8eb80a2d00 0x7f8eb80a51c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x562a0a965540 tx=0x7f8ec000a250 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:06.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.640+0000 7f8edb436740 1 --2- 172.21.15.84:0/623805519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a963f80 0x562a0a964360 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009ac0 tx=0x7f8ec8009f80 comp rx=0 tx=0).stop 2024-05-22T23:12:06.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.640+0000 7f8edb436740 1 -- 172.21.15.84:0/623805519 shutdown_connections 2024-05-22T23:12:06.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.640+0000 7f8edb436740 1 --2- 172.21.15.84:0/623805519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0a96ef70 0x562a0a971360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.640+0000 7f8edb436740 1 --2- 172.21.15.84:0/623805519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a0a9648a0 0x562a0a96ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.640+0000 7f8edb436740 1 --2- 172.21.15.84:0/623805519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a963f80 0x562a0a964360 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.640+0000 7f8edb436740 1 -- 172.21.15.84:0/623805519 >> 172.21.15.84:0/623805519 conn(0x562a0a9602a0 msgr2=0x562a0a960680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:06.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.640+0000 7f8edb436740 1 -- 172.21.15.84:0/623805519 shutdown_connections 2024-05-22T23:12:06.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.640+0000 7f8edb436740 1 -- 172.21.15.84:0/623805519 wait complete. 2024-05-22T23:12:06.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.640+0000 7f8edb436740 1 -- start start 2024-05-22T23:12:06.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.640+0000 7f8edb436740 1 Processor -- start 2024-05-22T23:12:06.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.640+0000 7f8edb436740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a963f80 0x562a0a96be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.640+0000 7f8edb436740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a0a9648a0 0x562a0a96c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.640+0000 7f8edb436740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0a96ef70 0x562a0a96c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.640+0000 7f8edb436740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562a0a971d60 con 0x562a0a963f80 2024-05-22T23:12:06.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.640+0000 7f8edb436740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562a0a8832e0 con 0x562a0a96ef70 2024-05-22T23:12:06.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.640+0000 7f8edb436740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562a0a971ee0 con 0x562a0a9648a0 2024-05-22T23:12:06.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.640+0000 7f8edb436740 10 client.0 _ll_register_callbacks cb 0x562a0a88ca70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:12:06.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.640+0000 7f8ed64c5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a963f80 0x562a0a96be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.640+0000 7f8ed64c5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a963f80 0x562a0a96be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49200/0 (socket says 172.21.15.84:49200) 2024-05-22T23:12:06.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.640+0000 7f8ed64c5640 1 -- 172.21.15.84:0/3677398381 learned_addr learned my addr 172.21.15.84:0/3677398381 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:06.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.641+0000 7f8ed5cc4640 1 --2- 172.21.15.84:0/3677398381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a0a9648a0 0x562a0a96c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.641+0000 7f8ed6cc6640 1 --2- 172.21.15.84:0/3677398381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0a96ef70 0x562a0a96c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.641+0000 7f8ed64c5640 1 -- 172.21.15.84:0/3677398381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a0a9648a0 msgr2=0x562a0a96c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:12:06.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.641+0000 7f8ed64c5640 1 --2- 172.21.15.84:0/3677398381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a0a9648a0 0x562a0a96c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.641+0000 7f8ed64c5640 1 -- 172.21.15.84:0/3677398381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0a96ef70 msgr2=0x562a0a96c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:12:06.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.641+0000 7f8ed64c5640 1 --2- 172.21.15.84:0/3677398381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0a96ef70 0x562a0a96c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.641+0000 7f8ed64c5640 1 -- 172.21.15.84:0/3677398381 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a0a5dc3f0 con 0x562a0a963f80 2024-05-22T23:12:06.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.641+0000 7f8ed6cc6640 1 --2- 172.21.15.84:0/3677398381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0a96ef70 0x562a0a96c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:12:06.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.641+0000 7f8ed64c5640 1 --2- 172.21.15.84:0/3677398381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a963f80 0x562a0a96be90 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0005940 tx=0x7f8ec00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9d32ca6272213d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:06.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.641+0000 7f8eb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:12:06.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.641+0000 7f8eb5ffb640 1 -- 172.21.15.84:0/3677398381 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec002a060 con 0x562a0a963f80 2024-05-22T23:12:06.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.641+0000 7f8eb5ffb640 1 -- 172.21.15.84:0/3677398381 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8571+0+0 (secure 0 0 0) 0x7f8ec001b460 con 0x562a0a963f80 2024-05-22T23:12:06.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.641+0000 7f8edb436740 1 -- 172.21.15.84:0/3677398381 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec8005500 con 0x562a0a963f80 2024-05-22T23:12:06.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.641+0000 7f8eb5ffb640 1 -- 172.21.15.84:0/3677398381 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f8ec001b660 con 0x562a0a963f80 2024-05-22T23:12:06.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.641+0000 7f8edb436740 10 client.17058 fetch_fsmap learned FSMap version 18 2024-05-22T23:12:06.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.641+0000 7f8edb436740 1 -- 172.21.15.84:0/3677398381 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x562a0a6a67d0 con 0x562a0a963f80 2024-05-22T23:12:06.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.641+0000 7f8eb5ffb640 1 -- 172.21.15.84:0/3677398381 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec8005500 con 0x562a0a963f80 2024-05-22T23:12:06.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.642+0000 7f8eb5ffb640 1 -- 172.21.15.84:0/3677398381 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f8ec0019040 con 0x562a0a963f80 2024-05-22T23:12:06.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.642+0000 7f8edb436740 10 client.17058 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:12:06.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.642+0000 7f8edb436740 10 client.17058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:12:06.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.642+0000 7f8edb436740 20 client.17058 populate_metadata read hostname 'smithi084' 2024-05-22T23:12:06.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.642+0000 7f8edb436740 1 --2- 172.21.15.84:0/3677398381 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x562a0a97f230 0x562a0a9816b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.642+0000 7f8edb436740 4 client.17058 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T23:12:06.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.642+0000 7f8edb436740 1 -- 172.21.15.84:0/3677398381 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a0a8d64e0 con 0x562a0a97f230 2024-05-22T23:12:06.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.642+0000 7f8ed5cc4640 1 --2- 172.21.15.84:0/3677398381 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x562a0a97f230 0x562a0a9816b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.642+0000 7f8eb5ffb640 10 client.17058 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:12:06.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.642+0000 7f8ed5cc4640 1 --2- 172.21.15.84:0/3677398381 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x562a0a97f230 0x562a0a9816b0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac127e52b3ff790a server_cookie=82decba0569c0e08 in_seq=0 out_seq=0 2024-05-22T23:12:06.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.643+0000 7f8eb5ffb640 1 -- 172.21.15.84:0/3677398381 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562a0a8d64e0 con 0x562a0a97f230 2024-05-22T23:12:06.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.643+0000 7f8eb5ffb640 10 client.17058 handle_command_reply: tid=0 2024-05-22T23:12:06.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.643+0000 7f8edb436740 1 client.17058 shutdown 2024-05-22T23:12:06.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.643+0000 7f8edb436740 1 -- 172.21.15.84:0/3677398381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a963f80 msgr2=0x562a0a96be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:06.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.643+0000 7f8edb436740 1 --2- 172.21.15.84:0/3677398381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a963f80 0x562a0a96be90 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0005940 tx=0x7f8ec00046c0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.643+0000 7f8edb436740 1 -- 172.21.15.84:0/3677398381 shutdown_connections 2024-05-22T23:12:06.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.643+0000 7f8edb436740 1 --2- 172.21.15.84:0/3677398381 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x562a0a97f230 0x562a0a9816b0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.643+0000 7f8edb436740 1 --2- 172.21.15.84:0/3677398381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0a96ef70 0x562a0a96c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.643+0000 7f8edb436740 1 --2- 172.21.15.84:0/3677398381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a0a9648a0 0x562a0a96c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.643+0000 7f8eb5ffb640 0 client.17058 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:12:06.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.643+0000 7f8edb436740 1 --2- 172.21.15.84:0/3677398381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a963f80 0x562a0a96be90 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.643+0000 7f8edb436740 1 -- 172.21.15.84:0/3677398381 >> 172.21.15.84:0/3677398381 conn(0x562a0a9602a0 msgr2=0x562a0a965fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:06.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.643+0000 7f8edb436740 1 -- 172.21.15.84:0/3677398381 shutdown_connections 2024-05-22T23:12:06.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.643+0000 7f8edb436740 1 -- 172.21.15.84:0/3677398381 wait complete. 2024-05-22T23:12:06.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.643+0000 7f8edb436740 20 client.17058 trim_cache size 0 max 16384 2024-05-22T23:12:06.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.656+0000 7f8edb436740 1 Processor -- start 2024-05-22T23:12:06.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.656+0000 7f8edb436740 1 -- start start 2024-05-22T23:12:06.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.656+0000 7f8edb436740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a0a963f80 0x562a0a97ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.656+0000 7f8edb436740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a9648a0 0x562a0a96a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.656+0000 7f8edb436740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0a96ef70 0x562a0a96c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.656+0000 7f8edb436740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed01ab630 con 0x562a0a9648a0 2024-05-22T23:12:06.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.656+0000 7f8edb436740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed01ade90 con 0x562a0a96ef70 2024-05-22T23:12:06.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.656+0000 7f8edb436740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed01abc80 con 0x562a0a963f80 2024-05-22T23:12:06.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.656+0000 7f8ed5cc4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a9648a0 0x562a0a96a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.656+0000 7f8ed5cc4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a9648a0 0x562a0a96a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49216/0 (socket says 172.21.15.84:49216) 2024-05-22T23:12:06.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.656+0000 7f8ed5cc4640 1 -- 172.21.15.84:0/2680451710 learned_addr learned my addr 172.21.15.84:0/2680451710 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:06.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.656+0000 7f8ed6cc6640 1 --2- 172.21.15.84:0/2680451710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0a96ef70 0x562a0a96c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.656+0000 7f8ed64c5640 1 --2- 172.21.15.84:0/2680451710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a0a963f80 0x562a0a97ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.657+0000 7f8ed5cc4640 1 -- 172.21.15.84:0/2680451710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a0a963f80 msgr2=0x562a0a97ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:12:06.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.657+0000 7f8ed5cc4640 1 --2- 172.21.15.84:0/2680451710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a0a963f80 0x562a0a97ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.657+0000 7f8ed5cc4640 1 -- 172.21.15.84:0/2680451710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0a96ef70 msgr2=0x562a0a96c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:12:06.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.657+0000 7f8ed5cc4640 1 --2- 172.21.15.84:0/2680451710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0a96ef70 0x562a0a96c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.657+0000 7f8ed5cc4640 1 -- 172.21.15.84:0/2680451710 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a0a960be0 con 0x562a0a9648a0 2024-05-22T23:12:06.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.657+0000 7f8ed64c5640 1 --2- 172.21.15.84:0/2680451710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a0a963f80 0x562a0a97ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:12:06.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.657+0000 7f8ed5cc4640 1 --2- 172.21.15.84:0/2680451710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a9648a0 0x562a0a96a630 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40ee840 tx=0x7f8ec40f2e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbc9f2bf1fc5895 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:06.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.657+0000 7f8eb7fff640 1 -- 172.21.15.84:0/2680451710 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec40bd820 con 0x562a0a9648a0 2024-05-22T23:12:06.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.657+0000 7f8eb7fff640 1 -- 172.21.15.84:0/2680451710 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec4036040 con 0x562a0a9648a0 2024-05-22T23:12:06.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.657+0000 7f8eb7fff640 1 -- 172.21.15.84:0/2680451710 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec40d6110 con 0x562a0a9648a0 2024-05-22T23:12:06.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.657+0000 7f8edb436740 1 -- 172.21.15.84:0/2680451710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a9648a0 msgr2=0x562a0a96a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:06.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.657+0000 7f8edb436740 1 --2- 172.21.15.84:0/2680451710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a9648a0 0x562a0a96a630 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40ee840 tx=0x7f8ec40f2e30 comp rx=0 tx=0).stop 2024-05-22T23:12:06.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.657+0000 7f8edb436740 1 -- 172.21.15.84:0/2680451710 shutdown_connections 2024-05-22T23:12:06.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.657+0000 7f8edb436740 1 --2- 172.21.15.84:0/2680451710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0a96ef70 0x562a0a96c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.657+0000 7f8edb436740 1 --2- 172.21.15.84:0/2680451710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a9648a0 0x562a0a96a630 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.657+0000 7f8edb436740 1 --2- 172.21.15.84:0/2680451710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a0a963f80 0x562a0a97ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.657+0000 7f8edb436740 1 -- 172.21.15.84:0/2680451710 >> 172.21.15.84:0/2680451710 conn(0x562a0a9602a0 msgr2=0x562a0a965f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:06.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.658+0000 7f8edb436740 1 -- 172.21.15.84:0/2680451710 shutdown_connections 2024-05-22T23:12:06.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.658+0000 7f8edb436740 1 -- 172.21.15.84:0/2680451710 wait complete. 2024-05-22T23:12:06.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.658+0000 7f8edb436740 1 -- start start 2024-05-22T23:12:06.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.658+0000 7f8edb436740 1 Processor -- start 2024-05-22T23:12:06.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.658+0000 7f8edb436740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0a963f80 0x562a0a972b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.658+0000 7f8edb436740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a0a9648a0 0x562a0a973060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.658+0000 7f8edb436740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a96ef70 0x562a0a9735a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.658+0000 7f8edb436740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed01ac350 con 0x562a0a96ef70 2024-05-22T23:12:06.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.658+0000 7f8edb436740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed01ad7c0 con 0x562a0a963f80 2024-05-22T23:12:06.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.658+0000 7f8edb436740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed01aca20 con 0x562a0a9648a0 2024-05-22T23:12:06.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.658+0000 7f8edb436740 10 client.0 _ll_register_callbacks cb 0x7f8ec000a890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T23:12:06.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.658+0000 7f8ed6cc6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a96ef70 0x562a0a9735a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.658+0000 7f8ed6cc6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a96ef70 0x562a0a9735a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49220/0 (socket says 172.21.15.84:49220) 2024-05-22T23:12:06.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.658+0000 7f8ed6cc6640 1 -- 172.21.15.84:0/3545751542 learned_addr learned my addr 172.21.15.84:0/3545751542 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:06.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.658+0000 7f8ed64c5640 1 --2- 172.21.15.84:0/3545751542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0a963f80 0x562a0a972b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.658+0000 7f8ed6cc6640 1 -- 172.21.15.84:0/3545751542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a0a9648a0 msgr2=0x562a0a973060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:12:06.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.658+0000 7f8ed6cc6640 1 --2- 172.21.15.84:0/3545751542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a0a9648a0 0x562a0a973060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.658+0000 7f8ed6cc6640 1 -- 172.21.15.84:0/3545751542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0a963f80 msgr2=0x562a0a972b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T23:12:06.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.658+0000 7f8ed6cc6640 1 --2- 172.21.15.84:0/3545751542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0a963f80 0x562a0a972b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.658+0000 7f8ed6cc6640 1 -- 172.21.15.84:0/3545751542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a0a9e47f0 con 0x562a0a96ef70 2024-05-22T23:12:06.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.659+0000 7f8ed6cc6640 1 --2- 172.21.15.84:0/3545751542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a96ef70 0x562a0a9735a0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8017000 tx=0x7f8ec8004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4de35695dcbcfa server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:06.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.659+0000 7f8eb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-22T23:12:06.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.659+0000 7f8eb5ffb640 1 -- 172.21.15.84:0/3545751542 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8016040 con 0x562a0a96ef70 2024-05-22T23:12:06.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.659+0000 7f8eb5ffb640 1 -- 172.21.15.84:0/3545751542 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8571+0+0 (secure 0 0 0) 0x7f8ec8011670 con 0x562a0a96ef70 2024-05-22T23:12:06.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.659+0000 7f8edb436740 1 -- 172.21.15.84:0/3545751542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec4036040 con 0x562a0a96ef70 2024-05-22T23:12:06.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.659+0000 7f8eb5ffb640 1 -- 172.21.15.84:0/3545751542 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f8ec8011870 con 0x562a0a96ef70 2024-05-22T23:12:06.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.659+0000 7f8edb436740 10 client.17064 fetch_fsmap learned FSMap version 18 2024-05-22T23:12:06.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.659+0000 7f8edb436740 1 -- 172.21.15.84:0/3545751542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x562a0a9e5a30 con 0x562a0a96ef70 2024-05-22T23:12:06.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.659+0000 7f8eb5ffb640 1 -- 172.21.15.84:0/3545751542 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec4036040 con 0x562a0a96ef70 2024-05-22T23:12:06.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.659+0000 7f8eb5ffb640 1 -- 172.21.15.84:0/3545751542 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f8ec800f040 con 0x562a0a96ef70 2024-05-22T23:12:06.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.659+0000 7f8edb436740 10 client.17064 fetch_fsmap finished waiting for FSMap version 18 2024-05-22T23:12:06.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.659+0000 7f8edb436740 10 client.17064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-22T23:12:06.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.659+0000 7f8edb436740 20 client.17064 populate_metadata read hostname 'smithi084' 2024-05-22T23:12:06.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.659+0000 7f8edb436740 1 --2- 172.21.15.84:0/3545751542 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x562a0a9e60d0 0x562a0a9e84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:06.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.659+0000 7f8edb436740 4 client.17064 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-22T23:12:06.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.659+0000 7f8edb436740 1 -- 172.21.15.84:0/3545751542 --> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a0a8d0830 con 0x562a0a9e60d0 2024-05-22T23:12:06.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.659+0000 7f8ed64c5640 1 --2- 172.21.15.84:0/3545751542 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x562a0a9e60d0 0x562a0a9e84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:06.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.660+0000 7f8ed64c5640 1 --2- 172.21.15.84:0/3545751542 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x562a0a9e60d0 0x562a0a9e84f0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fd4a152e2067a12 server_cookie=e46fdb8c45b1d6ec in_seq=0 out_seq=0 2024-05-22T23:12:06.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.660+0000 7f8eb5ffb640 10 client.17064 ms_handle_connect on v2:172.21.15.96:6838/3272422729 2024-05-22T23:12:06.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.660+0000 7f8eb5ffb640 1 -- 172.21.15.84:0/3545751542 <== mds.0 v2:172.21.15.96:6838/3272422729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a0a8d0830 con 0x562a0a9e60d0 2024-05-22T23:12:06.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.660+0000 7f8eb5ffb640 10 client.17064 handle_command_reply: tid=0 2024-05-22T23:12:06.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.660+0000 7f8edb436740 1 client.17064 shutdown 2024-05-22T23:12:06.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.660+0000 7f8edb436740 1 -- 172.21.15.84:0/3545751542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a96ef70 msgr2=0x562a0a9735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:06.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.660+0000 7f8edb436740 1 --2- 172.21.15.84:0/3545751542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a96ef70 0x562a0a9735a0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8017000 tx=0x7f8ec8004a30 comp rx=0 tx=0).stop 2024-05-22T23:12:06.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.660+0000 7f8edb436740 1 -- 172.21.15.84:0/3545751542 shutdown_connections 2024-05-22T23:12:06.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.660+0000 7f8edb436740 1 --2- 172.21.15.84:0/3545751542 >> [v2:172.21.15.96:6838/3272422729,v1:172.21.15.96:6839/3272422729] conn(0x562a0a9e60d0 0x562a0a9e84f0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.660+0000 7f8edb436740 1 --2- 172.21.15.84:0/3545751542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a0a96ef70 0x562a0a9735a0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.660+0000 7f8edb436740 1 --2- 172.21.15.84:0/3545751542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a0a9648a0 0x562a0a973060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.660+0000 7f8eb5ffb640 0 client.17064 ms_handle_reset on v2:172.21.15.96:6838/3272422729 2024-05-22T23:12:06.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.660+0000 7f8edb436740 1 --2- 172.21.15.84:0/3545751542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0a963f80 0x562a0a972b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.660+0000 7f8edb436740 1 -- 172.21.15.84:0/3545751542 >> 172.21.15.84:0/3545751542 conn(0x562a0a9602a0 msgr2=0x562a0a95ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:06.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.660+0000 7f8edb436740 1 -- 172.21.15.84:0/3545751542 shutdown_connections 2024-05-22T23:12:06.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.660+0000 7f8edb436740 1 -- 172.21.15.84:0/3545751542 wait complete. 2024-05-22T23:12:06.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.660+0000 7f8edb436740 20 client.17064 trim_cache size 0 max 16384 2024-05-22T23:12:06.678 INFO:teuthology.orchestra.run.smithi084.stdout:[] 2024-05-22T23:12:06.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.661+0000 7f8ed8729640 1 -- 172.21.15.84:0/3357618 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f8eb80a2d00 msgr2=0x7f8eb80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:06.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.661+0000 7f8ed8729640 1 --2- 172.21.15.84:0/3357618 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f8eb80a2d00 0x7f8eb80a51c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x562a0a965540 tx=0x7f8ec000a250 comp rx=0 tx=0).stop 2024-05-22T23:12:06.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.661+0000 7f8ed8729640 1 -- 172.21.15.84:0/3357618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ed0102200 msgr2=0x7f8ed0198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:06.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.661+0000 7f8ed8729640 1 --2- 172.21.15.84:0/3357618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ed0102200 0x7f8ed0198720 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8ec402ae00 tx=0x7f8ec4003970 comp rx=0 tx=0).stop 2024-05-22T23:12:06.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.661+0000 7f8ed8729640 1 -- 172.21.15.84:0/3357618 shutdown_connections 2024-05-22T23:12:06.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.661+0000 7f8ed8729640 1 --2- 172.21.15.84:0/3357618 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f8eb80a2d00 0x7f8eb80a51c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.661+0000 7f8ed8729640 1 --2- 172.21.15.84:0/3357618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ed0107850 0x7f8ed019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.661+0000 7f8ed8729640 1 --2- 172.21.15.84:0/3357618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ed0102200 0x7f8ed0198720 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.661+0000 7f8ed8729640 1 --2- 172.21.15.84:0/3357618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed0101840 0x7f8ed01981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:06.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.661+0000 7f8ed8729640 1 -- 172.21.15.84:0/3357618 >> 172.21.15.84:0/3357618 conn(0x7f8ed00fd2b0 msgr2=0x7f8ed00fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:06.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.661+0000 7f8ed8729640 1 -- 172.21.15.84:0/3357618 shutdown_connections 2024-05-22T23:12:06.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:06.661+0000 7f8ed8729640 1 -- 172.21.15.84:0/3357618 wait complete. 2024-05-22T23:12:07.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:12:07.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:12:07.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:12:07.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:12:07.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:12:07.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:12:07.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: pgmap v319: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-22T23:12:07.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2923248207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:12:07.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: Adjusting osd_memory_target on smithi084 to 176.2M 2024-05-22T23:12:07.059 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-05-22T23:12:07.059 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: Unable to set osd_memory_target on smithi084 to 184766668: error parsing value: Value '184766668' is below minimum 939524096 2024-05-22T23:12:07.059 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: Unable to set osd_memory_target on smithi096 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-05-22T23:12:07.059 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.059 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.059 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.059 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.060 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: Adjusting osd_memory_target on smithi204 to 176.2M 2024-05-22T23:12:07.060 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: Unable to set osd_memory_target on smithi204 to 184759500: error parsing value: Value '184759500' is below minimum 939524096 2024-05-22T23:12:07.060 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:12:07.060 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:12:07.060 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:06 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:12:07.131 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:12:07.132 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:12:07.132 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:12:07.132 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:12:07.132 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:12:07.132 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:12:07.132 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: pgmap v319: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-22T23:12:07.132 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2923248207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:12:07.132 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.133 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.133 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.133 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.133 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.133 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.133 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.133 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.133 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: Adjusting osd_memory_target on smithi084 to 176.2M 2024-05-22T23:12:07.134 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-05-22T23:12:07.134 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: Unable to set osd_memory_target on smithi084 to 184766668: error parsing value: Value '184766668' is below minimum 939524096 2024-05-22T23:12:07.134 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: Unable to set osd_memory_target on smithi096 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-05-22T23:12:07.134 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.134 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.134 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.134 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.135 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: Adjusting osd_memory_target on smithi204 to 176.2M 2024-05-22T23:12:07.135 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: Unable to set osd_memory_target on smithi204 to 184759500: error parsing value: Value '184759500' is below minimum 939524096 2024-05-22T23:12:07.135 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:12:07.135 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:12:07.135 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:06 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:12:07.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:12:07.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:12:07.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:12:07.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:12:07.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:12:07.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:12:07.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: pgmap v319: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-22T23:12:07.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2923248207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:12:07.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.169 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.169 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.169 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.169 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: Adjusting osd_memory_target on smithi084 to 176.2M 2024-05-22T23:12:07.169 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-05-22T23:12:07.169 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: Unable to set osd_memory_target on smithi084 to 184766668: error parsing value: Value '184766668' is below minimum 939524096 2024-05-22T23:12:07.169 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: Unable to set osd_memory_target on smithi096 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-05-22T23:12:07.169 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.170 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.170 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.170 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-22T23:12:07.170 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: Adjusting osd_memory_target on smithi204 to 176.2M 2024-05-22T23:12:07.170 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: Unable to set osd_memory_target on smithi204 to 184759500: error parsing value: Value '184759500' is below minimum 939524096 2024-05-22T23:12:07.170 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-22T23:12:07.171 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-22T23:12:07.171 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:06 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' 2024-05-22T23:12:07.683 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-05-22T23:12:07.683 INFO:tasks.fwd_scrub:done joining 2024-05-22T23:12:07.683 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-22T23:12:07.693 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T23:12:07.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.794+0000 7f68bfb3b640 1 -- 172.21.15.84:0/894131030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68b81007c0 msgr2=0x7f68b8100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:07.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.794+0000 7f68bfb3b640 1 --2- 172.21.15.84:0/894131030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68b81007c0 0x7f68b8100c40 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009cd0 tx=0x7f68ac02a8e0 comp rx=0 tx=0).stop 2024-05-22T23:12:07.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.794+0000 7f68bfb3b640 1 -- 172.21.15.84:0/894131030 shutdown_connections 2024-05-22T23:12:07.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.794+0000 7f68bfb3b640 1 --2- 172.21.15.84:0/894131030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b8101180 0x7f68b810d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:07.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.794+0000 7f68bfb3b640 1 --2- 172.21.15.84:0/894131030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68b81007c0 0x7f68b8100c40 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:07.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.794+0000 7f68bfb3b640 1 --2- 172.21.15.84:0/894131030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68b8106780 0x7f68b8106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:07.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.794+0000 7f68bfb3b640 1 -- 172.21.15.84:0/894131030 >> 172.21.15.84:0/894131030 conn(0x7f68b80778d0 msgr2=0x7f68b8077ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:07.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.794+0000 7f68bfb3b640 1 -- 172.21.15.84:0/894131030 shutdown_connections 2024-05-22T23:12:07.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.794+0000 7f68bfb3b640 1 -- 172.21.15.84:0/894131030 wait complete. 2024-05-22T23:12:07.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.795+0000 7f68bfb3b640 1 Processor -- start 2024-05-22T23:12:07.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.795+0000 7f68bfb3b640 1 -- start start 2024-05-22T23:12:07.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.795+0000 7f68bfb3b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68b81007c0 0x7f68b8196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:07.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.795+0000 7f68bfb3b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68b8101180 0x7f68b81974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:07.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.795+0000 7f68bfb3b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b8106780 0x7f68b819b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:07.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.795+0000 7f68bfb3b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f68b8110b90 con 0x7f68b8101180 2024-05-22T23:12:07.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.795+0000 7f68bfb3b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f68b8110a10 con 0x7f68b8106780 2024-05-22T23:12:07.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.795+0000 7f68bfb3b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f68b8110d10 con 0x7f68b81007c0 2024-05-22T23:12:07.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.795+0000 7f68bd0d6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68b8101180 0x7f68b81974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:07.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.795+0000 7f68bd0d6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68b8101180 0x7f68b81974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49236/0 (socket says 172.21.15.84:49236) 2024-05-22T23:12:07.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.795+0000 7f68bd0d6640 1 -- 172.21.15.84:0/2640761320 learned_addr learned my addr 172.21.15.84:0/2640761320 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:07.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.795+0000 7f68bd8d7640 1 --2- 172.21.15.84:0/2640761320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68b81007c0 0x7f68b8196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:07.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.795+0000 7f68be0d8640 1 --2- 172.21.15.84:0/2640761320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b8106780 0x7f68b819b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:07.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.795+0000 7f68bd0d6640 1 -- 172.21.15.84:0/2640761320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68b81007c0 msgr2=0x7f68b8196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:07.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.795+0000 7f68bd0d6640 1 --2- 172.21.15.84:0/2640761320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68b81007c0 0x7f68b8196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:07.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.795+0000 7f68bd0d6640 1 -- 172.21.15.84:0/2640761320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b8106780 msgr2=0x7f68b819b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:07.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.795+0000 7f68bd0d6640 1 --2- 172.21.15.84:0/2640761320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b8106780 0x7f68b819b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:07.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.795+0000 7f68bd0d6640 1 -- 172.21.15.84:0/2640761320 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b819bfa0 con 0x7f68b8101180 2024-05-22T23:12:07.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.795+0000 7f68be0d8640 1 --2- 172.21.15.84:0/2640761320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b8106780 0x7f68b819b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:12:07.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.795+0000 7f68bd0d6640 1 --2- 172.21.15.84:0/2640761320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68b8101180 0x7f68b81974f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f68ac02ae00 tx=0x7f68ac003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:07.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.796+0000 7f68b6ffd640 1 -- 172.21.15.84:0/2640761320 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac004040 con 0x7f68b8101180 2024-05-22T23:12:07.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.796+0000 7f68b6ffd640 1 -- 172.21.15.84:0/2640761320 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68ac0046c0 con 0x7f68b8101180 2024-05-22T23:12:07.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.796+0000 7f68bfb3b640 1 -- 172.21.15.84:0/2640761320 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b819c230 con 0x7f68b8101180 2024-05-22T23:12:07.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.796+0000 7f68b6ffd640 1 -- 172.21.15.84:0/2640761320 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac046480 con 0x7f68b8101180 2024-05-22T23:12:07.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.796+0000 7f68bfb3b640 1 -- 172.21.15.84:0/2640761320 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b81a3b20 con 0x7f68b8101180 2024-05-22T23:12:07.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.797+0000 7f68bfb3b640 1 -- 172.21.15.84:0/2640761320 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68b8068ae0 con 0x7f68b8101180 2024-05-22T23:12:07.807 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.798+0000 7f68b6ffd640 1 -- 172.21.15.84:0/2640761320 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f68ac0305a0 con 0x7f68b8101180 2024-05-22T23:12:07.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.798+0000 7f68b6ffd640 1 --2- 172.21.15.84:0/2640761320 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f68a00a2d00 0x7f68a00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:07.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.798+0000 7f68bd8d7640 1 --2- 172.21.15.84:0/2640761320 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f68a00a2d00 0x7f68a00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:07.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.798+0000 7f68b6ffd640 1 -- 172.21.15.84:0/2640761320 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8615+0+0 (secure 0 0 0) 0x7f68ac02e030 con 0x7f68b8101180 2024-05-22T23:12:07.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.799+0000 7f68bd8d7640 1 --2- 172.21.15.84:0/2640761320 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f68a00a2d00 0x7f68a00a51c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f68b000cac0 tx=0x7f68b000c4a0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:07.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.803+0000 7f68b6ffd640 1 -- 172.21.15.84:0/2640761320 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f68ac04b060 con 0x7f68b8101180 2024-05-22T23:12:07.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.983+0000 7f68bfb3b640 1 -- 172.21.15.84:0/2640761320 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68b802d200 con 0x7f68b8101180 2024-05-22T23:12:07.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.985+0000 7f68b6ffd640 1 -- 172.21.15.84:0/2640761320 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+23257 (secure 0 0 0) 0x7f68ac030890 con 0x7f68b8101180 2024-05-22T23:12:07.991 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:12:07.992 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":98,"fsid":"c1135a12-188d-11ef-bc9a-c7b262605968","created":"2024-05-22T22:51:49.878218+0000","modified":"2024-05-22T23:12:06.889352+0000","last_up_change":"2024-05-22T22:58:58.550278+0000","last_in_change":"2024-05-22T22:58:37.322139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T22:55:07.719394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T22:59:50.150511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T22:59:51.111182+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762df065-fb4f-4c71-aab8-a62d2282c253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6803","nonce":1559945826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6805","nonce":1559945826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6809","nonce":1559945826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6807","nonce":1559945826}]},"public_addr":"172.21.15.84:6803/1559945826","cluster_addr":"172.21.15.84:6805/1559945826","heartbeat_back_addr":"172.21.15.84:6809/1559945826","heartbeat_front_addr":"172.21.15.84:6807/1559945826","state":["exists","up"]},{"osd":1,"uuid":"3fb2385d-6a34-49e3-88e1-116237beb346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6801","nonce":2996505353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6803","nonce":2996505353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6807","nonce":2996505353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6805","nonce":2996505353}]},"public_addr":"172.21.15.96:6801/2996505353","cluster_addr":"172.21.15.96:6803/2996505353","heartbeat_back_addr":"172.21.15.96:6807/2996505353","heartbeat_front_addr":"172.21.15.96:6805/2996505353","state":["exists","up"]},{"osd":2,"uuid":"e306e88e-a308-4585-ae11-c8af10b84038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6801","nonce":1499577897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6803","nonce":1499577897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6807","nonce":1499577897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6805","nonce":1499577897}]},"public_addr":"172.21.15.204:6801/1499577897","cluster_addr":"172.21.15.204:6803/1499577897","heartbeat_back_addr":"172.21.15.204:6807/1499577897","heartbeat_front_addr":"172.21.15.204:6805/1499577897","state":["exists","up"]},{"osd":3,"uuid":"d2a5fb97-6f60-4d45-9f18-447cae480974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6811","nonce":2869340648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6813","nonce":2869340648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6817","nonce":2869340648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6815","nonce":2869340648}]},"public_addr":"172.21.15.84:6811/2869340648","cluster_addr":"172.21.15.84:6813/2869340648","heartbeat_back_addr":"172.21.15.84:6817/2869340648","heartbeat_front_addr":"172.21.15.84:6815/2869340648","state":["exists","up"]},{"osd":4,"uuid":"4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6809","nonce":2554265648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6811","nonce":2554265648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6815","nonce":2554265648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6813","nonce":2554265648}]},"public_addr":"172.21.15.96:6809/2554265648","cluster_addr":"172.21.15.96:6811/2554265648","heartbeat_back_addr":"172.21.15.96:6815/2554265648","heartbeat_front_addr":"172.21.15.96:6813/2554265648","state":["exists","up"]},{"osd":5,"uuid":"7996bf5e-93a5-41b0-a9b1-99e7ef4514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6809","nonce":4075476986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6811","nonce":4075476986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6815","nonce":4075476986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6813","nonce":4075476986}]},"public_addr":"172.21.15.204:6809/4075476986","cluster_addr":"172.21.15.204:6811/4075476986","heartbeat_back_addr":"172.21.15.204:6815/4075476986","heartbeat_front_addr":"172.21.15.204:6813/4075476986","state":["exists","up"]},{"osd":6,"uuid":"f4b7c5de-1836-4d16-8603-3c517524d9fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6819","nonce":1806202865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6821","nonce":1806202865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6825","nonce":1806202865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6823","nonce":1806202865}]},"public_addr":"172.21.15.84:6819/1806202865","cluster_addr":"172.21.15.84:6821/1806202865","heartbeat_back_addr":"172.21.15.84:6825/1806202865","heartbeat_front_addr":"172.21.15.84:6823/1806202865","state":["exists","up"]},{"osd":7,"uuid":"69a18c43-5447-480d-9466-190830070c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6817","nonce":3195792774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6819","nonce":3195792774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6823","nonce":3195792774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6821","nonce":3195792774}]},"public_addr":"172.21.15.96:6817/3195792774","cluster_addr":"172.21.15.96:6819/3195792774","heartbeat_back_addr":"172.21.15.96:6823/3195792774","heartbeat_front_addr":"172.21.15.96:6821/3195792774","state":["exists","up"]},{"osd":8,"uuid":"2c5cadca-689a-441e-847c-ba370d400ef1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6817","nonce":606997927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6819","nonce":606997927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6823","nonce":606997927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6821","nonce":606997927}]},"public_addr":"172.21.15.204:6817/606997927","cluster_addr":"172.21.15.204:6819/606997927","heartbeat_back_addr":"172.21.15.204:6823/606997927","heartbeat_front_addr":"172.21.15.204:6821/606997927","state":["exists","up"]},{"osd":9,"uuid":"e3984fb2-7c00-4275-a10e-bca7c859f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6827","nonce":2786189012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6829","nonce":2786189012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6833","nonce":2786189012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6831","nonce":2786189012}]},"public_addr":"172.21.15.84:6827/2786189012","cluster_addr":"172.21.15.84:6829/2786189012","heartbeat_back_addr":"172.21.15.84:6833/2786189012","heartbeat_front_addr":"172.21.15.84:6831/2786189012","state":["exists","up"]},{"osd":10,"uuid":"b4938160-a808-4ded-91c2-d245c99fff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6825","nonce":1985398526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6827","nonce":1985398526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6831","nonce":1985398526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6829","nonce":1985398526}]},"public_addr":"172.21.15.96:6825/1985398526","cluster_addr":"172.21.15.96:6827/1985398526","heartbeat_back_addr":"172.21.15.96:6831/1985398526","heartbeat_front_addr":"172.21.15.96:6829/1985398526","state":["exists","up"]},{"osd":11,"uuid":"238922ef-d046-40cc-833d-25b7e7a523cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6825","nonce":3106116519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6827","nonce":3106116519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6831","nonce":3106116519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6829","nonce":3106116519}]},"public_addr":"172.21.15.204:6825/3106116519","cluster_addr":"172.21.15.204:6827/3106116519","heartbeat_back_addr":"172.21.15.204:6831/3106116519","heartbeat_front_addr":"172.21.15.204:6829/3106116519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:16.443480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:39.898133+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:03.664544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:28.838687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:53.829733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:19.269307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:45.166910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:10.771614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:36.678516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:03.150592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:29.327769+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:56.315938+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2317224739":"2024-05-23T23:01:49.869856+0000","172.21.15.84:0/4170743349":"2024-05-23T23:01:49.869856+0000","172.21.15.84:6801/4223589140":"2024-05-23T23:01:49.869856+0000","172.21.15.84:6800/4223589140":"2024-05-23T23:01:49.869856+0000","172.21.15.84:6801/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:6800/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/2766288608":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/950455999":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/3490117794":"2024-05-23T23:01:49.869856+0000","172.21.15.84:0/4000289608":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1650467363":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/4077200298":"2024-05-23T23:01:49.869856+0000","172.21.15.84:0/1052746114":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6800/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6801/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/14610813":"2024-05-23T22:52:45.395791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T23:12:07.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.987+0000 7f68bfb3b640 1 -- 172.21.15.84:0/2640761320 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f68a00a2d00 msgr2=0x7f68a00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:07.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.987+0000 7f68bfb3b640 1 --2- 172.21.15.84:0/2640761320 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f68a00a2d00 0x7f68a00a51c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f68b000cac0 tx=0x7f68b000c4a0 comp rx=0 tx=0).stop 2024-05-22T23:12:07.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.987+0000 7f68bfb3b640 1 -- 172.21.15.84:0/2640761320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68b8101180 msgr2=0x7f68b81974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:07.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.987+0000 7f68bfb3b640 1 --2- 172.21.15.84:0/2640761320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68b8101180 0x7f68b81974f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f68ac02ae00 tx=0x7f68ac003970 comp rx=0 tx=0).stop 2024-05-22T23:12:07.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.987+0000 7f68bfb3b640 1 -- 172.21.15.84:0/2640761320 shutdown_connections 2024-05-22T23:12:07.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.987+0000 7f68bfb3b640 1 --2- 172.21.15.84:0/2640761320 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f68a00a2d00 0x7f68a00a51c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:07.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.987+0000 7f68bfb3b640 1 --2- 172.21.15.84:0/2640761320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b8106780 0x7f68b819b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:07.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.987+0000 7f68bfb3b640 1 --2- 172.21.15.84:0/2640761320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68b8101180 0x7f68b81974f0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:07.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.987+0000 7f68bfb3b640 1 --2- 172.21.15.84:0/2640761320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68b81007c0 0x7f68b8196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:07.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.987+0000 7f68bfb3b640 1 -- 172.21.15.84:0/2640761320 >> 172.21.15.84:0/2640761320 conn(0x7f68b80778d0 msgr2=0x7f68b81032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:07.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.987+0000 7f68bfb3b640 1 -- 172.21.15.84:0/2640761320 shutdown_connections 2024-05-22T23:12:07.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:07.987+0000 7f68bfb3b640 1 -- 172.21.15.84:0/2640761320 wait complete. 2024-05-22T23:12:08.003 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:07 smithi084 ceph-mon[32186]: osdmap e98: 12 total, 12 up, 12 in 2024-05-22T23:12:08.005 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T22:55:07.719394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T22:59:50.150511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T22:59:51.111182+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 98, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T23:12:08.005 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T23:12:08.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.103+0000 7fcd6d914640 1 -- 172.21.15.84:0/122811921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd68102af0 msgr2=0x7fcd68102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:08.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.103+0000 7fcd6d914640 1 --2- 172.21.15.84:0/122811921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd68102af0 0x7fcd68102ed0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009ca0 tx=0x7fcd5802a9e0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.103+0000 7fcd6d914640 1 -- 172.21.15.84:0/122811921 shutdown_connections 2024-05-22T23:12:08.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.103+0000 7fcd6d914640 1 --2- 172.21.15.84:0/122811921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd6810be30 0x7fcd6810e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.103+0000 7fcd6d914640 1 --2- 172.21.15.84:0/122811921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd68103410 0x7fcd6810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.103+0000 7fcd6d914640 1 --2- 172.21.15.84:0/122811921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd68102af0 0x7fcd68102ed0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.103+0000 7fcd6d914640 1 -- 172.21.15.84:0/122811921 >> 172.21.15.84:0/122811921 conn(0x7fcd68077870 msgr2=0x7fcd68077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:08.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.104+0000 7fcd6d914640 1 -- 172.21.15.84:0/122811921 shutdown_connections 2024-05-22T23:12:08.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.104+0000 7fcd6d914640 1 -- 172.21.15.84:0/122811921 wait complete. 2024-05-22T23:12:08.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.104+0000 7fcd6d914640 1 Processor -- start 2024-05-22T23:12:08.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.104+0000 7fcd6d914640 1 -- start start 2024-05-22T23:12:08.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.104+0000 7fcd6d914640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd68102af0 0x7fcd68198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:08.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.104+0000 7fcd6d914640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd68103410 0x7fcd68198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:08.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.104+0000 7fcd6d914640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd6810be30 0x7fcd6819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:08.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.104+0000 7fcd6d914640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcd68111c30 con 0x7fcd6810be30 2024-05-22T23:12:08.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.104+0000 7fcd6d914640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcd68111ab0 con 0x7fcd68102af0 2024-05-22T23:12:08.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.104+0000 7fcd6d914640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcd68111db0 con 0x7fcd68103410 2024-05-22T23:12:08.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.105+0000 7fcd677fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd6810be30 0x7fcd6819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:08.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.105+0000 7fcd677fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd6810be30 0x7fcd6819c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49250/0 (socket says 172.21.15.84:49250) 2024-05-22T23:12:08.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.105+0000 7fcd677fe640 1 -- 172.21.15.84:0/2821206781 learned_addr learned my addr 172.21.15.84:0/2821206781 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:08.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.105+0000 7fcd667fc640 1 --2- 172.21.15.84:0/2821206781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd68103410 0x7fcd68198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:08.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.105+0000 7fcd66ffd640 1 --2- 172.21.15.84:0/2821206781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd68102af0 0x7fcd68198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:08.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.105+0000 7fcd677fe640 1 -- 172.21.15.84:0/2821206781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd68103410 msgr2=0x7fcd68198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:08.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.105+0000 7fcd677fe640 1 --2- 172.21.15.84:0/2821206781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd68103410 0x7fcd68198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.105+0000 7fcd677fe640 1 -- 172.21.15.84:0/2821206781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd68102af0 msgr2=0x7fcd68198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:08.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.105+0000 7fcd677fe640 1 --2- 172.21.15.84:0/2821206781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd68102af0 0x7fcd68198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.105+0000 7fcd677fe640 1 -- 172.21.15.84:0/2821206781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6819d0d0 con 0x7fcd6810be30 2024-05-22T23:12:08.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.105+0000 7fcd66ffd640 1 --2- 172.21.15.84:0/2821206781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd68102af0 0x7fcd68198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:12:08.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.105+0000 7fcd677fe640 1 --2- 172.21.15.84:0/2821206781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd6810be30 0x7fcd6819c9a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000c970 tx=0x7fcd6000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:08.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.105+0000 7fcd6c912640 1 -- 172.21.15.84:0/2821206781 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60007bb0 con 0x7fcd6810be30 2024-05-22T23:12:08.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.105+0000 7fcd6c912640 1 -- 172.21.15.84:0/2821206781 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd60007d50 con 0x7fcd6810be30 2024-05-22T23:12:08.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.105+0000 7fcd6d914640 1 -- 172.21.15.84:0/2821206781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd6819d3c0 con 0x7fcd6810be30 2024-05-22T23:12:08.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.105+0000 7fcd6c912640 1 -- 172.21.15.84:0/2821206781 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60018800 con 0x7fcd6810be30 2024-05-22T23:12:08.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.105+0000 7fcd6d914640 1 -- 172.21.15.84:0/2821206781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd681a4c50 con 0x7fcd6810be30 2024-05-22T23:12:08.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.106+0000 7fcd6d914640 1 -- 172.21.15.84:0/2821206781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd68068690 con 0x7fcd6810be30 2024-05-22T23:12:08.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.107+0000 7fcd6c912640 1 -- 172.21.15.84:0/2821206781 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcd6000fa50 con 0x7fcd6810be30 2024-05-22T23:12:08.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.108+0000 7fcd6c912640 1 --2- 172.21.15.84:0/2821206781 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fcd500a2d00 0x7fcd500a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:08.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.108+0000 7fcd66ffd640 1 --2- 172.21.15.84:0/2821206781 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fcd500a2d00 0x7fcd500a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:08.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.108+0000 7fcd6c912640 1 -- 172.21.15.84:0/2821206781 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8615+0+0 (secure 0 0 0) 0x7fcd600cf540 con 0x7fcd6810be30 2024-05-22T23:12:08.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.108+0000 7fcd66ffd640 1 --2- 172.21.15.84:0/2821206781 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fcd500a2d00 0x7fcd500a51c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009ca0 tx=0x7fcd58005f80 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:08.118 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.112+0000 7fcd6c912640 1 -- 172.21.15.84:0/2821206781 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcd60014030 con 0x7fcd6810be30 2024-05-22T23:12:08.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:07 smithi204 ceph-mon[36211]: osdmap e98: 12 total, 12 up, 12 in 2024-05-22T23:12:08.295 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.288+0000 7fcd6d914640 1 -- 172.21.15.84:0/2821206781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcd6802d200 con 0x7fcd6810be30 2024-05-22T23:12:08.295 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.288+0000 7fcd6c912640 1 -- 172.21.15.84:0/2821206781 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7fcd6009c100 con 0x7fcd6810be30 2024-05-22T23:12:08.295 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2024-05-22T23:12:08.297 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.290+0000 7fcd6d914640 1 -- 172.21.15.84:0/2821206781 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fcd500a2d00 msgr2=0x7fcd500a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:08.297 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.290+0000 7fcd6d914640 1 --2- 172.21.15.84:0/2821206781 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fcd500a2d00 0x7fcd500a51c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009ca0 tx=0x7fcd58005f80 comp rx=0 tx=0).stop 2024-05-22T23:12:08.297 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.290+0000 7fcd6d914640 1 -- 172.21.15.84:0/2821206781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd6810be30 msgr2=0x7fcd6819c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:08.297 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.290+0000 7fcd6d914640 1 --2- 172.21.15.84:0/2821206781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd6810be30 0x7fcd6819c9a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000c970 tx=0x7fcd6000ce30 comp rx=0 tx=0).stop 2024-05-22T23:12:08.297 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.290+0000 7fcd6d914640 1 -- 172.21.15.84:0/2821206781 shutdown_connections 2024-05-22T23:12:08.298 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.290+0000 7fcd6d914640 1 --2- 172.21.15.84:0/2821206781 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fcd500a2d00 0x7fcd500a51c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.298 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.290+0000 7fcd6d914640 1 --2- 172.21.15.84:0/2821206781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd6810be30 0x7fcd6819c9a0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.298 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.290+0000 7fcd6d914640 1 --2- 172.21.15.84:0/2821206781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd68103410 0x7fcd68198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.298 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.290+0000 7fcd6d914640 1 --2- 172.21.15.84:0/2821206781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd68102af0 0x7fcd68198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.298 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.290+0000 7fcd6d914640 1 -- 172.21.15.84:0/2821206781 >> 172.21.15.84:0/2821206781 conn(0x7fcd68077870 msgr2=0x7fcd680fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:08.298 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.290+0000 7fcd6d914640 1 -- 172.21.15.84:0/2821206781 shutdown_connections 2024-05-22T23:12:08.298 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.291+0000 7fcd6d914640 1 -- 172.21.15.84:0/2821206781 wait complete. 2024-05-22T23:12:08.305 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:07 smithi096 ceph-mon[36392]: osdmap e98: 12 total, 12 up, 12 in 2024-05-22T23:12:08.308 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T23:12:08.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.405+0000 7f72b0f84640 1 -- 172.21.15.84:0/1740172646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72ac106750 msgr2=0x7f72ac106b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:08.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.405+0000 7f72b0f84640 1 --2- 172.21.15.84:0/1740172646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72ac106750 0x7f72ac106b30 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f729c009930 tx=0x7f729c02a420 comp rx=0 tx=0).stop 2024-05-22T23:12:08.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.405+0000 7f72b0f84640 1 -- 172.21.15.84:0/1740172646 shutdown_connections 2024-05-22T23:12:08.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.405+0000 7f72b0f84640 1 --2- 172.21.15.84:0/1740172646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72ac101100 0x7f72ac10d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.405+0000 7f72b0f84640 1 --2- 172.21.15.84:0/1740172646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72ac100740 0x7f72ac100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.405+0000 7f72b0f84640 1 --2- 172.21.15.84:0/1740172646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72ac106750 0x7f72ac106b30 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.405+0000 7f72b0f84640 1 -- 172.21.15.84:0/1740172646 >> 172.21.15.84:0/1740172646 conn(0x7f72ac077870 msgr2=0x7f72ac077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:08.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.405+0000 7f72b0f84640 1 -- 172.21.15.84:0/1740172646 shutdown_connections 2024-05-22T23:12:08.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.405+0000 7f72b0f84640 1 -- 172.21.15.84:0/1740172646 wait complete. 2024-05-22T23:12:08.415 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.406+0000 7f72b0f84640 1 Processor -- start 2024-05-22T23:12:08.415 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.406+0000 7f72b0f84640 1 -- start start 2024-05-22T23:12:08.415 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.406+0000 7f72b0f84640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72ac100740 0x7f72ac197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:08.415 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.406+0000 7f72b0f84640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72ac101100 0x7f72ac1975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:08.415 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.406+0000 7f72b0f84640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72ac106750 0x7f72ac19b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:08.415 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.406+0000 7f72b0f84640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac110b20 con 0x7f72ac101100 2024-05-22T23:12:08.415 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.406+0000 7f72b0f84640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac1109a0 con 0x7f72ac100740 2024-05-22T23:12:08.415 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.406+0000 7f72b0f84640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac110ca0 con 0x7f72ac106750 2024-05-22T23:12:08.416 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.406+0000 7f72a3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72ac101100 0x7f72ac1975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:08.416 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.406+0000 7f72a3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72ac101100 0x7f72ac1975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49264/0 (socket says 172.21.15.84:49264) 2024-05-22T23:12:08.416 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.406+0000 7f72a3fff640 1 -- 172.21.15.84:0/3316047529 learned_addr learned my addr 172.21.15.84:0/3316047529 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:08.416 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.406+0000 7f72aad9d640 1 --2- 172.21.15.84:0/3316047529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72ac106750 0x7f72ac19b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:08.416 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.406+0000 7f72a3fff640 1 -- 172.21.15.84:0/3316047529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72ac106750 msgr2=0x7f72ac19b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:08.416 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.406+0000 7f72a3fff640 1 --2- 172.21.15.84:0/3316047529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72ac106750 0x7f72ac19b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.416 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.406+0000 7f72a3fff640 1 -- 172.21.15.84:0/3316047529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72ac100740 msgr2=0x7f72ac197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:08.416 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.406+0000 7f72a3fff640 1 --2- 172.21.15.84:0/3316047529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72ac100740 0x7f72ac197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.406+0000 7f72a3fff640 1 -- 172.21.15.84:0/3316047529 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ac19c050 con 0x7f72ac101100 2024-05-22T23:12:08.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.406+0000 7f72a3fff640 1 --2- 172.21.15.84:0/3316047529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72ac101100 0x7f72ac1975a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7294009970 tx=0x7f7294009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:08.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.406+0000 7f72a37fe640 1 -- 172.21.15.84:0/3316047529 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729400d020 con 0x7f72ac101100 2024-05-22T23:12:08.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.406+0000 7f72b0f84640 1 -- 172.21.15.84:0/3316047529 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72ac19c340 con 0x7f72ac101100 2024-05-22T23:12:08.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.406+0000 7f72a37fe640 1 -- 172.21.15.84:0/3316047529 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7294007d40 con 0x7f72ac101100 2024-05-22T23:12:08.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.407+0000 7f72a37fe640 1 -- 172.21.15.84:0/3316047529 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729400ccc0 con 0x7f72ac101100 2024-05-22T23:12:08.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.407+0000 7f72b0f84640 1 -- 172.21.15.84:0/3316047529 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72ac068a00 con 0x7f72ac101100 2024-05-22T23:12:08.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.407+0000 7f72b0f84640 1 -- 172.21.15.84:0/3316047529 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72ac068690 con 0x7f72ac101100 2024-05-22T23:12:08.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.409+0000 7f72a37fe640 1 -- 172.21.15.84:0/3316047529 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f729400c460 con 0x7f72ac101100 2024-05-22T23:12:08.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.409+0000 7f72a37fe640 1 --2- 172.21.15.84:0/3316047529 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f72980a2d00 0x7f72980a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:08.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.410+0000 7f72aa59c640 1 --2- 172.21.15.84:0/3316047529 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f72980a2d00 0x7f72980a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:08.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.410+0000 7f72a37fe640 1 -- 172.21.15.84:0/3316047529 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8615+0+0 (secure 0 0 0) 0x7f72940d09f0 con 0x7f72ac101100 2024-05-22T23:12:08.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.410+0000 7f72aa59c640 1 --2- 172.21.15.84:0/3316047529 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f72980a2d00 0x7f72980a51c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f729c02a940 tx=0x7f729c000bc0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:08.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.414+0000 7f72a37fe640 1 -- 172.21.15.84:0/3316047529 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f729409d5b0 con 0x7f72ac101100 2024-05-22T23:12:08.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.589+0000 7f72b0f84640 1 -- 172.21.15.84:0/3316047529 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f72ac069530 con 0x7f72ac101100 2024-05-22T23:12:08.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.589+0000 7f72a37fe640 1 -- 172.21.15.84:0/3316047529 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) ==== 104+0+11 (secure 0 0 0) 0x7f729409ceb0 con 0x7f72ac101100 2024-05-22T23:12:08.596 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2024-05-22T23:12:08.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.591+0000 7f72b0f84640 1 -- 172.21.15.84:0/3316047529 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f72980a2d00 msgr2=0x7f72980a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:08.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.591+0000 7f72b0f84640 1 --2- 172.21.15.84:0/3316047529 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f72980a2d00 0x7f72980a51c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f729c02a940 tx=0x7f729c000bc0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.591+0000 7f72b0f84640 1 -- 172.21.15.84:0/3316047529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72ac101100 msgr2=0x7f72ac1975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:08.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.591+0000 7f72b0f84640 1 --2- 172.21.15.84:0/3316047529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72ac101100 0x7f72ac1975a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7294009970 tx=0x7f7294009e30 comp rx=0 tx=0).stop 2024-05-22T23:12:08.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.591+0000 7f72b0f84640 1 -- 172.21.15.84:0/3316047529 shutdown_connections 2024-05-22T23:12:08.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.591+0000 7f72b0f84640 1 --2- 172.21.15.84:0/3316047529 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f72980a2d00 0x7f72980a51c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.591+0000 7f72b0f84640 1 --2- 172.21.15.84:0/3316047529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72ac106750 0x7f72ac19b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.591+0000 7f72b0f84640 1 --2- 172.21.15.84:0/3316047529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72ac101100 0x7f72ac1975a0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.591+0000 7f72b0f84640 1 --2- 172.21.15.84:0/3316047529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72ac100740 0x7f72ac197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.591+0000 7f72b0f84640 1 -- 172.21.15.84:0/3316047529 >> 172.21.15.84:0/3316047529 conn(0x7f72ac077870 msgr2=0x7f72ac103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:08.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.591+0000 7f72b0f84640 1 -- 172.21.15.84:0/3316047529 shutdown_connections 2024-05-22T23:12:08.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.591+0000 7f72b0f84640 1 -- 172.21.15.84:0/3316047529 wait complete. 2024-05-22T23:12:08.609 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T23:12:08.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.706+0000 7f1159a88640 1 -- 172.21.15.84:0/2186570398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1154100750 msgr2=0x7f1154100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:08.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.706+0000 7f1159a88640 1 --2- 172.21.15.84:0/2186570398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1154100750 0x7f1154100bd0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f113c009930 tx=0x7f113c02a420 comp rx=0 tx=0).stop 2024-05-22T23:12:08.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.706+0000 7f1159a88640 1 -- 172.21.15.84:0/2186570398 shutdown_connections 2024-05-22T23:12:08.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.706+0000 7f1159a88640 1 --2- 172.21.15.84:0/2186570398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1154101110 0x7f115410d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.706+0000 7f1159a88640 1 --2- 172.21.15.84:0/2186570398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1154100750 0x7f1154100bd0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.706+0000 7f1159a88640 1 --2- 172.21.15.84:0/2186570398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1154106760 0x7f1154106b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.706+0000 7f1159a88640 1 -- 172.21.15.84:0/2186570398 >> 172.21.15.84:0/2186570398 conn(0x7f1154077900 msgr2=0x7f1154077d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:08.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.706+0000 7f1159a88640 1 -- 172.21.15.84:0/2186570398 shutdown_connections 2024-05-22T23:12:08.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.706+0000 7f1159a88640 1 -- 172.21.15.84:0/2186570398 wait complete. 2024-05-22T23:12:08.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.707+0000 7f1159a88640 1 Processor -- start 2024-05-22T23:12:08.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.707+0000 7f1159a88640 1 -- start start 2024-05-22T23:12:08.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.707+0000 7f1159a88640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1154100750 0x7f1154198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:08.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.707+0000 7f1159a88640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1154101110 0x7f1154199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:08.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.707+0000 7f1159a88640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1154106760 0x7f11541a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:08.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.707+0000 7f1159a88640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1154110b30 con 0x7f1154101110 2024-05-22T23:12:08.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.707+0000 7f1159a88640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f11541109b0 con 0x7f1154100750 2024-05-22T23:12:08.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.707+0000 7f1159a88640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1154110cb0 con 0x7f1154106760 2024-05-22T23:12:08.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.707+0000 7f114bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1154101110 0x7f1154199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:08.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.707+0000 7f114bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1154101110 0x7f1154199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49274/0 (socket says 172.21.15.84:49274) 2024-05-22T23:12:08.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.707+0000 7f114bfff640 1 -- 172.21.15.84:0/2916666737 learned_addr learned my addr 172.21.15.84:0/2916666737 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:08.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.707+0000 7f11537fe640 1 --2- 172.21.15.84:0/2916666737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1154100750 0x7f1154198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:08.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.707+0000 7f1153fff640 1 --2- 172.21.15.84:0/2916666737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1154106760 0x7f11541a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:08.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.707+0000 7f114bfff640 1 -- 172.21.15.84:0/2916666737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1154106760 msgr2=0x7f11541a3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:08.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.707+0000 7f114bfff640 1 --2- 172.21.15.84:0/2916666737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1154106760 0x7f11541a3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.707+0000 7f114bfff640 1 -- 172.21.15.84:0/2916666737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1154100750 msgr2=0x7f1154198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:08.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.707+0000 7f114bfff640 1 --2- 172.21.15.84:0/2916666737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1154100750 0x7f1154198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.707+0000 7f114bfff640 1 -- 172.21.15.84:0/2916666737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11541a4510 con 0x7f1154101110 2024-05-22T23:12:08.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.707+0000 7f114bfff640 1 --2- 172.21.15.84:0/2916666737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1154101110 0x7f1154199490 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f113c009230 tx=0x7f113c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:08.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.708+0000 7f11517fa640 1 -- 172.21.15.84:0/2916666737 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c03f030 con 0x7f1154101110 2024-05-22T23:12:08.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.708+0000 7f11517fa640 1 -- 172.21.15.84:0/2916666737 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f113c003ad0 con 0x7f1154101110 2024-05-22T23:12:08.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.708+0000 7f1159a88640 1 -- 172.21.15.84:0/2916666737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11541a46e0 con 0x7f1154101110 2024-05-22T23:12:08.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.708+0000 7f11517fa640 1 -- 172.21.15.84:0/2916666737 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c035070 con 0x7f1154101110 2024-05-22T23:12:08.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.708+0000 7f1159a88640 1 -- 172.21.15.84:0/2916666737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11541a4b40 con 0x7f1154101110 2024-05-22T23:12:08.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.709+0000 7f1159a88640 1 -- 172.21.15.84:0/2916666737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1154068690 con 0x7f1154101110 2024-05-22T23:12:08.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.710+0000 7f11517fa640 1 -- 172.21.15.84:0/2916666737 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f113c042030 con 0x7f1154101110 2024-05-22T23:12:08.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.710+0000 7f11517fa640 1 --2- 172.21.15.84:0/2916666737 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f11400a2d00 0x7f11400a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:08.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.710+0000 7f11537fe640 1 --2- 172.21.15.84:0/2916666737 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f11400a2d00 0x7f11400a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:08.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.710+0000 7f11517fa640 1 -- 172.21.15.84:0/2916666737 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8615+0+0 (secure 0 0 0) 0x7f113c0efb40 con 0x7f1154101110 2024-05-22T23:12:08.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.711+0000 7f11537fe640 1 --2- 172.21.15.84:0/2916666737 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f11400a2d00 0x7f11400a51c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f114400ab60 tx=0x7f114400a2d0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:08.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.715+0000 7f11517fa640 1 -- 172.21.15.84:0/2916666737 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f113c0bc700 con 0x7f1154101110 2024-05-22T23:12:08.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.891+0000 7f1159a88640 1 -- 172.21.15.84:0/2916666737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1154108ce0 con 0x7f1154101110 2024-05-22T23:12:08.898 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.892+0000 7f11517fa640 1 -- 172.21.15.84:0/2916666737 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) ==== 100+0+11 (secure 0 0 0) 0x7f113c0bc000 con 0x7f1154101110 2024-05-22T23:12:08.899 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2024-05-22T23:12:08.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.894+0000 7f1159a88640 1 -- 172.21.15.84:0/2916666737 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f11400a2d00 msgr2=0x7f11400a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:08.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.894+0000 7f1159a88640 1 --2- 172.21.15.84:0/2916666737 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f11400a2d00 0x7f11400a51c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f114400ab60 tx=0x7f114400a2d0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.894+0000 7f1159a88640 1 -- 172.21.15.84:0/2916666737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1154101110 msgr2=0x7f1154199490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:08.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.894+0000 7f1159a88640 1 --2- 172.21.15.84:0/2916666737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1154101110 0x7f1154199490 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f113c009230 tx=0x7f113c004270 comp rx=0 tx=0).stop 2024-05-22T23:12:08.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.894+0000 7f1159a88640 1 -- 172.21.15.84:0/2916666737 shutdown_connections 2024-05-22T23:12:08.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.894+0000 7f1159a88640 1 --2- 172.21.15.84:0/2916666737 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f11400a2d00 0x7f11400a51c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.894+0000 7f1159a88640 1 --2- 172.21.15.84:0/2916666737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1154106760 0x7f11541a3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.894+0000 7f1159a88640 1 --2- 172.21.15.84:0/2916666737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1154101110 0x7f1154199490 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.894+0000 7f1159a88640 1 --2- 172.21.15.84:0/2916666737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1154100750 0x7f1154198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:08.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.894+0000 7f1159a88640 1 -- 172.21.15.84:0/2916666737 >> 172.21.15.84:0/2916666737 conn(0x7f1154077900 msgr2=0x7f1154102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:08.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.894+0000 7f1159a88640 1 -- 172.21.15.84:0/2916666737 shutdown_connections 2024-05-22T23:12:08.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:08.894+0000 7f1159a88640 1 -- 172.21.15.84:0/2916666737 wait complete. 2024-05-22T23:12:08.914 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-22T23:12:09.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.009+0000 7fde6e2d2640 1 -- 172.21.15.84:0/3329424539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde6810be90 msgr2=0x7fde6810e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:09.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.009+0000 7fde6e2d2640 1 --2- 172.21.15.84:0/3329424539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde6810be90 0x7fde6810e300 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fde60009ca0 tx=0x7fde6002a950 comp rx=0 tx=0).stop 2024-05-22T23:12:09.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.009+0000 7fde6e2d2640 1 -- 172.21.15.84:0/3329424539 shutdown_connections 2024-05-22T23:12:09.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.009+0000 7fde6e2d2640 1 --2- 172.21.15.84:0/3329424539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde6810be90 0x7fde6810e300 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:09.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.009+0000 7fde6e2d2640 1 --2- 172.21.15.84:0/3329424539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde68103470 0x7fde6810b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:09.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.009+0000 7fde6e2d2640 1 --2- 172.21.15.84:0/3329424539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde68102b50 0x7fde68102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:09.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.009+0000 7fde6e2d2640 1 -- 172.21.15.84:0/3329424539 >> 172.21.15.84:0/3329424539 conn(0x7fde680778d0 msgr2=0x7fde68077ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:09.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.009+0000 7fde6e2d2640 1 -- 172.21.15.84:0/3329424539 shutdown_connections 2024-05-22T23:12:09.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.009+0000 7fde6e2d2640 1 -- 172.21.15.84:0/3329424539 wait complete. 2024-05-22T23:12:09.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.009+0000 7fde6e2d2640 1 Processor -- start 2024-05-22T23:12:09.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.009+0000 7fde6e2d2640 1 -- start start 2024-05-22T23:12:09.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.009+0000 7fde6e2d2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde68102b50 0x7fde68198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:09.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.009+0000 7fde6e2d2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde68103470 0x7fde68198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:09.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.009+0000 7fde6e2d2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde6810be90 0x7fde6819c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:09.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.009+0000 7fde6e2d2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fde68111c60 con 0x7fde68102b50 2024-05-22T23:12:09.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.009+0000 7fde6e2d2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fde68111ae0 con 0x7fde6810be90 2024-05-22T23:12:09.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.009+0000 7fde6e2d2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fde68111de0 con 0x7fde68103470 2024-05-22T23:12:09.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.010+0000 7fde67fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde68102b50 0x7fde68198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:09.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.010+0000 7fde67fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde68102b50 0x7fde68198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49290/0 (socket says 172.21.15.84:49290) 2024-05-22T23:12:09.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.010+0000 7fde67fff640 1 -- 172.21.15.84:0/3616890991 learned_addr learned my addr 172.21.15.84:0/3616890991 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:09.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.010+0000 7fde5ffff640 1 --2- 172.21.15.84:0/3616890991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde68103470 0x7fde68198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:09.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.010+0000 7fde67fff640 1 -- 172.21.15.84:0/3616890991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde68103470 msgr2=0x7fde68198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:09.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.010+0000 7fde67fff640 1 --2- 172.21.15.84:0/3616890991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde68103470 0x7fde68198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:09.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.010+0000 7fde67fff640 1 -- 172.21.15.84:0/3616890991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde6810be90 msgr2=0x7fde6819c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:09.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.010+0000 7fde67fff640 1 --2- 172.21.15.84:0/3616890991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde6810be90 0x7fde6819c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:09.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.010+0000 7fde67fff640 1 -- 172.21.15.84:0/3616890991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde6819d0c0 con 0x7fde68102b50 2024-05-22T23:12:09.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.010+0000 7fde67fff640 1 --2- 172.21.15.84:0/3616890991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde68102b50 0x7fde68198120 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fde5800d950 tx=0x7fde5800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:09.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.010+0000 7fde65ffb640 1 -- 172.21.15.84:0/3616890991 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde58017030 con 0x7fde68102b50 2024-05-22T23:12:09.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.010+0000 7fde65ffb640 1 -- 172.21.15.84:0/3616890991 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde58004020 con 0x7fde68102b50 2024-05-22T23:12:09.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.010+0000 7fde6e2d2640 1 -- 172.21.15.84:0/3616890991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde6819d3b0 con 0x7fde68102b50 2024-05-22T23:12:09.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.010+0000 7fde65ffb640 1 -- 172.21.15.84:0/3616890991 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde580047a0 con 0x7fde68102b50 2024-05-22T23:12:09.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.010+0000 7fde6e2d2640 1 -- 172.21.15.84:0/3616890991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde681a4c40 con 0x7fde68102b50 2024-05-22T23:12:09.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.011+0000 7fde6e2d2640 1 -- 172.21.15.84:0/3616890991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde68068ae0 con 0x7fde68102b50 2024-05-22T23:12:09.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.012+0000 7fde65ffb640 1 -- 172.21.15.84:0/3616890991 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fde580041d0 con 0x7fde68102b50 2024-05-22T23:12:09.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.013+0000 7fde65ffb640 1 --2- 172.21.15.84:0/3616890991 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fde540a2d00 0x7fde540a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:09.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.013+0000 7fde5ffff640 1 --2- 172.21.15.84:0/3616890991 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fde540a2d00 0x7fde540a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:09.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.013+0000 7fde65ffb640 1 -- 172.21.15.84:0/3616890991 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8615+0+0 (secure 0 0 0) 0x7fde580cef30 con 0x7fde68102b50 2024-05-22T23:12:09.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.014+0000 7fde5ffff640 1 --2- 172.21.15.84:0/3616890991 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fde540a2d00 0x7fde540a51c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fde50005fd0 tx=0x7fde5000b510 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:09.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.018+0000 7fde65ffb640 1 -- 172.21.15.84:0/3616890991 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fde5809baf0 con 0x7fde68102b50 2024-05-22T23:12:09.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:08 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2640761320' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:12:09.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:08 smithi204 ceph-mon[36211]: pgmap v321: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-05-22T23:12:09.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:08 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2821206781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:12:09.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:08 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3316047529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:12:09.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:08 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2916666737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:12:09.249 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.243+0000 7fde6e2d2640 1 -- 172.21.15.84:0/3616890991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fde681a4f90 con 0x7fde68102b50 2024-05-22T23:12:09.257 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.251+0000 7fde65ffb640 1 -- 172.21.15.84:0/3616890991 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+740902 (secure 0 0 0) 0x7fde5809b3f0 con 0x7fde68102b50 2024-05-22T23:12:09.267 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:12:09.267 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:12:09.267 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 25, 2024-05-22T23:12:09.268 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.268 INFO:teuthology.orchestra.run.smithi084.stdout: "active_gid": 24851, 2024-05-22T23:12:09.268 INFO:teuthology.orchestra.run.smithi084.stdout: "active_name": "x", 2024-05-22T23:12:09.268 INFO:teuthology.orchestra.run.smithi084.stdout: "active_addrs": { 2024-05-22T23:12:09.268 INFO:teuthology.orchestra.run.smithi084.stdout: "addrvec": [ 2024-05-22T23:12:09.268 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.268 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v2", 2024-05-22T23:12:09.268 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "172.21.15.84:6800", 2024-05-22T23:12:09.269 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 3358259585 2024-05-22T23:12:09.269 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.269 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.269 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2024-05-22T23:12:09.269 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "172.21.15.84:6801", 2024-05-22T23:12:09.269 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 3358259585 2024-05-22T23:12:09.269 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.269 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-05-22T23:12:09.270 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.270 INFO:teuthology.orchestra.run.smithi084.stdout: "active_addr": "172.21.15.84:6801/3358259585", 2024-05-22T23:12:09.270 INFO:teuthology.orchestra.run.smithi084.stdout: "active_change": "2024-05-22T23:01:49.870066+0000", 2024-05-22T23:12:09.270 INFO:teuthology.orchestra.run.smithi084.stdout: "active_mgr_features": 4540701547738038271, 2024-05-22T23:12:09.270 INFO:teuthology.orchestra.run.smithi084.stdout: "available": true, 2024-05-22T23:12:09.270 INFO:teuthology.orchestra.run.smithi084.stdout: "standbys": [ 2024-05-22T23:12:09.270 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.270 INFO:teuthology.orchestra.run.smithi084.stdout: "gid": 24760, 2024-05-22T23:12:09.271 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "z", 2024-05-22T23:12:09.271 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr_features": 4540701547738038271, 2024-05-22T23:12:09.271 INFO:teuthology.orchestra.run.smithi084.stdout: "available_modules": [ 2024-05-22T23:12:09.271 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.271 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "alerts", 2024-05-22T23:12:09.271 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.271 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.271 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.272 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-05-22T23:12:09.272 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-05-22T23:12:09.272 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:09.272 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.272 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.272 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-05-22T23:12:09.272 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.272 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.273 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.273 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "How frequently to reexamine health status", 2024-05-22T23:12:09.273 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.273 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.273 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.273 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.273 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.273 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.274 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.274 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.274 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.274 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.274 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.274 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.274 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.274 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.275 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.275 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.275 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.275 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.275 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.275 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.275 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.275 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.276 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.276 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.276 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.276 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.276 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.276 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.276 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.277 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.277 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.277 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.277 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.277 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.277 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.277 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.277 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.277 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.278 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.278 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.278 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.278 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.278 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.278 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.278 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.278 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.279 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.279 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.279 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.279 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.279 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.279 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.279 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.279 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.279 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.280 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.280 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.280 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.280 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.280 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.280 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.280 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.280 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.281 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.281 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.281 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.281 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.281 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.281 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.281 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.281 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.282 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.282 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.282 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.282 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_destination": { 2024-05-22T23:12:09.282 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_destination", 2024-05-22T23:12:09.282 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.282 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.282 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.283 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.283 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.283 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.283 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.283 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Email address to send alerts to", 2024-05-22T23:12:09.283 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.283 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.283 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.283 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.284 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_from_name": { 2024-05-22T23:12:09.284 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_from_name", 2024-05-22T23:12:09.284 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.284 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.284 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.284 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "Ceph", 2024-05-22T23:12:09.284 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.284 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.285 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.285 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Email From: name", 2024-05-22T23:12:09.285 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.285 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.285 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.285 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.285 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_host": { 2024-05-22T23:12:09.285 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_host", 2024-05-22T23:12:09.286 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.286 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.286 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.286 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.286 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.286 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.286 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.286 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "SMTP server", 2024-05-22T23:12:09.287 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.287 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.287 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.287 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.287 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_password": { 2024-05-22T23:12:09.287 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_password", 2024-05-22T23:12:09.287 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.287 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.288 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.288 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.288 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.288 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.288 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.288 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Password to authenticate with", 2024-05-22T23:12:09.288 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.288 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.289 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.289 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.289 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_port": { 2024-05-22T23:12:09.289 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_port", 2024-05-22T23:12:09.289 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.289 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.289 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.289 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "465", 2024-05-22T23:12:09.289 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.290 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.290 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.290 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "SMTP port", 2024-05-22T23:12:09.290 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.290 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.290 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.290 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.290 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_sender": { 2024-05-22T23:12:09.291 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_sender", 2024-05-22T23:12:09.291 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.291 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.291 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.291 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.291 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.291 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.291 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.292 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "SMTP envelope sender", 2024-05-22T23:12:09.292 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.292 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.292 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.292 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.292 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_ssl": { 2024-05-22T23:12:09.292 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_ssl", 2024-05-22T23:12:09.292 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.293 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.293 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.293 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.293 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.293 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.293 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.293 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-22T23:12:09.293 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.294 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.294 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.294 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.294 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_user": { 2024-05-22T23:12:09.294 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_user", 2024-05-22T23:12:09.294 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.294 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.294 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.295 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.295 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.295 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.295 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.295 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "User to authenticate as", 2024-05-22T23:12:09.295 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.295 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.295 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.295 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.296 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.296 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.296 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.296 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "balancer", 2024-05-22T23:12:09.296 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.296 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.296 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.296 INFO:teuthology.orchestra.run.smithi084.stdout: "active": { 2024-05-22T23:12:09.297 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "active", 2024-05-22T23:12:09.297 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.297 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.297 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.297 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.297 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.297 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.297 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.298 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "automatically balance PGs across cluster", 2024-05-22T23:12:09.298 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.298 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.298 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.298 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.298 INFO:teuthology.orchestra.run.smithi084.stdout: "begin_time": { 2024-05-22T23:12:09.298 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "begin_time", 2024-05-22T23:12:09.299 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.299 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.299 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.299 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0000", 2024-05-22T23:12:09.299 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.299 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.299 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.299 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "beginning time of day to automatically balance", 2024-05-22T23:12:09.300 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-22T23:12:09.300 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.300 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.300 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.300 INFO:teuthology.orchestra.run.smithi084.stdout: "begin_weekday": { 2024-05-22T23:12:09.300 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "begin_weekday", 2024-05-22T23:12:09.300 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-05-22T23:12:09.300 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.301 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.301 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-05-22T23:12:09.301 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "0", 2024-05-22T23:12:09.301 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "6", 2024-05-22T23:12:09.301 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.301 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-22T23:12:09.301 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-22T23:12:09.302 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.302 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.302 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.302 INFO:teuthology.orchestra.run.smithi084.stdout: "crush_compat_max_iterations": { 2024-05-22T23:12:09.302 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crush_compat_max_iterations", 2024-05-22T23:12:09.302 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-05-22T23:12:09.302 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.302 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.303 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "25", 2024-05-22T23:12:09.303 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1", 2024-05-22T23:12:09.303 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "250", 2024-05-22T23:12:09.303 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.303 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-22T23:12:09.303 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.303 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.304 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.304 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.304 INFO:teuthology.orchestra.run.smithi084.stdout: "crush_compat_metrics": { 2024-05-22T23:12:09.304 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crush_compat_metrics", 2024-05-22T23:12:09.304 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.304 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.304 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.304 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "pgs,objects,bytes", 2024-05-22T23:12:09.305 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.305 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.305 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.305 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-22T23:12:09.305 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-22T23:12:09.305 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.305 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.305 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.306 INFO:teuthology.orchestra.run.smithi084.stdout: "crush_compat_step": { 2024-05-22T23:12:09.306 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crush_compat_step", 2024-05-22T23:12:09.306 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-05-22T23:12:09.306 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.306 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.306 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0.5", 2024-05-22T23:12:09.306 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "0.001", 2024-05-22T23:12:09.307 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "0.999", 2024-05-22T23:12:09.307 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.307 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "aggressiveness of optimization", 2024-05-22T23:12:09.307 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-22T23:12:09.307 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.307 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.307 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.307 INFO:teuthology.orchestra.run.smithi084.stdout: "end_time": { 2024-05-22T23:12:09.308 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "end_time", 2024-05-22T23:12:09.308 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.308 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.308 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.308 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "2359", 2024-05-22T23:12:09.308 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.308 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.309 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.309 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "ending time of day to automatically balance", 2024-05-22T23:12:09.309 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-22T23:12:09.309 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.309 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.309 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.309 INFO:teuthology.orchestra.run.smithi084.stdout: "end_weekday": { 2024-05-22T23:12:09.309 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "end_weekday", 2024-05-22T23:12:09.310 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-05-22T23:12:09.310 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.310 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.310 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-05-22T23:12:09.310 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "0", 2024-05-22T23:12:09.310 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "6", 2024-05-22T23:12:09.310 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.310 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-22T23:12:09.311 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-22T23:12:09.311 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.311 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.311 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.311 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.311 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.311 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.312 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.312 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.312 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.312 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.312 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.312 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.312 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.312 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.313 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.313 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.313 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.313 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.313 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.313 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.313 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.313 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.314 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.314 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.314 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.314 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.314 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.314 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.314 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.314 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.315 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.315 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.315 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.315 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.315 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.315 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.315 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.316 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.316 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.316 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.316 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.316 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.317 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.317 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.317 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.317 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.317 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.317 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.317 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.317 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.318 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.318 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.318 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.318 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.318 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.318 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.318 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.319 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.319 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.319 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.319 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.319 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.319 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.319 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.319 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.320 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.320 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.320 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.320 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.320 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.320 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.320 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.320 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.321 INFO:teuthology.orchestra.run.smithi084.stdout: "min_score": { 2024-05-22T23:12:09.321 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "min_score", 2024-05-22T23:12:09.321 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-05-22T23:12:09.321 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.321 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.321 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-05-22T23:12:09.321 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.322 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.322 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.322 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-22T23:12:09.322 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.322 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.322 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.322 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.323 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": { 2024-05-22T23:12:09.323 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mode", 2024-05-22T23:12:09.323 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.323 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.323 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.323 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "upmap", 2024-05-22T23:12:09.323 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.323 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.324 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.324 INFO:teuthology.orchestra.run.smithi084.stdout: "crush-compat", 2024-05-22T23:12:09.324 INFO:teuthology.orchestra.run.smithi084.stdout: "none", 2024-05-22T23:12:09.324 INFO:teuthology.orchestra.run.smithi084.stdout: "read", 2024-05-22T23:12:09.324 INFO:teuthology.orchestra.run.smithi084.stdout: "upmap", 2024-05-22T23:12:09.324 INFO:teuthology.orchestra.run.smithi084.stdout: "upmap-read" 2024-05-22T23:12:09.324 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.324 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Balancer mode", 2024-05-22T23:12:09.325 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.325 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.325 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.325 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.325 INFO:teuthology.orchestra.run.smithi084.stdout: "pool_ids": { 2024-05-22T23:12:09.325 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "pool_ids", 2024-05-22T23:12:09.325 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.325 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.326 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.326 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.326 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.326 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.326 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.326 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-22T23:12:09.326 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.326 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.327 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.327 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.327 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-05-22T23:12:09.327 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-05-22T23:12:09.327 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:09.327 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.327 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.328 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-05-22T23:12:09.328 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.328 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.328 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.328 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-22T23:12:09.328 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.328 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.328 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.329 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.329 INFO:teuthology.orchestra.run.smithi084.stdout: "upmap_max_deviation": { 2024-05-22T23:12:09.329 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "upmap_max_deviation", 2024-05-22T23:12:09.329 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.329 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.329 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.329 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5", 2024-05-22T23:12:09.329 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1", 2024-05-22T23:12:09.330 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.330 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.330 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-22T23:12:09.330 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-22T23:12:09.330 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.330 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.330 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.331 INFO:teuthology.orchestra.run.smithi084.stdout: "upmap_max_optimizations": { 2024-05-22T23:12:09.331 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "upmap_max_optimizations", 2024-05-22T23:12:09.331 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-05-22T23:12:09.331 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.331 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.331 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-05-22T23:12:09.331 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.331 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.332 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.332 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-22T23:12:09.332 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.332 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.332 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.332 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.332 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.332 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.332 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.333 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephadm", 2024-05-22T23:12:09.333 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.333 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.333 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.333 INFO:teuthology.orchestra.run.smithi084.stdout: "agent_down_multiplier": { 2024-05-22T23:12:09.333 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "agent_down_multiplier", 2024-05-22T23:12:09.333 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-05-22T23:12:09.334 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.334 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.334 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "3.0", 2024-05-22T23:12:09.334 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.334 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.334 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.334 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-22T23:12:09.334 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.335 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.335 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.335 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.335 INFO:teuthology.orchestra.run.smithi084.stdout: "agent_refresh_rate": { 2024-05-22T23:12:09.335 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "agent_refresh_rate", 2024-05-22T23:12:09.335 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:09.335 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.335 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.335 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "20", 2024-05-22T23:12:09.336 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.336 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.336 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.336 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-22T23:12:09.336 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.336 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.336 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.336 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.336 INFO:teuthology.orchestra.run.smithi084.stdout: "agent_starting_port": { 2024-05-22T23:12:09.337 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "agent_starting_port", 2024-05-22T23:12:09.337 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.337 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.337 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.337 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "4721", 2024-05-22T23:12:09.337 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.337 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.338 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.338 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-22T23:12:09.338 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.338 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.338 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.338 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.338 INFO:teuthology.orchestra.run.smithi084.stdout: "allow_ptrace": { 2024-05-22T23:12:09.339 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "allow_ptrace", 2024-05-22T23:12:09.339 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.339 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.339 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.339 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.339 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.339 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.340 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.340 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-22T23:12:09.340 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-22T23:12:09.340 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.340 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.340 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.340 INFO:teuthology.orchestra.run.smithi084.stdout: "autotune_interval": { 2024-05-22T23:12:09.340 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "autotune_interval", 2024-05-22T23:12:09.341 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:09.341 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.341 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.341 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-05-22T23:12:09.341 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.341 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.341 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.341 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-22T23:12:09.341 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.342 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.342 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.342 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.342 INFO:teuthology.orchestra.run.smithi084.stdout: "autotune_memory_target_ratio": { 2024-05-22T23:12:09.342 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "autotune_memory_target_ratio", 2024-05-22T23:12:09.342 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-05-22T23:12:09.342 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.342 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.343 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0.7", 2024-05-22T23:12:09.343 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.343 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.343 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.343 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-22T23:12:09.343 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.343 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.343 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.343 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.344 INFO:teuthology.orchestra.run.smithi084.stdout: "cephadm_log_destination": { 2024-05-22T23:12:09.344 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephadm_log_destination", 2024-05-22T23:12:09.344 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.344 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.344 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.344 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.344 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.344 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.345 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.345 INFO:teuthology.orchestra.run.smithi084.stdout: "file", 2024-05-22T23:12:09.345 INFO:teuthology.orchestra.run.smithi084.stdout: "file,syslog", 2024-05-22T23:12:09.345 INFO:teuthology.orchestra.run.smithi084.stdout: "syslog" 2024-05-22T23:12:09.345 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.345 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-22T23:12:09.345 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.345 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.345 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.346 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.346 INFO:teuthology.orchestra.run.smithi084.stdout: "cgroups_split": { 2024-05-22T23:12:09.346 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cgroups_split", 2024-05-22T23:12:09.346 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.346 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.346 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.346 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.347 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.347 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.347 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.347 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-22T23:12:09.347 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.347 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.347 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.347 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.348 INFO:teuthology.orchestra.run.smithi084.stdout: "config_checks_enabled": { 2024-05-22T23:12:09.348 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "config_checks_enabled", 2024-05-22T23:12:09.348 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.348 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.348 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.348 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.348 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.348 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.349 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.349 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-22T23:12:09.349 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.349 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.349 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.349 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.349 INFO:teuthology.orchestra.run.smithi084.stdout: "config_dashboard": { 2024-05-22T23:12:09.349 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "config_dashboard", 2024-05-22T23:12:09.350 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.350 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.350 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.350 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.350 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.350 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.350 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.350 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-22T23:12:09.351 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.351 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.351 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.351 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.351 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_alertmanager": { 2024-05-22T23:12:09.351 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_alertmanager", 2024-05-22T23:12:09.351 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.352 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.352 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.352 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-22T23:12:09.352 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.352 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.352 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.352 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Prometheus container image", 2024-05-22T23:12:09.352 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.353 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.353 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.353 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.353 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_base": { 2024-05-22T23:12:09.353 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_base", 2024-05-22T23:12:09.353 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.353 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.354 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.354 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-22T23:12:09.354 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.354 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.354 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.354 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Container image name, without the tag", 2024-05-22T23:12:09.354 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.354 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.355 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.355 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.355 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_elasticsearch": { 2024-05-22T23:12:09.355 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_elasticsearch", 2024-05-22T23:12:09.355 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.355 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.355 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.356 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-22T23:12:09.356 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.356 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.356 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.356 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "elasticsearch container image", 2024-05-22T23:12:09.356 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.356 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.357 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.357 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.357 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_grafana": { 2024-05-22T23:12:09.357 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_grafana", 2024-05-22T23:12:09.357 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.357 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.358 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.358 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-22T23:12:09.358 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.358 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.358 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.358 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Prometheus container image", 2024-05-22T23:12:09.358 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.358 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.359 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.359 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.359 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_haproxy": { 2024-05-22T23:12:09.359 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_haproxy", 2024-05-22T23:12:09.359 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.359 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.359 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.360 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-22T23:12:09.360 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.360 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.360 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.360 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "HAproxy container image", 2024-05-22T23:12:09.360 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.360 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.361 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.361 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.361 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_jaeger_agent": { 2024-05-22T23:12:09.361 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_jaeger_agent", 2024-05-22T23:12:09.361 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.361 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.361 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.361 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-22T23:12:09.362 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.362 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.362 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.362 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Jaeger agent container image", 2024-05-22T23:12:09.362 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.362 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.362 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.362 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.363 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_jaeger_collector": { 2024-05-22T23:12:09.363 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_jaeger_collector", 2024-05-22T23:12:09.363 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.363 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.363 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.363 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-22T23:12:09.363 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.364 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.364 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.364 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Jaeger collector container image", 2024-05-22T23:12:09.364 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.364 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.364 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.364 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.364 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_jaeger_query": { 2024-05-22T23:12:09.365 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_jaeger_query", 2024-05-22T23:12:09.365 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.365 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.365 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.365 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-22T23:12:09.365 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.365 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.365 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.366 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Jaeger query container image", 2024-05-22T23:12:09.366 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.366 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.366 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.366 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.366 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_keepalived": { 2024-05-22T23:12:09.366 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_keepalived", 2024-05-22T23:12:09.366 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.367 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.367 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.367 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-22T23:12:09.367 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.367 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.367 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.367 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Keepalived container image", 2024-05-22T23:12:09.367 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.368 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.368 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.368 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.368 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_loki": { 2024-05-22T23:12:09.368 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_loki", 2024-05-22T23:12:09.368 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.368 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.369 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.369 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-22T23:12:09.369 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.369 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.369 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.369 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Loki container image", 2024-05-22T23:12:09.369 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.369 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.370 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.370 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.370 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_node_exporter": { 2024-05-22T23:12:09.370 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_node_exporter", 2024-05-22T23:12:09.370 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.370 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.370 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.370 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-22T23:12:09.371 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.371 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.371 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.371 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Prometheus container image", 2024-05-22T23:12:09.371 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.371 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.371 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.371 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.372 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_nvmeof": { 2024-05-22T23:12:09.372 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_nvmeof", 2024-05-22T23:12:09.372 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.372 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.372 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.372 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-22T23:12:09.372 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.372 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.373 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.373 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Nvme-of container image", 2024-05-22T23:12:09.373 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.373 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.373 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.373 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.373 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_prometheus": { 2024-05-22T23:12:09.373 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_prometheus", 2024-05-22T23:12:09.373 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.374 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.374 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.374 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-22T23:12:09.374 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.374 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.374 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.374 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Prometheus container image", 2024-05-22T23:12:09.374 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.375 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.375 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.375 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.375 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_promtail": { 2024-05-22T23:12:09.375 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_promtail", 2024-05-22T23:12:09.375 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.375 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.375 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.375 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-22T23:12:09.376 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.376 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.376 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.376 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Promtail container image", 2024-05-22T23:12:09.376 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.376 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.376 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.376 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.376 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_samba": { 2024-05-22T23:12:09.377 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_samba", 2024-05-22T23:12:09.377 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.377 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.377 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.377 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-22T23:12:09.377 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.377 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.377 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.378 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Samba/SMB container image", 2024-05-22T23:12:09.378 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.378 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.378 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.378 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.378 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_snmp_gateway": { 2024-05-22T23:12:09.378 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_snmp_gateway", 2024-05-22T23:12:09.378 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.379 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.379 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.379 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-22T23:12:09.379 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.379 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.379 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.379 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "SNMP Gateway container image", 2024-05-22T23:12:09.379 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.380 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.380 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.380 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.380 INFO:teuthology.orchestra.run.smithi084.stdout: "container_init": { 2024-05-22T23:12:09.380 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_init", 2024-05-22T23:12:09.380 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.380 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.381 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.381 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.381 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.381 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.381 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.381 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Run podman/docker with `--init`", 2024-05-22T23:12:09.381 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.381 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.382 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.382 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.382 INFO:teuthology.orchestra.run.smithi084.stdout: "daemon_cache_timeout": { 2024-05-22T23:12:09.382 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "daemon_cache_timeout", 2024-05-22T23:12:09.382 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:09.382 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.382 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.383 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-05-22T23:12:09.383 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.383 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.383 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.383 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-22T23:12:09.383 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.383 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.383 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.383 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.384 INFO:teuthology.orchestra.run.smithi084.stdout: "default_cephadm_command_timeout": { 2024-05-22T23:12:09.384 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "default_cephadm_command_timeout", 2024-05-22T23:12:09.384 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.384 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.384 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.384 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "900", 2024-05-22T23:12:09.384 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.384 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.385 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.385 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-22T23:12:09.385 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.385 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.385 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.385 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.385 INFO:teuthology.orchestra.run.smithi084.stdout: "default_registry": { 2024-05-22T23:12:09.385 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "default_registry", 2024-05-22T23:12:09.386 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.386 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.386 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.386 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "docker.io", 2024-05-22T23:12:09.386 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.386 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.386 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.386 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-22T23:12:09.387 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.387 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.387 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.387 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.387 INFO:teuthology.orchestra.run.smithi084.stdout: "device_cache_timeout": { 2024-05-22T23:12:09.387 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "device_cache_timeout", 2024-05-22T23:12:09.387 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:09.388 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.388 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.388 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "1800", 2024-05-22T23:12:09.388 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.388 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.388 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.388 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "seconds to cache device inventory", 2024-05-22T23:12:09.388 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.388 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.389 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.389 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.389 INFO:teuthology.orchestra.run.smithi084.stdout: "device_enhanced_scan": { 2024-05-22T23:12:09.389 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "device_enhanced_scan", 2024-05-22T23:12:09.389 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.389 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.389 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.389 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.389 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.390 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.390 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.390 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-22T23:12:09.390 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.390 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.390 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.390 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.390 INFO:teuthology.orchestra.run.smithi084.stdout: "facts_cache_timeout": { 2024-05-22T23:12:09.390 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "facts_cache_timeout", 2024-05-22T23:12:09.391 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:09.391 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.391 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.391 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-05-22T23:12:09.391 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.391 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.391 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.391 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "seconds to cache host facts data", 2024-05-22T23:12:09.392 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.392 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.392 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.392 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.392 INFO:teuthology.orchestra.run.smithi084.stdout: "grafana_dashboards_path": { 2024-05-22T23:12:09.392 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "grafana_dashboards_path", 2024-05-22T23:12:09.392 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.392 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.393 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.393 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-22T23:12:09.393 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.393 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.393 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.393 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-22T23:12:09.393 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.393 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.394 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.394 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.394 INFO:teuthology.orchestra.run.smithi084.stdout: "host_check_interval": { 2024-05-22T23:12:09.394 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "host_check_interval", 2024-05-22T23:12:09.394 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:09.394 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.394 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.394 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-05-22T23:12:09.395 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.395 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.395 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.395 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to perform a host check", 2024-05-22T23:12:09.395 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.395 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.395 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.395 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.396 INFO:teuthology.orchestra.run.smithi084.stdout: "hw_monitoring": { 2024-05-22T23:12:09.396 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "hw_monitoring", 2024-05-22T23:12:09.396 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.396 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.396 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.396 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.396 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.396 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.397 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.397 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-22T23:12:09.397 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.397 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.397 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.397 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.397 INFO:teuthology.orchestra.run.smithi084.stdout: "inventory_list_all": { 2024-05-22T23:12:09.398 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "inventory_list_all", 2024-05-22T23:12:09.398 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.398 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.398 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.398 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.398 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.398 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.398 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.399 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-22T23:12:09.399 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.399 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.399 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.399 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.399 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.399 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.399 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.400 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.400 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.400 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.400 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.400 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.400 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.400 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.401 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.401 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.401 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.401 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.401 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.401 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.401 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.401 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.401 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.402 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.402 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.402 INFO:teuthology.orchestra.run.smithi084.stdout: "log_refresh_metadata": { 2024-05-22T23:12:09.402 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_refresh_metadata", 2024-05-22T23:12:09.402 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.402 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.402 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.403 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.403 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.403 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.403 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.403 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-22T23:12:09.403 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.403 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.403 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.404 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.404 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.404 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.404 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.404 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.404 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.404 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.404 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.405 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.405 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.405 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-22T23:12:09.405 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.405 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.405 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.405 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.405 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.405 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.406 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.406 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.406 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.406 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.406 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.406 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.406 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.406 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.407 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.407 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.407 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.407 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.407 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.407 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.407 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.408 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.408 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.408 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.408 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.408 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.408 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.408 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.408 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.409 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.409 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.409 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.409 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.409 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.409 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.409 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.409 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.410 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.410 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.410 INFO:teuthology.orchestra.run.smithi084.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-22T23:12:09.410 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-22T23:12:09.410 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.410 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.410 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.411 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.411 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.411 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.411 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.411 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-22T23:12:09.411 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.411 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.411 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.412 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.412 INFO:teuthology.orchestra.run.smithi084.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-22T23:12:09.412 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-22T23:12:09.412 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.412 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.412 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.412 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "*", 2024-05-22T23:12:09.412 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.413 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.413 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.413 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-22T23:12:09.413 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.413 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.413 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.413 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.413 INFO:teuthology.orchestra.run.smithi084.stdout: "max_count_per_host": { 2024-05-22T23:12:09.413 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_count_per_host", 2024-05-22T23:12:09.414 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.414 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.414 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.414 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-05-22T23:12:09.414 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.414 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.414 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.414 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "max number of daemons per service per host", 2024-05-22T23:12:09.415 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.415 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.415 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.415 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.415 INFO:teuthology.orchestra.run.smithi084.stdout: "max_osd_draining_count": { 2024-05-22T23:12:09.415 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_osd_draining_count", 2024-05-22T23:12:09.415 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.415 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.416 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.416 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-05-22T23:12:09.416 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.416 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.416 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.416 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-22T23:12:09.416 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.416 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.417 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.417 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.417 INFO:teuthology.orchestra.run.smithi084.stdout: "migration_current": { 2024-05-22T23:12:09.417 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "migration_current", 2024-05-22T23:12:09.417 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.417 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.417 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.417 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.418 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.418 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.418 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.418 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "internal - do not modify", 2024-05-22T23:12:09.418 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.418 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.418 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.418 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.419 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": { 2024-05-22T23:12:09.419 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mode", 2024-05-22T23:12:09.419 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.419 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.419 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.419 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "root", 2024-05-22T23:12:09.419 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.419 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.420 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.420 INFO:teuthology.orchestra.run.smithi084.stdout: "cephadm-package", 2024-05-22T23:12:09.420 INFO:teuthology.orchestra.run.smithi084.stdout: "root" 2024-05-22T23:12:09.420 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.420 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "mode for remote execution of cephadm", 2024-05-22T23:12:09.420 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.420 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.420 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.421 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.421 INFO:teuthology.orchestra.run.smithi084.stdout: "oob_default_addr": { 2024-05-22T23:12:09.421 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "oob_default_addr", 2024-05-22T23:12:09.421 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.421 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.421 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.421 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "169.254.1.1", 2024-05-22T23:12:09.421 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.422 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.422 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.422 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-22T23:12:09.422 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.422 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.422 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.422 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.422 INFO:teuthology.orchestra.run.smithi084.stdout: "prometheus_alerts_path": { 2024-05-22T23:12:09.423 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "prometheus_alerts_path", 2024-05-22T23:12:09.423 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.423 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.423 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.423 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-22T23:12:09.423 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.423 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.423 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.424 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-22T23:12:09.424 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.424 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.424 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.424 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.424 INFO:teuthology.orchestra.run.smithi084.stdout: "registry_insecure": { 2024-05-22T23:12:09.424 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "registry_insecure", 2024-05-22T23:12:09.425 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.425 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.425 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.425 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.425 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.425 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.425 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.425 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-22T23:12:09.426 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.426 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.426 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.426 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.426 INFO:teuthology.orchestra.run.smithi084.stdout: "registry_password": { 2024-05-22T23:12:09.426 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "registry_password", 2024-05-22T23:12:09.426 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.426 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.427 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.427 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.427 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.427 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.427 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.427 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-22T23:12:09.427 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.427 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.427 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.428 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.428 INFO:teuthology.orchestra.run.smithi084.stdout: "registry_url": { 2024-05-22T23:12:09.428 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "registry_url", 2024-05-22T23:12:09.428 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.428 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.428 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.428 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.428 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.428 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.429 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.429 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-22T23:12:09.429 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.429 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.429 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.429 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.429 INFO:teuthology.orchestra.run.smithi084.stdout: "registry_username": { 2024-05-22T23:12:09.429 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "registry_username", 2024-05-22T23:12:09.429 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.430 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.430 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.430 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.430 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.430 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.430 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.430 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-22T23:12:09.431 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.431 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.431 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.431 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.431 INFO:teuthology.orchestra.run.smithi084.stdout: "secure_monitoring_stack": { 2024-05-22T23:12:09.431 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "secure_monitoring_stack", 2024-05-22T23:12:09.431 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.431 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.432 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.432 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.432 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.432 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.432 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.432 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-22T23:12:09.432 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.433 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.433 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.433 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.433 INFO:teuthology.orchestra.run.smithi084.stdout: "service_discovery_port": { 2024-05-22T23:12:09.433 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "service_discovery_port", 2024-05-22T23:12:09.433 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.433 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.433 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.434 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8765", 2024-05-22T23:12:09.434 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.434 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.434 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.434 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "cephadm service discovery port", 2024-05-22T23:12:09.434 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.434 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.434 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.435 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.435 INFO:teuthology.orchestra.run.smithi084.stdout: "ssh_config_file": { 2024-05-22T23:12:09.435 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ssh_config_file", 2024-05-22T23:12:09.435 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.435 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.435 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.435 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.435 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.436 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.436 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.436 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-22T23:12:09.436 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.436 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.436 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.436 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.436 INFO:teuthology.orchestra.run.smithi084.stdout: "use_agent": { 2024-05-22T23:12:09.437 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "use_agent", 2024-05-22T23:12:09.437 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.437 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.437 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.437 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.437 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.437 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.437 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.438 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-22T23:12:09.438 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.438 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.438 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.438 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.438 INFO:teuthology.orchestra.run.smithi084.stdout: "use_repo_digest": { 2024-05-22T23:12:09.438 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "use_repo_digest", 2024-05-22T23:12:09.439 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.439 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.439 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.439 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.439 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.439 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.439 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.439 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-22T23:12:09.439 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.440 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.440 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.440 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.440 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_on_failed_host_check": { 2024-05-22T23:12:09.440 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_on_failed_host_check", 2024-05-22T23:12:09.440 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.440 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.440 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.440 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.441 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.441 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.441 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.441 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "raise a health warning if the host check fails", 2024-05-22T23:12:09.441 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.441 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.441 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.441 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.441 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_on_stray_daemons": { 2024-05-22T23:12:09.442 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_on_stray_daemons", 2024-05-22T23:12:09.442 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.442 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.442 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.442 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.442 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.442 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.443 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.443 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-22T23:12:09.443 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.443 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.443 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.443 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.443 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_on_stray_hosts": { 2024-05-22T23:12:09.443 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_on_stray_hosts", 2024-05-22T23:12:09.444 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.444 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.444 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.444 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.444 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.444 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.444 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.444 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-22T23:12:09.444 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.445 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.445 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.445 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.445 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.445 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.445 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.445 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crash", 2024-05-22T23:12:09.445 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.446 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.446 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.446 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.446 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.446 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.446 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.446 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.446 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.447 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.447 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.447 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.447 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.447 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.447 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.447 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.447 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.448 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.448 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.448 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.448 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.448 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.448 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.448 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.448 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.449 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.449 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.449 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.449 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.449 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.449 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.449 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.449 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.449 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.450 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.450 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.450 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.450 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.450 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.450 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.450 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.450 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.451 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.451 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.451 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.451 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.453 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.453 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.453 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.453 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.454 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.454 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.454 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.454 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.454 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.454 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.454 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.454 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.455 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.455 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.455 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.455 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.455 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.455 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.455 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.455 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.456 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.456 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.456 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.456 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.456 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.456 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.456 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.456 INFO:teuthology.orchestra.run.smithi084.stdout: "retain_interval": { 2024-05-22T23:12:09.457 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "retain_interval", 2024-05-22T23:12:09.457 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:09.457 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.457 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.457 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "31536000", 2024-05-22T23:12:09.457 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.457 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.457 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.458 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-22T23:12:09.458 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.458 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.458 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.458 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.458 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_recent_interval": { 2024-05-22T23:12:09.458 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_recent_interval", 2024-05-22T23:12:09.459 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:09.459 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.459 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.459 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "1209600", 2024-05-22T23:12:09.459 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.459 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.459 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.459 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-22T23:12:09.460 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.460 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.460 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.460 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.460 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.460 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.460 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.460 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "dashboard", 2024-05-22T23:12:09.461 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.461 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.461 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.461 INFO:teuthology.orchestra.run.smithi084.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-22T23:12:09.461 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-22T23:12:09.461 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.461 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.461 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.461 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-05-22T23:12:09.462 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.462 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.462 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.462 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.462 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.462 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.462 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.463 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.463 INFO:teuthology.orchestra.run.smithi084.stdout: "ALERTMANAGER_API_HOST": { 2024-05-22T23:12:09.463 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-22T23:12:09.463 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.463 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.463 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.463 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.463 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.463 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.464 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.464 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.464 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.464 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.464 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.464 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.464 INFO:teuthology.orchestra.run.smithi084.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-22T23:12:09.465 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-22T23:12:09.465 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.465 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.465 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.465 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.465 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.465 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.465 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.466 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.466 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.466 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.466 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.466 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.466 INFO:teuthology.orchestra.run.smithi084.stdout: "AUDIT_API_ENABLED": { 2024-05-22T23:12:09.466 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "AUDIT_API_ENABLED", 2024-05-22T23:12:09.466 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.467 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.467 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.467 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.467 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.467 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.467 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.467 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.467 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.468 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.468 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.468 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.468 INFO:teuthology.orchestra.run.smithi084.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-22T23:12:09.468 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-22T23:12:09.468 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.468 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.468 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.468 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.469 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.469 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.469 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.469 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.469 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.469 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.469 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.469 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.470 INFO:teuthology.orchestra.run.smithi084.stdout: "ENABLE_BROWSABLE_API": { 2024-05-22T23:12:09.470 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-22T23:12:09.470 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.470 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.470 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.470 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.470 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.471 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.471 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.471 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.471 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.471 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.471 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.471 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.471 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-22T23:12:09.472 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-22T23:12:09.472 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.472 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.472 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.472 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.472 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.472 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.472 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.473 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.473 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.473 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.473 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.473 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.473 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-22T23:12:09.473 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-22T23:12:09.473 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.474 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.474 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.474 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.474 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.474 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.474 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.474 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.474 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.475 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.475 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.475 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.475 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-22T23:12:09.475 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-22T23:12:09.475 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.475 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.475 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.476 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.476 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.476 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.476 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.476 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.476 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.476 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.476 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.477 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.477 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-22T23:12:09.477 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-22T23:12:09.477 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.477 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.477 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.477 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.477 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.478 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.478 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.478 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.478 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.478 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.478 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.478 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.478 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-22T23:12:09.479 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-22T23:12:09.479 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.479 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.479 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.479 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.479 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.479 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.479 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.480 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.480 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.480 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.480 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.480 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.480 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-22T23:12:09.480 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-22T23:12:09.480 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.481 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.481 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.481 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.481 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.481 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.481 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.481 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.482 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.482 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.482 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.482 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.482 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-22T23:12:09.482 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-22T23:12:09.482 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.482 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.483 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.483 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.483 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.483 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.483 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.483 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.483 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.483 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.484 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.484 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.484 INFO:teuthology.orchestra.run.smithi084.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-22T23:12:09.484 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-22T23:12:09.484 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.484 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.484 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.484 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.485 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.485 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.485 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.485 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.485 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.485 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.485 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.485 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.486 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_API_PASSWORD": { 2024-05-22T23:12:09.486 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-22T23:12:09.486 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.486 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.486 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.486 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "admin", 2024-05-22T23:12:09.486 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.486 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.487 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.487 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.487 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.487 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.487 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.487 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.487 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-22T23:12:09.488 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-22T23:12:09.488 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.488 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.488 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.488 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.488 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.488 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.488 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.488 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.489 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.489 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.489 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.489 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.489 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_API_URL": { 2024-05-22T23:12:09.489 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_API_URL", 2024-05-22T23:12:09.489 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.490 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.490 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.490 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.490 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.490 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.490 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.490 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.490 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.491 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.491 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.491 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.491 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_API_USERNAME": { 2024-05-22T23:12:09.491 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-22T23:12:09.491 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.491 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.491 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.492 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "admin", 2024-05-22T23:12:09.492 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.492 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.492 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.492 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.492 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.492 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.492 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.493 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.493 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-22T23:12:09.493 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-22T23:12:09.493 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.493 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.493 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.493 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.493 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.494 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.494 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.494 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.494 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.494 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.494 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.494 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.494 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-22T23:12:09.495 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-22T23:12:09.495 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.495 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.495 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.495 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.495 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.495 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.496 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.496 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.496 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.496 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.496 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.496 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.496 INFO:teuthology.orchestra.run.smithi084.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-22T23:12:09.496 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-22T23:12:09.497 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.497 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.497 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.497 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.497 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.497 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.497 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.497 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.497 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.498 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.498 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.498 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.498 INFO:teuthology.orchestra.run.smithi084.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-22T23:12:09.498 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-22T23:12:09.498 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.498 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.498 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.499 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.499 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.499 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.499 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.499 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.499 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.499 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.499 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.500 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.500 INFO:teuthology.orchestra.run.smithi084.stdout: "PROMETHEUS_API_HOST": { 2024-05-22T23:12:09.500 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-22T23:12:09.500 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.500 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.500 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.500 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.500 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.501 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.501 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.501 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.501 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.501 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.501 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.501 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.501 INFO:teuthology.orchestra.run.smithi084.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-22T23:12:09.502 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-22T23:12:09.502 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.502 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.502 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.502 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.502 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.502 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.502 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.503 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.503 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.503 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.503 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.503 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.503 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-22T23:12:09.503 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-22T23:12:09.503 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.504 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.504 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.504 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.504 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.504 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.504 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.504 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.504 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.504 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.505 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.505 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.505 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-22T23:12:09.505 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-22T23:12:09.505 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.505 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.505 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.505 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.506 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.506 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.506 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.506 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.506 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.506 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.506 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.507 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.507 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-22T23:12:09.507 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-22T23:12:09.507 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.507 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.507 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.507 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.507 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.508 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.508 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.508 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.508 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.508 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.508 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.508 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.508 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-22T23:12:09.509 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-22T23:12:09.509 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.509 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.509 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.509 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.509 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.509 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.509 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.510 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.510 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.510 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.510 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.510 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.510 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-22T23:12:09.510 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-22T23:12:09.510 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.511 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.511 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.511 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.511 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.511 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.511 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.511 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.511 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.512 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.512 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.512 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.512 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-22T23:12:09.512 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-22T23:12:09.512 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.512 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.512 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.513 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.513 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.513 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.513 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.513 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.513 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.513 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.514 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.514 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.514 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-22T23:12:09.514 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-22T23:12:09.514 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.514 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.514 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.514 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.514 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.515 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.515 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.515 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.515 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.515 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.515 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.515 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.515 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_ENABLED": { 2024-05-22T23:12:09.516 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-22T23:12:09.516 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.516 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.516 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.516 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.516 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.516 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.516 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.517 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.517 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.517 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.517 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.517 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.517 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-22T23:12:09.517 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-22T23:12:09.518 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.518 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.518 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.518 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-22T23:12:09.518 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.518 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.518 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.518 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.519 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.519 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.519 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.519 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.519 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-22T23:12:09.519 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-22T23:12:09.519 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.519 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.520 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.520 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-05-22T23:12:09.520 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.520 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.520 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.520 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.520 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.521 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.521 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.521 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.521 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-22T23:12:09.521 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-22T23:12:09.521 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.521 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.521 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.522 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8", 2024-05-22T23:12:09.522 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.522 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.522 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.522 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.522 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.522 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.522 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.523 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.523 INFO:teuthology.orchestra.run.smithi084.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-22T23:12:09.523 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-22T23:12:09.523 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.523 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.523 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.523 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "45", 2024-05-22T23:12:09.523 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.524 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.524 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.524 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.524 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.524 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.524 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.524 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.524 INFO:teuthology.orchestra.run.smithi084.stdout: "RGW_API_ACCESS_KEY": { 2024-05-22T23:12:09.525 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-22T23:12:09.525 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.525 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.525 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.525 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.525 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.525 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.526 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.526 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.526 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.526 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.526 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.526 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.526 INFO:teuthology.orchestra.run.smithi084.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-22T23:12:09.526 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-22T23:12:09.527 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.527 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.527 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.527 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "admin", 2024-05-22T23:12:09.527 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.527 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.527 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.528 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.528 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.528 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.528 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.528 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.528 INFO:teuthology.orchestra.run.smithi084.stdout: "RGW_API_SECRET_KEY": { 2024-05-22T23:12:09.528 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-22T23:12:09.528 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.528 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.529 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.529 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.529 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.529 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.529 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.529 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.529 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.530 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.530 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.530 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.530 INFO:teuthology.orchestra.run.smithi084.stdout: "RGW_API_SSL_VERIFY": { 2024-05-22T23:12:09.530 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-22T23:12:09.530 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.530 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.530 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.531 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.531 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.531 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.531 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.531 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.531 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.531 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.531 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.532 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.532 INFO:teuthology.orchestra.run.smithi084.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-22T23:12:09.532 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-22T23:12:09.532 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.532 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.532 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.532 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-05-22T23:12:09.532 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.533 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.533 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.533 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.533 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.533 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.533 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.533 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.533 INFO:teuthology.orchestra.run.smithi084.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-22T23:12:09.534 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-22T23:12:09.534 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.534 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.534 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.534 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-05-22T23:12:09.534 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.534 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.534 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.535 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.535 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.535 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.535 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.535 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.535 INFO:teuthology.orchestra.run.smithi084.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-22T23:12:09.535 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-22T23:12:09.535 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.536 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.536 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.536 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5", 2024-05-22T23:12:09.536 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.536 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.536 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.536 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.536 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.537 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.537 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.537 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.537 INFO:teuthology.orchestra.run.smithi084.stdout: "cross_origin_url": { 2024-05-22T23:12:09.537 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cross_origin_url", 2024-05-22T23:12:09.537 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.537 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.537 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.537 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.538 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.538 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.538 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.538 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.538 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.538 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.538 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.539 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.539 INFO:teuthology.orchestra.run.smithi084.stdout: "crt_file": { 2024-05-22T23:12:09.539 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crt_file", 2024-05-22T23:12:09.539 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.539 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.539 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.539 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.539 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.540 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.540 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.540 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.540 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.540 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.540 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.540 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.540 INFO:teuthology.orchestra.run.smithi084.stdout: "debug": { 2024-05-22T23:12:09.541 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "debug", 2024-05-22T23:12:09.541 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.541 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.541 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.541 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.541 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.541 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.541 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.542 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Enable/disable debug options", 2024-05-22T23:12:09.542 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.542 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.542 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.542 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.542 INFO:teuthology.orchestra.run.smithi084.stdout: "jwt_token_ttl": { 2024-05-22T23:12:09.542 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "jwt_token_ttl", 2024-05-22T23:12:09.543 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.543 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.543 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.543 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "28800", 2024-05-22T23:12:09.543 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.543 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.543 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.543 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.544 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.544 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.544 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.544 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.544 INFO:teuthology.orchestra.run.smithi084.stdout: "key_file": { 2024-05-22T23:12:09.544 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "key_file", 2024-05-22T23:12:09.544 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.544 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.545 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.545 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.545 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.545 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.545 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.545 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.545 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.546 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.546 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.546 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.546 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.546 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.546 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.546 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.547 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.547 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.547 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.547 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.547 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.547 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.547 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.547 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.548 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.548 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.548 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.548 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.548 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.548 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.548 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.548 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.549 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.549 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.549 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.549 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.549 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.549 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.549 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.549 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.550 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.550 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.550 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.550 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.550 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.550 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.550 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.551 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.551 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.551 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.551 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.551 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.551 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.551 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.551 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.552 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.552 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.552 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.552 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.552 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.552 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.552 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.552 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.553 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.553 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.553 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.553 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.553 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.553 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.553 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.553 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.554 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.554 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.554 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.554 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.554 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.554 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.554 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.555 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.555 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.555 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.555 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.555 INFO:teuthology.orchestra.run.smithi084.stdout: "motd": { 2024-05-22T23:12:09.555 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "motd", 2024-05-22T23:12:09.555 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.555 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.556 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.556 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.556 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.556 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.556 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.556 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "The message of the day", 2024-05-22T23:12:09.556 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.556 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.557 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.557 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.557 INFO:teuthology.orchestra.run.smithi084.stdout: "redirect_resolve_ip_addr": { 2024-05-22T23:12:09.557 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "redirect_resolve_ip_addr", 2024-05-22T23:12:09.557 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.557 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.557 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.557 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.558 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.558 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.558 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.558 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.558 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.558 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.558 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.558 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.559 INFO:teuthology.orchestra.run.smithi084.stdout: "server_addr": { 2024-05-22T23:12:09.559 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_addr", 2024-05-22T23:12:09.559 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.559 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.559 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.559 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "::", 2024-05-22T23:12:09.559 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.559 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.560 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.560 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.560 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.560 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.560 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.560 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.560 INFO:teuthology.orchestra.run.smithi084.stdout: "server_port": { 2024-05-22T23:12:09.560 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_port", 2024-05-22T23:12:09.561 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.561 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.561 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.561 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8080", 2024-05-22T23:12:09.561 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.561 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.561 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.561 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.562 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.562 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.562 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.562 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.562 INFO:teuthology.orchestra.run.smithi084.stdout: "ssl": { 2024-05-22T23:12:09.562 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ssl", 2024-05-22T23:12:09.562 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.562 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.563 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.563 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.563 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.563 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.563 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.563 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.563 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.563 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.564 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.564 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.564 INFO:teuthology.orchestra.run.smithi084.stdout: "ssl_server_port": { 2024-05-22T23:12:09.564 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ssl_server_port", 2024-05-22T23:12:09.564 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.564 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.564 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.564 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8443", 2024-05-22T23:12:09.564 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.565 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.565 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.565 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.565 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.565 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.565 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.565 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.566 INFO:teuthology.orchestra.run.smithi084.stdout: "standby_behaviour": { 2024-05-22T23:12:09.566 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "standby_behaviour", 2024-05-22T23:12:09.566 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.566 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.566 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.566 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "redirect", 2024-05-22T23:12:09.566 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.566 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.567 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.567 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.567 INFO:teuthology.orchestra.run.smithi084.stdout: "redirect" 2024-05-22T23:12:09.567 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.567 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.567 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.567 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.567 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.568 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.568 INFO:teuthology.orchestra.run.smithi084.stdout: "standby_error_status_code": { 2024-05-22T23:12:09.568 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "standby_error_status_code", 2024-05-22T23:12:09.568 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.568 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.568 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.568 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "500", 2024-05-22T23:12:09.569 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "400", 2024-05-22T23:12:09.569 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "599", 2024-05-22T23:12:09.569 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.569 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.569 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.569 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.569 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.569 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.570 INFO:teuthology.orchestra.run.smithi084.stdout: "url_prefix": { 2024-05-22T23:12:09.570 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "url_prefix", 2024-05-22T23:12:09.570 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.570 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.570 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.570 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.570 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.570 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.571 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.571 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.571 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.571 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.571 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.571 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.571 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.571 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.572 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.572 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "devicehealth", 2024-05-22T23:12:09.572 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.572 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.572 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.572 INFO:teuthology.orchestra.run.smithi084.stdout: "enable_monitoring": { 2024-05-22T23:12:09.572 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "enable_monitoring", 2024-05-22T23:12:09.572 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.573 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.573 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.573 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.573 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.573 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.573 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.573 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "monitor device health metrics", 2024-05-22T23:12:09.573 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.574 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.574 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.574 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.574 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.574 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.574 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.574 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.575 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.575 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.575 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.575 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.575 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.575 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.575 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.575 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.576 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.576 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.576 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.576 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.576 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.576 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.576 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.576 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.577 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.577 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.577 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.577 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.577 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.577 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.577 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.578 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.578 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.578 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.578 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.578 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.578 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.578 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.578 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.579 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.579 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.579 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.579 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.579 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.579 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.579 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.579 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.580 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.580 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.580 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.580 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.580 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.580 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.580 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.580 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.581 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.581 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.581 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.581 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.581 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.581 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.581 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.581 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.582 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.582 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.582 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.582 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.582 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.582 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.582 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.582 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.583 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.583 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.583 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.583 INFO:teuthology.orchestra.run.smithi084.stdout: "mark_out_threshold": { 2024-05-22T23:12:09.583 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mark_out_threshold", 2024-05-22T23:12:09.583 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:09.583 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.583 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.584 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "2419200", 2024-05-22T23:12:09.584 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.584 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.584 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.584 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-22T23:12:09.584 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.584 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.584 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.585 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.585 INFO:teuthology.orchestra.run.smithi084.stdout: "pool_name": { 2024-05-22T23:12:09.585 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "pool_name", 2024-05-22T23:12:09.585 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.585 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.585 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.585 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "device_health_metrics", 2024-05-22T23:12:09.586 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.586 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.586 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.586 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-22T23:12:09.586 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.586 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.586 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.586 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.587 INFO:teuthology.orchestra.run.smithi084.stdout: "retention_period": { 2024-05-22T23:12:09.587 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "retention_period", 2024-05-22T23:12:09.587 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:09.587 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.587 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.587 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "15552000", 2024-05-22T23:12:09.587 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.587 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.588 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.588 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how long to retain device health metrics", 2024-05-22T23:12:09.588 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.588 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.588 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.588 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.588 INFO:teuthology.orchestra.run.smithi084.stdout: "scrape_frequency": { 2024-05-22T23:12:09.588 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "scrape_frequency", 2024-05-22T23:12:09.589 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:09.589 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.589 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.589 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "86400", 2024-05-22T23:12:09.589 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.589 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.589 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.589 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-22T23:12:09.590 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.590 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.590 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.590 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.590 INFO:teuthology.orchestra.run.smithi084.stdout: "self_heal": { 2024-05-22T23:12:09.590 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "self_heal", 2024-05-22T23:12:09.590 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.590 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.591 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.591 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.591 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.591 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.591 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.591 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-22T23:12:09.591 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.591 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.592 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.592 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.592 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-05-22T23:12:09.592 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-05-22T23:12:09.592 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:09.592 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.592 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.593 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-05-22T23:12:09.593 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.593 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.593 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.593 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to wake up and check device health", 2024-05-22T23:12:09.593 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.593 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.593 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.594 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.594 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_threshold": { 2024-05-22T23:12:09.594 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_threshold", 2024-05-22T23:12:09.594 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:09.594 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.594 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.594 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "7257600", 2024-05-22T23:12:09.594 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.595 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.595 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.595 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-22T23:12:09.595 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.595 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.595 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.595 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.595 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.596 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.596 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.596 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "diskprediction_local", 2024-05-22T23:12:09.596 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.596 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.596 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.596 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.596 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.597 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.597 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.597 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.597 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.597 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.597 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.597 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.598 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.598 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.598 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.598 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.598 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.598 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.598 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.598 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.599 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.599 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.599 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.599 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.599 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.599 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.599 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.599 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.600 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.600 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.600 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.600 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.600 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.600 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.600 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.600 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.601 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.601 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.601 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.601 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.601 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.601 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.601 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.601 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.601 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.602 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.602 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.602 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.602 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.602 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.602 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.602 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.602 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.603 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.603 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.603 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.603 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.603 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.603 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.603 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.603 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.604 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.604 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.604 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.604 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.604 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.604 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.604 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.604 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.604 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.605 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.605 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.605 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.605 INFO:teuthology.orchestra.run.smithi084.stdout: "predict_interval": { 2024-05-22T23:12:09.605 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "predict_interval", 2024-05-22T23:12:09.605 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.605 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.606 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.606 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "86400", 2024-05-22T23:12:09.606 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.606 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.606 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.606 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.606 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.606 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.606 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.607 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.607 INFO:teuthology.orchestra.run.smithi084.stdout: "predictor_model": { 2024-05-22T23:12:09.607 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "predictor_model", 2024-05-22T23:12:09.607 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.607 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.607 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.607 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "prophetstor", 2024-05-22T23:12:09.607 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.608 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.608 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.608 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.608 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.608 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.608 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.608 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.608 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-05-22T23:12:09.609 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-05-22T23:12:09.609 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.609 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.609 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.609 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-05-22T23:12:09.609 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.609 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.609 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.610 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.610 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.610 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.610 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.610 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.610 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.610 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.610 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.611 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "influx", 2024-05-22T23:12:09.611 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": false, 2024-05-22T23:12:09.611 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "influxdb python module not found", 2024-05-22T23:12:09.611 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.611 INFO:teuthology.orchestra.run.smithi084.stdout: "batch_size": { 2024-05-22T23:12:09.611 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "batch_size", 2024-05-22T23:12:09.611 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.611 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.612 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.612 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5000", 2024-05-22T23:12:09.612 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.612 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.612 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.612 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-22T23:12:09.612 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.612 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.613 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.613 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.613 INFO:teuthology.orchestra.run.smithi084.stdout: "database": { 2024-05-22T23:12:09.613 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "database", 2024-05-22T23:12:09.613 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.613 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.613 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.613 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "ceph", 2024-05-22T23:12:09.614 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.614 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.614 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.614 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-22T23:12:09.614 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.614 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.614 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.614 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.615 INFO:teuthology.orchestra.run.smithi084.stdout: "hostname": { 2024-05-22T23:12:09.615 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "hostname", 2024-05-22T23:12:09.615 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.615 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.615 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.615 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.615 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.615 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.616 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.616 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "InfluxDB server hostname", 2024-05-22T23:12:09.616 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.616 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.616 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.616 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.616 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-05-22T23:12:09.616 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-05-22T23:12:09.617 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:09.617 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.617 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.617 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "30", 2024-05-22T23:12:09.617 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "5", 2024-05-22T23:12:09.617 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.617 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.617 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-22T23:12:09.618 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.618 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.618 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.618 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.618 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.618 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.618 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.618 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.619 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.619 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.619 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.619 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.619 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.619 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.619 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.619 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.620 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.620 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.620 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.620 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.620 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.620 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.620 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.620 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.621 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.621 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.621 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.621 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.621 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.621 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.621 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.621 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.622 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.622 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.622 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.622 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.622 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.622 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.622 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.622 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.623 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.623 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.623 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.623 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.623 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.623 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.623 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.623 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.624 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.624 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.624 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.624 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.624 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.624 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.624 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.624 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.625 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.625 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.625 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.625 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.625 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.625 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.625 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.625 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.626 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.626 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.626 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.626 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.626 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.626 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.626 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.626 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.627 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.627 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.627 INFO:teuthology.orchestra.run.smithi084.stdout: "password": { 2024-05-22T23:12:09.627 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "password", 2024-05-22T23:12:09.627 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.627 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.627 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.628 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.628 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.628 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.628 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.628 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "password of InfluxDB server user", 2024-05-22T23:12:09.628 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.628 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.628 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.629 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.629 INFO:teuthology.orchestra.run.smithi084.stdout: "port": { 2024-05-22T23:12:09.629 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "port", 2024-05-22T23:12:09.629 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.629 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.629 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.629 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8086", 2024-05-22T23:12:09.629 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.629 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.630 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.630 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "InfluxDB server port", 2024-05-22T23:12:09.630 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.630 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.630 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.630 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.630 INFO:teuthology.orchestra.run.smithi084.stdout: "ssl": { 2024-05-22T23:12:09.631 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ssl", 2024-05-22T23:12:09.631 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.631 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.631 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.631 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "false", 2024-05-22T23:12:09.631 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.631 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.631 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.632 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-22T23:12:09.632 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.632 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.632 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.632 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.632 INFO:teuthology.orchestra.run.smithi084.stdout: "threads": { 2024-05-22T23:12:09.632 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "threads", 2024-05-22T23:12:09.632 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.633 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.633 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.633 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5", 2024-05-22T23:12:09.633 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1", 2024-05-22T23:12:09.633 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "32", 2024-05-22T23:12:09.633 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.633 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-22T23:12:09.633 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.634 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.634 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.634 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.634 INFO:teuthology.orchestra.run.smithi084.stdout: "username": { 2024-05-22T23:12:09.634 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "username", 2024-05-22T23:12:09.634 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.634 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.634 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.634 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.635 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.635 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.635 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.635 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "username of InfluxDB server user", 2024-05-22T23:12:09.635 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.635 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.635 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.636 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.636 INFO:teuthology.orchestra.run.smithi084.stdout: "verify_ssl": { 2024-05-22T23:12:09.636 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "verify_ssl", 2024-05-22T23:12:09.636 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.636 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.636 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.636 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "true", 2024-05-22T23:12:09.636 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.637 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.637 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.637 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-22T23:12:09.637 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.637 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.637 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.637 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.637 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.638 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.638 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.638 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "insights", 2024-05-22T23:12:09.638 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.638 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.638 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.638 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.639 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.639 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.639 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.639 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.639 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.639 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.639 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.639 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.640 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.640 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.640 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.640 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.640 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.640 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.640 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.640 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.640 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.641 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.641 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.641 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.641 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.641 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.641 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.641 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.641 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.642 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.642 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.642 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.642 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.642 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.642 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.642 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.643 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.643 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.643 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.643 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.643 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.643 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.643 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.643 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.644 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.644 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.644 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.644 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.644 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.644 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.644 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.645 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.645 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.645 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.645 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.645 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.645 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.645 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.645 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.646 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.646 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.646 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.646 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.646 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.646 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.646 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.646 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.647 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.647 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.647 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.647 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.647 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.647 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.647 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.647 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.648 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.648 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "iostat", 2024-05-22T23:12:09.648 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.648 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.648 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.648 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.648 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.648 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.649 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.649 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.649 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.649 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.649 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.649 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.649 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.650 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.650 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.650 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.650 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.650 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.650 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.650 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.651 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.651 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.651 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.651 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.651 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.651 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.651 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.651 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.652 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.652 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.652 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.652 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.652 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.652 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.652 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.652 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.653 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.653 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.653 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.653 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.653 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.653 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.653 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.653 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.654 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.654 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.654 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.654 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.654 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.654 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.654 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.654 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.655 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.655 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.655 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.655 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.655 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.655 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.655 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.655 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.656 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.656 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.656 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.656 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.656 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.656 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.656 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.657 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.657 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.657 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.657 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.657 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.657 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.657 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.657 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.658 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.658 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "k8sevents", 2024-05-22T23:12:09.658 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.658 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.658 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.658 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_event_retention_days": { 2024-05-22T23:12:09.658 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ceph_event_retention_days", 2024-05-22T23:12:09.658 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.659 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.659 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.659 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "7", 2024-05-22T23:12:09.659 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.659 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.659 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.659 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-22T23:12:09.660 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.660 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.660 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.660 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.660 INFO:teuthology.orchestra.run.smithi084.stdout: "config_check_secs": { 2024-05-22T23:12:09.660 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "config_check_secs", 2024-05-22T23:12:09.660 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.660 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.661 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.661 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-05-22T23:12:09.661 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "10", 2024-05-22T23:12:09.661 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.661 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.661 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-22T23:12:09.661 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.662 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.662 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.662 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.662 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.662 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.662 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.662 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.662 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.663 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.663 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.663 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.663 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.663 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.663 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.663 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.663 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.664 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.664 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.664 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.664 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.664 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.664 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.664 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.665 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.665 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.665 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.665 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.665 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.665 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.665 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.665 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.666 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.666 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.666 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.666 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.666 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.666 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.666 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.666 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.667 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.667 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.667 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.667 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.667 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.667 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.667 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.667 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.668 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.668 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.668 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.668 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.668 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.668 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.668 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.668 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.669 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.669 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.669 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.669 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.669 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.669 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.669 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.669 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.670 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.670 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.670 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.670 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.670 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.670 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.670 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.671 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.671 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.671 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.671 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.671 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.671 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.671 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "localpool", 2024-05-22T23:12:09.671 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.671 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.672 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.672 INFO:teuthology.orchestra.run.smithi084.stdout: "failure_domain": { 2024-05-22T23:12:09.672 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "failure_domain", 2024-05-22T23:12:09.672 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.672 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.672 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.672 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "host", 2024-05-22T23:12:09.672 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.673 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.673 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.673 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "failure domain for any created local pool", 2024-05-22T23:12:09.673 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-22T23:12:09.673 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.673 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.673 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.673 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.674 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.674 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.674 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.674 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.674 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.674 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.674 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.675 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.675 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.675 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.675 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.675 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.675 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.675 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.675 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.676 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.676 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.676 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.676 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.676 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.676 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.676 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.676 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.676 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.677 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.677 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.677 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.677 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.677 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.677 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.677 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.677 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.678 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.678 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.678 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.678 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.678 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.678 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.678 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.678 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.678 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.679 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.679 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.679 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.679 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.679 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.679 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.679 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.679 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.680 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.680 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.680 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.680 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.680 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.680 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.680 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.680 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.681 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.681 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.681 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.681 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.681 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.681 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.681 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.681 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.682 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.682 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.682 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.682 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.682 INFO:teuthology.orchestra.run.smithi084.stdout: "min_size": { 2024-05-22T23:12:09.682 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "min_size", 2024-05-22T23:12:09.682 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.682 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.683 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.683 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.683 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.683 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.683 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.683 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "default min_size for any created local pool", 2024-05-22T23:12:09.683 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-22T23:12:09.683 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.684 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.684 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.684 INFO:teuthology.orchestra.run.smithi084.stdout: "num_rep": { 2024-05-22T23:12:09.684 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "num_rep", 2024-05-22T23:12:09.684 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.684 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.684 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.685 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "3", 2024-05-22T23:12:09.685 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.685 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.685 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.685 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "default replica count for any created local pool", 2024-05-22T23:12:09.685 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.685 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.685 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.686 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.686 INFO:teuthology.orchestra.run.smithi084.stdout: "pg_num": { 2024-05-22T23:12:09.686 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "pg_num", 2024-05-22T23:12:09.686 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.686 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.686 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.686 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "128", 2024-05-22T23:12:09.687 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.687 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.687 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.687 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "default pg_num for any created local pool", 2024-05-22T23:12:09.687 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.687 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.687 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.687 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.687 INFO:teuthology.orchestra.run.smithi084.stdout: "prefix": { 2024-05-22T23:12:09.688 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "prefix", 2024-05-22T23:12:09.688 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.688 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.688 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.688 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.688 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.688 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.688 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.689 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "name prefix for any created local pool", 2024-05-22T23:12:09.689 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.689 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.689 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.689 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.689 INFO:teuthology.orchestra.run.smithi084.stdout: "subtree": { 2024-05-22T23:12:09.689 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "subtree", 2024-05-22T23:12:09.689 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.690 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.690 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.690 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "rack", 2024-05-22T23:12:09.690 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.690 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.690 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.690 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-22T23:12:09.690 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-22T23:12:09.691 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.691 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.691 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.691 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.691 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.691 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.691 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mds_autoscaler", 2024-05-22T23:12:09.691 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.691 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.692 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.692 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.692 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.692 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.692 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.692 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.692 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.692 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.693 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.693 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.693 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.693 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.693 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.693 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.693 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.693 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.693 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.694 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.694 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.694 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.694 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.694 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.694 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.694 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.694 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.695 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.695 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.695 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.695 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.695 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.695 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.695 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.695 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.696 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.696 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.696 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.696 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.696 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.696 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.696 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.696 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.697 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.697 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.697 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.697 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.697 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.697 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.697 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.697 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.698 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.698 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.698 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.698 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.698 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.698 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.698 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.699 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.699 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.699 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.699 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.699 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.699 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.699 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.699 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.699 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.700 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.700 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.700 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.700 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.700 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.700 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.700 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.700 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.701 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.701 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mirroring", 2024-05-22T23:12:09.701 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.701 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.701 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.701 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.701 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.701 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.702 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.702 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.702 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.702 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.702 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.702 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.702 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.702 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.703 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.703 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.703 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.703 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.703 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.703 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.703 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.703 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.704 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.704 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.704 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.704 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.704 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.704 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.704 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.704 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.705 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.705 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.705 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.705 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.705 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.705 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.705 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.705 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.706 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.706 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.706 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.706 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.706 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.706 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.706 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.706 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.707 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.707 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.707 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.707 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.707 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.707 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.707 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.707 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.708 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.708 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.708 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.708 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.708 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.708 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.708 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.708 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.709 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.709 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.709 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.709 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.709 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.709 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.709 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.710 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.710 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.710 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.710 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.710 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.710 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.710 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.711 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "nfs", 2024-05-22T23:12:09.711 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.711 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.711 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.711 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.711 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.711 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.712 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.712 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.712 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.712 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.712 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.712 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.712 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.712 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.713 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.713 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.713 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.713 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.713 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.713 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.713 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.713 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.713 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.714 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.714 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.714 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.714 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.714 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.714 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.714 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.714 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.715 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.715 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.715 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.715 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.715 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.715 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.715 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.715 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.716 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.716 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.716 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.716 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.716 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.716 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.716 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.716 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.717 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.717 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.717 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.717 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.717 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.717 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.717 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.717 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.718 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.718 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.718 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.718 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.718 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.718 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.718 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.718 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.719 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.719 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.719 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.719 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.719 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.719 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.719 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.719 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.720 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.720 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.720 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.720 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.720 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.720 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "orchestrator", 2024-05-22T23:12:09.720 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.720 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.721 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.721 INFO:teuthology.orchestra.run.smithi084.stdout: "fail_fs": { 2024-05-22T23:12:09.721 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "fail_fs", 2024-05-22T23:12:09.721 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.721 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.721 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.721 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.721 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.722 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.722 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.722 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-22T23:12:09.722 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.722 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.722 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.722 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.722 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.723 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.723 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.723 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.723 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.723 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.723 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.723 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.723 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.724 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.724 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.724 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.724 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.724 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.724 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.724 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.725 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.725 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.725 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.725 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.725 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.725 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.725 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.725 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.726 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.726 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.726 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.726 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.726 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.726 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.726 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.726 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.727 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.727 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.727 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.727 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.727 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.727 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.727 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.727 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.728 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.728 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.728 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.728 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.728 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.728 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.728 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.728 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.729 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.729 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.729 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.729 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.729 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.729 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.729 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.730 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.730 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.730 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.730 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.730 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.730 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.730 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.730 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.731 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.731 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.731 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.731 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.731 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.731 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.731 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.731 INFO:teuthology.orchestra.run.smithi084.stdout: "orchestrator": { 2024-05-22T23:12:09.732 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "orchestrator", 2024-05-22T23:12:09.732 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.732 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.732 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.732 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.732 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.732 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.732 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.733 INFO:teuthology.orchestra.run.smithi084.stdout: "cephadm", 2024-05-22T23:12:09.733 INFO:teuthology.orchestra.run.smithi084.stdout: "rook", 2024-05-22T23:12:09.733 INFO:teuthology.orchestra.run.smithi084.stdout: "test_orchestrator" 2024-05-22T23:12:09.733 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.733 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Orchestrator backend", 2024-05-22T23:12:09.733 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.733 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.733 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.734 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.734 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.734 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.734 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.734 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "osd_perf_query", 2024-05-22T23:12:09.734 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.734 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.735 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.735 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.735 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.735 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.735 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.735 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.735 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.735 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.736 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.736 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.736 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.736 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.736 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.736 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.736 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.737 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.737 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.737 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.737 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.737 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.737 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.737 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.737 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.737 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.738 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.738 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.738 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.738 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.738 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.738 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.738 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.739 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.739 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.739 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.739 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.739 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.739 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.739 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.740 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.740 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.740 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.740 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.740 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.740 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.740 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.740 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.741 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.741 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.741 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.741 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.741 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.741 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.741 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.741 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.742 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.742 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.742 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.742 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.742 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.742 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.742 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.742 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.743 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.743 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.743 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.743 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.743 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.743 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.743 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.744 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.744 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.744 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.744 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.744 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.744 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "osd_support", 2024-05-22T23:12:09.744 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.744 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.745 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.745 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.745 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.745 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.745 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.745 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.745 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.746 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.746 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.746 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.746 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.746 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.746 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.746 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.746 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.747 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.747 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.747 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.747 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.747 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.747 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.747 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.748 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.748 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.748 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.748 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.748 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.748 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.748 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.748 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.749 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.749 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.749 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.749 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.749 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.749 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.749 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.750 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.750 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.750 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.750 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.750 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.750 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.750 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.750 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.751 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.751 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.751 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.751 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.751 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.751 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.751 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.752 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.752 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.752 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.752 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.752 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.752 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.752 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.752 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.753 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.753 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.753 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.753 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.753 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.753 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.753 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.753 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.753 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.754 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.754 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.754 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.754 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.754 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.754 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "pg_autoscaler", 2024-05-22T23:12:09.754 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.754 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.755 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.755 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.755 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.755 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.755 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.755 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.755 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.755 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.756 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.756 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.756 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.756 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.756 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.756 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.756 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.756 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.757 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.757 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.757 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.757 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.757 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.757 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.757 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.757 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.758 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.758 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.758 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.758 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.758 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.758 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.758 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.758 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.759 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.759 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.759 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.759 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.759 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.759 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.759 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.760 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.760 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.760 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.760 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.760 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.760 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.760 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.760 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.761 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.761 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.761 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.761 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.761 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.761 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.761 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.761 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.762 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.762 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.762 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.762 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.762 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.762 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.762 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.762 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.763 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.763 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.763 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.763 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.763 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.763 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.763 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.763 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.763 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-05-22T23:12:09.764 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-05-22T23:12:09.764 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:09.764 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.764 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.764 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-05-22T23:12:09.764 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.764 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.764 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.765 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.765 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.765 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.765 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.765 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.765 INFO:teuthology.orchestra.run.smithi084.stdout: "threshold": { 2024-05-22T23:12:09.765 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "threshold", 2024-05-22T23:12:09.765 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-05-22T23:12:09.766 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.766 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.766 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "3.0", 2024-05-22T23:12:09.766 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1.0", 2024-05-22T23:12:09.766 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.766 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.766 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "scaling threshold", 2024-05-22T23:12:09.766 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-22T23:12:09.767 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.767 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.767 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.767 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.767 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.767 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.767 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "progress", 2024-05-22T23:12:09.767 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.768 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.768 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.768 INFO:teuthology.orchestra.run.smithi084.stdout: "allow_pg_recovery_event": { 2024-05-22T23:12:09.768 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "allow_pg_recovery_event", 2024-05-22T23:12:09.768 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.768 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.768 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.768 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.768 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.769 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.769 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.769 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-22T23:12:09.769 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.769 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.769 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.769 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.769 INFO:teuthology.orchestra.run.smithi084.stdout: "enabled": { 2024-05-22T23:12:09.770 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "enabled", 2024-05-22T23:12:09.770 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.770 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.770 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.770 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.770 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.770 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.770 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.771 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.771 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.771 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.771 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.771 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.771 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.771 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.771 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.772 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.772 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.772 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.772 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.772 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.772 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.772 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.772 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.772 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.773 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.773 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.773 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.773 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.773 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.773 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.773 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.773 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.774 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.774 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.774 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.774 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.774 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.774 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.774 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.774 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.775 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.775 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.775 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.775 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.775 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.775 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.775 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.775 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.776 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.776 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.776 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.776 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.776 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.776 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.776 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.776 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.777 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.777 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.777 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.777 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.777 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.777 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.777 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.777 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.777 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.778 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.778 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.778 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.778 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.778 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.778 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.778 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.779 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.779 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.779 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.779 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.779 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.779 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.779 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.779 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.779 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.780 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.780 INFO:teuthology.orchestra.run.smithi084.stdout: "max_completed_events": { 2024-05-22T23:12:09.780 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_completed_events", 2024-05-22T23:12:09.780 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.780 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.780 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.780 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "50", 2024-05-22T23:12:09.780 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.780 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.781 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.781 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "number of past completed events to remember", 2024-05-22T23:12:09.781 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.781 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.781 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.781 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.781 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-05-22T23:12:09.781 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-05-22T23:12:09.782 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:09.782 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.782 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.782 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5", 2024-05-22T23:12:09.782 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.782 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.782 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.782 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how long the module is going to sleep", 2024-05-22T23:12:09.783 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.783 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.783 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.783 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.783 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.783 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.783 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.784 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "prometheus", 2024-05-22T23:12:09.784 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.784 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.784 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.784 INFO:teuthology.orchestra.run.smithi084.stdout: "cache": { 2024-05-22T23:12:09.784 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cache", 2024-05-22T23:12:09.784 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.784 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.784 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.785 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.785 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.785 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.785 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.785 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.785 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.785 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.785 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.786 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.786 INFO:teuthology.orchestra.run.smithi084.stdout: "exclude_perf_counters": { 2024-05-22T23:12:09.786 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "exclude_perf_counters", 2024-05-22T23:12:09.786 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.786 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.786 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.786 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.786 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.787 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.787 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.787 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-22T23:12:09.787 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-22T23:12:09.787 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.787 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.787 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.788 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.788 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.788 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.788 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.788 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.788 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.788 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.788 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.789 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.789 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.789 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.789 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.789 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.789 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.789 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.790 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.790 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.790 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.790 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.790 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.790 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.790 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.790 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.791 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.791 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.791 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.791 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.791 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.791 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.791 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.791 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.792 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.792 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.792 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.792 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.792 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.792 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.792 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.792 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.793 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.793 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.793 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.793 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.793 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.793 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.793 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.793 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.793 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.794 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.794 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.794 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.794 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.794 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.794 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.794 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.794 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.795 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.795 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.795 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.795 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.795 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.795 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.795 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.795 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.796 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.796 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.796 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.796 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.796 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.796 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.796 INFO:teuthology.orchestra.run.smithi084.stdout: "rbd_stats_pools": { 2024-05-22T23:12:09.796 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rbd_stats_pools", 2024-05-22T23:12:09.796 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.796 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.797 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.797 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.797 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.797 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.797 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.797 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.797 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.797 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.798 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.798 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.798 INFO:teuthology.orchestra.run.smithi084.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-22T23:12:09.798 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-22T23:12:09.798 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.798 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.798 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.799 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "300", 2024-05-22T23:12:09.799 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.799 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.799 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.799 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.799 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.799 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.799 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.799 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.800 INFO:teuthology.orchestra.run.smithi084.stdout: "scrape_interval": { 2024-05-22T23:12:09.800 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "scrape_interval", 2024-05-22T23:12:09.800 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-05-22T23:12:09.800 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.800 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.800 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "15.0", 2024-05-22T23:12:09.800 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.800 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.800 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.801 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.801 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.801 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.801 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.801 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.801 INFO:teuthology.orchestra.run.smithi084.stdout: "server_addr": { 2024-05-22T23:12:09.801 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_addr", 2024-05-22T23:12:09.801 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.801 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.802 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.802 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "::", 2024-05-22T23:12:09.802 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.802 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.802 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.802 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-22T23:12:09.802 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.802 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.803 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.803 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.803 INFO:teuthology.orchestra.run.smithi084.stdout: "server_port": { 2024-05-22T23:12:09.803 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_port", 2024-05-22T23:12:09.803 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.803 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.803 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.803 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "9283", 2024-05-22T23:12:09.804 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.804 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.804 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.804 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-22T23:12:09.804 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.804 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.804 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.804 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.805 INFO:teuthology.orchestra.run.smithi084.stdout: "stale_cache_strategy": { 2024-05-22T23:12:09.805 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "stale_cache_strategy", 2024-05-22T23:12:09.805 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.805 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.805 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.805 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "log", 2024-05-22T23:12:09.805 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.805 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.806 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.806 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.806 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.806 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.806 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.806 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.806 INFO:teuthology.orchestra.run.smithi084.stdout: "standby_behaviour": { 2024-05-22T23:12:09.807 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "standby_behaviour", 2024-05-22T23:12:09.807 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.807 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.807 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.807 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "default", 2024-05-22T23:12:09.807 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.807 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.807 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.808 INFO:teuthology.orchestra.run.smithi084.stdout: "default", 2024-05-22T23:12:09.808 INFO:teuthology.orchestra.run.smithi084.stdout: "error" 2024-05-22T23:12:09.808 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.808 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.808 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.808 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.808 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.808 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.809 INFO:teuthology.orchestra.run.smithi084.stdout: "standby_error_status_code": { 2024-05-22T23:12:09.809 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "standby_error_status_code", 2024-05-22T23:12:09.809 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.809 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.809 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.809 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "500", 2024-05-22T23:12:09.809 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "400", 2024-05-22T23:12:09.809 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "599", 2024-05-22T23:12:09.809 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.810 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.810 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.810 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.810 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.810 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.810 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.810 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.811 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.811 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rbd_support", 2024-05-22T23:12:09.811 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.811 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.811 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.811 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.811 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.811 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.812 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.812 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.812 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.812 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.812 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.812 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.812 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.812 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.813 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.813 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.813 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.813 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.813 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.813 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.813 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.814 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.814 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.814 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.814 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.814 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.814 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.814 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.814 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.815 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.815 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.815 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.815 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.815 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.815 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.815 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.816 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.816 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.816 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.816 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.816 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.816 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.816 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.816 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.817 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.817 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.817 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.817 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.817 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.817 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.817 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.817 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.818 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.818 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.818 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.818 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.818 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.818 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.818 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.818 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.819 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.819 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.819 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.819 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.819 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.819 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.819 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.819 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.820 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.820 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.820 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.820 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.820 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.820 INFO:teuthology.orchestra.run.smithi084.stdout: "max_concurrent_snap_create": { 2024-05-22T23:12:09.820 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_concurrent_snap_create", 2024-05-22T23:12:09.821 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.821 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.821 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.821 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-05-22T23:12:09.821 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.821 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.821 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.822 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.822 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.822 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.822 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.822 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.822 INFO:teuthology.orchestra.run.smithi084.stdout: "mirror_snapshot_schedule": { 2024-05-22T23:12:09.822 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mirror_snapshot_schedule", 2024-05-22T23:12:09.822 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.823 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.823 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.823 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.823 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.823 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.823 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.823 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.823 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.824 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.824 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.824 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.824 INFO:teuthology.orchestra.run.smithi084.stdout: "trash_purge_schedule": { 2024-05-22T23:12:09.824 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "trash_purge_schedule", 2024-05-22T23:12:09.824 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.824 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.824 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.824 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.825 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.825 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.825 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.825 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.825 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.825 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.825 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.825 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.826 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.826 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.826 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.826 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "restful", 2024-05-22T23:12:09.826 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.826 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.826 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.826 INFO:teuthology.orchestra.run.smithi084.stdout: "enable_auth": { 2024-05-22T23:12:09.827 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "enable_auth", 2024-05-22T23:12:09.827 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.827 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.827 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.827 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.827 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.827 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.827 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.827 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.828 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.828 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.828 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.828 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.828 INFO:teuthology.orchestra.run.smithi084.stdout: "key_file": { 2024-05-22T23:12:09.828 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "key_file", 2024-05-22T23:12:09.828 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.828 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.829 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.829 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.829 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.829 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.829 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.829 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.829 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.830 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.830 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.830 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.830 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.830 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.830 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.830 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.831 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.831 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.831 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.831 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.831 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.831 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.831 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.831 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.832 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.832 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.832 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.832 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.832 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.832 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.832 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.832 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.832 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.833 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.833 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.833 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.833 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.833 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.833 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.833 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.834 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.834 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.834 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.834 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.834 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.834 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.834 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.834 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.835 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.835 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.835 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.835 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.835 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.835 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.835 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.835 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.836 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.836 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.836 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.836 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.836 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.836 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.836 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.836 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.837 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.837 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.837 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.837 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.837 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.837 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.837 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.837 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.837 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.838 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.838 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.838 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.838 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.838 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.838 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.838 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.838 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.839 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.839 INFO:teuthology.orchestra.run.smithi084.stdout: "server_addr": { 2024-05-22T23:12:09.839 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_addr", 2024-05-22T23:12:09.839 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.839 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.839 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.839 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.840 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.840 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.840 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.840 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.840 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.840 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.840 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.840 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.841 INFO:teuthology.orchestra.run.smithi084.stdout: "server_port": { 2024-05-22T23:12:09.841 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_port", 2024-05-22T23:12:09.841 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.841 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.841 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.841 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.841 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.841 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.842 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.842 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.842 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.842 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.842 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.842 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.842 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.842 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.843 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.843 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rgw", 2024-05-22T23:12:09.843 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.843 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.843 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.843 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.843 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.843 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.844 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.844 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.844 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.844 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.844 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.844 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.844 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.844 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.844 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.845 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.845 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.845 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.845 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.845 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.845 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.845 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.845 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.846 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.846 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.846 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.846 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.846 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.846 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.846 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.846 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.847 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.847 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.847 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.847 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.847 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.847 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.847 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.847 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.848 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.848 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.848 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.848 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.848 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.848 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.848 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.848 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.848 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.849 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.849 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.849 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.849 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.849 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.849 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.849 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.849 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.850 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.850 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.850 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.850 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.850 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.850 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.850 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.850 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.851 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.851 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.851 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.851 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.851 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.851 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.851 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.851 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.852 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.852 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.852 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.852 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.852 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rook", 2024-05-22T23:12:09.852 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.852 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.852 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.853 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.853 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.853 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.853 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.853 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.853 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.853 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.853 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.854 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.854 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.854 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.854 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.854 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.854 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.854 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.855 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.855 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.855 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.855 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.855 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.855 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.855 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.855 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.856 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.856 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.856 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.856 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.856 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.856 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.856 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.857 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.857 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.857 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.857 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.857 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.857 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.857 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.857 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.858 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.858 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.858 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.858 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.858 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.858 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.858 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.858 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.861 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.861 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.861 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.862 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.862 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.862 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.862 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.862 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.862 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.862 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.862 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.863 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.863 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.863 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.863 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.863 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.863 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.863 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.863 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.864 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.864 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.864 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.864 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.864 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.864 INFO:teuthology.orchestra.run.smithi084.stdout: "storage_class": { 2024-05-22T23:12:09.864 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "storage_class", 2024-05-22T23:12:09.864 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.865 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.865 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.865 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "local", 2024-05-22T23:12:09.865 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.865 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.865 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.865 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-22T23:12:09.866 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.866 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.866 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.866 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.866 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.866 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.866 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.866 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "selftest", 2024-05-22T23:12:09.867 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.867 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.867 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.867 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.867 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.867 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.867 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.867 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.868 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.868 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.868 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.868 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.868 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.868 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.868 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.868 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.868 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.869 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.869 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.869 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.869 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.869 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.869 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.869 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.869 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.870 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.870 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.870 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.870 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.870 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.870 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.870 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.871 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.871 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.871 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.871 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.871 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.871 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.871 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.871 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.872 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.872 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.872 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.872 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.872 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.872 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.872 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.872 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.873 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.873 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.873 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.873 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.873 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.873 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.873 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.874 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.874 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.874 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.874 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.874 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.874 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.874 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.874 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.875 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.875 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.875 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.875 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.875 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.875 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.875 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.875 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.876 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.876 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.876 INFO:teuthology.orchestra.run.smithi084.stdout: "roption1": { 2024-05-22T23:12:09.876 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "roption1", 2024-05-22T23:12:09.876 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.876 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.876 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.876 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.877 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.877 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.877 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.877 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.877 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.877 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.877 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.877 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.878 INFO:teuthology.orchestra.run.smithi084.stdout: "roption2": { 2024-05-22T23:12:09.878 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "roption2", 2024-05-22T23:12:09.878 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.878 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.878 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.878 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "xyz", 2024-05-22T23:12:09.878 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.878 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.879 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.879 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.879 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.879 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.879 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.879 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.879 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption1": { 2024-05-22T23:12:09.879 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption1", 2024-05-22T23:12:09.880 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.880 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.880 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.880 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.880 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.880 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.880 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.881 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.881 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.881 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.881 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.881 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.881 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption2": { 2024-05-22T23:12:09.881 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption2", 2024-05-22T23:12:09.881 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.882 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.882 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.882 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.882 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.882 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.882 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.882 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.883 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.883 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.883 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.883 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.883 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption3": { 2024-05-22T23:12:09.883 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption3", 2024-05-22T23:12:09.883 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-05-22T23:12:09.883 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.884 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.884 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.884 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.884 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.884 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.884 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.884 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.885 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.885 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.885 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.885 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption4": { 2024-05-22T23:12:09.885 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption4", 2024-05-22T23:12:09.885 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.885 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.885 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.885 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.886 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.886 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.886 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.886 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.886 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.886 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.886 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.886 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.887 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption5": { 2024-05-22T23:12:09.887 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption5", 2024-05-22T23:12:09.887 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.887 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.887 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.887 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.887 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.887 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.888 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.888 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.888 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.888 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.888 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.888 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.888 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption6": { 2024-05-22T23:12:09.888 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption6", 2024-05-22T23:12:09.889 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.889 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.889 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.889 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.889 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.889 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.889 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.889 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.889 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.890 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.890 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.890 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.890 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption7": { 2024-05-22T23:12:09.890 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption7", 2024-05-22T23:12:09.890 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.890 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.890 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.891 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.891 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1", 2024-05-22T23:12:09.891 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "42", 2024-05-22T23:12:09.891 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.891 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.891 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.891 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.891 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.892 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.892 INFO:teuthology.orchestra.run.smithi084.stdout: "testkey": { 2024-05-22T23:12:09.892 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "testkey", 2024-05-22T23:12:09.892 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.892 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.892 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.892 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.892 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.892 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.893 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.893 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.893 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.893 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.893 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.893 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.893 INFO:teuthology.orchestra.run.smithi084.stdout: "testlkey": { 2024-05-22T23:12:09.893 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "testlkey", 2024-05-22T23:12:09.893 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.894 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.894 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.894 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.894 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.894 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.894 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.894 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.894 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.895 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.895 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.895 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.895 INFO:teuthology.orchestra.run.smithi084.stdout: "testnewline": { 2024-05-22T23:12:09.895 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "testnewline", 2024-05-22T23:12:09.895 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.895 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.896 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.896 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.896 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.896 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.896 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.896 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.896 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.896 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.897 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.897 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.897 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.897 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.897 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.897 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "snap_schedule", 2024-05-22T23:12:09.897 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.897 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.897 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.898 INFO:teuthology.orchestra.run.smithi084.stdout: "allow_m_granularity": { 2024-05-22T23:12:09.898 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "allow_m_granularity", 2024-05-22T23:12:09.898 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.898 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.898 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.898 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.898 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.899 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.899 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.899 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "allow minute scheduled snapshots", 2024-05-22T23:12:09.899 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.899 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.899 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.899 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.899 INFO:teuthology.orchestra.run.smithi084.stdout: "dump_on_update": { 2024-05-22T23:12:09.900 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "dump_on_update", 2024-05-22T23:12:09.900 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.900 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.900 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.900 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.900 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.900 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.900 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.901 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "dump database to debug log on update", 2024-05-22T23:12:09.901 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.901 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.901 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.901 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.901 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.901 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.901 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.902 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.902 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.902 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.902 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.902 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.902 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.902 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.902 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.903 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.903 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.903 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.903 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.903 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.903 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.903 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.904 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.904 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.904 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.904 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.904 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.904 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.904 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.904 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.904 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.905 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.905 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.905 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.905 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.905 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.905 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.905 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.905 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.906 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.906 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.906 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.906 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.906 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.906 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.906 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.907 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.907 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.907 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.907 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.907 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.907 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.907 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.907 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.907 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.908 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.908 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.908 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.908 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.908 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.908 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.908 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.908 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.908 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.909 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.909 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.909 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.909 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.909 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.909 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.909 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.909 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.910 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.910 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.910 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.910 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.910 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.910 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "stats", 2024-05-22T23:12:09.910 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.910 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.911 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.911 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.911 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.911 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.911 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.911 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.911 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.911 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.912 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.912 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.912 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.912 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.912 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.912 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.912 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.912 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.913 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.913 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.913 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.913 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.913 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.913 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.913 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.913 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.913 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.914 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.914 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.914 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.914 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.914 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.914 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.914 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.915 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.915 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.915 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.915 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.915 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.915 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.915 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.915 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.916 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.916 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.916 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.916 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.916 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.916 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.916 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.916 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.916 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.917 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.917 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.917 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.917 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.917 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.917 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.917 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.917 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.918 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.918 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.918 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.918 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.918 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.918 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.918 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.918 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.919 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.919 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.919 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.919 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.919 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.919 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.919 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.919 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.920 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.920 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "status", 2024-05-22T23:12:09.920 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.920 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.920 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.920 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.920 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.920 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.920 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.921 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.921 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.921 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.921 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.921 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.921 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.921 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.921 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.922 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.922 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.922 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.922 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.922 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.922 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.922 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.923 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.923 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.923 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.923 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.923 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.923 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.923 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.923 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.924 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.924 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.924 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.924 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.924 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.924 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.924 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.924 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.924 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.925 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.925 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.925 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.925 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.925 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.925 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.925 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.925 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.926 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.926 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.926 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.926 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.926 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.926 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.926 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.927 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.927 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.927 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.927 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.927 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.927 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.927 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.927 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.928 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.928 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.928 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.928 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.928 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.928 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.928 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.928 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.929 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.929 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.929 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.929 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.929 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.929 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.929 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "telegraf", 2024-05-22T23:12:09.929 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.930 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.930 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.930 INFO:teuthology.orchestra.run.smithi084.stdout: "address": { 2024-05-22T23:12:09.930 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "address", 2024-05-22T23:12:09.930 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.930 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.930 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.931 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-22T23:12:09.931 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.931 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.931 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.931 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.931 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.931 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.931 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.932 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.932 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-05-22T23:12:09.932 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-05-22T23:12:09.932 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:09.932 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.932 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.932 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "15", 2024-05-22T23:12:09.932 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.933 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.933 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.933 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.933 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.933 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.933 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.933 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.933 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.934 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.934 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.934 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.934 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.934 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.934 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.934 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.934 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.935 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.935 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.935 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.935 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.935 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.935 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.935 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.935 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.936 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.936 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.936 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.936 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.936 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.936 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.936 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.936 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.937 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.937 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.937 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.937 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.937 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.937 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.937 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.938 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.938 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.938 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.938 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.938 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.938 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.938 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.938 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.939 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.939 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.939 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.939 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.939 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.939 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.939 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.939 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.940 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.940 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.940 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.940 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.940 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.940 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.940 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.941 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.941 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.941 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.941 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.941 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.941 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.941 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.941 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.942 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.942 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.942 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.942 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.942 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.942 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.942 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.942 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.943 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.943 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.943 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "telemetry", 2024-05-22T23:12:09.943 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.943 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.943 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.943 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_basic": { 2024-05-22T23:12:09.943 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_basic", 2024-05-22T23:12:09.943 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.944 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.944 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.944 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.944 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.944 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.944 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.944 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-22T23:12:09.944 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.944 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.945 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.945 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.945 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_crash": { 2024-05-22T23:12:09.945 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_crash", 2024-05-22T23:12:09.945 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.945 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.945 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.945 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.946 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.946 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.946 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.946 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-22T23:12:09.946 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.946 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.946 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.946 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.947 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_device": { 2024-05-22T23:12:09.947 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_device", 2024-05-22T23:12:09.947 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.947 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.947 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.947 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:09.947 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.947 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.947 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.948 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-22T23:12:09.948 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.948 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.948 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.948 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.948 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_ident": { 2024-05-22T23:12:09.948 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_ident", 2024-05-22T23:12:09.948 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.948 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.949 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.949 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.949 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.949 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.949 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.949 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-22T23:12:09.949 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.949 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.949 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.950 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.950 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_perf": { 2024-05-22T23:12:09.950 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_perf", 2024-05-22T23:12:09.950 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.950 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.950 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.950 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.950 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.951 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.951 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.951 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-22T23:12:09.951 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.951 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.951 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.951 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.951 INFO:teuthology.orchestra.run.smithi084.stdout: "contact": { 2024-05-22T23:12:09.952 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "contact", 2024-05-22T23:12:09.952 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.952 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.952 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.952 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.952 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.952 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.953 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.953 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.953 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.953 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.953 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.953 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.953 INFO:teuthology.orchestra.run.smithi084.stdout: "description": { 2024-05-22T23:12:09.953 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "description", 2024-05-22T23:12:09.954 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.954 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.954 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.954 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.954 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.954 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.954 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.955 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.955 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.955 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.955 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.955 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.955 INFO:teuthology.orchestra.run.smithi084.stdout: "device_url": { 2024-05-22T23:12:09.955 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "device_url", 2024-05-22T23:12:09.955 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.956 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.956 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.956 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-22T23:12:09.956 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.956 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.956 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.956 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.956 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.957 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.957 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.957 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.957 INFO:teuthology.orchestra.run.smithi084.stdout: "enabled": { 2024-05-22T23:12:09.957 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "enabled", 2024-05-22T23:12:09.957 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.957 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.957 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.958 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.958 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.958 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.958 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.958 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.958 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.958 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.959 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.959 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.959 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-05-22T23:12:09.959 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-05-22T23:12:09.959 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.959 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.959 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.959 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "24", 2024-05-22T23:12:09.960 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "8", 2024-05-22T23:12:09.960 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.960 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.960 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.960 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.960 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.960 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.960 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.961 INFO:teuthology.orchestra.run.smithi084.stdout: "last_opt_revision": { 2024-05-22T23:12:09.961 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "last_opt_revision", 2024-05-22T23:12:09.961 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:09.961 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.961 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.961 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "1", 2024-05-22T23:12:09.961 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.962 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.962 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.962 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.962 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.962 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.962 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.962 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.962 INFO:teuthology.orchestra.run.smithi084.stdout: "leaderboard": { 2024-05-22T23:12:09.963 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "leaderboard", 2024-05-22T23:12:09.963 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.963 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.963 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.963 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.963 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.963 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.963 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.964 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.964 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.964 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.964 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.964 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.964 INFO:teuthology.orchestra.run.smithi084.stdout: "leaderboard_description": { 2024-05-22T23:12:09.964 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "leaderboard_description", 2024-05-22T23:12:09.964 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.965 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.965 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.965 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.965 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.965 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.965 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.965 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.965 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.966 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.966 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.966 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.966 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.966 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.966 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.966 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.967 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.967 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.967 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.967 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.967 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.967 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.967 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.967 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.968 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.968 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.968 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.968 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.968 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.968 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.968 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.968 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.969 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.969 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.969 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.969 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.969 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.969 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.969 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.969 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.970 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.970 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.970 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.970 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.970 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.970 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.970 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.970 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.971 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.971 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.971 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.971 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.971 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.971 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.971 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.971 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.972 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.972 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.972 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.972 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.972 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.972 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.972 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.972 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.973 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.973 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.973 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.973 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.973 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.973 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.973 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.973 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.974 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.974 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.974 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.974 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.974 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.974 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.974 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.974 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.975 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.975 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.975 INFO:teuthology.orchestra.run.smithi084.stdout: "organization": { 2024-05-22T23:12:09.975 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "organization", 2024-05-22T23:12:09.975 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.975 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.975 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.975 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.976 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.976 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.976 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.976 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.976 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.976 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.976 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.976 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.977 INFO:teuthology.orchestra.run.smithi084.stdout: "proxy": { 2024-05-22T23:12:09.977 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "proxy", 2024-05-22T23:12:09.977 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.977 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.977 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.977 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.977 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.978 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.978 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.978 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.978 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.978 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.978 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.978 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.979 INFO:teuthology.orchestra.run.smithi084.stdout: "url": { 2024-05-22T23:12:09.979 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "url", 2024-05-22T23:12:09.979 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.979 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.979 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:09.979 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-22T23:12:09.979 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.979 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.980 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.980 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.980 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.980 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.980 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.980 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.980 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.980 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.981 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.981 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "test_orchestrator", 2024-05-22T23:12:09.981 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.981 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.981 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.981 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.981 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.981 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.982 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.982 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.982 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.982 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.982 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.982 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.982 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.982 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.983 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.983 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.983 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.983 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.983 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.983 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.983 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.983 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.984 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.984 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.984 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.984 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.984 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.984 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.984 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.984 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.985 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.985 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.985 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.985 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.985 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.985 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.985 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.985 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.986 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.986 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.986 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.986 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.986 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.986 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.986 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.987 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.987 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.987 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.987 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.987 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.987 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.987 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.987 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.988 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.988 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.988 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.988 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.988 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.988 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.988 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.988 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.989 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.989 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.989 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.989 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.989 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.989 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.989 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.989 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.990 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.990 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.990 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.990 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.990 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:09.990 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.990 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:09.990 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "volumes", 2024-05-22T23:12:09.991 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:09.991 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:09.991 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:09.991 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:09.991 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:09.991 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.991 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.991 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.992 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:09.992 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.992 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.992 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.992 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.992 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.992 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.992 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.993 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.993 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.993 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.993 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.993 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.993 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.993 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.993 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.994 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:09.994 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:09.994 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.994 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.994 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.994 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.994 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.995 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.995 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.995 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.995 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.995 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.995 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.995 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.995 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:09.996 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:09.996 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:09.996 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.996 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.996 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:09.996 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.996 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.996 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:09.997 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:09.997 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:09.997 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:09.997 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:09.997 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:09.997 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:09.997 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:09.997 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.998 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.998 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:09.998 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:09.998 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:09.998 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:09.998 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:09.998 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:09.998 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:09.999 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:09.999 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:09.999 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:09.999 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:09.999 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:09.999 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:09.999 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:09.999 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.000 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.000 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.000 INFO:teuthology.orchestra.run.smithi084.stdout: "max_concurrent_clones": { 2024-05-22T23:12:10.000 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_concurrent_clones", 2024-05-22T23:12:10.000 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.000 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.000 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.000 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "4", 2024-05-22T23:12:10.001 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.001 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.001 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.001 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-22T23:12:10.001 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.001 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.001 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.002 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.002 INFO:teuthology.orchestra.run.smithi084.stdout: "periodic_async_work": { 2024-05-22T23:12:10.002 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "periodic_async_work", 2024-05-22T23:12:10.002 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.002 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.002 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.002 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.002 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.003 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.003 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.003 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Periodically check for async work", 2024-05-22T23:12:10.003 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.003 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.003 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.004 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.004 INFO:teuthology.orchestra.run.smithi084.stdout: "snapshot_clone_delay": { 2024-05-22T23:12:10.004 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "snapshot_clone_delay", 2024-05-22T23:12:10.004 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.004 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.004 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.004 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-05-22T23:12:10.004 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.005 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.005 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.005 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-22T23:12:10.005 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.005 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.005 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.005 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.006 INFO:teuthology.orchestra.run.smithi084.stdout: "snapshot_clone_no_wait": { 2024-05-22T23:12:10.006 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "snapshot_clone_no_wait", 2024-05-22T23:12:10.006 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.006 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.006 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.006 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.006 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.006 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.007 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.007 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-22T23:12:10.007 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.007 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.007 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.007 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.007 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.007 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.008 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.008 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "zabbix", 2024-05-22T23:12:10.008 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.008 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.008 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.008 INFO:teuthology.orchestra.run.smithi084.stdout: "discovery_interval": { 2024-05-22T23:12:10.008 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "discovery_interval", 2024-05-22T23:12:10.008 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-05-22T23:12:10.009 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.009 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.009 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "100", 2024-05-22T23:12:10.009 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.009 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.009 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.009 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.009 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.010 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.010 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.010 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.010 INFO:teuthology.orchestra.run.smithi084.stdout: "identifier": { 2024-05-22T23:12:10.010 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "identifier", 2024-05-22T23:12:10.010 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.010 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.010 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.011 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.011 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.011 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.011 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.011 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.011 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.011 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.011 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.012 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.012 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-05-22T23:12:10.012 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-05-22T23:12:10.012 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.012 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.012 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.012 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-05-22T23:12:10.012 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.013 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.013 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.013 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.013 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.013 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.013 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.013 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.013 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.014 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.014 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.014 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.014 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.014 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.014 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.014 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.015 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.015 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.015 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.015 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.015 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.015 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.015 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.015 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.016 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.016 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.016 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.016 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.016 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.016 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.016 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.016 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.017 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.017 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.017 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.017 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.017 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.017 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.017 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.017 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.018 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.018 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.018 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.018 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.018 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.018 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.018 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.018 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.019 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.019 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.019 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.019 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.019 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.019 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.019 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.019 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.020 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.020 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.020 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.020 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.020 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.020 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.020 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.020 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.021 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.021 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.021 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.021 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.021 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.021 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.021 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.021 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.022 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.022 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.022 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.022 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.022 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.022 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.022 INFO:teuthology.orchestra.run.smithi084.stdout: "zabbix_host": { 2024-05-22T23:12:10.022 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "zabbix_host", 2024-05-22T23:12:10.023 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.023 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.023 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.023 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.023 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.023 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.023 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.024 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.024 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.024 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.024 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.024 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.024 INFO:teuthology.orchestra.run.smithi084.stdout: "zabbix_port": { 2024-05-22T23:12:10.024 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "zabbix_port", 2024-05-22T23:12:10.024 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.025 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.025 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.025 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10051", 2024-05-22T23:12:10.025 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.025 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.025 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.025 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.025 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.026 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.026 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.026 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.026 INFO:teuthology.orchestra.run.smithi084.stdout: "zabbix_sender": { 2024-05-22T23:12:10.026 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "zabbix_sender", 2024-05-22T23:12:10.026 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.026 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.026 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.027 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-22T23:12:10.027 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.027 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.027 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.027 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.027 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.027 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.028 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.028 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.028 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.028 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.028 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-05-22T23:12:10.028 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.028 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.029 INFO:teuthology.orchestra.run.smithi084.stdout: "gid": 24842, 2024-05-22T23:12:10.029 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "y", 2024-05-22T23:12:10.029 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr_features": 4540701547738038271, 2024-05-22T23:12:10.029 INFO:teuthology.orchestra.run.smithi084.stdout: "available_modules": [ 2024-05-22T23:12:10.029 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.029 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "alerts", 2024-05-22T23:12:10.029 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.029 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.030 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.030 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-05-22T23:12:10.030 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-05-22T23:12:10.030 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.030 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.030 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.030 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-05-22T23:12:10.030 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.031 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.031 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.031 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "How frequently to reexamine health status", 2024-05-22T23:12:10.031 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.031 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.031 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.031 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.031 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.032 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.032 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.032 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.032 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.032 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.032 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.032 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.033 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.033 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.033 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.033 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.033 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.033 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.033 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.033 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.034 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.034 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.034 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.034 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.034 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.034 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.034 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.034 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.035 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.035 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.035 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.035 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.035 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.035 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.035 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.035 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.036 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.036 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.036 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.036 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.036 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.036 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.036 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.036 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.037 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.037 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.037 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.037 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.037 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.037 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.037 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.037 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.038 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.038 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.038 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.038 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.038 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.038 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.038 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.038 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.039 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.039 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.039 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.039 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.039 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.039 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.039 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.039 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.040 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.040 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.040 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.040 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.040 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.040 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.040 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_destination": { 2024-05-22T23:12:10.040 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_destination", 2024-05-22T23:12:10.041 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.041 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.041 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.041 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.041 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.041 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.041 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.041 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Email address to send alerts to", 2024-05-22T23:12:10.042 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.042 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.042 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.042 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.042 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_from_name": { 2024-05-22T23:12:10.042 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_from_name", 2024-05-22T23:12:10.042 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.043 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.043 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.043 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "Ceph", 2024-05-22T23:12:10.043 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.043 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.043 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.043 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Email From: name", 2024-05-22T23:12:10.043 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.044 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.044 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.044 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.044 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_host": { 2024-05-22T23:12:10.044 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_host", 2024-05-22T23:12:10.044 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.044 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.045 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.045 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.045 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.045 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.045 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.045 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "SMTP server", 2024-05-22T23:12:10.045 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.045 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.045 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.046 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.046 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_password": { 2024-05-22T23:12:10.046 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_password", 2024-05-22T23:12:10.046 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.046 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.046 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.046 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.046 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.047 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.047 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.047 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Password to authenticate with", 2024-05-22T23:12:10.047 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.047 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.047 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.047 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.048 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_port": { 2024-05-22T23:12:10.048 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_port", 2024-05-22T23:12:10.048 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.048 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.048 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.048 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "465", 2024-05-22T23:12:10.048 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.048 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.049 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.049 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "SMTP port", 2024-05-22T23:12:10.049 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.049 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.049 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.049 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.049 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_sender": { 2024-05-22T23:12:10.049 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_sender", 2024-05-22T23:12:10.049 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.050 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.050 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.050 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.050 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.050 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.050 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.050 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "SMTP envelope sender", 2024-05-22T23:12:10.051 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.051 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.051 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.051 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.051 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_ssl": { 2024-05-22T23:12:10.051 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_ssl", 2024-05-22T23:12:10.051 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.051 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.052 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.052 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.052 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.052 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.052 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.052 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-22T23:12:10.052 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.052 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.053 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.053 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.053 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_user": { 2024-05-22T23:12:10.053 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_user", 2024-05-22T23:12:10.053 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.053 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.053 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.053 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.054 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.054 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.054 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.054 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "User to authenticate as", 2024-05-22T23:12:10.054 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.054 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.054 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.054 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.055 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.055 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.055 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.055 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "balancer", 2024-05-22T23:12:10.055 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.055 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.055 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.055 INFO:teuthology.orchestra.run.smithi084.stdout: "active": { 2024-05-22T23:12:10.056 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "active", 2024-05-22T23:12:10.056 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.056 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.056 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.056 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.056 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.056 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.056 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.057 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "automatically balance PGs across cluster", 2024-05-22T23:12:10.057 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.057 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.057 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.057 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.057 INFO:teuthology.orchestra.run.smithi084.stdout: "begin_time": { 2024-05-22T23:12:10.057 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "begin_time", 2024-05-22T23:12:10.057 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.058 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.058 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.058 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0000", 2024-05-22T23:12:10.058 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.058 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.058 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.058 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "beginning time of day to automatically balance", 2024-05-22T23:12:10.059 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-22T23:12:10.059 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.059 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.059 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.059 INFO:teuthology.orchestra.run.smithi084.stdout: "begin_weekday": { 2024-05-22T23:12:10.059 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "begin_weekday", 2024-05-22T23:12:10.059 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-05-22T23:12:10.059 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.060 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.060 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-05-22T23:12:10.060 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "0", 2024-05-22T23:12:10.060 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "6", 2024-05-22T23:12:10.060 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.060 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-22T23:12:10.060 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-22T23:12:10.060 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.061 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.061 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.061 INFO:teuthology.orchestra.run.smithi084.stdout: "crush_compat_max_iterations": { 2024-05-22T23:12:10.061 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crush_compat_max_iterations", 2024-05-22T23:12:10.061 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-05-22T23:12:10.061 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.061 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.062 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "25", 2024-05-22T23:12:10.062 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1", 2024-05-22T23:12:10.062 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "250", 2024-05-22T23:12:10.062 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.062 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-22T23:12:10.062 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.062 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.062 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.063 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.063 INFO:teuthology.orchestra.run.smithi084.stdout: "crush_compat_metrics": { 2024-05-22T23:12:10.063 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crush_compat_metrics", 2024-05-22T23:12:10.063 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.063 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.063 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.063 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "pgs,objects,bytes", 2024-05-22T23:12:10.063 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.064 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.064 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.064 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-22T23:12:10.064 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-22T23:12:10.064 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.064 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.064 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.064 INFO:teuthology.orchestra.run.smithi084.stdout: "crush_compat_step": { 2024-05-22T23:12:10.065 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crush_compat_step", 2024-05-22T23:12:10.065 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-05-22T23:12:10.065 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.065 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.065 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0.5", 2024-05-22T23:12:10.065 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "0.001", 2024-05-22T23:12:10.065 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "0.999", 2024-05-22T23:12:10.065 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.066 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "aggressiveness of optimization", 2024-05-22T23:12:10.066 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-22T23:12:10.066 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.066 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.066 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.066 INFO:teuthology.orchestra.run.smithi084.stdout: "end_time": { 2024-05-22T23:12:10.066 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "end_time", 2024-05-22T23:12:10.066 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.067 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.067 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.067 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "2359", 2024-05-22T23:12:10.067 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.067 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.067 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.067 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "ending time of day to automatically balance", 2024-05-22T23:12:10.067 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-22T23:12:10.068 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.068 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.068 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.068 INFO:teuthology.orchestra.run.smithi084.stdout: "end_weekday": { 2024-05-22T23:12:10.068 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "end_weekday", 2024-05-22T23:12:10.068 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-05-22T23:12:10.068 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.069 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.069 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-05-22T23:12:10.069 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "0", 2024-05-22T23:12:10.069 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "6", 2024-05-22T23:12:10.069 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.069 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-22T23:12:10.069 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-22T23:12:10.069 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.070 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.070 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.070 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.070 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.070 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.070 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.070 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.070 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.071 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.071 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.071 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.071 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.071 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.071 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.071 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.071 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.071 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.072 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.072 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.072 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.072 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.072 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.072 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.072 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.072 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.073 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.073 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.073 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.073 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.073 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.073 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.073 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.073 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.074 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.074 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.074 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.074 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.074 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.074 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.074 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.075 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.075 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.075 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.075 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.075 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.075 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.075 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.075 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.076 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.076 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.076 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.076 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.076 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.076 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.076 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.076 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.077 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.077 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.077 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.077 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.077 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.077 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.077 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.078 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.078 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.078 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.078 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.078 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.078 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.078 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.078 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.078 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.079 INFO:teuthology.orchestra.run.smithi084.stdout: "min_score": { 2024-05-22T23:12:10.079 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "min_score", 2024-05-22T23:12:10.079 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-05-22T23:12:10.079 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.079 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.079 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-05-22T23:12:10.079 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.079 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.080 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.080 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-22T23:12:10.080 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.080 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.080 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.080 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.080 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": { 2024-05-22T23:12:10.080 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mode", 2024-05-22T23:12:10.081 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.081 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.081 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.081 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "upmap", 2024-05-22T23:12:10.081 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.081 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.081 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.082 INFO:teuthology.orchestra.run.smithi084.stdout: "crush-compat", 2024-05-22T23:12:10.082 INFO:teuthology.orchestra.run.smithi084.stdout: "none", 2024-05-22T23:12:10.082 INFO:teuthology.orchestra.run.smithi084.stdout: "read", 2024-05-22T23:12:10.082 INFO:teuthology.orchestra.run.smithi084.stdout: "upmap", 2024-05-22T23:12:10.082 INFO:teuthology.orchestra.run.smithi084.stdout: "upmap-read" 2024-05-22T23:12:10.082 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.082 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Balancer mode", 2024-05-22T23:12:10.082 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.083 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.083 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.083 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.083 INFO:teuthology.orchestra.run.smithi084.stdout: "pool_ids": { 2024-05-22T23:12:10.083 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "pool_ids", 2024-05-22T23:12:10.083 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.083 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.083 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.084 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.084 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.084 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.084 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.084 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-22T23:12:10.084 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.084 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.084 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.085 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.085 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-05-22T23:12:10.085 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-05-22T23:12:10.085 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.085 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.085 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.085 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-05-22T23:12:10.085 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.086 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.086 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.086 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-22T23:12:10.086 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.086 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.086 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.086 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.086 INFO:teuthology.orchestra.run.smithi084.stdout: "upmap_max_deviation": { 2024-05-22T23:12:10.087 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "upmap_max_deviation", 2024-05-22T23:12:10.087 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.087 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.087 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.087 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5", 2024-05-22T23:12:10.087 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1", 2024-05-22T23:12:10.087 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.087 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.088 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-22T23:12:10.088 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-22T23:12:10.088 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.088 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.088 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.088 INFO:teuthology.orchestra.run.smithi084.stdout: "upmap_max_optimizations": { 2024-05-22T23:12:10.088 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "upmap_max_optimizations", 2024-05-22T23:12:10.088 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-05-22T23:12:10.089 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.089 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.089 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-05-22T23:12:10.089 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.089 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.089 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.089 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-22T23:12:10.089 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.090 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.090 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.090 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.090 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.090 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.090 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.090 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephadm", 2024-05-22T23:12:10.090 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.091 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.091 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.091 INFO:teuthology.orchestra.run.smithi084.stdout: "agent_down_multiplier": { 2024-05-22T23:12:10.091 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "agent_down_multiplier", 2024-05-22T23:12:10.091 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-05-22T23:12:10.091 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.091 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.091 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "3.0", 2024-05-22T23:12:10.092 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.092 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.092 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.092 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-22T23:12:10.092 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.092 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.092 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.092 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.093 INFO:teuthology.orchestra.run.smithi084.stdout: "agent_refresh_rate": { 2024-05-22T23:12:10.093 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "agent_refresh_rate", 2024-05-22T23:12:10.093 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.093 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.093 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.093 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "20", 2024-05-22T23:12:10.093 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.093 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.094 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.094 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-22T23:12:10.094 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.094 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.094 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.094 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.094 INFO:teuthology.orchestra.run.smithi084.stdout: "agent_starting_port": { 2024-05-22T23:12:10.095 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "agent_starting_port", 2024-05-22T23:12:10.095 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.095 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.095 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.095 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "4721", 2024-05-22T23:12:10.095 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.095 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.095 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.096 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-22T23:12:10.096 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.096 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.096 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.096 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.096 INFO:teuthology.orchestra.run.smithi084.stdout: "allow_ptrace": { 2024-05-22T23:12:10.096 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "allow_ptrace", 2024-05-22T23:12:10.096 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.097 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.097 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.097 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.097 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.097 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.097 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.097 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-22T23:12:10.098 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-22T23:12:10.098 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.098 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.098 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.098 INFO:teuthology.orchestra.run.smithi084.stdout: "autotune_interval": { 2024-05-22T23:12:10.098 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "autotune_interval", 2024-05-22T23:12:10.098 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.098 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.099 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.099 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-05-22T23:12:10.099 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.099 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.099 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.099 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-22T23:12:10.099 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.099 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.100 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.100 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.100 INFO:teuthology.orchestra.run.smithi084.stdout: "autotune_memory_target_ratio": { 2024-05-22T23:12:10.100 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "autotune_memory_target_ratio", 2024-05-22T23:12:10.100 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-05-22T23:12:10.100 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.100 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.100 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0.7", 2024-05-22T23:12:10.101 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.101 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.101 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.101 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-22T23:12:10.101 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.101 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.101 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.102 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.102 INFO:teuthology.orchestra.run.smithi084.stdout: "cephadm_log_destination": { 2024-05-22T23:12:10.102 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephadm_log_destination", 2024-05-22T23:12:10.102 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.102 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.102 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.102 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.102 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.103 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.103 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.103 INFO:teuthology.orchestra.run.smithi084.stdout: "file", 2024-05-22T23:12:10.103 INFO:teuthology.orchestra.run.smithi084.stdout: "file,syslog", 2024-05-22T23:12:10.103 INFO:teuthology.orchestra.run.smithi084.stdout: "syslog" 2024-05-22T23:12:10.103 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.103 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-22T23:12:10.103 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.104 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.104 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.104 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.104 INFO:teuthology.orchestra.run.smithi084.stdout: "cgroups_split": { 2024-05-22T23:12:10.104 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cgroups_split", 2024-05-22T23:12:10.104 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.104 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.104 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.105 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.105 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.105 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.105 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.105 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-22T23:12:10.105 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.105 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.105 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.106 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.106 INFO:teuthology.orchestra.run.smithi084.stdout: "config_checks_enabled": { 2024-05-22T23:12:10.106 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "config_checks_enabled", 2024-05-22T23:12:10.106 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.106 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.106 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.106 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.106 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.107 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.107 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.107 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-22T23:12:10.107 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.107 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.107 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.107 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.107 INFO:teuthology.orchestra.run.smithi084.stdout: "config_dashboard": { 2024-05-22T23:12:10.108 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "config_dashboard", 2024-05-22T23:12:10.108 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.108 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.108 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.108 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.108 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.108 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.109 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.109 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-22T23:12:10.109 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.109 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.109 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.109 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.109 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_alertmanager": { 2024-05-22T23:12:10.109 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_alertmanager", 2024-05-22T23:12:10.110 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.110 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.110 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.110 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-22T23:12:10.110 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.110 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.110 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.110 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Prometheus container image", 2024-05-22T23:12:10.111 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.111 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.111 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.111 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.111 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_base": { 2024-05-22T23:12:10.111 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_base", 2024-05-22T23:12:10.111 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.111 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.112 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.112 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-22T23:12:10.112 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.112 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.112 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.112 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Container image name, without the tag", 2024-05-22T23:12:10.112 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.112 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.112 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.113 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.113 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_elasticsearch": { 2024-05-22T23:12:10.113 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_elasticsearch", 2024-05-22T23:12:10.113 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.113 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.113 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.113 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-22T23:12:10.113 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.114 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.114 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.114 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "elasticsearch container image", 2024-05-22T23:12:10.114 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.114 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.114 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.114 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.114 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_grafana": { 2024-05-22T23:12:10.115 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_grafana", 2024-05-22T23:12:10.115 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.115 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.115 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.115 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-22T23:12:10.115 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.115 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.115 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.116 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Prometheus container image", 2024-05-22T23:12:10.116 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.116 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.116 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.116 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.116 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_haproxy": { 2024-05-22T23:12:10.116 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_haproxy", 2024-05-22T23:12:10.116 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.117 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.117 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.117 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-22T23:12:10.117 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.117 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.117 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.117 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "HAproxy container image", 2024-05-22T23:12:10.117 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.118 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.118 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.118 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.118 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_jaeger_agent": { 2024-05-22T23:12:10.118 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_jaeger_agent", 2024-05-22T23:12:10.118 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.118 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.119 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.119 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-22T23:12:10.119 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.119 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.119 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.119 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Jaeger agent container image", 2024-05-22T23:12:10.119 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.120 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.120 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.120 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.120 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_jaeger_collector": { 2024-05-22T23:12:10.120 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_jaeger_collector", 2024-05-22T23:12:10.120 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.120 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.120 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.121 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-22T23:12:10.121 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.121 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.121 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.121 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Jaeger collector container image", 2024-05-22T23:12:10.121 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.121 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.122 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.122 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.122 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_jaeger_query": { 2024-05-22T23:12:10.122 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_jaeger_query", 2024-05-22T23:12:10.122 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.122 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.122 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.122 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-22T23:12:10.123 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.123 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.123 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.123 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Jaeger query container image", 2024-05-22T23:12:10.123 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.123 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.123 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.124 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.124 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_keepalived": { 2024-05-22T23:12:10.124 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_keepalived", 2024-05-22T23:12:10.124 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.124 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.124 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.124 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-22T23:12:10.124 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.125 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.125 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.125 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Keepalived container image", 2024-05-22T23:12:10.125 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.125 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.125 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.125 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.125 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_loki": { 2024-05-22T23:12:10.126 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_loki", 2024-05-22T23:12:10.126 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.126 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.126 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.126 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-22T23:12:10.126 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.126 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.127 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.127 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Loki container image", 2024-05-22T23:12:10.127 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.127 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.127 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.127 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.127 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_node_exporter": { 2024-05-22T23:12:10.128 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_node_exporter", 2024-05-22T23:12:10.128 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.128 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.128 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.128 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-22T23:12:10.128 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.128 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.128 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.129 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Prometheus container image", 2024-05-22T23:12:10.129 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.129 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.129 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.129 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.129 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_nvmeof": { 2024-05-22T23:12:10.129 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_nvmeof", 2024-05-22T23:12:10.130 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.130 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.130 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.130 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-22T23:12:10.130 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.130 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.130 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.130 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Nvme-of container image", 2024-05-22T23:12:10.131 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.131 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.131 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.131 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.131 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_prometheus": { 2024-05-22T23:12:10.131 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_prometheus", 2024-05-22T23:12:10.131 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.131 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.132 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.132 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-22T23:12:10.132 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.132 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.132 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.132 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Prometheus container image", 2024-05-22T23:12:10.132 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.133 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.133 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.133 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.133 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_promtail": { 2024-05-22T23:12:10.133 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_promtail", 2024-05-22T23:12:10.133 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.133 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.133 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.134 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-22T23:12:10.134 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.134 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.134 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.134 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Promtail container image", 2024-05-22T23:12:10.134 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.134 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.134 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.135 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.135 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_samba": { 2024-05-22T23:12:10.135 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_samba", 2024-05-22T23:12:10.135 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.135 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.135 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.135 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-22T23:12:10.136 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.136 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.136 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.136 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Samba/SMB container image", 2024-05-22T23:12:10.136 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.136 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.136 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.136 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.137 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_snmp_gateway": { 2024-05-22T23:12:10.137 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_snmp_gateway", 2024-05-22T23:12:10.137 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.137 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.137 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.137 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-22T23:12:10.137 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.137 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.137 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.138 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "SNMP Gateway container image", 2024-05-22T23:12:10.138 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.138 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.138 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.138 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.138 INFO:teuthology.orchestra.run.smithi084.stdout: "container_init": { 2024-05-22T23:12:10.138 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_init", 2024-05-22T23:12:10.139 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.139 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.139 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.139 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.139 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.139 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.139 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.139 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Run podman/docker with `--init`", 2024-05-22T23:12:10.140 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.140 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.140 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.140 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.140 INFO:teuthology.orchestra.run.smithi084.stdout: "daemon_cache_timeout": { 2024-05-22T23:12:10.140 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "daemon_cache_timeout", 2024-05-22T23:12:10.140 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.140 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.141 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.141 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-05-22T23:12:10.141 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.141 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.141 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.141 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-22T23:12:10.141 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.142 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.142 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.142 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.142 INFO:teuthology.orchestra.run.smithi084.stdout: "default_cephadm_command_timeout": { 2024-05-22T23:12:10.142 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "default_cephadm_command_timeout", 2024-05-22T23:12:10.142 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.142 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.143 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.143 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "900", 2024-05-22T23:12:10.143 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.143 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.143 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.143 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-22T23:12:10.143 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.143 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.144 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.144 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.144 INFO:teuthology.orchestra.run.smithi084.stdout: "default_registry": { 2024-05-22T23:12:10.144 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "default_registry", 2024-05-22T23:12:10.144 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.144 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.144 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.144 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "docker.io", 2024-05-22T23:12:10.145 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.145 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.145 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.145 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-22T23:12:10.145 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.145 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.145 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.145 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.146 INFO:teuthology.orchestra.run.smithi084.stdout: "device_cache_timeout": { 2024-05-22T23:12:10.146 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "device_cache_timeout", 2024-05-22T23:12:10.146 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.146 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.146 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.146 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "1800", 2024-05-22T23:12:10.146 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.146 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.147 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.147 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "seconds to cache device inventory", 2024-05-22T23:12:10.147 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.147 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.147 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.147 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.147 INFO:teuthology.orchestra.run.smithi084.stdout: "device_enhanced_scan": { 2024-05-22T23:12:10.147 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "device_enhanced_scan", 2024-05-22T23:12:10.148 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.148 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.148 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.148 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.148 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.148 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.148 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.148 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-22T23:12:10.149 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.149 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.149 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.149 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.149 INFO:teuthology.orchestra.run.smithi084.stdout: "facts_cache_timeout": { 2024-05-22T23:12:10.149 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "facts_cache_timeout", 2024-05-22T23:12:10.149 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.149 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.150 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.150 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-05-22T23:12:10.150 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.150 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.150 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.150 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "seconds to cache host facts data", 2024-05-22T23:12:10.150 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.150 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.151 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.151 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.151 INFO:teuthology.orchestra.run.smithi084.stdout: "grafana_dashboards_path": { 2024-05-22T23:12:10.151 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "grafana_dashboards_path", 2024-05-22T23:12:10.151 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.151 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.151 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.151 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-22T23:12:10.152 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.152 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.152 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.152 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-22T23:12:10.152 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.152 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.152 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.153 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.153 INFO:teuthology.orchestra.run.smithi084.stdout: "host_check_interval": { 2024-05-22T23:12:10.153 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "host_check_interval", 2024-05-22T23:12:10.153 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.153 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.153 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.153 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-05-22T23:12:10.153 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.154 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.154 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.154 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to perform a host check", 2024-05-22T23:12:10.154 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.154 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.154 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.154 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.155 INFO:teuthology.orchestra.run.smithi084.stdout: "hw_monitoring": { 2024-05-22T23:12:10.155 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "hw_monitoring", 2024-05-22T23:12:10.155 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.155 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.155 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.155 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.155 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.155 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.156 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.156 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-22T23:12:10.156 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.156 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.156 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.156 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.156 INFO:teuthology.orchestra.run.smithi084.stdout: "inventory_list_all": { 2024-05-22T23:12:10.156 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "inventory_list_all", 2024-05-22T23:12:10.156 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.157 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.157 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.157 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.157 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.157 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.157 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.157 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-22T23:12:10.158 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.158 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.158 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.158 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.158 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.158 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.158 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.158 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.159 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.159 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.159 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.159 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.159 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.159 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.159 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.159 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.160 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.160 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.160 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.160 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.160 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.160 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.160 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.161 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.161 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.161 INFO:teuthology.orchestra.run.smithi084.stdout: "log_refresh_metadata": { 2024-05-22T23:12:10.161 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_refresh_metadata", 2024-05-22T23:12:10.161 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.161 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.161 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.161 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.161 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.162 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.162 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.162 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-22T23:12:10.162 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.162 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.162 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.162 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.163 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.163 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.163 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.163 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.163 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.163 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.163 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.163 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.164 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.164 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-22T23:12:10.164 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.164 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.164 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.164 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.164 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.165 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.165 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.165 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.165 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.165 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.165 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.165 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.165 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.166 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.166 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.166 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.166 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.166 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.166 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.166 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.166 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.167 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.167 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.167 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.167 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.167 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.167 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.167 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.167 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.168 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.168 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.168 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.168 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.168 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.168 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.168 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.168 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.169 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.169 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.169 INFO:teuthology.orchestra.run.smithi084.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-22T23:12:10.169 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-22T23:12:10.169 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.169 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.169 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.169 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.169 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.170 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.170 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.170 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-22T23:12:10.170 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.170 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.170 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.170 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.171 INFO:teuthology.orchestra.run.smithi084.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-22T23:12:10.171 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-22T23:12:10.171 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.171 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.171 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.171 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "*", 2024-05-22T23:12:10.171 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.171 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.172 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.172 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-22T23:12:10.172 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.172 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.172 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.172 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.172 INFO:teuthology.orchestra.run.smithi084.stdout: "max_count_per_host": { 2024-05-22T23:12:10.172 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_count_per_host", 2024-05-22T23:12:10.173 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.173 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.173 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.173 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-05-22T23:12:10.173 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.173 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.173 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.173 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "max number of daemons per service per host", 2024-05-22T23:12:10.174 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.174 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.174 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.174 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.174 INFO:teuthology.orchestra.run.smithi084.stdout: "max_osd_draining_count": { 2024-05-22T23:12:10.174 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_osd_draining_count", 2024-05-22T23:12:10.174 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.174 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.175 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.175 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-05-22T23:12:10.175 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.175 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.175 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.175 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-22T23:12:10.175 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.175 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.176 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.176 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.176 INFO:teuthology.orchestra.run.smithi084.stdout: "migration_current": { 2024-05-22T23:12:10.176 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "migration_current", 2024-05-22T23:12:10.176 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.176 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.176 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.176 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.177 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.177 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.177 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.177 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "internal - do not modify", 2024-05-22T23:12:10.177 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.177 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.177 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.177 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.178 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": { 2024-05-22T23:12:10.178 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mode", 2024-05-22T23:12:10.178 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.178 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.178 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.178 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "root", 2024-05-22T23:12:10.178 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.178 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.179 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.179 INFO:teuthology.orchestra.run.smithi084.stdout: "cephadm-package", 2024-05-22T23:12:10.179 INFO:teuthology.orchestra.run.smithi084.stdout: "root" 2024-05-22T23:12:10.179 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.179 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "mode for remote execution of cephadm", 2024-05-22T23:12:10.179 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.179 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.179 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.180 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.180 INFO:teuthology.orchestra.run.smithi084.stdout: "oob_default_addr": { 2024-05-22T23:12:10.180 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "oob_default_addr", 2024-05-22T23:12:10.180 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.180 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.180 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.180 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "169.254.1.1", 2024-05-22T23:12:10.180 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.181 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.181 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.181 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-22T23:12:10.181 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.181 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.181 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.181 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.181 INFO:teuthology.orchestra.run.smithi084.stdout: "prometheus_alerts_path": { 2024-05-22T23:12:10.182 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "prometheus_alerts_path", 2024-05-22T23:12:10.182 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.182 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.182 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.182 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-22T23:12:10.182 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.182 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.182 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.183 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-22T23:12:10.183 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.183 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.183 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.183 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.183 INFO:teuthology.orchestra.run.smithi084.stdout: "registry_insecure": { 2024-05-22T23:12:10.183 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "registry_insecure", 2024-05-22T23:12:10.183 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.184 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.184 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.184 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.184 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.184 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.184 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.184 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-22T23:12:10.184 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.185 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.185 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.185 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.185 INFO:teuthology.orchestra.run.smithi084.stdout: "registry_password": { 2024-05-22T23:12:10.185 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "registry_password", 2024-05-22T23:12:10.185 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.185 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.185 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.186 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.186 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.186 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.186 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.186 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-22T23:12:10.186 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.186 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.187 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.187 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.187 INFO:teuthology.orchestra.run.smithi084.stdout: "registry_url": { 2024-05-22T23:12:10.187 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "registry_url", 2024-05-22T23:12:10.187 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.187 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.187 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.187 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.188 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.188 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.188 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.188 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-22T23:12:10.188 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.188 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.188 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.188 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.189 INFO:teuthology.orchestra.run.smithi084.stdout: "registry_username": { 2024-05-22T23:12:10.189 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "registry_username", 2024-05-22T23:12:10.189 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.189 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.189 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.189 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.189 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.189 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.190 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.190 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-22T23:12:10.190 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.190 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.190 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.190 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.190 INFO:teuthology.orchestra.run.smithi084.stdout: "secure_monitoring_stack": { 2024-05-22T23:12:10.190 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "secure_monitoring_stack", 2024-05-22T23:12:10.191 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.191 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.191 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.191 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.191 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.191 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.191 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.191 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-22T23:12:10.192 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.192 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.192 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.192 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.192 INFO:teuthology.orchestra.run.smithi084.stdout: "service_discovery_port": { 2024-05-22T23:12:10.192 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "service_discovery_port", 2024-05-22T23:12:10.192 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.192 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.193 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.193 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8765", 2024-05-22T23:12:10.193 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.193 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.193 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.193 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "cephadm service discovery port", 2024-05-22T23:12:10.193 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.193 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.193 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.194 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.194 INFO:teuthology.orchestra.run.smithi084.stdout: "ssh_config_file": { 2024-05-22T23:12:10.194 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ssh_config_file", 2024-05-22T23:12:10.194 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.194 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.194 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.194 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.195 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.195 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.195 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.195 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-22T23:12:10.195 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.195 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.195 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.195 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.195 INFO:teuthology.orchestra.run.smithi084.stdout: "use_agent": { 2024-05-22T23:12:10.196 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "use_agent", 2024-05-22T23:12:10.196 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.196 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.196 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.196 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.196 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.196 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.196 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.197 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-22T23:12:10.197 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.197 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.197 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.197 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.197 INFO:teuthology.orchestra.run.smithi084.stdout: "use_repo_digest": { 2024-05-22T23:12:10.197 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "use_repo_digest", 2024-05-22T23:12:10.197 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.198 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.198 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.198 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.198 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.198 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.198 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.198 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-22T23:12:10.198 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.199 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.199 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.199 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.199 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_on_failed_host_check": { 2024-05-22T23:12:10.199 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_on_failed_host_check", 2024-05-22T23:12:10.199 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.199 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.199 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.200 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.200 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.200 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.200 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.200 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "raise a health warning if the host check fails", 2024-05-22T23:12:10.200 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.200 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.200 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.201 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.201 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_on_stray_daemons": { 2024-05-22T23:12:10.201 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_on_stray_daemons", 2024-05-22T23:12:10.201 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.201 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.201 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.201 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.201 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.202 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.202 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.202 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-22T23:12:10.202 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.202 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.202 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.202 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.202 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_on_stray_hosts": { 2024-05-22T23:12:10.203 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_on_stray_hosts", 2024-05-22T23:12:10.203 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.203 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.203 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.203 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.203 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.203 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.203 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.204 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-22T23:12:10.204 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.204 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.204 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.204 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.204 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.204 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.204 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.205 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crash", 2024-05-22T23:12:10.205 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.205 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.205 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.205 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.205 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.205 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.205 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.206 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.206 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.206 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.206 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.206 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.206 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.206 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.206 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.207 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.207 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.207 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.207 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.207 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.207 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.207 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.207 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.208 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.208 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.208 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.208 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.208 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.208 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.208 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.208 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.209 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.209 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.209 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.209 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.209 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.209 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.209 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.209 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.210 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.210 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.210 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.210 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.210 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.210 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.210 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.210 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.211 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.211 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.211 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.211 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.211 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.211 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.211 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.211 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.212 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.212 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.212 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.212 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.212 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.212 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.212 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.212 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.213 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.213 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.213 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.213 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.213 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.213 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.213 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.213 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.214 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.214 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.214 INFO:teuthology.orchestra.run.smithi084.stdout: "retain_interval": { 2024-05-22T23:12:10.214 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "retain_interval", 2024-05-22T23:12:10.214 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.214 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.214 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.214 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "31536000", 2024-05-22T23:12:10.215 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.215 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.215 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.215 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-22T23:12:10.215 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.215 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.215 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.215 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.216 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_recent_interval": { 2024-05-22T23:12:10.216 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_recent_interval", 2024-05-22T23:12:10.216 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.216 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.216 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.216 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "1209600", 2024-05-22T23:12:10.216 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.216 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.217 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.217 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-22T23:12:10.217 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.217 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.217 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.217 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.217 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.217 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.218 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.218 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "dashboard", 2024-05-22T23:12:10.218 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.218 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.218 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.218 INFO:teuthology.orchestra.run.smithi084.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-22T23:12:10.218 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-22T23:12:10.218 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.219 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.219 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.219 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-05-22T23:12:10.219 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.219 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.219 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.219 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.219 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.220 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.220 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.220 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.220 INFO:teuthology.orchestra.run.smithi084.stdout: "ALERTMANAGER_API_HOST": { 2024-05-22T23:12:10.220 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-22T23:12:10.220 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.220 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.220 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.221 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.221 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.221 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.221 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.221 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.221 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.221 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.221 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.221 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.222 INFO:teuthology.orchestra.run.smithi084.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-22T23:12:10.222 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-22T23:12:10.222 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.222 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.222 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.222 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.222 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.223 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.223 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.223 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.223 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.223 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.223 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.223 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.223 INFO:teuthology.orchestra.run.smithi084.stdout: "AUDIT_API_ENABLED": { 2024-05-22T23:12:10.224 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "AUDIT_API_ENABLED", 2024-05-22T23:12:10.224 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.224 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.224 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.224 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.224 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.224 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.224 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.225 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.225 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.225 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.225 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.225 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.225 INFO:teuthology.orchestra.run.smithi084.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-22T23:12:10.225 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-22T23:12:10.225 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.226 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.226 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.226 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.226 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.226 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.226 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.226 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.226 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.227 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.227 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.227 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.227 INFO:teuthology.orchestra.run.smithi084.stdout: "ENABLE_BROWSABLE_API": { 2024-05-22T23:12:10.227 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-22T23:12:10.227 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.227 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.227 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.228 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.228 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.228 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.228 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.228 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.228 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.228 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.228 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.229 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.229 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-22T23:12:10.229 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-22T23:12:10.229 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.229 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.229 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.229 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.229 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.230 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.230 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.230 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.230 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.230 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.230 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.230 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.231 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-22T23:12:10.231 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-22T23:12:10.231 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.231 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.231 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.231 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.231 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.231 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.232 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.232 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.232 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.232 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.232 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.232 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.232 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-22T23:12:10.232 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-22T23:12:10.233 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.233 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.233 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.233 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.233 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.233 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.233 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.233 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.234 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.234 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.234 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.234 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.234 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-22T23:12:10.234 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-22T23:12:10.234 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.235 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.235 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.235 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.235 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.235 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.235 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.235 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.235 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.235 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.236 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.236 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.236 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-22T23:12:10.236 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-22T23:12:10.236 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.236 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.236 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.236 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.237 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.237 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.237 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.237 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.237 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.237 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.237 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.237 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.238 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-22T23:12:10.238 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-22T23:12:10.238 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.238 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.238 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.238 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.238 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.238 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.239 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.239 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.239 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.239 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.239 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.239 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.239 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-22T23:12:10.239 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-22T23:12:10.240 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.240 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.240 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.240 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.240 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.240 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.240 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.241 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.241 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.241 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.241 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.241 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.241 INFO:teuthology.orchestra.run.smithi084.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-22T23:12:10.241 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-22T23:12:10.241 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.242 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.242 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.242 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.242 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.242 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.242 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.242 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.243 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.243 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.243 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.243 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.243 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_API_PASSWORD": { 2024-05-22T23:12:10.243 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-22T23:12:10.243 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.243 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.244 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.244 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "admin", 2024-05-22T23:12:10.244 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.244 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.244 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.244 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.244 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.244 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.245 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.245 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.245 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-22T23:12:10.245 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-22T23:12:10.245 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.245 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.245 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.245 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.246 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.246 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.246 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.246 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.246 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.246 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.246 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.247 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.247 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_API_URL": { 2024-05-22T23:12:10.247 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_API_URL", 2024-05-22T23:12:10.247 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.247 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.247 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.247 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.247 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.248 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.248 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.248 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.248 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.248 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.248 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.248 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.248 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_API_USERNAME": { 2024-05-22T23:12:10.248 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-22T23:12:10.249 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.249 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.249 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.249 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "admin", 2024-05-22T23:12:10.249 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.249 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.249 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.249 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.249 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.250 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.250 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.250 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.250 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-22T23:12:10.250 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-22T23:12:10.250 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.250 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.251 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.251 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.251 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.251 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.251 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.251 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.251 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.251 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.251 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.252 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.252 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-22T23:12:10.252 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-22T23:12:10.252 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.252 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.252 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.252 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.252 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.252 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.253 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.253 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.253 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.253 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.253 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.253 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.253 INFO:teuthology.orchestra.run.smithi084.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-22T23:12:10.253 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-22T23:12:10.254 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.254 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.254 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.254 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.254 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.254 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.254 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.254 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.255 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.255 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.255 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.255 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.255 INFO:teuthology.orchestra.run.smithi084.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-22T23:12:10.255 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-22T23:12:10.255 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.255 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.256 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.256 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.256 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.256 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.256 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.256 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.256 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.256 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.257 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.257 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.257 INFO:teuthology.orchestra.run.smithi084.stdout: "PROMETHEUS_API_HOST": { 2024-05-22T23:12:10.257 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-22T23:12:10.257 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.257 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.257 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.257 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.257 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.258 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.258 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.258 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.258 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.258 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.258 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.258 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.258 INFO:teuthology.orchestra.run.smithi084.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-22T23:12:10.259 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-22T23:12:10.259 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.259 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.259 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.259 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.259 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.259 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.260 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.260 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.260 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.260 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.260 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.260 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.260 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-22T23:12:10.260 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-22T23:12:10.261 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.261 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.261 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.261 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.261 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.261 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.261 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.261 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.262 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.262 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.262 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.262 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.262 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-22T23:12:10.262 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-22T23:12:10.262 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.263 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.263 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.263 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.263 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.263 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.263 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.263 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.263 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.264 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.264 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.264 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.264 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-22T23:12:10.264 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-22T23:12:10.264 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.264 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.264 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.265 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.265 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.265 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.265 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.265 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.265 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.265 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.265 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.266 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.266 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-22T23:12:10.266 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-22T23:12:10.266 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.266 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.266 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.266 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.266 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.267 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.267 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.267 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.267 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.267 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.267 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.267 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.268 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-22T23:12:10.268 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-22T23:12:10.268 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.268 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.268 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.268 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.268 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.268 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.269 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.269 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.269 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.269 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.269 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.269 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.269 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-22T23:12:10.269 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-22T23:12:10.269 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.270 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.270 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.270 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.270 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.270 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.270 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.270 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.270 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.271 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.271 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.271 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.271 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-22T23:12:10.271 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-22T23:12:10.271 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.271 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.272 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.274 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.274 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.274 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.274 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.274 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.274 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.274 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.275 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.275 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.275 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_ENABLED": { 2024-05-22T23:12:10.275 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-22T23:12:10.275 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.275 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.275 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.275 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.275 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.276 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.276 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.276 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.276 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.276 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.276 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.276 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.276 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-22T23:12:10.277 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-22T23:12:10.277 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.277 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.277 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.277 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-22T23:12:10.277 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.277 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.277 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.278 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.278 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.278 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.278 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.278 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.278 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-22T23:12:10.278 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-22T23:12:10.279 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.279 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.279 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.279 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-05-22T23:12:10.279 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.279 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.279 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.279 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.280 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.280 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.280 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.280 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.280 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-22T23:12:10.280 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-22T23:12:10.280 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.280 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.280 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.281 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8", 2024-05-22T23:12:10.281 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.281 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.281 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.281 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.281 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.281 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.281 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.282 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.282 INFO:teuthology.orchestra.run.smithi084.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-22T23:12:10.282 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-22T23:12:10.282 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.282 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.282 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.282 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "45", 2024-05-22T23:12:10.282 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.283 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.283 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.283 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.283 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.283 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.283 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.283 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.284 INFO:teuthology.orchestra.run.smithi084.stdout: "RGW_API_ACCESS_KEY": { 2024-05-22T23:12:10.284 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-22T23:12:10.284 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.284 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.284 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.284 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.284 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.284 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.285 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.285 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.285 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.285 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.285 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.285 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.285 INFO:teuthology.orchestra.run.smithi084.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-22T23:12:10.285 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-22T23:12:10.286 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.286 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.286 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.286 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "admin", 2024-05-22T23:12:10.286 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.286 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.286 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.286 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.287 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.287 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.287 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.287 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.287 INFO:teuthology.orchestra.run.smithi084.stdout: "RGW_API_SECRET_KEY": { 2024-05-22T23:12:10.287 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-22T23:12:10.287 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.287 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.287 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.288 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.288 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.288 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.288 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.288 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.288 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.288 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.288 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.289 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.289 INFO:teuthology.orchestra.run.smithi084.stdout: "RGW_API_SSL_VERIFY": { 2024-05-22T23:12:10.289 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-22T23:12:10.289 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.289 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.289 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.289 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.290 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.290 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.290 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.290 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.290 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.290 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.290 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.291 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.291 INFO:teuthology.orchestra.run.smithi084.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-22T23:12:10.291 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-22T23:12:10.291 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.291 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.291 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.291 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-05-22T23:12:10.291 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.291 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.292 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.292 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.292 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.292 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.292 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.292 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.292 INFO:teuthology.orchestra.run.smithi084.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-22T23:12:10.292 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-22T23:12:10.293 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.293 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.293 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.293 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-05-22T23:12:10.293 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.293 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.293 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.293 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.294 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.294 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.294 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.294 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.294 INFO:teuthology.orchestra.run.smithi084.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-22T23:12:10.294 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-22T23:12:10.294 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.294 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.294 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.295 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5", 2024-05-22T23:12:10.295 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.295 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.295 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.295 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.295 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.295 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.295 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.296 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.296 INFO:teuthology.orchestra.run.smithi084.stdout: "cross_origin_url": { 2024-05-22T23:12:10.296 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cross_origin_url", 2024-05-22T23:12:10.296 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.296 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.296 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.296 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.296 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.296 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.297 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.297 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.297 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.297 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.297 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.297 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.297 INFO:teuthology.orchestra.run.smithi084.stdout: "crt_file": { 2024-05-22T23:12:10.297 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crt_file", 2024-05-22T23:12:10.297 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.298 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.298 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.298 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.298 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.298 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.298 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.298 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.298 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.299 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.299 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.299 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.299 INFO:teuthology.orchestra.run.smithi084.stdout: "debug": { 2024-05-22T23:12:10.299 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "debug", 2024-05-22T23:12:10.299 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.299 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.299 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.299 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.300 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.300 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.300 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.300 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Enable/disable debug options", 2024-05-22T23:12:10.300 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.300 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.300 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.300 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.301 INFO:teuthology.orchestra.run.smithi084.stdout: "jwt_token_ttl": { 2024-05-22T23:12:10.301 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "jwt_token_ttl", 2024-05-22T23:12:10.301 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.301 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.301 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.301 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "28800", 2024-05-22T23:12:10.301 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.301 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.302 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.302 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.302 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.302 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.302 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.302 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.302 INFO:teuthology.orchestra.run.smithi084.stdout: "key_file": { 2024-05-22T23:12:10.302 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "key_file", 2024-05-22T23:12:10.302 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.303 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.303 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.303 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.303 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.303 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.303 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.303 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.303 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.304 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.304 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.304 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.304 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.304 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.304 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.304 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.304 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.304 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.305 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.305 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.305 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.305 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.305 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.305 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.305 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.305 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.306 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.306 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.306 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.306 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.306 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.306 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.306 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.306 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.307 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.307 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.307 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.307 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.307 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.307 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.307 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.307 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.307 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.308 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.308 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.308 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.308 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.308 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.308 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.308 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.308 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.308 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.309 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.309 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.309 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.309 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.309 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.309 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.309 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.310 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.310 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.310 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.310 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.310 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.310 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.310 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.310 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.310 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.311 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.311 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.311 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.311 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.311 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.311 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.311 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.311 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.312 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.312 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.312 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.312 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.312 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.312 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.312 INFO:teuthology.orchestra.run.smithi084.stdout: "motd": { 2024-05-22T23:12:10.313 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "motd", 2024-05-22T23:12:10.313 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.313 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.313 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.313 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.313 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.313 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.313 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.314 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "The message of the day", 2024-05-22T23:12:10.314 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.314 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.314 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.314 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.314 INFO:teuthology.orchestra.run.smithi084.stdout: "redirect_resolve_ip_addr": { 2024-05-22T23:12:10.314 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "redirect_resolve_ip_addr", 2024-05-22T23:12:10.314 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.315 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.315 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.315 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.315 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.315 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.315 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.315 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.315 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.316 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.316 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.316 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.316 INFO:teuthology.orchestra.run.smithi084.stdout: "server_addr": { 2024-05-22T23:12:10.316 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_addr", 2024-05-22T23:12:10.316 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.316 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.317 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.317 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "::", 2024-05-22T23:12:10.317 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.317 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.317 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.317 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.317 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.317 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.318 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.318 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.318 INFO:teuthology.orchestra.run.smithi084.stdout: "server_port": { 2024-05-22T23:12:10.318 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_port", 2024-05-22T23:12:10.318 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.318 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.318 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.318 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8080", 2024-05-22T23:12:10.319 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.319 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.319 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.319 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.319 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.319 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.319 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.319 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.320 INFO:teuthology.orchestra.run.smithi084.stdout: "ssl": { 2024-05-22T23:12:10.320 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ssl", 2024-05-22T23:12:10.320 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.320 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.320 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.320 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.320 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.320 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.321 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.321 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.321 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.321 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.321 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.321 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.321 INFO:teuthology.orchestra.run.smithi084.stdout: "ssl_server_port": { 2024-05-22T23:12:10.321 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ssl_server_port", 2024-05-22T23:12:10.322 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.322 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.322 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.322 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8443", 2024-05-22T23:12:10.322 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.322 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.322 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.323 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.323 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.323 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.323 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.323 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.323 INFO:teuthology.orchestra.run.smithi084.stdout: "standby_behaviour": { 2024-05-22T23:12:10.323 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "standby_behaviour", 2024-05-22T23:12:10.323 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.324 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.324 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.324 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "redirect", 2024-05-22T23:12:10.324 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.324 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.324 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.324 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.324 INFO:teuthology.orchestra.run.smithi084.stdout: "redirect" 2024-05-22T23:12:10.325 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.325 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.325 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.325 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.325 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.325 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.325 INFO:teuthology.orchestra.run.smithi084.stdout: "standby_error_status_code": { 2024-05-22T23:12:10.325 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "standby_error_status_code", 2024-05-22T23:12:10.326 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.326 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.326 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.326 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "500", 2024-05-22T23:12:10.326 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "400", 2024-05-22T23:12:10.326 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "599", 2024-05-22T23:12:10.326 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.326 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.327 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.327 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.327 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.327 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.327 INFO:teuthology.orchestra.run.smithi084.stdout: "url_prefix": { 2024-05-22T23:12:10.327 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "url_prefix", 2024-05-22T23:12:10.327 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.327 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.328 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.328 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.328 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.328 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.328 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.328 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.328 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.328 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.329 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.329 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.329 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.329 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.329 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.329 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "devicehealth", 2024-05-22T23:12:10.329 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.329 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.330 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.330 INFO:teuthology.orchestra.run.smithi084.stdout: "enable_monitoring": { 2024-05-22T23:12:10.330 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "enable_monitoring", 2024-05-22T23:12:10.330 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.330 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.330 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.330 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.330 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.331 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.331 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.331 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "monitor device health metrics", 2024-05-22T23:12:10.331 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.331 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.331 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.331 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.331 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.331 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.332 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.332 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.332 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.332 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.332 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.332 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.332 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.332 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.333 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.333 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.333 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.333 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.333 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.333 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.333 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.333 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.333 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.334 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.334 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.334 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.334 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.334 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.334 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.334 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.335 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.335 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.335 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.335 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.335 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.335 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.335 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.335 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.335 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.336 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.336 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.336 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.336 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.336 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.336 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.336 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.336 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.337 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.337 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.337 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.337 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.337 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.337 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.337 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.337 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.338 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.338 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.338 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.338 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.338 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.338 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.338 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.338 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.339 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.339 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.339 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.339 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.339 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.339 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.339 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.340 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.340 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.340 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.340 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.340 INFO:teuthology.orchestra.run.smithi084.stdout: "mark_out_threshold": { 2024-05-22T23:12:10.340 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mark_out_threshold", 2024-05-22T23:12:10.340 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.340 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.341 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.341 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "2419200", 2024-05-22T23:12:10.341 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.341 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.341 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.341 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-22T23:12:10.341 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.342 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.342 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.342 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.342 INFO:teuthology.orchestra.run.smithi084.stdout: "pool_name": { 2024-05-22T23:12:10.342 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "pool_name", 2024-05-22T23:12:10.342 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.342 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.342 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.342 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "device_health_metrics", 2024-05-22T23:12:10.343 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.343 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.343 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.343 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-22T23:12:10.343 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.343 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.343 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.344 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.344 INFO:teuthology.orchestra.run.smithi084.stdout: "retention_period": { 2024-05-22T23:12:10.344 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "retention_period", 2024-05-22T23:12:10.344 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.344 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.344 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.344 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "15552000", 2024-05-22T23:12:10.344 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.345 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.345 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.345 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how long to retain device health metrics", 2024-05-22T23:12:10.345 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.345 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.345 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.345 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.346 INFO:teuthology.orchestra.run.smithi084.stdout: "scrape_frequency": { 2024-05-22T23:12:10.346 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "scrape_frequency", 2024-05-22T23:12:10.346 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.346 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.346 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.346 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "86400", 2024-05-22T23:12:10.346 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.346 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.347 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.347 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-22T23:12:10.347 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.347 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.347 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.347 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.347 INFO:teuthology.orchestra.run.smithi084.stdout: "self_heal": { 2024-05-22T23:12:10.348 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "self_heal", 2024-05-22T23:12:10.348 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.348 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.348 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.348 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.348 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.348 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.349 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.349 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-22T23:12:10.349 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.349 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.349 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.349 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.349 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-05-22T23:12:10.349 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-05-22T23:12:10.350 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.350 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.350 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.350 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-05-22T23:12:10.350 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.350 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.350 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.350 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to wake up and check device health", 2024-05-22T23:12:10.351 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.351 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.351 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.351 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.351 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_threshold": { 2024-05-22T23:12:10.351 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_threshold", 2024-05-22T23:12:10.351 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.351 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.352 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.352 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "7257600", 2024-05-22T23:12:10.352 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.352 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.352 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.352 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-22T23:12:10.352 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.352 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.352 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.353 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.353 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.353 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.353 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.353 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "diskprediction_local", 2024-05-22T23:12:10.353 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.353 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.353 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.354 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.354 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.354 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.354 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.354 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.354 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.354 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.355 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.355 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.355 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.355 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.355 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.355 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.355 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.355 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.355 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.356 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.356 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.356 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.356 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.356 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.356 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.356 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.357 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.357 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.357 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.357 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.357 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.357 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.357 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.357 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.358 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.358 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.358 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.358 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.358 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.358 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.358 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.358 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.359 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.359 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.359 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.359 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.359 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.359 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.359 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.359 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.360 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.360 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.360 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.360 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.360 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.360 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.360 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.360 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.361 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.361 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.361 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.361 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.361 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.361 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.361 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.361 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.362 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.362 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.362 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.362 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.362 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.362 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.362 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.363 INFO:teuthology.orchestra.run.smithi084.stdout: "predict_interval": { 2024-05-22T23:12:10.363 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "predict_interval", 2024-05-22T23:12:10.363 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.363 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.363 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.363 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "86400", 2024-05-22T23:12:10.363 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.363 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.364 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.364 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.364 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.364 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.364 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.364 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.364 INFO:teuthology.orchestra.run.smithi084.stdout: "predictor_model": { 2024-05-22T23:12:10.364 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "predictor_model", 2024-05-22T23:12:10.365 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.365 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.365 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.365 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "prophetstor", 2024-05-22T23:12:10.365 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.365 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.365 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.365 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.366 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.366 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.366 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.366 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.366 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-05-22T23:12:10.366 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-05-22T23:12:10.366 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.366 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.367 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.367 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-05-22T23:12:10.367 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.367 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.367 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.367 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.367 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.367 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.368 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.368 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.368 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.368 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.368 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.368 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "influx", 2024-05-22T23:12:10.368 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": false, 2024-05-22T23:12:10.368 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "influxdb python module not found", 2024-05-22T23:12:10.369 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.369 INFO:teuthology.orchestra.run.smithi084.stdout: "batch_size": { 2024-05-22T23:12:10.369 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "batch_size", 2024-05-22T23:12:10.369 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.369 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.369 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.369 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5000", 2024-05-22T23:12:10.369 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.370 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.370 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.370 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-22T23:12:10.370 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.370 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.370 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.370 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.370 INFO:teuthology.orchestra.run.smithi084.stdout: "database": { 2024-05-22T23:12:10.371 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "database", 2024-05-22T23:12:10.371 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.371 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.371 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.371 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "ceph", 2024-05-22T23:12:10.371 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.371 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.371 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.372 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-22T23:12:10.372 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.372 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.372 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.372 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.372 INFO:teuthology.orchestra.run.smithi084.stdout: "hostname": { 2024-05-22T23:12:10.372 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "hostname", 2024-05-22T23:12:10.372 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.373 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.373 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.373 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.373 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.373 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.373 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.373 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "InfluxDB server hostname", 2024-05-22T23:12:10.373 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.374 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.374 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.374 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.374 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-05-22T23:12:10.374 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-05-22T23:12:10.374 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.374 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.375 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.375 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "30", 2024-05-22T23:12:10.375 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "5", 2024-05-22T23:12:10.375 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.375 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.375 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-22T23:12:10.375 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.375 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.376 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.376 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.376 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.376 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.376 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.376 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.376 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.376 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.376 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.377 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.377 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.377 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.377 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.377 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.377 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.377 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.377 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.378 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.378 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.378 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.378 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.378 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.378 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.378 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.379 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.379 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.379 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.379 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.379 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.379 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.379 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.379 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.380 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.380 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.380 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.380 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.380 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.380 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.380 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.380 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.381 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.381 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.381 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.381 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.381 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.381 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.381 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.381 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.382 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.382 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.382 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.382 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.382 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.382 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.382 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.382 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.383 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.383 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.383 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.383 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.383 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.383 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.383 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.383 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.384 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.384 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.384 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.384 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.384 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.384 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.384 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.384 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.385 INFO:teuthology.orchestra.run.smithi084.stdout: "password": { 2024-05-22T23:12:10.385 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "password", 2024-05-22T23:12:10.385 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.385 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.385 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.385 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.385 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.385 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.386 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.386 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "password of InfluxDB server user", 2024-05-22T23:12:10.386 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.386 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.386 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.386 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.386 INFO:teuthology.orchestra.run.smithi084.stdout: "port": { 2024-05-22T23:12:10.387 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "port", 2024-05-22T23:12:10.387 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.387 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.387 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.387 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8086", 2024-05-22T23:12:10.387 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.387 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.387 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.388 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "InfluxDB server port", 2024-05-22T23:12:10.388 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.388 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.388 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.388 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.388 INFO:teuthology.orchestra.run.smithi084.stdout: "ssl": { 2024-05-22T23:12:10.388 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ssl", 2024-05-22T23:12:10.388 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.389 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.389 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.389 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "false", 2024-05-22T23:12:10.389 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.389 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.389 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.389 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-22T23:12:10.390 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.390 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.390 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.390 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.390 INFO:teuthology.orchestra.run.smithi084.stdout: "threads": { 2024-05-22T23:12:10.390 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "threads", 2024-05-22T23:12:10.390 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.390 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.391 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.391 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5", 2024-05-22T23:12:10.391 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1", 2024-05-22T23:12:10.391 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "32", 2024-05-22T23:12:10.391 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.391 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-22T23:12:10.391 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.391 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.392 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.392 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.392 INFO:teuthology.orchestra.run.smithi084.stdout: "username": { 2024-05-22T23:12:10.392 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "username", 2024-05-22T23:12:10.392 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.392 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.392 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.392 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.393 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.393 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.393 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.393 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "username of InfluxDB server user", 2024-05-22T23:12:10.393 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.393 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.393 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.393 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.394 INFO:teuthology.orchestra.run.smithi084.stdout: "verify_ssl": { 2024-05-22T23:12:10.394 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "verify_ssl", 2024-05-22T23:12:10.394 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.394 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.394 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.394 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "true", 2024-05-22T23:12:10.394 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.395 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.395 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.395 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-22T23:12:10.395 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.395 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.395 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.395 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.395 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.396 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.396 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.396 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "insights", 2024-05-22T23:12:10.396 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.396 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.396 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.396 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.396 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.396 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.397 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.397 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.397 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.397 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.397 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.397 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.397 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.397 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.398 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.398 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.398 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.398 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.398 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.398 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.398 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.399 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.399 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.399 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.399 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.399 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.399 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.399 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.399 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.400 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.400 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.400 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.400 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.400 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.400 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.400 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.401 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.401 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.401 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.401 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.401 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.401 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.401 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.401 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.402 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.402 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.402 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.402 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.402 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.402 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.402 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.403 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.403 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.403 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.403 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.403 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.403 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.403 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.403 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.403 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.404 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.404 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.404 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.404 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.404 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.404 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.404 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.404 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.404 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.405 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.405 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.405 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.405 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.405 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.405 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.405 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.405 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "iostat", 2024-05-22T23:12:10.405 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.406 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.406 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.406 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.406 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.406 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.406 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.406 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.406 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.407 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.407 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.407 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.407 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.407 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.407 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.407 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.407 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.407 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.408 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.408 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.408 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.408 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.408 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.408 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.408 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.408 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.408 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.409 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.409 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.409 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.409 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.409 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.409 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.409 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.409 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.410 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.410 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.410 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.410 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.410 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.410 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.410 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.411 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.411 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.411 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.411 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.411 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.411 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.411 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.411 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.412 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.412 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.412 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.412 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.412 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.412 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.412 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.412 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.413 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.413 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.413 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.413 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.413 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.413 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.413 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.413 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.414 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.414 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.414 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.414 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.414 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.414 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.414 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.415 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.415 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.415 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.415 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "k8sevents", 2024-05-22T23:12:10.415 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.415 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.415 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.415 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_event_retention_days": { 2024-05-22T23:12:10.416 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ceph_event_retention_days", 2024-05-22T23:12:10.416 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.416 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.416 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.416 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "7", 2024-05-22T23:12:10.416 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.416 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.416 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.417 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-22T23:12:10.417 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.417 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.417 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.417 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.417 INFO:teuthology.orchestra.run.smithi084.stdout: "config_check_secs": { 2024-05-22T23:12:10.417 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "config_check_secs", 2024-05-22T23:12:10.417 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.418 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.418 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.418 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-05-22T23:12:10.418 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "10", 2024-05-22T23:12:10.418 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.418 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.418 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-22T23:12:10.418 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.419 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.419 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.419 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.419 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.419 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.419 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.419 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.419 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.420 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.420 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.420 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.420 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.420 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.420 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.420 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.421 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.421 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.421 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.421 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.421 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.421 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.421 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.421 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.422 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.422 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.422 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.422 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.422 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.422 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.422 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.423 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.423 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.423 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.423 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.423 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.423 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.423 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.423 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.424 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.424 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.424 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.424 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.424 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.424 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.424 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.424 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.425 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.425 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.425 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.425 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.425 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.425 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.425 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.425 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.426 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.426 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.426 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.426 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.426 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.426 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.426 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.427 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.427 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.427 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.427 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.427 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.427 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.427 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.428 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.428 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.428 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.428 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.428 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.428 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.428 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.428 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.429 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "localpool", 2024-05-22T23:12:10.429 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.429 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.429 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.429 INFO:teuthology.orchestra.run.smithi084.stdout: "failure_domain": { 2024-05-22T23:12:10.429 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "failure_domain", 2024-05-22T23:12:10.429 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.429 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.430 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.430 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "host", 2024-05-22T23:12:10.430 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.430 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.430 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.430 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "failure domain for any created local pool", 2024-05-22T23:12:10.430 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-22T23:12:10.430 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.431 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.431 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.431 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.431 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.431 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.431 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.431 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.431 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.432 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.432 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.432 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.432 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.432 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.432 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.432 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.432 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.433 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.433 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.433 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.433 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.433 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.433 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.433 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.433 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.434 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.434 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.434 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.434 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.434 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.434 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.434 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.435 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.435 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.435 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.435 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.435 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.435 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.435 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.435 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.436 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.436 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.436 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.436 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.436 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.436 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.436 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.436 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.437 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.437 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.437 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.437 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.437 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.437 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.437 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.437 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.438 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.438 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.438 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.438 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.438 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.438 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.438 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.438 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.439 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.439 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.439 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.439 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.439 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.439 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.439 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.440 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.440 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.440 INFO:teuthology.orchestra.run.smithi084.stdout: "min_size": { 2024-05-22T23:12:10.440 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "min_size", 2024-05-22T23:12:10.440 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.440 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.440 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.440 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.441 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.441 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.441 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.441 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "default min_size for any created local pool", 2024-05-22T23:12:10.441 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-22T23:12:10.441 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.441 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.441 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.442 INFO:teuthology.orchestra.run.smithi084.stdout: "num_rep": { 2024-05-22T23:12:10.442 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "num_rep", 2024-05-22T23:12:10.442 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.442 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.442 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.442 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "3", 2024-05-22T23:12:10.442 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.442 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.443 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.443 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "default replica count for any created local pool", 2024-05-22T23:12:10.443 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.443 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.443 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.443 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.443 INFO:teuthology.orchestra.run.smithi084.stdout: "pg_num": { 2024-05-22T23:12:10.443 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "pg_num", 2024-05-22T23:12:10.444 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.444 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.444 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.444 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "128", 2024-05-22T23:12:10.444 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.444 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.445 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.445 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "default pg_num for any created local pool", 2024-05-22T23:12:10.445 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.445 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.445 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.445 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.445 INFO:teuthology.orchestra.run.smithi084.stdout: "prefix": { 2024-05-22T23:12:10.445 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "prefix", 2024-05-22T23:12:10.446 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.446 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.446 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.446 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.446 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.446 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.446 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.447 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "name prefix for any created local pool", 2024-05-22T23:12:10.447 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.447 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.447 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.447 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.447 INFO:teuthology.orchestra.run.smithi084.stdout: "subtree": { 2024-05-22T23:12:10.447 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "subtree", 2024-05-22T23:12:10.447 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.448 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.448 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.448 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "rack", 2024-05-22T23:12:10.448 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.448 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.448 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.448 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-22T23:12:10.448 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-22T23:12:10.449 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.449 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.449 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.449 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.449 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.449 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.449 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mds_autoscaler", 2024-05-22T23:12:10.449 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.450 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.450 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.450 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.450 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.450 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.450 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.450 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.450 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.451 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.451 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.451 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.451 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.451 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.451 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.451 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.451 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.452 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.452 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.452 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.452 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.452 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.452 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.452 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.453 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.453 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.453 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.453 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.453 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.453 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.453 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.453 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.453 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.454 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.454 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.454 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.454 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.454 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.454 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.454 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.455 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.455 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.455 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.455 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.455 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.455 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.455 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.455 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.456 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.456 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.456 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.456 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.456 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.456 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.456 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.456 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.456 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.457 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.457 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.457 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.457 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.457 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.457 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.457 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.458 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.458 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.458 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.458 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.458 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.458 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.458 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.458 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.459 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.459 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.459 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.459 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.459 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mirroring", 2024-05-22T23:12:10.459 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.459 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.459 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.460 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.460 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.460 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.460 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.460 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.460 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.460 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.460 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.461 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.461 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.461 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.461 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.461 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.461 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.461 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.461 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.462 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.462 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.462 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.462 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.462 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.462 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.462 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.463 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.463 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.463 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.463 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.463 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.463 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.463 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.463 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.464 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.464 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.464 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.464 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.464 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.464 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.464 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.465 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.465 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.465 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.465 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.465 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.465 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.465 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.465 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.465 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.466 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.466 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.466 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.466 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.466 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.466 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.466 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.467 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.467 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.467 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.467 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.467 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.467 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.467 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.467 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.468 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.468 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.468 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.468 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.468 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.468 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.468 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.468 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.469 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.469 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.469 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.469 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "nfs", 2024-05-22T23:12:10.469 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.469 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.469 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.469 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.470 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.470 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.470 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.470 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.470 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.470 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.470 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.470 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.471 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.471 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.471 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.471 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.471 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.471 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.471 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.471 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.472 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.472 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.472 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.472 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.472 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.472 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.472 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.472 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.473 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.473 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.473 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.473 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.473 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.473 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.473 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.473 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.474 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.474 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.474 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.474 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.474 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.477 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.477 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.477 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.478 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.478 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.478 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.478 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.478 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.478 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.478 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.478 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.479 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.479 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.479 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.479 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.479 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.479 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.479 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.479 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.480 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.480 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.480 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.480 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.480 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.480 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.480 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.480 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.481 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.481 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.481 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.481 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.481 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.481 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.481 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.481 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.482 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "orchestrator", 2024-05-22T23:12:10.482 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.482 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.482 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.482 INFO:teuthology.orchestra.run.smithi084.stdout: "fail_fs": { 2024-05-22T23:12:10.482 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "fail_fs", 2024-05-22T23:12:10.482 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.482 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.483 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.483 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.483 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.483 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.483 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.483 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-22T23:12:10.483 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.483 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.484 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.484 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.484 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.484 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.484 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.484 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.484 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.484 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.485 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.485 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.485 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.485 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.485 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.485 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.485 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.485 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.486 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.486 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.486 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.486 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.486 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.486 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.486 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.486 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.487 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.487 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.487 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.487 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.487 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.487 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.487 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.487 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.488 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.488 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.488 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.488 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.488 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.488 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.488 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.488 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.489 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.489 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.489 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.489 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.489 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.489 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.489 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.490 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.490 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.490 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.490 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.490 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.490 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.490 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.490 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.491 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.491 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.491 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.491 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.491 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.491 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.491 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.491 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.492 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.492 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.492 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.492 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.492 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.492 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.492 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.492 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.493 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.493 INFO:teuthology.orchestra.run.smithi084.stdout: "orchestrator": { 2024-05-22T23:12:10.493 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "orchestrator", 2024-05-22T23:12:10.493 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.493 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.493 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.493 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.493 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.493 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.494 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.494 INFO:teuthology.orchestra.run.smithi084.stdout: "cephadm", 2024-05-22T23:12:10.494 INFO:teuthology.orchestra.run.smithi084.stdout: "rook", 2024-05-22T23:12:10.494 INFO:teuthology.orchestra.run.smithi084.stdout: "test_orchestrator" 2024-05-22T23:12:10.494 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.494 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Orchestrator backend", 2024-05-22T23:12:10.494 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.494 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.495 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.495 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.495 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.495 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.495 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.495 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "osd_perf_query", 2024-05-22T23:12:10.495 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.495 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.496 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.496 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.496 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.496 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.496 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.496 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.496 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.496 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.497 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.497 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.497 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.497 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.497 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.497 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.497 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.498 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.498 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.498 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.498 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.498 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.498 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.498 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.499 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.499 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.499 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.499 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.499 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.499 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.499 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.499 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.499 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.500 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.500 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.500 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.500 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.500 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.500 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.500 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.500 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.501 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.501 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.501 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.501 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.501 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.501 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.501 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.501 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.502 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.502 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.502 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.502 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.502 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.502 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.502 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.503 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.503 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.503 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.503 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.503 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.503 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.503 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.503 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.504 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.504 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.504 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.504 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.504 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.504 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.504 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.504 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.505 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.505 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.505 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.505 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.505 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "osd_support", 2024-05-22T23:12:10.505 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.505 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.505 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.506 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.506 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.506 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.506 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.506 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.506 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.506 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.506 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.507 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.507 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.507 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.507 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.507 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.507 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.507 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.507 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.507 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.508 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.508 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.508 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.508 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.508 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.508 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.508 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.509 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.509 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.509 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.509 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.509 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.509 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.509 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.509 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.510 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.510 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.510 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.510 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.510 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.510 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.510 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.510 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.511 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.511 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.511 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.511 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.511 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.511 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.511 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.511 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.512 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.512 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.512 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.512 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.512 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.512 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.512 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.512 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.513 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.513 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.513 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.513 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.513 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.513 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.513 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.513 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.514 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.514 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.514 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.514 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.514 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.514 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.514 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.514 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.515 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.515 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "pg_autoscaler", 2024-05-22T23:12:10.515 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.515 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.515 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.515 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.515 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.516 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.516 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.516 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.516 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.516 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.516 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.516 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.516 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.517 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.517 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.517 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.517 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.517 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.517 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.517 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.517 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.518 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.518 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.518 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.518 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.518 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.518 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.518 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.518 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.519 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.519 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.519 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.519 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.519 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.519 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.519 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.519 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.519 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.520 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.520 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.520 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.520 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.520 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.520 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.520 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.520 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.521 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.521 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.521 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.521 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.521 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.521 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.521 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.521 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.522 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.522 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.522 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.522 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.522 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.522 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.522 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.523 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.523 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.523 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.523 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.523 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.523 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.523 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.523 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.524 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.524 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.524 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.524 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.524 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-05-22T23:12:10.524 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-05-22T23:12:10.524 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.524 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.525 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.525 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-05-22T23:12:10.525 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.525 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.525 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.525 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.525 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.525 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.526 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.526 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.526 INFO:teuthology.orchestra.run.smithi084.stdout: "threshold": { 2024-05-22T23:12:10.526 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "threshold", 2024-05-22T23:12:10.526 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-05-22T23:12:10.526 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.526 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.527 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "3.0", 2024-05-22T23:12:10.527 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1.0", 2024-05-22T23:12:10.527 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.527 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.527 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "scaling threshold", 2024-05-22T23:12:10.527 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-22T23:12:10.527 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.527 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.528 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.528 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.528 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.528 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.528 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "progress", 2024-05-22T23:12:10.528 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.528 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.528 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.528 INFO:teuthology.orchestra.run.smithi084.stdout: "allow_pg_recovery_event": { 2024-05-22T23:12:10.529 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "allow_pg_recovery_event", 2024-05-22T23:12:10.529 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.529 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.529 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.529 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.529 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.529 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.529 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.530 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-22T23:12:10.530 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.530 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.530 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.530 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.530 INFO:teuthology.orchestra.run.smithi084.stdout: "enabled": { 2024-05-22T23:12:10.530 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "enabled", 2024-05-22T23:12:10.531 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.531 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.531 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.531 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.531 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.531 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.531 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.531 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.532 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.532 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.532 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.532 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.532 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.532 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.532 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.532 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.533 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.533 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.533 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.533 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.533 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.533 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.533 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.533 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.534 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.534 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.534 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.534 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.534 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.534 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.534 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.534 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.535 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.535 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.535 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.535 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.535 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.535 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.535 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.536 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.536 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.536 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.536 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.536 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.536 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.536 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.536 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.537 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.537 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.537 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.537 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.537 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.537 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.537 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.537 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.538 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.538 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.538 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.538 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.538 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.538 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.538 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.538 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.538 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.539 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.539 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.539 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.539 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.539 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.539 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.539 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.539 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.540 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.540 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.540 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.540 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.540 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.540 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.540 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.541 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.541 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.541 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.541 INFO:teuthology.orchestra.run.smithi084.stdout: "max_completed_events": { 2024-05-22T23:12:10.541 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_completed_events", 2024-05-22T23:12:10.541 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.541 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.541 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.542 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "50", 2024-05-22T23:12:10.542 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.542 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.542 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.542 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "number of past completed events to remember", 2024-05-22T23:12:10.542 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.542 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.542 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.543 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.543 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-05-22T23:12:10.543 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-05-22T23:12:10.543 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.543 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.543 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.543 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5", 2024-05-22T23:12:10.543 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.544 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.544 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.544 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how long the module is going to sleep", 2024-05-22T23:12:10.544 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.544 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.544 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.544 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.544 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.545 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.545 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.545 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "prometheus", 2024-05-22T23:12:10.545 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.545 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.545 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.545 INFO:teuthology.orchestra.run.smithi084.stdout: "cache": { 2024-05-22T23:12:10.545 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cache", 2024-05-22T23:12:10.546 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.546 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.546 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.546 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.546 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.546 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.546 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.546 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.547 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.547 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.547 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.547 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.547 INFO:teuthology.orchestra.run.smithi084.stdout: "exclude_perf_counters": { 2024-05-22T23:12:10.547 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "exclude_perf_counters", 2024-05-22T23:12:10.547 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.547 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.548 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.548 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.548 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.548 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.548 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.548 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-22T23:12:10.548 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-22T23:12:10.549 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.549 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.549 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.549 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.549 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.549 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.549 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.549 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.550 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.550 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.550 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.550 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.550 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.550 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.550 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.550 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.551 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.551 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.551 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.551 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.551 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.551 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.551 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.551 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.552 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.552 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.552 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.552 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.552 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.552 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.552 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.552 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.553 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.553 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.553 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.553 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.553 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.553 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.553 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.553 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.554 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.554 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.554 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.554 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.554 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.554 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.554 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.554 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.555 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.555 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.555 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.555 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.555 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.555 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.555 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.555 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.556 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.556 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.556 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.556 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.556 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.556 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.556 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.556 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.557 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.557 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.557 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.557 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.557 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.557 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.557 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.557 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.558 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.558 INFO:teuthology.orchestra.run.smithi084.stdout: "rbd_stats_pools": { 2024-05-22T23:12:10.558 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rbd_stats_pools", 2024-05-22T23:12:10.558 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.558 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.558 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.558 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.559 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.559 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.559 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.559 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.559 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.559 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.559 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.559 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.560 INFO:teuthology.orchestra.run.smithi084.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-22T23:12:10.560 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-22T23:12:10.560 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.560 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.560 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.560 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "300", 2024-05-22T23:12:10.560 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.560 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.561 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.561 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.561 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.561 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.561 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.561 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.561 INFO:teuthology.orchestra.run.smithi084.stdout: "scrape_interval": { 2024-05-22T23:12:10.561 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "scrape_interval", 2024-05-22T23:12:10.562 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-05-22T23:12:10.562 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.562 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.562 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "15.0", 2024-05-22T23:12:10.562 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.562 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.562 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.563 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.563 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.563 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.563 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.563 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.563 INFO:teuthology.orchestra.run.smithi084.stdout: "server_addr": { 2024-05-22T23:12:10.563 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_addr", 2024-05-22T23:12:10.563 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.564 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.564 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.564 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "::", 2024-05-22T23:12:10.564 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.564 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.564 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.564 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-22T23:12:10.564 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.565 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.565 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.565 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.565 INFO:teuthology.orchestra.run.smithi084.stdout: "server_port": { 2024-05-22T23:12:10.565 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_port", 2024-05-22T23:12:10.565 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.565 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.565 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.565 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "9283", 2024-05-22T23:12:10.566 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.566 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.566 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.566 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-22T23:12:10.566 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.566 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.566 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.567 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.567 INFO:teuthology.orchestra.run.smithi084.stdout: "stale_cache_strategy": { 2024-05-22T23:12:10.567 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "stale_cache_strategy", 2024-05-22T23:12:10.567 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.567 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.567 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.567 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "log", 2024-05-22T23:12:10.567 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.568 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.568 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.568 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.568 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.568 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.568 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.568 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.568 INFO:teuthology.orchestra.run.smithi084.stdout: "standby_behaviour": { 2024-05-22T23:12:10.569 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "standby_behaviour", 2024-05-22T23:12:10.569 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.569 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.569 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.569 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "default", 2024-05-22T23:12:10.569 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.569 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.569 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.570 INFO:teuthology.orchestra.run.smithi084.stdout: "default", 2024-05-22T23:12:10.570 INFO:teuthology.orchestra.run.smithi084.stdout: "error" 2024-05-22T23:12:10.570 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.570 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.570 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.570 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.570 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.570 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.571 INFO:teuthology.orchestra.run.smithi084.stdout: "standby_error_status_code": { 2024-05-22T23:12:10.571 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "standby_error_status_code", 2024-05-22T23:12:10.571 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.571 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.571 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.571 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "500", 2024-05-22T23:12:10.571 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "400", 2024-05-22T23:12:10.571 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "599", 2024-05-22T23:12:10.572 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.572 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.572 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.572 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.572 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.572 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.572 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.573 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.573 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.573 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rbd_support", 2024-05-22T23:12:10.573 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.573 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.573 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.573 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.573 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.574 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.574 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.574 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.574 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.574 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.574 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.574 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.574 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.575 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.575 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.575 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.575 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.575 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.575 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.575 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.575 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.576 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.576 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.576 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.576 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.576 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.576 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.576 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.576 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.577 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.577 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.577 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.577 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.577 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.577 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.577 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.577 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.578 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.578 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.578 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.578 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.578 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.578 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.578 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.578 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.579 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.579 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.579 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.579 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.579 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.579 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.579 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.579 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.580 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.580 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.580 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.580 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.580 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.580 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.580 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.580 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.581 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.581 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.581 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.581 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.581 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.581 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.581 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.581 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.582 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.582 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.582 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.582 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.582 INFO:teuthology.orchestra.run.smithi084.stdout: "max_concurrent_snap_create": { 2024-05-22T23:12:10.582 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_concurrent_snap_create", 2024-05-22T23:12:10.582 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.583 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.583 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.583 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-05-22T23:12:10.583 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.583 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.583 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.583 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.583 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.584 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.584 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.584 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.584 INFO:teuthology.orchestra.run.smithi084.stdout: "mirror_snapshot_schedule": { 2024-05-22T23:12:10.584 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mirror_snapshot_schedule", 2024-05-22T23:12:10.584 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.584 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.584 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.585 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.585 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.585 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.585 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.585 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.585 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.585 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.585 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.586 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.586 INFO:teuthology.orchestra.run.smithi084.stdout: "trash_purge_schedule": { 2024-05-22T23:12:10.586 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "trash_purge_schedule", 2024-05-22T23:12:10.586 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.586 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.586 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.586 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.586 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.587 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.587 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.587 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.587 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.587 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.587 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.587 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.587 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.588 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.588 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.588 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "restful", 2024-05-22T23:12:10.588 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.588 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.588 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.588 INFO:teuthology.orchestra.run.smithi084.stdout: "enable_auth": { 2024-05-22T23:12:10.588 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "enable_auth", 2024-05-22T23:12:10.589 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.589 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.589 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.589 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.589 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.589 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.589 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.589 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.590 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.590 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.590 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.590 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.590 INFO:teuthology.orchestra.run.smithi084.stdout: "key_file": { 2024-05-22T23:12:10.590 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "key_file", 2024-05-22T23:12:10.590 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.590 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.591 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.591 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.591 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.591 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.591 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.591 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.591 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.591 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.591 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.592 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.592 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.592 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.592 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.592 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.592 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.592 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.592 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.593 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.593 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.593 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.593 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.593 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.593 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.593 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.593 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.594 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.594 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.594 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.594 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.594 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.594 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.594 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.595 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.595 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.595 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.595 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.595 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.595 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.595 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.595 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.595 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.596 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.596 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.596 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.596 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.596 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.596 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.596 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.596 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.597 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.597 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.597 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.597 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.597 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.597 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.597 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.598 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.598 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.598 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.598 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.598 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.598 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.598 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.598 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.599 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.599 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.599 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.599 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.599 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.599 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.599 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.599 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.599 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.600 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.600 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.600 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.600 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.600 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.600 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.600 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.601 INFO:teuthology.orchestra.run.smithi084.stdout: "server_addr": { 2024-05-22T23:12:10.601 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_addr", 2024-05-22T23:12:10.601 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.601 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.601 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.601 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.601 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.601 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.602 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.602 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.602 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.602 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.602 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.602 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.602 INFO:teuthology.orchestra.run.smithi084.stdout: "server_port": { 2024-05-22T23:12:10.602 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_port", 2024-05-22T23:12:10.603 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.603 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.603 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.603 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.603 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.603 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.603 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.603 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.604 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.604 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.604 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.604 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.604 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.604 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.604 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.604 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rgw", 2024-05-22T23:12:10.605 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.605 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.605 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.605 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.605 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.605 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.605 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.605 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.606 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.606 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.606 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.606 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.606 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.606 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.606 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.606 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.607 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.607 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.607 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.607 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.607 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.607 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.607 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.608 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.608 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.608 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.608 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.608 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.608 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.608 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.608 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.609 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.609 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.609 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.609 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.609 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.609 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.609 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.610 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.610 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.610 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.610 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.610 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.610 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.610 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.611 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.611 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.611 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.611 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.611 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.611 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.611 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.612 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.612 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.612 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.612 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.612 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.612 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.612 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.613 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.613 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.613 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.613 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.613 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.613 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.613 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.613 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.614 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.614 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.614 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.614 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.614 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.614 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.614 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.615 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.615 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.615 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rook", 2024-05-22T23:12:10.615 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.615 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.615 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.615 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.615 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.616 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.616 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.616 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.616 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.616 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.616 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.616 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.616 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.616 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.617 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.617 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.617 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.617 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.617 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.617 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.617 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.617 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.618 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.618 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.618 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.618 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.618 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.618 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.618 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.618 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.619 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.619 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.619 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.619 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.619 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.619 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.619 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.619 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.620 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.620 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.620 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.620 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.620 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.620 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.620 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.620 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.620 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.621 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.621 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.621 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.621 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.621 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.621 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.621 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.622 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.622 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.622 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.622 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.622 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.622 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.622 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.622 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.623 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.623 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.623 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.623 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.623 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.623 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.623 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.623 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.624 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.624 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.624 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.624 INFO:teuthology.orchestra.run.smithi084.stdout: "storage_class": { 2024-05-22T23:12:10.624 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "storage_class", 2024-05-22T23:12:10.624 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.624 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.624 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.625 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "local", 2024-05-22T23:12:10.625 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.625 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.625 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.625 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-22T23:12:10.625 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.625 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.625 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.626 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.626 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.626 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.626 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.626 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "selftest", 2024-05-22T23:12:10.626 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.626 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.626 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.627 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.627 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.627 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.627 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.627 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.627 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.627 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.627 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.628 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.628 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.628 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.628 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.628 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.628 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.628 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.628 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.629 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.629 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.629 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.629 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.629 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.629 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.629 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.629 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.630 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.630 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.630 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.630 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.630 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.630 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.630 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.630 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.631 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.631 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.631 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.631 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.631 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.631 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.631 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.632 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.632 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.632 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.632 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.632 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.632 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.632 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.632 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.633 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.633 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.633 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.633 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.633 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.633 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.633 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.633 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.634 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.634 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.634 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.634 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.634 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.634 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.634 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.634 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.635 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.635 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.635 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.635 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.635 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.635 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.635 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.635 INFO:teuthology.orchestra.run.smithi084.stdout: "roption1": { 2024-05-22T23:12:10.636 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "roption1", 2024-05-22T23:12:10.636 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.636 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.636 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.636 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.636 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.636 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.636 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.637 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.637 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.637 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.637 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.637 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.637 INFO:teuthology.orchestra.run.smithi084.stdout: "roption2": { 2024-05-22T23:12:10.637 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "roption2", 2024-05-22T23:12:10.638 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.638 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.638 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.638 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "xyz", 2024-05-22T23:12:10.638 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.638 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.638 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.638 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.639 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.639 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.639 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.639 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.639 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption1": { 2024-05-22T23:12:10.639 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption1", 2024-05-22T23:12:10.639 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.639 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.640 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.640 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.640 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.640 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.640 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.640 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.640 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.640 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.641 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.641 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.641 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption2": { 2024-05-22T23:12:10.641 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption2", 2024-05-22T23:12:10.641 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.641 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.641 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.642 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.642 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.642 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.642 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.642 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.642 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.642 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.642 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.643 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.643 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption3": { 2024-05-22T23:12:10.643 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption3", 2024-05-22T23:12:10.643 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-05-22T23:12:10.643 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.643 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.643 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.644 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.644 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.644 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.644 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.644 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.644 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.644 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.644 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.645 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption4": { 2024-05-22T23:12:10.645 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption4", 2024-05-22T23:12:10.645 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.645 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.645 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.645 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.645 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.645 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.646 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.646 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.646 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.646 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.646 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.646 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.646 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption5": { 2024-05-22T23:12:10.647 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption5", 2024-05-22T23:12:10.647 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.647 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.647 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.647 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.647 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.647 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.647 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.648 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.648 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.648 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.648 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.648 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.648 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption6": { 2024-05-22T23:12:10.648 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption6", 2024-05-22T23:12:10.649 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.649 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.649 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.649 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.649 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.649 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.649 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.649 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.650 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.650 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.650 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.650 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.650 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption7": { 2024-05-22T23:12:10.650 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption7", 2024-05-22T23:12:10.650 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.650 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.651 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.651 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.651 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1", 2024-05-22T23:12:10.651 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "42", 2024-05-22T23:12:10.651 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.651 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.651 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.651 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.652 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.652 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.652 INFO:teuthology.orchestra.run.smithi084.stdout: "testkey": { 2024-05-22T23:12:10.652 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "testkey", 2024-05-22T23:12:10.652 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.652 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.652 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.652 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.652 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.653 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.653 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.653 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.653 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.653 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.653 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.653 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.653 INFO:teuthology.orchestra.run.smithi084.stdout: "testlkey": { 2024-05-22T23:12:10.654 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "testlkey", 2024-05-22T23:12:10.654 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.654 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.654 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.654 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.654 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.654 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.654 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.655 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.655 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.655 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.655 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.655 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.655 INFO:teuthology.orchestra.run.smithi084.stdout: "testnewline": { 2024-05-22T23:12:10.655 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "testnewline", 2024-05-22T23:12:10.655 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.656 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.656 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.656 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.656 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.656 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.656 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.656 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.656 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.656 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.657 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.657 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.657 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.657 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.657 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.657 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "snap_schedule", 2024-05-22T23:12:10.657 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.658 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.658 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.658 INFO:teuthology.orchestra.run.smithi084.stdout: "allow_m_granularity": { 2024-05-22T23:12:10.658 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "allow_m_granularity", 2024-05-22T23:12:10.658 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.658 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.658 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.658 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.659 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.659 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.659 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.659 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "allow minute scheduled snapshots", 2024-05-22T23:12:10.659 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.659 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.659 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.660 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.660 INFO:teuthology.orchestra.run.smithi084.stdout: "dump_on_update": { 2024-05-22T23:12:10.660 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "dump_on_update", 2024-05-22T23:12:10.660 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.660 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.660 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.660 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.660 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.661 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.661 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.661 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "dump database to debug log on update", 2024-05-22T23:12:10.661 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.661 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.661 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.661 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.661 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.662 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.662 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.662 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.662 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.662 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.662 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.662 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.662 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.663 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.663 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.663 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.663 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.663 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.663 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.663 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.663 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.664 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.664 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.664 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.664 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.664 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.664 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.664 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.664 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.665 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.665 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.665 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.665 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.665 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.665 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.665 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.665 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.666 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.666 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.666 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.666 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.666 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.666 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.666 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.666 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.667 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.667 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.667 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.667 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.667 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.667 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.668 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.668 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.668 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.668 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.668 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.668 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.668 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.668 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.669 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.669 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.669 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.669 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.669 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.669 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.669 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.669 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.670 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.670 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.670 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.670 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.670 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.670 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.670 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.671 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.671 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.671 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.671 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "stats", 2024-05-22T23:12:10.671 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.671 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.671 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.671 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.672 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.672 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.672 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.672 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.672 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.672 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.672 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.672 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.673 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.673 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.673 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.673 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.673 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.673 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.673 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.673 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.674 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.674 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.674 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.674 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.674 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.674 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.674 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.675 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.675 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.675 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.675 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.675 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.675 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.675 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.675 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.676 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.676 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.676 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.676 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.676 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.676 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.676 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.676 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.677 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.677 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.677 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.677 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.677 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.677 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.677 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.677 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.678 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.678 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.678 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.678 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.678 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.678 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.678 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.678 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.679 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.679 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.679 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.679 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.679 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.679 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.679 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.679 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.680 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.680 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.680 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.680 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.680 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.680 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.680 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.681 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.681 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.681 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "status", 2024-05-22T23:12:10.681 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.681 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.681 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.681 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.681 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.682 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.682 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.682 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.682 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.682 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.682 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.682 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.682 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.683 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.683 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.683 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.683 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.683 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.683 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.683 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.683 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.684 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.684 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.684 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.684 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.684 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.684 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.684 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.684 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.685 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.685 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.685 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.685 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.685 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.685 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.685 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.685 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.685 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.686 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.686 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.686 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.686 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.686 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.686 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.686 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.686 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.687 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.687 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.687 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.687 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.687 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.687 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.687 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.687 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.688 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.688 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.688 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.688 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.688 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.688 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.688 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.688 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.689 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.689 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.689 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.689 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.689 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.689 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.689 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.689 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.690 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.690 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.690 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.690 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.690 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.690 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.690 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "telegraf", 2024-05-22T23:12:10.690 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.691 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.691 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.691 INFO:teuthology.orchestra.run.smithi084.stdout: "address": { 2024-05-22T23:12:10.691 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "address", 2024-05-22T23:12:10.691 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.691 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.691 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.691 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-22T23:12:10.692 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.692 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.692 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.692 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.692 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.692 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.692 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.692 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.692 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-05-22T23:12:10.693 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-05-22T23:12:10.693 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.693 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.693 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.693 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "15", 2024-05-22T23:12:10.693 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.693 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.693 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.694 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.694 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.694 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.694 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.694 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.694 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.694 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.694 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.695 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.695 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.695 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.695 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.695 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.695 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.695 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.695 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.696 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.696 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.696 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.696 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.696 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.696 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.696 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.696 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.696 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.697 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.697 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.697 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.697 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.697 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.697 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.697 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.697 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.698 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.698 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.698 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.698 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.698 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.698 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.698 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.698 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.699 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.699 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.699 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.699 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.699 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.699 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.699 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.699 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.700 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.700 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.700 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.700 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.700 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.700 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.700 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.700 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.701 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.701 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.701 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.701 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.701 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.701 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.701 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.701 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.701 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.702 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.702 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.702 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.702 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.702 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.702 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.702 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.703 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.703 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.703 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.703 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.703 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.703 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "telemetry", 2024-05-22T23:12:10.703 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.703 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.704 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.704 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_basic": { 2024-05-22T23:12:10.704 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_basic", 2024-05-22T23:12:10.704 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.704 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.704 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.704 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.704 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.705 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.705 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.705 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-22T23:12:10.705 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.705 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.705 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.705 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.706 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_crash": { 2024-05-22T23:12:10.706 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_crash", 2024-05-22T23:12:10.706 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.706 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.706 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.706 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.706 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.706 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.707 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.707 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-22T23:12:10.707 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.707 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.707 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.707 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.707 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_device": { 2024-05-22T23:12:10.707 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_device", 2024-05-22T23:12:10.708 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.708 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.708 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.708 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.708 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.708 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.708 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.708 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-22T23:12:10.708 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.709 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.709 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.709 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.709 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_ident": { 2024-05-22T23:12:10.709 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_ident", 2024-05-22T23:12:10.709 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.709 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.709 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.710 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.710 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.710 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.710 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.710 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-22T23:12:10.710 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.710 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.710 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.711 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.711 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_perf": { 2024-05-22T23:12:10.711 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_perf", 2024-05-22T23:12:10.711 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.711 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.711 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.711 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.711 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.712 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.712 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.712 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-22T23:12:10.712 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.712 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.712 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.712 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.712 INFO:teuthology.orchestra.run.smithi084.stdout: "contact": { 2024-05-22T23:12:10.712 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "contact", 2024-05-22T23:12:10.713 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.713 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.713 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.713 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.713 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.713 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.713 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.713 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.714 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.714 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.714 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.714 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.714 INFO:teuthology.orchestra.run.smithi084.stdout: "description": { 2024-05-22T23:12:10.714 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "description", 2024-05-22T23:12:10.714 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.715 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.715 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.715 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.715 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.715 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.715 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.715 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.715 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.716 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.716 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.716 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.716 INFO:teuthology.orchestra.run.smithi084.stdout: "device_url": { 2024-05-22T23:12:10.716 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "device_url", 2024-05-22T23:12:10.716 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.716 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.716 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.716 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-22T23:12:10.717 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.717 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.717 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.717 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.717 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.717 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.717 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.717 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.718 INFO:teuthology.orchestra.run.smithi084.stdout: "enabled": { 2024-05-22T23:12:10.718 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "enabled", 2024-05-22T23:12:10.718 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.718 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.718 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.718 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.718 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.718 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.719 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.719 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.719 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.719 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.719 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.719 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.719 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-05-22T23:12:10.720 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-05-22T23:12:10.720 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.720 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.720 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.720 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "24", 2024-05-22T23:12:10.720 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "8", 2024-05-22T23:12:10.720 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.720 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.721 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.721 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.721 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.721 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.721 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.721 INFO:teuthology.orchestra.run.smithi084.stdout: "last_opt_revision": { 2024-05-22T23:12:10.721 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "last_opt_revision", 2024-05-22T23:12:10.721 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.722 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.722 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.722 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "1", 2024-05-22T23:12:10.722 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.722 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.722 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.722 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.723 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.723 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.723 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.723 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.723 INFO:teuthology.orchestra.run.smithi084.stdout: "leaderboard": { 2024-05-22T23:12:10.723 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "leaderboard", 2024-05-22T23:12:10.723 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.723 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.723 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.724 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.724 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.724 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.724 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.724 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.724 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.724 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.724 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.725 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.725 INFO:teuthology.orchestra.run.smithi084.stdout: "leaderboard_description": { 2024-05-22T23:12:10.725 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "leaderboard_description", 2024-05-22T23:12:10.725 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.725 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.725 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.725 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.725 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.725 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.726 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.726 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.726 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.726 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.726 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.726 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.726 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.726 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.727 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.727 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.727 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.727 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.727 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.727 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.727 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.727 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.728 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.728 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.728 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.728 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.728 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.728 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.728 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.728 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.729 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.729 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.729 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.729 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.729 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.729 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.729 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.729 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.730 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.730 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.730 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.730 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.730 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.730 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.730 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.730 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.731 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.731 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.731 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.731 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.731 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.731 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.731 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.732 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.732 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.732 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.732 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.732 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.732 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.732 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.732 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.733 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.733 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.733 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.733 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.733 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.733 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.733 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.733 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.734 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.734 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.734 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.734 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.734 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.734 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.734 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.734 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.735 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.735 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.735 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.735 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.735 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.735 INFO:teuthology.orchestra.run.smithi084.stdout: "organization": { 2024-05-22T23:12:10.735 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "organization", 2024-05-22T23:12:10.735 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.736 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.736 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.736 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.736 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.736 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.736 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.736 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.736 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.737 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.737 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.737 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.737 INFO:teuthology.orchestra.run.smithi084.stdout: "proxy": { 2024-05-22T23:12:10.737 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "proxy", 2024-05-22T23:12:10.737 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.737 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.737 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.737 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.738 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.738 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.738 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.738 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.738 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.738 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.738 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.739 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.739 INFO:teuthology.orchestra.run.smithi084.stdout: "url": { 2024-05-22T23:12:10.739 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "url", 2024-05-22T23:12:10.739 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.739 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.739 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.739 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-22T23:12:10.739 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.740 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.740 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.740 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.740 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.740 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.740 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.740 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.741 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.741 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.741 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.741 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "test_orchestrator", 2024-05-22T23:12:10.741 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.741 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.741 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.742 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.742 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.742 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.742 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.742 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.742 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.742 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.742 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.743 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.743 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.743 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.743 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.743 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.743 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.743 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.743 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.744 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.744 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.744 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.744 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.744 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.744 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.744 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.744 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.745 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.745 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.745 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.745 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.745 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.745 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.745 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.746 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.746 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.746 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.746 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.746 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.746 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.746 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.746 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.747 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.747 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.747 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.747 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.747 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.747 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.747 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.748 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.748 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.748 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.748 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.748 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.748 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.748 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.748 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.749 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.749 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.749 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.749 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.749 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.749 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.749 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.749 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.750 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.750 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.750 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.750 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.750 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.750 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.750 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.751 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.751 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.751 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.751 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.751 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "volumes", 2024-05-22T23:12:10.751 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.751 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.751 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.752 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.752 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.752 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.752 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.752 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.752 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.752 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.752 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.753 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.753 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.753 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.753 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.753 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.753 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.753 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.753 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.754 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.754 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.754 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.754 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.754 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.754 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.754 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.755 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.755 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.755 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.755 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.755 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.755 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.755 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.755 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.755 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.756 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.756 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.756 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.756 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.756 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.756 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.756 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.756 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.757 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.757 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.757 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.757 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.757 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.757 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.757 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.757 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.758 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.758 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.758 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.758 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.758 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.758 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.758 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.759 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.759 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.759 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.759 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.759 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.759 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.759 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.759 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.760 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.760 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.760 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.760 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.760 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.760 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.760 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.760 INFO:teuthology.orchestra.run.smithi084.stdout: "max_concurrent_clones": { 2024-05-22T23:12:10.761 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_concurrent_clones", 2024-05-22T23:12:10.761 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.761 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.761 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.761 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "4", 2024-05-22T23:12:10.761 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.761 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.761 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.762 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-22T23:12:10.762 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.762 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.762 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.762 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.762 INFO:teuthology.orchestra.run.smithi084.stdout: "periodic_async_work": { 2024-05-22T23:12:10.762 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "periodic_async_work", 2024-05-22T23:12:10.762 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.763 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.763 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.763 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.763 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.763 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.763 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.763 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Periodically check for async work", 2024-05-22T23:12:10.763 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.764 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.764 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.764 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.764 INFO:teuthology.orchestra.run.smithi084.stdout: "snapshot_clone_delay": { 2024-05-22T23:12:10.764 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "snapshot_clone_delay", 2024-05-22T23:12:10.764 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.764 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.764 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.764 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-05-22T23:12:10.765 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.765 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.765 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.765 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-22T23:12:10.765 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.765 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.765 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.765 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.765 INFO:teuthology.orchestra.run.smithi084.stdout: "snapshot_clone_no_wait": { 2024-05-22T23:12:10.766 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "snapshot_clone_no_wait", 2024-05-22T23:12:10.766 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.766 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.766 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.766 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.766 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.766 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.766 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.767 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-22T23:12:10.767 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.767 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.767 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.767 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.767 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.767 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.767 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.767 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "zabbix", 2024-05-22T23:12:10.768 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.768 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.768 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.768 INFO:teuthology.orchestra.run.smithi084.stdout: "discovery_interval": { 2024-05-22T23:12:10.768 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "discovery_interval", 2024-05-22T23:12:10.768 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-05-22T23:12:10.768 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.768 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.769 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "100", 2024-05-22T23:12:10.769 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.769 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.769 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.769 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.769 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.769 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.769 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.770 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.770 INFO:teuthology.orchestra.run.smithi084.stdout: "identifier": { 2024-05-22T23:12:10.770 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "identifier", 2024-05-22T23:12:10.770 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.770 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.770 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.770 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.771 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.771 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.771 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.771 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.771 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.771 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.771 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.772 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.772 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-05-22T23:12:10.772 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-05-22T23:12:10.772 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.772 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.772 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.772 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-05-22T23:12:10.772 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.773 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.773 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.773 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.773 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.773 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.773 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.773 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.774 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.774 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.774 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.774 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.774 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.774 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.774 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.774 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.775 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.775 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.775 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.775 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.775 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.775 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.775 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.776 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.776 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.776 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.776 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.776 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.776 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.776 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.776 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.777 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.777 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.777 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.777 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.777 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.777 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.777 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.777 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.778 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.778 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.778 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.778 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.778 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.778 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.778 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.779 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.779 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.779 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.779 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.779 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.779 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.779 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.780 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.780 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.780 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.780 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.780 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.780 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.780 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.780 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.781 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.781 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.781 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.781 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.781 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.781 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.781 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.782 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.782 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.782 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.782 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.782 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.782 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.782 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.782 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.783 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.783 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.783 INFO:teuthology.orchestra.run.smithi084.stdout: "zabbix_host": { 2024-05-22T23:12:10.783 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "zabbix_host", 2024-05-22T23:12:10.783 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.783 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.783 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.783 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.784 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.784 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.784 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.784 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.784 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.784 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.784 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.784 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.784 INFO:teuthology.orchestra.run.smithi084.stdout: "zabbix_port": { 2024-05-22T23:12:10.785 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "zabbix_port", 2024-05-22T23:12:10.785 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.785 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.785 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.785 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10051", 2024-05-22T23:12:10.785 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.785 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.785 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.785 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.786 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.786 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.786 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.786 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.786 INFO:teuthology.orchestra.run.smithi084.stdout: "zabbix_sender": { 2024-05-22T23:12:10.786 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "zabbix_sender", 2024-05-22T23:12:10.786 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.786 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.787 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.787 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-22T23:12:10.787 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.787 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.787 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.787 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.787 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.787 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.788 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.788 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.788 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.788 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.788 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-05-22T23:12:10.788 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.788 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.788 INFO:teuthology.orchestra.run.smithi084.stdout: "modules": [ 2024-05-22T23:12:10.788 INFO:teuthology.orchestra.run.smithi084.stdout: "cephadm", 2024-05-22T23:12:10.789 INFO:teuthology.orchestra.run.smithi084.stdout: "dashboard", 2024-05-22T23:12:10.789 INFO:teuthology.orchestra.run.smithi084.stdout: "iostat", 2024-05-22T23:12:10.789 INFO:teuthology.orchestra.run.smithi084.stdout: "nfs", 2024-05-22T23:12:10.789 INFO:teuthology.orchestra.run.smithi084.stdout: "restful", 2024-05-22T23:12:10.789 INFO:teuthology.orchestra.run.smithi084.stdout: "snap_schedule" 2024-05-22T23:12:10.789 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.789 INFO:teuthology.orchestra.run.smithi084.stdout: "available_modules": [ 2024-05-22T23:12:10.789 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.789 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "alerts", 2024-05-22T23:12:10.790 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.790 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.790 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.790 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-05-22T23:12:10.790 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-05-22T23:12:10.790 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.790 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.791 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.791 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-05-22T23:12:10.791 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.791 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.791 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.791 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "How frequently to reexamine health status", 2024-05-22T23:12:10.791 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.792 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.792 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.792 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.792 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.792 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.792 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.792 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.792 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.793 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.793 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.793 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.793 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.793 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.793 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.793 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.793 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.794 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.794 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.794 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.794 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.794 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.794 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.794 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.794 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.795 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.795 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.795 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.795 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.795 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.795 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.795 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.795 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.795 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.796 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.796 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.796 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.796 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.796 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.796 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.796 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.796 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.796 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.797 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.797 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.797 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.797 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.797 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.797 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.797 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.797 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.798 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.798 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.798 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.798 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.798 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.798 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.798 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.798 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.798 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.799 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.799 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.799 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.799 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.799 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.799 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.799 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.800 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.800 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.800 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.800 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.800 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.800 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.800 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.800 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_destination": { 2024-05-22T23:12:10.801 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_destination", 2024-05-22T23:12:10.801 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.801 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.801 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.801 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.801 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.801 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.801 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.802 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Email address to send alerts to", 2024-05-22T23:12:10.802 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.802 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.802 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.802 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.802 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_from_name": { 2024-05-22T23:12:10.802 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_from_name", 2024-05-22T23:12:10.802 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.803 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.803 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.803 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "Ceph", 2024-05-22T23:12:10.803 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.803 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.803 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.803 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Email From: name", 2024-05-22T23:12:10.803 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.803 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.804 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.804 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.804 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_host": { 2024-05-22T23:12:10.804 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_host", 2024-05-22T23:12:10.804 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.804 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.804 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.804 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.805 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.805 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.805 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.805 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "SMTP server", 2024-05-22T23:12:10.805 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.805 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.805 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.805 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.806 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_password": { 2024-05-22T23:12:10.806 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_password", 2024-05-22T23:12:10.806 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.806 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.806 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.806 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.806 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.806 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.807 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.807 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Password to authenticate with", 2024-05-22T23:12:10.807 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.807 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.807 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.807 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.807 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_port": { 2024-05-22T23:12:10.807 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_port", 2024-05-22T23:12:10.808 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.808 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.808 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.808 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "465", 2024-05-22T23:12:10.808 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.808 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.808 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.808 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "SMTP port", 2024-05-22T23:12:10.809 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.809 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.809 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.809 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.809 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_sender": { 2024-05-22T23:12:10.809 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_sender", 2024-05-22T23:12:10.809 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.809 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.810 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.810 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.810 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.810 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.810 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.810 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "SMTP envelope sender", 2024-05-22T23:12:10.810 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.811 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.811 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.811 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.811 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_ssl": { 2024-05-22T23:12:10.811 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_ssl", 2024-05-22T23:12:10.811 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.811 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.811 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.812 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.812 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.812 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.812 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.812 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-22T23:12:10.812 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.812 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.812 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.813 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.813 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_user": { 2024-05-22T23:12:10.813 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_user", 2024-05-22T23:12:10.813 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.813 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.813 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.813 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.813 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.814 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.814 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.814 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "User to authenticate as", 2024-05-22T23:12:10.814 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.814 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.814 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.814 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.814 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.815 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.815 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.815 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "balancer", 2024-05-22T23:12:10.815 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.815 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.815 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.815 INFO:teuthology.orchestra.run.smithi084.stdout: "active": { 2024-05-22T23:12:10.815 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "active", 2024-05-22T23:12:10.816 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.816 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.816 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.816 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.816 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.816 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.816 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.816 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "automatically balance PGs across cluster", 2024-05-22T23:12:10.817 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.817 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.817 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.817 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.817 INFO:teuthology.orchestra.run.smithi084.stdout: "begin_time": { 2024-05-22T23:12:10.817 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "begin_time", 2024-05-22T23:12:10.817 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.817 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.818 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.818 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0000", 2024-05-22T23:12:10.818 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.818 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.818 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.818 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "beginning time of day to automatically balance", 2024-05-22T23:12:10.818 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-22T23:12:10.818 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.819 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.819 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.819 INFO:teuthology.orchestra.run.smithi084.stdout: "begin_weekday": { 2024-05-22T23:12:10.819 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "begin_weekday", 2024-05-22T23:12:10.819 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-05-22T23:12:10.819 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.819 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.819 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-05-22T23:12:10.820 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "0", 2024-05-22T23:12:10.820 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "6", 2024-05-22T23:12:10.820 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.820 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-22T23:12:10.820 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-22T23:12:10.820 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.820 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.820 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.821 INFO:teuthology.orchestra.run.smithi084.stdout: "crush_compat_max_iterations": { 2024-05-22T23:12:10.821 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crush_compat_max_iterations", 2024-05-22T23:12:10.821 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-05-22T23:12:10.821 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.821 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.821 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "25", 2024-05-22T23:12:10.821 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1", 2024-05-22T23:12:10.821 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "250", 2024-05-22T23:12:10.822 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.822 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-22T23:12:10.822 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.822 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.822 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.822 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.822 INFO:teuthology.orchestra.run.smithi084.stdout: "crush_compat_metrics": { 2024-05-22T23:12:10.822 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crush_compat_metrics", 2024-05-22T23:12:10.823 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.823 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.823 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.823 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "pgs,objects,bytes", 2024-05-22T23:12:10.823 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.823 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.823 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.823 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-22T23:12:10.824 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-22T23:12:10.824 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.824 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.824 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.824 INFO:teuthology.orchestra.run.smithi084.stdout: "crush_compat_step": { 2024-05-22T23:12:10.824 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crush_compat_step", 2024-05-22T23:12:10.824 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-05-22T23:12:10.825 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.825 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.825 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0.5", 2024-05-22T23:12:10.825 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "0.001", 2024-05-22T23:12:10.825 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "0.999", 2024-05-22T23:12:10.825 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.825 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "aggressiveness of optimization", 2024-05-22T23:12:10.825 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-22T23:12:10.825 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.826 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.826 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.826 INFO:teuthology.orchestra.run.smithi084.stdout: "end_time": { 2024-05-22T23:12:10.826 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "end_time", 2024-05-22T23:12:10.826 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.826 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.826 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.827 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "2359", 2024-05-22T23:12:10.827 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.827 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.827 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.827 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "ending time of day to automatically balance", 2024-05-22T23:12:10.827 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-22T23:12:10.827 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.827 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.828 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.828 INFO:teuthology.orchestra.run.smithi084.stdout: "end_weekday": { 2024-05-22T23:12:10.828 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "end_weekday", 2024-05-22T23:12:10.828 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-05-22T23:12:10.828 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.828 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.828 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-05-22T23:12:10.829 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "0", 2024-05-22T23:12:10.829 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "6", 2024-05-22T23:12:10.829 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.829 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-22T23:12:10.829 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-22T23:12:10.829 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.829 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.829 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.830 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.830 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.830 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.830 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.830 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.830 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.830 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.831 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.831 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.831 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.831 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.831 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.831 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.831 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.831 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.832 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.832 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.832 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.832 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.832 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.832 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.832 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.832 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.833 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.833 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.833 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.833 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.833 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.833 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.833 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.833 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.834 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.834 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.834 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.834 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.834 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.834 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.834 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.835 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.835 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.835 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.835 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.835 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.835 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.835 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.835 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.836 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.836 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.836 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.836 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.836 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.836 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.836 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.836 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.837 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.837 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.837 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.837 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.837 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.837 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.837 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.837 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.838 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.838 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.838 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.838 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.838 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.838 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.838 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.838 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.839 INFO:teuthology.orchestra.run.smithi084.stdout: "min_score": { 2024-05-22T23:12:10.839 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "min_score", 2024-05-22T23:12:10.839 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-05-22T23:12:10.839 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.839 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.839 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-05-22T23:12:10.839 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.840 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.840 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.840 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-22T23:12:10.840 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.840 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.840 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.840 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.840 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": { 2024-05-22T23:12:10.841 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mode", 2024-05-22T23:12:10.841 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.841 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.841 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.841 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "upmap", 2024-05-22T23:12:10.841 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.841 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.841 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.842 INFO:teuthology.orchestra.run.smithi084.stdout: "crush-compat", 2024-05-22T23:12:10.842 INFO:teuthology.orchestra.run.smithi084.stdout: "none", 2024-05-22T23:12:10.842 INFO:teuthology.orchestra.run.smithi084.stdout: "read", 2024-05-22T23:12:10.842 INFO:teuthology.orchestra.run.smithi084.stdout: "upmap", 2024-05-22T23:12:10.842 INFO:teuthology.orchestra.run.smithi084.stdout: "upmap-read" 2024-05-22T23:12:10.842 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.842 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Balancer mode", 2024-05-22T23:12:10.842 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.843 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.843 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.843 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.843 INFO:teuthology.orchestra.run.smithi084.stdout: "pool_ids": { 2024-05-22T23:12:10.843 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "pool_ids", 2024-05-22T23:12:10.843 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.843 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.843 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.844 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.844 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.844 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.844 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.844 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-22T23:12:10.844 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.844 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.844 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.845 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.845 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-05-22T23:12:10.845 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-05-22T23:12:10.845 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.845 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.845 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.845 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-05-22T23:12:10.845 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.846 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.846 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.846 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-22T23:12:10.846 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.846 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.846 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.846 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.846 INFO:teuthology.orchestra.run.smithi084.stdout: "upmap_max_deviation": { 2024-05-22T23:12:10.847 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "upmap_max_deviation", 2024-05-22T23:12:10.847 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.847 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.847 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.847 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5", 2024-05-22T23:12:10.847 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1", 2024-05-22T23:12:10.847 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.847 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.848 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-22T23:12:10.848 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-22T23:12:10.848 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.848 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.848 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.848 INFO:teuthology.orchestra.run.smithi084.stdout: "upmap_max_optimizations": { 2024-05-22T23:12:10.848 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "upmap_max_optimizations", 2024-05-22T23:12:10.848 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-05-22T23:12:10.848 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.849 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.849 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-05-22T23:12:10.849 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.849 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.849 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.849 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-22T23:12:10.849 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.849 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.850 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.850 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.850 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.850 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.850 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.850 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephadm", 2024-05-22T23:12:10.850 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.851 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.851 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.851 INFO:teuthology.orchestra.run.smithi084.stdout: "agent_down_multiplier": { 2024-05-22T23:12:10.851 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "agent_down_multiplier", 2024-05-22T23:12:10.851 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-05-22T23:12:10.851 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.851 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.851 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "3.0", 2024-05-22T23:12:10.852 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.852 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.852 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.852 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-22T23:12:10.852 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.852 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.852 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.853 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.853 INFO:teuthology.orchestra.run.smithi084.stdout: "agent_refresh_rate": { 2024-05-22T23:12:10.853 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "agent_refresh_rate", 2024-05-22T23:12:10.853 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.853 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.853 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.853 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "20", 2024-05-22T23:12:10.853 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.854 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.854 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.854 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-22T23:12:10.854 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.854 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.854 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.854 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.855 INFO:teuthology.orchestra.run.smithi084.stdout: "agent_starting_port": { 2024-05-22T23:12:10.855 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "agent_starting_port", 2024-05-22T23:12:10.855 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.855 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.855 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.855 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "4721", 2024-05-22T23:12:10.855 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.855 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.856 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.856 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-22T23:12:10.856 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.856 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.856 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.856 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.856 INFO:teuthology.orchestra.run.smithi084.stdout: "allow_ptrace": { 2024-05-22T23:12:10.857 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "allow_ptrace", 2024-05-22T23:12:10.857 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.857 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.857 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.857 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.857 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.857 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.857 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.858 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-22T23:12:10.858 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-22T23:12:10.858 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.858 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.858 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.858 INFO:teuthology.orchestra.run.smithi084.stdout: "autotune_interval": { 2024-05-22T23:12:10.858 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "autotune_interval", 2024-05-22T23:12:10.859 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.859 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.859 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.859 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-05-22T23:12:10.859 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.859 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.859 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.859 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-22T23:12:10.860 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.860 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.860 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.860 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.860 INFO:teuthology.orchestra.run.smithi084.stdout: "autotune_memory_target_ratio": { 2024-05-22T23:12:10.860 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "autotune_memory_target_ratio", 2024-05-22T23:12:10.860 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-05-22T23:12:10.860 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.861 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.861 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0.7", 2024-05-22T23:12:10.861 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.861 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.861 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.861 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-22T23:12:10.861 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.861 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.862 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.862 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.862 INFO:teuthology.orchestra.run.smithi084.stdout: "cephadm_log_destination": { 2024-05-22T23:12:10.862 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephadm_log_destination", 2024-05-22T23:12:10.862 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.862 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.862 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.862 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.863 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.863 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.863 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.863 INFO:teuthology.orchestra.run.smithi084.stdout: "file", 2024-05-22T23:12:10.863 INFO:teuthology.orchestra.run.smithi084.stdout: "file,syslog", 2024-05-22T23:12:10.863 INFO:teuthology.orchestra.run.smithi084.stdout: "syslog" 2024-05-22T23:12:10.863 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.863 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-22T23:12:10.864 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.864 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.864 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.864 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.864 INFO:teuthology.orchestra.run.smithi084.stdout: "cgroups_split": { 2024-05-22T23:12:10.864 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cgroups_split", 2024-05-22T23:12:10.864 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.865 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.865 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.865 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.865 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.865 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.865 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.865 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-22T23:12:10.865 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.865 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.866 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.866 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.866 INFO:teuthology.orchestra.run.smithi084.stdout: "config_checks_enabled": { 2024-05-22T23:12:10.866 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "config_checks_enabled", 2024-05-22T23:12:10.866 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.866 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.866 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.867 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.867 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.867 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.867 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.867 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-22T23:12:10.867 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.867 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.867 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.868 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.868 INFO:teuthology.orchestra.run.smithi084.stdout: "config_dashboard": { 2024-05-22T23:12:10.868 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "config_dashboard", 2024-05-22T23:12:10.868 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.868 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.868 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.868 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.868 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.869 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.869 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.869 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-22T23:12:10.869 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.869 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.869 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.869 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.869 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_alertmanager": { 2024-05-22T23:12:10.870 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_alertmanager", 2024-05-22T23:12:10.870 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.870 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.870 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.870 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-22T23:12:10.870 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.870 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.871 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.871 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Prometheus container image", 2024-05-22T23:12:10.871 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.871 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.871 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.871 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.871 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_base": { 2024-05-22T23:12:10.871 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_base", 2024-05-22T23:12:10.872 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.872 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.872 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.872 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-22T23:12:10.872 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.872 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.872 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.873 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Container image name, without the tag", 2024-05-22T23:12:10.873 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.873 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.873 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.873 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.873 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_elasticsearch": { 2024-05-22T23:12:10.873 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_elasticsearch", 2024-05-22T23:12:10.873 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.874 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.874 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.874 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-22T23:12:10.874 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.874 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.874 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.874 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "elasticsearch container image", 2024-05-22T23:12:10.875 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.875 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.875 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.875 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.875 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_grafana": { 2024-05-22T23:12:10.875 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_grafana", 2024-05-22T23:12:10.875 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.875 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.876 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.876 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-22T23:12:10.876 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.876 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.876 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.876 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Prometheus container image", 2024-05-22T23:12:10.876 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.876 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.876 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.877 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.877 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_haproxy": { 2024-05-22T23:12:10.877 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_haproxy", 2024-05-22T23:12:10.877 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.877 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.877 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.877 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-22T23:12:10.877 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.878 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.878 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.878 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "HAproxy container image", 2024-05-22T23:12:10.878 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.878 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.878 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.878 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.878 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_jaeger_agent": { 2024-05-22T23:12:10.879 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_jaeger_agent", 2024-05-22T23:12:10.879 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.879 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.879 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.879 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-22T23:12:10.879 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.879 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.879 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.880 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Jaeger agent container image", 2024-05-22T23:12:10.880 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.880 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.880 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.880 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.880 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_jaeger_collector": { 2024-05-22T23:12:10.880 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_jaeger_collector", 2024-05-22T23:12:10.880 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.881 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.881 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.881 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-22T23:12:10.881 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.881 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.881 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.881 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Jaeger collector container image", 2024-05-22T23:12:10.881 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.882 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.882 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.882 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.882 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_jaeger_query": { 2024-05-22T23:12:10.882 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_jaeger_query", 2024-05-22T23:12:10.882 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.882 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.882 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.883 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-22T23:12:10.883 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.883 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.883 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.883 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Jaeger query container image", 2024-05-22T23:12:10.883 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.883 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.883 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.883 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.884 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_keepalived": { 2024-05-22T23:12:10.884 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_keepalived", 2024-05-22T23:12:10.884 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.884 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.884 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.884 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-22T23:12:10.884 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.884 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.885 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.885 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Keepalived container image", 2024-05-22T23:12:10.885 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.885 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.885 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.885 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.885 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_loki": { 2024-05-22T23:12:10.885 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_loki", 2024-05-22T23:12:10.885 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.886 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.886 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.886 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-22T23:12:10.886 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.886 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.886 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.886 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Loki container image", 2024-05-22T23:12:10.886 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.887 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.887 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.887 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.887 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_node_exporter": { 2024-05-22T23:12:10.887 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_node_exporter", 2024-05-22T23:12:10.887 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.887 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.887 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.888 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-22T23:12:10.888 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.888 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.888 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.888 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Prometheus container image", 2024-05-22T23:12:10.888 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.888 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.888 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.888 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.889 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_nvmeof": { 2024-05-22T23:12:10.889 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_nvmeof", 2024-05-22T23:12:10.889 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.889 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.889 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.889 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-22T23:12:10.889 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.889 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.889 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.890 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Nvme-of container image", 2024-05-22T23:12:10.890 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.890 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.890 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.890 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.890 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_prometheus": { 2024-05-22T23:12:10.890 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_prometheus", 2024-05-22T23:12:10.891 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.891 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.891 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.891 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-22T23:12:10.891 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.891 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.891 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.891 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Prometheus container image", 2024-05-22T23:12:10.891 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.892 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.892 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.892 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.892 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_promtail": { 2024-05-22T23:12:10.892 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_promtail", 2024-05-22T23:12:10.892 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.892 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.893 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.893 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-22T23:12:10.893 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.893 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.893 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.893 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Promtail container image", 2024-05-22T23:12:10.893 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.893 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.894 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.894 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.894 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_samba": { 2024-05-22T23:12:10.894 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_samba", 2024-05-22T23:12:10.894 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.894 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.894 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.894 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-22T23:12:10.895 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.895 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.895 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.895 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Samba/SMB container image", 2024-05-22T23:12:10.895 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.895 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.895 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.895 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.896 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_snmp_gateway": { 2024-05-22T23:12:10.896 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_snmp_gateway", 2024-05-22T23:12:10.896 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.896 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.896 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.896 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-22T23:12:10.896 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.896 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.897 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.897 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "SNMP Gateway container image", 2024-05-22T23:12:10.897 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.897 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.897 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.897 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.897 INFO:teuthology.orchestra.run.smithi084.stdout: "container_init": { 2024-05-22T23:12:10.898 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_init", 2024-05-22T23:12:10.898 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.898 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.898 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.898 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.898 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.898 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.898 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.899 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Run podman/docker with `--init`", 2024-05-22T23:12:10.899 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.899 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.899 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.899 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.899 INFO:teuthology.orchestra.run.smithi084.stdout: "daemon_cache_timeout": { 2024-05-22T23:12:10.899 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "daemon_cache_timeout", 2024-05-22T23:12:10.899 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.900 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.900 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.900 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-05-22T23:12:10.900 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.900 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.900 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.900 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-22T23:12:10.900 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.900 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.901 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.901 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.901 INFO:teuthology.orchestra.run.smithi084.stdout: "default_cephadm_command_timeout": { 2024-05-22T23:12:10.901 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "default_cephadm_command_timeout", 2024-05-22T23:12:10.901 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.901 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.901 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.901 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "900", 2024-05-22T23:12:10.901 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.902 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.902 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.902 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-22T23:12:10.902 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.902 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.902 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.902 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.903 INFO:teuthology.orchestra.run.smithi084.stdout: "default_registry": { 2024-05-22T23:12:10.903 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "default_registry", 2024-05-22T23:12:10.903 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.903 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.903 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.903 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "docker.io", 2024-05-22T23:12:10.903 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.904 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.904 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.904 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-22T23:12:10.904 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.904 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.904 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.904 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.904 INFO:teuthology.orchestra.run.smithi084.stdout: "device_cache_timeout": { 2024-05-22T23:12:10.905 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "device_cache_timeout", 2024-05-22T23:12:10.905 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.905 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.905 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.905 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "1800", 2024-05-22T23:12:10.905 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.905 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.905 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.906 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "seconds to cache device inventory", 2024-05-22T23:12:10.906 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.906 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.906 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.906 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.906 INFO:teuthology.orchestra.run.smithi084.stdout: "device_enhanced_scan": { 2024-05-22T23:12:10.906 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "device_enhanced_scan", 2024-05-22T23:12:10.906 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.907 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.907 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.907 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.907 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.907 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.907 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.907 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-22T23:12:10.907 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.907 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.908 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.908 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.908 INFO:teuthology.orchestra.run.smithi084.stdout: "facts_cache_timeout": { 2024-05-22T23:12:10.908 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "facts_cache_timeout", 2024-05-22T23:12:10.908 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.908 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.908 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.908 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-05-22T23:12:10.908 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.909 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.909 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.909 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "seconds to cache host facts data", 2024-05-22T23:12:10.909 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.909 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.909 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.909 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.909 INFO:teuthology.orchestra.run.smithi084.stdout: "grafana_dashboards_path": { 2024-05-22T23:12:10.910 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "grafana_dashboards_path", 2024-05-22T23:12:10.910 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.910 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.910 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.910 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-22T23:12:10.910 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.910 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.910 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.911 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-22T23:12:10.911 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.911 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.911 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.911 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.911 INFO:teuthology.orchestra.run.smithi084.stdout: "host_check_interval": { 2024-05-22T23:12:10.911 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "host_check_interval", 2024-05-22T23:12:10.911 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.912 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.912 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.912 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-05-22T23:12:10.912 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.912 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.912 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.912 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to perform a host check", 2024-05-22T23:12:10.912 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.912 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.913 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.913 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.913 INFO:teuthology.orchestra.run.smithi084.stdout: "hw_monitoring": { 2024-05-22T23:12:10.913 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "hw_monitoring", 2024-05-22T23:12:10.913 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.913 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.913 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.913 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.913 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.914 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.914 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.914 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-22T23:12:10.914 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.914 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.914 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.914 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.915 INFO:teuthology.orchestra.run.smithi084.stdout: "inventory_list_all": { 2024-05-22T23:12:10.915 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "inventory_list_all", 2024-05-22T23:12:10.915 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.915 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.915 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.915 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.915 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.915 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.915 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.916 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-22T23:12:10.916 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.916 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.916 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.916 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.916 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.916 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.916 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.917 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.917 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.917 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.917 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.917 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.917 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.917 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.917 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.917 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.918 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.918 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.918 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.918 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.918 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.918 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.918 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.919 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.919 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.919 INFO:teuthology.orchestra.run.smithi084.stdout: "log_refresh_metadata": { 2024-05-22T23:12:10.919 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_refresh_metadata", 2024-05-22T23:12:10.919 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.919 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.919 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.919 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.920 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.920 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.920 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.920 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-22T23:12:10.920 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.920 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.920 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.920 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.921 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.921 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.921 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.921 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.921 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.921 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.921 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.921 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.921 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.922 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-22T23:12:10.922 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.922 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.922 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.922 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.922 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.922 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.923 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.923 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.923 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.923 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.923 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.923 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.923 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.923 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.924 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.924 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.924 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.924 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.924 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.924 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.924 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.924 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.925 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.925 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.925 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.925 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.925 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.925 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.925 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.925 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.926 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.926 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.926 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.926 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.926 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.926 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.926 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.926 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.927 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.927 INFO:teuthology.orchestra.run.smithi084.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-22T23:12:10.927 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-22T23:12:10.927 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.927 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.927 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.927 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.927 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.928 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.928 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.928 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-22T23:12:10.928 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.928 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.928 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.928 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.928 INFO:teuthology.orchestra.run.smithi084.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-22T23:12:10.929 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-22T23:12:10.929 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.929 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.929 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.929 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "*", 2024-05-22T23:12:10.929 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.929 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.929 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.930 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-22T23:12:10.930 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.930 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.930 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.930 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.930 INFO:teuthology.orchestra.run.smithi084.stdout: "max_count_per_host": { 2024-05-22T23:12:10.930 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_count_per_host", 2024-05-22T23:12:10.930 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.931 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.931 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.931 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-05-22T23:12:10.931 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.931 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.931 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.931 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "max number of daemons per service per host", 2024-05-22T23:12:10.932 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.932 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.932 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.932 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.932 INFO:teuthology.orchestra.run.smithi084.stdout: "max_osd_draining_count": { 2024-05-22T23:12:10.932 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_osd_draining_count", 2024-05-22T23:12:10.932 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.932 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.933 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.933 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-05-22T23:12:10.933 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.933 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.933 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.933 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-22T23:12:10.933 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.933 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.934 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.934 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.934 INFO:teuthology.orchestra.run.smithi084.stdout: "migration_current": { 2024-05-22T23:12:10.934 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "migration_current", 2024-05-22T23:12:10.934 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.934 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.934 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.934 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.935 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.935 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.935 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.935 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "internal - do not modify", 2024-05-22T23:12:10.935 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.935 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.935 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.935 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.936 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": { 2024-05-22T23:12:10.936 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mode", 2024-05-22T23:12:10.936 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.936 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.936 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.936 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "root", 2024-05-22T23:12:10.936 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.936 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.937 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.937 INFO:teuthology.orchestra.run.smithi084.stdout: "cephadm-package", 2024-05-22T23:12:10.937 INFO:teuthology.orchestra.run.smithi084.stdout: "root" 2024-05-22T23:12:10.937 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.937 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "mode for remote execution of cephadm", 2024-05-22T23:12:10.937 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.937 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.937 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.937 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.938 INFO:teuthology.orchestra.run.smithi084.stdout: "oob_default_addr": { 2024-05-22T23:12:10.938 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "oob_default_addr", 2024-05-22T23:12:10.938 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.938 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.938 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.938 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "169.254.1.1", 2024-05-22T23:12:10.938 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.939 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.939 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.939 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-22T23:12:10.939 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.939 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.939 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.939 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.939 INFO:teuthology.orchestra.run.smithi084.stdout: "prometheus_alerts_path": { 2024-05-22T23:12:10.940 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "prometheus_alerts_path", 2024-05-22T23:12:10.940 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.940 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.940 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.940 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-22T23:12:10.940 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.940 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.940 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.941 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-22T23:12:10.941 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.941 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.941 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.941 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.941 INFO:teuthology.orchestra.run.smithi084.stdout: "registry_insecure": { 2024-05-22T23:12:10.941 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "registry_insecure", 2024-05-22T23:12:10.941 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.942 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.942 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.942 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.942 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.942 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.942 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.942 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-22T23:12:10.942 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.943 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.943 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.943 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.943 INFO:teuthology.orchestra.run.smithi084.stdout: "registry_password": { 2024-05-22T23:12:10.943 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "registry_password", 2024-05-22T23:12:10.943 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.943 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.943 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.944 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.944 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.944 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.944 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.944 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-22T23:12:10.944 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.944 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.944 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.944 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.945 INFO:teuthology.orchestra.run.smithi084.stdout: "registry_url": { 2024-05-22T23:12:10.945 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "registry_url", 2024-05-22T23:12:10.945 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.945 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.945 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.945 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.945 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.945 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.946 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.946 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-22T23:12:10.946 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.946 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.946 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.946 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.946 INFO:teuthology.orchestra.run.smithi084.stdout: "registry_username": { 2024-05-22T23:12:10.947 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "registry_username", 2024-05-22T23:12:10.947 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.947 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.947 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.947 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.947 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.947 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.947 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.947 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-22T23:12:10.948 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.948 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.948 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.948 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.948 INFO:teuthology.orchestra.run.smithi084.stdout: "secure_monitoring_stack": { 2024-05-22T23:12:10.948 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "secure_monitoring_stack", 2024-05-22T23:12:10.948 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.948 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.948 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.949 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.949 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.949 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.949 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.949 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-22T23:12:10.949 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.949 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.949 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.950 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.950 INFO:teuthology.orchestra.run.smithi084.stdout: "service_discovery_port": { 2024-05-22T23:12:10.950 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "service_discovery_port", 2024-05-22T23:12:10.950 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.950 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.950 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.950 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8765", 2024-05-22T23:12:10.950 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.951 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.951 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.951 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "cephadm service discovery port", 2024-05-22T23:12:10.951 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.951 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.951 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.951 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.951 INFO:teuthology.orchestra.run.smithi084.stdout: "ssh_config_file": { 2024-05-22T23:12:10.951 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ssh_config_file", 2024-05-22T23:12:10.952 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.952 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.952 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.952 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.952 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.952 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.952 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.952 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-22T23:12:10.953 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.953 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.953 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.953 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.953 INFO:teuthology.orchestra.run.smithi084.stdout: "use_agent": { 2024-05-22T23:12:10.953 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "use_agent", 2024-05-22T23:12:10.953 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.953 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.953 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.954 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.954 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.954 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.954 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.954 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-22T23:12:10.954 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.954 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.954 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.955 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.955 INFO:teuthology.orchestra.run.smithi084.stdout: "use_repo_digest": { 2024-05-22T23:12:10.955 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "use_repo_digest", 2024-05-22T23:12:10.955 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.955 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.955 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.955 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.955 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.956 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.956 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.956 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-22T23:12:10.956 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.956 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.956 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.956 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.956 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_on_failed_host_check": { 2024-05-22T23:12:10.956 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_on_failed_host_check", 2024-05-22T23:12:10.957 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.957 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.957 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.957 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.957 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.957 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.957 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.957 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "raise a health warning if the host check fails", 2024-05-22T23:12:10.957 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.958 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.958 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.958 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.958 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_on_stray_daemons": { 2024-05-22T23:12:10.958 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_on_stray_daemons", 2024-05-22T23:12:10.958 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.958 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.958 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.959 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.959 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.959 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.959 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.959 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-22T23:12:10.959 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.959 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.959 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.960 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.960 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_on_stray_hosts": { 2024-05-22T23:12:10.960 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_on_stray_hosts", 2024-05-22T23:12:10.960 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.960 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.960 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.960 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.960 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.961 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.961 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.961 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-22T23:12:10.961 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.961 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.961 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.961 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.961 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.962 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.962 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.962 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crash", 2024-05-22T23:12:10.962 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.962 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.962 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.962 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:10.963 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:10.963 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.963 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.963 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.963 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.963 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.963 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.963 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.964 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.964 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.964 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.964 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.964 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.964 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.964 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.965 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.965 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.965 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.965 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.965 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.965 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:10.965 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:10.965 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.966 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.966 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.966 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.966 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.966 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.966 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.966 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.966 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.967 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.967 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.967 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.967 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:10.967 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:10.967 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.967 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.968 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.968 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:10.968 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.968 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.968 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:10.968 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:10.968 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:10.968 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:10.969 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:10.969 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:10.969 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:10.969 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:10.969 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.969 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.969 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.969 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.970 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.970 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:10.970 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:10.970 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.970 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.970 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.970 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.970 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.971 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.971 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.971 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.971 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.971 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.971 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.971 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.971 INFO:teuthology.orchestra.run.smithi084.stdout: "retain_interval": { 2024-05-22T23:12:10.972 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "retain_interval", 2024-05-22T23:12:10.972 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.972 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.972 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.972 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "31536000", 2024-05-22T23:12:10.972 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.972 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.972 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.973 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-22T23:12:10.973 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.973 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.973 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.973 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.973 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_recent_interval": { 2024-05-22T23:12:10.973 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_recent_interval", 2024-05-22T23:12:10.973 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:10.974 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.974 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:10.974 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "1209600", 2024-05-22T23:12:10.974 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.974 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.974 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.974 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-22T23:12:10.974 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.975 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.975 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.975 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.975 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:10.975 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.975 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:10.975 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "dashboard", 2024-05-22T23:12:10.975 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:10.976 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:10.976 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:10.976 INFO:teuthology.orchestra.run.smithi084.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-22T23:12:10.976 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-22T23:12:10.976 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:10.976 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.976 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.977 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-05-22T23:12:10.977 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.977 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.977 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.977 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.977 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.977 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.977 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.977 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.978 INFO:teuthology.orchestra.run.smithi084.stdout: "ALERTMANAGER_API_HOST": { 2024-05-22T23:12:10.978 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-22T23:12:10.978 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.978 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.978 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.978 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.978 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.979 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.979 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.979 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.979 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.979 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.979 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.979 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.980 INFO:teuthology.orchestra.run.smithi084.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-22T23:12:10.980 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-22T23:12:10.980 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.980 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.980 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.980 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.980 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.981 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.981 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.981 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.981 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.981 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.981 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.981 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.981 INFO:teuthology.orchestra.run.smithi084.stdout: "AUDIT_API_ENABLED": { 2024-05-22T23:12:10.982 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "AUDIT_API_ENABLED", 2024-05-22T23:12:10.982 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.982 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.982 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.982 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:10.982 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.982 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.982 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.983 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.983 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.983 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.983 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.983 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.983 INFO:teuthology.orchestra.run.smithi084.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-22T23:12:10.983 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-22T23:12:10.983 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.984 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.984 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.984 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.984 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.984 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.984 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.984 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.984 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.985 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.985 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.985 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.985 INFO:teuthology.orchestra.run.smithi084.stdout: "ENABLE_BROWSABLE_API": { 2024-05-22T23:12:10.985 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-22T23:12:10.985 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.985 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.985 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.986 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.986 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.986 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.986 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.986 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.986 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.986 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.986 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.987 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.987 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-22T23:12:10.987 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-22T23:12:10.987 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.987 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.987 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.987 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.987 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.988 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.988 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.988 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.988 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.988 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.988 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.988 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.988 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-22T23:12:10.989 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-22T23:12:10.989 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.989 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.989 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.989 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.989 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.989 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.989 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.990 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.990 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.990 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.990 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.990 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.990 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-22T23:12:10.990 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-22T23:12:10.990 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.991 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.991 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.991 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.991 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.991 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.991 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.991 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.991 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.992 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.992 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.992 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.992 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-22T23:12:10.992 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-22T23:12:10.992 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.992 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.992 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.992 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.993 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.993 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.993 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.993 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.993 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.993 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.993 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.993 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.994 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-22T23:12:10.994 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-22T23:12:10.994 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.994 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.994 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.994 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.994 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.995 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.995 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.995 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.995 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.995 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.995 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.995 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.995 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-22T23:12:10.995 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-22T23:12:10.996 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.996 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.996 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.996 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.996 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.996 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.996 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.996 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.997 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.997 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.997 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.997 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.997 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-22T23:12:10.997 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-22T23:12:10.997 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:10.997 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.997 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.998 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:10.998 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.998 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:10.998 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:10.998 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:10.998 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:10.998 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:10.998 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:10.999 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:10.999 INFO:teuthology.orchestra.run.smithi084.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-22T23:12:10.999 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-22T23:12:10.999 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:10.999 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:10.999 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:10.999 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:10.999 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:10.999 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.000 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.000 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.000 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.000 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.000 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.000 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.000 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_API_PASSWORD": { 2024-05-22T23:12:11.000 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-22T23:12:11.000 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.001 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.001 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.001 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "admin", 2024-05-22T23:12:11.001 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.001 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.001 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.001 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.001 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.002 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.002 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.002 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.002 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-22T23:12:11.002 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-22T23:12:11.002 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.002 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.002 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.003 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:11.003 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.003 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.003 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.003 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.003 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.003 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.003 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.004 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.004 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_API_URL": { 2024-05-22T23:12:11.004 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_API_URL", 2024-05-22T23:12:11.004 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.004 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.004 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.004 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.004 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.005 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.005 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.005 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.005 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.005 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.005 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.005 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.006 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_API_USERNAME": { 2024-05-22T23:12:11.006 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-22T23:12:11.006 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.006 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.006 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.006 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "admin", 2024-05-22T23:12:11.006 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.006 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.007 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.007 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.007 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.007 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.007 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.007 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.007 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-22T23:12:11.007 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-22T23:12:11.008 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.008 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.008 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.008 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.008 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.008 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.008 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.008 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.009 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.009 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.009 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.009 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.009 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-22T23:12:11.009 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-22T23:12:11.009 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.009 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.010 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.010 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.010 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.010 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.010 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.010 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.010 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.010 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.011 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.011 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.011 INFO:teuthology.orchestra.run.smithi084.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-22T23:12:11.011 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-22T23:12:11.011 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.011 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.011 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.012 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:11.012 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.012 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.012 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.012 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.012 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.012 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.012 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.013 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.013 INFO:teuthology.orchestra.run.smithi084.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-22T23:12:11.013 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-22T23:12:11.013 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.013 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.013 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.013 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.013 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.014 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.014 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.014 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.014 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.014 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.014 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.014 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.014 INFO:teuthology.orchestra.run.smithi084.stdout: "PROMETHEUS_API_HOST": { 2024-05-22T23:12:11.015 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-22T23:12:11.015 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.015 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.015 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.015 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.015 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.015 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.015 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.016 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.016 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.016 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.016 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.016 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.016 INFO:teuthology.orchestra.run.smithi084.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-22T23:12:11.016 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-22T23:12:11.016 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.017 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.017 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.017 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:11.017 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.017 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.017 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.017 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.017 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.018 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.018 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.018 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.018 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-22T23:12:11.018 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-22T23:12:11.018 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.018 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.018 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.019 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.019 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.019 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.019 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.019 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.019 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.019 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.019 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.020 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.020 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-22T23:12:11.020 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-22T23:12:11.020 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.020 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.020 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.020 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.020 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.021 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.021 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.021 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.021 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.021 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.021 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.021 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.021 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-22T23:12:11.022 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-22T23:12:11.022 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.022 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.022 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.022 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:11.022 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.022 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.022 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.023 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.023 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.023 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.023 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.023 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.023 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-22T23:12:11.023 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-22T23:12:11.023 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.024 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.024 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.024 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:11.024 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.024 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.024 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.024 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.024 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.025 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.025 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.025 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.025 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-22T23:12:11.025 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-22T23:12:11.025 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.025 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.025 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.026 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.026 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.026 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.026 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.026 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.026 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.026 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.027 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.027 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.027 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-22T23:12:11.027 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-22T23:12:11.027 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.027 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.027 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.027 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.027 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.028 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.028 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.028 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.028 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.028 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.028 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.028 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.029 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-22T23:12:11.029 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-22T23:12:11.029 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.029 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.029 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.029 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.029 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.029 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.030 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.030 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.030 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.030 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.030 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.030 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.030 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_ENABLED": { 2024-05-22T23:12:11.030 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-22T23:12:11.031 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.031 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.031 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.031 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:11.031 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.031 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.031 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.031 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.032 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.032 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.032 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.032 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.032 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-22T23:12:11.032 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-22T23:12:11.032 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.032 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.033 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.033 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-22T23:12:11.033 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.033 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.033 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.033 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.033 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.033 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.034 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.034 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.034 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-22T23:12:11.034 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-22T23:12:11.034 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.034 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.034 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.035 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-05-22T23:12:11.035 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.035 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.035 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.035 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.035 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.035 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.035 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.036 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.036 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-22T23:12:11.036 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-22T23:12:11.036 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.036 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.036 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.036 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8", 2024-05-22T23:12:11.036 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.037 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.037 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.037 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.037 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.037 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.037 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.037 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.037 INFO:teuthology.orchestra.run.smithi084.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-22T23:12:11.038 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-22T23:12:11.038 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.038 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.038 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.038 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "45", 2024-05-22T23:12:11.038 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.038 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.038 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.039 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.039 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.039 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.039 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.039 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.039 INFO:teuthology.orchestra.run.smithi084.stdout: "RGW_API_ACCESS_KEY": { 2024-05-22T23:12:11.039 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-22T23:12:11.039 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.040 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.040 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.040 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.040 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.040 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.040 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.040 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.040 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.041 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.041 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.041 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.041 INFO:teuthology.orchestra.run.smithi084.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-22T23:12:11.041 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-22T23:12:11.041 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.041 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.042 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.042 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "admin", 2024-05-22T23:12:11.042 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.042 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.042 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.042 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.042 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.043 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.043 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.043 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.043 INFO:teuthology.orchestra.run.smithi084.stdout: "RGW_API_SECRET_KEY": { 2024-05-22T23:12:11.043 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-22T23:12:11.043 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.043 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.043 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.043 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.044 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.044 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.044 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.044 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.045 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.045 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.045 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.045 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.046 INFO:teuthology.orchestra.run.smithi084.stdout: "RGW_API_SSL_VERIFY": { 2024-05-22T23:12:11.046 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-22T23:12:11.046 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.046 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.046 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.046 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:11.046 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.047 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.047 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.047 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.047 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.047 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.047 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.047 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.047 INFO:teuthology.orchestra.run.smithi084.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-22T23:12:11.048 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-22T23:12:11.048 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.048 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.048 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.048 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-05-22T23:12:11.048 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.048 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.048 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.049 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.049 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.049 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.049 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.049 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.049 INFO:teuthology.orchestra.run.smithi084.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-22T23:12:11.049 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-22T23:12:11.049 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.050 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.050 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.050 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-05-22T23:12:11.050 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.050 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.050 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.050 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.051 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.051 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.051 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.051 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.051 INFO:teuthology.orchestra.run.smithi084.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-22T23:12:11.051 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-22T23:12:11.051 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.051 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.052 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.052 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5", 2024-05-22T23:12:11.052 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.052 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.052 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.052 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.052 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.053 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.053 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.053 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.053 INFO:teuthology.orchestra.run.smithi084.stdout: "cross_origin_url": { 2024-05-22T23:12:11.053 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cross_origin_url", 2024-05-22T23:12:11.053 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.053 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.053 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.054 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.054 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.054 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.054 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.054 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.054 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.054 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.054 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.055 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.055 INFO:teuthology.orchestra.run.smithi084.stdout: "crt_file": { 2024-05-22T23:12:11.055 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crt_file", 2024-05-22T23:12:11.055 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.055 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.055 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.055 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.055 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.056 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.056 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.056 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.056 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.056 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.056 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.056 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.057 INFO:teuthology.orchestra.run.smithi084.stdout: "debug": { 2024-05-22T23:12:11.057 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "debug", 2024-05-22T23:12:11.057 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.057 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.057 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.057 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.057 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.057 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.058 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.058 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Enable/disable debug options", 2024-05-22T23:12:11.058 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.058 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.058 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.058 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.058 INFO:teuthology.orchestra.run.smithi084.stdout: "jwt_token_ttl": { 2024-05-22T23:12:11.059 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "jwt_token_ttl", 2024-05-22T23:12:11.059 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.059 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.059 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.059 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "28800", 2024-05-22T23:12:11.059 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.059 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.059 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.060 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.060 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.060 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.060 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.060 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.060 INFO:teuthology.orchestra.run.smithi084.stdout: "key_file": { 2024-05-22T23:12:11.060 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "key_file", 2024-05-22T23:12:11.060 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.061 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.061 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.061 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.061 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.061 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.061 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.061 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.062 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.062 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.062 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.062 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.062 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.062 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.062 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.062 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.063 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.063 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.063 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.063 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.063 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.063 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.063 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.063 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.063 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.064 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.064 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.064 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.064 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.064 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.064 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.064 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.065 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.065 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.065 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.065 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.065 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.065 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.065 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.066 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.066 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.066 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.066 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.066 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.066 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.066 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.067 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.067 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.067 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.067 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.067 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.067 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.067 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.067 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.068 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.068 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.068 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.068 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.068 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.068 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.068 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.068 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.069 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.069 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.069 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.069 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.069 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.069 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.069 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.070 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.070 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.070 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.070 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.070 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.070 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.070 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.070 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.071 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.071 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.071 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.071 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.071 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.071 INFO:teuthology.orchestra.run.smithi084.stdout: "motd": { 2024-05-22T23:12:11.071 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "motd", 2024-05-22T23:12:11.071 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.072 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.072 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.072 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.072 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.072 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.072 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.072 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "The message of the day", 2024-05-22T23:12:11.073 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.073 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.073 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.073 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.073 INFO:teuthology.orchestra.run.smithi084.stdout: "redirect_resolve_ip_addr": { 2024-05-22T23:12:11.073 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "redirect_resolve_ip_addr", 2024-05-22T23:12:11.073 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.073 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.074 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.074 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.074 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.074 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.074 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.074 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.074 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.074 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.075 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.075 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.075 INFO:teuthology.orchestra.run.smithi084.stdout: "server_addr": { 2024-05-22T23:12:11.075 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_addr", 2024-05-22T23:12:11.075 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.075 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.075 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.075 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "::", 2024-05-22T23:12:11.075 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.076 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.076 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.076 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.076 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.076 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.076 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.076 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.076 INFO:teuthology.orchestra.run.smithi084.stdout: "server_port": { 2024-05-22T23:12:11.077 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_port", 2024-05-22T23:12:11.077 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.077 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.077 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.077 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8080", 2024-05-22T23:12:11.077 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.077 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.077 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.078 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.078 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.078 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.078 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.078 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.078 INFO:teuthology.orchestra.run.smithi084.stdout: "ssl": { 2024-05-22T23:12:11.078 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ssl", 2024-05-22T23:12:11.078 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.079 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.079 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.079 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:11.079 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.079 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.079 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.079 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.079 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.079 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.080 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.080 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.080 INFO:teuthology.orchestra.run.smithi084.stdout: "ssl_server_port": { 2024-05-22T23:12:11.080 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ssl_server_port", 2024-05-22T23:12:11.080 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.080 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.080 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.080 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8443", 2024-05-22T23:12:11.080 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.081 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.081 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.081 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.081 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.081 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.081 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.081 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.081 INFO:teuthology.orchestra.run.smithi084.stdout: "standby_behaviour": { 2024-05-22T23:12:11.082 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "standby_behaviour", 2024-05-22T23:12:11.082 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.082 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.082 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.082 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "redirect", 2024-05-22T23:12:11.082 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.082 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.082 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.083 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.083 INFO:teuthology.orchestra.run.smithi084.stdout: "redirect" 2024-05-22T23:12:11.083 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.083 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.083 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.083 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.083 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.084 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.084 INFO:teuthology.orchestra.run.smithi084.stdout: "standby_error_status_code": { 2024-05-22T23:12:11.084 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "standby_error_status_code", 2024-05-22T23:12:11.084 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.084 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.084 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.084 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "500", 2024-05-22T23:12:11.084 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "400", 2024-05-22T23:12:11.085 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "599", 2024-05-22T23:12:11.085 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.085 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.085 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.085 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.085 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.085 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.085 INFO:teuthology.orchestra.run.smithi084.stdout: "url_prefix": { 2024-05-22T23:12:11.086 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "url_prefix", 2024-05-22T23:12:11.086 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.086 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.086 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.086 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.086 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.086 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.087 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.087 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.087 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.087 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.087 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.087 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.087 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.088 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.088 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.088 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "devicehealth", 2024-05-22T23:12:11.088 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.088 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.088 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.088 INFO:teuthology.orchestra.run.smithi084.stdout: "enable_monitoring": { 2024-05-22T23:12:11.088 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "enable_monitoring", 2024-05-22T23:12:11.089 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.089 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.089 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.089 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:11.089 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.089 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.089 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.090 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "monitor device health metrics", 2024-05-22T23:12:11.090 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.090 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.090 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.090 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.090 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.090 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.090 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.091 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.091 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.091 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.091 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.091 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.091 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.091 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.091 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.092 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.092 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.092 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.092 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.092 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.092 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.092 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.093 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.093 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.093 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.093 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.093 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.093 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.093 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.093 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.094 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.094 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.094 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.094 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.094 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.094 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.094 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.094 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.095 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.095 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.095 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.095 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.095 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.095 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.095 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.095 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.096 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.096 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.096 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.096 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.096 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.096 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.096 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.096 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.097 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.097 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.097 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.097 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.097 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.097 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.097 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.097 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.098 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.098 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.098 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.098 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.098 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.098 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.098 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.098 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.099 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.099 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.099 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.099 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.099 INFO:teuthology.orchestra.run.smithi084.stdout: "mark_out_threshold": { 2024-05-22T23:12:11.099 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mark_out_threshold", 2024-05-22T23:12:11.099 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:11.099 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.099 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.100 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "2419200", 2024-05-22T23:12:11.100 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.100 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.100 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.100 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-22T23:12:11.100 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.100 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.100 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.100 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.101 INFO:teuthology.orchestra.run.smithi084.stdout: "pool_name": { 2024-05-22T23:12:11.101 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "pool_name", 2024-05-22T23:12:11.101 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.101 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.101 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.101 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "device_health_metrics", 2024-05-22T23:12:11.101 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.101 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.102 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.102 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-22T23:12:11.102 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.102 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.102 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.102 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.102 INFO:teuthology.orchestra.run.smithi084.stdout: "retention_period": { 2024-05-22T23:12:11.102 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "retention_period", 2024-05-22T23:12:11.103 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:11.103 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.103 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.103 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "15552000", 2024-05-22T23:12:11.103 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.103 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.103 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.103 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how long to retain device health metrics", 2024-05-22T23:12:11.104 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.104 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.104 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.104 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.104 INFO:teuthology.orchestra.run.smithi084.stdout: "scrape_frequency": { 2024-05-22T23:12:11.104 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "scrape_frequency", 2024-05-22T23:12:11.104 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:11.104 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.105 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.105 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "86400", 2024-05-22T23:12:11.105 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.105 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.105 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.105 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-22T23:12:11.105 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.105 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.106 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.106 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.106 INFO:teuthology.orchestra.run.smithi084.stdout: "self_heal": { 2024-05-22T23:12:11.106 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "self_heal", 2024-05-22T23:12:11.106 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.106 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.106 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.106 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:11.107 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.107 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.107 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.107 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-22T23:12:11.107 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.107 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.107 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.107 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.108 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-05-22T23:12:11.108 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-05-22T23:12:11.108 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:11.108 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.108 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.108 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-05-22T23:12:11.108 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.109 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.109 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.109 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to wake up and check device health", 2024-05-22T23:12:11.109 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.109 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.109 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.109 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.109 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_threshold": { 2024-05-22T23:12:11.110 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_threshold", 2024-05-22T23:12:11.110 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:11.110 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.110 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.110 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "7257600", 2024-05-22T23:12:11.110 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.110 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.110 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.111 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-22T23:12:11.111 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.111 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.111 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.111 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.111 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.111 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.111 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.112 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "diskprediction_local", 2024-05-22T23:12:11.112 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.112 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.112 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.112 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.112 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.112 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.112 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.113 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.113 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.113 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.113 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.113 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.113 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.113 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.113 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.114 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.114 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.114 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.114 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.114 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.114 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.114 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.114 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.115 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.115 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.115 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.115 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.115 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.115 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.115 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.115 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.116 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.116 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.116 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.116 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.116 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.116 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.116 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.116 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.117 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.117 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.117 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.117 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.117 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.117 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.117 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.117 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.118 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.118 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.118 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.118 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.118 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.118 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.118 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.119 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.119 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.119 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.119 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.119 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.119 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.119 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.119 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.120 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.120 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.120 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.120 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.120 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.120 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.120 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.120 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.121 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.121 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.121 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.121 INFO:teuthology.orchestra.run.smithi084.stdout: "predict_interval": { 2024-05-22T23:12:11.121 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "predict_interval", 2024-05-22T23:12:11.121 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.121 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.122 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.122 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "86400", 2024-05-22T23:12:11.122 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.122 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.122 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.122 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.122 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.123 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.123 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.123 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.123 INFO:teuthology.orchestra.run.smithi084.stdout: "predictor_model": { 2024-05-22T23:12:11.123 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "predictor_model", 2024-05-22T23:12:11.123 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.123 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.123 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.124 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "prophetstor", 2024-05-22T23:12:11.124 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.124 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.124 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.124 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.124 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.124 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.124 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.125 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.125 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-05-22T23:12:11.125 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-05-22T23:12:11.125 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.125 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.125 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.125 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-05-22T23:12:11.126 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.126 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.126 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.126 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.126 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.126 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.126 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.126 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.127 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.127 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.127 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.127 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "influx", 2024-05-22T23:12:11.127 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": false, 2024-05-22T23:12:11.127 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "influxdb python module not found", 2024-05-22T23:12:11.127 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.127 INFO:teuthology.orchestra.run.smithi084.stdout: "batch_size": { 2024-05-22T23:12:11.128 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "batch_size", 2024-05-22T23:12:11.128 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.128 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.128 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.128 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5000", 2024-05-22T23:12:11.128 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.128 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.128 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.129 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-22T23:12:11.129 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.129 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.129 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.129 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.129 INFO:teuthology.orchestra.run.smithi084.stdout: "database": { 2024-05-22T23:12:11.129 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "database", 2024-05-22T23:12:11.129 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.130 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.130 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.130 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "ceph", 2024-05-22T23:12:11.130 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.130 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.130 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.130 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-22T23:12:11.130 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.131 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.131 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.131 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.131 INFO:teuthology.orchestra.run.smithi084.stdout: "hostname": { 2024-05-22T23:12:11.131 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "hostname", 2024-05-22T23:12:11.131 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.131 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.132 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.132 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.132 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.132 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.132 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.132 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "InfluxDB server hostname", 2024-05-22T23:12:11.132 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.132 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.133 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.133 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.133 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-05-22T23:12:11.133 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-05-22T23:12:11.133 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:11.133 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.133 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.133 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "30", 2024-05-22T23:12:11.134 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "5", 2024-05-22T23:12:11.134 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.134 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.134 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-22T23:12:11.134 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.134 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.134 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.135 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.135 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.135 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.135 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.135 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.135 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.135 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.135 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.136 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.136 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.136 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.136 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.136 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.136 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.136 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.136 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.137 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.137 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.137 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.137 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.137 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.137 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.137 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.137 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.138 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.138 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.138 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.138 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.138 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.138 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.138 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.138 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.139 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.139 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.139 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.139 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.139 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.139 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.139 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.139 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.140 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.140 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.140 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.140 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.140 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.140 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.140 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.140 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.140 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.141 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.141 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.141 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.141 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.141 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.141 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.141 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.141 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.142 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.142 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.142 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.142 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.142 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.142 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.142 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.142 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.143 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.143 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.143 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.143 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.143 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.143 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.143 INFO:teuthology.orchestra.run.smithi084.stdout: "password": { 2024-05-22T23:12:11.143 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "password", 2024-05-22T23:12:11.143 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.144 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.144 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.144 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.144 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.144 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.144 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.144 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "password of InfluxDB server user", 2024-05-22T23:12:11.144 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.145 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.145 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.145 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.145 INFO:teuthology.orchestra.run.smithi084.stdout: "port": { 2024-05-22T23:12:11.145 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "port", 2024-05-22T23:12:11.145 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.145 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.145 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.145 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8086", 2024-05-22T23:12:11.146 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.146 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.146 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.146 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "InfluxDB server port", 2024-05-22T23:12:11.146 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.146 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.146 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.146 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.147 INFO:teuthology.orchestra.run.smithi084.stdout: "ssl": { 2024-05-22T23:12:11.147 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ssl", 2024-05-22T23:12:11.147 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.147 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.147 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.147 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "false", 2024-05-22T23:12:11.147 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.147 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.147 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.148 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-22T23:12:11.148 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.148 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.148 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.148 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.148 INFO:teuthology.orchestra.run.smithi084.stdout: "threads": { 2024-05-22T23:12:11.148 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "threads", 2024-05-22T23:12:11.148 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.148 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.149 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.149 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5", 2024-05-22T23:12:11.149 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1", 2024-05-22T23:12:11.149 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "32", 2024-05-22T23:12:11.149 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.149 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-22T23:12:11.149 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.149 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.150 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.150 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.150 INFO:teuthology.orchestra.run.smithi084.stdout: "username": { 2024-05-22T23:12:11.150 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "username", 2024-05-22T23:12:11.150 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.150 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.150 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.151 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.151 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.151 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.151 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.151 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "username of InfluxDB server user", 2024-05-22T23:12:11.151 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.151 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.151 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.151 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.152 INFO:teuthology.orchestra.run.smithi084.stdout: "verify_ssl": { 2024-05-22T23:12:11.152 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "verify_ssl", 2024-05-22T23:12:11.152 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.152 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.152 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.152 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "true", 2024-05-22T23:12:11.152 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.152 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.152 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.153 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-22T23:12:11.153 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.153 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.153 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.153 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.153 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.153 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.153 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.154 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "insights", 2024-05-22T23:12:11.154 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.154 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.154 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.154 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.154 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.154 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.154 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.155 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.155 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.155 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.155 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.155 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.155 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.155 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.155 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.156 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.156 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.156 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.156 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.156 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.156 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.156 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.156 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.157 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.157 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.157 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.157 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.157 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.157 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.157 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.157 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.158 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.158 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.158 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.158 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.158 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.158 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.158 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.159 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.159 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.159 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.159 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.159 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.159 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.159 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.159 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.160 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.160 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.160 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.160 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.160 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.160 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.160 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.161 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.161 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.161 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.161 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.161 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.161 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.161 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.161 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.162 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.162 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.162 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.162 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.162 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.162 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.162 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.162 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.163 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.163 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.163 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.163 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.163 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.163 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.163 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.163 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "iostat", 2024-05-22T23:12:11.164 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.164 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.164 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.164 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.164 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.164 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.164 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.164 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.164 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.165 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.165 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.165 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.165 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.165 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.165 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.165 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.165 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.166 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.166 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.166 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.166 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.166 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.166 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.166 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.166 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.167 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.167 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.167 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.167 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.167 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.167 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.167 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.167 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.168 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.168 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.168 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.168 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.168 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.168 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.168 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.169 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.169 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.169 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.169 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.169 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.169 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.169 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.169 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.170 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.170 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.170 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.170 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.170 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.170 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.170 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.170 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.171 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.171 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.171 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.171 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.171 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.171 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.171 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.171 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.172 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.172 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.172 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.172 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.172 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.172 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.172 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.173 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.173 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.173 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.173 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.173 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.173 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "k8sevents", 2024-05-22T23:12:11.173 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.173 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.174 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.174 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_event_retention_days": { 2024-05-22T23:12:11.174 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ceph_event_retention_days", 2024-05-22T23:12:11.174 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.174 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.174 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.174 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "7", 2024-05-22T23:12:11.174 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.175 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.175 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.175 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-22T23:12:11.175 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.175 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.175 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.175 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.175 INFO:teuthology.orchestra.run.smithi084.stdout: "config_check_secs": { 2024-05-22T23:12:11.176 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "config_check_secs", 2024-05-22T23:12:11.176 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.176 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.176 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.176 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-05-22T23:12:11.176 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "10", 2024-05-22T23:12:11.176 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.176 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.177 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-22T23:12:11.177 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.177 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.177 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.177 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.177 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.177 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.178 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.178 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.178 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.178 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.178 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.178 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.178 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.179 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.179 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.179 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.179 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.179 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.179 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.179 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.179 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.180 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.180 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.180 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.180 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.180 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.180 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.180 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.180 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.181 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.181 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.181 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.181 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.181 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.181 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.181 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.181 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.182 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.182 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.182 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.182 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.182 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.182 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.182 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.182 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.183 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.183 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.183 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.183 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.183 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.183 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.183 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.184 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.184 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.184 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.184 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.184 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.184 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.184 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.184 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.185 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.185 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.185 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.185 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.185 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.185 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.185 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.185 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.185 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.186 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.186 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.186 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.186 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.186 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.186 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.186 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.186 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.187 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "localpool", 2024-05-22T23:12:11.187 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.187 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.187 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.187 INFO:teuthology.orchestra.run.smithi084.stdout: "failure_domain": { 2024-05-22T23:12:11.187 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "failure_domain", 2024-05-22T23:12:11.187 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.187 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.188 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.188 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "host", 2024-05-22T23:12:11.188 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.188 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.188 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.188 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "failure domain for any created local pool", 2024-05-22T23:12:11.188 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-22T23:12:11.188 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.189 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.189 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.189 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.189 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.189 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.189 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.189 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.189 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.189 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.190 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.190 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.190 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.190 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.190 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.190 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.190 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.191 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.191 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.191 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.191 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.191 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.191 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.191 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.191 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.191 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.192 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.192 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.192 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.192 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.192 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.192 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.192 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.193 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.193 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.193 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.193 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.193 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.193 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.193 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.193 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.193 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.194 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.194 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.194 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.194 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.194 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.194 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.194 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.195 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.195 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.195 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.195 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.195 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.195 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.195 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.195 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.195 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.196 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.196 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.196 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.196 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.196 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.196 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.196 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.196 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.196 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.197 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.197 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.197 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.197 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.197 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.197 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.197 INFO:teuthology.orchestra.run.smithi084.stdout: "min_size": { 2024-05-22T23:12:11.197 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "min_size", 2024-05-22T23:12:11.198 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.198 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.198 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.198 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.198 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.198 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.198 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.198 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "default min_size for any created local pool", 2024-05-22T23:12:11.198 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-22T23:12:11.199 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.199 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.199 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.199 INFO:teuthology.orchestra.run.smithi084.stdout: "num_rep": { 2024-05-22T23:12:11.199 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "num_rep", 2024-05-22T23:12:11.199 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.199 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.199 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.199 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "3", 2024-05-22T23:12:11.200 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.200 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.200 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.200 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "default replica count for any created local pool", 2024-05-22T23:12:11.200 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.200 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.200 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.200 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.201 INFO:teuthology.orchestra.run.smithi084.stdout: "pg_num": { 2024-05-22T23:12:11.201 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "pg_num", 2024-05-22T23:12:11.201 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.201 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.201 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.201 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "128", 2024-05-22T23:12:11.201 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.201 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.201 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.202 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "default pg_num for any created local pool", 2024-05-22T23:12:11.202 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.202 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.202 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.202 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.202 INFO:teuthology.orchestra.run.smithi084.stdout: "prefix": { 2024-05-22T23:12:11.202 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "prefix", 2024-05-22T23:12:11.203 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.203 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.203 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.203 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.203 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.203 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.203 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.203 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "name prefix for any created local pool", 2024-05-22T23:12:11.203 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.204 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.204 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.204 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.204 INFO:teuthology.orchestra.run.smithi084.stdout: "subtree": { 2024-05-22T23:12:11.204 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "subtree", 2024-05-22T23:12:11.204 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.204 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.205 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.205 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "rack", 2024-05-22T23:12:11.205 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.205 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.205 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.205 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-22T23:12:11.205 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-22T23:12:11.205 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.206 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.206 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.206 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.206 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.206 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.206 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mds_autoscaler", 2024-05-22T23:12:11.206 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.206 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.207 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.207 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.207 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.207 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.207 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.207 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.207 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.207 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.207 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.208 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.208 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.208 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.208 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.208 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.208 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.208 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.208 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.209 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.209 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.209 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.209 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.209 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.209 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.209 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.209 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.209 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.210 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.210 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.210 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.210 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.210 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.210 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.210 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.211 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.211 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.211 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.211 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.211 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.211 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.211 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.211 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.212 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.212 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.212 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.212 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.212 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.212 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.212 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.212 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.213 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.213 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.213 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.213 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.213 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.213 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.213 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.213 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.213 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.214 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.214 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.214 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.214 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.214 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.214 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.214 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.214 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.214 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.215 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.215 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.215 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.215 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.215 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.215 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.215 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.215 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mirroring", 2024-05-22T23:12:11.216 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.216 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.216 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.216 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.216 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.216 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.216 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.217 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.217 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.217 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.217 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.217 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.217 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.217 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.217 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.217 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.218 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.218 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.218 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.218 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.218 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.218 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.218 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.219 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.219 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.219 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.219 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.219 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.219 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.219 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.219 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.220 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.220 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.220 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.220 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.220 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.220 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.220 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.221 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.221 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.221 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.221 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.221 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.221 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.221 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.222 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.222 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.222 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.222 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.222 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.222 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.222 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.222 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.223 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.223 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.223 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.223 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.223 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.223 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.223 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.224 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.224 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.224 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.224 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.224 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.224 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.224 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.224 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.225 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.225 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.225 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.225 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.225 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.225 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.225 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.225 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.225 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "nfs", 2024-05-22T23:12:11.226 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.226 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.226 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.226 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.226 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.226 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.226 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.226 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.227 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.227 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.227 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.227 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.227 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.227 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.227 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.227 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.227 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.228 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.228 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.228 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.228 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.228 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.228 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.228 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.228 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.228 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.229 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.229 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.229 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.229 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.229 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.229 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.229 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.230 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.230 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.230 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.230 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.230 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.230 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.230 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.230 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.231 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.231 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.231 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.231 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.231 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.231 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.231 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.232 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.232 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.232 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.232 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.232 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.232 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.232 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.232 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.233 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.233 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.233 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.233 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.233 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.233 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.233 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.233 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.234 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.234 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.234 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.234 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.234 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.234 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.234 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.235 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.235 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.235 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.235 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.235 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.235 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "orchestrator", 2024-05-22T23:12:11.235 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.235 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.236 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.236 INFO:teuthology.orchestra.run.smithi084.stdout: "fail_fs": { 2024-05-22T23:12:11.236 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "fail_fs", 2024-05-22T23:12:11.236 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.236 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.236 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.236 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.236 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.237 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.237 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.237 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-22T23:12:11.237 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.237 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.237 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.237 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.237 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.238 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.238 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.238 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.238 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.238 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.238 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.238 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.238 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.239 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.239 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.239 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.239 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.239 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.239 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.239 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.240 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.240 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.240 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.240 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.240 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.240 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.240 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.240 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.241 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.241 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.241 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.241 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.241 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.241 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.241 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.241 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.242 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.242 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.242 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.242 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.242 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.242 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.242 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.242 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.243 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.243 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.243 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.243 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.243 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.243 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.243 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.243 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.244 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.244 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.244 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.244 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.244 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.244 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.244 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.244 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.245 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.245 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.245 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.245 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.245 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.245 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.245 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.245 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.246 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.246 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.246 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.246 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.246 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.246 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.246 INFO:teuthology.orchestra.run.smithi084.stdout: "orchestrator": { 2024-05-22T23:12:11.246 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "orchestrator", 2024-05-22T23:12:11.247 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.247 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.247 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.247 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.247 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.247 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.247 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.247 INFO:teuthology.orchestra.run.smithi084.stdout: "cephadm", 2024-05-22T23:12:11.248 INFO:teuthology.orchestra.run.smithi084.stdout: "rook", 2024-05-22T23:12:11.248 INFO:teuthology.orchestra.run.smithi084.stdout: "test_orchestrator" 2024-05-22T23:12:11.248 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.248 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Orchestrator backend", 2024-05-22T23:12:11.248 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.248 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.248 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.248 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.249 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.249 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.249 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.249 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "osd_perf_query", 2024-05-22T23:12:11.249 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.249 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.249 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.249 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.250 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.250 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.250 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.250 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.250 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.250 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.250 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.250 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.251 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.251 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.251 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.251 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.251 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.251 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.251 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.251 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.252 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.252 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.252 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.252 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.252 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.252 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.252 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.252 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.253 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.253 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.253 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.253 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.253 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.253 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.253 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.253 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.254 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.254 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.254 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.254 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.254 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.254 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.254 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.255 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.255 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.255 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.255 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.255 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.255 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.255 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.255 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.256 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.256 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.256 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.256 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.256 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.256 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.256 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.256 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.257 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.257 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.257 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.257 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.257 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.257 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.257 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.257 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.257 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.258 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.258 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.258 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.258 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.258 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.258 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.258 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.258 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.259 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "osd_support", 2024-05-22T23:12:11.259 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.259 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.259 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.259 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.259 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.259 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.259 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.260 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.260 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.260 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.260 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.260 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.260 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.260 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.260 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.261 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.261 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.261 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.261 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.261 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.261 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.261 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.261 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.262 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.262 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.262 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.262 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.262 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.262 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.262 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.263 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.263 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.263 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.263 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.263 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.263 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.263 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.263 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.264 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.264 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.264 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.264 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.264 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.264 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.264 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.265 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.265 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.265 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.265 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.265 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.265 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.265 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.265 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.266 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.266 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.266 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.266 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.266 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.266 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.266 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.266 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.267 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.267 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.267 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.267 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.267 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.267 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.267 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.267 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.268 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.268 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.268 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.268 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.268 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.268 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.268 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.268 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "pg_autoscaler", 2024-05-22T23:12:11.269 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.269 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.269 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.269 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.269 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.269 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.269 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.269 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.269 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.270 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.270 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.270 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.270 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.270 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.270 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.270 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.271 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.271 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.271 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.271 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.271 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.271 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.271 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.271 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.272 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.272 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.272 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.272 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.272 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.272 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.272 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.272 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.273 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.273 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.273 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.273 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.273 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.273 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.273 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.274 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.274 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.274 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.274 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.274 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.274 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.274 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.275 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.275 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.275 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.275 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.275 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.275 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.275 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.275 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.276 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.276 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.276 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.276 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.276 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.276 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.276 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.276 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.277 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.277 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.277 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.277 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.277 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.277 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.277 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.277 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.278 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.278 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.278 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.278 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-05-22T23:12:11.278 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-05-22T23:12:11.278 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:11.278 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.278 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.279 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-05-22T23:12:11.279 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.279 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.279 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.279 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.279 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.279 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.279 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.280 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.280 INFO:teuthology.orchestra.run.smithi084.stdout: "threshold": { 2024-05-22T23:12:11.280 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "threshold", 2024-05-22T23:12:11.280 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-05-22T23:12:11.280 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.280 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.280 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "3.0", 2024-05-22T23:12:11.281 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1.0", 2024-05-22T23:12:11.281 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.281 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.281 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "scaling threshold", 2024-05-22T23:12:11.281 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-22T23:12:11.281 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.281 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.281 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.282 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.282 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.282 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.282 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "progress", 2024-05-22T23:12:11.282 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.282 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.282 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.283 INFO:teuthology.orchestra.run.smithi084.stdout: "allow_pg_recovery_event": { 2024-05-22T23:12:11.283 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "allow_pg_recovery_event", 2024-05-22T23:12:11.283 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.283 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.283 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.283 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.283 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.284 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.284 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.284 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-22T23:12:11.284 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.284 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.284 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.284 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.284 INFO:teuthology.orchestra.run.smithi084.stdout: "enabled": { 2024-05-22T23:12:11.285 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "enabled", 2024-05-22T23:12:11.285 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.285 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.285 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.285 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:11.285 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.285 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.286 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.286 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.286 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.286 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.286 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.286 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.286 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.286 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.287 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.287 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.287 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.287 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.287 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.287 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.287 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.288 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.288 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.288 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.288 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.288 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.288 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.288 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.288 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.289 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.289 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.289 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.289 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.289 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.289 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.289 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.289 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.289 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.290 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.290 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.290 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.290 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.290 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.290 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.290 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.290 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.291 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.291 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.291 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.291 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.293 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.293 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.294 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.294 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.294 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.294 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.294 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.294 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.294 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.295 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.295 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.295 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.295 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.295 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.295 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.295 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.295 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.296 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.296 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.296 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.296 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.296 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.296 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.296 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.296 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.296 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.297 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.297 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.297 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.297 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.297 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.297 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.297 INFO:teuthology.orchestra.run.smithi084.stdout: "max_completed_events": { 2024-05-22T23:12:11.297 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_completed_events", 2024-05-22T23:12:11.298 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.298 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.298 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.298 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "50", 2024-05-22T23:12:11.298 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.298 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.298 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.298 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "number of past completed events to remember", 2024-05-22T23:12:11.299 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.299 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.299 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.299 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.299 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-05-22T23:12:11.299 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-05-22T23:12:11.299 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:11.299 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.300 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.300 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5", 2024-05-22T23:12:11.300 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.300 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.300 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.300 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how long the module is going to sleep", 2024-05-22T23:12:11.300 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.300 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.301 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.301 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.301 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.301 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.301 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.301 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "prometheus", 2024-05-22T23:12:11.301 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.301 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.302 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.302 INFO:teuthology.orchestra.run.smithi084.stdout: "cache": { 2024-05-22T23:12:11.302 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cache", 2024-05-22T23:12:11.302 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.302 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.302 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.302 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:11.303 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.303 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.303 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.303 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.303 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.303 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.303 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.303 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.304 INFO:teuthology.orchestra.run.smithi084.stdout: "exclude_perf_counters": { 2024-05-22T23:12:11.304 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "exclude_perf_counters", 2024-05-22T23:12:11.304 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.304 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.304 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.304 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:11.304 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.304 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.305 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.305 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-22T23:12:11.305 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-22T23:12:11.305 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.305 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.305 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.305 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.305 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.306 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.306 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.306 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.306 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.306 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.306 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.306 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.307 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.307 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.307 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.307 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.307 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.307 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.307 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.307 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.308 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.308 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.308 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.308 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.308 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.308 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.308 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.308 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.309 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.309 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.309 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.309 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.309 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.309 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.309 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.309 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.310 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.310 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.310 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.310 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.310 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.310 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.310 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.310 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.311 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.311 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.311 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.311 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.311 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.311 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.311 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.311 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.312 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.312 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.312 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.312 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.312 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.312 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.312 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.312 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.313 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.313 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.313 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.313 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.313 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.313 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.313 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.313 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.314 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.314 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.314 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.314 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.314 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.314 INFO:teuthology.orchestra.run.smithi084.stdout: "rbd_stats_pools": { 2024-05-22T23:12:11.314 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rbd_stats_pools", 2024-05-22T23:12:11.314 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.315 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.315 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.315 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.315 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.315 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.315 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.315 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.316 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.316 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.316 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.316 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.316 INFO:teuthology.orchestra.run.smithi084.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-22T23:12:11.316 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-22T23:12:11.316 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.316 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.317 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.317 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "300", 2024-05-22T23:12:11.317 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.317 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.317 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.317 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.317 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.317 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.318 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.318 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.318 INFO:teuthology.orchestra.run.smithi084.stdout: "scrape_interval": { 2024-05-22T23:12:11.318 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "scrape_interval", 2024-05-22T23:12:11.318 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-05-22T23:12:11.318 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.318 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.319 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "15.0", 2024-05-22T23:12:11.319 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.319 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.319 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.319 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.319 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.319 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.319 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.320 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.320 INFO:teuthology.orchestra.run.smithi084.stdout: "server_addr": { 2024-05-22T23:12:11.320 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_addr", 2024-05-22T23:12:11.320 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.320 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.320 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.320 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "::", 2024-05-22T23:12:11.320 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.321 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.321 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.321 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-22T23:12:11.321 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.321 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.321 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.321 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.322 INFO:teuthology.orchestra.run.smithi084.stdout: "server_port": { 2024-05-22T23:12:11.322 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_port", 2024-05-22T23:12:11.322 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.322 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.322 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.322 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "9283", 2024-05-22T23:12:11.322 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.322 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.323 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.323 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-22T23:12:11.323 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.323 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.323 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.323 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.323 INFO:teuthology.orchestra.run.smithi084.stdout: "stale_cache_strategy": { 2024-05-22T23:12:11.324 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "stale_cache_strategy", 2024-05-22T23:12:11.324 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.324 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.324 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.324 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "log", 2024-05-22T23:12:11.324 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.324 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.324 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.325 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.325 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.325 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.325 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.325 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.325 INFO:teuthology.orchestra.run.smithi084.stdout: "standby_behaviour": { 2024-05-22T23:12:11.325 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "standby_behaviour", 2024-05-22T23:12:11.325 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.325 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.326 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.326 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "default", 2024-05-22T23:12:11.326 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.326 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.326 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.326 INFO:teuthology.orchestra.run.smithi084.stdout: "default", 2024-05-22T23:12:11.326 INFO:teuthology.orchestra.run.smithi084.stdout: "error" 2024-05-22T23:12:11.326 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.327 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.327 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.327 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.327 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.327 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.327 INFO:teuthology.orchestra.run.smithi084.stdout: "standby_error_status_code": { 2024-05-22T23:12:11.327 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "standby_error_status_code", 2024-05-22T23:12:11.327 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.328 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.328 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.328 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "500", 2024-05-22T23:12:11.328 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "400", 2024-05-22T23:12:11.328 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "599", 2024-05-22T23:12:11.328 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.328 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.328 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.329 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.329 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.329 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.329 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.329 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.329 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.329 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rbd_support", 2024-05-22T23:12:11.329 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.330 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.330 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.330 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.330 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.330 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.330 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.330 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.330 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.331 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.331 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.331 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.331 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.331 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.331 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.331 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.331 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.332 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.332 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.332 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.332 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.332 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.332 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.332 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.332 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.333 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.333 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.333 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.333 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.333 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.333 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.333 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.333 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.334 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.334 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.334 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.334 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.334 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.334 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.334 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.334 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.335 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.335 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.335 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.335 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.335 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.335 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.335 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.335 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.336 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.336 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.336 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.336 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.336 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.336 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.336 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.336 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.337 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.337 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.337 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.337 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.337 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.337 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.337 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.337 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.337 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.338 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.338 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.338 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.338 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.338 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.338 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.338 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.338 INFO:teuthology.orchestra.run.smithi084.stdout: "max_concurrent_snap_create": { 2024-05-22T23:12:11.339 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_concurrent_snap_create", 2024-05-22T23:12:11.339 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.339 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.339 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.339 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-05-22T23:12:11.339 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.339 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.340 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.340 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.340 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.340 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.340 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.340 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.340 INFO:teuthology.orchestra.run.smithi084.stdout: "mirror_snapshot_schedule": { 2024-05-22T23:12:11.340 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mirror_snapshot_schedule", 2024-05-22T23:12:11.341 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.341 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.341 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.341 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.341 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.341 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.341 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.341 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.341 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.342 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.342 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.342 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.342 INFO:teuthology.orchestra.run.smithi084.stdout: "trash_purge_schedule": { 2024-05-22T23:12:11.342 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "trash_purge_schedule", 2024-05-22T23:12:11.342 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.342 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.343 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.343 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.343 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.343 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.343 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.343 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.343 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.343 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.344 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.344 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.344 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.344 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.344 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.344 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "restful", 2024-05-22T23:12:11.344 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.344 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.345 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.345 INFO:teuthology.orchestra.run.smithi084.stdout: "enable_auth": { 2024-05-22T23:12:11.345 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "enable_auth", 2024-05-22T23:12:11.345 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.345 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.345 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.345 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:11.345 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.346 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.346 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.346 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.346 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.346 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.346 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.346 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.347 INFO:teuthology.orchestra.run.smithi084.stdout: "key_file": { 2024-05-22T23:12:11.347 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "key_file", 2024-05-22T23:12:11.347 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.347 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.347 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.347 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.347 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.347 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.348 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.348 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.348 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.348 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.348 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.348 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.348 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.348 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.349 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.349 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.349 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.349 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.349 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.349 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.349 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.349 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.350 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.350 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.350 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.350 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.350 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.350 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.350 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.351 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.351 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.351 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.351 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.351 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.351 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.351 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.351 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.352 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.352 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.352 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.352 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.352 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.352 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.352 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.352 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.353 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.353 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.353 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.353 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.353 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.353 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.353 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.353 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.354 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.354 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.354 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.354 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.354 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.354 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.354 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.354 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.355 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.355 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.355 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.355 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.355 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.355 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.355 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.355 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.356 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.356 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.356 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.356 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.356 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.356 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.356 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.356 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.357 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.357 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.357 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.357 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.357 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.357 INFO:teuthology.orchestra.run.smithi084.stdout: "server_addr": { 2024-05-22T23:12:11.357 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_addr", 2024-05-22T23:12:11.357 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.358 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.358 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.358 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.358 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.358 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.358 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.358 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.358 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.359 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.359 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.359 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.359 INFO:teuthology.orchestra.run.smithi084.stdout: "server_port": { 2024-05-22T23:12:11.359 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_port", 2024-05-22T23:12:11.359 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.359 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.359 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.359 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.360 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.360 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.360 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.360 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.360 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.360 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.360 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.360 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.361 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.361 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.361 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.361 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rgw", 2024-05-22T23:12:11.361 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.361 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.361 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.362 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.362 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.362 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.362 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.362 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.362 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.362 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.363 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.363 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.363 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.363 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.363 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.363 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.363 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.363 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.364 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.364 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.364 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.364 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.364 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.364 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.364 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.364 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.365 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.365 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.365 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.365 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.365 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.365 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.365 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.366 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.366 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.366 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.366 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.366 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.366 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.366 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.366 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.367 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.367 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.367 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.367 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.367 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.367 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.367 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.367 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.368 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.368 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.368 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.368 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.368 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.368 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.368 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.368 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.369 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.369 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.369 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.369 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.369 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.369 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.369 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.370 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.370 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.370 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.370 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.370 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.370 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.370 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.370 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.371 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.371 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.371 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.371 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.371 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rook", 2024-05-22T23:12:11.371 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.371 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.371 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.372 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.372 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.372 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.372 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.372 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.372 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.372 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.372 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.372 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.373 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.373 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.373 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.373 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.373 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.373 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.373 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.374 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.374 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.374 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.374 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.374 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.374 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.374 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.374 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.375 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.375 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.375 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.375 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.375 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.375 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.375 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.375 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.376 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.376 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.376 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.376 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.376 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.376 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.376 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.376 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.377 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.377 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.377 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.377 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.377 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.377 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.377 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.377 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.378 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.378 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.378 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.378 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.378 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.378 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.378 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.379 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.379 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.379 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.379 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.379 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.379 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.379 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.379 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.380 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.380 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.380 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.380 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.380 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.380 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.380 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.380 INFO:teuthology.orchestra.run.smithi084.stdout: "storage_class": { 2024-05-22T23:12:11.380 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "storage_class", 2024-05-22T23:12:11.381 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.381 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.381 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.381 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "local", 2024-05-22T23:12:11.381 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.381 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.381 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.381 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-22T23:12:11.382 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.382 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.382 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.382 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.382 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.382 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.382 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.383 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "selftest", 2024-05-22T23:12:11.383 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.383 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.383 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.383 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.383 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.383 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.383 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.384 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.384 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.384 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.384 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.384 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.384 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.384 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.384 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.384 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.385 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.385 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.385 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.385 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.385 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.385 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.385 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.385 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.385 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.386 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.386 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.386 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.386 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.386 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.386 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.386 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.386 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.387 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.387 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.387 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.387 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.387 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.387 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.387 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.387 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.388 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.388 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.388 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.388 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.388 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.388 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.388 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.388 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.388 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.389 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.389 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.389 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.389 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.389 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.389 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.389 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.389 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.389 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.390 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.390 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.390 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.390 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.390 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.390 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.390 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.390 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.391 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.391 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.391 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.391 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.391 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.391 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.391 INFO:teuthology.orchestra.run.smithi084.stdout: "roption1": { 2024-05-22T23:12:11.391 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "roption1", 2024-05-22T23:12:11.391 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.392 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.392 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.392 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.392 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.392 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.392 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.392 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.392 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.392 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.393 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.393 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.393 INFO:teuthology.orchestra.run.smithi084.stdout: "roption2": { 2024-05-22T23:12:11.393 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "roption2", 2024-05-22T23:12:11.393 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.393 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.393 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.393 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "xyz", 2024-05-22T23:12:11.394 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.394 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.394 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.394 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.394 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.394 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.394 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.395 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.395 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption1": { 2024-05-22T23:12:11.395 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption1", 2024-05-22T23:12:11.395 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.395 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.395 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.395 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.395 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.395 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.396 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.396 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.396 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.396 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.396 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.396 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.396 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption2": { 2024-05-22T23:12:11.396 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption2", 2024-05-22T23:12:11.397 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.397 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.397 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.397 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.397 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.397 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.397 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.397 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.397 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.398 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.398 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.398 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.398 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption3": { 2024-05-22T23:12:11.398 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption3", 2024-05-22T23:12:11.398 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-05-22T23:12:11.398 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.398 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.399 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.399 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.399 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.399 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.399 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.399 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.399 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.399 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.399 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.400 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption4": { 2024-05-22T23:12:11.400 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption4", 2024-05-22T23:12:11.400 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.400 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.400 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.400 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.400 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.400 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.400 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.401 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.401 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.401 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.401 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.401 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.401 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption5": { 2024-05-22T23:12:11.401 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption5", 2024-05-22T23:12:11.401 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.401 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.402 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.402 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.402 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.402 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.402 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.402 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.402 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.403 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.403 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.403 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.403 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption6": { 2024-05-22T23:12:11.403 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption6", 2024-05-22T23:12:11.403 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.403 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.403 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.403 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:11.404 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.404 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.404 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.404 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.404 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.404 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.404 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.404 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.404 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption7": { 2024-05-22T23:12:11.405 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption7", 2024-05-22T23:12:11.405 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.405 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.405 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.405 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.405 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1", 2024-05-22T23:12:11.405 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "42", 2024-05-22T23:12:11.405 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.406 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.406 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.406 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.406 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.406 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.406 INFO:teuthology.orchestra.run.smithi084.stdout: "testkey": { 2024-05-22T23:12:11.406 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "testkey", 2024-05-22T23:12:11.406 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.407 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.407 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.407 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.407 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.407 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.407 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.407 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.407 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.407 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.408 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.408 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.408 INFO:teuthology.orchestra.run.smithi084.stdout: "testlkey": { 2024-05-22T23:12:11.408 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "testlkey", 2024-05-22T23:12:11.408 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.408 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.408 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.408 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.409 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.409 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.409 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.409 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.409 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.409 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.409 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.409 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.409 INFO:teuthology.orchestra.run.smithi084.stdout: "testnewline": { 2024-05-22T23:12:11.410 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "testnewline", 2024-05-22T23:12:11.410 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.410 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.410 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.410 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.410 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.410 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.410 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.411 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.411 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.411 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.411 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.411 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.411 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.411 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.411 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.412 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "snap_schedule", 2024-05-22T23:12:11.412 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.412 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.412 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.412 INFO:teuthology.orchestra.run.smithi084.stdout: "allow_m_granularity": { 2024-05-22T23:12:11.412 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "allow_m_granularity", 2024-05-22T23:12:11.412 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.412 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.412 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.413 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.413 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.413 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.413 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.413 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "allow minute scheduled snapshots", 2024-05-22T23:12:11.413 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.413 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.413 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.413 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.414 INFO:teuthology.orchestra.run.smithi084.stdout: "dump_on_update": { 2024-05-22T23:12:11.414 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "dump_on_update", 2024-05-22T23:12:11.414 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.414 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.414 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.414 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.414 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.414 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.415 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.415 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "dump database to debug log on update", 2024-05-22T23:12:11.415 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.415 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.415 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.415 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.415 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.415 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.416 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.416 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.416 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.416 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.416 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.416 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.416 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.416 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.417 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.417 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.417 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.417 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.417 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.417 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.417 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.417 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.417 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.418 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.418 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.418 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.418 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.418 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.418 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.418 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.419 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.419 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.419 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.419 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.419 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.419 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.419 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.419 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.419 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.420 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.420 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.420 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.420 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.420 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.420 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.420 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.420 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.421 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.421 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.421 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.421 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.421 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.421 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.421 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.421 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.422 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.422 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.422 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.422 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.422 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.422 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.422 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.422 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.423 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.423 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.423 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.423 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.423 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.423 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.423 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.423 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.424 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.424 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.424 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.424 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.424 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.424 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.424 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "stats", 2024-05-22T23:12:11.424 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.424 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.425 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.425 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.425 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.425 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.425 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.425 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.425 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.425 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.425 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.426 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.426 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.426 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.426 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.426 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.426 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.426 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.426 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.427 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.427 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.427 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.427 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.427 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.427 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.427 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.427 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.427 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.428 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.428 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.428 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.428 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.428 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.428 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.428 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.428 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.429 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.429 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.429 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.429 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.429 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.429 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.429 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.429 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.429 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.430 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.430 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.430 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.430 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.430 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.430 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.430 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.430 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.431 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.431 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.431 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.431 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.431 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.431 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.431 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.431 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.431 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.432 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.432 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.432 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.432 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.432 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.432 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.432 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.433 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.433 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.433 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.433 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.433 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.433 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.433 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.433 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "status", 2024-05-22T23:12:11.433 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.434 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.434 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.434 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.434 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.434 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.434 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.434 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.435 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.435 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.435 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.435 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.435 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.435 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.435 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.435 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.436 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.436 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.436 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.436 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.436 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.436 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.436 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.436 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.437 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.437 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.437 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.437 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.437 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.437 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.437 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.437 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.438 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.438 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.438 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.438 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.438 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.438 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.438 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.439 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.439 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.439 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.439 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.439 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.439 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.439 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.439 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.440 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.440 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.440 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.440 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.440 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.440 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.440 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.440 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.440 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.441 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.441 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.441 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.441 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.441 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.441 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.441 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.441 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.442 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.442 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.442 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.442 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.442 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.442 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.442 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.443 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.443 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.443 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.443 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.443 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.443 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "telegraf", 2024-05-22T23:12:11.443 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.443 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.443 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.444 INFO:teuthology.orchestra.run.smithi084.stdout: "address": { 2024-05-22T23:12:11.444 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "address", 2024-05-22T23:12:11.444 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.444 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.444 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.444 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-22T23:12:11.444 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.444 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.445 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.445 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.445 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.445 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.445 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.445 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.445 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-05-22T23:12:11.445 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-05-22T23:12:11.446 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:11.446 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.446 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.446 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "15", 2024-05-22T23:12:11.446 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.446 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.446 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.446 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.447 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.447 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.447 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.447 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.447 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.447 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.447 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.447 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.448 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.448 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.448 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.448 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.448 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.448 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.448 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.448 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.449 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.449 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.449 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.449 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.449 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.449 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.449 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.449 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.450 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.450 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.450 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.450 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.450 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.450 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.450 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.451 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.451 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.451 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.451 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.451 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.451 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.451 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.451 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.452 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.452 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.452 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.452 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.452 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.452 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.452 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.452 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.453 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.453 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.453 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.453 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.453 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.453 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.453 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.453 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.454 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.454 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.454 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.454 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.454 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.454 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.454 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.454 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.455 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.455 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.455 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.455 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.455 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.455 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.455 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.455 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.455 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.456 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.456 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.456 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.456 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.456 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.456 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "telemetry", 2024-05-22T23:12:11.456 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.456 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.457 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.457 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_basic": { 2024-05-22T23:12:11.457 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_basic", 2024-05-22T23:12:11.457 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.457 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.457 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.457 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:11.457 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.458 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.458 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.458 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-22T23:12:11.458 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.458 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.458 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.458 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.458 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_crash": { 2024-05-22T23:12:11.459 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_crash", 2024-05-22T23:12:11.459 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.459 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.459 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.459 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:11.459 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.459 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.459 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.459 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-22T23:12:11.460 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.460 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.460 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.460 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.460 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_device": { 2024-05-22T23:12:11.460 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_device", 2024-05-22T23:12:11.460 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.460 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.461 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.461 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:11.461 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.461 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.461 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.461 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-22T23:12:11.461 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.461 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.462 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.462 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.462 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_ident": { 2024-05-22T23:12:11.462 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_ident", 2024-05-22T23:12:11.462 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.462 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.462 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.463 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.463 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.463 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.463 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.463 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-22T23:12:11.463 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.463 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.463 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.464 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.464 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_perf": { 2024-05-22T23:12:11.464 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_perf", 2024-05-22T23:12:11.464 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.464 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.464 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.464 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.464 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.465 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.465 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.465 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-22T23:12:11.465 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.465 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.465 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.465 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.466 INFO:teuthology.orchestra.run.smithi084.stdout: "contact": { 2024-05-22T23:12:11.466 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "contact", 2024-05-22T23:12:11.466 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.466 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.466 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.466 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.466 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.466 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.467 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.467 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.467 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.467 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.467 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.467 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.467 INFO:teuthology.orchestra.run.smithi084.stdout: "description": { 2024-05-22T23:12:11.468 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "description", 2024-05-22T23:12:11.468 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.468 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.468 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.468 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.468 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.468 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.468 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.469 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.469 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.469 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.469 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.469 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.469 INFO:teuthology.orchestra.run.smithi084.stdout: "device_url": { 2024-05-22T23:12:11.469 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "device_url", 2024-05-22T23:12:11.469 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.469 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.470 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.470 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-22T23:12:11.470 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.470 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.470 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.470 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.470 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.471 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.471 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.471 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.471 INFO:teuthology.orchestra.run.smithi084.stdout: "enabled": { 2024-05-22T23:12:11.471 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "enabled", 2024-05-22T23:12:11.471 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.471 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.471 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.472 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.472 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.472 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.472 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.472 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.472 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.472 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.472 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.473 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.473 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-05-22T23:12:11.473 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-05-22T23:12:11.473 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.473 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.473 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.473 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "24", 2024-05-22T23:12:11.473 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "8", 2024-05-22T23:12:11.474 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.474 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.474 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.474 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.474 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.474 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.474 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.474 INFO:teuthology.orchestra.run.smithi084.stdout: "last_opt_revision": { 2024-05-22T23:12:11.475 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "last_opt_revision", 2024-05-22T23:12:11.475 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.475 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.475 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.475 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "1", 2024-05-22T23:12:11.475 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.475 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.476 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.476 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.476 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.476 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.476 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.476 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.476 INFO:teuthology.orchestra.run.smithi084.stdout: "leaderboard": { 2024-05-22T23:12:11.476 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "leaderboard", 2024-05-22T23:12:11.477 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.477 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.477 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.477 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.477 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.477 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.477 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.478 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.478 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.478 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.478 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.478 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.478 INFO:teuthology.orchestra.run.smithi084.stdout: "leaderboard_description": { 2024-05-22T23:12:11.478 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "leaderboard_description", 2024-05-22T23:12:11.478 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.479 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.479 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.479 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.479 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.479 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.479 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.479 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.479 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.480 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.480 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.480 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.480 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.480 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.480 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.480 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.480 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.481 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.481 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.481 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.481 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.481 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.481 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.481 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.481 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.482 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.482 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.482 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.482 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.482 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.482 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.482 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.482 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.483 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.483 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.483 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.483 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.483 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.483 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.483 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.484 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.484 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.484 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.484 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.484 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.484 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.484 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.485 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.485 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.485 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.485 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.485 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.485 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.485 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.485 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.486 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.486 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.486 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.486 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.486 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.486 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.486 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.486 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.487 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.487 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.487 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.487 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.487 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.487 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.487 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.487 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.487 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.488 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.488 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.488 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.488 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.488 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.488 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.488 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.488 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.489 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.489 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.489 INFO:teuthology.orchestra.run.smithi084.stdout: "organization": { 2024-05-22T23:12:11.489 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "organization", 2024-05-22T23:12:11.489 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.489 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.489 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.489 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.489 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.490 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.490 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.490 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.490 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.490 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.490 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.490 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.490 INFO:teuthology.orchestra.run.smithi084.stdout: "proxy": { 2024-05-22T23:12:11.490 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "proxy", 2024-05-22T23:12:11.491 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.491 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.491 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.491 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.491 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.491 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.491 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.491 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.492 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.492 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.492 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.492 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.492 INFO:teuthology.orchestra.run.smithi084.stdout: "url": { 2024-05-22T23:12:11.492 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "url", 2024-05-22T23:12:11.492 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.492 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.493 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.493 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-22T23:12:11.493 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.493 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.493 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.493 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.493 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.493 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.494 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.494 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.494 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.494 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.494 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.494 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "test_orchestrator", 2024-05-22T23:12:11.494 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.494 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.495 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.495 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.495 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.495 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.495 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.495 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.495 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.496 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.496 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.496 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.496 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.496 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.496 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.496 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.496 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.497 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.497 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.497 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.497 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.497 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.497 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.497 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.498 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.498 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.498 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.498 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.498 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.498 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.498 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.499 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.499 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.499 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.499 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.499 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.499 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.499 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.499 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.500 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.500 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.500 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.500 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.500 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.500 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.500 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.501 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.501 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.501 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.501 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.501 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.501 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.501 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.501 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.501 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.502 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.502 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.502 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.502 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.502 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.502 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.502 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.503 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.503 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.503 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.503 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.503 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.503 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.503 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.503 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.503 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.504 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.504 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.504 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.504 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.504 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.504 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "volumes", 2024-05-22T23:12:11.504 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.504 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.504 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.505 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.505 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.505 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.505 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.505 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.505 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.505 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.505 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.505 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.506 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.506 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.506 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.506 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.506 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.506 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.506 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.507 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.507 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.507 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.507 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.507 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.507 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.507 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.507 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.507 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.508 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.508 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.508 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.508 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.508 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.508 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.508 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.508 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.508 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.509 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.509 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.509 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.509 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.509 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.509 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.509 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.509 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.510 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.510 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.510 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.510 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.510 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.510 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.510 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.511 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.511 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.511 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.511 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.511 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.511 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.511 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.511 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.512 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.512 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.512 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.512 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.512 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.512 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.512 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.512 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.513 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.513 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.513 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.513 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.513 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.513 INFO:teuthology.orchestra.run.smithi084.stdout: "max_concurrent_clones": { 2024-05-22T23:12:11.513 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_concurrent_clones", 2024-05-22T23:12:11.514 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.514 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.514 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.514 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "4", 2024-05-22T23:12:11.514 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.514 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.514 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.514 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-22T23:12:11.515 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.515 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.515 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.515 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.515 INFO:teuthology.orchestra.run.smithi084.stdout: "periodic_async_work": { 2024-05-22T23:12:11.515 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "periodic_async_work", 2024-05-22T23:12:11.515 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.515 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.515 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.516 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.516 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.516 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.516 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.516 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Periodically check for async work", 2024-05-22T23:12:11.516 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.516 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.516 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.516 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.517 INFO:teuthology.orchestra.run.smithi084.stdout: "snapshot_clone_delay": { 2024-05-22T23:12:11.517 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "snapshot_clone_delay", 2024-05-22T23:12:11.517 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.517 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.517 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.517 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-05-22T23:12:11.517 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.517 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.518 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.518 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-22T23:12:11.518 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.518 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.518 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.518 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.518 INFO:teuthology.orchestra.run.smithi084.stdout: "snapshot_clone_no_wait": { 2024-05-22T23:12:11.518 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "snapshot_clone_no_wait", 2024-05-22T23:12:11.519 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.519 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.519 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.519 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-05-22T23:12:11.519 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.519 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.519 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.519 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-22T23:12:11.520 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.520 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.520 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.520 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.520 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.520 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.520 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.520 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "zabbix", 2024-05-22T23:12:11.521 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-05-22T23:12:11.521 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-05-22T23:12:11.521 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-05-22T23:12:11.521 INFO:teuthology.orchestra.run.smithi084.stdout: "discovery_interval": { 2024-05-22T23:12:11.521 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "discovery_interval", 2024-05-22T23:12:11.521 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-05-22T23:12:11.521 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.521 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.522 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "100", 2024-05-22T23:12:11.522 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.522 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.522 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.522 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.522 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.522 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.522 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.523 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.523 INFO:teuthology.orchestra.run.smithi084.stdout: "identifier": { 2024-05-22T23:12:11.523 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "identifier", 2024-05-22T23:12:11.523 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.523 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.523 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.523 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.523 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.524 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.524 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.524 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.524 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.524 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.524 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.524 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.524 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-05-22T23:12:11.525 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-05-22T23:12:11.525 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-05-22T23:12:11.525 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.525 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.525 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-05-22T23:12:11.525 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.525 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.525 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.526 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.526 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.526 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.526 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.526 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.526 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-05-22T23:12:11.526 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-05-22T23:12:11.526 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.527 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.527 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.527 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.527 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.527 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.527 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.527 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.527 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.527 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.528 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.528 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.528 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.528 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.528 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.528 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.528 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.528 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.528 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.529 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-05-22T23:12:11.529 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-05-22T23:12:11.529 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.529 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.529 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.529 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.529 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.529 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.530 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.530 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.530 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.530 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.530 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.530 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.530 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-05-22T23:12:11.530 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-05-22T23:12:11.531 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.531 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.531 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.531 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-05-22T23:12:11.531 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.531 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.531 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-05-22T23:12:11.531 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-05-22T23:12:11.532 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-05-22T23:12:11.532 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-05-22T23:12:11.532 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-05-22T23:12:11.532 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-05-22T23:12:11.532 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-05-22T23:12:11.532 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.532 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.532 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.533 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.533 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.533 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.533 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-05-22T23:12:11.533 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-05-22T23:12:11.533 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-05-22T23:12:11.533 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.533 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-05-22T23:12:11.533 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-05-22T23:12:11.534 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.534 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.534 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.534 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.534 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.534 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.534 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.535 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.535 INFO:teuthology.orchestra.run.smithi084.stdout: "zabbix_host": { 2024-05-22T23:12:11.535 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "zabbix_host", 2024-05-22T23:12:11.535 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.535 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.535 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.535 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-05-22T23:12:11.535 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.536 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.536 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.536 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.536 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.536 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.536 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.536 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.536 INFO:teuthology.orchestra.run.smithi084.stdout: "zabbix_port": { 2024-05-22T23:12:11.537 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "zabbix_port", 2024-05-22T23:12:11.537 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-05-22T23:12:11.537 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.537 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.537 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10051", 2024-05-22T23:12:11.537 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.537 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.537 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.538 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.538 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.538 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.538 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.538 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.538 INFO:teuthology.orchestra.run.smithi084.stdout: "zabbix_sender": { 2024-05-22T23:12:11.538 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "zabbix_sender", 2024-05-22T23:12:11.539 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-05-22T23:12:11.539 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-05-22T23:12:11.539 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-05-22T23:12:11.539 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-22T23:12:11.539 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-05-22T23:12:11.539 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-05-22T23:12:11.539 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-05-22T23:12:11.539 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-05-22T23:12:11.539 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-05-22T23:12:11.540 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-05-22T23:12:11.540 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-05-22T23:12:11.540 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.540 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.540 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.540 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.540 INFO:teuthology.orchestra.run.smithi084.stdout: "services": { 2024-05-22T23:12:11.541 INFO:teuthology.orchestra.run.smithi084.stdout: "dashboard": "https://172.21.15.84:8443/" 2024-05-22T23:12:11.541 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.541 INFO:teuthology.orchestra.run.smithi084.stdout: "always_on_modules": { 2024-05-22T23:12:11.541 INFO:teuthology.orchestra.run.smithi084.stdout: "octopus": [ 2024-05-22T23:12:11.541 INFO:teuthology.orchestra.run.smithi084.stdout: "balancer", 2024-05-22T23:12:11.541 INFO:teuthology.orchestra.run.smithi084.stdout: "crash", 2024-05-22T23:12:11.541 INFO:teuthology.orchestra.run.smithi084.stdout: "devicehealth", 2024-05-22T23:12:11.541 INFO:teuthology.orchestra.run.smithi084.stdout: "orchestrator", 2024-05-22T23:12:11.542 INFO:teuthology.orchestra.run.smithi084.stdout: "pg_autoscaler", 2024-05-22T23:12:11.542 INFO:teuthology.orchestra.run.smithi084.stdout: "progress", 2024-05-22T23:12:11.542 INFO:teuthology.orchestra.run.smithi084.stdout: "rbd_support", 2024-05-22T23:12:11.542 INFO:teuthology.orchestra.run.smithi084.stdout: "status", 2024-05-22T23:12:11.542 INFO:teuthology.orchestra.run.smithi084.stdout: "telemetry", 2024-05-22T23:12:11.542 INFO:teuthology.orchestra.run.smithi084.stdout: "volumes" 2024-05-22T23:12:11.542 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.543 INFO:teuthology.orchestra.run.smithi084.stdout: "pacific": [ 2024-05-22T23:12:11.543 INFO:teuthology.orchestra.run.smithi084.stdout: "balancer", 2024-05-22T23:12:11.543 INFO:teuthology.orchestra.run.smithi084.stdout: "crash", 2024-05-22T23:12:11.543 INFO:teuthology.orchestra.run.smithi084.stdout: "devicehealth", 2024-05-22T23:12:11.543 INFO:teuthology.orchestra.run.smithi084.stdout: "orchestrator", 2024-05-22T23:12:11.543 INFO:teuthology.orchestra.run.smithi084.stdout: "pg_autoscaler", 2024-05-22T23:12:11.543 INFO:teuthology.orchestra.run.smithi084.stdout: "progress", 2024-05-22T23:12:11.543 INFO:teuthology.orchestra.run.smithi084.stdout: "rbd_support", 2024-05-22T23:12:11.544 INFO:teuthology.orchestra.run.smithi084.stdout: "status", 2024-05-22T23:12:11.544 INFO:teuthology.orchestra.run.smithi084.stdout: "telemetry", 2024-05-22T23:12:11.544 INFO:teuthology.orchestra.run.smithi084.stdout: "volumes" 2024-05-22T23:12:11.544 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.544 INFO:teuthology.orchestra.run.smithi084.stdout: "quincy": [ 2024-05-22T23:12:11.544 INFO:teuthology.orchestra.run.smithi084.stdout: "balancer", 2024-05-22T23:12:11.544 INFO:teuthology.orchestra.run.smithi084.stdout: "crash", 2024-05-22T23:12:11.544 INFO:teuthology.orchestra.run.smithi084.stdout: "devicehealth", 2024-05-22T23:12:11.545 INFO:teuthology.orchestra.run.smithi084.stdout: "orchestrator", 2024-05-22T23:12:11.545 INFO:teuthology.orchestra.run.smithi084.stdout: "pg_autoscaler", 2024-05-22T23:12:11.545 INFO:teuthology.orchestra.run.smithi084.stdout: "progress", 2024-05-22T23:12:11.545 INFO:teuthology.orchestra.run.smithi084.stdout: "rbd_support", 2024-05-22T23:12:11.545 INFO:teuthology.orchestra.run.smithi084.stdout: "status", 2024-05-22T23:12:11.545 INFO:teuthology.orchestra.run.smithi084.stdout: "telemetry", 2024-05-22T23:12:11.545 INFO:teuthology.orchestra.run.smithi084.stdout: "volumes" 2024-05-22T23:12:11.545 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.546 INFO:teuthology.orchestra.run.smithi084.stdout: "reef": [ 2024-05-22T23:12:11.546 INFO:teuthology.orchestra.run.smithi084.stdout: "balancer", 2024-05-22T23:12:11.546 INFO:teuthology.orchestra.run.smithi084.stdout: "crash", 2024-05-22T23:12:11.546 INFO:teuthology.orchestra.run.smithi084.stdout: "devicehealth", 2024-05-22T23:12:11.546 INFO:teuthology.orchestra.run.smithi084.stdout: "orchestrator", 2024-05-22T23:12:11.546 INFO:teuthology.orchestra.run.smithi084.stdout: "pg_autoscaler", 2024-05-22T23:12:11.546 INFO:teuthology.orchestra.run.smithi084.stdout: "progress", 2024-05-22T23:12:11.546 INFO:teuthology.orchestra.run.smithi084.stdout: "rbd_support", 2024-05-22T23:12:11.547 INFO:teuthology.orchestra.run.smithi084.stdout: "status", 2024-05-22T23:12:11.547 INFO:teuthology.orchestra.run.smithi084.stdout: "telemetry", 2024-05-22T23:12:11.547 INFO:teuthology.orchestra.run.smithi084.stdout: "volumes" 2024-05-22T23:12:11.547 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-05-22T23:12:11.547 INFO:teuthology.orchestra.run.smithi084.stdout: "squid": [ 2024-05-22T23:12:11.547 INFO:teuthology.orchestra.run.smithi084.stdout: "balancer", 2024-05-22T23:12:11.547 INFO:teuthology.orchestra.run.smithi084.stdout: "crash", 2024-05-22T23:12:11.547 INFO:teuthology.orchestra.run.smithi084.stdout: "devicehealth", 2024-05-22T23:12:11.547 INFO:teuthology.orchestra.run.smithi084.stdout: "orchestrator", 2024-05-22T23:12:11.548 INFO:teuthology.orchestra.run.smithi084.stdout: "pg_autoscaler", 2024-05-22T23:12:11.548 INFO:teuthology.orchestra.run.smithi084.stdout: "progress", 2024-05-22T23:12:11.548 INFO:teuthology.orchestra.run.smithi084.stdout: "rbd_support", 2024-05-22T23:12:11.548 INFO:teuthology.orchestra.run.smithi084.stdout: "status", 2024-05-22T23:12:11.548 INFO:teuthology.orchestra.run.smithi084.stdout: "telemetry", 2024-05-22T23:12:11.548 INFO:teuthology.orchestra.run.smithi084.stdout: "volumes" 2024-05-22T23:12:11.548 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-05-22T23:12:11.548 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.548 INFO:teuthology.orchestra.run.smithi084.stdout: "last_failure_osd_epoch": 89, 2024-05-22T23:12:11.549 INFO:teuthology.orchestra.run.smithi084.stdout: "active_clients": [ 2024-05-22T23:12:11.549 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.549 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "devicehealth", 2024-05-22T23:12:11.549 INFO:teuthology.orchestra.run.smithi084.stdout: "addrvec": [ 2024-05-22T23:12:11.549 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.549 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v2", 2024-05-22T23:12:11.549 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "172.21.15.84:0", 2024-05-22T23:12:11.549 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 136490549 2024-05-22T23:12:11.549 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.550 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-05-22T23:12:11.550 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.550 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.550 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "libcephsqlite", 2024-05-22T23:12:11.550 INFO:teuthology.orchestra.run.smithi084.stdout: "addrvec": [ 2024-05-22T23:12:11.550 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.550 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v2", 2024-05-22T23:12:11.550 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "172.21.15.84:0", 2024-05-22T23:12:11.551 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 3665246464 2024-05-22T23:12:11.551 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.551 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-05-22T23:12:11.551 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.551 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.551 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rbd_support", 2024-05-22T23:12:11.551 INFO:teuthology.orchestra.run.smithi084.stdout: "addrvec": [ 2024-05-22T23:12:11.551 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.551 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v2", 2024-05-22T23:12:11.552 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "172.21.15.84:0", 2024-05-22T23:12:11.552 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 1701822245 2024-05-22T23:12:11.552 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.552 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-05-22T23:12:11.552 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.552 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.552 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "snap_schedule", 2024-05-22T23:12:11.553 INFO:teuthology.orchestra.run.smithi084.stdout: "addrvec": [ 2024-05-22T23:12:11.553 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.553 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v2", 2024-05-22T23:12:11.553 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "172.21.15.84:0", 2024-05-22T23:12:11.553 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 3779740686 2024-05-22T23:12:11.553 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.553 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-05-22T23:12:11.553 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.554 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.554 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "snap_schedule", 2024-05-22T23:12:11.554 INFO:teuthology.orchestra.run.smithi084.stdout: "addrvec": [ 2024-05-22T23:12:11.554 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.554 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v2", 2024-05-22T23:12:11.554 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "172.21.15.84:0", 2024-05-22T23:12:11.554 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 3103675305 2024-05-22T23:12:11.555 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.555 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-05-22T23:12:11.555 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:11.555 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.555 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "volumes", 2024-05-22T23:12:11.555 INFO:teuthology.orchestra.run.smithi084.stdout: "addrvec": [ 2024-05-22T23:12:11.555 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-05-22T23:12:11.555 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v2", 2024-05-22T23:12:11.556 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "172.21.15.84:0", 2024-05-22T23:12:11.556 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 1053725225 2024-05-22T23:12:11.556 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.556 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-05-22T23:12:11.556 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:11.556 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-05-22T23:12:11.556 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:12:11.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.256+0000 7fde6e2d2640 1 -- 172.21.15.84:0/3616890991 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fde540a2d00 msgr2=0x7fde540a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:11.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.256+0000 7fde6e2d2640 1 --2- 172.21.15.84:0/3616890991 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fde540a2d00 0x7fde540a51c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fde50005fd0 tx=0x7fde5000b510 comp rx=0 tx=0).stop 2024-05-22T23:12:11.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.256+0000 7fde6e2d2640 1 -- 172.21.15.84:0/3616890991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde68102b50 msgr2=0x7fde68198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:11.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.256+0000 7fde6e2d2640 1 --2- 172.21.15.84:0/3616890991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde68102b50 0x7fde68198120 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fde5800d950 tx=0x7fde5800de10 comp rx=0 tx=0).stop 2024-05-22T23:12:11.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.256+0000 7fde6e2d2640 1 -- 172.21.15.84:0/3616890991 shutdown_connections 2024-05-22T23:12:11.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.256+0000 7fde6e2d2640 1 --2- 172.21.15.84:0/3616890991 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fde540a2d00 0x7fde540a51c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:11.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.256+0000 7fde6e2d2640 1 --2- 172.21.15.84:0/3616890991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde6810be90 0x7fde6819c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:11.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.256+0000 7fde6e2d2640 1 --2- 172.21.15.84:0/3616890991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde68103470 0x7fde68198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:11.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.256+0000 7fde6e2d2640 1 --2- 172.21.15.84:0/3616890991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde68102b50 0x7fde68198120 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:11.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.256+0000 7fde6e2d2640 1 -- 172.21.15.84:0/3616890991 >> 172.21.15.84:0/3616890991 conn(0x7fde680778d0 msgr2=0x7fde680fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:11.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.256+0000 7fde6e2d2640 1 -- 172.21.15.84:0/3616890991 shutdown_connections 2024-05-22T23:12:11.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:09.256+0000 7fde6e2d2640 1 -- 172.21.15.84:0/3616890991 wait complete. 2024-05-22T23:12:11.560 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:08 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2640761320' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:12:11.560 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:08 smithi084 ceph-mon[32186]: pgmap v321: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-05-22T23:12:11.560 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:08 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2821206781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:12:11.560 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:08 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3316047529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:12:11.560 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:08 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2916666737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:12:11.560 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:09 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3616890991' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-22T23:12:11.560 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:10 smithi204 ceph-mon[36211]: pgmap v322: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-05-22T23:12:11.561 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:08 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2640761320' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:12:11.561 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:08 smithi096 ceph-mon[36392]: pgmap v321: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-05-22T23:12:11.561 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:08 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2821206781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:12:11.561 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:08 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3316047529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:12:11.561 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:08 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2916666737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:12:11.562 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:09 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3616890991' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-22T23:12:11.562 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:10 smithi096 ceph-mon[36392]: pgmap v322: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-05-22T23:12:11.569 DEBUG:tasks.check_counter:Getting stats from a 2024-05-22T23:12:11.569 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-22T23:12:11.570 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:09 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3616890991' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-22T23:12:11.570 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:10 smithi084 ceph-mon[32186]: pgmap v322: 97 pgs: 97 active+clean; 431 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-05-22T23:12:11.818 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:12:13.310 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:13 smithi084 ceph-mon[32186]: pgmap v323: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-22T23:12:13.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:13 smithi204 ceph-mon[36211]: pgmap v323: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-22T23:12:13.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:13 smithi096 ceph-mon[36392]: pgmap v323: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-22T23:12:13.560 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:12:13.560 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-0": { 2024-05-22T23:12:13.560 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 4, 2024-05-22T23:12:13.561 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 4, 2024-05-22T23:12:13.561 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 2992, 2024-05-22T23:12:13.561 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 448, 2024-05-22T23:12:13.561 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 4, 2024-05-22T23:12:13.561 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 0, 2024-05-22T23:12:13.561 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.001468289, 2024-05-22T23:12:13.561 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.000044122, 2024-05-22T23:12:13.562 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.000499210, 2024-05-22T23:12:13.562 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-22T23:12:13.562 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:13.562 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 3, 2024-05-22T23:12:13.562 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000706329, 2024-05-22T23:12:13.562 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000235443 2024-05-22T23:12:13.562 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:13.562 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:13.563 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-22T23:12:13.563 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-22T23:12:13.563 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-22T23:12:13.563 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:13.563 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-22T23:12:13.563 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 448 2024-05-22T23:12:13.563 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:13.563 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-1": { 2024-05-22T23:12:13.564 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 2, 2024-05-22T23:12:13.564 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 154, 2024-05-22T23:12:13.564 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 320, 2024-05-22T23:12:13.564 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 182576, 2024-05-22T23:12:13.564 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 5, 2024-05-22T23:12:13.564 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 3, 2024-05-22T23:12:13.564 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.011362945, 2024-05-22T23:12:13.564 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.007676152, 2024-05-22T23:12:13.564 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.001624184, 2024-05-22T23:12:13.565 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-22T23:12:13.565 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:13.565 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 151, 2024-05-22T23:12:13.565 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.013826796, 2024-05-22T23:12:13.565 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000091568 2024-05-22T23:12:13.565 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:13.565 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:13.565 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-22T23:12:13.565 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-22T23:12:13.566 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-22T23:12:13.566 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:13.566 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-22T23:12:13.566 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 182576 2024-05-22T23:12:13.566 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:13.566 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-2": { 2024-05-22T23:12:13.566 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 212, 2024-05-22T23:12:13.567 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 202, 2024-05-22T23:12:13.567 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 1440320, 2024-05-22T23:12:13.567 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 124064, 2024-05-22T23:12:13.567 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 2, 2024-05-22T23:12:13.567 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 1, 2024-05-22T23:12:13.567 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.038540277, 2024-05-22T23:12:13.567 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.011456182, 2024-05-22T23:12:13.568 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.031702941, 2024-05-22T23:12:13.568 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.003599519, 2024-05-22T23:12:13.568 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:13.568 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 201, 2024-05-22T23:12:13.568 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.008637784, 2024-05-22T23:12:13.568 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000042974 2024-05-22T23:12:13.568 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:13.569 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:13.569 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-22T23:12:13.569 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-22T23:12:13.569 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-22T23:12:13.569 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:13.569 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 1440320, 2024-05-22T23:12:13.569 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 124064 2024-05-22T23:12:13.570 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:13.570 INFO:teuthology.orchestra.run.smithi084.stdout: "cct": { 2024-05-22T23:12:13.570 INFO:teuthology.orchestra.run.smithi084.stdout: "total_workers": 0, 2024-05-22T23:12:13.570 INFO:teuthology.orchestra.run.smithi084.stdout: "unhealthy_workers": 0 2024-05-22T23:12:13.570 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:13.570 INFO:teuthology.orchestra.run.smithi084.stdout: "mempool": { 2024-05-22T23:12:13.570 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_bytes": 0, 2024-05-22T23:12:13.570 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_items": 0, 2024-05-22T23:12:13.571 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_bytes": 0, 2024-05-22T23:12:13.571 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_items": 0, 2024-05-22T23:12:13.571 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_bytes": 0, 2024-05-22T23:12:13.571 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_items": 0, 2024-05-22T23:12:13.571 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-22T23:12:13.571 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_items": 0, 2024-05-22T23:12:13.571 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-22T23:12:13.571 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_items": 0, 2024-05-22T23:12:13.572 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_bytes": 0, 2024-05-22T23:12:13.572 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_items": 0, 2024-05-22T23:12:13.572 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-22T23:12:13.572 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_items": 0, 2024-05-22T23:12:13.572 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_bytes": 0, 2024-05-22T23:12:13.572 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_items": 0, 2024-05-22T23:12:13.572 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_bytes": 0, 2024-05-22T23:12:13.572 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_items": 0, 2024-05-22T23:12:13.572 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-22T23:12:13.573 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_items": 0, 2024-05-22T23:12:13.573 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-22T23:12:13.573 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_items": 0, 2024-05-22T23:12:13.573 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_bytes": 0, 2024-05-22T23:12:13.573 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_items": 0, 2024-05-22T23:12:13.573 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_bytes": 0, 2024-05-22T23:12:13.573 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_items": 0, 2024-05-22T23:12:13.573 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-22T23:12:13.574 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_items": 0, 2024-05-22T23:12:13.574 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_bytes": 0, 2024-05-22T23:12:13.574 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_items": 0, 2024-05-22T23:12:13.574 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_bytes": 0, 2024-05-22T23:12:13.574 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_items": 0, 2024-05-22T23:12:13.574 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_bytes": 0, 2024-05-22T23:12:13.574 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_items": 0, 2024-05-22T23:12:13.574 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_bytes": 0, 2024-05-22T23:12:13.575 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_items": 0, 2024-05-22T23:12:13.575 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_bytes": 148977, 2024-05-22T23:12:13.575 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_items": 30, 2024-05-22T23:12:13.575 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_bytes": 0, 2024-05-22T23:12:13.575 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_items": 0, 2024-05-22T23:12:13.575 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_bytes": 0, 2024-05-22T23:12:13.575 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_items": 0, 2024-05-22T23:12:13.575 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_bytes": 0, 2024-05-22T23:12:13.575 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_items": 0, 2024-05-22T23:12:13.576 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_bytes": 0, 2024-05-22T23:12:13.576 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_items": 0, 2024-05-22T23:12:13.576 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_bytes": 0, 2024-05-22T23:12:13.576 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_items": 0, 2024-05-22T23:12:13.576 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_bytes": 0, 2024-05-22T23:12:13.576 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_items": 0, 2024-05-22T23:12:13.576 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_bytes": 0, 2024-05-22T23:12:13.577 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_items": 0, 2024-05-22T23:12:13.577 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_bytes": 928, 2024-05-22T23:12:13.577 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_items": 2, 2024-05-22T23:12:13.577 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_bytes": 0, 2024-05-22T23:12:13.577 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_items": 0, 2024-05-22T23:12:13.577 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_bytes": 0, 2024-05-22T23:12:13.577 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_items": 0 2024-05-22T23:12:13.577 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:13.578 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-22T23:12:13.578 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-05-22T23:12:13.578 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-05-22T23:12:13.578 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-05-22T23:12:13.578 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 214, 2024-05-22T23:12:13.578 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 1410436, 2024-05-22T23:12:13.578 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-05-22T23:12:13.578 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 214, 2024-05-22T23:12:13.579 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-05-22T23:12:13.579 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-05-22T23:12:13.579 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 214, 2024-05-22T23:12:13.579 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 1410436, 2024-05-22T23:12:13.579 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-05-22T23:12:13.579 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-22T23:12:13.579 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-22T23:12:13.579 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-22T23:12:13.580 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:13.580 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:13.580 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:12:14.175 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-05-22T23:12:14.175 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-05-22T23:12:14.175 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-05-22T23:12:14.175 DEBUG:tasks.check_counter:Getting stats from d 2024-05-22T23:12:14.176 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-22T23:12:14.416 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:12:15.381 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:15 smithi084 ceph-mon[32186]: pgmap v324: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-22T23:12:15.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:15 smithi204 ceph-mon[36211]: pgmap v324: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-22T23:12:15.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:15 smithi096 ceph-mon[36392]: pgmap v324: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-22T23:12:16.112 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:12:16.112 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-0": { 2024-05-22T23:12:16.112 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 222, 2024-05-22T23:12:16.112 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 210, 2024-05-22T23:12:16.113 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 1444944, 2024-05-22T23:12:16.113 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 128352, 2024-05-22T23:12:16.113 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 4, 2024-05-22T23:12:16.113 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 0, 2024-05-22T23:12:16.113 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.041069242, 2024-05-22T23:12:16.113 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.012337689, 2024-05-22T23:12:16.113 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.032975673, 2024-05-22T23:12:16.113 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.003938278, 2024-05-22T23:12:16.114 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:16.114 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 210, 2024-05-22T23:12:16.114 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.010107874, 2024-05-22T23:12:16.114 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000048132 2024-05-22T23:12:16.114 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:16.114 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:16.114 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-22T23:12:16.114 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-22T23:12:16.115 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-22T23:12:16.115 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:16.115 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 1444944, 2024-05-22T23:12:16.115 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 128352 2024-05-22T23:12:16.115 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:16.115 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-1": { 2024-05-22T23:12:16.115 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 2, 2024-05-22T23:12:16.115 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 159, 2024-05-22T23:12:16.116 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 320, 2024-05-22T23:12:16.116 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 183456, 2024-05-22T23:12:16.116 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 5, 2024-05-22T23:12:16.116 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 3, 2024-05-22T23:12:16.116 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.012501407, 2024-05-22T23:12:16.116 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.008597320, 2024-05-22T23:12:16.116 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.001778117, 2024-05-22T23:12:16.116 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-22T23:12:16.117 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:16.117 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 156, 2024-05-22T23:12:16.117 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.010725115, 2024-05-22T23:12:16.117 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000068750 2024-05-22T23:12:16.117 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:16.117 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:16.117 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-22T23:12:16.117 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-22T23:12:16.117 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-22T23:12:16.118 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:16.118 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-22T23:12:16.118 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 183456 2024-05-22T23:12:16.118 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:16.118 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-2": { 2024-05-22T23:12:16.118 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 0, 2024-05-22T23:12:16.118 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 2, 2024-05-22T23:12:16.118 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 0, 2024-05-22T23:12:16.119 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 0, 2024-05-22T23:12:16.119 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 2, 2024-05-22T23:12:16.119 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 1, 2024-05-22T23:12:16.119 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.000301732, 2024-05-22T23:12:16.119 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.000000000, 2024-05-22T23:12:16.119 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.000215256, 2024-05-22T23:12:16.119 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-22T23:12:16.119 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:16.120 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-22T23:12:16.120 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-22T23:12:16.120 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-22T23:12:16.120 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:16.120 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:16.120 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-22T23:12:16.120 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-22T23:12:16.120 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-22T23:12:16.121 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:16.121 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-22T23:12:16.121 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 0 2024-05-22T23:12:16.121 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:16.121 INFO:teuthology.orchestra.run.smithi084.stdout: "cct": { 2024-05-22T23:12:16.121 INFO:teuthology.orchestra.run.smithi084.stdout: "total_workers": 0, 2024-05-22T23:12:16.121 INFO:teuthology.orchestra.run.smithi084.stdout: "unhealthy_workers": 0 2024-05-22T23:12:16.121 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:16.122 INFO:teuthology.orchestra.run.smithi084.stdout: "mempool": { 2024-05-22T23:12:16.122 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_bytes": 0, 2024-05-22T23:12:16.122 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_items": 0, 2024-05-22T23:12:16.122 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_bytes": 0, 2024-05-22T23:12:16.122 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_items": 0, 2024-05-22T23:12:16.122 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_bytes": 0, 2024-05-22T23:12:16.122 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_items": 0, 2024-05-22T23:12:16.122 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-22T23:12:16.123 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_items": 0, 2024-05-22T23:12:16.123 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-22T23:12:16.123 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_items": 0, 2024-05-22T23:12:16.123 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_bytes": 0, 2024-05-22T23:12:16.123 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_items": 0, 2024-05-22T23:12:16.123 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-22T23:12:16.123 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_items": 0, 2024-05-22T23:12:16.123 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_bytes": 0, 2024-05-22T23:12:16.123 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_items": 0, 2024-05-22T23:12:16.124 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_bytes": 0, 2024-05-22T23:12:16.124 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_items": 0, 2024-05-22T23:12:16.124 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-22T23:12:16.124 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_items": 0, 2024-05-22T23:12:16.124 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-22T23:12:16.124 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_items": 0, 2024-05-22T23:12:16.124 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_bytes": 0, 2024-05-22T23:12:16.124 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_items": 0, 2024-05-22T23:12:16.125 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_bytes": 0, 2024-05-22T23:12:16.125 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_items": 0, 2024-05-22T23:12:16.125 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-22T23:12:16.125 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_items": 0, 2024-05-22T23:12:16.125 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_bytes": 0, 2024-05-22T23:12:16.125 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_items": 0, 2024-05-22T23:12:16.125 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_bytes": 0, 2024-05-22T23:12:16.125 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_items": 0, 2024-05-22T23:12:16.126 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_bytes": 0, 2024-05-22T23:12:16.126 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_items": 0, 2024-05-22T23:12:16.126 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_bytes": 0, 2024-05-22T23:12:16.126 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_items": 0, 2024-05-22T23:12:16.126 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_bytes": 148977, 2024-05-22T23:12:16.126 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_items": 30, 2024-05-22T23:12:16.126 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_bytes": 0, 2024-05-22T23:12:16.126 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_items": 0, 2024-05-22T23:12:16.127 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_bytes": 0, 2024-05-22T23:12:16.127 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_items": 0, 2024-05-22T23:12:16.127 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_bytes": 0, 2024-05-22T23:12:16.127 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_items": 0, 2024-05-22T23:12:16.127 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_bytes": 0, 2024-05-22T23:12:16.127 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_items": 0, 2024-05-22T23:12:16.127 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_bytes": 0, 2024-05-22T23:12:16.127 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_items": 0, 2024-05-22T23:12:16.128 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_bytes": 0, 2024-05-22T23:12:16.128 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_items": 0, 2024-05-22T23:12:16.128 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_bytes": 0, 2024-05-22T23:12:16.128 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_items": 0, 2024-05-22T23:12:16.128 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_bytes": 928, 2024-05-22T23:12:16.128 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_items": 2, 2024-05-22T23:12:16.128 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_bytes": 0, 2024-05-22T23:12:16.128 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_items": 0, 2024-05-22T23:12:16.128 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_bytes": 0, 2024-05-22T23:12:16.129 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_items": 0 2024-05-22T23:12:16.129 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:16.129 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-22T23:12:16.129 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-05-22T23:12:16.129 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-05-22T23:12:16.129 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-05-22T23:12:16.129 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 220, 2024-05-22T23:12:16.129 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 1411216, 2024-05-22T23:12:16.130 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-05-22T23:12:16.130 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 220, 2024-05-22T23:12:16.130 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-05-22T23:12:16.130 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-05-22T23:12:16.130 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 220, 2024-05-22T23:12:16.130 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 1411216, 2024-05-22T23:12:16.130 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-05-22T23:12:16.130 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-22T23:12:16.131 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-22T23:12:16.131 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-22T23:12:16.131 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:16.131 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:16.131 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:12:16.802 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-05-22T23:12:16.802 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-05-22T23:12:16.802 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-05-22T23:12:16.802 DEBUG:tasks.check_counter:Getting stats from g 2024-05-22T23:12:16.803 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-22T23:12:17.041 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:12:17.381 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:17 smithi084 ceph-mon[32186]: pgmap v325: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-22T23:12:17.381 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:17 smithi084 ceph-mon[32186]: osdmap e99: 12 total, 12 up, 12 in 2024-05-22T23:12:17.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:17 smithi204 ceph-mon[36211]: pgmap v325: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-22T23:12:17.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:17 smithi204 ceph-mon[36211]: osdmap e99: 12 total, 12 up, 12 in 2024-05-22T23:12:17.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:17 smithi096 ceph-mon[36392]: pgmap v325: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-22T23:12:17.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:17 smithi096 ceph-mon[36392]: osdmap e99: 12 total, 12 up, 12 in 2024-05-22T23:12:18.738 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:12:18.738 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-0": { 2024-05-22T23:12:18.738 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 0, 2024-05-22T23:12:18.738 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 2, 2024-05-22T23:12:18.738 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 0, 2024-05-22T23:12:18.739 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 0, 2024-05-22T23:12:18.739 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 4, 2024-05-22T23:12:18.739 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 0, 2024-05-22T23:12:18.739 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.001122376, 2024-05-22T23:12:18.739 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.000000000, 2024-05-22T23:12:18.739 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.000208342, 2024-05-22T23:12:18.739 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-22T23:12:18.739 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:18.740 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-22T23:12:18.740 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-22T23:12:18.740 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-22T23:12:18.740 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:18.740 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:18.740 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-22T23:12:18.740 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-22T23:12:18.740 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-22T23:12:18.740 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:18.741 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-22T23:12:18.741 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 0 2024-05-22T23:12:18.741 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:18.741 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-1": { 2024-05-22T23:12:18.741 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 2, 2024-05-22T23:12:18.741 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 156, 2024-05-22T23:12:18.741 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 320, 2024-05-22T23:12:18.741 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 182928, 2024-05-22T23:12:18.742 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 5, 2024-05-22T23:12:18.742 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 3, 2024-05-22T23:12:18.742 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.012390102, 2024-05-22T23:12:18.742 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.008373671, 2024-05-22T23:12:18.742 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.001792859, 2024-05-22T23:12:18.742 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-22T23:12:18.742 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:18.742 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 153, 2024-05-22T23:12:18.743 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.013870962, 2024-05-22T23:12:18.743 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000090659 2024-05-22T23:12:18.743 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:18.743 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:18.743 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-22T23:12:18.743 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-22T23:12:18.743 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-22T23:12:18.743 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:18.744 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-22T23:12:18.744 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 182928 2024-05-22T23:12:18.744 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:18.744 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-2": { 2024-05-22T23:12:18.744 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 218, 2024-05-22T23:12:18.744 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 206, 2024-05-22T23:12:18.744 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 1443856, 2024-05-22T23:12:18.744 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 125776, 2024-05-22T23:12:18.745 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 2, 2024-05-22T23:12:18.745 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 1, 2024-05-22T23:12:18.745 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.036998885, 2024-05-22T23:12:18.745 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.011766579, 2024-05-22T23:12:18.745 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.030702474, 2024-05-22T23:12:18.745 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.003682742, 2024-05-22T23:12:18.745 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:18.745 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 206, 2024-05-22T23:12:18.746 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.009057803, 2024-05-22T23:12:18.746 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000043969 2024-05-22T23:12:18.746 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:18.746 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:18.746 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-22T23:12:18.746 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-22T23:12:18.746 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-22T23:12:18.746 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:18.747 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 1443856, 2024-05-22T23:12:18.747 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 125776 2024-05-22T23:12:18.747 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:18.747 INFO:teuthology.orchestra.run.smithi084.stdout: "cct": { 2024-05-22T23:12:18.747 INFO:teuthology.orchestra.run.smithi084.stdout: "total_workers": 0, 2024-05-22T23:12:18.747 INFO:teuthology.orchestra.run.smithi084.stdout: "unhealthy_workers": 0 2024-05-22T23:12:18.747 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:18.747 INFO:teuthology.orchestra.run.smithi084.stdout: "mempool": { 2024-05-22T23:12:18.747 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_bytes": 0, 2024-05-22T23:12:18.748 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_items": 0, 2024-05-22T23:12:18.748 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_bytes": 0, 2024-05-22T23:12:18.748 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_items": 0, 2024-05-22T23:12:18.748 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_bytes": 0, 2024-05-22T23:12:18.748 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_items": 0, 2024-05-22T23:12:18.748 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-22T23:12:18.748 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_items": 0, 2024-05-22T23:12:18.748 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-22T23:12:18.749 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_items": 0, 2024-05-22T23:12:18.749 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_bytes": 0, 2024-05-22T23:12:18.749 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_items": 0, 2024-05-22T23:12:18.749 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-22T23:12:18.749 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_items": 0, 2024-05-22T23:12:18.749 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_bytes": 0, 2024-05-22T23:12:18.749 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_items": 0, 2024-05-22T23:12:18.749 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_bytes": 0, 2024-05-22T23:12:18.750 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_items": 0, 2024-05-22T23:12:18.750 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-22T23:12:18.750 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_items": 0, 2024-05-22T23:12:18.750 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-22T23:12:18.750 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_items": 0, 2024-05-22T23:12:18.750 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_bytes": 0, 2024-05-22T23:12:18.750 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_items": 0, 2024-05-22T23:12:18.751 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_bytes": 0, 2024-05-22T23:12:18.751 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_items": 0, 2024-05-22T23:12:18.751 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-22T23:12:18.751 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_items": 0, 2024-05-22T23:12:18.751 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_bytes": 0, 2024-05-22T23:12:18.751 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_items": 0, 2024-05-22T23:12:18.751 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_bytes": 0, 2024-05-22T23:12:18.751 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_items": 0, 2024-05-22T23:12:18.751 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_bytes": 0, 2024-05-22T23:12:18.752 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_items": 0, 2024-05-22T23:12:18.752 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_bytes": 0, 2024-05-22T23:12:18.752 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_items": 0, 2024-05-22T23:12:18.752 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_bytes": 148967, 2024-05-22T23:12:18.752 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_items": 29, 2024-05-22T23:12:18.752 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_bytes": 0, 2024-05-22T23:12:18.752 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_items": 0, 2024-05-22T23:12:18.752 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_bytes": 0, 2024-05-22T23:12:18.753 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_items": 0, 2024-05-22T23:12:18.753 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_bytes": 0, 2024-05-22T23:12:18.753 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_items": 0, 2024-05-22T23:12:18.753 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_bytes": 0, 2024-05-22T23:12:18.753 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_items": 0, 2024-05-22T23:12:18.753 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_bytes": 0, 2024-05-22T23:12:18.753 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_items": 0, 2024-05-22T23:12:18.753 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_bytes": 0, 2024-05-22T23:12:18.754 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_items": 0, 2024-05-22T23:12:18.754 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_bytes": 0, 2024-05-22T23:12:18.754 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_items": 0, 2024-05-22T23:12:18.754 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_bytes": 928, 2024-05-22T23:12:18.754 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_items": 2, 2024-05-22T23:12:18.754 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_bytes": 0, 2024-05-22T23:12:18.754 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_items": 0, 2024-05-22T23:12:18.754 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_bytes": 0, 2024-05-22T23:12:18.755 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_items": 0 2024-05-22T23:12:18.755 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:18.755 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-22T23:12:18.755 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-05-22T23:12:18.755 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-05-22T23:12:18.755 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-05-22T23:12:18.755 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 216, 2024-05-22T23:12:18.755 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 1410696, 2024-05-22T23:12:18.756 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-05-22T23:12:18.756 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 216, 2024-05-22T23:12:18.756 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-05-22T23:12:18.756 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-05-22T23:12:18.756 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 216, 2024-05-22T23:12:18.756 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 1410696, 2024-05-22T23:12:18.756 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-05-22T23:12:18.756 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-22T23:12:18.757 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-22T23:12:18.757 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-22T23:12:18.757 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:18.757 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:18.757 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:12:19.381 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:19 smithi084 ceph-mon[32186]: pgmap v327: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-22T23:12:19.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:19 smithi204 ceph-mon[36211]: pgmap v327: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-22T23:12:19.462 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-05-22T23:12:19.463 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-05-22T23:12:19.463 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-05-22T23:12:19.463 DEBUG:tasks.check_counter:Getting stats from j 2024-05-22T23:12:19.464 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-22T23:12:19.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:19 smithi096 ceph-mon[36392]: pgmap v327: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-22T23:12:19.707 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.a/config 2024-05-22T23:12:21.379 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:21 smithi084 ceph-mon[32186]: pgmap v328: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-22T23:12:21.412 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-22T23:12:21.413 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-0": { 2024-05-22T23:12:21.413 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 0, 2024-05-22T23:12:21.413 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 2, 2024-05-22T23:12:21.413 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 0, 2024-05-22T23:12:21.413 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 0, 2024-05-22T23:12:21.413 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 4, 2024-05-22T23:12:21.413 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 0, 2024-05-22T23:12:21.414 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.001154664, 2024-05-22T23:12:21.414 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.000000000, 2024-05-22T23:12:21.414 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.000221865, 2024-05-22T23:12:21.414 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-22T23:12:21.414 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:21.414 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-22T23:12:21.415 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-22T23:12:21.415 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-22T23:12:21.415 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:21.415 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:21.415 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-22T23:12:21.415 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-22T23:12:21.415 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-22T23:12:21.415 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:21.416 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-22T23:12:21.416 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 0 2024-05-22T23:12:21.416 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:21.416 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-1": { 2024-05-22T23:12:21.416 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 2, 2024-05-22T23:12:21.416 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 161, 2024-05-22T23:12:21.416 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 320, 2024-05-22T23:12:21.416 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 183808, 2024-05-22T23:12:21.417 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 5, 2024-05-22T23:12:21.417 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 3, 2024-05-22T23:12:21.417 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.011406770, 2024-05-22T23:12:21.417 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.007505117, 2024-05-22T23:12:21.417 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.002019563, 2024-05-22T23:12:21.417 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-22T23:12:21.417 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:21.417 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 158, 2024-05-22T23:12:21.418 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.014918904, 2024-05-22T23:12:21.418 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000094423 2024-05-22T23:12:21.418 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:21.418 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:21.418 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-22T23:12:21.418 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-22T23:12:21.418 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-22T23:12:21.419 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:21.419 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-22T23:12:21.419 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 183808 2024-05-22T23:12:21.419 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:21.419 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-2": { 2024-05-22T23:12:21.419 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 224, 2024-05-22T23:12:21.419 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 212, 2024-05-22T23:12:21.419 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 1445488, 2024-05-22T23:12:21.420 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 129632, 2024-05-22T23:12:21.420 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 2, 2024-05-22T23:12:21.420 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 1, 2024-05-22T23:12:21.420 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.040001902, 2024-05-22T23:12:21.420 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.012161617, 2024-05-22T23:12:21.420 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.032900425, 2024-05-22T23:12:21.420 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.003895905, 2024-05-22T23:12:21.420 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:21.420 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 212, 2024-05-22T23:12:21.421 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.009189547, 2024-05-22T23:12:21.421 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000043346 2024-05-22T23:12:21.421 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:21.421 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:21.421 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-22T23:12:21.421 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-22T23:12:21.421 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-22T23:12:21.422 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:21.422 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 1445488, 2024-05-22T23:12:21.422 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 129632 2024-05-22T23:12:21.422 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:21.422 INFO:teuthology.orchestra.run.smithi084.stdout: "cct": { 2024-05-22T23:12:21.422 INFO:teuthology.orchestra.run.smithi084.stdout: "total_workers": 0, 2024-05-22T23:12:21.422 INFO:teuthology.orchestra.run.smithi084.stdout: "unhealthy_workers": 0 2024-05-22T23:12:21.423 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:21.423 INFO:teuthology.orchestra.run.smithi084.stdout: "mempool": { 2024-05-22T23:12:21.423 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_bytes": 0, 2024-05-22T23:12:21.423 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_items": 0, 2024-05-22T23:12:21.423 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_bytes": 0, 2024-05-22T23:12:21.423 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_items": 0, 2024-05-22T23:12:21.423 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_bytes": 0, 2024-05-22T23:12:21.423 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_items": 0, 2024-05-22T23:12:21.423 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-22T23:12:21.424 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_items": 0, 2024-05-22T23:12:21.424 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-22T23:12:21.424 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_items": 0, 2024-05-22T23:12:21.424 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_bytes": 0, 2024-05-22T23:12:21.424 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_items": 0, 2024-05-22T23:12:21.424 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-22T23:12:21.424 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_items": 0, 2024-05-22T23:12:21.424 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_bytes": 0, 2024-05-22T23:12:21.425 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_items": 0, 2024-05-22T23:12:21.425 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_bytes": 0, 2024-05-22T23:12:21.425 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_items": 0, 2024-05-22T23:12:21.425 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-22T23:12:21.425 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_items": 0, 2024-05-22T23:12:21.425 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-22T23:12:21.425 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_items": 0, 2024-05-22T23:12:21.425 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_bytes": 0, 2024-05-22T23:12:21.425 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_items": 0, 2024-05-22T23:12:21.426 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_bytes": 0, 2024-05-22T23:12:21.426 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_items": 0, 2024-05-22T23:12:21.426 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-22T23:12:21.426 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_items": 0, 2024-05-22T23:12:21.426 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_bytes": 0, 2024-05-22T23:12:21.426 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_items": 0, 2024-05-22T23:12:21.426 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_bytes": 0, 2024-05-22T23:12:21.426 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_items": 0, 2024-05-22T23:12:21.427 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_bytes": 0, 2024-05-22T23:12:21.427 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_items": 0, 2024-05-22T23:12:21.427 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_bytes": 0, 2024-05-22T23:12:21.427 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_items": 0, 2024-05-22T23:12:21.427 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_bytes": 149017, 2024-05-22T23:12:21.427 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_items": 31, 2024-05-22T23:12:21.427 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_bytes": 0, 2024-05-22T23:12:21.427 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_items": 0, 2024-05-22T23:12:21.427 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_bytes": 0, 2024-05-22T23:12:21.427 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_items": 0, 2024-05-22T23:12:21.428 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_bytes": 0, 2024-05-22T23:12:21.428 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_items": 0, 2024-05-22T23:12:21.428 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_bytes": 0, 2024-05-22T23:12:21.428 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_items": 0, 2024-05-22T23:12:21.428 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_bytes": 0, 2024-05-22T23:12:21.428 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_items": 0, 2024-05-22T23:12:21.428 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_bytes": 0, 2024-05-22T23:12:21.428 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_items": 0, 2024-05-22T23:12:21.428 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_bytes": 0, 2024-05-22T23:12:21.429 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_items": 0, 2024-05-22T23:12:21.429 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_bytes": 928, 2024-05-22T23:12:21.429 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_items": 2, 2024-05-22T23:12:21.429 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_bytes": 0, 2024-05-22T23:12:21.429 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_items": 0, 2024-05-22T23:12:21.429 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_bytes": 0, 2024-05-22T23:12:21.429 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_items": 0 2024-05-22T23:12:21.429 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-22T23:12:21.429 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-22T23:12:21.430 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-05-22T23:12:21.430 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-05-22T23:12:21.430 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-05-22T23:12:21.430 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 222, 2024-05-22T23:12:21.430 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 1411476, 2024-05-22T23:12:21.430 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-05-22T23:12:21.430 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 222, 2024-05-22T23:12:21.430 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-05-22T23:12:21.431 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-05-22T23:12:21.431 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 222, 2024-05-22T23:12:21.431 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 1411476, 2024-05-22T23:12:21.431 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-05-22T23:12:21.431 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-22T23:12:21.432 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-22T23:12:21.432 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-22T23:12:21.432 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:21.432 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-22T23:12:21.432 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-22T23:12:21.432 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:21 smithi204 ceph-mon[36211]: pgmap v328: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-22T23:12:21.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:21 smithi096 ceph-mon[36392]: pgmap v328: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-22T23:12:22.129 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-22T23:12:22.129 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-22T23:12:22.129 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-22T23:12:22.129 DEBUG:tasks.check_counter:Getting stats from b 2024-05-22T23:12:22.129 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-22T23:12:22.380 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.b/config 2024-05-22T23:12:23.381 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:23 smithi084 ceph-mon[32186]: pgmap v329: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-22T23:12:23.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:23 smithi204 ceph-mon[36211]: pgmap v329: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-22T23:12:23.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:23 smithi096 ceph-mon[36392]: pgmap v329: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-22T23:12:24.258 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-05-22T23:12:24.259 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-05-22T23:12:24.259 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 39479, 2024-05-22T23:12:24.259 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 38890, 2024-05-22T23:12:24.259 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 10023942, 2024-05-22T23:12:24.259 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 263844210, 2024-05-22T23:12:24.259 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 14, 2024-05-22T23:12:24.259 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 5, 2024-05-22T23:12:24.260 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 3.337966637, 2024-05-22T23:12:24.260 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 1.399898343, 2024-05-22T23:12:24.260 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.645950023, 2024-05-22T23:12:24.260 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 1.058977886, 2024-05-22T23:12:24.260 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:24.260 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 38888, 2024-05-22T23:12:24.260 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 4.253715212, 2024-05-22T23:12:24.260 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000109383 2024-05-22T23:12:24.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.261 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:24.261 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 915, 2024-05-22T23:12:24.261 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000148283, 2024-05-22T23:12:24.261 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000162 2024-05-22T23:12:24.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.261 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 330016, 2024-05-22T23:12:24.261 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 99232 2024-05-22T23:12:24.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.262 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-05-22T23:12:24.262 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 40585, 2024-05-22T23:12:24.262 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 40491, 2024-05-22T23:12:24.262 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 14969532, 2024-05-22T23:12:24.262 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 278787848, 2024-05-22T23:12:24.262 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 173, 2024-05-22T23:12:24.262 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 5, 2024-05-22T23:12:24.263 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 3.249241871, 2024-05-22T23:12:24.263 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 1.273235998, 2024-05-22T23:12:24.263 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.708784194, 2024-05-22T23:12:24.263 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 1.046614352, 2024-05-22T23:12:24.263 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:24.263 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 40490, 2024-05-22T23:12:24.263 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 3.566529525, 2024-05-22T23:12:24.263 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000088084 2024-05-22T23:12:24.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.264 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:24.264 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 271, 2024-05-22T23:12:24.264 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000039581, 2024-05-22T23:12:24.264 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000146 2024-05-22T23:12:24.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.264 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1521904, 2024-05-22T23:12:24.264 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 162864 2024-05-22T23:12:24.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.265 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-05-22T23:12:24.265 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 280645, 2024-05-22T23:12:24.265 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 457124, 2024-05-22T23:12:24.265 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 82965758, 2024-05-22T23:12:24.265 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 439495116, 2024-05-22T23:12:24.265 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 15, 2024-05-22T23:12:24.265 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 5, 2024-05-22T23:12:24.266 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 12.163750841, 2024-05-22T23:12:24.266 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 6.230822286, 2024-05-22T23:12:24.266 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 7.552484861, 2024-05-22T23:12:24.266 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.552764035, 2024-05-22T23:12:24.266 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:24.266 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 457122, 2024-05-22T23:12:24.267 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 10.354644010, 2024-05-22T23:12:24.267 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000022651 2024-05-22T23:12:24.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.267 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:24.267 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 413097, 2024-05-22T23:12:24.267 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.064427817, 2024-05-22T23:12:24.267 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000155 2024-05-22T23:12:24.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.268 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 99856, 2024-05-22T23:12:24.268 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 479600 2024-05-22T23:12:24.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.268 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-05-22T23:12:24.268 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-05-22T23:12:24.268 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-05-22T23:12:24.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.268 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-05-22T23:12:24.268 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-05-22T23:12:24.269 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-05-22T23:12:24.269 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 104409, 2024-05-22T23:12:24.269 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 96.139430191, 2024-05-22T23:12:24.269 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000920796 2024-05-22T23:12:24.269 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-22T23:12:24.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.269 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-05-22T23:12:24.269 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-05-22T23:12:24.270 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-05-22T23:12:24.270 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 28452, 2024-05-22T23:12:24.270 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 6.903321937, 2024-05-22T23:12:24.270 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000242630 2024-05-22T23:12:24.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-22T23:12:24.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.270 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-05-22T23:12:24.270 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 195236, 2024-05-22T23:12:24.271 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 195236, 2024-05-22T23:12:24.271 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-05-22T23:12:24.271 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 195236, 2024-05-22T23:12:24.271 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 258.494807739, 2024-05-22T23:12:24.271 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001324012 2024-05-22T23:12:24.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.271 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-05-22T23:12:24.272 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 0, 2024-05-22T23:12:24.272 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-05-22T23:12:24.272 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-05-22T23:12:24.272 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 234, 2024-05-22T23:12:24.272 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 19, 2024-05-22T23:12:24.272 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 10, 2024-05-22T23:12:24.272 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 3051, 2024-05-22T23:12:24.273 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 900, 2024-05-22T23:12:24.273 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 2151, 2024-05-22T23:12:24.273 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 0, 2024-05-22T23:12:24.273 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 1765, 2024-05-22T23:12:24.273 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 0, 2024-05-22T23:12:24.273 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 71, 2024-05-22T23:12:24.273 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 72, 2024-05-22T23:12:24.273 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 2, 2024-05-22T23:12:24.274 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 270103, 2024-05-22T23:12:24.274 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 161386, 2024-05-22T23:12:24.274 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 0, 2024-05-22T23:12:24.274 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 0, 2024-05-22T23:12:24.274 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-05-22T23:12:24.274 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-05-22T23:12:24.274 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 2363, 2024-05-22T23:12:24.274 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-05-22T23:12:24.274 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-05-22T23:12:24.275 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 0, 2024-05-22T23:12:24.275 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 0, 2024-05-22T23:12:24.275 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 0, 2024-05-22T23:12:24.275 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 0, 2024-05-22T23:12:24.275 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-05-22T23:12:24.275 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 1684, 2024-05-22T23:12:24.275 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-05-22T23:12:24.275 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 68, 2024-05-22T23:12:24.276 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 21196213, 2024-05-22T23:12:24.276 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 6, 2024-05-22T23:12:24.276 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 1684, 2024-05-22T23:12:24.276 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-05-22T23:12:24.276 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-05-22T23:12:24.276 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-05-22T23:12:24.276 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 409, 2024-05-22T23:12:24.276 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 22, 2024-05-22T23:12:24.276 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 387, 2024-05-22T23:12:24.277 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 1275, 2024-05-22T23:12:24.277 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 0, 2024-05-22T23:12:24.277 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 7162, 2024-05-22T23:12:24.277 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 139590, 2024-05-22T23:12:24.277 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-05-22T23:12:24.277 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-22T23:12:24.277 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-22T23:12:24.277 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 52777, 2024-05-22T23:12:24.278 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 26572, 2024-05-22T23:12:24.278 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 13897, 2024-05-22T23:12:24.278 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 121632 2024-05-22T23:12:24.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.278 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-05-22T23:12:24.278 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 0, 2024-05-22T23:12:24.278 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 0, 2024-05-22T23:12:24.278 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 0, 2024-05-22T23:12:24.279 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 0, 2024-05-22T23:12:24.279 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 0, 2024-05-22T23:12:24.279 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 0, 2024-05-22T23:12:24.279 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-05-22T23:12:24.279 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-05-22T23:12:24.279 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 34004, 2024-05-22T23:12:24.279 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 34004, 2024-05-22T23:12:24.279 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-05-22T23:12:24.280 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-05-22T23:12:24.280 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-05-22T23:12:24.280 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-05-22T23:12:24.280 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-05-22T23:12:24.280 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-05-22T23:12:24.280 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-05-22T23:12:24.280 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 0, 2024-05-22T23:12:24.280 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 0, 2024-05-22T23:12:24.281 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 22, 2024-05-22T23:12:24.281 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 0, 2024-05-22T23:12:24.281 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-05-22T23:12:24.281 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 29, 2024-05-22T23:12:24.281 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-05-22T23:12:24.281 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-05-22T23:12:24.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.281 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-05-22T23:12:24.282 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-05-22T23:12:24.282 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 153822, 2024-05-22T23:12:24.282 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 65667, 2024-05-22T23:12:24.282 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 61451, 2024-05-22T23:12:24.282 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 92371, 2024-05-22T23:12:24.282 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-05-22T23:12:24.282 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 4216, 2024-05-22T23:12:24.282 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 221, 2024-05-22T23:12:24.283 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 93, 2024-05-22T23:12:24.283 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 87, 2024-05-22T23:12:24.283 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 134, 2024-05-22T23:12:24.283 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 8, 2024-05-22T23:12:24.283 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-05-22T23:12:24.283 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 6, 2024-05-22T23:12:24.283 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 177477184, 2024-05-22T23:12:24.283 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 577888240, 2024-05-22T23:12:24.284 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 0, 2024-05-22T23:12:24.284 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-05-22T23:12:24.284 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 52894, 2024-05-22T23:12:24.284 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 99.105491705, 2024-05-22T23:12:24.284 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001873662 2024-05-22T23:12:24.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.284 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 0 2024-05-22T23:12:24.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.284 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-05-22T23:12:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 443, 2024-05-22T23:12:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 34469, 2024-05-22T23:12:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 34026, 2024-05-22T23:12:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 109, 2024-05-22T23:12:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 194, 2024-05-22T23:12:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 85, 2024-05-22T23:12:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 3051, 2024-05-22T23:12:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 69561, 2024-05-22T23:12:24.286 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 66510, 2024-05-22T23:12:24.286 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 72, 2024-05-22T23:12:24.286 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 35629, 2024-05-22T23:12:24.286 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 35557, 2024-05-22T23:12:24.286 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 87408, 2024-05-22T23:12:24.286 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 231804 2024-05-22T23:12:24.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.286 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-05-22T23:12:24.287 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 239464, 2024-05-22T23:12:24.287 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 0, 2024-05-22T23:12:24.287 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 195236, 2024-05-22T23:12:24.287 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 4802, 2024-05-22T23:12:24.287 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 0, 2024-05-22T23:12:24.287 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-05-22T23:12:24.287 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 34072, 2024-05-22T23:12:24.287 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 38.843281399, 2024-05-22T23:12:24.288 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001140035 2024-05-22T23:12:24.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.288 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-05-22T23:12:24.288 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 343, 2024-05-22T23:12:24.288 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.711224138, 2024-05-22T23:12:24.288 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002073539 2024-05-22T23:12:24.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.288 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-05-22T23:12:24.289 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:24.289 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:24.289 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:24.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.289 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-05-22T23:12:24.289 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:24.289 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:24.289 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:24.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.290 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-05-22T23:12:24.290 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 65093, 2024-05-22T23:12:24.290 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 28.940909762, 2024-05-22T23:12:24.290 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000444608 2024-05-22T23:12:24.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.290 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-05-22T23:12:24.290 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:24.291 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:24.291 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:24.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.291 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-05-22T23:12:24.291 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2, 2024-05-22T23:12:24.291 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000438527, 2024-05-22T23:12:24.291 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000219263 2024-05-22T23:12:24.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.292 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-05-22T23:12:24.292 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:24.292 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:24.292 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:24.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.292 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-05-22T23:12:24.292 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:24.292 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:24.293 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:24.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.293 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-05-22T23:12:24.293 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:24.293 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:24.294 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:24.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.294 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-05-22T23:12:24.294 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 11, 2024-05-22T23:12:24.294 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.005809454, 2024-05-22T23:12:24.295 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000528132 2024-05-22T23:12:24.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.295 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-05-22T23:12:24.295 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 20, 2024-05-22T23:12:24.295 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.020673433, 2024-05-22T23:12:24.295 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001033671 2024-05-22T23:12:24.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.295 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-05-22T23:12:24.296 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:24.296 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:24.296 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:24.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.296 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-05-22T23:12:24.296 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 11, 2024-05-22T23:12:24.296 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.084081253, 2024-05-22T23:12:24.296 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.007643750 2024-05-22T23:12:24.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.297 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-05-22T23:12:24.297 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1, 2024-05-22T23:12:24.297 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000312404, 2024-05-22T23:12:24.297 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000312404 2024-05-22T23:12:24.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.297 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-05-22T23:12:24.297 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 53461, 2024-05-22T23:12:24.298 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 132.868392016, 2024-05-22T23:12:24.298 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002485333 2024-05-22T23:12:24.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.298 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-05-22T23:12:24.298 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 7113, 2024-05-22T23:12:24.298 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 11.128035497, 2024-05-22T23:12:24.298 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001564464 2024-05-22T23:12:24.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.299 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-05-22T23:12:24.299 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:24.299 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:24.299 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:24.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.299 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-05-22T23:12:24.299 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:24.300 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:24.300 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:24.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.300 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-05-22T23:12:24.300 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:24.300 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:24.300 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:24.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.301 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-05-22T23:12:24.301 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 5, 2024-05-22T23:12:24.301 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.029290562, 2024-05-22T23:12:24.301 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.005858112 2024-05-22T23:12:24.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.301 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-05-22T23:12:24.301 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:24.301 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:24.302 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:24.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.302 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-05-22T23:12:24.302 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1, 2024-05-22T23:12:24.302 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.001367215, 2024-05-22T23:12:24.302 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001367215 2024-05-22T23:12:24.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.302 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-05-22T23:12:24.303 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:24.303 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:24.303 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:24.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.303 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-05-22T23:12:24.303 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1092, 2024-05-22T23:12:24.303 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.597370667, 2024-05-22T23:12:24.303 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000547042 2024-05-22T23:12:24.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.304 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-05-22T23:12:24.304 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:24.304 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:24.304 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:24.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.304 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-05-22T23:12:24.304 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 7, 2024-05-22T23:12:24.305 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.016155275, 2024-05-22T23:12:24.305 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002307896 2024-05-22T23:12:24.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.305 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-05-22T23:12:24.305 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:24.305 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:24.305 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:24.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.305 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-05-22T23:12:24.306 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 34004, 2024-05-22T23:12:24.306 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 45.247466137, 2024-05-22T23:12:24.306 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001330651 2024-05-22T23:12:24.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.306 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-05-22T23:12:24.306 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-05-22T23:12:24.306 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-05-22T23:12:24.306 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:24.307 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:24.307 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:24.307 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-22T23:12:24.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.307 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-05-22T23:12:24.307 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 2, 2024-05-22T23:12:24.307 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 5, 2024-05-22T23:12:24.307 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 3, 2024-05-22T23:12:24.308 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 2, 2024-05-22T23:12:24.308 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-05-22T23:12:24.308 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 6019, 2024-05-22T23:12:24.308 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 3009, 2024-05-22T23:12:24.308 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 627, 2024-05-22T23:12:24.308 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-05-22T23:12:24.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.308 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-05-22T23:12:24.309 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-05-22T23:12:24.309 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-05-22T23:12:24.309 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-05-22T23:12:24.309 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-05-22T23:12:24.309 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-05-22T23:12:24.309 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-05-22T23:12:24.309 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-22T23:12:24.309 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-05-22T23:12:24.310 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-22T23:12:24.310 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-05-22T23:12:24.310 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-05-22T23:12:24.310 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-05-22T23:12:24.310 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-22T23:12:24.310 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-05-22T23:12:24.310 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-05-22T23:12:24.310 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-05-22T23:12:24.311 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-05-22T23:12:24.311 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-05-22T23:12:24.311 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-22T23:12:24.311 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-05-22T23:12:24.311 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-22T23:12:24.311 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-05-22T23:12:24.311 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-05-22T23:12:24.311 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-05-22T23:12:24.312 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-05-22T23:12:24.312 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-05-22T23:12:24.312 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-22T23:12:24.312 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-05-22T23:12:24.312 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-05-22T23:12:24.312 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-05-22T23:12:24.312 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-05-22T23:12:24.312 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-05-22T23:12:24.313 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-05-22T23:12:24.313 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-05-22T23:12:24.313 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-05-22T23:12:24.313 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-05-22T23:12:24.313 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 328213, 2024-05-22T23:12:24.313 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 372, 2024-05-22T23:12:24.313 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 440, 2024-05-22T23:12:24.313 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 5, 2024-05-22T23:12:24.314 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-05-22T23:12:24.314 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-05-22T23:12:24.314 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-05-22T23:12:24.314 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-05-22T23:12:24.314 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-05-22T23:12:24.314 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-05-22T23:12:24.314 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 8224, 2024-05-22T23:12:24.315 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 171, 2024-05-22T23:12:24.315 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-05-22T23:12:24.315 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-05-22T23:12:24.315 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-05-22T23:12:24.315 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-05-22T23:12:24.315 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 3634454, 2024-05-22T23:12:24.315 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 39777, 2024-05-22T23:12:24.315 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-05-22T23:12:24.316 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-05-22T23:12:24.316 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-05-22T23:12:24.316 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-05-22T23:12:24.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.316 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-05-22T23:12:24.316 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-05-22T23:12:24.316 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-05-22T23:12:24.316 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 91998, 2024-05-22T23:12:24.317 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 582577786, 2024-05-22T23:12:24.317 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 31, 2024-05-22T23:12:24.317 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 91998, 2024-05-22T23:12:24.317 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-05-22T23:12:24.317 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 91998, 2024-05-22T23:12:24.317 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 304.985089531, 2024-05-22T23:12:24.317 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.003315127 2024-05-22T23:12:24.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.318 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-05-22T23:12:24.318 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-05-22T23:12:24.318 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 91998, 2024-05-22T23:12:24.318 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 93101 2024-05-22T23:12:24.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.318 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 91998, 2024-05-22T23:12:24.318 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 2524, 2024-05-22T23:12:24.318 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 89474, 2024-05-22T23:12:24.319 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-05-22T23:12:24.319 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-05-22T23:12:24.319 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 40, 2024-05-22T23:12:24.319 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 47, 2024-05-22T23:12:24.319 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 818, 2024-05-22T23:12:24.319 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 53690, 2024-05-22T23:12:24.319 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 349, 2024-05-22T23:12:24.320 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-05-22T23:12:24.320 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-05-22T23:12:24.320 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-05-22T23:12:24.320 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 19, 2024-05-22T23:12:24.320 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 34759, 2024-05-22T23:12:24.320 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-05-22T23:12:24.320 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-05-22T23:12:24.320 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-05-22T23:12:24.320 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 1706, 2024-05-22T23:12:24.321 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 94, 2024-05-22T23:12:24.321 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-05-22T23:12:24.321 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-05-22T23:12:24.321 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-05-22T23:12:24.321 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-05-22T23:12:24.321 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-05-22T23:12:24.321 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-05-22T23:12:24.321 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-05-22T23:12:24.322 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-05-22T23:12:24.322 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-05-22T23:12:24.322 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-05-22T23:12:24.322 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-05-22T23:12:24.322 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-05-22T23:12:24.322 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-05-22T23:12:24.322 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-05-22T23:12:24.322 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-05-22T23:12:24.323 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-05-22T23:12:24.323 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-05-22T23:12:24.323 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-05-22T23:12:24.323 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-05-22T23:12:24.323 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-05-22T23:12:24.323 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-05-22T23:12:24.323 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-05-22T23:12:24.323 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-05-22T23:12:24.324 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-05-22T23:12:24.324 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-05-22T23:12:24.324 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-05-22T23:12:24.324 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 99, 2024-05-22T23:12:24.324 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-05-22T23:12:24.324 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 14, 2024-05-22T23:12:24.324 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 12, 2024-05-22T23:12:24.324 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 12, 2024-05-22T23:12:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 0, 2024-05-22T23:12:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-05-22T23:12:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 1185, 2024-05-22T23:12:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 0, 2024-05-22T23:12:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 394 2024-05-22T23:12:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-05-22T23:12:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 1, 2024-05-22T23:12:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-05-22T23:12:24.326 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-05-22T23:12:24.326 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-05-22T23:12:24.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.326 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-05-22T23:12:24.326 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 0 2024-05-22T23:12:24.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.326 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-05-22T23:12:24.326 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-05-22T23:12:24.327 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 32, 2024-05-22T23:12:24.327 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-05-22T23:12:24.327 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 32, 2024-05-22T23:12:24.327 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 34004, 2024-05-22T23:12:24.327 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 34004, 2024-05-22T23:12:24.327 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-05-22T23:12:24.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.327 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-22T23:12:24.328 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-22T23:12:24.328 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-05-22T23:12:24.328 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-22T23:12:24.328 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 360705, 2024-05-22T23:12:24.328 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 79702488, 2024-05-22T23:12:24.328 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-22T23:12:24.328 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 360705, 2024-05-22T23:12:24.328 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-22T23:12:24.329 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-22T23:12:24.329 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 360705, 2024-05-22T23:12:24.329 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 79702488, 2024-05-22T23:12:24.329 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-22T23:12:24.329 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:24.329 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:24.329 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:24.329 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-22T23:12:24.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.330 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-05-22T23:12:24.330 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-22T23:12:24.330 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-05-22T23:12:24.330 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-22T23:12:24.330 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-05-22T23:12:24.330 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-05-22T23:12:24.330 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-22T23:12:24.331 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-05-22T23:12:24.331 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 91998, 2024-05-22T23:12:24.331 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 586014902, 2024-05-22T23:12:24.331 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 91998, 2024-05-22T23:12:24.331 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 586014902, 2024-05-22T23:12:24.331 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-22T23:12:24.331 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:24.331 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:24.331 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:24.332 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-22T23:12:24.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.332 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-05-22T23:12:24.332 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-22T23:12:24.332 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-05-22T23:12:24.332 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-22T23:12:24.332 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-05-22T23:12:24.332 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-05-22T23:12:24.332 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-22T23:12:24.333 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-05-22T23:12:24.333 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 91998, 2024-05-22T23:12:24.333 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 91998, 2024-05-22T23:12:24.333 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 91998, 2024-05-22T23:12:24.333 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 91998, 2024-05-22T23:12:24.333 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-22T23:12:24.333 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:24.333 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:24.334 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:24.334 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-22T23:12:24.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.334 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-05-22T23:12:24.334 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-22T23:12:24.334 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-05-22T23:12:24.334 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-22T23:12:24.334 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 34004, 2024-05-22T23:12:24.335 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 3437116, 2024-05-22T23:12:24.335 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-22T23:12:24.335 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 34004, 2024-05-22T23:12:24.335 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-22T23:12:24.335 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-22T23:12:24.335 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 796, 2024-05-22T23:12:24.335 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 3437116, 2024-05-22T23:12:24.335 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-22T23:12:24.336 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:24.336 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:24.336 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:24.336 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-22T23:12:24.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:24.336 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x559315d42500": { 2024-05-22T23:12:24.336 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-22T23:12:24.336 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-05-22T23:12:24.337 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-22T23:12:24.337 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 153822, 2024-05-22T23:12:24.337 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 573693936, 2024-05-22T23:12:24.337 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-22T23:12:24.337 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 153822, 2024-05-22T23:12:24.337 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-22T23:12:24.337 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-22T23:12:24.337 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 52894, 2024-05-22T23:12:24.338 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 573693936, 2024-05-22T23:12:24.338 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-22T23:12:24.338 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:24.338 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:24.338 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:24.338 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-22T23:12:24.338 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-22T23:12:24.338 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-05-22T23:12:25.076 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-05-22T23:12:25.077 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-05-22T23:12:25.077 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-05-22T23:12:25.077 DEBUG:tasks.check_counter:Getting stats from e 2024-05-22T23:12:25.077 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-22T23:12:25.305 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:25 smithi096 ceph-mon[36392]: pgmap v330: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-22T23:12:25.322 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.b/config 2024-05-22T23:12:25.381 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:25 smithi084 ceph-mon[32186]: pgmap v330: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-22T23:12:25.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:25 smithi204 ceph-mon[36211]: pgmap v330: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-22T23:12:26.985 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-05-22T23:12:26.986 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-05-22T23:12:26.986 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 221, 2024-05-22T23:12:26.986 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 211, 2024-05-22T23:12:26.986 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1442832, 2024-05-22T23:12:26.986 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 129344, 2024-05-22T23:12:26.986 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-05-22T23:12:26.986 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 0, 2024-05-22T23:12:26.987 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.039083095, 2024-05-22T23:12:26.987 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.011021821, 2024-05-22T23:12:26.987 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.031232899, 2024-05-22T23:12:26.987 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.004700257, 2024-05-22T23:12:26.987 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:26.987 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 210, 2024-05-22T23:12:26.987 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.008532561, 2024-05-22T23:12:26.987 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000040631 2024-05-22T23:12:26.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:26.988 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:26.988 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:26.988 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:26.988 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:26.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:26.988 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1442832, 2024-05-22T23:12:26.988 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 129344 2024-05-22T23:12:26.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:26.989 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-05-22T23:12:26.989 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 6, 2024-05-22T23:12:26.989 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 163, 2024-05-22T23:12:26.989 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 3312, 2024-05-22T23:12:26.989 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 184256, 2024-05-22T23:12:26.989 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 5, 2024-05-22T23:12:26.989 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 3, 2024-05-22T23:12:26.990 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.008770628, 2024-05-22T23:12:26.990 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.005515160, 2024-05-22T23:12:26.990 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.001642257, 2024-05-22T23:12:26.990 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-22T23:12:26.990 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:26.990 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 161, 2024-05-22T23:12:26.990 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.013754843, 2024-05-22T23:12:26.990 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000085433 2024-05-22T23:12:26.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:26.991 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:26.991 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:26.991 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:26.991 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:26.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:26.991 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-22T23:12:26.991 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 184256 2024-05-22T23:12:26.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:26.992 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-05-22T23:12:26.992 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 0, 2024-05-22T23:12:26.992 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 2, 2024-05-22T23:12:26.992 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 0, 2024-05-22T23:12:26.992 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 0, 2024-05-22T23:12:26.992 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 2, 2024-05-22T23:12:26.992 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 1, 2024-05-22T23:12:26.993 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.000318595, 2024-05-22T23:12:26.993 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.000000000, 2024-05-22T23:12:26.993 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.000247831, 2024-05-22T23:12:26.993 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-22T23:12:26.993 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:26.993 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:26.993 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:26.993 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:26.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:26.994 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:26.994 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:26.994 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:26.994 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:26.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:26.994 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-22T23:12:26.994 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 0 2024-05-22T23:12:26.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:26.995 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-05-22T23:12:26.995 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 0, 2024-05-22T23:12:26.995 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-05-22T23:12:26.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:26.995 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-05-22T23:12:26.995 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-05-22T23:12:26.995 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-05-22T23:12:26.996 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-05-22T23:12:26.996 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-05-22T23:12:26.996 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-05-22T23:12:26.996 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-05-22T23:12:26.996 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-22T23:12:26.996 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-05-22T23:12:26.996 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-22T23:12:26.996 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-05-22T23:12:26.997 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-05-22T23:12:26.997 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-05-22T23:12:26.997 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-22T23:12:26.997 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-05-22T23:12:26.997 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-05-22T23:12:26.997 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-05-22T23:12:26.998 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-05-22T23:12:26.998 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-05-22T23:12:26.998 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-22T23:12:26.998 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-05-22T23:12:26.999 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-22T23:12:26.999 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-05-22T23:12:26.999 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-05-22T23:12:27.000 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-05-22T23:12:27.000 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-05-22T23:12:27.000 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-05-22T23:12:27.000 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-22T23:12:27.000 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-05-22T23:12:27.000 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-05-22T23:12:27.001 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-05-22T23:12:27.001 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-05-22T23:12:27.001 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-05-22T23:12:27.001 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-05-22T23:12:27.001 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-05-22T23:12:27.001 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-05-22T23:12:27.001 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-05-22T23:12:27.001 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 148967, 2024-05-22T23:12:27.002 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 29, 2024-05-22T23:12:27.002 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-05-22T23:12:27.002 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-05-22T23:12:27.002 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-05-22T23:12:27.002 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-05-22T23:12:27.002 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-05-22T23:12:27.003 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-05-22T23:12:27.003 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-05-22T23:12:27.003 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-05-22T23:12:27.003 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 0, 2024-05-22T23:12:27.003 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 0, 2024-05-22T23:12:27.003 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-05-22T23:12:27.003 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-05-22T23:12:27.004 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-05-22T23:12:27.004 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-05-22T23:12:27.004 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 928, 2024-05-22T23:12:27.004 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 2, 2024-05-22T23:12:27.004 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-05-22T23:12:27.004 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-05-22T23:12:27.004 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-05-22T23:12:27.004 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-05-22T23:12:27.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:27.005 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-22T23:12:27.005 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-22T23:12:27.005 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-05-22T23:12:27.005 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-22T23:12:27.005 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 223, 2024-05-22T23:12:27.005 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 1411670, 2024-05-22T23:12:27.005 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-22T23:12:27.006 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 223, 2024-05-22T23:12:27.006 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-22T23:12:27.006 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-22T23:12:27.006 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 223, 2024-05-22T23:12:27.006 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 1411670, 2024-05-22T23:12:27.006 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-22T23:12:27.006 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:27.006 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:27.007 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:27.007 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-22T23:12:27.007 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-22T23:12:27.007 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-05-22T23:12:27.305 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:27 smithi096 ceph-mon[36392]: pgmap v331: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-22T23:12:27.381 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:27 smithi084 ceph-mon[32186]: pgmap v331: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-22T23:12:27.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:27 smithi204 ceph-mon[36211]: pgmap v331: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-22T23:12:27.683 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-05-22T23:12:27.683 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-05-22T23:12:27.683 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-05-22T23:12:27.683 DEBUG:tasks.check_counter:Getting stats from h 2024-05-22T23:12:27.683 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-22T23:12:27.923 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.b/config 2024-05-22T23:12:29.381 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:29 smithi084 ceph-mon[32186]: pgmap v332: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 934 B/s rd, 1 op/s 2024-05-22T23:12:29.382 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:29 smithi096 ceph-mon[36392]: pgmap v332: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 934 B/s rd, 1 op/s 2024-05-22T23:12:29.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:29 smithi204 ceph-mon[36211]: pgmap v332: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 934 B/s rd, 1 op/s 2024-05-22T23:12:29.644 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-05-22T23:12:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-05-22T23:12:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 1688, 2024-05-22T23:12:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 1689, 2024-05-22T23:12:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 226993693, 2024-05-22T23:12:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 536476, 2024-05-22T23:12:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 8, 2024-05-22T23:12:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 4, 2024-05-22T23:12:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.218573602, 2024-05-22T23:12:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.044960722, 2024-05-22T23:12:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.124046393, 2024-05-22T23:12:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.037396370, 2024-05-22T23:12:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1687, 2024-05-22T23:12:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.048126170, 2024-05-22T23:12:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000028527 2024-05-22T23:12:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-22T23:12:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 0 2024-05-22T23:12:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-05-22T23:12:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 955, 2024-05-22T23:12:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 944, 2024-05-22T23:12:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 169433825, 2024-05-22T23:12:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 359098, 2024-05-22T23:12:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 6, 2024-05-22T23:12:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 5, 2024-05-22T23:12:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.164406853, 2024-05-22T23:12:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.024255914, 2024-05-22T23:12:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.123564898, 2024-05-22T23:12:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.020877577, 2024-05-22T23:12:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 943, 2024-05-22T23:12:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.026411067, 2024-05-22T23:12:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000028007 2024-05-22T23:12:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1463472, 2024-05-22T23:12:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 131344 2024-05-22T23:12:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-05-22T23:12:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 862, 2024-05-22T23:12:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 1017, 2024-05-22T23:12:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 179652411, 2024-05-22T23:12:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 717073, 2024-05-22T23:12:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 8, 2024-05-22T23:12:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 6, 2024-05-22T23:12:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.162506286, 2024-05-22T23:12:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.021739421, 2024-05-22T23:12:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.114063841, 2024-05-22T23:12:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.019287521, 2024-05-22T23:12:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1015, 2024-05-22T23:12:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.031999886, 2024-05-22T23:12:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000031526 2024-05-22T23:12:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-22T23:12:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 444864 2024-05-22T23:12:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-05-22T23:12:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-05-22T23:12:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-05-22T23:12:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-05-22T23:12:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-05-22T23:12:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-05-22T23:12:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 5065, 2024-05-22T23:12:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.128221282, 2024-05-22T23:12:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000025315 2024-05-22T23:12:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-22T23:12:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-05-22T23:12:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-05-22T23:12:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-05-22T23:12:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-22T23:12:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-05-22T23:12:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 0, 2024-05-22T23:12:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 0, 2024-05-22T23:12:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-05-22T23:12:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-05-22T23:12:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 0, 2024-05-22T23:12:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-05-22T23:12:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-05-22T23:12:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 0, 2024-05-22T23:12:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 0, 2024-05-22T23:12:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-05-22T23:12:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 34519, 2024-05-22T23:12:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 9129, 2024-05-22T23:12:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 25390, 2024-05-22T23:12:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 0, 2024-05-22T23:12:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 21478, 2024-05-22T23:12:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 0, 2024-05-22T23:12:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 0, 2024-05-22T23:12:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 0, 2024-05-22T23:12:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 2, 2024-05-22T23:12:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 0, 2024-05-22T23:12:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 0, 2024-05-22T23:12:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 0, 2024-05-22T23:12:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 0, 2024-05-22T23:12:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-05-22T23:12:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-05-22T23:12:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 0, 2024-05-22T23:12:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-05-22T23:12:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-05-22T23:12:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 0, 2024-05-22T23:12:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 0, 2024-05-22T23:12:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 0, 2024-05-22T23:12:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 0, 2024-05-22T23:12:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-05-22T23:12:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 0, 2024-05-22T23:12:29.661 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-05-22T23:12:29.661 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 68, 2024-05-22T23:12:29.661 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 21196213, 2024-05-22T23:12:29.661 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 6, 2024-05-22T23:12:29.661 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 0, 2024-05-22T23:12:29.661 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-05-22T23:12:29.661 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-05-22T23:12:29.662 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-05-22T23:12:29.662 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 0, 2024-05-22T23:12:29.662 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 0, 2024-05-22T23:12:29.662 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 0, 2024-05-22T23:12:29.662 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 0, 2024-05-22T23:12:29.662 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 0, 2024-05-22T23:12:29.662 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 0, 2024-05-22T23:12:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 0, 2024-05-22T23:12:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-05-22T23:12:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-22T23:12:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-22T23:12:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 0, 2024-05-22T23:12:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 0, 2024-05-22T23:12:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 0, 2024-05-22T23:12:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 0 2024-05-22T23:12:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-05-22T23:12:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 0, 2024-05-22T23:12:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 0, 2024-05-22T23:12:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 0, 2024-05-22T23:12:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 0, 2024-05-22T23:12:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 0, 2024-05-22T23:12:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 0, 2024-05-22T23:12:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-05-22T23:12:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-05-22T23:12:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 0, 2024-05-22T23:12:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 0, 2024-05-22T23:12:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-05-22T23:12:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-05-22T23:12:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-05-22T23:12:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-05-22T23:12:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-05-22T23:12:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-05-22T23:12:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-05-22T23:12:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 0, 2024-05-22T23:12:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 0, 2024-05-22T23:12:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 0, 2024-05-22T23:12:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 0, 2024-05-22T23:12:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-05-22T23:12:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 0, 2024-05-22T23:12:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-05-22T23:12:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-05-22T23:12:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-05-22T23:12:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-05-22T23:12:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 0, 2024-05-22T23:12:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-05-22T23:12:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-05-22T23:12:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 153822, 2024-05-22T23:12:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-05-22T23:12:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-05-22T23:12:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 0, 2024-05-22T23:12:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-05-22T23:12:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-05-22T23:12:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 134, 2024-05-22T23:12:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 14, 2024-05-22T23:12:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-05-22T23:12:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-05-22T23:12:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 177477184, 2024-05-22T23:12:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 577888240, 2024-05-22T23:12:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 577888128, 2024-05-22T23:12:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-05-22T23:12:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 153822 2024-05-22T23:12:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-05-22T23:12:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 443, 2024-05-22T23:12:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 34447, 2024-05-22T23:12:29.671 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 34004, 2024-05-22T23:12:29.671 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 109, 2024-05-22T23:12:29.671 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 194, 2024-05-22T23:12:29.671 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 85, 2024-05-22T23:12:29.671 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 34519, 2024-05-22T23:12:29.671 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 34519, 2024-05-22T23:12:29.671 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 0, 2024-05-22T23:12:29.671 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 0, 2024-05-22T23:12:29.672 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 0, 2024-05-22T23:12:29.672 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 0, 2024-05-22T23:12:29.672 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 87052, 2024-05-22T23:12:29.672 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 215420 2024-05-22T23:12:29.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.672 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-05-22T23:12:29.672 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 0, 2024-05-22T23:12:29.672 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 0, 2024-05-22T23:12:29.673 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 0, 2024-05-22T23:12:29.673 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 0, 2024-05-22T23:12:29.673 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 0, 2024-05-22T23:12:29.673 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-05-22T23:12:29.673 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.673 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.673 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.673 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-05-22T23:12:29.674 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.674 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.674 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.674 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-05-22T23:12:29.674 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.674 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.674 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.675 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-05-22T23:12:29.675 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.675 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.675 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.675 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-05-22T23:12:29.675 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-05-22T23:12:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-05-22T23:12:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-05-22T23:12:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-05-22T23:12:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-05-22T23:12:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-05-22T23:12:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-05-22T23:12:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-05-22T23:12:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-05-22T23:12:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-05-22T23:12:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-05-22T23:12:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-05-22T23:12:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-05-22T23:12:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-05-22T23:12:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-05-22T23:12:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-05-22T23:12:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-05-22T23:12:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-05-22T23:12:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-05-22T23:12:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-05-22T23:12:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-05-22T23:12:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-05-22T23:12:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-05-22T23:12:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-05-22T23:12:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-05-22T23:12:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-05-22T23:12:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-05-22T23:12:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-22T23:12:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-05-22T23:12:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 2, 2024-05-22T23:12:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 5, 2024-05-22T23:12:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 3, 2024-05-22T23:12:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 4, 2024-05-22T23:12:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-05-22T23:12:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 0, 2024-05-22T23:12:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 0, 2024-05-22T23:12:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 3314, 2024-05-22T23:12:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-05-22T23:12:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-05-22T23:12:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-05-22T23:12:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-05-22T23:12:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-05-22T23:12:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-05-22T23:12:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-05-22T23:12:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-05-22T23:12:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-22T23:12:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-05-22T23:12:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-22T23:12:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-05-22T23:12:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-05-22T23:12:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-05-22T23:12:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-22T23:12:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-05-22T23:12:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-05-22T23:12:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-05-22T23:12:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-05-22T23:12:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-05-22T23:12:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-22T23:12:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-05-22T23:12:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-22T23:12:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-05-22T23:12:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-05-22T23:12:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-05-22T23:12:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-05-22T23:12:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-05-22T23:12:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-22T23:12:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-05-22T23:12:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-05-22T23:12:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-05-22T23:12:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-05-22T23:12:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-05-22T23:12:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-05-22T23:12:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-05-22T23:12:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-05-22T23:12:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-05-22T23:12:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 258226, 2024-05-22T23:12:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 605, 2024-05-22T23:12:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-05-22T23:12:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-05-22T23:12:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-05-22T23:12:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-05-22T23:12:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-05-22T23:12:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-05-22T23:12:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-05-22T23:12:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-05-22T23:12:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 8224, 2024-05-22T23:12:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 171, 2024-05-22T23:12:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-05-22T23:12:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-05-22T23:12:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-05-22T23:12:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-05-22T23:12:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 25650089, 2024-05-22T23:12:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 150521, 2024-05-22T23:12:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-05-22T23:12:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-05-22T23:12:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-05-22T23:12:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-05-22T23:12:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-05-22T23:12:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-05-22T23:12:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-05-22T23:12:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 3259, 2024-05-22T23:12:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 26, 2024-05-22T23:12:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 4, 2024-05-22T23:12:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 3259, 2024-05-22T23:12:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-05-22T23:12:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3259, 2024-05-22T23:12:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 4.476898499, 2024-05-22T23:12:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001373703 2024-05-22T23:12:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-05-22T23:12:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-05-22T23:12:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3259, 2024-05-22T23:12:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 3260 2024-05-22T23:12:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 3259, 2024-05-22T23:12:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 3259, 2024-05-22T23:12:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 0, 2024-05-22T23:12:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-05-22T23:12:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-05-22T23:12:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 2006, 2024-05-22T23:12:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 0, 2024-05-22T23:12:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 1250, 2024-05-22T23:12:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 0, 2024-05-22T23:12:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 0, 2024-05-22T23:12:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-05-22T23:12:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-05-22T23:12:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-05-22T23:12:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 0, 2024-05-22T23:12:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 0, 2024-05-22T23:12:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-05-22T23:12:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-05-22T23:12:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-05-22T23:12:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 2, 2024-05-22T23:12:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 0, 2024-05-22T23:12:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-05-22T23:12:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-05-22T23:12:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-05-22T23:12:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-05-22T23:12:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-05-22T23:12:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-05-22T23:12:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-05-22T23:12:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-05-22T23:12:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-05-22T23:12:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-05-22T23:12:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-05-22T23:12:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-05-22T23:12:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-05-22T23:12:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-05-22T23:12:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-05-22T23:12:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-05-22T23:12:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-05-22T23:12:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-05-22T23:12:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-05-22T23:12:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-05-22T23:12:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-05-22T23:12:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-05-22T23:12:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-05-22T23:12:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-05-22T23:12:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-05-22T23:12:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-05-22T23:12:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 99, 2024-05-22T23:12:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-05-22T23:12:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 13, 2024-05-22T23:12:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 11, 2024-05-22T23:12:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 11, 2024-05-22T23:12:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 0, 2024-05-22T23:12:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-05-22T23:12:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 0, 2024-05-22T23:12:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 2, 2024-05-22T23:12:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 0 2024-05-22T23:12:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-05-22T23:12:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 0, 2024-05-22T23:12:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-05-22T23:12:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-05-22T23:12:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-05-22T23:12:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-05-22T23:12:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 0 2024-05-22T23:12:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-05-22T23:12:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-05-22T23:12:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 0, 2024-05-22T23:12:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-05-22T23:12:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 0, 2024-05-22T23:12:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 0, 2024-05-22T23:12:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 0, 2024-05-22T23:12:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-05-22T23:12:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-22T23:12:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-22T23:12:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-05-22T23:12:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-22T23:12:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 3501, 2024-05-22T23:12:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 575749615, 2024-05-22T23:12:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-22T23:12:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 3501, 2024-05-22T23:12:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-22T23:12:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-22T23:12:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 3501, 2024-05-22T23:12:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 575749615, 2024-05-22T23:12:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-22T23:12:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-22T23:12:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-05-22T23:12:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-22T23:12:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-05-22T23:12:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-22T23:12:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-05-22T23:12:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-05-22T23:12:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-22T23:12:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-05-22T23:12:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 3259, 2024-05-22T23:12:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 573716901, 2024-05-22T23:12:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 3259, 2024-05-22T23:12:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 573716901, 2024-05-22T23:12:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-22T23:12:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-22T23:12:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-05-22T23:12:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-22T23:12:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-05-22T23:12:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-22T23:12:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-05-22T23:12:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-05-22T23:12:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-22T23:12:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-05-22T23:12:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 3259, 2024-05-22T23:12:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 3259, 2024-05-22T23:12:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 3259, 2024-05-22T23:12:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 3259, 2024-05-22T23:12:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-22T23:12:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-22T23:12:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-05-22T23:12:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-22T23:12:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-05-22T23:12:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-22T23:12:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-05-22T23:12:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-05-22T23:12:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-22T23:12:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-05-22T23:12:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-22T23:12:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-22T23:12:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-05-22T23:12:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-05-22T23:12:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-22T23:12:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-22T23:12:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x55ac8fca06e0": { 2024-05-22T23:12:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-22T23:12:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-05-22T23:12:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-22T23:12:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-05-22T23:12:29.725 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-05-22T23:12:29.725 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-22T23:12:29.725 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-05-22T23:12:29.725 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-22T23:12:29.725 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-22T23:12:29.725 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-05-22T23:12:29.725 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-05-22T23:12:29.725 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-22T23:12:29.726 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:29.726 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:29.726 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:29.726 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-22T23:12:29.726 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-22T23:12:29.726 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-05-22T23:12:30.435 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-05-22T23:12:30.435 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-22T23:12:30.435 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-22T23:12:30.435 DEBUG:tasks.check_counter:Getting stats from k 2024-05-22T23:12:30.435 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-22T23:12:30.674 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.b/config 2024-05-22T23:12:31.381 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:31 smithi084 ceph-mon[32186]: pgmap v333: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:12:31.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:31 smithi204 ceph-mon[36211]: pgmap v333: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:12:31.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:31 smithi096 ceph-mon[36392]: pgmap v333: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:12:32.378 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-05-22T23:12:32.378 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-05-22T23:12:32.378 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 0, 2024-05-22T23:12:32.378 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 2, 2024-05-22T23:12:32.379 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 0, 2024-05-22T23:12:32.379 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 0, 2024-05-22T23:12:32.379 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-05-22T23:12:32.379 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 0, 2024-05-22T23:12:32.379 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.001058894, 2024-05-22T23:12:32.379 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.000000000, 2024-05-22T23:12:32.379 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.000273183, 2024-05-22T23:12:32.379 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-22T23:12:32.379 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:32.380 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:32.380 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:32.380 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:32.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:32.380 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:32.380 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:32.380 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:32.380 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:32.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:32.380 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-22T23:12:32.381 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 0 2024-05-22T23:12:32.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:32.381 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-05-22T23:12:32.381 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 228, 2024-05-22T23:12:32.381 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 216, 2024-05-22T23:12:32.381 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1446640, 2024-05-22T23:12:32.381 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 131712, 2024-05-22T23:12:32.381 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 2, 2024-05-22T23:12:32.381 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 1, 2024-05-22T23:12:32.382 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.040996792, 2024-05-22T23:12:32.382 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.011796850, 2024-05-22T23:12:32.382 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.032927943, 2024-05-22T23:12:32.382 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.004744607, 2024-05-22T23:12:32.382 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:32.382 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 216, 2024-05-22T23:12:32.382 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.009446166, 2024-05-22T23:12:32.382 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000043732 2024-05-22T23:12:32.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:32.383 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:32.383 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:32.383 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:32.383 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:32.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:32.383 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1446640, 2024-05-22T23:12:32.383 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 131712 2024-05-22T23:12:32.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:32.384 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-05-22T23:12:32.384 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 2, 2024-05-22T23:12:32.384 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 164, 2024-05-22T23:12:32.384 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 320, 2024-05-22T23:12:32.384 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 184336, 2024-05-22T23:12:32.384 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 5, 2024-05-22T23:12:32.384 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 3, 2024-05-22T23:12:32.384 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.009595678, 2024-05-22T23:12:32.384 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.006254373, 2024-05-22T23:12:32.385 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.001460145, 2024-05-22T23:12:32.385 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-22T23:12:32.385 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:32.385 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 161, 2024-05-22T23:12:32.385 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.018086465, 2024-05-22T23:12:32.385 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000112338 2024-05-22T23:12:32.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:32.385 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:32.386 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:32.386 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:32.386 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:32.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:32.386 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-22T23:12:32.386 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 184336 2024-05-22T23:12:32.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:32.386 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-05-22T23:12:32.387 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 0, 2024-05-22T23:12:32.387 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-05-22T23:12:32.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:32.387 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-05-22T23:12:32.387 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-05-22T23:12:32.387 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-05-22T23:12:32.387 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-05-22T23:12:32.387 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-05-22T23:12:32.387 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-05-22T23:12:32.388 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-05-22T23:12:32.388 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-22T23:12:32.388 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-05-22T23:12:32.388 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-22T23:12:32.388 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-05-22T23:12:32.388 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-05-22T23:12:32.388 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-05-22T23:12:32.388 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-22T23:12:32.389 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-05-22T23:12:32.389 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-05-22T23:12:32.389 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-05-22T23:12:32.389 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-05-22T23:12:32.389 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-05-22T23:12:32.389 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-22T23:12:32.389 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-05-22T23:12:32.389 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-22T23:12:32.389 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-05-22T23:12:32.390 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-05-22T23:12:32.390 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-05-22T23:12:32.390 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-05-22T23:12:32.390 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-05-22T23:12:32.390 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-22T23:12:32.390 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-05-22T23:12:32.390 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-05-22T23:12:32.390 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-05-22T23:12:32.391 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-05-22T23:12:32.391 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-05-22T23:12:32.391 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-05-22T23:12:32.391 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-05-22T23:12:32.391 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-05-22T23:12:32.391 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-05-22T23:12:32.391 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 149047, 2024-05-22T23:12:32.391 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 31, 2024-05-22T23:12:32.392 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-05-22T23:12:32.392 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-05-22T23:12:32.392 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-05-22T23:12:32.392 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-05-22T23:12:32.392 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-05-22T23:12:32.392 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-05-22T23:12:32.392 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-05-22T23:12:32.392 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-05-22T23:12:32.393 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 0, 2024-05-22T23:12:32.393 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 0, 2024-05-22T23:12:32.393 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-05-22T23:12:32.393 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-05-22T23:12:32.393 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-05-22T23:12:32.393 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-05-22T23:12:32.393 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 928, 2024-05-22T23:12:32.393 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 2, 2024-05-22T23:12:32.393 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-05-22T23:12:32.394 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-05-22T23:12:32.394 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-05-22T23:12:32.394 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-05-22T23:12:32.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-22T23:12:32.394 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-22T23:12:32.394 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-22T23:12:32.394 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-05-22T23:12:32.394 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-22T23:12:32.395 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 226, 2024-05-22T23:12:32.395 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 1412060, 2024-05-22T23:12:32.395 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-22T23:12:32.395 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 226, 2024-05-22T23:12:32.395 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-22T23:12:32.395 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-22T23:12:32.395 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 226, 2024-05-22T23:12:32.395 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 1412060, 2024-05-22T23:12:32.396 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-22T23:12:32.396 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-22T23:12:32.396 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-22T23:12:32.396 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-22T23:12:32.396 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-22T23:12:32.396 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-22T23:12:32.396 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-05-22T23:12:33.071 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-05-22T23:12:33.071 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-05-22T23:12:33.072 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-05-22T23:12:33.072 DEBUG:tasks.check_counter:Getting stats from c 2024-05-22T23:12:33.072 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-22T23:12:33.321 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.c/config 2024-05-22T23:12:33.381 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:33 smithi084 ceph-mon[32186]: pgmap v334: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:12:33.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:33 smithi204 ceph-mon[36211]: pgmap v334: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:12:33.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:33 smithi096 ceph-mon[36392]: pgmap v334: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:12:35.198 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:35 smithi204 ceph-mon[36211]: pgmap v335: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:12:35.227 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-05-22T23:12:35.228 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-05-22T23:12:35.228 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 4, 2024-05-22T23:12:35.228 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 4, 2024-05-22T23:12:35.228 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 2992, 2024-05-22T23:12:35.228 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 448, 2024-05-22T23:12:35.228 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2024-05-22T23:12:35.229 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 0, 2024-05-22T23:12:35.229 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.001450131, 2024-05-22T23:12:35.229 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.000034849, 2024-05-22T23:12:35.229 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.000575729, 2024-05-22T23:12:35.229 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-22T23:12:35.229 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:35.229 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3, 2024-05-22T23:12:35.229 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000676515, 2024-05-22T23:12:35.229 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000225505 2024-05-22T23:12:35.230 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:35.230 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:35.230 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-22T23:12:35.230 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-22T23:12:35.230 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-22T23:12:35.230 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:35.230 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-22T23:12:35.230 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 448 2024-05-22T23:12:35.231 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:35.231 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-05-22T23:12:35.231 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 2, 2024-05-22T23:12:35.231 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 161, 2024-05-22T23:12:35.231 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 320, 2024-05-22T23:12:35.231 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 183808, 2024-05-22T23:12:35.231 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 5, 2024-05-22T23:12:35.231 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-05-22T23:12:35.232 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.010189446, 2024-05-22T23:12:35.232 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.006250395, 2024-05-22T23:12:35.232 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.001979032, 2024-05-22T23:12:35.232 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-22T23:12:35.232 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:35.232 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 158, 2024-05-22T23:12:35.232 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.014970599, 2024-05-22T23:12:35.232 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000094750 2024-05-22T23:12:35.233 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:35.233 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:35.233 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-22T23:12:35.233 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-22T23:12:35.233 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-22T23:12:35.233 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:35.233 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-22T23:12:35.234 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 183808 2024-05-22T23:12:35.234 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:35.234 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-05-22T23:12:35.234 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 221, 2024-05-22T23:12:35.234 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 211, 2024-05-22T23:12:35.234 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1442832, 2024-05-22T23:12:35.234 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 129344, 2024-05-22T23:12:35.234 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 2, 2024-05-22T23:12:35.235 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 1, 2024-05-22T23:12:35.238 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.040225696, 2024-05-22T23:12:35.238 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.012248521, 2024-05-22T23:12:35.238 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.030315055, 2024-05-22T23:12:35.238 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.004982820, 2024-05-22T23:12:35.238 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:35.238 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 210, 2024-05-22T23:12:35.238 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.009493509, 2024-05-22T23:12:35.239 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000045207 2024-05-22T23:12:35.239 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:35.239 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:35.239 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-22T23:12:35.239 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-22T23:12:35.239 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-22T23:12:35.239 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:35.239 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1442832, 2024-05-22T23:12:35.240 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 129344 2024-05-22T23:12:35.240 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:35.240 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-05-22T23:12:35.240 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 0, 2024-05-22T23:12:35.240 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-05-22T23:12:35.240 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:35.240 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-05-22T23:12:35.240 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-05-22T23:12:35.241 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-05-22T23:12:35.241 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-05-22T23:12:35.241 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-05-22T23:12:35.241 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-05-22T23:12:35.241 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-05-22T23:12:35.241 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-22T23:12:35.241 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-05-22T23:12:35.241 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-22T23:12:35.241 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-05-22T23:12:35.242 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-05-22T23:12:35.242 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-05-22T23:12:35.242 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-22T23:12:35.242 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-05-22T23:12:35.242 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-05-22T23:12:35.242 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-05-22T23:12:35.242 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-05-22T23:12:35.242 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-05-22T23:12:35.243 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-22T23:12:35.243 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-05-22T23:12:35.243 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-22T23:12:35.243 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-05-22T23:12:35.243 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-05-22T23:12:35.243 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-05-22T23:12:35.243 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-05-22T23:12:35.243 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-05-22T23:12:35.244 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-22T23:12:35.244 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-05-22T23:12:35.244 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-05-22T23:12:35.244 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-05-22T23:12:35.244 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-05-22T23:12:35.244 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-05-22T23:12:35.244 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-05-22T23:12:35.244 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-05-22T23:12:35.245 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-05-22T23:12:35.245 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-05-22T23:12:35.245 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 149057, 2024-05-22T23:12:35.245 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 32, 2024-05-22T23:12:35.245 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-05-22T23:12:35.245 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-05-22T23:12:35.245 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-05-22T23:12:35.245 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-05-22T23:12:35.246 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-05-22T23:12:35.246 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-05-22T23:12:35.246 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-05-22T23:12:35.246 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-05-22T23:12:35.246 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 0, 2024-05-22T23:12:35.246 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 0, 2024-05-22T23:12:35.246 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-05-22T23:12:35.246 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-05-22T23:12:35.247 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-05-22T23:12:35.247 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-05-22T23:12:35.247 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 928, 2024-05-22T23:12:35.247 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 2, 2024-05-22T23:12:35.247 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-05-22T23:12:35.247 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-05-22T23:12:35.247 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-05-22T23:12:35.247 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-05-22T23:12:35.248 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:35.248 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-22T23:12:35.248 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-05-22T23:12:35.248 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-05-22T23:12:35.248 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-05-22T23:12:35.248 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 223, 2024-05-22T23:12:35.248 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 1411670, 2024-05-22T23:12:35.248 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-05-22T23:12:35.249 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 223, 2024-05-22T23:12:35.249 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-05-22T23:12:35.249 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-05-22T23:12:35.249 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 223, 2024-05-22T23:12:35.249 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1411670, 2024-05-22T23:12:35.249 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-05-22T23:12:35.249 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-22T23:12:35.249 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-22T23:12:35.249 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-22T23:12:35.250 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-22T23:12:35.250 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-22T23:12:35.250 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-05-22T23:12:35.305 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:35 smithi096 ceph-mon[36392]: pgmap v335: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:12:35.381 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:35 smithi084 ceph-mon[32186]: pgmap v335: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:12:36.038 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.c 2024-05-22T23:12:36.038 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.c 2024-05-22T23:12:36.038 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.c 2024-05-22T23:12:36.038 DEBUG:tasks.check_counter:Getting stats from f 2024-05-22T23:12:36.039 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-22T23:12:36.281 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.c/config 2024-05-22T23:12:37.381 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:37 smithi084 ceph-mon[32186]: pgmap v336: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:12:37.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:37 smithi204 ceph-mon[36211]: pgmap v336: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:12:37.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:37 smithi096 ceph-mon[36392]: pgmap v336: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:12:38.091 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-05-22T23:12:38.091 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-05-22T23:12:38.092 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 223, 2024-05-22T23:12:38.092 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 213, 2024-05-22T23:12:38.092 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1443376, 2024-05-22T23:12:38.092 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 130624, 2024-05-22T23:12:38.092 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2024-05-22T23:12:38.092 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 0, 2024-05-22T23:12:38.092 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.042393437, 2024-05-22T23:12:38.093 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.012413492, 2024-05-22T23:12:38.093 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.032224090, 2024-05-22T23:12:38.093 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.004870254, 2024-05-22T23:12:38.093 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:38.093 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 212, 2024-05-22T23:12:38.093 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.010394896, 2024-05-22T23:12:38.093 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000049032 2024-05-22T23:12:38.094 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:38.094 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:38.094 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-22T23:12:38.094 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-22T23:12:38.094 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-22T23:12:38.094 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:38.094 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1443376, 2024-05-22T23:12:38.095 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 130624 2024-05-22T23:12:38.095 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:38.095 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-05-22T23:12:38.095 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 2, 2024-05-22T23:12:38.095 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 162, 2024-05-22T23:12:38.095 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 320, 2024-05-22T23:12:38.095 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 183984, 2024-05-22T23:12:38.095 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 5, 2024-05-22T23:12:38.096 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-05-22T23:12:38.096 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.010690279, 2024-05-22T23:12:38.096 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.007131978, 2024-05-22T23:12:38.096 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.001432571, 2024-05-22T23:12:38.096 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-22T23:12:38.096 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:38.096 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 159, 2024-05-22T23:12:38.097 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.013256194, 2024-05-22T23:12:38.097 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000083372 2024-05-22T23:12:38.097 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:38.097 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:38.097 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-22T23:12:38.097 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-22T23:12:38.097 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-22T23:12:38.097 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:38.097 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-22T23:12:38.097 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 183984 2024-05-22T23:12:38.098 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:38.098 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-05-22T23:12:38.098 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 4, 2024-05-22T23:12:38.098 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 4, 2024-05-22T23:12:38.098 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 2992, 2024-05-22T23:12:38.098 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 448, 2024-05-22T23:12:38.098 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 2, 2024-05-22T23:12:38.098 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 1, 2024-05-22T23:12:38.099 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.000753025, 2024-05-22T23:12:38.099 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.000035141, 2024-05-22T23:12:38.099 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.000679145, 2024-05-22T23:12:38.099 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-22T23:12:38.099 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:38.099 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3, 2024-05-22T23:12:38.099 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000759411, 2024-05-22T23:12:38.099 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000253137 2024-05-22T23:12:38.100 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:38.100 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:38.100 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-22T23:12:38.100 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-22T23:12:38.100 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-22T23:12:38.100 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:38.100 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-22T23:12:38.100 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 448 2024-05-22T23:12:38.101 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:38.101 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-05-22T23:12:38.101 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 0, 2024-05-22T23:12:38.101 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-05-22T23:12:38.101 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:38.101 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-05-22T23:12:38.101 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-05-22T23:12:38.101 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-05-22T23:12:38.102 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-05-22T23:12:38.102 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-05-22T23:12:38.102 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-05-22T23:12:38.102 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-05-22T23:12:38.102 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-22T23:12:38.102 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-05-22T23:12:38.102 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-22T23:12:38.102 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-05-22T23:12:38.103 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-05-22T23:12:38.103 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-05-22T23:12:38.103 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-22T23:12:38.103 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-05-22T23:12:38.103 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-05-22T23:12:38.103 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-05-22T23:12:38.103 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-05-22T23:12:38.103 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-05-22T23:12:38.103 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-22T23:12:38.104 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-05-22T23:12:38.104 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-22T23:12:38.104 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-05-22T23:12:38.104 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-05-22T23:12:38.104 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-05-22T23:12:38.104 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-05-22T23:12:38.104 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-05-22T23:12:38.104 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-22T23:12:38.105 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-05-22T23:12:38.105 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-05-22T23:12:38.105 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-05-22T23:12:38.105 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-05-22T23:12:38.105 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-05-22T23:12:38.105 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-05-22T23:12:38.105 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-05-22T23:12:38.105 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-05-22T23:12:38.105 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-05-22T23:12:38.106 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 148967, 2024-05-22T23:12:38.106 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 29, 2024-05-22T23:12:38.106 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-05-22T23:12:38.106 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-05-22T23:12:38.106 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-05-22T23:12:38.106 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-05-22T23:12:38.106 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-05-22T23:12:38.106 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-05-22T23:12:38.107 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-05-22T23:12:38.107 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-05-22T23:12:38.107 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 0, 2024-05-22T23:12:38.107 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 0, 2024-05-22T23:12:38.107 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-05-22T23:12:38.107 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-05-22T23:12:38.107 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-05-22T23:12:38.107 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-05-22T23:12:38.107 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 928, 2024-05-22T23:12:38.108 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 2, 2024-05-22T23:12:38.108 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-05-22T23:12:38.108 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-05-22T23:12:38.108 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-05-22T23:12:38.108 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-05-22T23:12:38.108 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:38.108 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-22T23:12:38.108 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-05-22T23:12:38.108 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-05-22T23:12:38.109 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-05-22T23:12:38.109 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 225, 2024-05-22T23:12:38.109 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 1411930, 2024-05-22T23:12:38.109 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-05-22T23:12:38.109 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 225, 2024-05-22T23:12:38.109 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-05-22T23:12:38.109 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-05-22T23:12:38.109 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 225, 2024-05-22T23:12:38.109 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1411930, 2024-05-22T23:12:38.110 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-05-22T23:12:38.110 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-22T23:12:38.110 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-22T23:12:38.110 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-22T23:12:38.110 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-22T23:12:38.110 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-22T23:12:38.110 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-05-22T23:12:38.832 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-05-22T23:12:38.833 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-05-22T23:12:38.833 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-05-22T23:12:38.833 DEBUG:tasks.check_counter:Getting stats from i 2024-05-22T23:12:38.833 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-22T23:12:39.071 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.c/config 2024-05-22T23:12:39.381 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:39 smithi084 ceph-mon[32186]: pgmap v337: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:12:39.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:39 smithi204 ceph-mon[36211]: pgmap v337: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:12:39.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:39 smithi096 ceph-mon[36392]: pgmap v337: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:12:40.860 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-05-22T23:12:40.860 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-05-22T23:12:40.860 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 4, 2024-05-22T23:12:40.861 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 4, 2024-05-22T23:12:40.861 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 2992, 2024-05-22T23:12:40.861 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 448, 2024-05-22T23:12:40.861 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2024-05-22T23:12:40.861 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 0, 2024-05-22T23:12:40.861 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.001466961, 2024-05-22T23:12:40.861 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.000033050, 2024-05-22T23:12:40.861 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.000485627, 2024-05-22T23:12:40.862 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-22T23:12:40.862 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:40.862 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3, 2024-05-22T23:12:40.862 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000710904, 2024-05-22T23:12:40.862 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000236968 2024-05-22T23:12:40.862 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:40.862 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:40.862 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-22T23:12:40.863 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-22T23:12:40.863 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-22T23:12:40.863 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:40.863 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-22T23:12:40.863 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 448 2024-05-22T23:12:40.863 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:40.863 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-05-22T23:12:40.863 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 221, 2024-05-22T23:12:40.864 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 211, 2024-05-22T23:12:40.864 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1442832, 2024-05-22T23:12:40.864 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 129344, 2024-05-22T23:12:40.864 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 2, 2024-05-22T23:12:40.864 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 1, 2024-05-22T23:12:40.864 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.038989465, 2024-05-22T23:12:40.864 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.012108879, 2024-05-22T23:12:40.864 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.028336668, 2024-05-22T23:12:40.865 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.004997920, 2024-05-22T23:12:40.865 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:40.865 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 210, 2024-05-22T23:12:40.865 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.010402477, 2024-05-22T23:12:40.865 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000049535 2024-05-22T23:12:40.865 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:40.865 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:40.865 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-22T23:12:40.865 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-22T23:12:40.866 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-22T23:12:40.866 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:40.866 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1442832, 2024-05-22T23:12:40.866 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 129344 2024-05-22T23:12:40.866 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:40.866 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-05-22T23:12:40.866 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 2, 2024-05-22T23:12:40.866 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 160, 2024-05-22T23:12:40.867 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 320, 2024-05-22T23:12:40.867 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 183632, 2024-05-22T23:12:40.867 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 5, 2024-05-22T23:12:40.867 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-05-22T23:12:40.867 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.010119793, 2024-05-22T23:12:40.867 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.006596912, 2024-05-22T23:12:40.867 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.001434016, 2024-05-22T23:12:40.867 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-22T23:12:40.868 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:40.868 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 157, 2024-05-22T23:12:40.868 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.014834282, 2024-05-22T23:12:40.868 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000094485 2024-05-22T23:12:40.868 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:40.868 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:40.868 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-22T23:12:40.868 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-22T23:12:40.868 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-22T23:12:40.869 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:40.869 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-22T23:12:40.869 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 183632 2024-05-22T23:12:40.869 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:40.869 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-05-22T23:12:40.869 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 0, 2024-05-22T23:12:40.869 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-05-22T23:12:40.869 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:40.870 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-05-22T23:12:40.870 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-05-22T23:12:40.870 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-05-22T23:12:40.870 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-05-22T23:12:40.870 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-05-22T23:12:40.870 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-05-22T23:12:40.870 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-05-22T23:12:40.870 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-22T23:12:40.871 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-05-22T23:12:40.871 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-22T23:12:40.871 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-05-22T23:12:40.871 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-05-22T23:12:40.871 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-05-22T23:12:40.871 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-22T23:12:40.871 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-05-22T23:12:40.871 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-05-22T23:12:40.871 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-05-22T23:12:40.872 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-05-22T23:12:40.872 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-05-22T23:12:40.872 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-22T23:12:40.872 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-05-22T23:12:40.872 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-22T23:12:40.872 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-05-22T23:12:40.872 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-05-22T23:12:40.872 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-05-22T23:12:40.873 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-05-22T23:12:40.873 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-05-22T23:12:40.873 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-22T23:12:40.873 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-05-22T23:12:40.873 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-05-22T23:12:40.873 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-05-22T23:12:40.873 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-05-22T23:12:40.873 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-05-22T23:12:40.874 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-05-22T23:12:40.874 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-05-22T23:12:40.874 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-05-22T23:12:40.874 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-05-22T23:12:40.874 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 149007, 2024-05-22T23:12:40.874 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 30, 2024-05-22T23:12:40.874 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-05-22T23:12:40.874 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-05-22T23:12:40.875 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-05-22T23:12:40.875 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-05-22T23:12:40.875 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-05-22T23:12:40.875 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-05-22T23:12:40.875 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-05-22T23:12:40.875 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-05-22T23:12:40.875 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 0, 2024-05-22T23:12:40.875 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 0, 2024-05-22T23:12:40.875 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-05-22T23:12:40.876 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-05-22T23:12:40.876 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-05-22T23:12:40.876 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-05-22T23:12:40.876 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 928, 2024-05-22T23:12:40.876 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 2, 2024-05-22T23:12:40.876 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-05-22T23:12:40.876 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-05-22T23:12:40.877 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-05-22T23:12:40.877 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-05-22T23:12:40.877 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:40.877 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-22T23:12:40.877 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-05-22T23:12:40.877 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-05-22T23:12:40.877 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-05-22T23:12:40.877 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 223, 2024-05-22T23:12:40.878 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 1411670, 2024-05-22T23:12:40.878 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-05-22T23:12:40.878 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 223, 2024-05-22T23:12:40.878 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-05-22T23:12:40.878 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-05-22T23:12:40.878 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 223, 2024-05-22T23:12:40.878 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1411670, 2024-05-22T23:12:40.878 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-05-22T23:12:40.879 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-22T23:12:40.879 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-22T23:12:40.879 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-22T23:12:40.879 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-22T23:12:40.879 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-22T23:12:40.879 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-05-22T23:12:41.381 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:41 smithi084 ceph-mon[32186]: pgmap v338: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:12:41.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:41 smithi204 ceph-mon[36211]: pgmap v338: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:12:41.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:41 smithi096 ceph-mon[36392]: pgmap v338: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:12:41.692 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.i 2024-05-22T23:12:41.693 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.i 2024-05-22T23:12:41.693 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.i 2024-05-22T23:12:41.693 DEBUG:tasks.check_counter:Getting stats from l 2024-05-22T23:12:41.693 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794 shell --fsid c1135a12-188d-11ef-bc9a-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-22T23:12:41.929 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/mon.c/config 2024-05-22T23:12:43.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:43 smithi204 ceph-mon[36211]: pgmap v339: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:12:43.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:43 smithi096 ceph-mon[36392]: pgmap v339: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:12:43.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:43 smithi084 ceph-mon[32186]: pgmap v339: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:12:43.719 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-05-22T23:12:43.720 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-05-22T23:12:43.720 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 0, 2024-05-22T23:12:43.720 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 2, 2024-05-22T23:12:43.720 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 0, 2024-05-22T23:12:43.720 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 0, 2024-05-22T23:12:43.720 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2024-05-22T23:12:43.720 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 0, 2024-05-22T23:12:43.721 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.001105762, 2024-05-22T23:12:43.721 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.000000000, 2024-05-22T23:12:43.721 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.000304401, 2024-05-22T23:12:43.721 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-22T23:12:43.721 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:43.721 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-22T23:12:43.721 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-22T23:12:43.721 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-22T23:12:43.722 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:43.722 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:43.722 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-22T23:12:43.722 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-22T23:12:43.722 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-22T23:12:43.722 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:43.722 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-22T23:12:43.722 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 0 2024-05-22T23:12:43.723 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:43.723 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-05-22T23:12:43.723 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 228, 2024-05-22T23:12:43.723 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 218, 2024-05-22T23:12:43.723 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1444736, 2024-05-22T23:12:43.723 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 133824, 2024-05-22T23:12:43.723 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 2, 2024-05-22T23:12:43.724 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 1, 2024-05-22T23:12:43.724 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.042865627, 2024-05-22T23:12:43.724 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.012715701, 2024-05-22T23:12:43.724 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.033738171, 2024-05-22T23:12:43.724 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.004854333, 2024-05-22T23:12:43.724 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:43.724 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 217, 2024-05-22T23:12:43.724 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.011776174, 2024-05-22T23:12:43.725 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000054268 2024-05-22T23:12:43.725 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:43.725 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:43.725 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-22T23:12:43.725 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-22T23:12:43.725 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-22T23:12:43.725 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:43.725 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1444736, 2024-05-22T23:12:43.726 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 133824 2024-05-22T23:12:43.726 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:43.726 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-05-22T23:12:43.726 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 6, 2024-05-22T23:12:43.726 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 168, 2024-05-22T23:12:43.726 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 3312, 2024-05-22T23:12:43.726 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 185136, 2024-05-22T23:12:43.726 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 5, 2024-05-22T23:12:43.727 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-05-22T23:12:43.727 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.015034506, 2024-05-22T23:12:43.727 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.006264234, 2024-05-22T23:12:43.727 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.006844900, 2024-05-22T23:12:43.727 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-22T23:12:43.727 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-05-22T23:12:43.727 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 166, 2024-05-22T23:12:43.727 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.017386345, 2024-05-22T23:12:43.727 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000104737 2024-05-22T23:12:43.728 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:43.728 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-05-22T23:12:43.728 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-22T23:12:43.728 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-22T23:12:43.728 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-22T23:12:43.728 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:43.728 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-22T23:12:43.728 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 185136 2024-05-22T23:12:43.729 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:43.729 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-05-22T23:12:43.729 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 0, 2024-05-22T23:12:43.729 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-05-22T23:12:43.729 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:43.729 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-05-22T23:12:43.729 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-05-22T23:12:43.729 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-05-22T23:12:43.730 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-05-22T23:12:43.730 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-05-22T23:12:43.730 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-05-22T23:12:43.730 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-05-22T23:12:43.730 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-22T23:12:43.730 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-05-22T23:12:43.730 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-22T23:12:43.730 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-05-22T23:12:43.731 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-05-22T23:12:43.731 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-05-22T23:12:43.731 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-22T23:12:43.731 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-05-22T23:12:43.731 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-05-22T23:12:43.731 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-05-22T23:12:43.731 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-05-22T23:12:43.731 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-05-22T23:12:43.732 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-22T23:12:43.732 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-05-22T23:12:43.732 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-22T23:12:43.732 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-05-22T23:12:43.732 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-05-22T23:12:43.732 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-05-22T23:12:43.732 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-05-22T23:12:43.732 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-05-22T23:12:43.733 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-22T23:12:43.733 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-05-22T23:12:43.733 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-05-22T23:12:43.733 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-05-22T23:12:43.733 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-05-22T23:12:43.733 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-05-22T23:12:43.733 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-05-22T23:12:43.733 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-05-22T23:12:43.733 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-05-22T23:12:43.734 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-05-22T23:12:43.734 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 149047, 2024-05-22T23:12:43.734 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 31, 2024-05-22T23:12:43.734 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-05-22T23:12:43.734 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-05-22T23:12:43.734 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-05-22T23:12:43.734 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-05-22T23:12:43.735 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-05-22T23:12:43.735 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-05-22T23:12:43.735 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-05-22T23:12:43.735 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-05-22T23:12:43.735 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 0, 2024-05-22T23:12:43.735 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 0, 2024-05-22T23:12:43.735 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-05-22T23:12:43.735 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-05-22T23:12:43.736 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-05-22T23:12:43.736 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-05-22T23:12:43.736 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 928, 2024-05-22T23:12:43.736 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 2, 2024-05-22T23:12:43.736 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-05-22T23:12:43.736 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-05-22T23:12:43.736 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-05-22T23:12:43.736 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-05-22T23:12:43.737 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-22T23:12:43.737 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-22T23:12:43.737 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-05-22T23:12:43.737 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-05-22T23:12:43.737 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-05-22T23:12:43.737 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 230, 2024-05-22T23:12:43.737 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 1412580, 2024-05-22T23:12:43.737 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-05-22T23:12:43.738 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 230, 2024-05-22T23:12:43.738 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-05-22T23:12:43.738 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-05-22T23:12:43.738 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 230, 2024-05-22T23:12:43.738 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1412580, 2024-05-22T23:12:43.738 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-05-22T23:12:43.738 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-22T23:12:43.738 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-22T23:12:43.739 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-22T23:12:43.739 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-22T23:12:43.739 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-22T23:12:43.739 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-05-22T23:12:44.410 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-05-22T23:12:44.410 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-05-22T23:12:44.410 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-05-22T23:12:44.411 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-22T23:12:44.421 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-22T23:12:44.421 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-22T23:12:44.422 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T23:12:44.443 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-22T23:12:44.444 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T23:12:44.502 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T23:12:44.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.646+0000 7fab91409640 1 -- 172.21.15.84:0/4161221205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab8c10e110 msgr2=0x7fab8c110500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:44.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.646+0000 7fab91409640 1 --2- 172.21.15.84:0/4161221205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab8c10e110 0x7fab8c110500 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fab84009ca0 tx=0x7fab8402a950 comp rx=0 tx=0).stop 2024-05-22T23:12:44.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.646+0000 7fab91409640 1 -- 172.21.15.84:0/4161221205 shutdown_connections 2024-05-22T23:12:44.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.646+0000 7fab91409640 1 --2- 172.21.15.84:0/4161221205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab8c10e110 0x7fab8c110500 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:44.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.646+0000 7fab91409640 1 --2- 172.21.15.84:0/4161221205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab8c074540 0x7fab8c10dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:44.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.646+0000 7fab91409640 1 --2- 172.21.15.84:0/4161221205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab8c073c20 0x7fab8c074000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:44.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.646+0000 7fab91409640 1 -- 172.21.15.84:0/4161221205 >> 172.21.15.84:0/4161221205 conn(0x7fab8c0fd300 msgr2=0x7fab8c0ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:44.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.646+0000 7fab91409640 1 -- 172.21.15.84:0/4161221205 shutdown_connections 2024-05-22T23:12:44.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.646+0000 7fab91409640 1 -- 172.21.15.84:0/4161221205 wait complete. 2024-05-22T23:12:44.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.646+0000 7fab91409640 1 Processor -- start 2024-05-22T23:12:44.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.647+0000 7fab91409640 1 -- start start 2024-05-22T23:12:44.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.647+0000 7fab91409640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab8c073c20 0x7fab8c19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:44.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.647+0000 7fab91409640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab8c074540 0x7fab8c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:44.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.647+0000 7fab91409640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab8c10e110 0x7fab8c19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:44.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.647+0000 7fab91409640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c113e30 con 0x7fab8c10e110 2024-05-22T23:12:44.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.647+0000 7fab91409640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c113cb0 con 0x7fab8c073c20 2024-05-22T23:12:44.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.647+0000 7fab91409640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c113fb0 con 0x7fab8c074540 2024-05-22T23:12:44.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.647+0000 7fab8affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab8c073c20 0x7fab8c19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:44.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.647+0000 7fab8b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab8c10e110 0x7fab8c19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:44.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.647+0000 7fab8affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab8c073c20 0x7fab8c19a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.84:43102/0 (socket says 172.21.15.84:43102) 2024-05-22T23:12:44.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.647+0000 7fab8b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab8c10e110 0x7fab8c19ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40072/0 (socket says 172.21.15.84:40072) 2024-05-22T23:12:44.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.647+0000 7fab8affd640 1 -- 172.21.15.84:0/2232507168 learned_addr learned my addr 172.21.15.84:0/2232507168 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:44.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.647+0000 7fab827fc640 1 --2- 172.21.15.84:0/2232507168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab8c074540 0x7fab8c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:44.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.647+0000 7fab8b7fe640 1 -- 172.21.15.84:0/2232507168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab8c074540 msgr2=0x7fab8c19a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:44.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.647+0000 7fab8b7fe640 1 --2- 172.21.15.84:0/2232507168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab8c074540 0x7fab8c19a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:44.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.647+0000 7fab8b7fe640 1 -- 172.21.15.84:0/2232507168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab8c073c20 msgr2=0x7fab8c19a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:44.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.647+0000 7fab8b7fe640 1 --2- 172.21.15.84:0/2232507168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab8c073c20 0x7fab8c19a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:44.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.647+0000 7fab8b7fe640 1 -- 172.21.15.84:0/2232507168 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c19f320 con 0x7fab8c10e110 2024-05-22T23:12:44.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.648+0000 7fab8b7fe640 1 --2- 172.21.15.84:0/2232507168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab8c10e110 0x7fab8c19ebf0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fab84005cc0 tx=0x7fab8402f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:44.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.648+0000 7fab88ff9640 1 -- 172.21.15.84:0/2232507168 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84034040 con 0x7fab8c10e110 2024-05-22T23:12:44.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.648+0000 7fab88ff9640 1 -- 172.21.15.84:0/2232507168 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab84030920 con 0x7fab8c10e110 2024-05-22T23:12:44.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.648+0000 7fab91409640 1 -- 172.21.15.84:0/2232507168 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8c19f5b0 con 0x7fab8c10e110 2024-05-22T23:12:44.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.648+0000 7fab88ff9640 1 -- 172.21.15.84:0/2232507168 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84004750 con 0x7fab8c10e110 2024-05-22T23:12:44.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.648+0000 7fab91409640 1 -- 172.21.15.84:0/2232507168 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8c1a6ea0 con 0x7fab8c10e110 2024-05-22T23:12:44.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.649+0000 7fab91409640 1 -- 172.21.15.84:0/2232507168 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab8c0752d0 con 0x7fab8c10e110 2024-05-22T23:12:44.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.650+0000 7fab88ff9640 1 -- 172.21.15.84:0/2232507168 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fab84037020 con 0x7fab8c10e110 2024-05-22T23:12:44.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.650+0000 7fab88ff9640 1 --2- 172.21.15.84:0/2232507168 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fab780a2d00 0x7fab780a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:44.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.651+0000 7fab8affd640 1 --2- 172.21.15.84:0/2232507168 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fab780a2d00 0x7fab780a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:44.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.651+0000 7fab88ff9640 1 -- 172.21.15.84:0/2232507168 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8571+0+0 (secure 0 0 0) 0x7fab840f02d0 con 0x7fab8c10e110 2024-05-22T23:12:44.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.651+0000 7fab8affd640 1 --2- 172.21.15.84:0/2232507168 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fab780a2d00 0x7fab780a51c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00bca0 tx=0x7fab7c00b410 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:44.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.655+0000 7fab88ff9640 1 -- 172.21.15.84:0/2232507168 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fab840bcfd0 con 0x7fab8c10e110 2024-05-22T23:12:44.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.832+0000 7fab91409640 1 -- 172.21.15.84:0/2232507168 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab8c109290 con 0x7fab8c10e110 2024-05-22T23:12:44.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.834+0000 7fab88ff9640 1 -- 172.21.15.84:0/2232507168 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+23190 (secure 0 0 0) 0x7fab840bc8d0 con 0x7fab8c10e110 2024-05-22T23:12:44.842 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:12:44.842 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":99,"fsid":"c1135a12-188d-11ef-bc9a-c7b262605968","created":"2024-05-22T22:51:49.878218+0000","modified":"2024-05-22T23:12:17.083988+0000","last_up_change":"2024-05-22T22:58:58.550278+0000","last_in_change":"2024-05-22T22:58:37.322139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T22:55:07.719394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T22:59:50.150511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T22:59:51.111182+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762df065-fb4f-4c71-aab8-a62d2282c253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6803","nonce":1559945826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6805","nonce":1559945826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6809","nonce":1559945826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6807","nonce":1559945826}]},"public_addr":"172.21.15.84:6803/1559945826","cluster_addr":"172.21.15.84:6805/1559945826","heartbeat_back_addr":"172.21.15.84:6809/1559945826","heartbeat_front_addr":"172.21.15.84:6807/1559945826","state":["exists","up"]},{"osd":1,"uuid":"3fb2385d-6a34-49e3-88e1-116237beb346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6801","nonce":2996505353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6803","nonce":2996505353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6807","nonce":2996505353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6805","nonce":2996505353}]},"public_addr":"172.21.15.96:6801/2996505353","cluster_addr":"172.21.15.96:6803/2996505353","heartbeat_back_addr":"172.21.15.96:6807/2996505353","heartbeat_front_addr":"172.21.15.96:6805/2996505353","state":["exists","up"]},{"osd":2,"uuid":"e306e88e-a308-4585-ae11-c8af10b84038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6801","nonce":1499577897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6803","nonce":1499577897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6807","nonce":1499577897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6805","nonce":1499577897}]},"public_addr":"172.21.15.204:6801/1499577897","cluster_addr":"172.21.15.204:6803/1499577897","heartbeat_back_addr":"172.21.15.204:6807/1499577897","heartbeat_front_addr":"172.21.15.204:6805/1499577897","state":["exists","up"]},{"osd":3,"uuid":"d2a5fb97-6f60-4d45-9f18-447cae480974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6811","nonce":2869340648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6813","nonce":2869340648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6817","nonce":2869340648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6815","nonce":2869340648}]},"public_addr":"172.21.15.84:6811/2869340648","cluster_addr":"172.21.15.84:6813/2869340648","heartbeat_back_addr":"172.21.15.84:6817/2869340648","heartbeat_front_addr":"172.21.15.84:6815/2869340648","state":["exists","up"]},{"osd":4,"uuid":"4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6809","nonce":2554265648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6811","nonce":2554265648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6815","nonce":2554265648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6813","nonce":2554265648}]},"public_addr":"172.21.15.96:6809/2554265648","cluster_addr":"172.21.15.96:6811/2554265648","heartbeat_back_addr":"172.21.15.96:6815/2554265648","heartbeat_front_addr":"172.21.15.96:6813/2554265648","state":["exists","up"]},{"osd":5,"uuid":"7996bf5e-93a5-41b0-a9b1-99e7ef4514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6809","nonce":4075476986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6811","nonce":4075476986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6815","nonce":4075476986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6813","nonce":4075476986}]},"public_addr":"172.21.15.204:6809/4075476986","cluster_addr":"172.21.15.204:6811/4075476986","heartbeat_back_addr":"172.21.15.204:6815/4075476986","heartbeat_front_addr":"172.21.15.204:6813/4075476986","state":["exists","up"]},{"osd":6,"uuid":"f4b7c5de-1836-4d16-8603-3c517524d9fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6819","nonce":1806202865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6821","nonce":1806202865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6825","nonce":1806202865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6823","nonce":1806202865}]},"public_addr":"172.21.15.84:6819/1806202865","cluster_addr":"172.21.15.84:6821/1806202865","heartbeat_back_addr":"172.21.15.84:6825/1806202865","heartbeat_front_addr":"172.21.15.84:6823/1806202865","state":["exists","up"]},{"osd":7,"uuid":"69a18c43-5447-480d-9466-190830070c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6817","nonce":3195792774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6819","nonce":3195792774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6823","nonce":3195792774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6821","nonce":3195792774}]},"public_addr":"172.21.15.96:6817/3195792774","cluster_addr":"172.21.15.96:6819/3195792774","heartbeat_back_addr":"172.21.15.96:6823/3195792774","heartbeat_front_addr":"172.21.15.96:6821/3195792774","state":["exists","up"]},{"osd":8,"uuid":"2c5cadca-689a-441e-847c-ba370d400ef1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6817","nonce":606997927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6819","nonce":606997927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6823","nonce":606997927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6821","nonce":606997927}]},"public_addr":"172.21.15.204:6817/606997927","cluster_addr":"172.21.15.204:6819/606997927","heartbeat_back_addr":"172.21.15.204:6823/606997927","heartbeat_front_addr":"172.21.15.204:6821/606997927","state":["exists","up"]},{"osd":9,"uuid":"e3984fb2-7c00-4275-a10e-bca7c859f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6827","nonce":2786189012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6829","nonce":2786189012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6833","nonce":2786189012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6831","nonce":2786189012}]},"public_addr":"172.21.15.84:6827/2786189012","cluster_addr":"172.21.15.84:6829/2786189012","heartbeat_back_addr":"172.21.15.84:6833/2786189012","heartbeat_front_addr":"172.21.15.84:6831/2786189012","state":["exists","up"]},{"osd":10,"uuid":"b4938160-a808-4ded-91c2-d245c99fff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6825","nonce":1985398526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6827","nonce":1985398526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6831","nonce":1985398526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6829","nonce":1985398526}]},"public_addr":"172.21.15.96:6825/1985398526","cluster_addr":"172.21.15.96:6827/1985398526","heartbeat_back_addr":"172.21.15.96:6831/1985398526","heartbeat_front_addr":"172.21.15.96:6829/1985398526","state":["exists","up"]},{"osd":11,"uuid":"238922ef-d046-40cc-833d-25b7e7a523cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6825","nonce":3106116519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6827","nonce":3106116519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6831","nonce":3106116519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6829","nonce":3106116519}]},"public_addr":"172.21.15.204:6825/3106116519","cluster_addr":"172.21.15.204:6827/3106116519","heartbeat_back_addr":"172.21.15.204:6831/3106116519","heartbeat_front_addr":"172.21.15.204:6829/3106116519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:16.443480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:39.898133+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:03.664544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:28.838687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:53.829733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:19.269307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:45.166910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:10.771614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:36.678516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:03.150592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:29.327769+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:56.315938+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2317224739":"2024-05-23T23:01:49.869856+0000","172.21.15.84:0/4170743349":"2024-05-23T23:01:49.869856+0000","172.21.15.84:6801/4223589140":"2024-05-23T23:01:49.869856+0000","172.21.15.84:6800/4223589140":"2024-05-23T23:01:49.869856+0000","172.21.15.84:6801/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:6800/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/2766288608":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/950455999":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/3490117794":"2024-05-23T23:01:49.869856+0000","172.21.15.84:0/4000289608":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1650467363":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/4077200298":"2024-05-23T23:01:49.869856+0000","172.21.15.84:0/1052746114":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6800/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6801/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/14610813":"2024-05-23T22:52:45.395791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T23:12:44.844 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.836+0000 7fab91409640 1 -- 172.21.15.84:0/2232507168 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fab780a2d00 msgr2=0x7fab780a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:44.844 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.836+0000 7fab91409640 1 --2- 172.21.15.84:0/2232507168 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fab780a2d00 0x7fab780a51c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00bca0 tx=0x7fab7c00b410 comp rx=0 tx=0).stop 2024-05-22T23:12:44.844 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.836+0000 7fab91409640 1 -- 172.21.15.84:0/2232507168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab8c10e110 msgr2=0x7fab8c19ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:44.844 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.836+0000 7fab91409640 1 --2- 172.21.15.84:0/2232507168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab8c10e110 0x7fab8c19ebf0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fab84005cc0 tx=0x7fab8402f630 comp rx=0 tx=0).stop 2024-05-22T23:12:44.844 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.836+0000 7fab91409640 1 -- 172.21.15.84:0/2232507168 shutdown_connections 2024-05-22T23:12:44.844 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.836+0000 7fab91409640 1 --2- 172.21.15.84:0/2232507168 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fab780a2d00 0x7fab780a51c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:44.845 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.836+0000 7fab91409640 1 --2- 172.21.15.84:0/2232507168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab8c10e110 0x7fab8c19ebf0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:44.845 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.836+0000 7fab91409640 1 --2- 172.21.15.84:0/2232507168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab8c074540 0x7fab8c19a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:44.845 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.836+0000 7fab91409640 1 --2- 172.21.15.84:0/2232507168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab8c073c20 0x7fab8c19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:44.845 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.836+0000 7fab91409640 1 -- 172.21.15.84:0/2232507168 >> 172.21.15.84:0/2232507168 conn(0x7fab8c0fd300 msgr2=0x7fab8c0fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:44.845 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.836+0000 7fab91409640 1 -- 172.21.15.84:0/2232507168 shutdown_connections 2024-05-22T23:12:44.845 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.836+0000 7fab91409640 1 -- 172.21.15.84:0/2232507168 wait complete. 2024-05-22T23:12:44.855 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T22:55:07.719394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T22:59:50.150511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T22:59:51.111182+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 98, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T23:12:44.856 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T23:12:44.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.953+0000 7f71de876640 1 -- 172.21.15.84:0/4163002207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71d8102af0 msgr2=0x7f71d8102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:44.962 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.953+0000 7f71de876640 1 --2- 172.21.15.84:0/4163002207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71d8102af0 0x7f71d8102ed0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f71c8009900 tx=0x7f71c802a510 comp rx=0 tx=0).stop 2024-05-22T23:12:44.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.953+0000 7f71de876640 1 -- 172.21.15.84:0/4163002207 shutdown_connections 2024-05-22T23:12:44.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.953+0000 7f71de876640 1 --2- 172.21.15.84:0/4163002207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71d810be30 0x7f71d810e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:44.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.953+0000 7f71de876640 1 --2- 172.21.15.84:0/4163002207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71d8103410 0x7f71d810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:44.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.953+0000 7f71de876640 1 --2- 172.21.15.84:0/4163002207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71d8102af0 0x7f71d8102ed0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:44.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.953+0000 7f71de876640 1 -- 172.21.15.84:0/4163002207 >> 172.21.15.84:0/4163002207 conn(0x7f71d8077870 msgr2=0x7f71d8077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:44.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.953+0000 7f71de876640 1 -- 172.21.15.84:0/4163002207 shutdown_connections 2024-05-22T23:12:44.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.953+0000 7f71de876640 1 -- 172.21.15.84:0/4163002207 wait complete. 2024-05-22T23:12:44.963 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.953+0000 7f71de876640 1 Processor -- start 2024-05-22T23:12:44.964 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.953+0000 7f71de876640 1 -- start start 2024-05-22T23:12:44.964 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.953+0000 7f71de876640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71d8102af0 0x7f71d8100740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:44.964 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.954+0000 7f71de876640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71d8103410 0x7f71d8100c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:44.964 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.954+0000 7f71de876640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71d810be30 0x7f71d81011c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:44.964 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.954+0000 7f71de876640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f71d8111c30 con 0x7f71d810be30 2024-05-22T23:12:44.964 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.954+0000 7f71de876640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f71d8111ab0 con 0x7f71d8103410 2024-05-22T23:12:44.964 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.954+0000 7f71de876640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f71d8111db0 con 0x7f71d8102af0 2024-05-22T23:12:44.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.954+0000 7f71dce13640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71d810be30 0x7f71d81011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:44.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.954+0000 7f71dce13640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71d810be30 0x7f71d81011c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40084/0 (socket says 172.21.15.84:40084) 2024-05-22T23:12:44.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.954+0000 7f71dce13640 1 -- 172.21.15.84:0/1925107690 learned_addr learned my addr 172.21.15.84:0/1925107690 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:44.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.954+0000 7f71dce13640 1 -- 172.21.15.84:0/1925107690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71d8102af0 msgr2=0x7f71d8100740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:44.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.954+0000 7f71dce13640 1 --2- 172.21.15.84:0/1925107690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71d8102af0 0x7f71d8100740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:44.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.954+0000 7f71dce13640 1 -- 172.21.15.84:0/1925107690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71d8103410 msgr2=0x7f71d8100c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:44.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.954+0000 7f71dce13640 1 --2- 172.21.15.84:0/1925107690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71d8103410 0x7f71d8100c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:44.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.954+0000 7f71dce13640 1 -- 172.21.15.84:0/1925107690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d81a6610 con 0x7f71d810be30 2024-05-22T23:12:44.965 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.954+0000 7f71dce13640 1 --2- 172.21.15.84:0/1925107690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71d810be30 0x7f71d81011c0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f71d000eaa0 tx=0x7f71d000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:44.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.954+0000 7f71d5ffb640 1 -- 172.21.15.84:0/1925107690 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d000cdb0 con 0x7f71d810be30 2024-05-22T23:12:44.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.954+0000 7f71d5ffb640 1 -- 172.21.15.84:0/1925107690 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d0005020 con 0x7f71d810be30 2024-05-22T23:12:44.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.954+0000 7f71d5ffb640 1 -- 172.21.15.84:0/1925107690 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d00116b0 con 0x7f71d810be30 2024-05-22T23:12:44.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.954+0000 7f71de876640 1 -- 172.21.15.84:0/1925107690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d81a6900 con 0x7f71d810be30 2024-05-22T23:12:44.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.954+0000 7f71de876640 1 -- 172.21.15.84:0/1925107690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d81a6e00 con 0x7f71d810be30 2024-05-22T23:12:44.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.955+0000 7f71de876640 1 -- 172.21.15.84:0/1925107690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d8068690 con 0x7f71d810be30 2024-05-22T23:12:44.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.956+0000 7f71d5ffb640 1 -- 172.21.15.84:0/1925107690 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f71d0005560 con 0x7f71d810be30 2024-05-22T23:12:44.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.957+0000 7f71d5ffb640 1 --2- 172.21.15.84:0/1925107690 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f71c40a2d80 0x7f71c40a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:44.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.957+0000 7f71d7fff640 1 --2- 172.21.15.84:0/1925107690 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f71c40a2d80 0x7f71c40a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:44.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.957+0000 7f71d5ffb640 1 -- 172.21.15.84:0/1925107690 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8571+0+0 (secure 0 0 0) 0x7f71d00cf660 con 0x7f71d810be30 2024-05-22T23:12:44.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.957+0000 7f71d7fff640 1 --2- 172.21.15.84:0/1925107690 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f71c40a2d80 0x7f71c40a5240 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f71c802aa30 tx=0x7f71c8000bc0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:44.969 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:44.961+0000 7f71d5ffb640 1 -- 172.21.15.84:0/1925107690 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f71d0016030 con 0x7f71d810be30 2024-05-22T23:12:45.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.137+0000 7f71de876640 1 -- 172.21.15.84:0/1925107690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f71d8106fb0 con 0x7f71d810be30 2024-05-22T23:12:45.145 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.137+0000 7f71d5ffb640 1 -- 172.21.15.84:0/1925107690 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7f71d009c250 con 0x7f71d810be30 2024-05-22T23:12:45.145 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2024-05-22T23:12:45.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.139+0000 7f71de876640 1 -- 172.21.15.84:0/1925107690 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f71c40a2d80 msgr2=0x7f71c40a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:45.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.139+0000 7f71de876640 1 --2- 172.21.15.84:0/1925107690 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f71c40a2d80 0x7f71c40a5240 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f71c802aa30 tx=0x7f71c8000bc0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.139+0000 7f71de876640 1 -- 172.21.15.84:0/1925107690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71d810be30 msgr2=0x7f71d81011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:45.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.139+0000 7f71de876640 1 --2- 172.21.15.84:0/1925107690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71d810be30 0x7f71d81011c0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f71d000eaa0 tx=0x7f71d000ef60 comp rx=0 tx=0).stop 2024-05-22T23:12:45.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.139+0000 7f71de876640 1 -- 172.21.15.84:0/1925107690 shutdown_connections 2024-05-22T23:12:45.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.139+0000 7f71de876640 1 --2- 172.21.15.84:0/1925107690 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f71c40a2d80 0x7f71c40a5240 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.139+0000 7f71de876640 1 --2- 172.21.15.84:0/1925107690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71d810be30 0x7f71d81011c0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.139+0000 7f71de876640 1 --2- 172.21.15.84:0/1925107690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71d8103410 0x7f71d8100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.139+0000 7f71de876640 1 --2- 172.21.15.84:0/1925107690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71d8102af0 0x7f71d8100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.139+0000 7f71de876640 1 -- 172.21.15.84:0/1925107690 >> 172.21.15.84:0/1925107690 conn(0x7f71d8077870 msgr2=0x7f71d80fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:45.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.139+0000 7f71de876640 1 -- 172.21.15.84:0/1925107690 shutdown_connections 2024-05-22T23:12:45.149 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.139+0000 7f71de876640 1 -- 172.21.15.84:0/1925107690 wait complete. 2024-05-22T23:12:45.158 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T23:12:45.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.254+0000 7ff90c190640 1 -- 172.21.15.84:0/4125200545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff904107860 msgr2=0x7ff904107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:45.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.254+0000 7ff90c190640 1 --2- 172.21.15.84:0/4125200545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff904107860 0x7ff904107c40 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7ff900009930 tx=0x7ff90002a420 comp rx=0 tx=0).stop 2024-05-22T23:12:45.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.254+0000 7ff90c190640 1 -- 172.21.15.84:0/4125200545 shutdown_connections 2024-05-22T23:12:45.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.254+0000 7ff90c190640 1 --2- 172.21.15.84:0/4125200545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff90406a8b0 0x7ff90410e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.254+0000 7ff90c190640 1 --2- 172.21.15.84:0/4125200545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff904069ef0 0x7ff90406a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.254+0000 7ff90c190640 1 --2- 172.21.15.84:0/4125200545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff904107860 0x7ff904107c40 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.254+0000 7ff90c190640 1 -- 172.21.15.84:0/4125200545 >> 172.21.15.84:0/4125200545 conn(0x7ff9040777d0 msgr2=0x7ff904077be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:45.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.254+0000 7ff90c190640 1 -- 172.21.15.84:0/4125200545 shutdown_connections 2024-05-22T23:12:45.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.254+0000 7ff90c190640 1 -- 172.21.15.84:0/4125200545 wait complete. 2024-05-22T23:12:45.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.255+0000 7ff90c190640 1 Processor -- start 2024-05-22T23:12:45.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.255+0000 7ff90c190640 1 -- start start 2024-05-22T23:12:45.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.255+0000 7ff90c190640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff904069ef0 0x7ff9041980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:45.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.255+0000 7ff90c190640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff90406a8b0 0x7ff904198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:45.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.255+0000 7ff90c190640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff904107860 0x7ff90419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:45.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.255+0000 7ff90c190640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff904111c70 con 0x7ff904107860 2024-05-22T23:12:45.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.255+0000 7ff90c190640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff904111af0 con 0x7ff90406a8b0 2024-05-22T23:12:45.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.255+0000 7ff90c190640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff904111df0 con 0x7ff904069ef0 2024-05-22T23:12:45.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.255+0000 7ff90a72d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff904107860 0x7ff90419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:45.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.255+0000 7ff90a72d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff904107860 0x7ff90419c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40104/0 (socket says 172.21.15.84:40104) 2024-05-22T23:12:45.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.255+0000 7ff90a72d640 1 -- 172.21.15.84:0/1839508304 learned_addr learned my addr 172.21.15.84:0/1839508304 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:45.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.255+0000 7ff909f2c640 1 --2- 172.21.15.84:0/1839508304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff904069ef0 0x7ff9041980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:45.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.255+0000 7ff90972b640 1 --2- 172.21.15.84:0/1839508304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff90406a8b0 0x7ff904198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:45.267 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.255+0000 7ff90a72d640 1 -- 172.21.15.84:0/1839508304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff904069ef0 msgr2=0x7ff9041980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:45.267 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.255+0000 7ff90a72d640 1 --2- 172.21.15.84:0/1839508304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff904069ef0 0x7ff9041980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.267 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.255+0000 7ff90a72d640 1 -- 172.21.15.84:0/1839508304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff90406a8b0 msgr2=0x7ff904198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:45.267 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.255+0000 7ff90a72d640 1 --2- 172.21.15.84:0/1839508304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff90406a8b0 0x7ff904198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.267 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.255+0000 7ff90a72d640 1 -- 172.21.15.84:0/1839508304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff90419d0e0 con 0x7ff904107860 2024-05-22T23:12:45.267 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.256+0000 7ff90a72d640 1 --2- 172.21.15.84:0/1839508304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff904107860 0x7ff90419c9b0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00eb10 tx=0x7ff8fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:45.267 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.256+0000 7ff8faffd640 1 -- 172.21.15.84:0/1839508304 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc004b20 con 0x7ff904107860 2024-05-22T23:12:45.267 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.256+0000 7ff8faffd640 1 -- 172.21.15.84:0/1839508304 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8fc004cc0 con 0x7ff904107860 2024-05-22T23:12:45.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.256+0000 7ff90c190640 1 -- 172.21.15.84:0/1839508304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff90419d3d0 con 0x7ff904107860 2024-05-22T23:12:45.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.256+0000 7ff8faffd640 1 -- 172.21.15.84:0/1839508304 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc010760 con 0x7ff904107860 2024-05-22T23:12:45.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.256+0000 7ff90c190640 1 -- 172.21.15.84:0/1839508304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9041a4c60 con 0x7ff904107860 2024-05-22T23:12:45.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.257+0000 7ff90c190640 1 -- 172.21.15.84:0/1839508304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff904068ae0 con 0x7ff904107860 2024-05-22T23:12:45.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.258+0000 7ff8faffd640 1 -- 172.21.15.84:0/1839508304 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff8fc005000 con 0x7ff904107860 2024-05-22T23:12:45.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.258+0000 7ff8faffd640 1 --2- 172.21.15.84:0/1839508304 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff8f00a2d80 0x7ff8f00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:45.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.259+0000 7ff909f2c640 1 --2- 172.21.15.84:0/1839508304 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff8f00a2d80 0x7ff8f00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:45.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.259+0000 7ff8faffd640 1 -- 172.21.15.84:0/1839508304 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8571+0+0 (secure 0 0 0) 0x7ff8fc0cfe40 con 0x7ff904107860 2024-05-22T23:12:45.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.259+0000 7ff909f2c640 1 --2- 172.21.15.84:0/1839508304 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff8f00a2d80 0x7ff8f00a5240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff900009900 tx=0x7ff9000058d0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:45.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.262+0000 7ff8faffd640 1 -- 172.21.15.84:0/1839508304 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff8fc09ca30 con 0x7ff904107860 2024-05-22T23:12:45.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:45 smithi204 ceph-mon[36211]: pgmap v340: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:12:45.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:45 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2232507168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:12:45.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.439+0000 7ff90c190640 1 -- 172.21.15.84:0/1839508304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff90402d200 con 0x7ff904107860 2024-05-22T23:12:45.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.439+0000 7ff8faffd640 1 -- 172.21.15.84:0/1839508304 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7ff8fc09c330 con 0x7ff904107860 2024-05-22T23:12:45.447 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2024-05-22T23:12:45.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.441+0000 7ff90c190640 1 -- 172.21.15.84:0/1839508304 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff8f00a2d80 msgr2=0x7ff8f00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:45.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.441+0000 7ff90c190640 1 --2- 172.21.15.84:0/1839508304 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff8f00a2d80 0x7ff8f00a5240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff900009900 tx=0x7ff9000058d0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.441+0000 7ff90c190640 1 -- 172.21.15.84:0/1839508304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff904107860 msgr2=0x7ff90419c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:45.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.441+0000 7ff90c190640 1 --2- 172.21.15.84:0/1839508304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff904107860 0x7ff90419c9b0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00eb10 tx=0x7ff8fc00c2d0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.441+0000 7ff90c190640 1 -- 172.21.15.84:0/1839508304 shutdown_connections 2024-05-22T23:12:45.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.441+0000 7ff90c190640 1 --2- 172.21.15.84:0/1839508304 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff8f00a2d80 0x7ff8f00a5240 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.441+0000 7ff90c190640 1 --2- 172.21.15.84:0/1839508304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff904107860 0x7ff90419c9b0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.450 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.441+0000 7ff90c190640 1 --2- 172.21.15.84:0/1839508304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff90406a8b0 0x7ff904198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.441+0000 7ff90c190640 1 --2- 172.21.15.84:0/1839508304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff904069ef0 0x7ff9041980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.441+0000 7ff90c190640 1 -- 172.21.15.84:0/1839508304 >> 172.21.15.84:0/1839508304 conn(0x7ff9040777d0 msgr2=0x7ff9040fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:45.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.442+0000 7ff90c190640 1 -- 172.21.15.84:0/1839508304 shutdown_connections 2024-05-22T23:12:45.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.442+0000 7ff90c190640 1 -- 172.21.15.84:0/1839508304 wait complete. 2024-05-22T23:12:45.459 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:45 smithi084 ceph-mon[32186]: pgmap v340: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:12:45.459 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:45 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2232507168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:12:45.460 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T23:12:45.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:45 smithi096 ceph-mon[36392]: pgmap v340: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-22T23:12:45.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:45 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2232507168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:12:45.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.556+0000 7f4050eb3640 1 -- 172.21.15.84:0/4094874116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f404c100740 msgr2=0x7f404c100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:45.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.556+0000 7f4050eb3640 1 --2- 172.21.15.84:0/4094874116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f404c100740 0x7f404c100bc0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f4034009930 tx=0x7f403402a420 comp rx=0 tx=0).stop 2024-05-22T23:12:45.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.556+0000 7f4050eb3640 1 -- 172.21.15.84:0/4094874116 shutdown_connections 2024-05-22T23:12:45.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.556+0000 7f4050eb3640 1 --2- 172.21.15.84:0/4094874116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f404c101100 0x7f404c10d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.556+0000 7f4050eb3640 1 --2- 172.21.15.84:0/4094874116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f404c100740 0x7f404c100bc0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.556+0000 7f4050eb3640 1 --2- 172.21.15.84:0/4094874116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f404c106750 0x7f404c106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.556+0000 7f4050eb3640 1 -- 172.21.15.84:0/4094874116 >> 172.21.15.84:0/4094874116 conn(0x7f404c077870 msgr2=0x7f404c077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:45.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.556+0000 7f4050eb3640 1 -- 172.21.15.84:0/4094874116 shutdown_connections 2024-05-22T23:12:45.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.556+0000 7f4050eb3640 1 -- 172.21.15.84:0/4094874116 wait complete. 2024-05-22T23:12:45.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.557+0000 7f4050eb3640 1 Processor -- start 2024-05-22T23:12:45.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.557+0000 7f4050eb3640 1 -- start start 2024-05-22T23:12:45.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.557+0000 7f4050eb3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f404c100740 0x7f404c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:45.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.557+0000 7f4050eb3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f404c101100 0x7f404c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:45.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.557+0000 7f4050eb3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f404c106750 0x7f404c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:45.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.557+0000 7f4050eb3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f404c110b60 con 0x7f404c106750 2024-05-22T23:12:45.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.557+0000 7f4050eb3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f404c1109e0 con 0x7f404c100740 2024-05-22T23:12:45.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.557+0000 7f4050eb3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f404c110ce0 con 0x7f404c101100 2024-05-22T23:12:45.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.557+0000 7f404ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f404c106750 0x7f404c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:45.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.557+0000 7f404ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f404c106750 0x7f404c19ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40122/0 (socket says 172.21.15.84:40122) 2024-05-22T23:12:45.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.557+0000 7f404ad9d640 1 -- 172.21.15.84:0/1076889820 learned_addr learned my addr 172.21.15.84:0/1076889820 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:45.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.557+0000 7f404a59c640 1 --2- 172.21.15.84:0/1076889820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f404c100740 0x7f404c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:45.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.557+0000 7f4043fff640 1 --2- 172.21.15.84:0/1076889820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f404c101100 0x7f404c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:45.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.557+0000 7f404ad9d640 1 -- 172.21.15.84:0/1076889820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f404c101100 msgr2=0x7f404c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:45.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.557+0000 7f404ad9d640 1 --2- 172.21.15.84:0/1076889820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f404c101100 0x7f404c198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.557+0000 7f404ad9d640 1 -- 172.21.15.84:0/1076889820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f404c100740 msgr2=0x7f404c1981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:45.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.557+0000 7f404ad9d640 1 --2- 172.21.15.84:0/1076889820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f404c100740 0x7f404c1981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.557+0000 7f404ad9d640 1 -- 172.21.15.84:0/1076889820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404c19d170 con 0x7f404c106750 2024-05-22T23:12:45.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.557+0000 7f4043fff640 1 --2- 172.21.15.84:0/1076889820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f404c101100 0x7f404c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:12:45.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.557+0000 7f404ad9d640 1 --2- 172.21.15.84:0/1076889820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f404c106750 0x7f404c19ca40 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f404400eaa0 tx=0x7f404400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:45.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.557+0000 7f40437fe640 1 -- 172.21.15.84:0/1076889820 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404400cdb0 con 0x7f404c106750 2024-05-22T23:12:45.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.558+0000 7f4050eb3640 1 -- 172.21.15.84:0/1076889820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404c19d460 con 0x7f404c106750 2024-05-22T23:12:45.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.558+0000 7f40437fe640 1 -- 172.21.15.84:0/1076889820 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4044005020 con 0x7f404c106750 2024-05-22T23:12:45.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.558+0000 7f40437fe640 1 -- 172.21.15.84:0/1076889820 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40440116b0 con 0x7f404c106750 2024-05-22T23:12:45.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.558+0000 7f4050eb3640 1 -- 172.21.15.84:0/1076889820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f404c1a4cf0 con 0x7f404c106750 2024-05-22T23:12:45.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.559+0000 7f4050eb3640 1 -- 172.21.15.84:0/1076889820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f404c068690 con 0x7f404c106750 2024-05-22T23:12:45.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.560+0000 7f40437fe640 1 -- 172.21.15.84:0/1076889820 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4044005560 con 0x7f404c106750 2024-05-22T23:12:45.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.560+0000 7f40437fe640 1 --2- 172.21.15.84:0/1076889820 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f40380a2d00 0x7f40380a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:45.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.561+0000 7f404a59c640 1 --2- 172.21.15.84:0/1076889820 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f40380a2d00 0x7f40380a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:45.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.561+0000 7f40437fe640 1 -- 172.21.15.84:0/1076889820 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8571+0+0 (secure 0 0 0) 0x7f40440cfa70 con 0x7f404c106750 2024-05-22T23:12:45.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.561+0000 7f404a59c640 1 --2- 172.21.15.84:0/1076889820 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f40380a2d00 0x7f40380a51c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f403c005fd0 tx=0x7f403c00b390 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:45.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.564+0000 7f40437fe640 1 -- 172.21.15.84:0/1076889820 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4044016030 con 0x7f404c106750 2024-05-22T23:12:45.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.739+0000 7f4050eb3640 1 -- 172.21.15.84:0/1076889820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f404c1a4f80 con 0x7f404c106750 2024-05-22T23:12:45.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.739+0000 7f40437fe640 1 -- 172.21.15.84:0/1076889820 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7f404409c660 con 0x7f404c106750 2024-05-22T23:12:45.747 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2024-05-22T23:12:45.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.741+0000 7f4050eb3640 1 -- 172.21.15.84:0/1076889820 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f40380a2d00 msgr2=0x7f40380a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:45.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.741+0000 7f4050eb3640 1 --2- 172.21.15.84:0/1076889820 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f40380a2d00 0x7f40380a51c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f403c005fd0 tx=0x7f403c00b390 comp rx=0 tx=0).stop 2024-05-22T23:12:45.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.741+0000 7f4050eb3640 1 -- 172.21.15.84:0/1076889820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f404c106750 msgr2=0x7f404c19ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:45.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.741+0000 7f4050eb3640 1 --2- 172.21.15.84:0/1076889820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f404c106750 0x7f404c19ca40 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f404400eaa0 tx=0x7f404400ef60 comp rx=0 tx=0).stop 2024-05-22T23:12:45.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.741+0000 7f4050eb3640 1 -- 172.21.15.84:0/1076889820 shutdown_connections 2024-05-22T23:12:45.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.741+0000 7f4050eb3640 1 --2- 172.21.15.84:0/1076889820 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f40380a2d00 0x7f40380a51c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.741+0000 7f4050eb3640 1 --2- 172.21.15.84:0/1076889820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f404c106750 0x7f404c19ca40 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.741+0000 7f4050eb3640 1 --2- 172.21.15.84:0/1076889820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f404c101100 0x7f404c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.741+0000 7f4050eb3640 1 --2- 172.21.15.84:0/1076889820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f404c100740 0x7f404c1981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.741+0000 7f4050eb3640 1 -- 172.21.15.84:0/1076889820 >> 172.21.15.84:0/1076889820 conn(0x7f404c077870 msgr2=0x7f404c0ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:45.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.741+0000 7f4050eb3640 1 -- 172.21.15.84:0/1076889820 shutdown_connections 2024-05-22T23:12:45.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.742+0000 7f4050eb3640 1 -- 172.21.15.84:0/1076889820 wait complete. 2024-05-22T23:12:45.760 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T23:12:45.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.854+0000 7f14f9753640 1 -- 172.21.15.84:0/1525604705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14f4101840 msgr2=0x7f14f4101cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:45.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.854+0000 7f14f9753640 1 --2- 172.21.15.84:0/1525604705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14f4101840 0x7f14f4101cc0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009930 tx=0x7f14dc02a420 comp rx=0 tx=0).stop 2024-05-22T23:12:45.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.854+0000 7f14f9753640 1 -- 172.21.15.84:0/1525604705 shutdown_connections 2024-05-22T23:12:45.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.854+0000 7f14f9753640 1 --2- 172.21.15.84:0/1525604705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14f4102200 0x7f14f410e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.854+0000 7f14f9753640 1 --2- 172.21.15.84:0/1525604705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14f4101840 0x7f14f4101cc0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.854+0000 7f14f9753640 1 --2- 172.21.15.84:0/1525604705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14f4107850 0x7f14f4107c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.854+0000 7f14f9753640 1 -- 172.21.15.84:0/1525604705 >> 172.21.15.84:0/1525604705 conn(0x7f14f4078cc0 msgr2=0x7f14f40790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:45.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.854+0000 7f14f9753640 1 -- 172.21.15.84:0/1525604705 shutdown_connections 2024-05-22T23:12:45.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.854+0000 7f14f9753640 1 -- 172.21.15.84:0/1525604705 wait complete. 2024-05-22T23:12:45.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.854+0000 7f14f9753640 1 Processor -- start 2024-05-22T23:12:45.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.855+0000 7f14f9753640 1 -- start start 2024-05-22T23:12:45.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.855+0000 7f14f9753640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14f4101840 0x7f14f4073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:45.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.855+0000 7f14f9753640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14f4102200 0x7f14f40740b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:45.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.855+0000 7f14f9753640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14f4107850 0x7f14f40745f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:45.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.855+0000 7f14f9753640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f14f4111c20 con 0x7f14f4101840 2024-05-22T23:12:45.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.855+0000 7f14f9753640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f14f4111aa0 con 0x7f14f4107850 2024-05-22T23:12:45.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.855+0000 7f14f9753640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f14f4111da0 con 0x7f14f4102200 2024-05-22T23:12:45.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.855+0000 7f14f2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14f4101840 0x7f14f4073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:45.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.855+0000 7f14f2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14f4101840 0x7f14f4073b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40132/0 (socket says 172.21.15.84:40132) 2024-05-22T23:12:45.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.855+0000 7f14f2ffd640 1 -- 172.21.15.84:0/231349279 learned_addr learned my addr 172.21.15.84:0/231349279 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:45.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.855+0000 7f14f2ffd640 1 -- 172.21.15.84:0/231349279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14f4102200 msgr2=0x7f14f40740b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:45.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.855+0000 7f14f2ffd640 1 --2- 172.21.15.84:0/231349279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14f4102200 0x7f14f40740b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.855+0000 7f14f2ffd640 1 -- 172.21.15.84:0/231349279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14f4107850 msgr2=0x7f14f40745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:45.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.855+0000 7f14f2ffd640 1 --2- 172.21.15.84:0/231349279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14f4107850 0x7f14f40745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:45.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.855+0000 7f14f2ffd640 1 -- 172.21.15.84:0/231349279 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f41a6660 con 0x7f14f4101840 2024-05-22T23:12:45.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.855+0000 7f14f2ffd640 1 --2- 172.21.15.84:0/231349279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14f4101840 0x7f14f4073b70 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f14e400c920 tx=0x7f14e400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:45.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.855+0000 7f14ebfff640 1 -- 172.21.15.84:0/231349279 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4007c70 con 0x7f14f4101840 2024-05-22T23:12:45.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.855+0000 7f14f9753640 1 -- 172.21.15.84:0/231349279 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f41a6950 con 0x7f14f4101840 2024-05-22T23:12:45.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.855+0000 7f14ebfff640 1 -- 172.21.15.84:0/231349279 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e4007e10 con 0x7f14f4101840 2024-05-22T23:12:45.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.855+0000 7f14ebfff640 1 -- 172.21.15.84:0/231349279 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4018890 con 0x7f14f4101840 2024-05-22T23:12:45.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.855+0000 7f14f9753640 1 -- 172.21.15.84:0/231349279 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f41a6ea0 con 0x7f14f4101840 2024-05-22T23:12:45.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.856+0000 7f14f9753640 1 -- 172.21.15.84:0/231349279 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14f4068ae0 con 0x7f14f4101840 2024-05-22T23:12:45.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.858+0000 7f14ebfff640 1 -- 172.21.15.84:0/231349279 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f14e4018a70 con 0x7f14f4101840 2024-05-22T23:12:45.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.858+0000 7f14ebfff640 1 --2- 172.21.15.84:0/231349279 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f14e00a2d00 0x7f14e00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:45.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.858+0000 7f14ebfff640 1 -- 172.21.15.84:0/231349279 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8571+0+0 (secure 0 0 0) 0x7f14e40cf600 con 0x7f14f4101840 2024-05-22T23:12:45.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.858+0000 7f14f27fc640 1 --2- 172.21.15.84:0/231349279 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f14e00a2d00 0x7f14e00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:45.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.859+0000 7f14f27fc640 1 --2- 172.21.15.84:0/231349279 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f14e00a2d00 0x7f14e00a51c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f14dc02a940 tx=0x7f14dc000bc0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:45.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:45.862+0000 7f14ebfff640 1 -- 172.21.15.84:0/231349279 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f14e4014030 con 0x7f14f4101840 2024-05-22T23:12:46.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.054+0000 7f14f9753640 1 -- 172.21.15.84:0/231349279 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f14f4109840 con 0x7f14f4101840 2024-05-22T23:12:46.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.054+0000 7f14ebfff640 1 -- 172.21.15.84:0/231349279 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v99) ==== 81+0+943 (secure 0 0 0) 0x7f14e409c1f0 con 0x7f14f4101840 2024-05-22T23:12:46.063 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2317224739 2024-05-23T23:01:49.869856+0000 2024-05-22T23:12:46.063 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/4170743349 2024-05-23T23:01:49.869856+0000 2024-05-22T23:12:46.063 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6801/4223589140 2024-05-23T23:01:49.869856+0000 2024-05-22T23:12:46.063 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6800/4223589140 2024-05-23T23:01:49.869856+0000 2024-05-22T23:12:46.063 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6801/3278959854 2024-05-23T22:52:45.395791+0000 2024-05-22T23:12:46.063 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6800/3278959854 2024-05-23T22:52:45.395791+0000 2024-05-22T23:12:46.063 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2766288608 2024-05-23T22:52:45.395791+0000 2024-05-22T23:12:46.064 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/950455999 2024-05-23T22:52:45.395791+0000 2024-05-22T23:12:46.064 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3490117794 2024-05-23T23:01:49.869856+0000 2024-05-22T23:12:46.064 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/4000289608 2024-05-23T22:52:18.615994+0000 2024-05-22T23:12:46.064 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1650467363 2024-05-23T22:52:18.615994+0000 2024-05-22T23:12:46.064 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/4077200298 2024-05-23T23:01:49.869856+0000 2024-05-22T23:12:46.064 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1052746114 2024-05-23T22:52:18.615994+0000 2024-05-22T23:12:46.064 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6800/4029123070 2024-05-23T22:52:18.615994+0000 2024-05-22T23:12:46.064 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6801/4029123070 2024-05-23T22:52:18.615994+0000 2024-05-22T23:12:46.065 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/14610813 2024-05-23T22:52:45.395791+0000 2024-05-22T23:12:46.065 INFO:teuthology.orchestra.run.smithi084.stderr:listed 16 entries 2024-05-22T23:12:46.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.056+0000 7f14f9753640 1 -- 172.21.15.84:0/231349279 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f14e00a2d00 msgr2=0x7f14e00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:46.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.056+0000 7f14f9753640 1 --2- 172.21.15.84:0/231349279 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f14e00a2d00 0x7f14e00a51c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f14dc02a940 tx=0x7f14dc000bc0 comp rx=0 tx=0).stop 2024-05-22T23:12:46.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.056+0000 7f14f9753640 1 -- 172.21.15.84:0/231349279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14f4101840 msgr2=0x7f14f4073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:46.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.056+0000 7f14f9753640 1 --2- 172.21.15.84:0/231349279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14f4101840 0x7f14f4073b70 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f14e400c920 tx=0x7f14e400cde0 comp rx=0 tx=0).stop 2024-05-22T23:12:46.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.056+0000 7f14f9753640 1 -- 172.21.15.84:0/231349279 shutdown_connections 2024-05-22T23:12:46.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.056+0000 7f14f9753640 1 --2- 172.21.15.84:0/231349279 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f14e00a2d00 0x7f14e00a51c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:46.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.056+0000 7f14f9753640 1 --2- 172.21.15.84:0/231349279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14f4107850 0x7f14f40745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:46.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.056+0000 7f14f9753640 1 --2- 172.21.15.84:0/231349279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14f4102200 0x7f14f40740b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:46.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.056+0000 7f14f9753640 1 --2- 172.21.15.84:0/231349279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14f4101840 0x7f14f4073b70 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:46.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.056+0000 7f14f9753640 1 -- 172.21.15.84:0/231349279 >> 172.21.15.84:0/231349279 conn(0x7f14f4078cc0 msgr2=0x7f14f40ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:46.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.056+0000 7f14f9753640 1 -- 172.21.15.84:0/231349279 shutdown_connections 2024-05-22T23:12:46.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.056+0000 7f14f9753640 1 -- 172.21.15.84:0/231349279 wait complete. 2024-05-22T23:12:46.075 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-22T23:12:46.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:12:46.076 DEBUG:teuthology.orchestra.run.smithi084:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-22T23:12:46.123 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2024-05-22T23:12:46.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:12:46.123 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-22T23:12:46.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:12:46.160 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2024-05-22T23:12:46.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:46 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1925107690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:12:46.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:46 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1839508304' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:12:46.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:46 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1076889820' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:12:46.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:46 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/231349279' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-22T23:12:46.215 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-22T23:12:46.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:12:46.215 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-22T23:12:46.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T23:12:46.281 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link delete ceph-brx 2024-05-22T23:12:46.343 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:46 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1925107690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:12:46.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:46 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1839508304' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:12:46.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:46 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1076889820' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:12:46.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:46 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/231349279' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-22T23:12:46.370 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-22T23:12:46.381 INFO:tasks.cephadm:Teardown begin 2024-05-22T23:12:46.381 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-22T23:12:46.381 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:12:46.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.505+0000 7f91f1570640 1 -- 172.21.15.84:0/2431061795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ec101880 msgr2=0x7f91ec101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:46.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.505+0000 7f91f1570640 1 --2- 172.21.15.84:0/2431061795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ec101880 0x7f91ec101d00 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f91d4009930 tx=0x7f91d402a420 comp rx=0 tx=0).stop 2024-05-22T23:12:46.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.505+0000 7f91f1570640 1 -- 172.21.15.84:0/2431061795 shutdown_connections 2024-05-22T23:12:46.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.505+0000 7f91f1570640 1 --2- 172.21.15.84:0/2431061795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91ec102240 0x7f91ec10e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:46.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.505+0000 7f91f1570640 1 --2- 172.21.15.84:0/2431061795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ec101880 0x7f91ec101d00 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:46.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.505+0000 7f91f1570640 1 --2- 172.21.15.84:0/2431061795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91ec107890 0x7f91ec107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:46.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.505+0000 7f91f1570640 1 -- 172.21.15.84:0/2431061795 >> 172.21.15.84:0/2431061795 conn(0x7f91ec078d00 msgr2=0x7f91ec079110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:46.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.505+0000 7f91f1570640 1 -- 172.21.15.84:0/2431061795 shutdown_connections 2024-05-22T23:12:46.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.505+0000 7f91f1570640 1 -- 172.21.15.84:0/2431061795 wait complete. 2024-05-22T23:12:46.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.505+0000 7f91f1570640 1 Processor -- start 2024-05-22T23:12:46.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.506+0000 7f91f1570640 1 -- start start 2024-05-22T23:12:46.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.506+0000 7f91f1570640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91ec101880 0x7f91ec198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:46.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.506+0000 7f91f1570640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91ec102240 0x7f91ec1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:46.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.506+0000 7f91f1570640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ec107890 0x7f91ec19ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:46.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.506+0000 7f91f1570640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f91ec111ca0 con 0x7f91ec107890 2024-05-22T23:12:46.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.506+0000 7f91f1570640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f91ec111b20 con 0x7f91ec101880 2024-05-22T23:12:46.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.506+0000 7f91f1570640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f91ec111e20 con 0x7f91ec102240 2024-05-22T23:12:46.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.506+0000 7f91eb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ec107890 0x7f91ec19ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:46.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.506+0000 7f91eb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ec107890 0x7f91ec19ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40162/0 (socket says 172.21.15.84:40162) 2024-05-22T23:12:46.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.506+0000 7f91eb7fe640 1 -- 172.21.15.84:0/1187410257 learned_addr learned my addr 172.21.15.84:0/1187410257 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:46.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.506+0000 7f91eaffd640 1 --2- 172.21.15.84:0/1187410257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91ec101880 0x7f91ec198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:46.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.506+0000 7f91ea7fc640 1 --2- 172.21.15.84:0/1187410257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91ec102240 0x7f91ec1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:46.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.506+0000 7f91eb7fe640 1 -- 172.21.15.84:0/1187410257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91ec102240 msgr2=0x7f91ec1986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:46.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.506+0000 7f91eb7fe640 1 --2- 172.21.15.84:0/1187410257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91ec102240 0x7f91ec1986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:46.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.506+0000 7f91eb7fe640 1 -- 172.21.15.84:0/1187410257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91ec101880 msgr2=0x7f91ec198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:46.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.506+0000 7f91eb7fe640 1 --2- 172.21.15.84:0/1187410257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91ec101880 0x7f91ec198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:46.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.506+0000 7f91eb7fe640 1 -- 172.21.15.84:0/1187410257 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ec19d150 con 0x7f91ec107890 2024-05-22T23:12:46.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.506+0000 7f91eb7fe640 1 --2- 172.21.15.84:0/1187410257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ec107890 0x7f91ec19ca20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f91dc00da40 tx=0x7f91dc00df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:46.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.507+0000 7f91e3fff640 1 -- 172.21.15.84:0/1187410257 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91dc017070 con 0x7f91ec107890 2024-05-22T23:12:46.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.507+0000 7f91e3fff640 1 -- 172.21.15.84:0/1187410257 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91dc003e50 con 0x7f91ec107890 2024-05-22T23:12:46.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.507+0000 7f91f1570640 1 -- 172.21.15.84:0/1187410257 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91ec19d440 con 0x7f91ec107890 2024-05-22T23:12:46.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.507+0000 7f91e3fff640 1 -- 172.21.15.84:0/1187410257 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91dc010460 con 0x7f91ec107890 2024-05-22T23:12:46.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.507+0000 7f91f1570640 1 -- 172.21.15.84:0/1187410257 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91ec1a4cd0 con 0x7f91ec107890 2024-05-22T23:12:46.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.508+0000 7f91f1570640 1 -- 172.21.15.84:0/1187410257 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91ec067e00 con 0x7f91ec107890 2024-05-22T23:12:46.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.509+0000 7f91e3fff640 1 -- 172.21.15.84:0/1187410257 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f91dc003970 con 0x7f91ec107890 2024-05-22T23:12:46.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.510+0000 7f91e3fff640 1 --2- 172.21.15.84:0/1187410257 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f91d80a2d00 0x7f91d80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:46.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.510+0000 7f91eaffd640 1 --2- 172.21.15.84:0/1187410257 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f91d80a2d00 0x7f91d80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:46.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.510+0000 7f91e3fff640 1 -- 172.21.15.84:0/1187410257 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8571+0+0 (secure 0 0 0) 0x7f91dc0cf0c0 con 0x7f91ec107890 2024-05-22T23:12:46.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.510+0000 7f91eaffd640 1 --2- 172.21.15.84:0/1187410257 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f91d80a2d00 0x7f91d80a51c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f91e4005d00 tx=0x7f91e400b410 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:46.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.514+0000 7f91e3fff640 1 -- 172.21.15.84:0/1187410257 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f91dc09bcb0 con 0x7f91ec107890 2024-05-22T23:12:46.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:46 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1925107690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-22T23:12:46.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:46 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1839508304' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-22T23:12:46.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:46 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1076889820' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-22T23:12:46.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:46 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/231349279' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-22T23:12:46.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.727+0000 7f91f1570640 1 -- 172.21.15.84:0/1187410257 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91ec109e50 con 0x7f91ec107890 2024-05-22T23:12:46.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.728+0000 7f91e3fff640 1 -- 172.21.15.84:0/1187410257 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7f91dc09b5b0 con 0x7f91ec107890 2024-05-22T23:12:46.736 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:12:46.736 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:12:46.736 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:12:46.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.730+0000 7f91f1570640 1 -- 172.21.15.84:0/1187410257 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f91d80a2d00 msgr2=0x7f91d80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:46.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.730+0000 7f91f1570640 1 --2- 172.21.15.84:0/1187410257 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f91d80a2d00 0x7f91d80a51c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f91e4005d00 tx=0x7f91e400b410 comp rx=0 tx=0).stop 2024-05-22T23:12:46.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.730+0000 7f91f1570640 1 -- 172.21.15.84:0/1187410257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ec107890 msgr2=0x7f91ec19ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:46.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.730+0000 7f91f1570640 1 --2- 172.21.15.84:0/1187410257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ec107890 0x7f91ec19ca20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f91dc00da40 tx=0x7f91dc00df00 comp rx=0 tx=0).stop 2024-05-22T23:12:46.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.730+0000 7f91f1570640 1 -- 172.21.15.84:0/1187410257 shutdown_connections 2024-05-22T23:12:46.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.730+0000 7f91f1570640 1 --2- 172.21.15.84:0/1187410257 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f91d80a2d00 0x7f91d80a51c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:46.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.730+0000 7f91f1570640 1 --2- 172.21.15.84:0/1187410257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ec107890 0x7f91ec19ca20 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:46.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.730+0000 7f91f1570640 1 --2- 172.21.15.84:0/1187410257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91ec102240 0x7f91ec1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:46.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.730+0000 7f91f1570640 1 --2- 172.21.15.84:0/1187410257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91ec101880 0x7f91ec198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:46.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.730+0000 7f91f1570640 1 -- 172.21.15.84:0/1187410257 >> 172.21.15.84:0/1187410257 conn(0x7f91ec078d00 msgr2=0x7f91ec0ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:46.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.730+0000 7f91f1570640 1 -- 172.21.15.84:0/1187410257 shutdown_connections 2024-05-22T23:12:46.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.730+0000 7f91f1570640 1 -- 172.21.15.84:0/1187410257 wait complete. 2024-05-22T23:12:46.749 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T23:12:46.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.842+0000 7ff18a04c640 1 -- 172.21.15.84:0/2478365569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff184107850 msgr2=0x7ff184107c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:46.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.842+0000 7ff18a04c640 1 --2- 172.21.15.84:0/2478365569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff184107850 0x7ff184107c30 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff174009930 tx=0x7ff17402a420 comp rx=0 tx=0).stop 2024-05-22T23:12:46.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.842+0000 7ff18a04c640 1 -- 172.21.15.84:0/2478365569 shutdown_connections 2024-05-22T23:12:46.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.842+0000 7ff18a04c640 1 --2- 172.21.15.84:0/2478365569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff184102200 0x7ff18410e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:46.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.842+0000 7ff18a04c640 1 --2- 172.21.15.84:0/2478365569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff184101840 0x7ff184101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:46.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.842+0000 7ff18a04c640 1 --2- 172.21.15.84:0/2478365569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff184107850 0x7ff184107c30 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:46.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.842+0000 7ff18a04c640 1 -- 172.21.15.84:0/2478365569 >> 172.21.15.84:0/2478365569 conn(0x7ff184078c30 msgr2=0x7ff184079040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:46.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.842+0000 7ff18a04c640 1 -- 172.21.15.84:0/2478365569 shutdown_connections 2024-05-22T23:12:46.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.842+0000 7ff18a04c640 1 -- 172.21.15.84:0/2478365569 wait complete. 2024-05-22T23:12:46.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.843+0000 7ff18a04c640 1 Processor -- start 2024-05-22T23:12:46.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.843+0000 7ff18a04c640 1 -- start start 2024-05-22T23:12:46.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.843+0000 7ff18a04c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff184101840 0x7ff184073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:46.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.843+0000 7ff18a04c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff184102200 0x7ff1840740b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:46.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.843+0000 7ff18a04c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff184107850 0x7ff1840745f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:46.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.843+0000 7ff18a04c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff184111c60 con 0x7ff184102200 2024-05-22T23:12:46.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.843+0000 7ff18a04c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff184111ae0 con 0x7ff184101840 2024-05-22T23:12:46.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.843+0000 7ff18a04c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff184111de0 con 0x7ff184107850 2024-05-22T23:12:46.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.843+0000 7ff17bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff184102200 0x7ff1840740b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:46.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.843+0000 7ff17bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff184102200 0x7ff1840740b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40184/0 (socket says 172.21.15.84:40184) 2024-05-22T23:12:46.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.843+0000 7ff17bfff640 1 -- 172.21.15.84:0/2034186762 learned_addr learned my addr 172.21.15.84:0/2034186762 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:46.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.843+0000 7ff1837fe640 1 --2- 172.21.15.84:0/2034186762 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff184101840 0x7ff184073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:46.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.843+0000 7ff183fff640 1 --2- 172.21.15.84:0/2034186762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff184107850 0x7ff1840745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:46.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.843+0000 7ff17bfff640 1 -- 172.21.15.84:0/2034186762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff184107850 msgr2=0x7ff1840745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:46.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.843+0000 7ff17bfff640 1 --2- 172.21.15.84:0/2034186762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff184107850 0x7ff1840745f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:46.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.843+0000 7ff17bfff640 1 -- 172.21.15.84:0/2034186762 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff184101840 msgr2=0x7ff184073b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:46.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.843+0000 7ff17bfff640 1 --2- 172.21.15.84:0/2034186762 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff184101840 0x7ff184073b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:46.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.843+0000 7ff17bfff640 1 -- 172.21.15.84:0/2034186762 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1841a66a0 con 0x7ff184102200 2024-05-22T23:12:46.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.843+0000 7ff183fff640 1 --2- 172.21.15.84:0/2034186762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff184107850 0x7ff1840745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:12:46.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.843+0000 7ff17bfff640 1 --2- 172.21.15.84:0/2034186762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff184102200 0x7ff1840740b0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00d950 tx=0x7ff16c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:46.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.844+0000 7ff1817fa640 1 -- 172.21.15.84:0/2034186762 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c017070 con 0x7ff184102200 2024-05-22T23:12:46.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.844+0000 7ff1817fa640 1 -- 172.21.15.84:0/2034186762 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff16c009c30 con 0x7ff184102200 2024-05-22T23:12:46.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.844+0000 7ff18a04c640 1 -- 172.21.15.84:0/2034186762 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1841a6990 con 0x7ff184102200 2024-05-22T23:12:46.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.844+0000 7ff1817fa640 1 -- 172.21.15.84:0/2034186762 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c00f6b0 con 0x7ff184102200 2024-05-22T23:12:46.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.844+0000 7ff18a04c640 1 -- 172.21.15.84:0/2034186762 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1841a6ee0 con 0x7ff184102200 2024-05-22T23:12:46.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.845+0000 7ff18a04c640 1 -- 172.21.15.84:0/2034186762 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff184068ae0 con 0x7ff184102200 2024-05-22T23:12:46.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.846+0000 7ff1817fa640 1 -- 172.21.15.84:0/2034186762 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff16c0109d0 con 0x7ff184102200 2024-05-22T23:12:46.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.846+0000 7ff1817fa640 1 --2- 172.21.15.84:0/2034186762 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff1700a2d00 0x7ff1700a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:46.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.846+0000 7ff1837fe640 1 --2- 172.21.15.84:0/2034186762 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff1700a2d00 0x7ff1700a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:46.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.846+0000 7ff1817fa640 1 -- 172.21.15.84:0/2034186762 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8571+0+0 (secure 0 0 0) 0x7ff16c0cf170 con 0x7ff184102200 2024-05-22T23:12:46.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.847+0000 7ff1837fe640 1 --2- 172.21.15.84:0/2034186762 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff1700a2d00 0x7ff1700a51c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff174000c00 tx=0x7ff174005a20 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:46.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:46.850+0000 7ff1817fa640 1 -- 172.21.15.84:0/2034186762 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff16c009de0 con 0x7ff184102200 2024-05-22T23:12:47.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.063+0000 7ff18a04c640 1 -- 172.21.15.84:0/2034186762 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1840750a0 con 0x7ff184102200 2024-05-22T23:12:47.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.064+0000 7ff1817fa640 1 -- 172.21.15.84:0/2034186762 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11808 (secure 0 0 0) 0x7ff16c09bd60 con 0x7ff184102200 2024-05-22T23:12:47.072 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:12:47.072 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2391815590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2391815590},{"type":"v1","addr":"172.21.15.84:6835","nonce":2391815590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4284308638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4284308638},{"type":"v1","addr":"172.21.15.84:6837","nonce":4284308638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/669165452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":669165452},{"type":"v1","addr":"172.21.15.84:6839","nonce":669165452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2709540650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2709540650},{"type":"v1","addr":"172.21.15.84:6841","nonce":2709540650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3932120262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3932120262},{"type":"v1","addr":"172.21.15.204:6833","nonce":3932120262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2890595273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2890595273},{"type":"v1","addr":"172.21.15.204:6835","nonce":2890595273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/3792500684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":3792500684},{"type":"v1","addr":"172.21.15.96:6833","nonce":3792500684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3662981631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3662981631},{"type":"v1","addr":"172.21.15.204:6837","nonce":3662981631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3328293038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3328293038},{"type":"v1","addr":"172.21.15.204:6839","nonce":3328293038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1115134082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1115134082},{"type":"v1","addr":"172.21.15.96:6835","nonce":1115134082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-22T22:59:51.926923+0000","modified":"2024-05-22T22:59:54.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/1191862158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1191862158},{"type":"v1","addr":"172.21.15.96:6837","nonce":1191862158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6839/3272422729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":3272422729},{"type":"v1","addr":"172.21.15.96:6839","nonce":3272422729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530]},"id":1}]} 2024-05-22T23:12:47.072 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2024-05-22T23:12:47.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.066+0000 7ff18a04c640 1 -- 172.21.15.84:0/2034186762 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff1700a2d00 msgr2=0x7ff1700a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:47.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.066+0000 7ff18a04c640 1 --2- 172.21.15.84:0/2034186762 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff1700a2d00 0x7ff1700a51c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff174000c00 tx=0x7ff174005a20 comp rx=0 tx=0).stop 2024-05-22T23:12:47.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.066+0000 7ff18a04c640 1 -- 172.21.15.84:0/2034186762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff184102200 msgr2=0x7ff1840740b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:47.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.066+0000 7ff18a04c640 1 --2- 172.21.15.84:0/2034186762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff184102200 0x7ff1840740b0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00d950 tx=0x7ff16c00de10 comp rx=0 tx=0).stop 2024-05-22T23:12:47.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.066+0000 7ff18a04c640 1 -- 172.21.15.84:0/2034186762 shutdown_connections 2024-05-22T23:12:47.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.066+0000 7ff18a04c640 1 --2- 172.21.15.84:0/2034186762 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7ff1700a2d00 0x7ff1700a51c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.066+0000 7ff18a04c640 1 --2- 172.21.15.84:0/2034186762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff184107850 0x7ff1840745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.066+0000 7ff18a04c640 1 --2- 172.21.15.84:0/2034186762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff184102200 0x7ff1840740b0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.066+0000 7ff18a04c640 1 --2- 172.21.15.84:0/2034186762 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff184101840 0x7ff184073b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.066+0000 7ff18a04c640 1 -- 172.21.15.84:0/2034186762 >> 172.21.15.84:0/2034186762 conn(0x7ff184078c30 msgr2=0x7ff184104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:47.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.066+0000 7ff18a04c640 1 -- 172.21.15.84:0/2034186762 shutdown_connections 2024-05-22T23:12:47.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.066+0000 7ff18a04c640 1 -- 172.21.15.84:0/2034186762 wait complete. 2024-05-22T23:12:47.085 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T23:12:47.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.178+0000 7fef716cd640 1 -- 172.21.15.84:0/2098087164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef6c107880 msgr2=0x7fef6c107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:47.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.178+0000 7fef716cd640 1 --2- 172.21.15.84:0/2098087164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef6c107880 0x7fef6c107c60 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fef5c009900 tx=0x7fef5c02a480 comp rx=0 tx=0).stop 2024-05-22T23:12:47.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.178+0000 7fef716cd640 1 -- 172.21.15.84:0/2098087164 shutdown_connections 2024-05-22T23:12:47.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.178+0000 7fef716cd640 1 --2- 172.21.15.84:0/2098087164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef6c102290 0x7fef6c10e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.188 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.178+0000 7fef716cd640 1 --2- 172.21.15.84:0/2098087164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef6c1018f0 0x7fef6c101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.178+0000 7fef716cd640 1 --2- 172.21.15.84:0/2098087164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef6c107880 0x7fef6c107c60 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.178+0000 7fef716cd640 1 -- 172.21.15.84:0/2098087164 >> 172.21.15.84:0/2098087164 conn(0x7fef6c078a00 msgr2=0x7fef6c078e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:47.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.178+0000 7fef716cd640 1 -- 172.21.15.84:0/2098087164 shutdown_connections 2024-05-22T23:12:47.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.178+0000 7fef716cd640 1 -- 172.21.15.84:0/2098087164 wait complete. 2024-05-22T23:12:47.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.178+0000 7fef716cd640 1 Processor -- start 2024-05-22T23:12:47.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.179+0000 7fef716cd640 1 -- start start 2024-05-22T23:12:47.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.179+0000 7fef716cd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef6c1018f0 0x7fef6c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:47.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.179+0000 7fef716cd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef6c102290 0x7fef6c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:47.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.179+0000 7fef716cd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef6c107880 0x7fef6c19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:47.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.179+0000 7fef716cd640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c111c40 con 0x7fef6c1018f0 2024-05-22T23:12:47.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.179+0000 7fef716cd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c111ac0 con 0x7fef6c102290 2024-05-22T23:12:47.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.179+0000 7fef716cd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c111dc0 con 0x7fef6c107880 2024-05-22T23:12:47.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.179+0000 7fef6affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef6c1018f0 0x7fef6c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:47.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.179+0000 7fef6affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef6c1018f0 0x7fef6c19a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40208/0 (socket says 172.21.15.84:40208) 2024-05-22T23:12:47.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.179+0000 7fef6affd640 1 -- 172.21.15.84:0/2750212628 learned_addr learned my addr 172.21.15.84:0/2750212628 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:47.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.179+0000 7fef6b7fe640 1 --2- 172.21.15.84:0/2750212628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef6c107880 0x7fef6c19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:47.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.179+0000 7fef627fc640 1 --2- 172.21.15.84:0/2750212628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef6c102290 0x7fef6c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:47.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.179+0000 7fef6affd640 1 -- 172.21.15.84:0/2750212628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef6c107880 msgr2=0x7fef6c19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:47.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.179+0000 7fef6affd640 1 --2- 172.21.15.84:0/2750212628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef6c107880 0x7fef6c19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.179+0000 7fef6affd640 1 -- 172.21.15.84:0/2750212628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef6c102290 msgr2=0x7fef6c19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:47.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.179+0000 7fef6affd640 1 --2- 172.21.15.84:0/2750212628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef6c102290 0x7fef6c19a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.179+0000 7fef6affd640 1 -- 172.21.15.84:0/2750212628 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6c19f220 con 0x7fef6c1018f0 2024-05-22T23:12:47.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.179+0000 7fef627fc640 1 --2- 172.21.15.84:0/2750212628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef6c102290 0x7fef6c19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T23:12:47.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.180+0000 7fef6affd640 1 --2- 172.21.15.84:0/2750212628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef6c1018f0 0x7fef6c19a230 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fef5c006010 tx=0x7fef5c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:47.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.180+0000 7fef68ff9640 1 -- 172.21.15.84:0/2750212628 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c03f030 con 0x7fef6c1018f0 2024-05-22T23:12:47.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.180+0000 7fef716cd640 1 -- 172.21.15.84:0/2750212628 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef6c19f4b0 con 0x7fef6c1018f0 2024-05-22T23:12:47.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.180+0000 7fef68ff9640 1 -- 172.21.15.84:0/2750212628 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef5c003ce0 con 0x7fef6c1018f0 2024-05-22T23:12:47.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.180+0000 7fef68ff9640 1 -- 172.21.15.84:0/2750212628 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c035070 con 0x7fef6c1018f0 2024-05-22T23:12:47.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.180+0000 7fef716cd640 1 -- 172.21.15.84:0/2750212628 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef6c1a6da0 con 0x7fef6c1018f0 2024-05-22T23:12:47.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.181+0000 7fef716cd640 1 -- 172.21.15.84:0/2750212628 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef6c068ae0 con 0x7fef6c1018f0 2024-05-22T23:12:47.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.182+0000 7fef68ff9640 1 -- 172.21.15.84:0/2750212628 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fef5c042030 con 0x7fef6c1018f0 2024-05-22T23:12:47.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.182+0000 7fef68ff9640 1 --2- 172.21.15.84:0/2750212628 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fef580a2d00 0x7fef580a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:47.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.183+0000 7fef627fc640 1 --2- 172.21.15.84:0/2750212628 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fef580a2d00 0x7fef580a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:47.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.183+0000 7fef68ff9640 1 -- 172.21.15.84:0/2750212628 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8571+0+0 (secure 0 0 0) 0x7fef5c0ef990 con 0x7fef6c1018f0 2024-05-22T23:12:47.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.183+0000 7fef627fc640 1 --2- 172.21.15.84:0/2750212628 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fef580a2d00 0x7fef580a51c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0792a0 tx=0x7fef54005d90 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:47.194 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.187+0000 7fef68ff9640 1 -- 172.21.15.84:0/2750212628 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fef5c0bc640 con 0x7fef6c1018f0 2024-05-22T23:12:47.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.361+0000 7fef716cd640 1 -- 172.21.15.84:0/2750212628 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef6c1a6f70 con 0x7fef6c1018f0 2024-05-22T23:12:47.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.363+0000 7fef68ff9640 1 -- 172.21.15.84:0/2750212628 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+23190 (secure 0 0 0) 0x7fef5c0bbf40 con 0x7fef6c1018f0 2024-05-22T23:12:47.371 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-22T23:12:47.372 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":99,"fsid":"c1135a12-188d-11ef-bc9a-c7b262605968","created":"2024-05-22T22:51:49.878218+0000","modified":"2024-05-22T23:12:17.083988+0000","last_up_change":"2024-05-22T22:58:58.550278+0000","last_in_change":"2024-05-22T22:58:37.322139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T22:55:07.719394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T22:59:50.150511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T22:59:51.111182+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762df065-fb4f-4c71-aab8-a62d2282c253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6803","nonce":1559945826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6805","nonce":1559945826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6809","nonce":1559945826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1559945826},{"type":"v1","addr":"172.21.15.84:6807","nonce":1559945826}]},"public_addr":"172.21.15.84:6803/1559945826","cluster_addr":"172.21.15.84:6805/1559945826","heartbeat_back_addr":"172.21.15.84:6809/1559945826","heartbeat_front_addr":"172.21.15.84:6807/1559945826","state":["exists","up"]},{"osd":1,"uuid":"3fb2385d-6a34-49e3-88e1-116237beb346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6801","nonce":2996505353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6803","nonce":2996505353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6807","nonce":2996505353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2996505353},{"type":"v1","addr":"172.21.15.96:6805","nonce":2996505353}]},"public_addr":"172.21.15.96:6801/2996505353","cluster_addr":"172.21.15.96:6803/2996505353","heartbeat_back_addr":"172.21.15.96:6807/2996505353","heartbeat_front_addr":"172.21.15.96:6805/2996505353","state":["exists","up"]},{"osd":2,"uuid":"e306e88e-a308-4585-ae11-c8af10b84038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6801","nonce":1499577897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6803","nonce":1499577897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6807","nonce":1499577897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1499577897},{"type":"v1","addr":"172.21.15.204:6805","nonce":1499577897}]},"public_addr":"172.21.15.204:6801/1499577897","cluster_addr":"172.21.15.204:6803/1499577897","heartbeat_back_addr":"172.21.15.204:6807/1499577897","heartbeat_front_addr":"172.21.15.204:6805/1499577897","state":["exists","up"]},{"osd":3,"uuid":"d2a5fb97-6f60-4d45-9f18-447cae480974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6811","nonce":2869340648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6813","nonce":2869340648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6817","nonce":2869340648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2869340648},{"type":"v1","addr":"172.21.15.84:6815","nonce":2869340648}]},"public_addr":"172.21.15.84:6811/2869340648","cluster_addr":"172.21.15.84:6813/2869340648","heartbeat_back_addr":"172.21.15.84:6817/2869340648","heartbeat_front_addr":"172.21.15.84:6815/2869340648","state":["exists","up"]},{"osd":4,"uuid":"4f6bbf7d-66d2-48e0-a7f4-8f4e7236551f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6809","nonce":2554265648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6811","nonce":2554265648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6815","nonce":2554265648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2554265648},{"type":"v1","addr":"172.21.15.96:6813","nonce":2554265648}]},"public_addr":"172.21.15.96:6809/2554265648","cluster_addr":"172.21.15.96:6811/2554265648","heartbeat_back_addr":"172.21.15.96:6815/2554265648","heartbeat_front_addr":"172.21.15.96:6813/2554265648","state":["exists","up"]},{"osd":5,"uuid":"7996bf5e-93a5-41b0-a9b1-99e7ef4514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6809","nonce":4075476986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6811","nonce":4075476986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6815","nonce":4075476986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4075476986},{"type":"v1","addr":"172.21.15.204:6813","nonce":4075476986}]},"public_addr":"172.21.15.204:6809/4075476986","cluster_addr":"172.21.15.204:6811/4075476986","heartbeat_back_addr":"172.21.15.204:6815/4075476986","heartbeat_front_addr":"172.21.15.204:6813/4075476986","state":["exists","up"]},{"osd":6,"uuid":"f4b7c5de-1836-4d16-8603-3c517524d9fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6819","nonce":1806202865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6821","nonce":1806202865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6825","nonce":1806202865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1806202865},{"type":"v1","addr":"172.21.15.84:6823","nonce":1806202865}]},"public_addr":"172.21.15.84:6819/1806202865","cluster_addr":"172.21.15.84:6821/1806202865","heartbeat_back_addr":"172.21.15.84:6825/1806202865","heartbeat_front_addr":"172.21.15.84:6823/1806202865","state":["exists","up"]},{"osd":7,"uuid":"69a18c43-5447-480d-9466-190830070c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6817","nonce":3195792774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6819","nonce":3195792774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6823","nonce":3195792774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3195792774},{"type":"v1","addr":"172.21.15.96:6821","nonce":3195792774}]},"public_addr":"172.21.15.96:6817/3195792774","cluster_addr":"172.21.15.96:6819/3195792774","heartbeat_back_addr":"172.21.15.96:6823/3195792774","heartbeat_front_addr":"172.21.15.96:6821/3195792774","state":["exists","up"]},{"osd":8,"uuid":"2c5cadca-689a-441e-847c-ba370d400ef1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6817","nonce":606997927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6819","nonce":606997927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6823","nonce":606997927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":606997927},{"type":"v1","addr":"172.21.15.204:6821","nonce":606997927}]},"public_addr":"172.21.15.204:6817/606997927","cluster_addr":"172.21.15.204:6819/606997927","heartbeat_back_addr":"172.21.15.204:6823/606997927","heartbeat_front_addr":"172.21.15.204:6821/606997927","state":["exists","up"]},{"osd":9,"uuid":"e3984fb2-7c00-4275-a10e-bca7c859f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6827","nonce":2786189012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6829","nonce":2786189012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6833","nonce":2786189012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2786189012},{"type":"v1","addr":"172.21.15.84:6831","nonce":2786189012}]},"public_addr":"172.21.15.84:6827/2786189012","cluster_addr":"172.21.15.84:6829/2786189012","heartbeat_back_addr":"172.21.15.84:6833/2786189012","heartbeat_front_addr":"172.21.15.84:6831/2786189012","state":["exists","up"]},{"osd":10,"uuid":"b4938160-a808-4ded-91c2-d245c99fff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6825","nonce":1985398526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6827","nonce":1985398526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6831","nonce":1985398526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1985398526},{"type":"v1","addr":"172.21.15.96:6829","nonce":1985398526}]},"public_addr":"172.21.15.96:6825/1985398526","cluster_addr":"172.21.15.96:6827/1985398526","heartbeat_back_addr":"172.21.15.96:6831/1985398526","heartbeat_front_addr":"172.21.15.96:6829/1985398526","state":["exists","up"]},{"osd":11,"uuid":"238922ef-d046-40cc-833d-25b7e7a523cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6825","nonce":3106116519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6827","nonce":3106116519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6831","nonce":3106116519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3106116519},{"type":"v1","addr":"172.21.15.204:6829","nonce":3106116519}]},"public_addr":"172.21.15.204:6825/3106116519","cluster_addr":"172.21.15.204:6827/3106116519","heartbeat_back_addr":"172.21.15.204:6831/3106116519","heartbeat_front_addr":"172.21.15.204:6829/3106116519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:16.443480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:54:39.898133+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:03.664544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:28.838687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:55:53.829733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:19.269307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:56:45.166910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:10.771614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:57:36.678516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:03.150592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:29.327769+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T22:58:56.315938+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2317224739":"2024-05-23T23:01:49.869856+0000","172.21.15.84:0/4170743349":"2024-05-23T23:01:49.869856+0000","172.21.15.84:6801/4223589140":"2024-05-23T23:01:49.869856+0000","172.21.15.84:6800/4223589140":"2024-05-23T23:01:49.869856+0000","172.21.15.84:6801/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:6800/3278959854":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/2766288608":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/950455999":"2024-05-23T22:52:45.395791+0000","172.21.15.84:0/3490117794":"2024-05-23T23:01:49.869856+0000","172.21.15.84:0/4000289608":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/1650467363":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/4077200298":"2024-05-23T23:01:49.869856+0000","172.21.15.84:0/1052746114":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6800/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:6801/4029123070":"2024-05-23T22:52:18.615994+0000","172.21.15.84:0/14610813":"2024-05-23T22:52:45.395791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T23:12:47.373 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.365+0000 7fef716cd640 1 -- 172.21.15.84:0/2750212628 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fef580a2d00 msgr2=0x7fef580a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:47.373 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.365+0000 7fef716cd640 1 --2- 172.21.15.84:0/2750212628 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fef580a2d00 0x7fef580a51c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0792a0 tx=0x7fef54005d90 comp rx=0 tx=0).stop 2024-05-22T23:12:47.373 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.365+0000 7fef716cd640 1 -- 172.21.15.84:0/2750212628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef6c1018f0 msgr2=0x7fef6c19a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:47.373 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.365+0000 7fef716cd640 1 --2- 172.21.15.84:0/2750212628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef6c1018f0 0x7fef6c19a230 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fef5c006010 tx=0x7fef5c0042f0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.373 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.365+0000 7fef716cd640 1 -- 172.21.15.84:0/2750212628 shutdown_connections 2024-05-22T23:12:47.373 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.365+0000 7fef716cd640 1 --2- 172.21.15.84:0/2750212628 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fef580a2d00 0x7fef580a51c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.374 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.365+0000 7fef716cd640 1 --2- 172.21.15.84:0/2750212628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef6c107880 0x7fef6c19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.374 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.365+0000 7fef716cd640 1 --2- 172.21.15.84:0/2750212628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef6c102290 0x7fef6c19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.374 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.365+0000 7fef716cd640 1 --2- 172.21.15.84:0/2750212628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef6c1018f0 0x7fef6c19a230 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.374 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.365+0000 7fef716cd640 1 -- 172.21.15.84:0/2750212628 >> 172.21.15.84:0/2750212628 conn(0x7fef6c078a00 msgr2=0x7fef6c10cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:47.374 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.365+0000 7fef716cd640 1 -- 172.21.15.84:0/2750212628 shutdown_connections 2024-05-22T23:12:47.374 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.365+0000 7fef716cd640 1 -- 172.21.15.84:0/2750212628 wait complete. 2024-05-22T23:12:47.381 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:47 smithi084 ceph-mon[32186]: pgmap v341: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:12:47.381 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:47 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1187410257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:12:47.381 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:47 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2034186762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:12:47.384 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-22T23:12:47.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:47 smithi204 ceph-mon[36211]: pgmap v341: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:12:47.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:47 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1187410257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:12:47.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:47 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2034186762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:12:47.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.478+0000 7f5f7a6e7640 1 -- 172.21.15.84:0/3270927078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f7410be40 msgr2=0x7f5f7410e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:47.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.478+0000 7f5f7a6e7640 1 --2- 172.21.15.84:0/3270927078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f7410be40 0x7f5f7410e2e0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009ca0 tx=0x7f5f6c02a9e0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.478+0000 7f5f7a6e7640 1 -- 172.21.15.84:0/3270927078 shutdown_connections 2024-05-22T23:12:47.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.478+0000 7f5f7a6e7640 1 --2- 172.21.15.84:0/3270927078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f7410be40 0x7f5f7410e2e0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.478+0000 7f5f7a6e7640 1 --2- 172.21.15.84:0/3270927078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f74103420 0x7f5f7410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.478+0000 7f5f7a6e7640 1 --2- 172.21.15.84:0/3270927078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f74102b00 0x7f5f74102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.478+0000 7f5f7a6e7640 1 -- 172.21.15.84:0/3270927078 >> 172.21.15.84:0/3270927078 conn(0x7f5f74076af0 msgr2=0x7f5f74076f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:47.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.478+0000 7f5f7a6e7640 1 -- 172.21.15.84:0/3270927078 shutdown_connections 2024-05-22T23:12:47.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.478+0000 7f5f7a6e7640 1 -- 172.21.15.84:0/3270927078 wait complete. 2024-05-22T23:12:47.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.479+0000 7f5f7a6e7640 1 Processor -- start 2024-05-22T23:12:47.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.479+0000 7f5f7a6e7640 1 -- start start 2024-05-22T23:12:47.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.479+0000 7f5f7a6e7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f74102b00 0x7f5f74198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:47.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.479+0000 7f5f7a6e7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f74103420 0x7f5f74198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:47.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.479+0000 7f5f7a6e7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f7410be40 0x7f5f7419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:47.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.479+0000 7f5f7a6e7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5f74111c40 con 0x7f5f74102b00 2024-05-22T23:12:47.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.479+0000 7f5f7a6e7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5f74111ac0 con 0x7f5f7410be40 2024-05-22T23:12:47.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.479+0000 7f5f7a6e7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5f74111dc0 con 0x7f5f74103420 2024-05-22T23:12:47.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.479+0000 7f5f73fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f74102b00 0x7f5f74198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:47.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.479+0000 7f5f73fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f74102b00 0x7f5f74198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40220/0 (socket says 172.21.15.84:40220) 2024-05-22T23:12:47.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.479+0000 7f5f73fff640 1 -- 172.21.15.84:0/1248514175 learned_addr learned my addr 172.21.15.84:0/1248514175 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:47.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.479+0000 7f5f6bfff640 1 --2- 172.21.15.84:0/1248514175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f74103420 0x7f5f74198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:47.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.479+0000 7f5f78c84640 1 --2- 172.21.15.84:0/1248514175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f7410be40 0x7f5f7419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:47.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.480+0000 7f5f73fff640 1 -- 172.21.15.84:0/1248514175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f74103420 msgr2=0x7f5f74198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:47.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.480+0000 7f5f73fff640 1 --2- 172.21.15.84:0/1248514175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f74103420 0x7f5f74198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.480+0000 7f5f73fff640 1 -- 172.21.15.84:0/1248514175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f7410be40 msgr2=0x7f5f7419c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:47.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.480+0000 7f5f73fff640 1 --2- 172.21.15.84:0/1248514175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f7410be40 0x7f5f7419c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.480+0000 7f5f73fff640 1 -- 172.21.15.84:0/1248514175 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7419d0e0 con 0x7f5f74102b00 2024-05-22T23:12:47.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.480+0000 7f5f73fff640 1 --2- 172.21.15.84:0/1248514175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f74102b00 0x7f5f74198140 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400e9f0 tx=0x7f5f6400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:47.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.480+0000 7f5f71ffb640 1 -- 172.21.15.84:0/1248514175 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6400cd80 con 0x7f5f74102b00 2024-05-22T23:12:47.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.480+0000 7f5f71ffb640 1 -- 172.21.15.84:0/1248514175 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f64005500 con 0x7f5f74102b00 2024-05-22T23:12:47.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.480+0000 7f5f7a6e7640 1 -- 172.21.15.84:0/1248514175 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f7419d3d0 con 0x7f5f74102b00 2024-05-22T23:12:47.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.480+0000 7f5f71ffb640 1 -- 172.21.15.84:0/1248514175 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64010690 con 0x7f5f74102b00 2024-05-22T23:12:47.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.480+0000 7f5f7a6e7640 1 -- 172.21.15.84:0/1248514175 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f741a4c60 con 0x7f5f74102b00 2024-05-22T23:12:47.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.481+0000 7f5f7a6e7640 1 -- 172.21.15.84:0/1248514175 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f74068ae0 con 0x7f5f74102b00 2024-05-22T23:12:47.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.482+0000 7f5f71ffb640 1 -- 172.21.15.84:0/1248514175 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5f640046c0 con 0x7f5f74102b00 2024-05-22T23:12:47.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.482+0000 7f5f71ffb640 1 --2- 172.21.15.84:0/1248514175 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f5f600a2d00 0x7f5f600a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:47.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.483+0000 7f5f6bfff640 1 --2- 172.21.15.84:0/1248514175 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f5f600a2d00 0x7f5f600a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:47.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.483+0000 7f5f71ffb640 1 -- 172.21.15.84:0/1248514175 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8571+0+0 (secure 0 0 0) 0x7f5f640cf950 con 0x7f5f74102b00 2024-05-22T23:12:47.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.483+0000 7f5f6bfff640 1 --2- 172.21.15.84:0/1248514175 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f5f600a2d00 0x7f5f600a51c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5f740fff80 tx=0x7f5f5c00b410 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:47.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.487+0000 7f5f71ffb640 1 -- 172.21.15.84:0/1248514175 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5f6409c540 con 0x7f5f74102b00 2024-05-22T23:12:47.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:47 smithi096 ceph-mon[36392]: pgmap v341: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-22T23:12:47.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:47 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1187410257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:12:47.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:47 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2034186762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-22T23:12:47.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.702+0000 7f5f7a6e7640 1 -- 172.21.15.84:0/1248514175 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5f74199440 con 0x7f5f74102b00 2024-05-22T23:12:47.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.705+0000 7f5f71ffb640 1 -- 172.21.15.84:0/1248514175 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7f5f6409be40 con 0x7f5f74102b00 2024-05-22T23:12:47.713 INFO:teuthology.orchestra.run.smithi084.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-22T23:12:47.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.707+0000 7f5f7a6e7640 1 -- 172.21.15.84:0/1248514175 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f5f600a2d00 msgr2=0x7f5f600a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:47.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.707+0000 7f5f7a6e7640 1 --2- 172.21.15.84:0/1248514175 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f5f600a2d00 0x7f5f600a51c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5f740fff80 tx=0x7f5f5c00b410 comp rx=0 tx=0).stop 2024-05-22T23:12:47.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.707+0000 7f5f7a6e7640 1 -- 172.21.15.84:0/1248514175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f74102b00 msgr2=0x7f5f74198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:47.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.707+0000 7f5f7a6e7640 1 --2- 172.21.15.84:0/1248514175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f74102b00 0x7f5f74198140 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400e9f0 tx=0x7f5f6400edf0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.707+0000 7f5f7a6e7640 1 -- 172.21.15.84:0/1248514175 shutdown_connections 2024-05-22T23:12:47.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.707+0000 7f5f7a6e7640 1 --2- 172.21.15.84:0/1248514175 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f5f600a2d00 0x7f5f600a51c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.707+0000 7f5f7a6e7640 1 --2- 172.21.15.84:0/1248514175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f7410be40 0x7f5f7419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.707+0000 7f5f7a6e7640 1 --2- 172.21.15.84:0/1248514175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f74103420 0x7f5f74198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.707+0000 7f5f7a6e7640 1 --2- 172.21.15.84:0/1248514175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f74102b00 0x7f5f74198140 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.707+0000 7f5f7a6e7640 1 -- 172.21.15.84:0/1248514175 >> 172.21.15.84:0/1248514175 conn(0x7f5f74076af0 msgr2=0x7f5f740fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:47.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.707+0000 7f5f7a6e7640 1 -- 172.21.15.84:0/1248514175 shutdown_connections 2024-05-22T23:12:47.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.707+0000 7f5f7a6e7640 1 -- 172.21.15.84:0/1248514175 wait complete. 2024-05-22T23:12:47.727 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-22T23:12:47.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.824+0000 7f3a3c7b0640 1 -- 172.21.15.84:0/4287986998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a34101100 msgr2=0x7f3a3410d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:47.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.824+0000 7f3a3c7b0640 1 --2- 172.21.15.84:0/4287986998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a34101100 0x7f3a3410d610 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c009cd0 tx=0x7f3a2c02a8e0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.824+0000 7f3a3c7b0640 1 -- 172.21.15.84:0/4287986998 shutdown_connections 2024-05-22T23:12:47.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.824+0000 7f3a3c7b0640 1 --2- 172.21.15.84:0/4287986998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a34101100 0x7f3a3410d610 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.824+0000 7f3a3c7b0640 1 --2- 172.21.15.84:0/4287986998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a34100740 0x7f3a34100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.824+0000 7f3a3c7b0640 1 --2- 172.21.15.84:0/4287986998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a34106750 0x7f3a34106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.824+0000 7f3a3c7b0640 1 -- 172.21.15.84:0/4287986998 >> 172.21.15.84:0/4287986998 conn(0x7f3a34077870 msgr2=0x7f3a34077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:47.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.824+0000 7f3a3c7b0640 1 -- 172.21.15.84:0/4287986998 shutdown_connections 2024-05-22T23:12:47.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.824+0000 7f3a3c7b0640 1 -- 172.21.15.84:0/4287986998 wait complete. 2024-05-22T23:12:47.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.824+0000 7f3a3c7b0640 1 Processor -- start 2024-05-22T23:12:47.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.824+0000 7f3a3c7b0640 1 -- start start 2024-05-22T23:12:47.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.825+0000 7f3a3c7b0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a34100740 0x7f3a341970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:47.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.825+0000 7f3a3c7b0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a34101100 0x7f3a34197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:47.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.825+0000 7f3a3c7b0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a34106750 0x7f3a3419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:47.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.825+0000 7f3a3c7b0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3a34110b60 con 0x7f3a34101100 2024-05-22T23:12:47.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.825+0000 7f3a3c7b0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3a341109e0 con 0x7f3a34106750 2024-05-22T23:12:47.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.825+0000 7f3a3c7b0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3a34110ce0 con 0x7f3a34100740 2024-05-22T23:12:47.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.825+0000 7f3a39d4b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a34101100 0x7f3a34197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:47.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.825+0000 7f3a39d4b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a34101100 0x7f3a34197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40244/0 (socket says 172.21.15.84:40244) 2024-05-22T23:12:47.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.825+0000 7f3a39d4b640 1 -- 172.21.15.84:0/1890311229 learned_addr learned my addr 172.21.15.84:0/1890311229 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:47.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.825+0000 7f3a3ad4d640 1 --2- 172.21.15.84:0/1890311229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a34106750 0x7f3a3419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:47.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.825+0000 7f3a3a54c640 1 --2- 172.21.15.84:0/1890311229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a34100740 0x7f3a341970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:47.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.825+0000 7f3a39d4b640 1 -- 172.21.15.84:0/1890311229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a34100740 msgr2=0x7f3a341970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:47.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.825+0000 7f3a39d4b640 1 --2- 172.21.15.84:0/1890311229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a34100740 0x7f3a341970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.825+0000 7f3a39d4b640 1 -- 172.21.15.84:0/1890311229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a34106750 msgr2=0x7f3a3419b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:47.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.825+0000 7f3a39d4b640 1 --2- 172.21.15.84:0/1890311229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a34106750 0x7f3a3419b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:47.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.825+0000 7f3a39d4b640 1 -- 172.21.15.84:0/1890311229 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a3419c0d0 con 0x7f3a34101100 2024-05-22T23:12:47.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.825+0000 7f3a39d4b640 1 --2- 172.21.15.84:0/1890311229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a34101100 0x7f3a34197620 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000d950 tx=0x7f3a3000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:47.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.825+0000 7f3a2b7fe640 1 -- 172.21.15.84:0/1890311229 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a30017070 con 0x7f3a34101100 2024-05-22T23:12:47.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.825+0000 7f3a2b7fe640 1 -- 172.21.15.84:0/1890311229 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a30009c30 con 0x7f3a34101100 2024-05-22T23:12:47.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.825+0000 7f3a3c7b0640 1 -- 172.21.15.84:0/1890311229 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a3419c3c0 con 0x7f3a34101100 2024-05-22T23:12:47.837 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.825+0000 7f3a2b7fe640 1 -- 172.21.15.84:0/1890311229 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3000f6b0 con 0x7f3a34101100 2024-05-22T23:12:47.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.825+0000 7f3a3c7b0640 1 -- 172.21.15.84:0/1890311229 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a341a3c50 con 0x7f3a34101100 2024-05-22T23:12:47.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.826+0000 7f3a3c7b0640 1 -- 172.21.15.84:0/1890311229 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a34068690 con 0x7f3a34101100 2024-05-22T23:12:47.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.827+0000 7f3a2b7fe640 1 -- 172.21.15.84:0/1890311229 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3a300109d0 con 0x7f3a34101100 2024-05-22T23:12:47.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.828+0000 7f3a2b7fe640 1 --2- 172.21.15.84:0/1890311229 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f3a200a2d00 0x7f3a200a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:47.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.828+0000 7f3a3a54c640 1 --2- 172.21.15.84:0/1890311229 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f3a200a2d00 0x7f3a200a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:47.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.828+0000 7f3a2b7fe640 1 -- 172.21.15.84:0/1890311229 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8613+0+0 (secure 0 0 0) 0x7f3a300cf050 con 0x7f3a34101100 2024-05-22T23:12:47.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.828+0000 7f3a3a54c640 1 --2- 172.21.15.84:0/1890311229 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f3a200a2d00 0x7f3a200a51c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3a24005fd0 tx=0x7f3a2400b390 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:47.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:47.832+0000 7f3a2b7fe640 1 -- 172.21.15.84:0/1890311229 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3a30009de0 con 0x7f3a34101100 2024-05-22T23:12:48.056 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 22 23:12:47 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-h[66549]: ignoring --setuser ceph since I am not root 2024-05-22T23:12:48.056 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 22 23:12:47 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-h[66549]: ignoring --setgroup ceph since I am not root 2024-05-22T23:12:48.056 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 22 23:12:47 smithi096 ceph-mds[66553]: -- 172.21.15.96:0/3218624945 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563ea5f4fa40 con 0x563ea5f22c00 2024-05-22T23:12:48.056 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 22 23:12:47 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-h[66549]: starting mds.h at 2024-05-22T23:12:48.057 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 22 23:12:47 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-b[66854]: ignoring --setuser ceph since I am not root 2024-05-22T23:12:48.057 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 22 23:12:47 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-b[66854]: ignoring --setgroup ceph since I am not root 2024-05-22T23:12:48.057 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 22 23:12:47 smithi096 ceph-mds[66858]: -- 172.21.15.96:0/2041415174 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563867b97a40 con 0x563867b6a800 2024-05-22T23:12:48.057 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 22 23:12:47 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-b[66854]: starting mds.b at 2024-05-22T23:12:48.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.050+0000 7f3a3c7b0640 1 -- 172.21.15.84:0/1890311229 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f3a34108cb0 con 0x7f3a34101100 2024-05-22T23:12:48.132 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:47 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-a[32182]: 2024-05-22T23:12:47.703+0000 7f02e191e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-22T23:12:48.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:48 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2750212628' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:12:48.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:48 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1248514175' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-22T23:12:48.418 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:48 smithi204 ceph-mon[36211]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-22T23:12:48.418 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:48 smithi204 ceph-mon[36211]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-22T23:12:48.418 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:48 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1248514175' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-22T23:12:48.418 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:48 smithi204 ceph-mon[36211]: osdmap e100: 12 total, 12 up, 12 in 2024-05-22T23:12:48.418 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:48 smithi204 ceph-mon[36211]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-05-22T23:12:48.418 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:48 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1890311229' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-22T23:12:48.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:48 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2750212628' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:12:48.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:48 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1248514175' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-22T23:12:48.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:48 smithi096 ceph-mon[36392]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-22T23:12:48.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:48 smithi096 ceph-mon[36392]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-22T23:12:48.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:48 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1248514175' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-22T23:12:48.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:48 smithi096 ceph-mon[36392]: osdmap e100: 12 total, 12 up, 12 in 2024-05-22T23:12:48.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:48 smithi096 ceph-mon[36392]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-05-22T23:12:48.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:48 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1890311229' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-22T23:12:48.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:48 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2750212628' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-22T23:12:48.632 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:48 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1248514175' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-22T23:12:48.632 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:48 smithi084 ceph-mon[32186]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-22T23:12:48.632 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:48 smithi084 ceph-mon[32186]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-22T23:12:48.632 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:48 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1248514175' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-22T23:12:48.632 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:48 smithi084 ceph-mon[32186]: osdmap e100: 12 total, 12 up, 12 in 2024-05-22T23:12:48.632 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:48 smithi084 ceph-mon[32186]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-05-22T23:12:48.633 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:48 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1890311229' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-22T23:12:48.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.712+0000 7f3a2b7fe640 1 -- 172.21.15.84:0/1890311229 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7f3a3009b5c0 con 0x7f3a34101100 2024-05-22T23:12:48.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.715+0000 7f3a3c7b0640 1 -- 172.21.15.84:0/1890311229 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f3a200a2d00 msgr2=0x7f3a200a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:48.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.715+0000 7f3a3c7b0640 1 --2- 172.21.15.84:0/1890311229 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f3a200a2d00 0x7f3a200a51c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3a24005fd0 tx=0x7f3a2400b390 comp rx=0 tx=0).stop 2024-05-22T23:12:48.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.715+0000 7f3a3c7b0640 1 -- 172.21.15.84:0/1890311229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a34101100 msgr2=0x7f3a34197620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:48.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.715+0000 7f3a3c7b0640 1 --2- 172.21.15.84:0/1890311229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a34101100 0x7f3a34197620 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000d950 tx=0x7f3a3000de10 comp rx=0 tx=0).stop 2024-05-22T23:12:48.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.715+0000 7f3a3c7b0640 1 -- 172.21.15.84:0/1890311229 shutdown_connections 2024-05-22T23:12:48.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.715+0000 7f3a3c7b0640 1 --2- 172.21.15.84:0/1890311229 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f3a200a2d00 0x7f3a200a51c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:48.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.715+0000 7f3a3c7b0640 1 --2- 172.21.15.84:0/1890311229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a34106750 0x7f3a3419b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:48.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.715+0000 7f3a3c7b0640 1 --2- 172.21.15.84:0/1890311229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a34101100 0x7f3a34197620 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:48.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.715+0000 7f3a3c7b0640 1 --2- 172.21.15.84:0/1890311229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a34100740 0x7f3a341970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:48.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.715+0000 7f3a3c7b0640 1 -- 172.21.15.84:0/1890311229 >> 172.21.15.84:0/1890311229 conn(0x7f3a34077870 msgr2=0x7f3a34102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:48.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.715+0000 7f3a3c7b0640 1 -- 172.21.15.84:0/1890311229 shutdown_connections 2024-05-22T23:12:48.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.716+0000 7f3a3c7b0640 1 -- 172.21.15.84:0/1890311229 wait complete. 2024-05-22T23:12:48.737 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-22T23:12:48.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.843+0000 7f0fe5a9a640 1 -- 172.21.15.84:0/2717526606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fe0102af0 msgr2=0x7f0fe0102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:48.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.843+0000 7f0fe5a9a640 1 --2- 172.21.15.84:0/2717526606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fe0102af0 0x7f0fe0102ed0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0009900 tx=0x7f0fd002a510 comp rx=0 tx=0).stop 2024-05-22T23:12:48.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.843+0000 7f0fe5a9a640 1 -- 172.21.15.84:0/2717526606 shutdown_connections 2024-05-22T23:12:48.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.843+0000 7f0fe5a9a640 1 --2- 172.21.15.84:0/2717526606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fe010be30 0x7f0fe010e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:48.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.843+0000 7f0fe5a9a640 1 --2- 172.21.15.84:0/2717526606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fe0103410 0x7f0fe010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:48.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.843+0000 7f0fe5a9a640 1 --2- 172.21.15.84:0/2717526606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fe0102af0 0x7f0fe0102ed0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:48.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.843+0000 7f0fe5a9a640 1 -- 172.21.15.84:0/2717526606 >> 172.21.15.84:0/2717526606 conn(0x7f0fe0077870 msgr2=0x7f0fe0077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:48.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.843+0000 7f0fe5a9a640 1 -- 172.21.15.84:0/2717526606 shutdown_connections 2024-05-22T23:12:48.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.843+0000 7f0fe5a9a640 1 -- 172.21.15.84:0/2717526606 wait complete. 2024-05-22T23:12:48.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.843+0000 7f0fe5a9a640 1 Processor -- start 2024-05-22T23:12:48.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.844+0000 7f0fe5a9a640 1 -- start start 2024-05-22T23:12:48.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.844+0000 7f0fe5a9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fe0102af0 0x7f0fe0198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:48.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.844+0000 7f0fe5a9a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fe0103410 0x7f0fe0198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:48.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.844+0000 7f0fe5a9a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fe010be30 0x7f0fe019cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:48.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.844+0000 7f0fe5a9a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe0111c70 con 0x7f0fe010be30 2024-05-22T23:12:48.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.844+0000 7f0fe5a9a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe0111af0 con 0x7f0fe0102af0 2024-05-22T23:12:48.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.844+0000 7f0fe5a9a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe0111df0 con 0x7f0fe0103410 2024-05-22T23:12:48.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.844+0000 7f0fdffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fe010be30 0x7f0fe019cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:48.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.844+0000 7f0fdffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fe010be30 0x7f0fe019cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40252/0 (socket says 172.21.15.84:40252) 2024-05-22T23:12:48.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.844+0000 7f0fdffff640 1 -- 172.21.15.84:0/3030030414 learned_addr learned my addr 172.21.15.84:0/3030030414 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:48.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.844+0000 7f0fdf7fe640 1 --2- 172.21.15.84:0/3030030414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fe0102af0 0x7f0fe0198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:48.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.844+0000 7f0fd7fff640 1 --2- 172.21.15.84:0/3030030414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fe0103410 0x7f0fe0198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:48.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.844+0000 7f0fdffff640 1 -- 172.21.15.84:0/3030030414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fe0103410 msgr2=0x7f0fe0198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:48.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.844+0000 7f0fdffff640 1 --2- 172.21.15.84:0/3030030414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fe0103410 0x7f0fe0198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:48.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.844+0000 7f0fdffff640 1 -- 172.21.15.84:0/3030030414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fe0102af0 msgr2=0x7f0fe0198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:48.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.844+0000 7f0fdffff640 1 --2- 172.21.15.84:0/3030030414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fe0102af0 0x7f0fe0198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:48.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.844+0000 7f0fdffff640 1 -- 172.21.15.84:0/3030030414 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe019d240 con 0x7f0fe010be30 2024-05-22T23:12:48.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.844+0000 7f0fd7fff640 1 --2- 172.21.15.84:0/3030030414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fe0103410 0x7f0fe0198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:12:48.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.844+0000 7f0fdffff640 1 --2- 172.21.15.84:0/3030030414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fe010be30 0x7f0fe019cb10 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800b970 tx=0x7f0fd800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:48.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.845+0000 7f0fdd7fa640 1 -- 172.21.15.84:0/3030030414 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8019070 con 0x7f0fe010be30 2024-05-22T23:12:48.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.845+0000 7f0fe5a9a640 1 -- 172.21.15.84:0/3030030414 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fe019d530 con 0x7f0fe010be30 2024-05-22T23:12:48.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.845+0000 7f0fdd7fa640 1 -- 172.21.15.84:0/3030030414 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fd800cb40 con 0x7f0fe010be30 2024-05-22T23:12:48.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.845+0000 7f0fdd7fa640 1 -- 172.21.15.84:0/3030030414 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8007650 con 0x7f0fe010be30 2024-05-22T23:12:48.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.845+0000 7f0fe5a9a640 1 -- 172.21.15.84:0/3030030414 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe01a4dc0 con 0x7f0fe010be30 2024-05-22T23:12:48.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.846+0000 7f0fe5a9a640 1 -- 172.21.15.84:0/3030030414 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fe0068690 con 0x7f0fe010be30 2024-05-22T23:12:48.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.848+0000 7f0fdd7fa640 1 -- 172.21.15.84:0/3030030414 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0fd801c070 con 0x7f0fe010be30 2024-05-22T23:12:48.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.848+0000 7f0fdd7fa640 1 --2- 172.21.15.84:0/3030030414 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f0fcc0a2d00 0x7f0fcc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:48.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.848+0000 7f0fdf7fe640 1 --2- 172.21.15.84:0/3030030414 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f0fcc0a2d00 0x7f0fcc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:48.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.848+0000 7f0fdd7fa640 1 -- 172.21.15.84:0/3030030414 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8613+0+0 (secure 0 0 0) 0x7f0fd80ce980 con 0x7f0fe010be30 2024-05-22T23:12:48.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.849+0000 7f0fdf7fe640 1 --2- 172.21.15.84:0/3030030414 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f0fcc0a2d00 0x7f0fcc0a51c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0009230 tx=0x7f0fd00058d0 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:48.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:48.853+0000 7f0fdd7fa640 1 -- 172.21.15.84:0/3030030414 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0fd809b000 con 0x7f0fe010be30 2024-05-22T23:12:49.044 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.036+0000 7f0fe5a9a640 1 -- 172.21.15.84:0/3030030414 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0fe002d200 con 0x7f0fe010be30 2024-05-22T23:12:49.381 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:49 smithi084 ceph-mon[32186]: pgmap v343: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-22T23:12:49.381 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:49 smithi084 ceph-mon[32186]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-22T23:12:49.381 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:49 smithi084 ceph-mon[32186]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-22T23:12:49.382 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:49 smithi084 ceph-mon[32186]: Cluster is now healthy 2024-05-22T23:12:49.382 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:49 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/1890311229' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-22T23:12:49.382 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:49 smithi084 ceph-mon[32186]: mds.? [v2:172.21.15.96:6836/3807250515,v1:172.21.15.96:6838/3807250515] up:boot 2024-05-22T23:12:49.382 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:49 smithi084 ceph-mon[32186]: mds.? [v2:172.21.15.96:6837/2748859733,v1:172.21.15.96:6839/2748859733] up:boot 2024-05-22T23:12:49.382 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:49 smithi084 ceph-mon[32186]: fsmap 12 up:standby 2024-05-22T23:12:49.382 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:49 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-22T23:12:49.382 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:49 smithi084 ceph-mon[32186]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-22T23:12:49.383 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:49 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3030030414' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-22T23:12:49.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:49 smithi204 ceph-mon[36211]: pgmap v343: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-22T23:12:49.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:49 smithi204 ceph-mon[36211]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-22T23:12:49.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:49 smithi204 ceph-mon[36211]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-22T23:12:49.417 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:49 smithi204 ceph-mon[36211]: Cluster is now healthy 2024-05-22T23:12:49.418 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:49 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/1890311229' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-22T23:12:49.418 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:49 smithi204 ceph-mon[36211]: mds.? [v2:172.21.15.96:6836/3807250515,v1:172.21.15.96:6838/3807250515] up:boot 2024-05-22T23:12:49.418 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:49 smithi204 ceph-mon[36211]: mds.? [v2:172.21.15.96:6837/2748859733,v1:172.21.15.96:6839/2748859733] up:boot 2024-05-22T23:12:49.418 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:49 smithi204 ceph-mon[36211]: fsmap 12 up:standby 2024-05-22T23:12:49.418 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:49 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-22T23:12:49.418 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:49 smithi204 ceph-mon[36211]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-22T23:12:49.418 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:49 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3030030414' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-22T23:12:49.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:49 smithi096 ceph-mon[36392]: pgmap v343: 97 pgs: 97 active+clean; 430 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-22T23:12:49.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:49 smithi096 ceph-mon[36392]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-22T23:12:49.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:49 smithi096 ceph-mon[36392]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-22T23:12:49.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:49 smithi096 ceph-mon[36392]: Cluster is now healthy 2024-05-22T23:12:49.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:49 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/1890311229' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-22T23:12:49.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:49 smithi096 ceph-mon[36392]: mds.? [v2:172.21.15.96:6836/3807250515,v1:172.21.15.96:6838/3807250515] up:boot 2024-05-22T23:12:49.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:49 smithi096 ceph-mon[36392]: mds.? [v2:172.21.15.96:6837/2748859733,v1:172.21.15.96:6839/2748859733] up:boot 2024-05-22T23:12:49.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:49 smithi096 ceph-mon[36392]: fsmap 12 up:standby 2024-05-22T23:12:49.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:49 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-22T23:12:49.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:49 smithi096 ceph-mon[36392]: from='mgr.24851 172.21.15.84:0/4084122650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-22T23:12:49.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:49 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3030030414' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-22T23:12:49.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.714+0000 7f0fdd7fa640 1 -- 172.21.15.84:0/3030030414 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v101) ==== 181+0+0 (secure 0 0 0) 0x7f0fd809f300 con 0x7f0fe010be30 2024-05-22T23:12:49.722 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' removed 2024-05-22T23:12:49.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.716+0000 7f0fe5a9a640 1 -- 172.21.15.84:0/3030030414 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f0fcc0a2d00 msgr2=0x7f0fcc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:49.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.716+0000 7f0fe5a9a640 1 --2- 172.21.15.84:0/3030030414 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f0fcc0a2d00 0x7f0fcc0a51c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0009230 tx=0x7f0fd00058d0 comp rx=0 tx=0).stop 2024-05-22T23:12:49.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.716+0000 7f0fe5a9a640 1 -- 172.21.15.84:0/3030030414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fe010be30 msgr2=0x7f0fe019cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:49.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.716+0000 7f0fe5a9a640 1 --2- 172.21.15.84:0/3030030414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fe010be30 0x7f0fe019cb10 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800b970 tx=0x7f0fd800be30 comp rx=0 tx=0).stop 2024-05-22T23:12:49.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.716+0000 7f0fe5a9a640 1 -- 172.21.15.84:0/3030030414 shutdown_connections 2024-05-22T23:12:49.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.716+0000 7f0fe5a9a640 1 --2- 172.21.15.84:0/3030030414 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7f0fcc0a2d00 0x7f0fcc0a51c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:49.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.716+0000 7f0fe5a9a640 1 --2- 172.21.15.84:0/3030030414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fe010be30 0x7f0fe019cb10 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:49.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.716+0000 7f0fe5a9a640 1 --2- 172.21.15.84:0/3030030414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fe0103410 0x7f0fe0198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:49.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.716+0000 7f0fe5a9a640 1 --2- 172.21.15.84:0/3030030414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fe0102af0 0x7f0fe0198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:49.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.717+0000 7f0fe5a9a640 1 -- 172.21.15.84:0/3030030414 >> 172.21.15.84:0/3030030414 conn(0x7f0fe0077870 msgr2=0x7f0fe00fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:49.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.717+0000 7f0fe5a9a640 1 -- 172.21.15.84:0/3030030414 shutdown_connections 2024-05-22T23:12:49.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.717+0000 7f0fe5a9a640 1 -- 172.21.15.84:0/3030030414 wait complete. 2024-05-22T23:12:49.739 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-22T23:12:49.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.847+0000 7fd59b5a9640 1 -- 172.21.15.84:0/3506854411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd594101100 msgr2=0x7fd59410d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:49.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.847+0000 7fd59b5a9640 1 --2- 172.21.15.84:0/3506854411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd594101100 0x7fd59410d610 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009cd0 tx=0x7fd58c02a8e0 comp rx=0 tx=0).stop 2024-05-22T23:12:49.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.847+0000 7fd59b5a9640 1 -- 172.21.15.84:0/3506854411 shutdown_connections 2024-05-22T23:12:49.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.847+0000 7fd59b5a9640 1 --2- 172.21.15.84:0/3506854411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd594101100 0x7fd59410d610 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:49.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.847+0000 7fd59b5a9640 1 --2- 172.21.15.84:0/3506854411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd594100740 0x7fd594100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:49.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.847+0000 7fd59b5a9640 1 --2- 172.21.15.84:0/3506854411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd594106750 0x7fd594106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:49.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.847+0000 7fd59b5a9640 1 -- 172.21.15.84:0/3506854411 >> 172.21.15.84:0/3506854411 conn(0x7fd594077870 msgr2=0x7fd594077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:49.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.847+0000 7fd59b5a9640 1 -- 172.21.15.84:0/3506854411 shutdown_connections 2024-05-22T23:12:49.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.847+0000 7fd59b5a9640 1 -- 172.21.15.84:0/3506854411 wait complete. 2024-05-22T23:12:49.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.847+0000 7fd59b5a9640 1 Processor -- start 2024-05-22T23:12:49.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.847+0000 7fd59b5a9640 1 -- start start 2024-05-22T23:12:49.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.848+0000 7fd59b5a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd594100740 0x7fd594198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:49.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.848+0000 7fd59b5a9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd594101100 0x7fd5941994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:49.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.848+0000 7fd59b5a9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd594106750 0x7fd5941a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:49.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.848+0000 7fd59b5a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd594110b60 con 0x7fd594100740 2024-05-22T23:12:49.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.848+0000 7fd59b5a9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd5941109e0 con 0x7fd594106750 2024-05-22T23:12:49.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.848+0000 7fd59b5a9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd594110ce0 con 0x7fd594101100 2024-05-22T23:12:49.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.848+0000 7fd599345640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd594100740 0x7fd594198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:49.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.848+0000 7fd599345640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd594100740 0x7fd594198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40274/0 (socket says 172.21.15.84:40274) 2024-05-22T23:12:49.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.848+0000 7fd599345640 1 -- 172.21.15.84:0/2495253614 learned_addr learned my addr 172.21.15.84:0/2495253614 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-22T23:12:49.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.848+0000 7fd598b44640 1 --2- 172.21.15.84:0/2495253614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd594101100 0x7fd5941994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:49.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.848+0000 7fd599b46640 1 --2- 172.21.15.84:0/2495253614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd594106750 0x7fd5941a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:49.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.848+0000 7fd599345640 1 -- 172.21.15.84:0/2495253614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd594101100 msgr2=0x7fd5941994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:49.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.848+0000 7fd599345640 1 --2- 172.21.15.84:0/2495253614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd594101100 0x7fd5941994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:49.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.848+0000 7fd599345640 1 -- 172.21.15.84:0/2495253614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd594106750 msgr2=0x7fd5941a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:49.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.848+0000 7fd599345640 1 --2- 172.21.15.84:0/2495253614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd594106750 0x7fd5941a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:49.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.848+0000 7fd599345640 1 -- 172.21.15.84:0/2495253614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5941a4540 con 0x7fd594100740 2024-05-22T23:12:49.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.848+0000 7fd599b46640 1 --2- 172.21.15.84:0/2495253614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd594106750 0x7fd5941a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T23:12:49.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.848+0000 7fd599345640 1 --2- 172.21.15.84:0/2495253614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd594100740 0x7fd594198f80 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd59000d950 tx=0x7fd59000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:49.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.848+0000 7fd58a7fc640 1 -- 172.21.15.84:0/2495253614 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590017070 con 0x7fd594100740 2024-05-22T23:12:49.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.848+0000 7fd58a7fc640 1 -- 172.21.15.84:0/2495253614 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd590009c30 con 0x7fd594100740 2024-05-22T23:12:49.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.848+0000 7fd59b5a9640 1 -- 172.21.15.84:0/2495253614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5941a4710 con 0x7fd594100740 2024-05-22T23:12:49.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.848+0000 7fd58a7fc640 1 -- 172.21.15.84:0/2495253614 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59000f6b0 con 0x7fd594100740 2024-05-22T23:12:49.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.848+0000 7fd59b5a9640 1 -- 172.21.15.84:0/2495253614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5941a4c30 con 0x7fd594100740 2024-05-22T23:12:49.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.849+0000 7fd59b5a9640 1 -- 172.21.15.84:0/2495253614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd594068690 con 0x7fd594100740 2024-05-22T23:12:49.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.851+0000 7fd58a7fc640 1 -- 172.21.15.84:0/2495253614 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd5900109d0 con 0x7fd594100740 2024-05-22T23:12:49.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.851+0000 7fd58a7fc640 1 --2- 172.21.15.84:0/2495253614 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd5800a2d00 0x7fd5800a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T23:12:49.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.852+0000 7fd598b44640 1 --2- 172.21.15.84:0/2495253614 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd5800a2d00 0x7fd5800a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T23:12:49.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.852+0000 7fd58a7fc640 1 -- 172.21.15.84:0/2495253614 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8059+0+0 (secure 0 0 0) 0x7fd5900cedd0 con 0x7fd594100740 2024-05-22T23:12:49.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.852+0000 7fd598b44640 1 --2- 172.21.15.84:0/2495253614 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd5800a2d00 0x7fd5800a51c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd594069550 tx=0x7fd584005f90 comp rx=0 tx=0).ready entity=mgr.24851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T23:12:49.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:49.856+0000 7fd58a7fc640 1 -- 172.21.15.84:0/2495253614 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd590009de0 con 0x7fd594100740 2024-05-22T23:12:50.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:50.038+0000 7fd59b5a9640 1 -- 172.21.15.84:0/2495253614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd594108cd0 con 0x7fd594100740 2024-05-22T23:12:50.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:50.720+0000 7fd58a7fc640 1 -- 172.21.15.84:0/2495253614 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v102) ==== 169+0+0 (secure 0 0 0) 0x7fd59009bc10 con 0x7fd594100740 2024-05-22T23:12:50.728 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' removed 2024-05-22T23:12:50.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:50.722+0000 7fd59b5a9640 1 -- 172.21.15.84:0/2495253614 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd5800a2d00 msgr2=0x7fd5800a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:50.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:50.722+0000 7fd59b5a9640 1 --2- 172.21.15.84:0/2495253614 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd5800a2d00 0x7fd5800a51c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd594069550 tx=0x7fd584005f90 comp rx=0 tx=0).stop 2024-05-22T23:12:50.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:50.722+0000 7fd59b5a9640 1 -- 172.21.15.84:0/2495253614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd594100740 msgr2=0x7fd594198f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T23:12:50.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:50.722+0000 7fd59b5a9640 1 --2- 172.21.15.84:0/2495253614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd594100740 0x7fd594198f80 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd59000d950 tx=0x7fd59000de10 comp rx=0 tx=0).stop 2024-05-22T23:12:50.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:50.722+0000 7fd59b5a9640 1 -- 172.21.15.84:0/2495253614 shutdown_connections 2024-05-22T23:12:50.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:50.722+0000 7fd59b5a9640 1 --2- 172.21.15.84:0/2495253614 >> [v2:172.21.15.84:6800/3358259585,v1:172.21.15.84:6801/3358259585] conn(0x7fd5800a2d00 0x7fd5800a51c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:50.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:50.723+0000 7fd59b5a9640 1 --2- 172.21.15.84:0/2495253614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd594106750 0x7fd5941a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:50.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:50.723+0000 7fd59b5a9640 1 --2- 172.21.15.84:0/2495253614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd594101100 0x7fd5941994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:50.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:50.723+0000 7fd59b5a9640 1 --2- 172.21.15.84:0/2495253614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd594100740 0x7fd594198f80 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T23:12:50.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:50.723+0000 7fd59b5a9640 1 -- 172.21.15.84:0/2495253614 >> 172.21.15.84:0/2495253614 conn(0x7fd594077870 msgr2=0x7fd594103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T23:12:50.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:50.723+0000 7fd59b5a9640 1 -- 172.21.15.84:0/2495253614 shutdown_connections 2024-05-22T23:12:50.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-22T23:12:50.723+0000 7fd59b5a9640 1 -- 172.21.15.84:0/2495253614 wait complete. 2024-05-22T23:12:50.746 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-22T23:12:50.772 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-22T23:12:50.799 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-22T23:12:50.826 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-22T23:12:50.826 DEBUG:teuthology.orchestra.run.smithi084:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-22T23:12:50.842 DEBUG:teuthology.orchestra.run.smithi096:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-22T23:12:50.857 DEBUG:teuthology.orchestra.run.smithi204:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-22T23:12:50.881 INFO:tasks.cephadm:Stopping all daemons... 2024-05-22T23:12:50.881 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-22T23:12:50.882 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mon.a 2024-05-22T23:12:51.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:50 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/3030030414' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-22T23:12:51.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:50 smithi096 ceph-mon[36392]: osdmap e101: 12 total, 12 up, 12 in 2024-05-22T23:12:51.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:50 smithi096 ceph-mon[36392]: from='client.? 172.21.15.84:0/2495253614' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-22T23:12:51.131 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:50 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/3030030414' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-22T23:12:51.131 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:50 smithi084 ceph-mon[32186]: osdmap e101: 12 total, 12 up, 12 in 2024-05-22T23:12:51.131 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:50 smithi084 ceph-mon[32186]: from='client.? 172.21.15.84:0/2495253614' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-22T23:12:51.132 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:50 smithi084 systemd[1]: Stopping Ceph mon.a for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:12:51.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:50 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/3030030414' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-22T23:12:51.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:50 smithi204 ceph-mon[36211]: osdmap e101: 12 total, 12 up, 12 in 2024-05-22T23:12:51.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:50 smithi204 ceph-mon[36211]: from='client.? 172.21.15.84:0/2495253614' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-22T23:12:51.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:51 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-a[32182]: 2024-05-22T23:12:51.321+0000 7f02e7129700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:12:51.631 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:51 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-a[32182]: 2024-05-22T23:12:51.321+0000 7f02e7129700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-22T23:12:51.632 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:51 smithi084 podman[94063]: 2024-05-22 23:12:51.373629194 +0000 UTC m=+0.306780199 container stop 16db2f34d801f4e3ada4ff7250f589fc39df5c0b7b5c46890567978aabd8656f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-a, org.label-schema.vendor=CentOS, RELEASE=squid-2e1f762, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240521) 2024-05-22T23:12:51.632 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:51 smithi084 podman[94063]: 2024-05-22 23:12:51.399270271 +0000 UTC m=+0.332421293 container died 16db2f34d801f4e3ada4ff7250f589fc39df5c0b7b5c46890567978aabd8656f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-2e1f762, CEPH_POINT_RELEASE=) 2024-05-22T23:12:51.989 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:51 smithi084 podman[94063]: 2024-05-22 23:12:51.773193337 +0000 UTC m=+0.706344360 container cleanup 16db2f34d801f4e3ada4ff7250f589fc39df5c0b7b5c46890567978aabd8656f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, RELEASE=squid-2e1f762, io.buildah.version=1.33.5, ceph=True) 2024-05-22T23:12:51.990 INFO:journalctl@ceph.mon.a.smithi084.stdout:May 22 23:12:51 smithi084 bash[94063]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-a 2024-05-22T23:12:52.205 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mon.a.service' 2024-05-22T23:12:52.251 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:12:52.251 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-22T23:12:52.251 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-22T23:12:52.252 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mon.b 2024-05-22T23:12:52.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:52 smithi096 systemd[1]: Stopping Ceph mon.b for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:12:52.937 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:52 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-b[36388]: 2024-05-22T23:12:52.561+0000 7f1e1db49700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:12:52.937 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:52 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-b[36388]: 2024-05-22T23:12:52.561+0000 7f1e1db49700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-22T23:12:52.937 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:52 smithi096 podman[72561]: 2024-05-22 23:12:52.623925133 +0000 UTC m=+0.186185540 container died 5d1e127aea7c509617fff2246174b529df32677202b4c909a8aa152fcaf9d73c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-2e1f762, org.label-schema.vendor=CentOS) 2024-05-22T23:12:53.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:53 smithi096 podman[72561]: 2024-05-22 23:12:53.021397062 +0000 UTC m=+0.583657459 container cleanup 5d1e127aea7c509617fff2246174b529df32677202b4c909a8aa152fcaf9d73c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-b, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-2e1f762, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-22T23:12:53.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 22 23:12:53 smithi096 bash[72561]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-b 2024-05-22T23:12:53.601 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mon.b.service' 2024-05-22T23:12:53.642 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:12:53.643 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-22T23:12:53.643 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-22T23:12:53.643 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mon.c 2024-05-22T23:12:53.963 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:53 smithi204 systemd[1]: Stopping Ceph mon.c for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:12:54.356 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:53 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-c[36194]: 2024-05-22T23:12:53.954+0000 7fe45e81b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:12:54.356 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:53 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-c[36194]: 2024-05-22T23:12:53.954+0000 7fe45e81b700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-22T23:12:54.356 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:54 smithi204 podman[72289]: 2024-05-22 23:12:54.026507348 +0000 UTC m=+0.193325481 container died 2617a778f6ff94ce63cabf3e3ac27b61b72d870d2a10e560ba4099e578d95a9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-c, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-2e1f762) 2024-05-22T23:12:54.667 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:54 smithi204 podman[72289]: 2024-05-22 23:12:54.456197527 +0000 UTC m=+0.623015653 container cleanup 2617a778f6ff94ce63cabf3e3ac27b61b72d870d2a10e560ba4099e578d95a9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-2e1f762, ceph=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-22T23:12:54.667 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:54 smithi204 bash[72289]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-c 2024-05-22T23:12:54.939 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mon.c.service' 2024-05-22T23:12:54.960 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:54 smithi204 podman[72301]: 2024-05-22 23:12:54.683265352 +0000 UTC m=+0.656883714 container remove 2617a778f6ff94ce63cabf3e3ac27b61b72d870d2a10e560ba4099e578d95a9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-2e1f762, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5) 2024-05-22T23:12:54.960 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:54 smithi204 systemd[1]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mon.c.service: Deactivated successfully. 2024-05-22T23:12:54.961 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:54 smithi204 systemd[1]: Stopped Ceph mon.c for c1135a12-188d-11ef-bc9a-c7b262605968. 2024-05-22T23:12:54.961 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 22 23:12:54 smithi204 systemd[1]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mon.c.service: Consumed 16.605s CPU time. 2024-05-22T23:12:54.982 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:12:54.982 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-22T23:12:54.983 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-22T23:12:54.983 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mgr.x 2024-05-22T23:12:55.381 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:12:55 smithi084 systemd[1]: Stopping Ceph mgr.x for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:12:55.744 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:12:55 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:12:55.410+0000 7f446f4fb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:12:55.744 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:12:55 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x[32417]: 2024-05-22T23:12:55.410+0000 7f446f4fb700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-22T23:12:55.744 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:12:55 smithi084 podman[94184]: 2024-05-22 23:12:55.443115196 +0000 UTC m=+0.265513802 container died 17eaaed076d3fbfec8c42bb33ca23f8ac7ae4cc9250993c73043f5c3f1f1cbb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.build-date=20240521, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=squid-2e1f762, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-22T23:12:56.112 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:12:55 smithi084 podman[94184]: 2024-05-22 23:12:55.845791195 +0000 UTC m=+0.668189796 container cleanup 17eaaed076d3fbfec8c42bb33ca23f8ac7ae4cc9250993c73043f5c3f1f1cbb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e1f76222e88afe0f1cda37775b2d45d88289794, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2e1f762, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521) 2024-05-22T23:12:56.113 INFO:journalctl@ceph.mgr.x.smithi084.stdout:May 22 23:12:55 smithi084 bash[94184]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-x 2024-05-22T23:12:56.269 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mgr.x.service' 2024-05-22T23:12:56.305 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:12:56.306 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-22T23:12:56.306 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-22T23:12:56.306 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mgr.y 2024-05-22T23:12:56.637 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:12:56 smithi096 systemd[1]: Stopping Ceph mgr.y for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:12:56.934 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:12:56 smithi096 podman[72694]: 2024-05-22 23:12:56.630160087 +0000 UTC m=+0.142311198 container died f4badc5353f9ad578122ed4e04c24053ff81529a90696c5983dfbad4fef01774 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2e1f762, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-22T23:12:57.185 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:12:57 smithi096 podman[72694]: 2024-05-22 23:12:57.001885424 +0000 UTC m=+0.514036526 container cleanup f4badc5353f9ad578122ed4e04c24053ff81529a90696c5983dfbad4fef01774 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-2e1f762, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-22T23:12:57.185 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:12:57 smithi096 bash[72694]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y 2024-05-22T23:12:57.420 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mgr.y.service' 2024-05-22T23:12:57.441 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:12:57 smithi096 podman[72707]: 2024-05-22 23:12:57.178756152 +0000 UTC m=+0.542890366 container remove f4badc5353f9ad578122ed4e04c24053ff81529a90696c5983dfbad4fef01774 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-y, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, RELEASE=squid-2e1f762, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD) 2024-05-22T23:12:57.441 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:12:57 smithi096 systemd[1]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-05-22T23:12:57.441 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:12:57 smithi096 systemd[1]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-05-22T23:12:57.441 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:12:57 smithi096 systemd[1]: Stopped Ceph mgr.y for c1135a12-188d-11ef-bc9a-c7b262605968. 2024-05-22T23:12:57.441 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 22 23:12:57 smithi096 systemd[1]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mgr.y.service: Consumed 19.716s CPU time. 2024-05-22T23:12:57.457 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:12:57.457 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-22T23:12:57.457 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-22T23:12:57.457 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mgr.z 2024-05-22T23:12:57.816 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:12:57 smithi204 systemd[1]: Stopping Ceph mgr.z for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:12:58.112 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:12:57 smithi204 podman[72420]: 2024-05-22 23:12:57.809786008 +0000 UTC m=+0.156496424 container died f67fd07efabb2fadfb2de9fe26b805056dd4333caff23ce2b97e558a6038f207 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2e1f762, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-22T23:12:58.417 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:12:58 smithi204 podman[72420]: 2024-05-22 23:12:58.204582958 +0000 UTC m=+0.551293373 container cleanup f67fd07efabb2fadfb2de9fe26b805056dd4333caff23ce2b97e558a6038f207 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-2e1f762, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-22T23:12:58.417 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 22 23:12:58 smithi204 bash[72420]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mgr-z 2024-05-22T23:12:58.637 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mgr.z.service' 2024-05-22T23:12:58.672 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:12:58.673 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-22T23:12:58.673 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-22T23:12:58.673 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.0 2024-05-22T23:12:58.999 INFO:journalctl@ceph.osd.0.smithi084.stdout:May 22 23:12:58 smithi084 systemd[1]: Stopping Ceph osd.0 for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:12:59.381 INFO:journalctl@ceph.osd.0.smithi084.stdout:May 22 23:12:58 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-0[44652]: 2024-05-22T23:12:58.991+0000 7f32f945d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:12:59.381 INFO:journalctl@ceph.osd.0.smithi084.stdout:May 22 23:12:58 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-0[44652]: 2024-05-22T23:12:58.991+0000 7f32f945d700 -1 osd.0 102 *** Got signal Terminated *** 2024-05-22T23:12:59.382 INFO:journalctl@ceph.osd.0.smithi084.stdout:May 22 23:12:58 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-0[44652]: 2024-05-22T23:12:58.991+0000 7f32f945d700 -1 osd.0 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-22T23:13:04.800 INFO:journalctl@ceph.osd.0.smithi084.stdout:May 22 23:13:04 smithi084 podman[94303]: 2024-05-22 23:13:04.463652565 +0000 UTC m=+5.598583984 container died 2b28939cd94ea55ce41a854b50722ee2b474a6b41a6c21acf65a3c6899abdfee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-0, org.label-schema.build-date=20240521, RELEASE=squid-2e1f762, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-22T23:13:05.109 INFO:journalctl@ceph.osd.0.smithi084.stdout:May 22 23:13:04 smithi084 podman[94303]: 2024-05-22 23:13:04.892258507 +0000 UTC m=+6.027189930 container cleanup 2b28939cd94ea55ce41a854b50722ee2b474a6b41a6c21acf65a3c6899abdfee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240521, RELEASE=squid-2e1f762, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-05-22T23:13:05.109 INFO:journalctl@ceph.osd.0.smithi084.stdout:May 22 23:13:04 smithi084 bash[94303]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-0 2024-05-22T23:13:05.381 INFO:journalctl@ceph.osd.0.smithi084.stdout:May 22 23:13:05 smithi084 podman[94317]: 2024-05-22 23:13:05.102309336 +0000 UTC m=+0.632529121 container remove 2b28939cd94ea55ce41a854b50722ee2b474a6b41a6c21acf65a3c6899abdfee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2e1f762, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-22T23:13:05.804 INFO:journalctl@ceph.osd.0.smithi084.stdout:May 22 23:13:05 smithi084 podman[94409]: 2024-05-22 23:13:05.414926442 +0000 UTC m=+0.017367425 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T23:13:05.805 INFO:journalctl@ceph.osd.0.smithi084.stdout:May 22 23:13:05 smithi084 podman[94409]: 2024-05-22 23:13:05.536559904 +0000 UTC m=+0.139000863 container create 596f2655bbb1828da69d68076be4533175a96c2f990cdb612b8812caac953c61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-0-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-2e1f762, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-22T23:13:05.805 INFO:journalctl@ceph.osd.0.smithi084.stdout:May 22 23:13:05 smithi084 podman[94409]: 2024-05-22 23:13:05.797512758 +0000 UTC m=+0.399953722 container init 596f2655bbb1828da69d68076be4533175a96c2f990cdb612b8812caac953c61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-2e1f762, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-22T23:13:06.098 INFO:journalctl@ceph.osd.0.smithi084.stdout:May 22 23:13:05 smithi084 podman[94409]: 2024-05-22 23:13:05.800887233 +0000 UTC m=+0.403328198 container start 596f2655bbb1828da69d68076be4533175a96c2f990cdb612b8812caac953c61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-2e1f762, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-22T23:13:06.099 INFO:journalctl@ceph.osd.0.smithi084.stdout:May 22 23:13:05 smithi084 podman[94409]: 2024-05-22 23:13:05.914353726 +0000 UTC m=+0.516794706 container attach 596f2655bbb1828da69d68076be4533175a96c2f990cdb612b8812caac953c61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-2e1f762, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-22T23:13:06.381 INFO:journalctl@ceph.osd.0.smithi084.stdout:May 22 23:13:06 smithi084 podman[94409]: 2024-05-22 23:13:06.091888926 +0000 UTC m=+0.694329899 container died 596f2655bbb1828da69d68076be4533175a96c2f990cdb612b8812caac953c61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-2e1f762, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-22T23:13:06.765 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.0.service' 2024-05-22T23:13:06.805 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:13:06.805 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-22T23:13:06.805 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-22T23:13:06.805 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.1 2024-05-22T23:13:07.122 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 22 23:13:06 smithi096 systemd[1]: Stopping Ceph osd.1 for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:13:07.556 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 22 23:13:07 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-1[40931]: 2024-05-22T23:13:07.114+0000 7f4f2c20e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:13:07.556 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 22 23:13:07 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-1[40931]: 2024-05-22T23:13:07.114+0000 7f4f2c20e700 -1 osd.1 102 *** Got signal Terminated *** 2024-05-22T23:13:07.556 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 22 23:13:07 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-1[40931]: 2024-05-22T23:13:07.114+0000 7f4f2c20e700 -1 osd.1 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-22T23:13:12.897 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 22 23:13:12 smithi096 podman[72822]: 2024-05-22 23:13:12.576244752 +0000 UTC m=+5.574999810 container died 17d2d24f431aeaaa5e468a422186a60911c08c350c54761f270e20b11d0b4104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-1, RELEASE=squid-2e1f762, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-22T23:13:13.256 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 22 23:13:12 smithi096 podman[72822]: 2024-05-22 23:13:12.981412903 +0000 UTC m=+5.980167958 container cleanup 17d2d24f431aeaaa5e468a422186a60911c08c350c54761f270e20b11d0b4104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-1, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-2e1f762, GIT_BRANCH=HEAD) 2024-05-22T23:13:13.257 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 22 23:13:12 smithi096 bash[72822]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-1 2024-05-22T23:13:13.556 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 22 23:13:13 smithi096 podman[72837]: 2024-05-22 23:13:13.249930248 +0000 UTC m=+0.668686599 container remove 17d2d24f431aeaaa5e468a422186a60911c08c350c54761f270e20b11d0b4104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-1, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-2e1f762, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-22T23:13:13.869 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 22 23:13:13 smithi096 podman[72928]: 2024-05-22 23:13:13.46755175 +0000 UTC m=+0.016937837 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T23:13:13.869 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 22 23:13:13 smithi096 podman[72928]: 2024-05-22 23:13:13.592713061 +0000 UTC m=+0.142099123 container create d74b48a6d8e2e9d88f1905c514a74d25dfc378cfc5b8c5357a2ec8588ac3975c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-2e1f762, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-22T23:13:13.869 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 22 23:13:13 smithi096 podman[72928]: 2024-05-22 23:13:13.86210335 +0000 UTC m=+0.411489422 container init d74b48a6d8e2e9d88f1905c514a74d25dfc378cfc5b8c5357a2ec8588ac3975c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-2e1f762, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-22T23:13:14.163 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 22 23:13:13 smithi096 podman[72928]: 2024-05-22 23:13:13.865696043 +0000 UTC m=+0.415082113 container start d74b48a6d8e2e9d88f1905c514a74d25dfc378cfc5b8c5357a2ec8588ac3975c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-2e1f762, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-22T23:13:14.163 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 22 23:13:13 smithi096 podman[72928]: 2024-05-22 23:13:13.962192908 +0000 UTC m=+0.511578989 container attach d74b48a6d8e2e9d88f1905c514a74d25dfc378cfc5b8c5357a2ec8588ac3975c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-2e1f762, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-22T23:13:14.440 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 22 23:13:14 smithi096 podman[72928]: 2024-05-22 23:13:14.157022783 +0000 UTC m=+0.706408860 container died d74b48a6d8e2e9d88f1905c514a74d25dfc378cfc5b8c5357a2ec8588ac3975c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-2e1f762, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-22T23:13:14.827 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.1.service' 2024-05-22T23:13:14.864 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:13:14.864 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-22T23:13:14.865 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-22T23:13:14.865 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.2 2024-05-22T23:13:15.167 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 22 23:13:14 smithi204 systemd[1]: Stopping Ceph osd.2 for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:13:15.667 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 22 23:13:15 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-2[41331]: 2024-05-22T23:13:15.175+0000 7f59dc90b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:13:15.667 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 22 23:13:15 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-2[41331]: 2024-05-22T23:13:15.175+0000 7f59dc90b700 -1 osd.2 102 *** Got signal Terminated *** 2024-05-22T23:13:15.667 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 22 23:13:15 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-2[41331]: 2024-05-22T23:13:15.175+0000 7f59dc90b700 -1 osd.2 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-22T23:13:20.909 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 22 23:13:20 smithi204 podman[72548]: 2024-05-22 23:13:20.592286737 +0000 UTC m=+5.536644463 container died 870ddf27a800864f4405dfe1358870a96e7f9eb3fae9b5ca25c043e4f90eaddf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2e1f762) 2024-05-22T23:13:21.167 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 22 23:13:21 smithi204 podman[72548]: 2024-05-22 23:13:21.001740032 +0000 UTC m=+5.946097757 container cleanup 870ddf27a800864f4405dfe1358870a96e7f9eb3fae9b5ca25c043e4f90eaddf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-2e1f762, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-22T23:13:21.167 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 22 23:13:21 smithi204 bash[72548]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-2 2024-05-22T23:13:21.667 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 22 23:13:21 smithi204 podman[72562]: 2024-05-22 23:13:21.25359914 +0000 UTC m=+0.660147610 container remove 870ddf27a800864f4405dfe1358870a96e7f9eb3fae9b5ca25c043e4f90eaddf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-2e1f762, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-22T23:13:21.997 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 22 23:13:21 smithi204 podman[72654]: 2024-05-22 23:13:21.584676446 +0000 UTC m=+0.017190892 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T23:13:21.997 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 22 23:13:21 smithi204 podman[72654]: 2024-05-22 23:13:21.712969773 +0000 UTC m=+0.145484253 container create 5ea0b50201b8456e2d412cc1fe27d4ad3e52bb31c9454029c008cd8745ffa3fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-2e1f762, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-22T23:13:21.998 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 22 23:13:21 smithi204 podman[72654]: 2024-05-22 23:13:21.990724639 +0000 UTC m=+0.423239076 container init 5ea0b50201b8456e2d412cc1fe27d4ad3e52bb31c9454029c008cd8745ffa3fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-2e1f762, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD) 2024-05-22T23:13:22.293 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 22 23:13:21 smithi204 podman[72654]: 2024-05-22 23:13:21.993966782 +0000 UTC m=+0.426481224 container start 5ea0b50201b8456e2d412cc1fe27d4ad3e52bb31c9454029c008cd8745ffa3fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-2e1f762, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521) 2024-05-22T23:13:22.293 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 22 23:13:22 smithi204 podman[72654]: 2024-05-22 23:13:22.116183437 +0000 UTC m=+0.548697890 container attach 5ea0b50201b8456e2d412cc1fe27d4ad3e52bb31c9454029c008cd8745ffa3fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-2e1f762, org.label-schema.build-date=20240521, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-22T23:13:22.561 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 22 23:13:22 smithi204 podman[72654]: 2024-05-22 23:13:22.286513257 +0000 UTC m=+0.719027698 container died 5ea0b50201b8456e2d412cc1fe27d4ad3e52bb31c9454029c008cd8745ffa3fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-2-deactivate, ceph=True, GIT_CLEAN=True, RELEASE=squid-2e1f762, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521) 2024-05-22T23:13:22.965 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.2.service' 2024-05-22T23:13:23.002 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:13:23.002 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-22T23:13:23.002 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-22T23:13:23.002 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.3 2024-05-22T23:13:23.381 INFO:journalctl@ceph.osd.3.smithi084.stdout:May 22 23:13:23 smithi084 systemd[1]: Stopping Ceph osd.3 for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:13:23.881 INFO:journalctl@ceph.osd.3.smithi084.stdout:May 22 23:13:23 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-3[51121]: 2024-05-22T23:13:23.394+0000 7f7c78611700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:13:23.881 INFO:journalctl@ceph.osd.3.smithi084.stdout:May 22 23:13:23 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-3[51121]: 2024-05-22T23:13:23.394+0000 7f7c78611700 -1 osd.3 102 *** Got signal Terminated *** 2024-05-22T23:13:23.882 INFO:journalctl@ceph.osd.3.smithi084.stdout:May 22 23:13:23 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-3[51121]: 2024-05-22T23:13:23.394+0000 7f7c78611700 -1 osd.3 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-22T23:13:25.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:25 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:25.577+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:26.167 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:25 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:25.899+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:26.881 INFO:journalctl@ceph.osd.3.smithi084.stdout:May 22 23:13:26 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-3[51121]: 2024-05-22T23:13:26.467+0000 7f7c70bfd700 -1 osd.3 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:03.909839+0000 front 2024-05-22T23:13:03.909868+0000 (oldest deadline 2024-05-22T23:13:26.210047+0000) 2024-05-22T23:13:26.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:26 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:26.582+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:27.167 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:26 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:26.856+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:27.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:26 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:26.816+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:27.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:27 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:27.616+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:27.882 INFO:journalctl@ceph.osd.3.smithi084.stdout:May 22 23:13:27 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-3[51121]: 2024-05-22T23:13:27.454+0000 7f7c70bfd700 -1 osd.3 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:03.909839+0000 front 2024-05-22T23:13:03.909868+0000 (oldest deadline 2024-05-22T23:13:26.210047+0000) 2024-05-22T23:13:28.167 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:27 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:27.838+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:28.168 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:27 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:27.880+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:03.782236+0000 front 2024-05-22T23:13:03.782340+0000 (oldest deadline 2024-05-22T23:13:27.282147+0000) 2024-05-22T23:13:28.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:27 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:27.831+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:29.062 INFO:journalctl@ceph.osd.3.smithi084.stdout:May 22 23:13:28 smithi084 podman[94519]: 2024-05-22 23:13:28.754258193 +0000 UTC m=+5.565789691 container died fa8878b06a16049543c97f4d2aa47ead4d6b63c873f059ecb9896b4e303b4ae9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2e1f762, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-22T23:13:29.062 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:28 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:28.644+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:29.167 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:28 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:28.888+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:03.782236+0000 front 2024-05-22T23:13:03.782340+0000 (oldest deadline 2024-05-22T23:13:27.282147+0000) 2024-05-22T23:13:29.167 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:28 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:28.847+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:29.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:28 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:28.830+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:29.336 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:29 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:29.100+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.290741+0000 front 2024-05-22T23:13:02.290722+0000 (oldest deadline 2024-05-22T23:13:28.190897+0000) 2024-05-22T23:13:29.336 INFO:journalctl@ceph.osd.3.smithi084.stdout:May 22 23:13:29 smithi084 podman[94519]: 2024-05-22 23:13:29.119103987 +0000 UTC m=+5.930635483 container cleanup fa8878b06a16049543c97f4d2aa47ead4d6b63c873f059ecb9896b4e303b4ae9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-2e1f762, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-22T23:13:29.336 INFO:journalctl@ceph.osd.3.smithi084.stdout:May 22 23:13:29 smithi084 bash[94519]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-3 2024-05-22T23:13:29.629 INFO:journalctl@ceph.osd.3.smithi084.stdout:May 22 23:13:29 smithi084 podman[94535]: 2024-05-22 23:13:29.328898169 +0000 UTC m=+0.570260064 container remove fa8878b06a16049543c97f4d2aa47ead4d6b63c873f059ecb9896b4e303b4ae9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-2e1f762, io.buildah.version=1.33.5) 2024-05-22T23:13:29.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:29 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:29.621+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:29.882 INFO:journalctl@ceph.osd.3.smithi084.stdout:May 22 23:13:29 smithi084 podman[94624]: 2024-05-22 23:13:29.646909626 +0000 UTC m=+0.016771190 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T23:13:29.882 INFO:journalctl@ceph.osd.3.smithi084.stdout:May 22 23:13:29 smithi084 podman[94624]: 2024-05-22 23:13:29.780178654 +0000 UTC m=+0.150040250 container create 985ddaa7acaf4c296c71cbb1202480776a6a85d34c641ad5b8d8145920609c75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, RELEASE=squid-2e1f762, ceph=True, io.buildah.version=1.33.5) 2024-05-22T23:13:30.167 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:29 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:29.914+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:03.782236+0000 front 2024-05-22T23:13:03.782340+0000 (oldest deadline 2024-05-22T23:13:27.282147+0000) 2024-05-22T23:13:30.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:29 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:29.849+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:30.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:29 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:29.856+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:30.313 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:30 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:30.124+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.290741+0000 front 2024-05-22T23:13:02.290722+0000 (oldest deadline 2024-05-22T23:13:28.190897+0000) 2024-05-22T23:13:30.313 INFO:journalctl@ceph.osd.3.smithi084.stdout:May 22 23:13:30 smithi084 podman[94624]: 2024-05-22 23:13:30.007763172 +0000 UTC m=+0.377624727 container init 985ddaa7acaf4c296c71cbb1202480776a6a85d34c641ad5b8d8145920609c75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-2e1f762, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-22T23:13:30.314 INFO:journalctl@ceph.osd.3.smithi084.stdout:May 22 23:13:30 smithi084 podman[94624]: 2024-05-22 23:13:30.01105083 +0000 UTC m=+0.380912383 container start 985ddaa7acaf4c296c71cbb1202480776a6a85d34c641ad5b8d8145920609c75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-3-deactivate, RELEASE=squid-2e1f762, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-22T23:13:30.314 INFO:journalctl@ceph.osd.3.smithi084.stdout:May 22 23:13:30 smithi084 podman[94624]: 2024-05-22 23:13:30.107633905 +0000 UTC m=+0.477495458 container attach 985ddaa7acaf4c296c71cbb1202480776a6a85d34c641ad5b8d8145920609c75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240521, RELEASE=squid-2e1f762, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-22T23:13:30.592 INFO:journalctl@ceph.osd.3.smithi084.stdout:May 22 23:13:30 smithi084 podman[94624]: 2024-05-22 23:13:30.30688216 +0000 UTC m=+0.676743722 container died 985ddaa7acaf4c296c71cbb1202480776a6a85d34c641ad5b8d8145920609c75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240521, RELEASE=squid-2e1f762, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-22T23:13:30.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:30 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:30.658+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:31.016 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.3.service' 2024-05-22T23:13:31.054 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:13:31.054 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-22T23:13:31.055 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-22T23:13:31.055 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.4 2024-05-22T23:13:31.167 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:30 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:30.937+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:03.782236+0000 front 2024-05-22T23:13:03.782340+0000 (oldest deadline 2024-05-22T23:13:27.282147+0000) 2024-05-22T23:13:31.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:30 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:30.855+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:31.306 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 22 23:13:31 smithi096 systemd[1]: Stopping Ceph osd.4 for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:13:31.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:30 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:30.903+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:31.381 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:31 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:31.156+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.290741+0000 front 2024-05-22T23:13:02.290722+0000 (oldest deadline 2024-05-22T23:13:28.190897+0000) 2024-05-22T23:13:31.806 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 22 23:13:31 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-4[47826]: 2024-05-22T23:13:31.351+0000 7f5fc219c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:13:31.806 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 22 23:13:31 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-4[47826]: 2024-05-22T23:13:31.351+0000 7f5fc219c700 -1 osd.4 102 *** Got signal Terminated *** 2024-05-22T23:13:31.806 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 22 23:13:31 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-4[47826]: 2024-05-22T23:13:31.351+0000 7f5fc219c700 -1 osd.4 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-22T23:13:31.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:31 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:31.699+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:32.167 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:31 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:31.958+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:03.782236+0000 front 2024-05-22T23:13:03.782340+0000 (oldest deadline 2024-05-22T23:13:27.282147+0000) 2024-05-22T23:13:32.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:31 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:31.887+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:32.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:31 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:31.879+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:32.631 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:32 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:32.146+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.290741+0000 front 2024-05-22T23:13:02.290722+0000 (oldest deadline 2024-05-22T23:13:28.190897+0000) 2024-05-22T23:13:33.131 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:32 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:32.649+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:33.136 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:32 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:32.864+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:33.167 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:32 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:32.979+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:03.782236+0000 front 2024-05-22T23:13:03.782340+0000 (oldest deadline 2024-05-22T23:13:27.282147+0000) 2024-05-22T23:13:33.167 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:32 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:32.870+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:33.556 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:33 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:33.128+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:33.615 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:33 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:33.170+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.290741+0000 front 2024-05-22T23:13:02.290722+0000 (oldest deadline 2024-05-22T23:13:28.190897+0000) 2024-05-22T23:13:33.615 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:33 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:33.170+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:08.191522+0000 front 2024-05-22T23:13:08.191670+0000 (oldest deadline 2024-05-22T23:13:32.891467+0000) 2024-05-22T23:13:33.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:33 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:33.607+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:33.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:33 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:33.607+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:34.182 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:33 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:33.815+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:34.417 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:33 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:33.981+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:03.782236+0000 front 2024-05-22T23:13:03.782340+0000 (oldest deadline 2024-05-22T23:13:27.282147+0000) 2024-05-22T23:13:34.418 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:33 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:33.981+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.282702+0000 front 2024-05-22T23:13:07.282766+0000 (oldest deadline 2024-05-22T23:13:33.182682+0000) 2024-05-22T23:13:34.418 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:33 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:33.915+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:34.556 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:34 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:34.175+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:34.588 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:34 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:34.184+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.290741+0000 front 2024-05-22T23:13:02.290722+0000 (oldest deadline 2024-05-22T23:13:28.190897+0000) 2024-05-22T23:13:34.588 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:34 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:34.184+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:08.191522+0000 front 2024-05-22T23:13:08.191670+0000 (oldest deadline 2024-05-22T23:13:32.891467+0000) 2024-05-22T23:13:34.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:34 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:34.580+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:34.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:34 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:34.580+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:35.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:34 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:34.776+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:35.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:34 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:34.776+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:35.167 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:35 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:35.024+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:03.782236+0000 front 2024-05-22T23:13:03.782340+0000 (oldest deadline 2024-05-22T23:13:27.282147+0000) 2024-05-22T23:13:35.167 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:35 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:35.024+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.282702+0000 front 2024-05-22T23:13:07.282766+0000 (oldest deadline 2024-05-22T23:13:33.182682+0000) 2024-05-22T23:13:35.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:34 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:34.860+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:35.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:34 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:34.917+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:35.556 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:35 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:35.141+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:35.566 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:35 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:35.185+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.290741+0000 front 2024-05-22T23:13:02.290722+0000 (oldest deadline 2024-05-22T23:13:28.190897+0000) 2024-05-22T23:13:35.566 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:35 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:35.185+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:08.191522+0000 front 2024-05-22T23:13:08.191670+0000 (oldest deadline 2024-05-22T23:13:32.891467+0000) 2024-05-22T23:13:35.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:35 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:35.558+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:35.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:35 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:35.558+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:36.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:35 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:35.784+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:36.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:35 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:35.784+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:36.167 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:36 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:36.060+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:03.782236+0000 front 2024-05-22T23:13:03.782340+0000 (oldest deadline 2024-05-22T23:13:27.282147+0000) 2024-05-22T23:13:36.167 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:36 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:36.060+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.282702+0000 front 2024-05-22T23:13:07.282766+0000 (oldest deadline 2024-05-22T23:13:33.182682+0000) 2024-05-22T23:13:36.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:35 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:35.925+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:36.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:35 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:35.860+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:36.556 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:36 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:36.113+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:36.612 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:36 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:36.157+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.290741+0000 front 2024-05-22T23:13:02.290722+0000 (oldest deadline 2024-05-22T23:13:28.190897+0000) 2024-05-22T23:13:36.613 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:36 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:36.157+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:08.191522+0000 front 2024-05-22T23:13:08.191670+0000 (oldest deadline 2024-05-22T23:13:32.891467+0000) 2024-05-22T23:13:36.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:36 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:36.604+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:36.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:36 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:36.604+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:37.056 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 22 23:13:36 smithi096 podman[73052]: 2024-05-22 23:13:36.880541695 +0000 UTC m=+5.631336032 container died 6c60197e2e007caa449759cad94d5c653f012f24e6d230b657197b86a2b6ae7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-2e1f762, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-22T23:13:37.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:36 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:36.752+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:37.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:36 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:36.752+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:37.167 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:36 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:36.815+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:37.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:36 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:36.929+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:37.168 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:37 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:37.061+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:03.782236+0000 front 2024-05-22T23:13:03.782340+0000 (oldest deadline 2024-05-22T23:13:27.282147+0000) 2024-05-22T23:13:37.168 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:37 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:37.061+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.282702+0000 front 2024-05-22T23:13:07.282766+0000 (oldest deadline 2024-05-22T23:13:33.182682+0000) 2024-05-22T23:13:37.518 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:37 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:37.126+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:37.519 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 22 23:13:37 smithi096 podman[73052]: 2024-05-22 23:13:37.235175368 +0000 UTC m=+5.985969702 container cleanup 6c60197e2e007caa449759cad94d5c653f012f24e6d230b657197b86a2b6ae7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-2e1f762, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-22T23:13:37.519 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 22 23:13:37 smithi096 bash[73052]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-4 2024-05-22T23:13:37.579 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:37 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:37.127+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.290741+0000 front 2024-05-22T23:13:02.290722+0000 (oldest deadline 2024-05-22T23:13:28.190897+0000) 2024-05-22T23:13:37.580 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:37 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:37.127+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:08.191522+0000 front 2024-05-22T23:13:08.191670+0000 (oldest deadline 2024-05-22T23:13:32.891467+0000) 2024-05-22T23:13:37.806 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 22 23:13:37 smithi096 podman[73069]: 2024-05-22 23:13:37.511481948 +0000 UTC m=+0.627925447 container remove 6c60197e2e007caa449759cad94d5c653f012f24e6d230b657197b86a2b6ae7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=squid-2e1f762, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-22T23:13:37.807 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:37 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:37.718+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:37.807 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:37 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:37.718+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:37.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:37 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:37.572+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:37.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:37 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:37.572+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:38.148 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 22 23:13:37 smithi096 podman[73158]: 2024-05-22 23:13:37.756243065 +0000 UTC m=+0.017557598 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T23:13:38.148 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 22 23:13:37 smithi096 podman[73158]: 2024-05-22 23:13:37.896007271 +0000 UTC m=+0.157321813 container create 19204659d0a0e41de3d34bcc7b5412df2c628b72f6f3040bb03e0e3c3272045e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-2e1f762, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-22T23:13:38.167 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:37 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:37.852+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:38.168 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:38 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:38.081+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:03.782236+0000 front 2024-05-22T23:13:03.782340+0000 (oldest deadline 2024-05-22T23:13:27.282147+0000) 2024-05-22T23:13:38.168 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:38 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:38.081+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.282702+0000 front 2024-05-22T23:13:07.282766+0000 (oldest deadline 2024-05-22T23:13:33.182682+0000) 2024-05-22T23:13:38.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:37 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:37.917+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:38.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:37 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:37.917+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:13:38.441 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:38 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:38.169+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:38.442 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 22 23:13:38 smithi096 podman[73158]: 2024-05-22 23:13:38.140572833 +0000 UTC m=+0.401887360 container init 19204659d0a0e41de3d34bcc7b5412df2c628b72f6f3040bb03e0e3c3272045e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, RELEASE=squid-2e1f762, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-22T23:13:38.442 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 22 23:13:38 smithi096 podman[73158]: 2024-05-22 23:13:38.143930673 +0000 UTC m=+0.405245199 container start 19204659d0a0e41de3d34bcc7b5412df2c628b72f6f3040bb03e0e3c3272045e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-4-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2e1f762, GIT_CLEAN=True) 2024-05-22T23:13:38.442 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 22 23:13:38 smithi096 podman[73158]: 2024-05-22 23:13:38.258805926 +0000 UTC m=+0.520120465 container attach 19204659d0a0e41de3d34bcc7b5412df2c628b72f6f3040bb03e0e3c3272045e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-4-deactivate, RELEASE=squid-2e1f762, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-22T23:13:38.601 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:38 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:38.165+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.290741+0000 front 2024-05-22T23:13:02.290722+0000 (oldest deadline 2024-05-22T23:13:28.190897+0000) 2024-05-22T23:13:38.601 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:38 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:38.165+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:08.191522+0000 front 2024-05-22T23:13:08.191670+0000 (oldest deadline 2024-05-22T23:13:32.891467+0000) 2024-05-22T23:13:38.733 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 22 23:13:38 smithi096 podman[73158]: 2024-05-22 23:13:38.434007416 +0000 UTC m=+0.695321938 container died 19204659d0a0e41de3d34bcc7b5412df2c628b72f6f3040bb03e0e3c3272045e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-2e1f762, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-22T23:13:38.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:38 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:38.593+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:38.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:38 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:38.593+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:39.018 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:38 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:38.733+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:39.018 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:38 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:38.733+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:39.037 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.4.service' 2024-05-22T23:13:39.075 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:13:39.075 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-22T23:13:39.075 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-22T23:13:39.076 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.5 2024-05-22T23:13:39.102 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:38 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:38.837+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:39.102 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:39 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:39.073+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:03.782236+0000 front 2024-05-22T23:13:03.782340+0000 (oldest deadline 2024-05-22T23:13:27.282147+0000) 2024-05-22T23:13:39.102 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:39 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:39.073+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.282702+0000 front 2024-05-22T23:13:07.282766+0000 (oldest deadline 2024-05-22T23:13:33.182682+0000) 2024-05-22T23:13:39.103 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:38 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:38.873+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:39.103 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:38 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:38.873+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:13:39.306 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:39 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:39.123+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:39.373 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:39 smithi204 systemd[1]: Stopping Ceph osd.5 for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:13:39.381 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:39 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:39.116+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.290741+0000 front 2024-05-22T23:13:02.290722+0000 (oldest deadline 2024-05-22T23:13:28.190897+0000) 2024-05-22T23:13:39.381 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:39 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:39.116+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:08.191522+0000 front 2024-05-22T23:13:08.191670+0000 (oldest deadline 2024-05-22T23:13:32.891467+0000) 2024-05-22T23:13:39.667 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:39 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:39.365+0000 7f0977745700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:13:39.667 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:39 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:39.365+0000 7f0977745700 -1 osd.5 102 *** Got signal Terminated *** 2024-05-22T23:13:39.667 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:39 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:39.365+0000 7f0977745700 -1 osd.5 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-22T23:13:39.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:39 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:39.545+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:39.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:39 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:39.545+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:40.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:39 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:39.751+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:40.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:39 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:39.751+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:40.167 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:39 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:39.871+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:40.168 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:40 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:40.114+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:03.782236+0000 front 2024-05-22T23:13:03.782340+0000 (oldest deadline 2024-05-22T23:13:27.282147+0000) 2024-05-22T23:13:40.168 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:40 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:40.114+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.282702+0000 front 2024-05-22T23:13:07.282766+0000 (oldest deadline 2024-05-22T23:13:33.182682+0000) 2024-05-22T23:13:40.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:39 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:39.872+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:40.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:39 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:39.872+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:13:40.381 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:40 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:40.075+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.290741+0000 front 2024-05-22T23:13:02.290722+0000 (oldest deadline 2024-05-22T23:13:28.190897+0000) 2024-05-22T23:13:40.381 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:40 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:40.075+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:08.191522+0000 front 2024-05-22T23:13:08.191670+0000 (oldest deadline 2024-05-22T23:13:32.891467+0000) 2024-05-22T23:13:40.556 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:40 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:40.117+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:40.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:40 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:40.569+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:40.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:40 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:40.569+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:41.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:40 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:40.776+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:41.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:40 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:40.776+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:41.084 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:40 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:40.824+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:41.084 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:40 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:40.824+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:13:41.084 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:40 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:40.856+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:41.085 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:40 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:40.856+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:13:41.381 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:41 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:41.049+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.290741+0000 front 2024-05-22T23:13:02.290722+0000 (oldest deadline 2024-05-22T23:13:28.190897+0000) 2024-05-22T23:13:41.382 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:41 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:41.049+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:08.191522+0000 front 2024-05-22T23:13:08.191670+0000 (oldest deadline 2024-05-22T23:13:32.891467+0000) 2024-05-22T23:13:41.417 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:41 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:41.076+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:03.782236+0000 front 2024-05-22T23:13:03.782340+0000 (oldest deadline 2024-05-22T23:13:27.282147+0000) 2024-05-22T23:13:41.417 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:41 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:41.076+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.282702+0000 front 2024-05-22T23:13:07.282766+0000 (oldest deadline 2024-05-22T23:13:33.182682+0000) 2024-05-22T23:13:41.556 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:41 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:41.151+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:41.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:41 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:41.567+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:41.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:41 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:41.567+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:42.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:41 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:41.768+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:42.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:41 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:41.768+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:42.082 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:41 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:41.777+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:42.082 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:41 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:41.777+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:13:42.082 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:41 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:41.835+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:42.083 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:41 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:41.835+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:13:42.381 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:42 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:42.073+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.290741+0000 front 2024-05-22T23:13:02.290722+0000 (oldest deadline 2024-05-22T23:13:28.190897+0000) 2024-05-22T23:13:42.381 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:42 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:42.073+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:08.191522+0000 front 2024-05-22T23:13:08.191670+0000 (oldest deadline 2024-05-22T23:13:32.891467+0000) 2024-05-22T23:13:42.382 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:42 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:42.073+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:17.592517+0000 front 2024-05-22T23:13:17.592574+0000 (oldest deadline 2024-05-22T23:13:41.692561+0000) 2024-05-22T23:13:42.417 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:42 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:42.074+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:03.782236+0000 front 2024-05-22T23:13:03.782340+0000 (oldest deadline 2024-05-22T23:13:27.282147+0000) 2024-05-22T23:13:42.417 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:42 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:42.074+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.282702+0000 front 2024-05-22T23:13:07.282766+0000 (oldest deadline 2024-05-22T23:13:33.182682+0000) 2024-05-22T23:13:42.556 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:42 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:42.172+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:42.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:42 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:42.548+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:42.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:42 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:42.548+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:43.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:42 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:42.770+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:43.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:42 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:42.770+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:43.167 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:42 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:42.806+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:43.167 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:42 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:42.806+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:13:43.168 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:43 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:43.041+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:03.782236+0000 front 2024-05-22T23:13:03.782340+0000 (oldest deadline 2024-05-22T23:13:27.282147+0000) 2024-05-22T23:13:43.168 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:43 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:43.041+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.282702+0000 front 2024-05-22T23:13:07.282766+0000 (oldest deadline 2024-05-22T23:13:33.182682+0000) 2024-05-22T23:13:43.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:42 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:42.826+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:43.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:42 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:42.826+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:13:43.381 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:43 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:43.052+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.290741+0000 front 2024-05-22T23:13:02.290722+0000 (oldest deadline 2024-05-22T23:13:28.190897+0000) 2024-05-22T23:13:43.381 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:43 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:43.052+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:08.191522+0000 front 2024-05-22T23:13:08.191670+0000 (oldest deadline 2024-05-22T23:13:32.891467+0000) 2024-05-22T23:13:43.382 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:43 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:43.052+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:17.592517+0000 front 2024-05-22T23:13:17.592574+0000 (oldest deadline 2024-05-22T23:13:41.692561+0000) 2024-05-22T23:13:43.556 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:43 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:43.128+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:43.556 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:43 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:43.128+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.496055+0000 front 2024-05-22T23:13:19.495914+0000 (oldest deadline 2024-05-22T23:13:42.995962+0000) 2024-05-22T23:13:43.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:43 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:43.528+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:43.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:43 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:43.528+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:44.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:43 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:43.734+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:44.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:43 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:43.734+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:44.167 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:43 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:43.796+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:44.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:43 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:43.796+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:13:44.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:43 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:43.796+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:13:44.168 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:44 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:44.006+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:03.782236+0000 front 2024-05-22T23:13:03.782340+0000 (oldest deadline 2024-05-22T23:13:27.282147+0000) 2024-05-22T23:13:44.168 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:44 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:44.006+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.282702+0000 front 2024-05-22T23:13:07.282766+0000 (oldest deadline 2024-05-22T23:13:33.182682+0000) 2024-05-22T23:13:44.169 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:44 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5[47857]: 2024-05-22T23:13:44.006+0000 7f096fd31700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.083742+0000 front 2024-05-22T23:13:19.083771+0000 (oldest deadline 2024-05-22T23:13:43.783908+0000) 2024-05-22T23:13:44.169 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:43 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:43.779+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:44.169 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:43 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:43.779+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:13:44.381 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:44 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:44.068+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.290741+0000 front 2024-05-22T23:13:02.290722+0000 (oldest deadline 2024-05-22T23:13:28.190897+0000) 2024-05-22T23:13:44.382 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:44 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:44.068+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:08.191522+0000 front 2024-05-22T23:13:08.191670+0000 (oldest deadline 2024-05-22T23:13:32.891467+0000) 2024-05-22T23:13:44.382 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:44 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:44.068+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:17.592517+0000 front 2024-05-22T23:13:17.592574+0000 (oldest deadline 2024-05-22T23:13:41.692561+0000) 2024-05-22T23:13:44.556 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:44 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:44.098+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:44.557 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:44 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:44.099+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.496055+0000 front 2024-05-22T23:13:19.495914+0000 (oldest deadline 2024-05-22T23:13:42.995962+0000) 2024-05-22T23:13:44.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:44 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:44.495+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:44.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:44 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:44.495+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:45.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:44 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:44.702+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:45.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:44 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:44.702+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:45.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:44 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:44.702+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:13:45.167 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:44 smithi204 podman[72772]: 2024-05-22 23:13:44.870053676 +0000 UTC m=+5.603688410 container died 695acd9c4cf18e79ca21db94a34c30033d21d9c6269b718b235d94221ab372e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-2e1f762, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-22T23:13:45.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:44 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:44.820+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:45.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:44 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:44.820+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:13:45.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:44 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:44.773+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:45.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:44 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:44.773+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:13:45.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:44 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:44.773+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:13:45.381 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:45 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:45.090+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.290741+0000 front 2024-05-22T23:13:02.290722+0000 (oldest deadline 2024-05-22T23:13:28.190897+0000) 2024-05-22T23:13:45.382 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:45 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:45.090+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:08.191522+0000 front 2024-05-22T23:13:08.191670+0000 (oldest deadline 2024-05-22T23:13:32.891467+0000) 2024-05-22T23:13:45.382 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:45 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:45.090+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:17.592517+0000 front 2024-05-22T23:13:17.592574+0000 (oldest deadline 2024-05-22T23:13:41.692561+0000) 2024-05-22T23:13:45.492 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:45 smithi204 podman[72772]: 2024-05-22 23:13:45.283080445 +0000 UTC m=+6.016715171 container cleanup 695acd9c4cf18e79ca21db94a34c30033d21d9c6269b718b235d94221ab372e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2e1f762, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521) 2024-05-22T23:13:45.492 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:45 smithi204 bash[72772]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5 2024-05-22T23:13:45.556 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:45 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:45.060+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:45.556 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:45 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:45.060+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.496055+0000 front 2024-05-22T23:13:19.495914+0000 (oldest deadline 2024-05-22T23:13:42.995962+0000) 2024-05-22T23:13:45.787 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:45 smithi204 podman[72790]: 2024-05-22 23:13:45.484652675 +0000 UTC m=+0.615412742 container remove 695acd9c4cf18e79ca21db94a34c30033d21d9c6269b718b235d94221ab372e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-2e1f762, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-22T23:13:45.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:45 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:45.537+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:45.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:45 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:45.537+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:45.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:45 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:45.537+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:13:46.038 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:45 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:45.862+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:46.038 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:45 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:45.862+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:13:46.038 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:45 smithi204 podman[72878]: 2024-05-22 23:13:45.812804316 +0000 UTC m=+0.016708587 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T23:13:46.038 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:45 smithi204 podman[72878]: 2024-05-22 23:13:45.918924106 +0000 UTC m=+0.122828427 container create 691c636e2e5b521602491fc94f0614748276e713b0bdebe545d4b7fc5fc0963f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-2e1f762) 2024-05-22T23:13:46.039 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:45 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:45.779+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:46.039 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:45 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:45.779+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:13:46.039 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:45 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:45.779+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:13:46.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:45 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:45.704+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:46.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:45 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:45.704+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:46.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:45 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:45.704+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:13:46.381 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:46 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:46.121+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.290741+0000 front 2024-05-22T23:13:02.290722+0000 (oldest deadline 2024-05-22T23:13:28.190897+0000) 2024-05-22T23:13:46.382 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:46 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:46.121+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:08.191522+0000 front 2024-05-22T23:13:08.191670+0000 (oldest deadline 2024-05-22T23:13:32.891467+0000) 2024-05-22T23:13:46.382 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:46 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:46.121+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:17.592517+0000 front 2024-05-22T23:13:17.592574+0000 (oldest deadline 2024-05-22T23:13:41.692561+0000) 2024-05-22T23:13:46.468 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:46 smithi204 podman[72878]: 2024-05-22 23:13:46.163010699 +0000 UTC m=+0.366914961 container init 691c636e2e5b521602491fc94f0614748276e713b0bdebe545d4b7fc5fc0963f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-2e1f762) 2024-05-22T23:13:46.468 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:46 smithi204 podman[72878]: 2024-05-22 23:13:46.166255924 +0000 UTC m=+0.370160187 container start 691c636e2e5b521602491fc94f0614748276e713b0bdebe545d4b7fc5fc0963f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-2e1f762, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-22T23:13:46.468 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:46 smithi204 podman[72878]: 2024-05-22 23:13:46.263167096 +0000 UTC m=+0.467071361 container attach 691c636e2e5b521602491fc94f0614748276e713b0bdebe545d4b7fc5fc0963f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-2e1f762, io.buildah.version=1.33.5) 2024-05-22T23:13:46.556 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:46 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:46.060+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:46.556 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:46 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:46.060+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.496055+0000 front 2024-05-22T23:13:19.495914+0000 (oldest deadline 2024-05-22T23:13:42.995962+0000) 2024-05-22T23:13:46.765 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 22 23:13:46 smithi204 podman[72878]: 2024-05-22 23:13:46.461198293 +0000 UTC m=+0.665102562 container died 691c636e2e5b521602491fc94f0614748276e713b0bdebe545d4b7fc5fc0963f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-5-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-2e1f762) 2024-05-22T23:13:46.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:46 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:46.519+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:46.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:46 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:46.519+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:46.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:46 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:46.519+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:13:47.022 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:46 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:46.695+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:47.022 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:46 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:46.695+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:47.023 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:46 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:46.695+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:13:47.070 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:46 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:46.785+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:47.071 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:46 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:46.785+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:13:47.071 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:46 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:46.785+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:13:47.071 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:46 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:46.868+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:47.071 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:46 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:46.868+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:13:47.091 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.5.service' 2024-05-22T23:13:47.174 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:13:47.175 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-22T23:13:47.175 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-22T23:13:47.175 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.6 2024-05-22T23:13:47.306 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:47 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:47.014+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:47.306 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:47 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:47.014+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.496055+0000 front 2024-05-22T23:13:19.495914+0000 (oldest deadline 2024-05-22T23:13:42.995962+0000) 2024-05-22T23:13:47.381 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:47 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:47.085+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.290741+0000 front 2024-05-22T23:13:02.290722+0000 (oldest deadline 2024-05-22T23:13:28.190897+0000) 2024-05-22T23:13:47.382 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:47 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:47.085+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:08.191522+0000 front 2024-05-22T23:13:08.191670+0000 (oldest deadline 2024-05-22T23:13:32.891467+0000) 2024-05-22T23:13:47.382 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:47 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:47.085+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:17.592517+0000 front 2024-05-22T23:13:17.592574+0000 (oldest deadline 2024-05-22T23:13:41.692561+0000) 2024-05-22T23:13:47.382 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:47 smithi084 systemd[1]: Stopping Ceph osd.6 for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:13:47.881 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:47 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:47.488+0000 7f19717a8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:13:47.882 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:47 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:47.488+0000 7f19717a8700 -1 osd.6 102 *** Got signal Terminated *** 2024-05-22T23:13:47.882 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:47 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:47.488+0000 7f19717a8700 -1 osd.6 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-22T23:13:47.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:47 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:47.519+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:47.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:47 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:47.519+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:47.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:47 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:47.519+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:13:48.056 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:47 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:47.991+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:48.056 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:47 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:47.991+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.496055+0000 front 2024-05-22T23:13:19.495914+0000 (oldest deadline 2024-05-22T23:13:42.995962+0000) 2024-05-22T23:13:48.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:47 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:47.743+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:48.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:47 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:47.743+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:48.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:47 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:47.743+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:13:48.167 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:47 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:47.868+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:48.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:47 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:47.868+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:13:48.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:47 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:47.740+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:48.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:47 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:47.740+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:13:48.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:47 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:47.740+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:13:48.381 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:48 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:48.108+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.290741+0000 front 2024-05-22T23:13:02.290722+0000 (oldest deadline 2024-05-22T23:13:28.190897+0000) 2024-05-22T23:13:48.382 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:48 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:48.108+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:08.191522+0000 front 2024-05-22T23:13:08.191670+0000 (oldest deadline 2024-05-22T23:13:32.891467+0000) 2024-05-22T23:13:48.382 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:48 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:48.108+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:17.592517+0000 front 2024-05-22T23:13:17.592574+0000 (oldest deadline 2024-05-22T23:13:41.692561+0000) 2024-05-22T23:13:48.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:48 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:48.526+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:48.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:48 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:48.526+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:48.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:48 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:48.526+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:13:49.056 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:48 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:48.954+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:49.056 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:48 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:48.954+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.496055+0000 front 2024-05-22T23:13:19.495914+0000 (oldest deadline 2024-05-22T23:13:42.995962+0000) 2024-05-22T23:13:49.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:48 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:48.773+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:49.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:48 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:48.773+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:49.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:48 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:48.773+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:13:49.167 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:48 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:48.900+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:49.167 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:48 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:48.900+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:13:49.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:48 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:48.773+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:49.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:48 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:48.773+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:13:49.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:48 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:48.773+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:13:49.381 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:49 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:49.098+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.290741+0000 front 2024-05-22T23:13:02.290722+0000 (oldest deadline 2024-05-22T23:13:28.190897+0000) 2024-05-22T23:13:49.382 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:49 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:49.098+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:08.191522+0000 front 2024-05-22T23:13:08.191670+0000 (oldest deadline 2024-05-22T23:13:32.891467+0000) 2024-05-22T23:13:49.382 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:49 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:49.098+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:17.592517+0000 front 2024-05-22T23:13:17.592574+0000 (oldest deadline 2024-05-22T23:13:41.692561+0000) 2024-05-22T23:13:49.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:49 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:49.491+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:49.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:49 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:49.491+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:49.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:49 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:49.491+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:13:49.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:49 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:49.491+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:13:50.167 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:49 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:49.874+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:50.167 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:49 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:49.874+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:13:50.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:49 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:49.751+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:50.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:49 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:49.751+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:13:50.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:49 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:49.751+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:13:50.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:49 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:49.751+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:13:50.306 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:49 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:49.944+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:50.306 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:49 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:49.944+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.496055+0000 front 2024-05-22T23:13:19.495914+0000 (oldest deadline 2024-05-22T23:13:42.995962+0000) 2024-05-22T23:13:50.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:49 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:49.808+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:50.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:49 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:49.808+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:50.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:49 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:49.808+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:13:50.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:49 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:49.808+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:13:50.381 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:50 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:50.082+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.290741+0000 front 2024-05-22T23:13:02.290722+0000 (oldest deadline 2024-05-22T23:13:28.190897+0000) 2024-05-22T23:13:50.381 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:50 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:50.082+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:08.191522+0000 front 2024-05-22T23:13:08.191670+0000 (oldest deadline 2024-05-22T23:13:32.891467+0000) 2024-05-22T23:13:50.382 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:50 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:50.082+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:17.592517+0000 front 2024-05-22T23:13:17.592574+0000 (oldest deadline 2024-05-22T23:13:41.692561+0000) 2024-05-22T23:13:50.382 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:50 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:50.082+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.593402+0000 front 2024-05-22T23:13:27.593382+0000 (oldest deadline 2024-05-22T23:13:49.293663+0000) 2024-05-22T23:13:50.803 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:50 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:50.794+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:50.804 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:50 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:50.794+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:50.804 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:50 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:50.794+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:13:50.804 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:50 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:50.794+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:13:50.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:50 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:50.531+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:50.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:50 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:50.531+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:50.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:50 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:50.531+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:13:50.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:50 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:50.531+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:13:51.056 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:50 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:50.929+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:51.056 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:50 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:50.929+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.496055+0000 front 2024-05-22T23:13:19.495914+0000 (oldest deadline 2024-05-22T23:13:42.995962+0000) 2024-05-22T23:13:51.167 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:50 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:50.832+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:51.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:50 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:50.832+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:13:51.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:50 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:50.799+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:51.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:50 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:50.799+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:13:51.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:50 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:50.799+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:13:51.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:50 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:50.799+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:13:51.381 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:51 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:51.101+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.290741+0000 front 2024-05-22T23:13:02.290722+0000 (oldest deadline 2024-05-22T23:13:28.190897+0000) 2024-05-22T23:13:51.382 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:51 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:51.101+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:08.191522+0000 front 2024-05-22T23:13:08.191670+0000 (oldest deadline 2024-05-22T23:13:32.891467+0000) 2024-05-22T23:13:51.382 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:51 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:51.101+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:17.592517+0000 front 2024-05-22T23:13:17.592574+0000 (oldest deadline 2024-05-22T23:13:41.692561+0000) 2024-05-22T23:13:51.382 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:51 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:51.101+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.593402+0000 front 2024-05-22T23:13:27.593382+0000 (oldest deadline 2024-05-22T23:13:49.293663+0000) 2024-05-22T23:13:51.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:51 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:51.569+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:51.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:51 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:51.569+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:51.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:51 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:51.569+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:13:51.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:51 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:51.569+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:13:52.056 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:51 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:51.979+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:52.056 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:51 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:51.979+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.496055+0000 front 2024-05-22T23:13:19.495914+0000 (oldest deadline 2024-05-22T23:13:42.995962+0000) 2024-05-22T23:13:52.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:51 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:51.750+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:52.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:51 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:51.750+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:52.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:51 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:51.750+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:13:52.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:51 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:51.750+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:13:52.167 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:51 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:51.846+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:52.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:51 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:51.846+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:13:52.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:51 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:51.828+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:52.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:51 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:51.828+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:13:52.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:51 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:51.828+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:13:52.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:51 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:51.828+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:13:52.381 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:52 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:52.078+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.290741+0000 front 2024-05-22T23:13:02.290722+0000 (oldest deadline 2024-05-22T23:13:28.190897+0000) 2024-05-22T23:13:52.382 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:52 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:52.078+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:08.191522+0000 front 2024-05-22T23:13:08.191670+0000 (oldest deadline 2024-05-22T23:13:32.891467+0000) 2024-05-22T23:13:52.382 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:52 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:52.078+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:17.592517+0000 front 2024-05-22T23:13:17.592574+0000 (oldest deadline 2024-05-22T23:13:41.692561+0000) 2024-05-22T23:13:52.382 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:52 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6[57890]: 2024-05-22T23:13:52.078+0000 7f196dd9c700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.593402+0000 front 2024-05-22T23:13:27.593382+0000 (oldest deadline 2024-05-22T23:13:49.293663+0000) 2024-05-22T23:13:52.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:52 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:52.554+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:52.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:52 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:52.554+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:52.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:52 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:52.554+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:13:52.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:52 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:52.554+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:13:53.056 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:52 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:52.985+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:53.056 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:52 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:52.985+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.496055+0000 front 2024-05-22T23:13:19.495914+0000 (oldest deadline 2024-05-22T23:13:42.995962+0000) 2024-05-22T23:13:53.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:52 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:52.766+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:53.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:52 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:52.766+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:53.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:52 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:52.766+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:13:53.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:52 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:52.766+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:13:53.167 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:52 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:52.804+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:53.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:52 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:52.804+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:13:53.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:52 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:52.804+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.619609+0000 front 2024-05-22T23:13:27.619667+0000 (oldest deadline 2024-05-22T23:13:52.319485+0000) 2024-05-22T23:13:53.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:52 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:52.815+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:53.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:52 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:52.815+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:13:53.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:52 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:52.815+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:13:53.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:52 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:52.815+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:13:53.288 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:52 smithi084 podman[94742]: 2024-05-22 23:13:52.966818561 +0000 UTC m=+5.603033034 container died 833460ad68322c1f4ae1bf33e50fbf54c2b1cffa7859ebddff5f1daec615b0ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-2e1f762, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-22T23:13:53.592 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:53 smithi084 podman[94742]: 2024-05-22 23:13:53.37951135 +0000 UTC m=+6.015725824 container cleanup 833460ad68322c1f4ae1bf33e50fbf54c2b1cffa7859ebddff5f1daec615b0ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6, CEPH_POINT_RELEASE=, RELEASE=squid-2e1f762, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-22T23:13:53.592 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:53 smithi084 bash[94742]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6 2024-05-22T23:13:53.881 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:53 smithi084 podman[94757]: 2024-05-22 23:13:53.589106158 +0000 UTC m=+0.625341575 container remove 833460ad68322c1f4ae1bf33e50fbf54c2b1cffa7859ebddff5f1daec615b0ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-2e1f762, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-22T23:13:53.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:53 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:53.584+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:53.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:53 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:53.584+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:53.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:53 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:53.584+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:13:53.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:53 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:53.584+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:13:54.056 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:53 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:53.996+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:54.056 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:53 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:53.996+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.496055+0000 front 2024-05-22T23:13:19.495914+0000 (oldest deadline 2024-05-22T23:13:42.995962+0000) 2024-05-22T23:13:54.056 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:53 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:53.996+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.697159+0000 front 2024-05-22T23:13:27.697139+0000 (oldest deadline 2024-05-22T23:13:52.997247+0000) 2024-05-22T23:13:54.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:53 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:53.756+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:54.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:53 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:53.756+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:54.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:53 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:53.756+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:13:54.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:53 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:53.756+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:13:54.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:53 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:53.822+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:54.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:53 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:53.822+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:13:54.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:53 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:53.822+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.619609+0000 front 2024-05-22T23:13:27.619667+0000 (oldest deadline 2024-05-22T23:13:52.319485+0000) 2024-05-22T23:13:54.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:53 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:53.842+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:54.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:53 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:53.842+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:13:54.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:53 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:53.842+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:13:54.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:53 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:53.842+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:13:54.291 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:53 smithi084 podman[94849]: 2024-05-22 23:13:53.897363243 +0000 UTC m=+0.017216082 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T23:13:54.291 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:54 smithi084 podman[94849]: 2024-05-22 23:13:54.027834745 +0000 UTC m=+0.147687583 container create f38eb7dbcd9b4bf8a0e6df5f4d0cfd85de818347a518808d09630f30d3055aff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-2e1f762, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-05-22T23:13:54.291 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:54 smithi084 podman[94849]: 2024-05-22 23:13:54.28424672 +0000 UTC m=+0.404099572 container init f38eb7dbcd9b4bf8a0e6df5f4d0cfd85de818347a518808d09630f30d3055aff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-2e1f762, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-22T23:13:54.582 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:54 smithi084 podman[94849]: 2024-05-22 23:13:54.287701633 +0000 UTC m=+0.407554477 container start f38eb7dbcd9b4bf8a0e6df5f4d0cfd85de818347a518808d09630f30d3055aff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=squid-2e1f762, org.label-schema.build-date=20240521, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-22T23:13:54.582 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:54 smithi084 podman[94849]: 2024-05-22 23:13:54.38427814 +0000 UTC m=+0.504130990 container attach f38eb7dbcd9b4bf8a0e6df5f4d0cfd85de818347a518808d09630f30d3055aff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-2e1f762, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD) 2024-05-22T23:13:54.881 INFO:journalctl@ceph.osd.6.smithi084.stdout:May 22 23:13:54 smithi084 podman[94849]: 2024-05-22 23:13:54.575434973 +0000 UTC m=+0.695287819 container died f38eb7dbcd9b4bf8a0e6df5f4d0cfd85de818347a518808d09630f30d3055aff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, RELEASE=squid-2e1f762, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-22T23:13:54.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:54 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:54.603+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:54.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:54 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:54.603+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:54.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:54 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:54.603+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:13:54.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:54 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:54.603+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:13:55.056 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:54 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:54.993+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:55.056 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:54 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:54.993+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.496055+0000 front 2024-05-22T23:13:19.495914+0000 (oldest deadline 2024-05-22T23:13:42.995962+0000) 2024-05-22T23:13:55.056 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:54 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:54.993+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.697159+0000 front 2024-05-22T23:13:27.697139+0000 (oldest deadline 2024-05-22T23:13:52.997247+0000) 2024-05-22T23:13:55.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:54 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:54.790+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:55.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:54 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:54.790+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:55.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:54 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:54.790+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:13:55.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:54 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:54.790+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:13:55.167 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:54 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:54.789+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:55.167 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:54 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:54.789+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:13:55.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:54 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:54.789+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.619609+0000 front 2024-05-22T23:13:27.619667+0000 (oldest deadline 2024-05-22T23:13:52.319485+0000) 2024-05-22T23:13:55.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:54 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:54.876+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:55.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:54 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:54.876+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:13:55.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:54 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:54.876+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:13:55.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:54 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:54.876+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:13:55.281 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.6.service' 2024-05-22T23:13:55.319 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:13:55.320 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-22T23:13:55.320 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-22T23:13:55.320 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.7 2024-05-22T23:13:55.670 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:55 smithi096 systemd[1]: Stopping Ceph osd.7 for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:13:55.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:55 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:55.589+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:55.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:55 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:55.589+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:55.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:55 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:55.589+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:13:55.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:55 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:55.589+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:13:56.033 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:55 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:55.745+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:56.033 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:55 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:55.745+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:56.033 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:55 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:55.745+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:13:56.033 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:55 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:55.745+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:13:56.033 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:55 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:55.661+0000 7f6b6ccb1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:13:56.034 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:55 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:55.661+0000 7f6b6ccb1700 -1 osd.7 102 *** Got signal Terminated *** 2024-05-22T23:13:56.034 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:55 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:55.661+0000 7f6b6ccb1700 -1 osd.7 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-22T23:13:56.034 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:56 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:56.025+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:56.034 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:56 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:56.025+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.496055+0000 front 2024-05-22T23:13:19.495914+0000 (oldest deadline 2024-05-22T23:13:42.995962+0000) 2024-05-22T23:13:56.034 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:56 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:56.025+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.697159+0000 front 2024-05-22T23:13:27.697139+0000 (oldest deadline 2024-05-22T23:13:52.997247+0000) 2024-05-22T23:13:56.167 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:55 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:55.826+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:56.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:55 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:55.826+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:13:56.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:55 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:55.826+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.619609+0000 front 2024-05-22T23:13:27.619667+0000 (oldest deadline 2024-05-22T23:13:52.319485+0000) 2024-05-22T23:13:56.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:55 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:55.905+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:56.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:55 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:55.905+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:13:56.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:55 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:55.905+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:13:56.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:55 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:55.905+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:13:56.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:56 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:56.600+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:56.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:56 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:56.600+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:56.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:56 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:56.600+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:13:56.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:56 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:56.600+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:13:57.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:56 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:56.758+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:57.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:56 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:56.758+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:57.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:56 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:56.758+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:13:57.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:56 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:56.758+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:13:57.084 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:56 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:56.857+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:57.084 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:56 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:56.857+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:13:57.084 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:56 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:56.857+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:13:57.084 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:56 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:56.857+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:13:57.085 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:56 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:56.792+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:57.085 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:56 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:56.792+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:13:57.085 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:56 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:56.792+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.619609+0000 front 2024-05-22T23:13:27.619667+0000 (oldest deadline 2024-05-22T23:13:52.319485+0000) 2024-05-22T23:13:57.556 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:57 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:57.072+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:57.556 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:57 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:57.072+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.496055+0000 front 2024-05-22T23:13:19.495914+0000 (oldest deadline 2024-05-22T23:13:42.995962+0000) 2024-05-22T23:13:57.556 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:57 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:57.072+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.697159+0000 front 2024-05-22T23:13:27.697139+0000 (oldest deadline 2024-05-22T23:13:52.997247+0000) 2024-05-22T23:13:57.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:57 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:57.592+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:57.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:57 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:57.592+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:57.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:57 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:57.592+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:13:57.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:57 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:57.592+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:13:58.167 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:57 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:57.779+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:58.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:57 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:57.779+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:13:58.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:57 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:57.779+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.619609+0000 front 2024-05-22T23:13:27.619667+0000 (oldest deadline 2024-05-22T23:13:52.319485+0000) 2024-05-22T23:13:58.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:57 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:57.779+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.820716+0000 front 2024-05-22T23:13:35.820763+0000 (oldest deadline 2024-05-22T23:13:57.520616+0000) 2024-05-22T23:13:58.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:57 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:57.862+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:58.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:57 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:57.862+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:13:58.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:57 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:57.862+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:13:58.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:57 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:57.862+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:13:58.306 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:58 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:58.029+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:58.306 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:58 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:58.029+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.496055+0000 front 2024-05-22T23:13:19.495914+0000 (oldest deadline 2024-05-22T23:13:42.995962+0000) 2024-05-22T23:13:58.306 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:58 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:58.029+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.697159+0000 front 2024-05-22T23:13:27.697139+0000 (oldest deadline 2024-05-22T23:13:52.997247+0000) 2024-05-22T23:13:58.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:57 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:57.799+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:58.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:57 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:57.799+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:58.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:57 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:57.799+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:13:58.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:57 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:57.799+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:13:58.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:58 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:58.584+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:58.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:58 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:58.584+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:58.883 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:58 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:58.584+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:13:58.883 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:58 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:58.584+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:13:59.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:58 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:58.753+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:13:59.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:58 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:58.753+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:13:59.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:58 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:58.753+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:13:59.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:58 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:58.753+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:13:59.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:58 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:58.753+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:13:59.167 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:58 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:58.824+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:59.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:58 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:58.824+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:13:59.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:58 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:58.824+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.619609+0000 front 2024-05-22T23:13:27.619667+0000 (oldest deadline 2024-05-22T23:13:52.319485+0000) 2024-05-22T23:13:59.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:58 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:58.824+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.820716+0000 front 2024-05-22T23:13:35.820763+0000 (oldest deadline 2024-05-22T23:13:57.520616+0000) 2024-05-22T23:13:59.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:58 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:58.909+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:59.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:58 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:58.909+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:13:59.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:58 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:58.909+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:13:59.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:58 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:58.909+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:13:59.556 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:59 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:59.064+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:13:59.557 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:59 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:59.064+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.496055+0000 front 2024-05-22T23:13:19.495914+0000 (oldest deadline 2024-05-22T23:13:42.995962+0000) 2024-05-22T23:13:59.557 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:59 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:59.064+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.697159+0000 front 2024-05-22T23:13:27.697139+0000 (oldest deadline 2024-05-22T23:13:52.997247+0000) 2024-05-22T23:13:59.557 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:13:59 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:13:59.064+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.498059+0000 front 2024-05-22T23:13:33.498030+0000 (oldest deadline 2024-05-22T23:13:58.098802+0000) 2024-05-22T23:13:59.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:59 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:59.598+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:13:59.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:59 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:59.598+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:13:59.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:59 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:59.598+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:13:59.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:13:59 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:13:59.598+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:13:59.905 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:59 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:59.842+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:13:59.906 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:59 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:59.842+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:13:59.906 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:59 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:59.842+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.619609+0000 front 2024-05-22T23:13:27.619667+0000 (oldest deadline 2024-05-22T23:13:52.319485+0000) 2024-05-22T23:13:59.906 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:13:59 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:13:59.842+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.820716+0000 front 2024-05-22T23:13:35.820763+0000 (oldest deadline 2024-05-22T23:13:57.520616+0000) 2024-05-22T23:13:59.906 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:59 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:59.895+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:13:59.906 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:59 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:59.895+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:13:59.906 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:59 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:59.895+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:13:59.907 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:59 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:59.895+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:13:59.907 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:13:59 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:13:59.895+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:00.044 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:59 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:59.713+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:00.044 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:59 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:59.713+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:00.044 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:59 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:59.713+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:00.044 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:59 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:59.713+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:00.044 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:13:59 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:13:59.713+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:00.306 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:14:00 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:14:00.036+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:10.194142+0000 front 2024-05-22T23:13:10.194171+0000 (oldest deadline 2024-05-22T23:13:33.094301+0000) 2024-05-22T23:14:00.306 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:14:00 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:14:00.036+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.496055+0000 front 2024-05-22T23:13:19.495914+0000 (oldest deadline 2024-05-22T23:13:42.995962+0000) 2024-05-22T23:14:00.307 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:14:00 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:14:00.036+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.697159+0000 front 2024-05-22T23:13:27.697139+0000 (oldest deadline 2024-05-22T23:13:52.997247+0000) 2024-05-22T23:14:00.307 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:14:00 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7[54792]: 2024-05-22T23:14:00.036+0000 7f6b6529d700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.498059+0000 front 2024-05-22T23:13:33.498030+0000 (oldest deadline 2024-05-22T23:13:58.098802+0000) 2024-05-22T23:14:00.984 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:00 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:00.639+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:14:00.984 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:00 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:00.639+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:14:00.984 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:00 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:00.639+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:14:00.984 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:00 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:00.639+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:14:01.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:00 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:00.749+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:01.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:00 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:00.749+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:01.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:00 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:00.749+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:01.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:00 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:00.749+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:01.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:00 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:00.749+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:01.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:00 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:00.834+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:14:01.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:00 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:00.834+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:14:01.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:00 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:00.834+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.619609+0000 front 2024-05-22T23:13:27.619667+0000 (oldest deadline 2024-05-22T23:13:52.319485+0000) 2024-05-22T23:14:01.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:00 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:00.834+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.820716+0000 front 2024-05-22T23:13:35.820763+0000 (oldest deadline 2024-05-22T23:13:57.520616+0000) 2024-05-22T23:14:01.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:00 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:00.874+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:01.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:00 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:00.874+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:01.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:00 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:00.874+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:01.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:00 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:00.874+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:01.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:00 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:00.874+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:01.528 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:14:01 smithi096 podman[73273]: 2024-05-22 23:14:01.21352388 +0000 UTC m=+5.694185298 container died f935196acd9b0171bfb3de9057be05e7a0a99acfb418604a281917f6e88b4d33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-2e1f762, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-22T23:14:01.794 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:14:01 smithi096 podman[73273]: 2024-05-22 23:14:01.612137535 +0000 UTC m=+6.092798994 container cleanup f935196acd9b0171bfb3de9057be05e7a0a99acfb418604a281917f6e88b4d33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, RELEASE=squid-2e1f762, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-22T23:14:01.794 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:14:01 smithi096 bash[73273]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7 2024-05-22T23:14:01.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:01 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:01.594+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:14:01.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:01 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:01.594+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:14:01.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:01 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:01.594+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:14:01.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:01 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:01.594+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:14:01.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:01 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:01.594+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:36.333758+0000 front 2024-05-22T23:13:36.333862+0000 (oldest deadline 2024-05-22T23:14:01.033534+0000) 2024-05-22T23:14:02.056 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:14:01 smithi096 podman[73291]: 2024-05-22 23:14:01.87216895 +0000 UTC m=+0.655050583 container remove f935196acd9b0171bfb3de9057be05e7a0a99acfb418604a281917f6e88b4d33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-2e1f762, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-22T23:14:02.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:01 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:01.786+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:02.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:01 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:01.786+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:02.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:01 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:01.786+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:02.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:01 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:01.786+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:02.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:01 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:01.786+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:02.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:01 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:01.847+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:14:02.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:01 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:01.847+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:14:02.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:01 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:01.847+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.619609+0000 front 2024-05-22T23:13:27.619667+0000 (oldest deadline 2024-05-22T23:13:52.319485+0000) 2024-05-22T23:14:02.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:01 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:01.847+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.820716+0000 front 2024-05-22T23:13:35.820763+0000 (oldest deadline 2024-05-22T23:13:57.520616+0000) 2024-05-22T23:14:02.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:01 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:01.838+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:02.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:01 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:01.838+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:02.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:01 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:01.838+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:02.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:01 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:01.838+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:02.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:01 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:01.838+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:02.541 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:14:02 smithi096 podman[73383]: 2024-05-22 23:14:02.13667674 +0000 UTC m=+0.017103586 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T23:14:02.542 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:14:02 smithi096 podman[73383]: 2024-05-22 23:14:02.256435613 +0000 UTC m=+0.136862453 container create 9a6040069aa5ae35abae20f0e5a71dd8f626c60b4e4ed29292f16a3580850f6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, GIT_CLEAN=True, RELEASE=squid-2e1f762, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-22T23:14:02.542 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:14:02 smithi096 podman[73383]: 2024-05-22 23:14:02.534484492 +0000 UTC m=+0.414911337 container init 9a6040069aa5ae35abae20f0e5a71dd8f626c60b4e4ed29292f16a3580850f6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, RELEASE=squid-2e1f762, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-22T23:14:02.806 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:14:02 smithi096 podman[73383]: 2024-05-22 23:14:02.537909935 +0000 UTC m=+0.418336789 container start 9a6040069aa5ae35abae20f0e5a71dd8f626c60b4e4ed29292f16a3580850f6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-2e1f762, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-22T23:14:02.806 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:14:02 smithi096 podman[73383]: 2024-05-22 23:14:02.609327481 +0000 UTC m=+0.489754328 container attach 9a6040069aa5ae35abae20f0e5a71dd8f626c60b4e4ed29292f16a3580850f6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, RELEASE=squid-2e1f762, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-22T23:14:03.132 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:02 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:02.626+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:14:03.132 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:02 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:02.626+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:14:03.132 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:02 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:02.626+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:14:03.132 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:02 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:02.626+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:14:03.132 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:02 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:02.626+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:36.333758+0000 front 2024-05-22T23:13:36.333862+0000 (oldest deadline 2024-05-22T23:14:01.033534+0000) 2024-05-22T23:14:03.135 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 22 23:14:02 smithi096 podman[73383]: 2024-05-22 23:14:02.828601206 +0000 UTC m=+0.709028053 container died 9a6040069aa5ae35abae20f0e5a71dd8f626c60b4e4ed29292f16a3580850f6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-2e1f762, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-05-22T23:14:03.136 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:02 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:02.834+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:03.136 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:02 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:02.834+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:03.136 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:02 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:02.834+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:03.136 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:02 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:02.834+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:03.136 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:02 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:02.834+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:03.167 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:02 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:02.845+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:14:03.167 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:02 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:02.845+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:14:03.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:02 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:02.845+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.619609+0000 front 2024-05-22T23:13:27.619667+0000 (oldest deadline 2024-05-22T23:13:52.319485+0000) 2024-05-22T23:14:03.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:02 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:02.845+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.820716+0000 front 2024-05-22T23:13:35.820763+0000 (oldest deadline 2024-05-22T23:13:57.520616+0000) 2024-05-22T23:14:03.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:02 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:02.873+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:03.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:02 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:02.873+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:03.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:02 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:02.873+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:03.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:02 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:02.873+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:03.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:02 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:02.873+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:03.449 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.7.service' 2024-05-22T23:14:03.489 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:14:03.490 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-22T23:14:03.490 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-22T23:14:03.490 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.8 2024-05-22T23:14:03.815 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:03 smithi204 systemd[1]: Stopping Ceph osd.8 for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:14:03.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:03 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:03.586+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:14:03.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:03 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:03.586+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:14:03.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:03 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:03.586+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:14:03.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:03 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:03.586+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:14:03.883 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:03 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:03.586+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:36.333758+0000 front 2024-05-22T23:13:36.333862+0000 (oldest deadline 2024-05-22T23:14:01.033534+0000) 2024-05-22T23:14:04.167 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:03 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:03.807+0000 7f34d62ee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:14:04.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:03 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:03.807+0000 7f34d62ee700 -1 osd.8 102 *** Got signal Terminated *** 2024-05-22T23:14:04.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:03 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:03.807+0000 7f34d62ee700 -1 osd.8 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-22T23:14:04.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:03 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:03.868+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:14:04.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:03 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:03.868+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:14:04.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:03 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:03.868+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.619609+0000 front 2024-05-22T23:13:27.619667+0000 (oldest deadline 2024-05-22T23:13:52.319485+0000) 2024-05-22T23:14:04.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:03 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:03.868+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.820716+0000 front 2024-05-22T23:13:35.820763+0000 (oldest deadline 2024-05-22T23:13:57.520616+0000) 2024-05-22T23:14:04.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:03 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:03.876+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:04.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:03 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:03.876+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:04.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:03 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:03.876+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:04.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:03 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:03.876+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:04.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:03 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:03.876+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:04.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:03 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:03.872+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:04.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:03 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:03.872+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:04.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:03 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:03.872+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:04.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:03 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:03.872+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:04.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:03 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:03.872+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:04.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:04 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:04.608+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:14:04.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:04 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:04.608+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:14:04.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:04 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:04.608+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:14:04.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:04 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:04.608+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:14:04.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:04 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:04.608+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:36.333758+0000 front 2024-05-22T23:13:36.333862+0000 (oldest deadline 2024-05-22T23:14:01.033534+0000) 2024-05-22T23:14:05.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:04 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:04.886+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:14:05.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:04 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:04.886+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:14:05.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:04 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:04.886+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.619609+0000 front 2024-05-22T23:13:27.619667+0000 (oldest deadline 2024-05-22T23:13:52.319485+0000) 2024-05-22T23:14:05.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:04 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:04.886+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.820716+0000 front 2024-05-22T23:13:35.820763+0000 (oldest deadline 2024-05-22T23:13:57.520616+0000) 2024-05-22T23:14:05.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:04 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:04.832+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:05.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:04 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:04.832+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:05.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:04 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:04.832+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:05.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:04 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:04.832+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:05.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:04 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:04.832+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:05.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:04 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:04.877+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:05.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:04 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:04.877+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:05.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:04 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:04.877+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:05.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:04 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:04.877+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:05.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:04 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:04.877+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:05.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:05 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:05.592+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:14:05.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:05 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:05.592+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:14:05.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:05 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:05.592+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:14:05.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:05 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:05.592+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:14:05.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:05 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:05.592+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:36.333758+0000 front 2024-05-22T23:13:36.333862+0000 (oldest deadline 2024-05-22T23:14:01.033534+0000) 2024-05-22T23:14:05.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:05 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:05.592+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.734405+0000 front 2024-05-22T23:13:42.734328+0000 (oldest deadline 2024-05-22T23:14:05.034285+0000) 2024-05-22T23:14:06.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:05 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:05.930+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:14:06.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:05 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:05.930+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:14:06.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:05 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:05.930+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.619609+0000 front 2024-05-22T23:13:27.619667+0000 (oldest deadline 2024-05-22T23:13:52.319485+0000) 2024-05-22T23:14:06.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:05 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:05.930+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.820716+0000 front 2024-05-22T23:13:35.820763+0000 (oldest deadline 2024-05-22T23:13:57.520616+0000) 2024-05-22T23:14:06.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:05 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:05.831+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:06.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:05 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:05.831+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:06.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:05 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:05.831+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:06.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:05 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:05.831+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:06.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:05 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:05.831+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:06.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:05 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:05.921+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:06.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:05 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:05.921+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:06.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:05 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:05.921+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:06.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:05 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:05.921+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:06.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:05 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:05.921+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:06.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:06 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:06.579+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:14:06.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:06 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:06.579+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:14:06.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:06 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:06.579+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:14:06.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:06 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:06.579+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:14:06.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:06 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:06.579+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:36.333758+0000 front 2024-05-22T23:13:36.333862+0000 (oldest deadline 2024-05-22T23:14:01.033534+0000) 2024-05-22T23:14:06.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:06 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:06.579+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.734405+0000 front 2024-05-22T23:13:42.734328+0000 (oldest deadline 2024-05-22T23:14:05.034285+0000) 2024-05-22T23:14:07.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:06 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:06.887+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:14:07.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:06 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:06.887+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:14:07.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:06 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:06.887+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.619609+0000 front 2024-05-22T23:13:27.619667+0000 (oldest deadline 2024-05-22T23:13:52.319485+0000) 2024-05-22T23:14:07.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:06 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:06.887+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.820716+0000 front 2024-05-22T23:13:35.820763+0000 (oldest deadline 2024-05-22T23:13:57.520616+0000) 2024-05-22T23:14:07.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:06 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:06.789+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:07.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:06 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:06.789+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:07.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:06 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:06.789+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:07.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:06 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:06.789+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:07.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:06 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:06.789+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:07.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:06 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:06.963+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:07.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:06 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:06.963+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:07.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:06 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:06.963+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:07.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:06 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:06.963+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:07.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:06 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:06.963+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:07.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:07 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:07.610+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:14:07.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:07 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:07.610+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:14:07.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:07 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:07.610+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:14:07.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:07 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:07.610+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:14:07.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:07 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:07.610+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:36.333758+0000 front 2024-05-22T23:13:36.333862+0000 (oldest deadline 2024-05-22T23:14:01.033534+0000) 2024-05-22T23:14:07.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:07 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:07.610+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.734405+0000 front 2024-05-22T23:13:42.734328+0000 (oldest deadline 2024-05-22T23:14:05.034285+0000) 2024-05-22T23:14:08.167 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:07 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:07.838+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.916521+0000 front 2024-05-22T23:13:11.916439+0000 (oldest deadline 2024-05-22T23:13:34.816480+0000) 2024-05-22T23:14:08.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:07 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:07.838+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:18.917611+0000 front 2024-05-22T23:13:18.917584+0000 (oldest deadline 2024-05-22T23:13:40.617773+0000) 2024-05-22T23:14:08.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:07 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:07.838+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.619609+0000 front 2024-05-22T23:13:27.619667+0000 (oldest deadline 2024-05-22T23:13:52.319485+0000) 2024-05-22T23:14:08.168 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:07 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8[54571]: 2024-05-22T23:14:07.838+0000 7f34ce8da700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.820716+0000 front 2024-05-22T23:13:35.820763+0000 (oldest deadline 2024-05-22T23:13:57.520616+0000) 2024-05-22T23:14:08.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:07 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:07.838+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:08.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:07 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:07.838+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:08.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:07 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:07.838+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:08.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:07 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:07.838+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:08.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:07 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:07.838+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:08.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:07 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:07.838+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:08.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:07 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:07.941+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:08.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:07 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:07.941+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:08.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:07 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:07.941+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:08.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:07 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:07.941+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:08.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:07 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:07.941+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:08.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:08 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:08.619+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:14:08.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:08 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:08.619+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:14:08.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:08 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:08.619+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:14:08.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:08 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:08.619+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:14:08.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:08 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:08.619+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:36.333758+0000 front 2024-05-22T23:13:36.333862+0000 (oldest deadline 2024-05-22T23:14:01.033534+0000) 2024-05-22T23:14:08.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:08 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:08.619+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.734405+0000 front 2024-05-22T23:13:42.734328+0000 (oldest deadline 2024-05-22T23:14:05.034285+0000) 2024-05-22T23:14:09.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:08 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:08.850+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:09.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:08 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:08.850+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:09.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:08 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:08.850+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:09.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:08 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:08.850+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:09.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:08 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:08.850+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:09.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:08 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:08.850+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:09.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:08 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:08.969+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:09.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:08 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:08.969+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:09.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:08 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:08.969+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:09.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:08 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:08.969+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:09.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:08 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:08.969+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:09.609 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:09 smithi204 podman[72996]: 2024-05-22 23:14:09.301919959 +0000 UTC m=+5.623231863 container died 67af32f59f3c2d14085ecb8674d2214bb49d1ce8545adbba3769e5f3009a58fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-2e1f762, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-22T23:14:09.860 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:09 smithi204 podman[72996]: 2024-05-22 23:14:09.708494521 +0000 UTC m=+6.029806433 container cleanup 67af32f59f3c2d14085ecb8674d2214bb49d1ce8545adbba3769e5f3009a58fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-2e1f762, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-22T23:14:09.861 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:09 smithi204 bash[72996]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8 2024-05-22T23:14:09.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:09 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:09.595+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:14:09.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:09 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:09.595+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:14:09.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:09 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:09.595+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:14:09.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:09 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:09.595+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:14:09.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:09 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:09.595+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:36.333758+0000 front 2024-05-22T23:13:36.333862+0000 (oldest deadline 2024-05-22T23:14:01.033534+0000) 2024-05-22T23:14:09.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:09 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:09.595+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.734405+0000 front 2024-05-22T23:13:42.734328+0000 (oldest deadline 2024-05-22T23:14:05.034285+0000) 2024-05-22T23:14:10.167 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:09 smithi204 podman[73010]: 2024-05-22 23:14:09.96839053 +0000 UTC m=+0.663191651 container remove 67af32f59f3c2d14085ecb8674d2214bb49d1ce8545adbba3769e5f3009a58fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-2e1f762, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux ) 2024-05-22T23:14:10.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:09 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:09.852+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:10.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:09 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:09.852+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:10.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:09 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:09.852+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:10.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:09 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:09.852+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:10.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:09 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:09.852+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:10.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:09 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:09.852+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:10.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:10 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:10.016+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:10.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:10 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:10.016+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:10.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:10 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:10.016+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:10.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:10 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:10.016+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:10.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:10 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:10.016+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:10.637 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:10 smithi204 podman[73102]: 2024-05-22 23:14:10.234950745 +0000 UTC m=+0.017531621 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T23:14:10.637 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:10 smithi204 podman[73102]: 2024-05-22 23:14:10.369179716 +0000 UTC m=+0.151760570 container create 35f4a59133369c83f226bbee29deabc35667b54591afaf5ea5f2eb838722b6f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8-deactivate, RELEASE=squid-2e1f762, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-22T23:14:10.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:10 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:10.598+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:14:10.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:10 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:10.598+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:14:10.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:10 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:10.598+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:14:10.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:10 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:10.598+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:14:10.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:10 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:10.598+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:36.333758+0000 front 2024-05-22T23:13:36.333862+0000 (oldest deadline 2024-05-22T23:14:01.033534+0000) 2024-05-22T23:14:10.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:10 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:10.598+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.734405+0000 front 2024-05-22T23:13:42.734328+0000 (oldest deadline 2024-05-22T23:14:05.034285+0000) 2024-05-22T23:14:10.917 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:10 smithi204 podman[73102]: 2024-05-22 23:14:10.630112975 +0000 UTC m=+0.412693827 container init 35f4a59133369c83f226bbee29deabc35667b54591afaf5ea5f2eb838722b6f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, RELEASE=squid-2e1f762, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-22T23:14:10.918 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:10 smithi204 podman[73102]: 2024-05-22 23:14:10.633250737 +0000 UTC m=+0.415831598 container start 35f4a59133369c83f226bbee29deabc35667b54591afaf5ea5f2eb838722b6f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-2e1f762, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-22T23:14:10.918 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:10 smithi204 podman[73102]: 2024-05-22 23:14:10.730166282 +0000 UTC m=+0.512747167 container attach 35f4a59133369c83f226bbee29deabc35667b54591afaf5ea5f2eb838722b6f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8-deactivate, RELEASE=squid-2e1f762, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-22T23:14:10.918 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:10 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:10.805+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:10.918 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:10 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:10.805+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:10.918 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:10 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:10.805+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:10.919 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:10 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:10.805+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:10.919 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:10 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:10.805+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:10.919 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:10 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:10.805+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:11.240 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 22 23:14:10 smithi204 podman[73102]: 2024-05-22 23:14:10.93465224 +0000 UTC m=+0.717233106 container died 35f4a59133369c83f226bbee29deabc35667b54591afaf5ea5f2eb838722b6f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-2e1f762, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-22T23:14:11.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:10 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:10.971+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:11.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:10 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:10.971+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:11.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:10 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:10.971+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:11.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:10 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:10.971+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:11.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:10 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:10.971+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:11.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:10 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:10.971+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:40.637512+0000 front 2024-05-22T23:13:40.637489+0000 (oldest deadline 2024-05-22T23:14:10.637674+0000) 2024-05-22T23:14:11.548 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.8.service' 2024-05-22T23:14:11.586 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:14:11.587 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-22T23:14:11.587 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-22T23:14:11.587 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.9 2024-05-22T23:14:11.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:11 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:11.551+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:14:11.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:11 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:11.551+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:14:11.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:11 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:11.551+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:14:11.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:11 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:11.551+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:14:11.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:11 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:11.551+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:36.333758+0000 front 2024-05-22T23:13:36.333862+0000 (oldest deadline 2024-05-22T23:14:01.033534+0000) 2024-05-22T23:14:11.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:11 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:11.551+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.734405+0000 front 2024-05-22T23:13:42.734328+0000 (oldest deadline 2024-05-22T23:14:05.034285+0000) 2024-05-22T23:14:11.883 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:11 smithi084 systemd[1]: Stopping Ceph osd.9 for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:14:12.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:11 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:11.790+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:12.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:11 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:11.790+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:12.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:11 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:11.790+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:12.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:11 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:11.790+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:12.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:11 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:11.790+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:12.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:11 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:11.790+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:12.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:11 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:11.956+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:12.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:11 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:11.956+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:12.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:11 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:11.956+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:12.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:11 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:11.956+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:12.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:11 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:11.956+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:12.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:11 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:11.956+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:40.637512+0000 front 2024-05-22T23:13:40.637489+0000 (oldest deadline 2024-05-22T23:14:10.637674+0000) 2024-05-22T23:14:12.382 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:11 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:11.889+0000 7f8f8a002700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:14:12.382 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:11 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:11.889+0000 7f8f8a002700 -1 osd.9 102 *** Got signal Terminated *** 2024-05-22T23:14:12.382 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:11 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:11.889+0000 7f8f8a002700 -1 osd.9 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-22T23:14:12.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:12 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:12.550+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:14:12.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:12 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:12.550+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:14:12.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:12 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:12.550+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:14:12.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:12 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:12.550+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:14:12.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:12 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:12.550+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:36.333758+0000 front 2024-05-22T23:13:36.333862+0000 (oldest deadline 2024-05-22T23:14:01.033534+0000) 2024-05-22T23:14:12.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:12 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:12.550+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.734405+0000 front 2024-05-22T23:13:42.734328+0000 (oldest deadline 2024-05-22T23:14:05.034285+0000) 2024-05-22T23:14:13.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:12 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:12.810+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:13.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:12 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:12.810+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:13.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:12 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:12.810+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:13.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:12 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:12.810+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:13.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:12 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:12.810+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:13.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:12 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:12.810+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:13.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:12 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:12.945+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:13.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:12 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:12.945+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:13.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:12 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:12.945+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:13.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:12 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:12.945+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:13.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:12 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:12.945+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:13.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:12 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:12.945+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:40.637512+0000 front 2024-05-22T23:13:40.637489+0000 (oldest deadline 2024-05-22T23:14:10.637674+0000) 2024-05-22T23:14:13.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:13 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:13.510+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:14:13.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:13 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:13.510+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:14:13.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:13 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:13.510+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:14:13.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:13 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:13.510+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:14:13.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:13 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:13.510+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:36.333758+0000 front 2024-05-22T23:13:36.333862+0000 (oldest deadline 2024-05-22T23:14:01.033534+0000) 2024-05-22T23:14:13.883 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:13 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:13.510+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.734405+0000 front 2024-05-22T23:13:42.734328+0000 (oldest deadline 2024-05-22T23:14:05.034285+0000) 2024-05-22T23:14:14.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:13 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:13.832+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:14.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:13 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:13.832+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:14.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:13 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:13.832+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:14.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:13 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:13.832+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:14.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:13 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:13.832+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:14.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:13 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:13.832+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:14.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:13 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:13.956+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:14.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:13 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:13.956+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:14.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:13 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:13.956+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:14.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:13 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:13.956+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:14.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:13 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:13.956+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:14.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:13 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:13.956+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:40.637512+0000 front 2024-05-22T23:13:40.637489+0000 (oldest deadline 2024-05-22T23:14:10.637674+0000) 2024-05-22T23:14:14.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:14 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:14.471+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:14:14.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:14 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:14.471+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:14:14.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:14 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:14.471+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:14:14.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:14 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:14.471+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:14:14.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:14 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:14.471+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:36.333758+0000 front 2024-05-22T23:13:36.333862+0000 (oldest deadline 2024-05-22T23:14:01.033534+0000) 2024-05-22T23:14:14.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:14 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:14.471+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.734405+0000 front 2024-05-22T23:13:42.734328+0000 (oldest deadline 2024-05-22T23:14:05.034285+0000) 2024-05-22T23:14:15.167 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:14 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:14.865+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:15.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:14 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:14.865+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:15.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:14 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:14.865+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:15.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:14 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:14.865+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:15.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:14 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:14.865+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:15.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:14 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:14.865+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:15.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:14 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:14.938+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:15.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:14 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:14.938+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:15.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:14 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:14.938+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:15.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:14 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:14.938+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:15.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:14 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:14.938+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:15.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:14 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:14.938+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:40.637512+0000 front 2024-05-22T23:13:40.637489+0000 (oldest deadline 2024-05-22T23:14:10.637674+0000) 2024-05-22T23:14:15.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:14 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:14.938+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-05-22T23:13:50.638427+0000 front 2024-05-22T23:13:50.638335+0000 (oldest deadline 2024-05-22T23:14:14.738143+0000) 2024-05-22T23:14:15.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:15 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:15.509+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:14:15.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:15 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:15.509+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:14:15.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:15 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:15.509+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:14:15.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:15 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:15.509+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:14:15.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:15 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:15.509+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:36.333758+0000 front 2024-05-22T23:13:36.333862+0000 (oldest deadline 2024-05-22T23:14:01.033534+0000) 2024-05-22T23:14:15.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:15 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:15.509+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.734405+0000 front 2024-05-22T23:13:42.734328+0000 (oldest deadline 2024-05-22T23:14:05.034285+0000) 2024-05-22T23:14:16.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:15 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:15.891+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:16.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:15 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:15.891+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:16.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:15 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:15.891+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:16.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:15 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:15.891+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:16.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:15 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:15.891+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:16.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:15 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:15.891+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:16.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:15 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:15.896+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:16.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:15 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:15.896+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:16.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:15 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:15.896+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:16.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:15 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:15.896+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:16.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:15 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:15.896+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:16.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:15 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:15.896+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:40.637512+0000 front 2024-05-22T23:13:40.637489+0000 (oldest deadline 2024-05-22T23:14:10.637674+0000) 2024-05-22T23:14:16.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:15 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:15.896+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-05-22T23:13:50.638427+0000 front 2024-05-22T23:13:50.638335+0000 (oldest deadline 2024-05-22T23:14:14.738143+0000) 2024-05-22T23:14:16.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:16 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:16.555+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:02.027425+0000 front 2024-05-22T23:13:02.027451+0000 (oldest deadline 2024-05-22T23:13:25.527550+0000) 2024-05-22T23:14:16.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:16 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:16.555+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:07.828588+0000 front 2024-05-22T23:13:07.828440+0000 (oldest deadline 2024-05-22T23:13:33.029317+0000) 2024-05-22T23:14:16.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:16 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:16.555+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:19.431019+0000 front 2024-05-22T23:13:19.431033+0000 (oldest deadline 2024-05-22T23:13:44.731026+0000) 2024-05-22T23:14:16.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:16 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:16.555+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:24.731576+0000 front 2024-05-22T23:13:24.731691+0000 (oldest deadline 2024-05-22T23:13:48.831738+0000) 2024-05-22T23:14:16.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:16 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:16.555+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:36.333758+0000 front 2024-05-22T23:13:36.333862+0000 (oldest deadline 2024-05-22T23:14:01.033534+0000) 2024-05-22T23:14:16.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:16 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:16.555+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.734405+0000 front 2024-05-22T23:13:42.734328+0000 (oldest deadline 2024-05-22T23:14:05.034285+0000) 2024-05-22T23:14:16.882 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:16 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9[64843]: 2024-05-22T23:14:16.555+0000 7f8f865f6700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-05-22T23:13:50.335133+0000 front 2024-05-22T23:13:50.335106+0000 (oldest deadline 2024-05-22T23:14:15.635151+0000) 2024-05-22T23:14:17.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:16 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:16.897+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:17.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:16 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:16.897+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:17.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:16 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:16.897+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:17.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:16 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:16.897+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:17.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:16 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:16.897+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:17.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:16 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:16.897+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:40.637512+0000 front 2024-05-22T23:13:40.637489+0000 (oldest deadline 2024-05-22T23:14:10.637674+0000) 2024-05-22T23:14:17.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:16 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:16.897+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-05-22T23:13:50.638427+0000 front 2024-05-22T23:13:50.638335+0000 (oldest deadline 2024-05-22T23:14:14.738143+0000) 2024-05-22T23:14:17.417 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:16 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:16.929+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:17.418 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:16 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:16.929+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:17.418 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:16 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:16.929+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:17.418 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:16 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:16.929+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:17.418 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:16 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:16.929+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:17.418 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:16 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:16.929+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:17.712 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:17 smithi084 podman[94967]: 2024-05-22 23:14:17.393131961 +0000 UTC m=+5.609950166 container died 7d9f57beecc6b388d592bd6e82f700f36f0d3beada98251bdcf26804282114cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-2e1f762, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-22T23:14:17.712 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:17 smithi084 podman[94967]: 2024-05-22 23:14:17.704854143 +0000 UTC m=+5.921672348 container cleanup 7d9f57beecc6b388d592bd6e82f700f36f0d3beada98251bdcf26804282114cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2e1f762, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-22T23:14:18.131 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:17 smithi084 bash[94967]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9 2024-05-22T23:14:18.132 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:17 smithi084 podman[94983]: 2024-05-22 23:14:17.930973532 +0000 UTC m=+0.533424649 container remove 7d9f57beecc6b388d592bd6e82f700f36f0d3beada98251bdcf26804282114cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-2e1f762, GIT_CLEAN=True) 2024-05-22T23:14:18.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:17 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:17.888+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:18.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:17 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:17.888+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:18.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:17 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:17.888+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:18.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:17 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:17.888+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:18.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:17 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:17.888+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:18.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:17 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:17.888+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:40.637512+0000 front 2024-05-22T23:13:40.637489+0000 (oldest deadline 2024-05-22T23:14:10.637674+0000) 2024-05-22T23:14:18.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:17 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:17.888+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-05-22T23:13:50.638427+0000 front 2024-05-22T23:13:50.638335+0000 (oldest deadline 2024-05-22T23:14:14.738143+0000) 2024-05-22T23:14:18.417 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:17 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:17.912+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:18.418 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:17 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:17.912+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:18.418 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:17 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:17.912+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:18.418 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:17 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:17.912+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:18.418 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:17 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:17.912+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:18.418 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:17 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:17.912+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:18.591 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:18 smithi084 podman[95068]: 2024-05-22 23:14:18.273397635 +0000 UTC m=+0.113863014 container create aaedf73fa0463c396c747221031e7456eb059801bdd4e6ff573b89d1f1cc3b50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-2e1f762, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-22T23:14:18.591 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:18 smithi084 podman[95068]: 2024-05-22 23:14:18.176709347 +0000 UTC m=+0.017174744 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T23:14:18.591 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:18 smithi084 podman[95068]: 2024-05-22 23:14:18.500672343 +0000 UTC m=+0.341137738 container init aaedf73fa0463c396c747221031e7456eb059801bdd4e6ff573b89d1f1cc3b50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2e1f762, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0) 2024-05-22T23:14:18.591 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:18 smithi084 podman[95068]: 2024-05-22 23:14:18.503928563 +0000 UTC m=+0.344393950 container start aaedf73fa0463c396c747221031e7456eb059801bdd4e6ff573b89d1f1cc3b50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2e1f762, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-22T23:14:18.592 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:18 smithi084 podman[95068]: 2024-05-22 23:14:18.583875548 +0000 UTC m=+0.424340945 container attach aaedf73fa0463c396c747221031e7456eb059801bdd4e6ff573b89d1f1cc3b50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-2e1f762, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-22T23:14:18.881 INFO:journalctl@ceph.osd.9.smithi084.stdout:May 22 23:14:18 smithi084 podman[95068]: 2024-05-22 23:14:18.803489432 +0000 UTC m=+0.643954838 container died aaedf73fa0463c396c747221031e7456eb059801bdd4e6ff573b89d1f1cc3b50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-9-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-2e1f762, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-22T23:14:19.167 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:18 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:18.907+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:19.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:18 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:18.907+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:19.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:18 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:18.907+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:19.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:18 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:18.907+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:19.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:18 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:18.907+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:19.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:18 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:18.907+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:19.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:18 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:18.862+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:19.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:18 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:18.862+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:19.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:18 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:18.862+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:19.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:18 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:18.862+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:19.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:18 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:18.862+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:19.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:18 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:18.862+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:40.637512+0000 front 2024-05-22T23:13:40.637489+0000 (oldest deadline 2024-05-22T23:14:10.637674+0000) 2024-05-22T23:14:19.308 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:18 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:18.862+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-05-22T23:13:50.638427+0000 front 2024-05-22T23:13:50.638335+0000 (oldest deadline 2024-05-22T23:14:14.738143+0000) 2024-05-22T23:14:19.375 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.9.service' 2024-05-22T23:14:19.414 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:14:19.414 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-22T23:14:19.414 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-22T23:14:19.414 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.10 2024-05-22T23:14:19.713 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:19 smithi096 systemd[1]: Stopping Ceph osd.10 for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:14:20.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:19 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:19.705+0000 7f6958608700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:14:20.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:19 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:19.705+0000 7f6958608700 -1 osd.10 102 *** Got signal Terminated *** 2024-05-22T23:14:20.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:19 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:19.705+0000 7f6958608700 -1 osd.10 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-22T23:14:20.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:19 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:19.878+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:20.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:19 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:19.878+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:20.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:19 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:19.878+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:20.058 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:19 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:19.878+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:20.058 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:19 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:19.878+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:20.058 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:19 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:19.878+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:40.637512+0000 front 2024-05-22T23:13:40.637489+0000 (oldest deadline 2024-05-22T23:14:10.637674+0000) 2024-05-22T23:14:20.058 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:19 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:19.878+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-05-22T23:13:50.638427+0000 front 2024-05-22T23:13:50.638335+0000 (oldest deadline 2024-05-22T23:14:14.738143+0000) 2024-05-22T23:14:20.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:19 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:19.904+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:20.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:19 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:19.904+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:20.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:19 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:19.904+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:20.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:19 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:19.904+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:20.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:19 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:19.904+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:20.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:19 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:19.904+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:21.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:20 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:20.862+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:21.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:20 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:20.862+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:21.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:20 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:20.862+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:21.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:20 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:20.862+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:21.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:20 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:20.862+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:21.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:20 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:20.862+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:21.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:20 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:20.865+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:21.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:20 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:20.865+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:21.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:20 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:20.865+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:21.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:20 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:20.865+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:21.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:20 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:20.865+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:21.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:20 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:20.865+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:40.637512+0000 front 2024-05-22T23:13:40.637489+0000 (oldest deadline 2024-05-22T23:14:10.637674+0000) 2024-05-22T23:14:21.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:20 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:20.865+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-05-22T23:13:50.638427+0000 front 2024-05-22T23:13:50.638335+0000 (oldest deadline 2024-05-22T23:14:14.738143+0000) 2024-05-22T23:14:22.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:21 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:21.814+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:22.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:21 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:21.814+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:22.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:21 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:21.814+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:22.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:21 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:21.814+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:22.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:21 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:21.814+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:22.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:21 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:21.814+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:22.306 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:21 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:21.826+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:22.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:21 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:21.826+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:22.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:21 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:21.826+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:22.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:21 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:21.826+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:22.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:21 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:21.826+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:22.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:21 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:21.826+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:40.637512+0000 front 2024-05-22T23:13:40.637489+0000 (oldest deadline 2024-05-22T23:14:10.637674+0000) 2024-05-22T23:14:22.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:21 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:21.826+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-05-22T23:13:50.638427+0000 front 2024-05-22T23:13:50.638335+0000 (oldest deadline 2024-05-22T23:14:14.738143+0000) 2024-05-22T23:14:23.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:22 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:22.783+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:23.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:22 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:22.783+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:23.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:22 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:22.783+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:23.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:22 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:22.783+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:23.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:22 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:22.783+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:23.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:22 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:22.783+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:40.637512+0000 front 2024-05-22T23:13:40.637489+0000 (oldest deadline 2024-05-22T23:14:10.637674+0000) 2024-05-22T23:14:23.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:22 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:22.783+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-05-22T23:13:50.638427+0000 front 2024-05-22T23:13:50.638335+0000 (oldest deadline 2024-05-22T23:14:14.738143+0000) 2024-05-22T23:14:23.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:22 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:22.837+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:23.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:22 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:22.837+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:23.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:22 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:22.837+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:23.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:22 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:22.837+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:23.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:22 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:22.837+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:23.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:22 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:22.837+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:23.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:22 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:22.837+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-22T23:13:56.547835+0000 front 2024-05-22T23:13:56.547770+0000 (oldest deadline 2024-05-22T23:14:22.447740+0000) 2024-05-22T23:14:24.056 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:23 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:23.738+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:13:01.830215+0000 front 2024-05-22T23:13:01.830333+0000 (oldest deadline 2024-05-22T23:13:26.530283+0000) 2024-05-22T23:14:24.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:23 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:23.738+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.231341+0000 front 2024-05-22T23:13:11.231435+0000 (oldest deadline 2024-05-22T23:13:34.631983+0000) 2024-05-22T23:14:24.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:23 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:23.738+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:14.632396+0000 front 2024-05-22T23:13:14.632495+0000 (oldest deadline 2024-05-22T23:13:44.632841+0000) 2024-05-22T23:14:24.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:23 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:23.738+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:27.534157+0000 front 2024-05-22T23:13:27.534212+0000 (oldest deadline 2024-05-22T23:13:49.234061+0000) 2024-05-22T23:14:24.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:23 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:23.738+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:33.835527+0000 front 2024-05-22T23:13:33.835485+0000 (oldest deadline 2024-05-22T23:13:58.535497+0000) 2024-05-22T23:14:24.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:23 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:23.738+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:40.637512+0000 front 2024-05-22T23:13:40.637489+0000 (oldest deadline 2024-05-22T23:14:10.637674+0000) 2024-05-22T23:14:24.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:23 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10[61643]: 2024-05-22T23:14:23.738+0000 7f6950bf4700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-05-22T23:13:50.638427+0000 front 2024-05-22T23:13:50.638335+0000 (oldest deadline 2024-05-22T23:14:14.738143+0000) 2024-05-22T23:14:24.167 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:23 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:23.833+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:24.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:23 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:23.833+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:24.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:23 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:23.833+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:24.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:23 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:23.833+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:24.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:23 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:23.833+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:24.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:23 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:23.833+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:24.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:23 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:23.833+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-22T23:13:56.547835+0000 front 2024-05-22T23:13:56.547770+0000 (oldest deadline 2024-05-22T23:14:22.447740+0000) 2024-05-22T23:14:25.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:24 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:24.829+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:25.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:24 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:24.829+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:25.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:24 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:24.829+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:25.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:24 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:24.829+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:25.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:24 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:24.829+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:25.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:24 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:24.829+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:25.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:24 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:24.829+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-22T23:13:56.547835+0000 front 2024-05-22T23:13:56.547770+0000 (oldest deadline 2024-05-22T23:14:22.447740+0000) 2024-05-22T23:14:25.480 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:25 smithi096 podman[73487]: 2024-05-22 23:14:25.169110552 +0000 UTC m=+5.566514859 container died 4d44d69606f25f3f61c14e729441178da636b564bbd32967e23bf9ddf6d29117 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-2e1f762, org.label-schema.vendor=CentOS) 2024-05-22T23:14:25.806 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:25 smithi096 podman[73487]: 2024-05-22 23:14:25.57217616 +0000 UTC m=+5.969580468 container cleanup 4d44d69606f25f3f61c14e729441178da636b564bbd32967e23bf9ddf6d29117 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-2e1f762, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux ) 2024-05-22T23:14:25.806 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:25 smithi096 bash[73487]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10 2024-05-22T23:14:26.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:25 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:25.830+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:26.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:25 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:25.830+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:26.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:25 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:25.830+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:26.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:25 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:25.830+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:26.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:25 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:25.830+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:26.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:25 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:25.830+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:26.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:25 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:25.830+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-22T23:13:56.547835+0000 front 2024-05-22T23:13:56.547770+0000 (oldest deadline 2024-05-22T23:14:22.447740+0000) 2024-05-22T23:14:26.202 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:25 smithi096 podman[73504]: 2024-05-22 23:14:25.831941377 +0000 UTC m=+0.659265521 container remove 4d44d69606f25f3f61c14e729441178da636b564bbd32967e23bf9ddf6d29117 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-2e1f762, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-22T23:14:26.525 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:26 smithi096 podman[73592]: 2024-05-22 23:14:26.097807037 +0000 UTC m=+0.017721712 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T23:14:26.526 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:26 smithi096 podman[73592]: 2024-05-22 23:14:26.199086028 +0000 UTC m=+0.119000727 container create 4b80fa176cea2d95c279f062bc92e868bcf1961b0efc4535867be1866154f5a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10-deactivate, RELEASE=squid-2e1f762, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-22T23:14:26.526 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:26 smithi096 podman[73592]: 2024-05-22 23:14:26.426807165 +0000 UTC m=+0.346721828 container init 4b80fa176cea2d95c279f062bc92e868bcf1961b0efc4535867be1866154f5a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=squid-2e1f762, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-22T23:14:26.526 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:26 smithi096 podman[73592]: 2024-05-22 23:14:26.429771762 +0000 UTC m=+0.349686427 container start 4b80fa176cea2d95c279f062bc92e868bcf1961b0efc4535867be1866154f5a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2e1f762, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-22T23:14:26.526 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:26 smithi096 podman[73592]: 2024-05-22 23:14:26.518345926 +0000 UTC m=+0.438260599 container attach 4b80fa176cea2d95c279f062bc92e868bcf1961b0efc4535867be1866154f5a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-2e1f762, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-05-22T23:14:26.806 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 22 23:14:26 smithi096 podman[73592]: 2024-05-22 23:14:26.720521434 +0000 UTC m=+0.640436098 container died 4b80fa176cea2d95c279f062bc92e868bcf1961b0efc4535867be1866154f5a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-2e1f762, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-22T23:14:27.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:26 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:26.861+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:27.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:26 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:26.861+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:27.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:26 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:26.861+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:27.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:26 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:26.861+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:27.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:26 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:26.861+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:27.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:26 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:26.861+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:27.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:26 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:26.861+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-22T23:13:56.547835+0000 front 2024-05-22T23:13:56.547770+0000 (oldest deadline 2024-05-22T23:14:22.447740+0000) 2024-05-22T23:14:27.303 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.10.service' 2024-05-22T23:14:27.341 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:14:27.341 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-22T23:14:27.342 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-22T23:14:27.342 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.11 2024-05-22T23:14:27.652 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:27 smithi204 systemd[1]: Stopping Ceph osd.11 for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:14:27.906 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:27 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:27.644+0000 7f78d08e3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:14:27.906 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:27 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:27.644+0000 7f78d08e3700 -1 osd.11 102 *** Got signal Terminated *** 2024-05-22T23:14:27.906 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:27 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:27.644+0000 7f78d08e3700 -1 osd.11 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-22T23:14:28.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:27 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:27.898+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:28.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:27 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:27.898+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:28.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:27 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:27.898+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:28.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:27 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:27.898+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:28.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:27 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:27.898+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:28.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:27 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:27.898+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:28.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:27 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:27.898+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-22T23:13:56.547835+0000 front 2024-05-22T23:13:56.547770+0000 (oldest deadline 2024-05-22T23:14:22.447740+0000) 2024-05-22T23:14:29.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:28 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:28.879+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:29.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:28 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:28.879+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:29.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:28 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:28.879+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:29.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:28 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:28.879+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:29.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:28 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:28.879+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:29.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:28 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:28.879+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:29.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:28 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:28.879+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-22T23:13:56.547835+0000 front 2024-05-22T23:13:56.547770+0000 (oldest deadline 2024-05-22T23:14:22.447740+0000) 2024-05-22T23:14:30.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:29 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:29.850+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:30.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:29 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:29.851+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:30.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:29 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:29.851+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:30.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:29 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:29.851+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:30.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:29 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:29.851+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:30.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:29 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:29.851+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:30.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:29 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:29.851+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-22T23:13:56.547835+0000 front 2024-05-22T23:13:56.547770+0000 (oldest deadline 2024-05-22T23:14:22.447740+0000) 2024-05-22T23:14:31.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:30 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:30.817+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:31.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:30 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:30.817+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:31.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:30 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:30.817+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:31.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:30 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:30.817+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:31.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:30 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:30.817+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:31.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:30 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:30.817+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:31.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:30 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:30.817+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-22T23:13:56.547835+0000 front 2024-05-22T23:13:56.547770+0000 (oldest deadline 2024-05-22T23:14:22.447740+0000) 2024-05-22T23:14:31.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:30 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:30.817+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-05-22T23:14:07.148653+0000 front 2024-05-22T23:14:07.148687+0000 (oldest deadline 2024-05-22T23:14:30.648657+0000) 2024-05-22T23:14:32.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:31 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:31.862+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-05-22T23:12:59.839209+0000 front 2024-05-22T23:12:59.839308+0000 (oldest deadline 2024-05-22T23:13:25.739071+0000) 2024-05-22T23:14:32.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:31 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:31.862+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-22T23:13:11.040405+0000 front 2024-05-22T23:13:11.040274+0000 (oldest deadline 2024-05-22T23:13:36.940392+0000) 2024-05-22T23:14:32.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:31 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:31.862+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-22T23:13:16.940850+0000 front 2024-05-22T23:13:16.940884+0000 (oldest deadline 2024-05-22T23:13:43.341647+0000) 2024-05-22T23:14:32.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:31 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:31.862+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-05-22T23:13:23.342349+0000 front 2024-05-22T23:13:23.342240+0000 (oldest deadline 2024-05-22T23:13:49.242180+0000) 2024-05-22T23:14:32.168 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:31 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:31.862+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-22T23:13:35.044185+0000 front 2024-05-22T23:13:35.044153+0000 (oldest deadline 2024-05-22T23:13:59.144179+0000) 2024-05-22T23:14:32.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:31 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:31.862+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-22T23:13:42.045081+0000 front 2024-05-22T23:13:42.045150+0000 (oldest deadline 2024-05-22T23:14:07.345273+0000) 2024-05-22T23:14:32.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:31 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:31.862+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-22T23:13:56.547835+0000 front 2024-05-22T23:13:56.547770+0000 (oldest deadline 2024-05-22T23:14:22.447740+0000) 2024-05-22T23:14:32.169 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:31 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11[61588]: 2024-05-22T23:14:31.862+0000 7f78c8ecf700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-05-22T23:14:07.148653+0000 front 2024-05-22T23:14:07.148687+0000 (oldest deadline 2024-05-22T23:14:30.648657+0000) 2024-05-22T23:14:33.370 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:33 smithi204 podman[73233]: 2024-05-22 23:14:33.074779184 +0000 UTC m=+5.559792697 container died dab3c44ea774d6f06e69866b7d34f045dc9ad6fc8237be9836de53323d993988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-2e1f762, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-22T23:14:33.668 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:33 smithi204 podman[73233]: 2024-05-22 23:14:33.486652017 +0000 UTC m=+5.971665525 container cleanup dab3c44ea774d6f06e69866b7d34f045dc9ad6fc8237be9836de53323d993988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, RELEASE=squid-2e1f762, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-22T23:14:33.668 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:33 smithi204 bash[73233]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11 2024-05-22T23:14:34.125 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:33 smithi204 podman[73247]: 2024-05-22 23:14:33.712777319 +0000 UTC m=+0.632043328 container remove dab3c44ea774d6f06e69866b7d34f045dc9ad6fc8237be9836de53323d993988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-2e1f762, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-22T23:14:34.418 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:34 smithi204 podman[73338]: 2024-05-22 23:14:34.02055598 +0000 UTC m=+0.016945285 image pull f0ae8441b76cd7a519a0186543d4b03eb8a47168c7797f2f5a8383edb0721f41 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036 2024-05-22T23:14:34.418 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:34 smithi204 podman[73338]: 2024-05-22 23:14:34.130186052 +0000 UTC m=+0.126575367 container create 62b0e62fc6bd3aed6eef2d9d79f1962c4dbd3db87555574d3a26ffdc181d1520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-2e1f762, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521) 2024-05-22T23:14:34.418 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:34 smithi204 podman[73338]: 2024-05-22 23:14:34.341257445 +0000 UTC m=+0.337646737 container init 62b0e62fc6bd3aed6eef2d9d79f1962c4dbd3db87555574d3a26ffdc181d1520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-2e1f762, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-22T23:14:34.418 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:34 smithi204 podman[73338]: 2024-05-22 23:14:34.3443309 +0000 UTC m=+0.340720197 container start 62b0e62fc6bd3aed6eef2d9d79f1962c4dbd3db87555574d3a26ffdc181d1520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2e1f762, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521) 2024-05-22T23:14:34.918 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:34 smithi204 podman[73338]: 2024-05-22 23:14:34.424524571 +0000 UTC m=+0.420913877 container attach 62b0e62fc6bd3aed6eef2d9d79f1962c4dbd3db87555574d3a26ffdc181d1520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-2e1f762, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-22T23:14:34.918 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 22 23:14:34 smithi204 podman[73338]: 2024-05-22 23:14:34.636325771 +0000 UTC m=+0.632715068 container died 62b0e62fc6bd3aed6eef2d9d79f1962c4dbd3db87555574d3a26ffdc181d1520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240521, ceph=True, RELEASE=squid-2e1f762, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-22T23:14:35.258 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@osd.11.service' 2024-05-22T23:14:35.300 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:14:35.301 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-22T23:14:35.301 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-22T23:14:35.301 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.a 2024-05-22T23:14:35.613 INFO:journalctl@ceph.mds.a.smithi084.stdout:May 22 23:14:35 smithi084 systemd[1]: Stopping Ceph mds.a for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:14:35.882 INFO:journalctl@ceph.mds.a.smithi084.stdout:May 22 23:14:35 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-a[71337]: 2024-05-22T23:14:35.604+0000 7f2200cf7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:14:35.882 INFO:journalctl@ceph.mds.a.smithi084.stdout:May 22 23:14:35 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-a[71337]: 2024-05-22T23:14:35.604+0000 7f2200cf7700 -1 mds.a *** got signal Terminated *** 2024-05-22T23:14:39.421 INFO:journalctl@ceph.mds.a.smithi084.stdout:May 22 23:14:39 smithi084 podman[95197]: 2024-05-22 23:14:39.158440739 +0000 UTC m=+3.670863350 container died d06a7e93c3c95de78db4b6b266e161e4ed9facaaa6bfb952fbe18b733b3d5c29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-a, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-2e1f762, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-22T23:14:39.796 INFO:journalctl@ceph.mds.a.smithi084.stdout:May 22 23:14:39 smithi084 podman[95197]: 2024-05-22 23:14:39.512423967 +0000 UTC m=+4.024846573 container cleanup d06a7e93c3c95de78db4b6b266e161e4ed9facaaa6bfb952fbe18b733b3d5c29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-2e1f762, org.label-schema.build-date=20240521) 2024-05-22T23:14:39.796 INFO:journalctl@ceph.mds.a.smithi084.stdout:May 22 23:14:39 smithi084 bash[95197]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-a 2024-05-22T23:14:39.956 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.a.service' 2024-05-22T23:14:39.991 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:14:39.991 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-22T23:14:39.992 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-22T23:14:39.992 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.d 2024-05-22T23:14:40.382 INFO:journalctl@ceph.mds.d.smithi084.stdout:May 22 23:14:40 smithi084 systemd[1]: Stopping Ceph mds.d for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:14:40.382 INFO:journalctl@ceph.mds.d.smithi084.stdout:May 22 23:14:40 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-d[70379]: 2024-05-22T23:14:40.291+0000 7fa488edb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:14:40.383 INFO:journalctl@ceph.mds.d.smithi084.stdout:May 22 23:14:40 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-d[70379]: 2024-05-22T23:14:40.291+0000 7fa488edb700 -1 mds.d *** got signal Terminated *** 2024-05-22T23:14:43.424 INFO:journalctl@ceph.mds.d.smithi084.stdout:May 22 23:14:43 smithi084 podman[95320]: 2024-05-22 23:14:43.138765525 +0000 UTC m=+2.933936629 container died 275f492e41f0d6f41e53bd175f22227d777cfff169306d3c33b0193ef25c7c47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-2e1f762, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-22T23:14:43.783 INFO:journalctl@ceph.mds.d.smithi084.stdout:May 22 23:14:43 smithi084 podman[95320]: 2024-05-22 23:14:43.499421903 +0000 UTC m=+3.294593008 container cleanup 275f492e41f0d6f41e53bd175f22227d777cfff169306d3c33b0193ef25c7c47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-d, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-2e1f762, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-05-22T23:14:43.783 INFO:journalctl@ceph.mds.d.smithi084.stdout:May 22 23:14:43 smithi084 bash[95320]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-d 2024-05-22T23:14:43.940 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.d.service' 2024-05-22T23:14:43.973 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:14:43.974 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-22T23:14:43.974 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-22T23:14:43.974 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.g 2024-05-22T23:14:44.327 INFO:journalctl@ceph.mds.g.smithi084.stdout:May 22 23:14:44 smithi084 systemd[1]: Stopping Ceph mds.g for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:14:44.632 INFO:journalctl@ceph.mds.g.smithi084.stdout:May 22 23:14:44 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-g[70981]: 2024-05-22T23:14:44.319+0000 7fef82229700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:14:44.632 INFO:journalctl@ceph.mds.g.smithi084.stdout:May 22 23:14:44 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-g[70981]: 2024-05-22T23:14:44.319+0000 7fef82229700 -1 mds.g *** got signal Terminated *** 2024-05-22T23:14:48.632 INFO:journalctl@ceph.mds.g.smithi084.stdout:May 22 23:14:48 smithi084 podman[95443]: 2024-05-22 23:14:48.325204266 +0000 UTC m=+4.141069423 container died dfa016465ba0dedaee560b8bf48747f3e208d696e262d619ba7c82eef82cd365 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-g, RELEASE=squid-2e1f762, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-22T23:14:48.961 INFO:journalctl@ceph.mds.g.smithi084.stdout:May 22 23:14:48 smithi084 podman[95443]: 2024-05-22 23:14:48.735850059 +0000 UTC m=+4.551715216 container cleanup dfa016465ba0dedaee560b8bf48747f3e208d696e262d619ba7c82eef82cd365 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-g, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, RELEASE=squid-2e1f762, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True) 2024-05-22T23:14:48.961 INFO:journalctl@ceph.mds.g.smithi084.stdout:May 22 23:14:48 smithi084 bash[95443]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-g 2024-05-22T23:14:49.108 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.g.service' 2024-05-22T23:14:49.140 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:14:49.140 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-22T23:14:49.141 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-22T23:14:49.141 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.j 2024-05-22T23:14:49.631 INFO:journalctl@ceph.mds.j.smithi084.stdout:May 22 23:14:49 smithi084 systemd[1]: Stopping Ceph mds.j for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:14:49.632 INFO:journalctl@ceph.mds.j.smithi084.stdout:May 22 23:14:49 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-j[69996]: 2024-05-22T23:14:49.447+0000 7f784dbe6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:14:49.632 INFO:journalctl@ceph.mds.j.smithi084.stdout:May 22 23:14:49 smithi084 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-j[69996]: 2024-05-22T23:14:49.447+0000 7f784dbe6700 -1 mds.j *** got signal Terminated *** 2024-05-22T23:14:52.434 INFO:journalctl@ceph.mds.j.smithi084.stdout:May 22 23:14:52 smithi084 podman[95566]: 2024-05-22 23:14:52.171912797 +0000 UTC m=+2.818835768 container died af78cff964907f6a461b2ae3ae823a71158c78a064d7f0d06817fafc008fc509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-2e1f762, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, ceph=True) 2024-05-22T23:14:52.691 INFO:journalctl@ceph.mds.j.smithi084.stdout:May 22 23:14:52 smithi084 podman[95566]: 2024-05-22 23:14:52.499744185 +0000 UTC m=+3.146667212 container cleanup af78cff964907f6a461b2ae3ae823a71158c78a064d7f0d06817fafc008fc509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-j, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, RELEASE=squid-2e1f762, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-22T23:14:52.691 INFO:journalctl@ceph.mds.j.smithi084.stdout:May 22 23:14:52 smithi084 bash[95566]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-j 2024-05-22T23:14:52.935 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.j.service' 2024-05-22T23:14:52.957 INFO:journalctl@ceph.mds.j.smithi084.stdout:May 22 23:14:52 smithi084 podman[95578]: 2024-05-22 23:14:52.683972003 +0000 UTC m=+0.505491705 container remove af78cff964907f6a461b2ae3ae823a71158c78a064d7f0d06817fafc008fc509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-j, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-2e1f762, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521) 2024-05-22T23:14:52.957 INFO:journalctl@ceph.mds.j.smithi084.stdout:May 22 23:14:52 smithi084 systemd[1]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.j.service: Deactivated successfully. 2024-05-22T23:14:52.957 INFO:journalctl@ceph.mds.j.smithi084.stdout:May 22 23:14:52 smithi084 systemd[1]: Stopped Ceph mds.j for c1135a12-188d-11ef-bc9a-c7b262605968. 2024-05-22T23:14:52.958 INFO:journalctl@ceph.mds.j.smithi084.stdout:May 22 23:14:52 smithi084 systemd[1]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.j.service: Consumed 1.035s CPU time. 2024-05-22T23:14:52.966 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:14:52.967 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-22T23:14:52.967 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-22T23:14:52.967 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.b 2024-05-22T23:14:53.262 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 22 23:14:53 smithi096 systemd[1]: Stopping Ceph mds.b for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:14:53.556 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 22 23:14:53 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-b[66854]: 2024-05-22T23:14:53.253+0000 7f3cf14f4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:14:53.557 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 22 23:14:53 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-b[66854]: 2024-05-22T23:14:53.253+0000 7f3cf14f4700 -1 mds.b *** got signal Terminated *** 2024-05-22T23:14:56.045 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 22 23:14:55 smithi096 podman[73709]: 2024-05-22 23:14:55.783420679 +0000 UTC m=+2.629319414 container died e4324ef0d993e90e1b81b6f1309395cbe5339294436e4a29b465bfe7c87f878b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2e1f762, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-05-22T23:14:56.306 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 22 23:14:56 smithi096 podman[73709]: 2024-05-22 23:14:56.153930931 +0000 UTC m=+2.999829661 container cleanup e4324ef0d993e90e1b81b6f1309395cbe5339294436e4a29b465bfe7c87f878b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=squid-2e1f762, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521) 2024-05-22T23:14:56.307 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 22 23:14:56 smithi096 bash[73709]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-b 2024-05-22T23:14:56.587 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.b.service' 2024-05-22T23:14:56.621 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:14:56.621 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-22T23:14:56.621 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-22T23:14:56.621 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.e 2024-05-22T23:14:57.057 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 22 23:14:56 smithi096 systemd[1]: Stopping Ceph mds.e for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:14:57.057 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 22 23:14:56 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-e[66243]: 2024-05-22T23:14:56.915+0000 7f5907808700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:14:57.057 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 22 23:14:56 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-e[66243]: 2024-05-22T23:14:56.915+0000 7f5907808700 -1 mds.e *** got signal Terminated *** 2024-05-22T23:14:58.306 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 22 23:14:58 smithi096 podman[73830]: 2024-05-22 23:14:58.029443109 +0000 UTC m=+1.202724708 container stop 61c5842649bc6a7a1d2ce53e223d35547ed7ec9d578bbc84347473a77e9c0c5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-2e1f762, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-22T23:14:58.307 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 22 23:14:58 smithi096 podman[73830]: 2024-05-22 23:14:58.055166291 +0000 UTC m=+1.228447899 container died 61c5842649bc6a7a1d2ce53e223d35547ed7ec9d578bbc84347473a77e9c0c5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-2e1f762) 2024-05-22T23:14:58.806 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 22 23:14:58 smithi096 podman[73830]: 2024-05-22 23:14:58.357422108 +0000 UTC m=+1.530703716 container cleanup 61c5842649bc6a7a1d2ce53e223d35547ed7ec9d578bbc84347473a77e9c0c5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-e, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, io.buildah.version=1.33.5, RELEASE=squid-2e1f762, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-22T23:14:58.807 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 22 23:14:58 smithi096 bash[73830]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-e 2024-05-22T23:14:58.807 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 22 23:14:58 smithi096 podman[73847]: 2024-05-22 23:14:58.525262919 +0000 UTC m=+0.490079610 container remove 61c5842649bc6a7a1d2ce53e223d35547ed7ec9d578bbc84347473a77e9c0c5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-2e1f762, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0) 2024-05-22T23:14:58.839 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.e.service' 2024-05-22T23:14:58.871 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:14:58.872 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-22T23:14:58.872 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-22T23:14:58.872 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.h 2024-05-22T23:14:59.203 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 22 23:14:58 smithi096 systemd[1]: Stopping Ceph mds.h for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:14:59.556 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 22 23:14:59 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-h[66549]: 2024-05-22T23:14:59.194+0000 7f9b94fea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:14:59.557 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 22 23:14:59 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-h[66549]: 2024-05-22T23:14:59.194+0000 7f9b94fea700 -1 mds.h *** got signal Terminated *** 2024-05-22T23:15:03.502 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 22 23:15:03 smithi096 podman[73962]: 2024-05-22 23:15:03.200396597 +0000 UTC m=+4.117803926 container died 1c2d2eeaf388409b6f06ff21b6b9a354760400e5f6e3ffe9285d02560b0125b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2e1f762, ceph=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-22T23:15:03.806 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 22 23:15:03 smithi096 podman[73962]: 2024-05-22 23:15:03.594368677 +0000 UTC m=+4.511775997 container cleanup 1c2d2eeaf388409b6f06ff21b6b9a354760400e5f6e3ffe9285d02560b0125b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2e1f762, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-22T23:15:03.807 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 22 23:15:03 smithi096 bash[73962]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-h 2024-05-22T23:15:03.981 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.h.service' 2024-05-22T23:15:04.013 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:15:04.013 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-22T23:15:04.014 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-22T23:15:04.014 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.k 2024-05-22T23:15:04.556 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 22 23:15:04 smithi096 systemd[1]: Stopping Ceph mds.k for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:15:04.557 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 22 23:15:04 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-k[65930]: 2024-05-22T23:15:04.331+0000 7f274baef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:15:04.557 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 22 23:15:04 smithi096 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-k[65930]: 2024-05-22T23:15:04.331+0000 7f274baef700 -1 mds.k *** got signal Terminated *** 2024-05-22T23:15:08.713 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 22 23:15:08 smithi096 podman[74082]: 2024-05-22 23:15:08.337168987 +0000 UTC m=+4.103284123 container died a9999061cbc9aff3b5d9d944ddf7cc0ee37a9959bb6ef2b1df889ad3d71ba71b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-k, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2e1f762, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240521, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-22T23:15:09.008 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 22 23:15:08 smithi096 podman[74082]: 2024-05-22 23:15:08.805639007 +0000 UTC m=+4.571754137 container cleanup a9999061cbc9aff3b5d9d944ddf7cc0ee37a9959bb6ef2b1df889ad3d71ba71b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-k, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2e1f762, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS) 2024-05-22T23:15:09.008 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 22 23:15:08 smithi096 bash[74082]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-k 2024-05-22T23:15:09.180 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.k.service' 2024-05-22T23:15:09.211 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:15:09.212 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-22T23:15:09.212 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-22T23:15:09.212 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.c 2024-05-22T23:15:09.519 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 22 23:15:09 smithi204 systemd[1]: Stopping Ceph mds.c for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:15:09.779 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 22 23:15:09 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-c[66646]: 2024-05-22T23:15:09.511+0000 7f09d8f23700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:15:09.779 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 22 23:15:09 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-c[66646]: 2024-05-22T23:15:09.511+0000 7f09d8f23700 -1 mds.c *** got signal Terminated *** 2024-05-22T23:15:12.077 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 22 23:15:11 smithi204 podman[73444]: 2024-05-22 23:15:11.732862852 +0000 UTC m=+2.331313870 container died c208e68ab1edc37670ce9bc65e707262cd199913cd45d765f479797316793510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-c, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-2e1f762, CEPH_POINT_RELEASE=) 2024-05-22T23:15:12.370 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 22 23:15:12 smithi204 podman[73444]: 2024-05-22 23:15:12.153136534 +0000 UTC m=+2.751587548 container cleanup c208e68ab1edc37670ce9bc65e707262cd199913cd45d765f479797316793510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-2e1f762, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-05-22T23:15:12.370 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 22 23:15:12 smithi204 bash[73444]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-c 2024-05-22T23:15:12.598 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.c.service' 2024-05-22T23:15:12.621 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 22 23:15:12 smithi204 podman[73471]: 2024-05-22 23:15:12.362653332 +0000 UTC m=+0.618579154 container remove c208e68ab1edc37670ce9bc65e707262cd199913cd45d765f479797316793510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, RELEASE=squid-2e1f762, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521) 2024-05-22T23:15:12.621 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 22 23:15:12 smithi204 systemd[1]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.c.service: Deactivated successfully. 2024-05-22T23:15:12.621 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 22 23:15:12 smithi204 systemd[1]: Stopped Ceph mds.c for c1135a12-188d-11ef-bc9a-c7b262605968. 2024-05-22T23:15:12.621 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 22 23:15:12 smithi204 systemd[1]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.c.service: Consumed 1.068s CPU time. 2024-05-22T23:15:12.633 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:15:12.634 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-22T23:15:12.634 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-22T23:15:12.634 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.f 2024-05-22T23:15:12.918 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 22 23:15:12 smithi204 systemd[1]: Stopping Ceph mds.f for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:15:13.418 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 22 23:15:12 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-f[66343]: 2024-05-22T23:15:12.948+0000 7f21433a6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:15:13.418 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 22 23:15:12 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-f[66343]: 2024-05-22T23:15:12.948+0000 7f21433a6700 -1 mds.f *** got signal Terminated *** 2024-05-22T23:15:17.149 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 22 23:15:16 smithi204 podman[73577]: 2024-05-22 23:15:16.861162265 +0000 UTC m=+4.012944926 container died a1ab7dd8ec58b743f7121085e5fd62878a0ee2ad85f12c83b562cef07c2f5fcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2e1f762, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-22T23:15:17.417 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 22 23:15:17 smithi204 podman[73577]: 2024-05-22 23:15:17.232166724 +0000 UTC m=+4.383949384 container cleanup a1ab7dd8ec58b743f7121085e5fd62878a0ee2ad85f12c83b562cef07c2f5fcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-f, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-2e1f762, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-22T23:15:17.418 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 22 23:15:17 smithi204 bash[73577]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-f 2024-05-22T23:15:17.669 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.f.service' 2024-05-22T23:15:17.701 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:15:17.701 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-22T23:15:17.701 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-22T23:15:17.701 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.i 2024-05-22T23:15:18.026 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 22 23:15:17 smithi204 systemd[1]: Stopping Ceph mds.i for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:15:18.418 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 22 23:15:18 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-i[66951]: 2024-05-22T23:15:18.018+0000 7f5265cee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:15:18.418 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 22 23:15:18 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-i[66951]: 2024-05-22T23:15:18.018+0000 7f5265cee700 -1 mds.i *** got signal Terminated *** 2024-05-22T23:15:22.002 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 22 23:15:21 smithi204 podman[73700]: 2024-05-22 23:15:21.7254913 +0000 UTC m=+3.817523929 container died 81b84a23d555ed38fe039a3a884dc62d1e90cf5d153a81c760264d744ef1b3b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-2e1f762, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-22T23:15:22.418 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 22 23:15:22 smithi204 podman[73700]: 2024-05-22 23:15:22.094506423 +0000 UTC m=+4.186539063 container cleanup 81b84a23d555ed38fe039a3a884dc62d1e90cf5d153a81c760264d744ef1b3b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-2e1f762, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-22T23:15:22.418 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 22 23:15:22 smithi204 bash[73700]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-i 2024-05-22T23:15:22.691 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.i.service' 2024-05-22T23:15:22.722 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:15:22.722 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-22T23:15:22.722 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-22T23:15:22.722 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.l 2024-05-22T23:15:23.048 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 22 23:15:22 smithi204 systemd[1]: Stopping Ceph mds.l for c1135a12-188d-11ef-bc9a-c7b262605968... 2024-05-22T23:15:23.418 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 22 23:15:23 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-l[66029]: 2024-05-22T23:15:23.039+0000 7f951aba7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-22T23:15:23.418 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 22 23:15:23 smithi204 ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-l[66029]: 2024-05-22T23:15:23.039+0000 7f951aba7700 -1 mds.l *** got signal Terminated *** 2024-05-22T23:15:26.648 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 22 23:15:26 smithi204 podman[73820]: 2024-05-22 23:15:26.31905193 +0000 UTC m=+3.384374085 container died ccde4371bca298a22ee0d67457a41509ed67e91a1b261d890259a252b3293fde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-l, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-2e1f762) 2024-05-22T23:15:26.648 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 22 23:15:26 smithi204 podman[73820]: 2024-05-22 23:15:26.64027559 +0000 UTC m=+3.705597736 container cleanup ccde4371bca298a22ee0d67457a41509ed67e91a1b261d890259a252b3293fde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, RELEASE=squid-2e1f762, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-05-22T23:15:26.918 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 22 23:15:26 smithi204 bash[73820]: ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-l 2024-05-22T23:15:26.918 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 22 23:15:26 smithi204 podman[73831]: 2024-05-22 23:15:26.833039722 +0000 UTC m=+0.509882129 container remove ccde4371bca298a22ee0d67457a41509ed67e91a1b261d890259a252b3293fde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d75b7fc6f1a567cd446c157fc5a821daa32aed37a514c93b3d2651e77acb3036, name=ceph-c1135a12-188d-11ef-bc9a-c7b262605968-mds-l, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-2e1f762, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-22T23:15:27.067 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c1135a12-188d-11ef-bc9a-c7b262605968@mds.l.service' 2024-05-22T23:15:27.097 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-22T23:15:27.098 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-22T23:15:27.098 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c1135a12-188d-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-22T23:15:27.288 INFO:teuthology.orchestra.run.smithi084.stdout:Deleting cluster with fsid: c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T23:15:29.709 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c1135a12-188d-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-22T23:15:29.897 INFO:teuthology.orchestra.run.smithi096.stdout:Deleting cluster with fsid: c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T23:15:32.289 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c1135a12-188d-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-22T23:15:32.480 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T23:15:34.859 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-22T23:15:34.893 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-22T23:15:34.925 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-22T23:15:34.950 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-22T23:15:34.953 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-22_21:24:03-fs-squid-distro-default-smithi/7721199/remote/smithi084/crash 2024-05-22T23:15:34.954 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/crash -- . 2024-05-22T23:15:34.998 INFO:teuthology.orchestra.run.smithi084.stderr:tar: /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-22T23:15:34.999 INFO:teuthology.orchestra.run.smithi084.stderr:tar: Error is not recoverable: exiting now 2024-05-22T23:15:35.001 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-22_21:24:03-fs-squid-distro-default-smithi/7721199/remote/smithi096/crash 2024-05-22T23:15:35.002 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/crash -- . 2024-05-22T23:15:35.075 INFO:teuthology.orchestra.run.smithi096.stderr:tar: /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-22T23:15:35.075 INFO:teuthology.orchestra.run.smithi096.stderr:tar: Error is not recoverable: exiting now 2024-05-22T23:15:35.077 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-22_21:24:03-fs-squid-distro-default-smithi/7721199/remote/smithi204/crash 2024-05-22T23:15:35.078 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/crash -- . 2024-05-22T23:15:35.114 INFO:teuthology.orchestra.run.smithi204.stderr:tar: /var/lib/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-22T23:15:35.115 INFO:teuthology.orchestra.run.smithi204.stderr:tar: Error is not recoverable: exiting now 2024-05-22T23:15:35.116 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-22T23:15:35.116 DEBUG:teuthology.orchestra.run.smithi084:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-22T23:15:35.157 INFO:tasks.cephadm:Compressing logs... 2024-05-22T23:15:35.157 DEBUG:teuthology.orchestra.run.smithi084:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-22T23:15:35.200 DEBUG:teuthology.orchestra.run.smithi096:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-22T23:15:35.203 DEBUG:teuthology.orchestra.run.smithi204:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-22T23:15:35.226 INFO:teuthology.orchestra.run.smithi204.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.log 2024-05-22T23:15:35.227 INFO:teuthology.orchestra.run.smithi204.stderr:: No such file or directory 2024-05-22T23:15:35.227 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.2.log 2024-05-22T23:15:35.227 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.f.log 2024-05-22T23:15:35.227 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-22T23:15:35.227 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.5.log 2024-05-22T23:15:35.227 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.c.log 2024-05-22T23:15:35.227 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.audit.log: /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mon.c.log 2024-05-22T23:15:35.239 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mgr.z.log 2024-05-22T23:15:35.239 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mon.c.log: /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mgr.z.log: 87.2% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-22T23:15:35.239 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.i.log 2024-05-22T23:15:35.239 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-22T23:15:35.239 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.i.log: 90.5% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-22T23:15:35.240 INFO:teuthology.orchestra.run.smithi204.stderr: 94.6% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.f.log.gz 2024-05-22T23:15:35.240 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.11.log 2024-05-22T23:15:35.240 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.8.log 2024-05-22T23:15:35.240 INFO:teuthology.orchestra.run.smithi204.stderr: 83.9% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-22T23:15:35.240 INFO:teuthology.orchestra.run.smithi204.stderr: 94.7% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.c.log.gz 2024-05-22T23:15:35.240 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.11.log: 94.6% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.i.log.gz 2024-05-22T23:15:35.240 INFO:teuthology.orchestra.run.smithi204.stderr: 92.0% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mgr.z.log.gz 2024-05-22T23:15:35.242 INFO:teuthology.orchestra.run.smithi084.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-22T23:15:35.243 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.log 2024-05-22T23:15:35.243 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-22T23:15:35.243 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.g.log 2024-05-22T23:15:35.243 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.log: /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.j.log 2024-05-22T23:15:35.243 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.3.log 2024-05-22T23:15:35.243 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.0.log 2024-05-22T23:15:35.243 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mgr.x.log 2024-05-22T23:15:35.244 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mon.a.log 2024-05-22T23:15:35.244 INFO:teuthology.orchestra.run.smithi084.stderr: 87.2% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-22T23:15:35.244 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.6.log 2024-05-22T23:15:35.244 INFO:teuthology.orchestra.run.smithi084.stderr: 90.3%/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mon.a.log: -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-22T23:15:35.244 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-22T23:15:35.244 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.a.log 2024-05-22T23:15:35.244 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.6.log: /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.cephadm.log: 94.6% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.g.log.gz 2024-05-22T23:15:35.244 INFO:teuthology.orchestra.run.smithi084.stderr: 84.2% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-22T23:15:35.245 INFO:teuthology.orchestra.run.smithi084.stderr: 94.7% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.j.log.gz 2024-05-22T23:15:35.245 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-22T23:15:35.245 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.d.log 2024-05-22T23:15:35.245 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.9.log 2024-05-22T23:15:35.245 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-volume.log: /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.a.log.gz 2024-05-22T23:15:35.245 INFO:teuthology.orchestra.run.smithi084.stderr: 94.6% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.d.log.gz 2024-05-22T23:15:35.250 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-22T23:15:35.250 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-22T23:15:35.251 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-22T23:15:35.251 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.l.log 2024-05-22T23:15:35.252 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-22T23:15:35.252 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-22T23:15:35.252 INFO:teuthology.orchestra.run.smithi204.stderr: 94.6% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.l.log.gz 2024-05-22T23:15:35.252 INFO:teuthology.orchestra.run.smithi096.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.log 2024-05-22T23:15:35.252 INFO:teuthology.orchestra.run.smithi096.stderr:: No such file or directory 2024-05-22T23:15:35.252 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-22T23:15:35.252 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.7.log 2024-05-22T23:15:35.253 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.log: /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.b.log 2024-05-22T23:15:35.253 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.4.log 2024-05-22T23:15:35.253 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.1.log 2024-05-22T23:15:35.253 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.b.log: /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mgr.y.log 2024-05-22T23:15:35.253 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.h.log 2024-05-22T23:15:35.253 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.10.log 2024-05-22T23:15:35.253 INFO:teuthology.orchestra.run.smithi096.stderr: 87.2%/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.h.log: -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-22T23:15:35.253 INFO:teuthology.orchestra.run.smithi096.stderr: 90.4% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-22T23:15:35.254 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-22T23:15:35.254 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.e.log 2024-05-22T23:15:35.254 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.cephadm.log: 84.0% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-22T23:15:35.254 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-22T23:15:35.254 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.e.log: 94.6% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.e.log.gz 2024-05-22T23:15:35.254 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mon.b.log 2024-05-22T23:15:35.255 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-volume.log: 91.9% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mgr.y.log.gz 2024-05-22T23:15:35.255 INFO:teuthology.orchestra.run.smithi204.stderr: 91.1% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-22T23:15:35.256 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.k.log 2024-05-22T23:15:35.257 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-22T23:15:35.257 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-22T23:15:35.257 INFO:teuthology.orchestra.run.smithi084.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-22T23:15:35.261 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.k.log: 94.5% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.k.log.gz 2024-05-22T23:15:35.263 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-22T23:15:35.476 INFO:teuthology.orchestra.run.smithi084.stderr: 90.1% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mgr.x.log.gz 2024-05-22T23:15:35.691 INFO:teuthology.orchestra.run.smithi096.stderr: 92.0% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mon.b.log.gz 2024-05-22T23:15:35.883 INFO:teuthology.orchestra.run.smithi204.stderr: 92.8% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mon.c.log.gz 2024-05-22T23:15:36.421 INFO:teuthology.orchestra.run.smithi084.stderr: 90.5% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mon.a.log.gz 2024-05-22T23:15:42.345 INFO:teuthology.orchestra.run.smithi084.stderr: 91.6% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.9.log.gz 2024-05-22T23:15:42.779 INFO:teuthology.orchestra.run.smithi204.stderr: 91.4% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.11.log.gz 2024-05-22T23:15:43.029 INFO:teuthology.orchestra.run.smithi096.stderr: 91.8% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.4.log.gz 2024-05-22T23:15:43.425 INFO:teuthology.orchestra.run.smithi084.stderr: 91.4% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.6.log.gz 2024-05-22T23:15:43.693 INFO:teuthology.orchestra.run.smithi084.stderr: 92.0% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.0.log.gz 2024-05-22T23:15:44.628 INFO:teuthology.orchestra.run.smithi096.stderr: 95.5% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.h.log.gz 2024-05-22T23:15:44.914 INFO:teuthology.orchestra.run.smithi204.stderr: 91.8% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.2.log.gz 2024-05-22T23:15:45.275 INFO:teuthology.orchestra.run.smithi084.stderr: 91.6% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.3.log.gz 2024-05-22T23:15:45.277 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-05-22T23:15:45.277 INFO:teuthology.orchestra.run.smithi084.stderr:real 0m10.062s 2024-05-22T23:15:45.277 INFO:teuthology.orchestra.run.smithi084.stderr:user 0m34.113s 2024-05-22T23:15:45.277 INFO:teuthology.orchestra.run.smithi084.stderr:sys 0m1.148s 2024-05-22T23:15:46.595 INFO:teuthology.orchestra.run.smithi204.stderr: 91.2% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.8.log.gz 2024-05-22T23:15:46.725 INFO:teuthology.orchestra.run.smithi096.stderr: 92.0% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.1.log.gz 2024-05-22T23:15:47.147 INFO:teuthology.orchestra.run.smithi204.stderr: 91.2% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.5.log.gz 2024-05-22T23:15:47.149 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-05-22T23:15:47.149 INFO:teuthology.orchestra.run.smithi204.stderr:real 0m11.932s 2024-05-22T23:15:47.149 INFO:teuthology.orchestra.run.smithi204.stderr:user 0m39.863s 2024-05-22T23:15:47.149 INFO:teuthology.orchestra.run.smithi204.stderr:sys 0m1.245s 2024-05-22T23:15:49.046 INFO:teuthology.orchestra.run.smithi096.stderr: 91.5% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.10.log.gz 2024-05-22T23:15:49.401 INFO:teuthology.orchestra.run.smithi096.stderr: 91.4% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-osd.7.log.gz 2024-05-22T23:18:00.192 INFO:teuthology.orchestra.run.smithi096.stderr: 93.6% -- replaced with /var/log/ceph/c1135a12-188d-11ef-bc9a-c7b262605968/ceph-mds.b.log.gz 2024-05-22T23:18:00.194 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-05-22T23:18:00.194 INFO:teuthology.orchestra.run.smithi096.stderr:real 2m24.975s 2024-05-22T23:18:00.194 INFO:teuthology.orchestra.run.smithi096.stderr:user 3m15.665s 2024-05-22T23:18:00.194 INFO:teuthology.orchestra.run.smithi096.stderr:sys 0m6.073s 2024-05-22T23:18:00.195 INFO:tasks.cephadm:Archiving logs... 2024-05-22T23:18:00.195 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-22_21:24:03-fs-squid-distro-default-smithi/7721199/remote/smithi084/log 2024-05-22T23:18:00.196 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-22T23:18:05.798 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-22_21:24:03-fs-squid-distro-default-smithi/7721199/remote/smithi096/log 2024-05-22T23:18:05.798 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-22T23:18:35.142 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-22_21:24:03-fs-squid-distro-default-smithi/7721199/remote/smithi204/log 2024-05-22T23:18:35.143 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-22T23:18:41.731 INFO:tasks.cephadm:Removing cluster... 2024-05-22T23:18:41.732 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c1135a12-188d-11ef-bc9a-c7b262605968 --force 2024-05-22T23:18:41.927 INFO:teuthology.orchestra.run.smithi084.stdout:Deleting cluster with fsid: c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T23:18:42.271 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c1135a12-188d-11ef-bc9a-c7b262605968 --force 2024-05-22T23:18:42.462 INFO:teuthology.orchestra.run.smithi096.stdout:Deleting cluster with fsid: c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T23:18:42.925 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c1135a12-188d-11ef-bc9a-c7b262605968 --force 2024-05-22T23:18:43.114 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: c1135a12-188d-11ef-bc9a-c7b262605968 2024-05-22T23:18:43.613 INFO:tasks.cephadm:Removing cephadm ... 2024-05-22T23:18:43.614 DEBUG:teuthology.orchestra.run.smithi084:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-22T23:18:43.631 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-22T23:18:43.647 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-22T23:18:43.662 INFO:tasks.cephadm:Teardown complete 2024-05-22T23:18:43.663 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-22T23:18:43.673 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-22T23:18:43.674 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-22T23:18:43.676 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-22T23:18:43.689 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-22T23:18:43.799 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-22T23:18:43.809 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-22T23:18:43.809 DEBUG:teuthology.orchestra.run.smithi084:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-22T23:18:43.812 DEBUG:teuthology.orchestra.run.smithi096:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-22T23:18:43.814 DEBUG:teuthology.orchestra.run.smithi204:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-22T23:18:43.824 INFO:teuthology.orchestra.run.smithi084.stderr:bash: line 1: ntpq: command not found 2024-05-22T23:18:43.826 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-05-22T23:18:43.828 INFO:teuthology.orchestra.run.smithi084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-22T23:18:43.828 INFO:teuthology.orchestra.run.smithi084.stdout:=============================================================================== 2024-05-22T23:18:43.828 INFO:teuthology.orchestra.run.smithi084.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 40 -364us[ -364us] +/- 57ms 2024-05-22T23:18:43.828 INFO:teuthology.orchestra.run.smithi084.stdout:^* hv02.front.sepia.ceph.com 3 6 377 41 -1081us[-1081us] +/- 52ms 2024-05-22T23:18:43.829 INFO:teuthology.orchestra.run.smithi084.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 +2333us[+2333us] +/- 80ms 2024-05-22T23:18:43.829 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-22T23:18:43.829 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-05-22T23:18:43.830 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-22T23:18:43.830 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-05-22T23:18:43.830 INFO:teuthology.orchestra.run.smithi096.stdout:^* hv01.front.sepia.ceph.com 3 8 377 167 -339us[ -386us] +/- 49ms 2024-05-22T23:18:43.830 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 42 -1115us[-1115us] +/- 52ms 2024-05-22T23:18:43.830 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 +2291us[+2291us] +/- 80ms 2024-05-22T23:18:43.831 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-22T23:18:43.832 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-22T23:18:43.832 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-05-22T23:18:43.833 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 39 -388us[ -388us] +/- 57ms 2024-05-22T23:18:43.833 INFO:teuthology.orchestra.run.smithi204.stdout:^* hv02.front.sepia.ceph.com 3 6 377 39 -1112us[-1111us] +/- 52ms 2024-05-22T23:18:43.833 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 +2305us[+2305us] +/- 80ms 2024-05-22T23:18:43.833 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-22T23:18:43.833 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-22T23:18:43.843 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-22T23:18:43.844 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-22T23:18:43.854 DEBUG:teuthology.orchestra.run.smithi084:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-22T23:18:43.945 DEBUG:teuthology.orchestra.run.smithi096:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-22T23:18:44.012 DEBUG:teuthology.orchestra.run.smithi204:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-22T23:18:44.075 DEBUG:teuthology.orchestra.run.smithi084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-22T23:18:44.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T23:18:44.104 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-22T23:18:44.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T23:18:44.132 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-22T23:18:44.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T23:18:44.159 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-22T23:18:44.170 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-22T23:18:44.182 INFO:teuthology.task.internal:Duration was 2068.499148 seconds 2024-05-22T23:18:44.183 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-22T23:18:44.192 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-22T23:18:44.192 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-22T23:18:44.195 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-22T23:18:44.197 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-22T23:18:44.231 INFO:teuthology.orchestra.run.smithi084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-22T23:18:44.231 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-22T23:18:44.235 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-22T23:18:44.596 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-22T23:18:44.597 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi084.front.sepia.ceph.com 2024-05-22T23:18:44.597 DEBUG:teuthology.orchestra.run.smithi084:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-22T23:18:44.618 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi096.front.sepia.ceph.com 2024-05-22T23:18:44.618 DEBUG:teuthology.orchestra.run.smithi096:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-22T23:18:44.659 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi204.front.sepia.ceph.com 2024-05-22T23:18:44.659 DEBUG:teuthology.orchestra.run.smithi204:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-22T23:18:44.694 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-22T23:18:44.694 DEBUG:teuthology.orchestra.run.smithi084:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-22T23:18:44.697 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-22T23:18:44.701 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-22T23:18:44.759 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-22T23:18:44.760 DEBUG:teuthology.orchestra.run.smithi084:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-22T23:18:44.764 DEBUG:teuthology.orchestra.run.smithi096:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-22T23:18:44.767 DEBUG:teuthology.orchestra.run.smithi204:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-22T23:18:45.148 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-22T23:18:45.160 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-22T23:18:45.160 DEBUG:teuthology.orchestra.run.smithi084:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-22T23:18:45.185 DEBUG:teuthology.orchestra.run.smithi096:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-22T23:18:45.209 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-22T23:18:45.233 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-22T23:18:45.244 DEBUG:teuthology.orchestra.run.smithi084:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-22T23:18:45.246 DEBUG:teuthology.orchestra.run.smithi096:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-22T23:18:45.251 DEBUG:teuthology.orchestra.run.smithi204:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-22T23:18:45.267 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = core 2024-05-22T23:18:45.271 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = core 2024-05-22T23:18:45.295 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = core 2024-05-22T23:18:45.331 DEBUG:teuthology.orchestra.run.smithi084:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-22T23:18:45.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T23:18:45.346 DEBUG:teuthology.orchestra.run.smithi096:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-22T23:18:45.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T23:18:45.360 DEBUG:teuthology.orchestra.run.smithi204:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-22T23:18:45.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T23:18:45.386 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-22T23:18:45.398 INFO:teuthology.task.internal:Transferring archived files... 2024-05-22T23:18:45.399 DEBUG:teuthology.misc:Transferring archived files from smithi084:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-22_21:24:03-fs-squid-distro-default-smithi/7721199/remote/smithi084 2024-05-22T23:18:45.399 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-22T23:18:45.442 DEBUG:teuthology.misc:Transferring archived files from smithi096:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-22_21:24:03-fs-squid-distro-default-smithi/7721199/remote/smithi096 2024-05-22T23:18:45.442 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-22T23:18:45.478 DEBUG:teuthology.misc:Transferring archived files from smithi204:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-22_21:24:03-fs-squid-distro-default-smithi/7721199/remote/smithi204 2024-05-22T23:18:45.478 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-22T23:18:45.513 INFO:teuthology.task.internal:Removing archive directory... 2024-05-22T23:18:45.514 DEBUG:teuthology.orchestra.run.smithi084:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-22T23:18:45.516 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-22T23:18:45.518 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-22T23:18:45.562 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-22T23:18:45.576 INFO:teuthology.task.internal:Not uploading archives. 2024-05-22T23:18:45.576 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-22T23:18:45.587 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-22T23:18:45.587 DEBUG:teuthology.orchestra.run.smithi084:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-22T23:18:45.590 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-22T23:18:45.592 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-22T23:18:45.602 INFO:teuthology.orchestra.run.smithi084.stdout: 265567 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 22 23:18 /home/ubuntu/cephtest 2024-05-22T23:18:45.604 INFO:teuthology.orchestra.run.smithi096.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 22 23:18 /home/ubuntu/cephtest 2024-05-22T23:18:45.616 INFO:teuthology.orchestra.run.smithi204.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 22 23:18 /home/ubuntu/cephtest 2024-05-22T23:18:45.618 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-22T23:18:45.629 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-22T23:18:45.659 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} duration: 2068.4991478919983 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi084 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=935217f1cb4dc722d5303f5ad364c883e9133376 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=4254f8882f53441ba212b9835c8b500d status: fail success: false 2024-05-22T23:18:45.659 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-22T23:18:45.738 INFO:teuthology.run:FAIL